Open Source AI Models Improve Capabilities Unlike Pharma Knockoffs
Reactions from ranked influencers
2 posts
Point 5 above encompasses the safety case, as changing the cost/benefit. Otherwise this tweet would've been an essay which would've become a book...
@krishnanrohit Agreed this is helpful clarity on the economic case — isn’t there a parallel safety/security case to unpack also? That case could also use some clarification especially around offense vs defense etc
Here's the steelman argument to be anti open source AI as it stands today (note: I don't believe it). The things you need to believe are that AI requires large scale capex as a necessity, and the open source movement is basically all akin to piracy, just duplication without effort. So the logic has to be: 1. Frontier AI requires large Capex, which depend on their ability to capture huge economic rents 2. OS AI today mostly is equivalent to piracy of what frontier AI does 3. It contributes little added capability or research, it will not expand the market or increase demand 4. It materially decreases the rents that would finance point 1 5. Gains from OS, access, competition, innovation, deployment, downstream application and benefits, are much smaller than the costs of lost frontier lab R&D And *if* this is the case, that larger AI labs are like pharma cos, then sure we should have the governments step in and license every part of that supply chain, with extreme restrictions on usage ("behind the counter", "prescription drugs"). Now, the reason this analogy doesn't work in reality is that OS AI models are not just duplicates, generic knockoffs of Claude. If I create knockoff Xanax, Pfizer loses and might invest less in new drugs, but the knockoff Xanax makers aren't making better alprazolam, much less other drugs. They're not improving the molecule, but OS model makers are! Giving folks an open model isn't like handing them a knockoff iPhone to use, it's capability to search the entire adjacent design space, a distributed search process. This gives tremendous option value, since it is not clear where the next best ideas come from. Maybe it's OAI spending $10 Trillion on a training run, but maybe not! The purpose of the acceleration-is-good thesis is not that giving OAI and Ant a de-facto monopoly will result in larger models, which will bring about societal largesse, but that this technology is useful and good, which is why more people building and developing it is useful. I wrote this because the quality of discourse on this is terrible, most advocates for both sides don't seem to be able to actually explain their worldview. If the best argument for why to ban/control OS is Ant needs an extra trillion for a larger pretraining run that's yet another marginal improvement on the same capabilities line, then the market ought to decide if that's worth it. If it sees enough value, it'll fund it. If it doesn't, it won't. And that's fine.