I honestly don't understand the assumption that there will be continued open weights models. At some point, China will regulate release of Mythos-class models or the companies making open weights models will switch to closed.

There will still be open weights, just not frontier.

@emollick You make it up in touring and merchandising.

@emollick Why didn't they stop already? My guess, at the national level LLMs are a sideshow for robotics/productivity speed up/healthcare. They need human care robotics in 10 years due to population dynamics and LLMs gets them there faster.

@emollick Not sure how to work the profit exactly. But as seen with electric car, solar panel, Ecommerce, and property over production profit motive is more vague in China vs national needs. If robotics/heathcare ends up mattering they could chase B2B especially live applications overseas.

@emollick Security wise: The US has lots of compute and a lot of software flows through big firms so Glasswing works. Elsewhere compute limited, no trust of US firms, many small companies. Maybe best to let firms find their own compute. OW might make more sense under that regime.

@emollick I totally agree, china open source models are just here to slow down US labs.

But it may also depend on the business model. It makes sense for Nvidia to release open source models, as they are making money on the hardware.

@emollick Every open weights release so far was someone else's marketing budget. Llama sold Meta's relevance, Gemma sells Google Cloud.

Hard to see who keeps paying once a training run costs nine figures.

@emollick One argument follows the "information wants to be free" and another follows the "frontier models are self replicating". Together they both push towards openness, whether it's allowed or not.

@emollick Do you think there might be a case like the Android Open Source Project - the trailers (Google, Apple, MS, Nvidia, Meta, maybe an alliance?) fund a frontieresque project to keep leverage with Anthropic/OpenAI?

Significantly more expensive than AOSP/Mozilla, but?

@emollick no, there are none

@emollick Open-weights models are very expensive and are temporary attempts at crashing the party and getting the spotlight.

Longer term, those will be the "lite" version and for anything of true value one will have to pay up.

@emollick I'm not sure what you mean by "continued" availability here, because there has never been an open weights model at the frontier. Sometimes they're not too far off, but there's always a significant gap.

@emollick Say someone creates a super powerful open source model and anything "load-bearing" (to borrow AI's words) are based in a country that doesn't care/lax laws.

Then what? Once something exists, making it illegal doesn't exactly stop it.

@emollick I think a lot of it will depend on architecture - if we maintain the current standard of "the more money you have, and the more GPUs, the better your model" then yes I think you're 100% right.

But one of the things Anthropic/Fable restricts with ML research is distributed ML...

@emollick Open weights models are only sustainable if you can either monetize them or subsidize them indefinitely. Most projects do neither

@emollick There is a finite (albeit increasing) amount of bad code, and all of it can be patched. New code will be better. Eventually less new code will be needed and existing code will be free.

@emollick Anyone that isn't the one AI monopoly, or duopoly, has massive incentive to keep models open, including and especially nvidia.

@emollick They will be maintained by the community once we get cheaper compute. Companies are making the mistake of trying to monetize something you just can't... At some point frontier won't even matter if your workflows already work and you can make them even cheaper with less "AI"

@emollick Even more - you need the compute. But yes, here is the case for the end result...without the profit:

@emollick Correct. All the models will become unavailble or un trustable.