Reactions from ranked influencers
2 posts
every single person i know working at the frontier is incredibly talented and hardworking, and they're not wasting compute or capital, and neither is america. some thoughts / opinions on this 1/ new capability is infinitely harder. i'm not saying chinese labs are just mass distilling (although there's some of that), but even a few teacher model calls to 5.6 sol pro for math or fable agent rollouts is OOMs cheaper than hiring human experts, sourcing data, cleaning data, etc. this is why mercor, scale and the other data vendors do so well. paying physics phds to help teach the models is not cheap, and china does indeed get a leg up here. this is also why i suspect most labs have the true frontier model internal only. the modus operandi seems to be to assume any model put into the hands of hyperscalers or released via api will be distilled or stolen. what the public sees is ~6 mo behind what's internal, and depending on how the chinese labs are operating, they're either ~1yr behind, or ~1.5yrs behind the true capability of the labs. there's also a fair amount of benchmaxxing from china, as they only have to show a good looking number to be considered great, not get sustained usage to stay alive (like the US labs), or be efficient at inference (which is also an optimization for the labs, more on this later). the modus operandi seems to be to assume any model put into the hands of hyperscalers or released via API will be distilled or stolen, which is 100% justified and right. talent and ip bleed is a also real thing. a lab may frequently have to run/burn thousands of tests to figure out the optimal config/architecture for a model. anyone on the training, inference team or one of the hyperscalers who hosts the model can just look at one model implementation/config file and know the truth and understand and replicate. you don't need to know what didn't work, only what did. 2/ data privacy requirements (or lack thereof) in mainland china. assuming reasonably that the six little tigers in china (look it up if you don't know about this) have access to some form of additional ccp data that we do not know about, it is a moderate advantage. the labs are incredibly thirsty for novel data. the internet has mostly exhausted its utility, at least with current techniques, and much like what ilya said, there is only one internet (like fossil fuels!). people are also deploying the models to things never seen before. i.e how is a model supposed to learn how to order food with the doordash cli unless its taught how to behave in this scenario / is superhuman at generalization. US enterprises are also vary of this, and therefore cannot and do not rely on chinese model apis. i don't think this is cope, although i realize many may feel this way 3/ inference efficiency: the modern labs are extremely good at inference. the gb300 is a beast, and even older chips are being used very well to serve the models. api pricing is meant to also recover the cost on training, and inference cost for intelligence continues to fall. all labs have really large sparse moes that are OOMs cheaper than what api pricing reflects. 4/ profit motive: there is an interesting dwarkesh podcast about this. you must stop looking at china from the worldview of they're trying to make money. this is just not true. they're trying to dominate via scale and volume, which is true for the way china works, but not the us. the people of china are incredible, extremely sharp and are not to be underestimated. 996 work culture and strong stem is a great source of talent and data for the labs. american labs must pay around $1m/yr for entry level talent in the bay, which also increases capex. also there's an undisclosed amount of capex coming from the ccp. i do not claim to know what the scale is, but i would suspect there's a fair amount of chips flowing through the gulf, khazakstan etc. (do we seriously believe khazakstan of all places is building $10B worth of datacenters with nvidia gpus all for/by themselves?) 5/ datacenter and build out. xai had to spend a lot of money, time and effort to build out the supercomputer in tennessee. anthropic pays them 10 digits to lease out to serve customers etc because of the lack of supply, electricity, zoning, regulation etc. ccp does not have this issue, and there are claims that they are indeed introducing misinformation into the us that data centers are bad for us. we have to get much much faster about addressing this and dealing with "muh water use" kind of statements. it's oneshotting our population, much like nuclear energy did. electricity is the final moat, over chips, models, intelligence, and data. regardless of your timelines, if you believe in rsi, then you know the physical world is the only bottleneck. china/the ccp has real, compounding structural edges on data access, deployment freedom, talent cost, speed of physical buildout, and now apparently on shaping the domestic narrative around our datacenters and energy. america is making the best use of it's most powerful resource, capital, to accelerate faster.
@lu_sichu i do think that openai and anthropic need to answer for why they are barely 4 months ahead but are spending like 10x the capex of chinese labs (maybe an understatement)
long stream of embarrassing excuse-making from a googler riddle me this, why are your examples of distillation Fable and Sol? They just came out. Why don't the chynese/CCP save money distilling the reliable Gemini for their shitty inefficient models? Get real lmao
every single person i know working at the frontier is incredibly talented and hardworking, and they're not wasting compute or capital, and neither is america. some thoughts / opinions on this 1/ new capability is infinitely harder. i'm not saying chinese labs are just mass distilling (although there's some of that), but even a few teacher model calls to 5.6 sol pro for math or fable agent rollouts is OOMs cheaper than hiring human experts, sourcing data, cleaning data, etc. this is why mercor, scale and the other data vendors do so well. paying physics phds to help teach the models is not cheap, and china does indeed get a leg up here. this is also why i suspect most labs have the true frontier model internal only. the modus operandi seems to be to assume any model put into the hands of hyperscalers or released via api will be distilled or stolen. what the public sees is ~6 mo behind what's internal, and depending on how the chinese labs are operating, they're either ~1yr behind, or ~1.5yrs behind the true capability of the labs. there's also a fair amount of benchmaxxing from china, as they only have to show a good looking number to be considered great, not get sustained usage to stay alive (like the US labs), or be efficient at inference (which is also an optimization for the labs, more on this later). the modus operandi seems to be to assume any model put into the hands of hyperscalers or released via API will be distilled or stolen, which is 100% justified and right. talent and ip bleed is a also real thing. a lab may frequently have to run/burn thousands of tests to figure out the optimal config/architecture for a model. anyone on the training, inference team or one of the hyperscalers who hosts the model can just look at one model implementation/config file and know the truth and understand and replicate. you don't need to know what didn't work, only what did. 2/ data privacy requirements (or lack thereof) in mainland china. assuming reasonably that the six little tigers in china (look it up if you don't know about this) have access to some form of additional ccp data that we do not know about, it is a moderate advantage. the labs are incredibly thirsty for novel data. the internet has mostly exhausted its utility, at least with current techniques, and much like what ilya said, there is only one internet (like fossil fuels!). people are also deploying the models to things never seen before. i.e how is a model supposed to learn how to order food with the doordash cli unless its taught how to behave in this scenario / is superhuman at generalization. US enterprises are also vary of this, and therefore cannot and do not rely on chinese model apis. i don't think this is cope, although i realize many may feel this way 3/ inference efficiency: the modern labs are extremely good at inference. the gb300 is a beast, and even older chips are being used very well to serve the models. api pricing is meant to also recover the cost on training, and inference cost for intelligence continues to fall. all labs have really large sparse moes that are OOMs cheaper than what api pricing reflects. 4/ profit motive: there is an interesting dwarkesh podcast about this. you must stop looking at china from the worldview of they're trying to make money. this is just not true. they're trying to dominate via scale and volume, which is true for the way china works, but not the us. the people of china are incredible, extremely sharp and are not to be underestimated. 996 work culture and strong stem is a great source of talent and data for the labs. american labs must pay around $1m/yr for entry level talent in the bay, which also increases capex. also there's an undisclosed amount of capex coming from the ccp. i do not claim to know what the scale is, but i would suspect there's a fair amount of chips flowing through the gulf, khazakstan etc. (do we seriously believe khazakstan of all places is building $10B worth of datacenters with nvidia gpus all for/by themselves?) 5/ datacenter and build out. xai had to spend a lot of money, time and effort to build out the supercomputer in tennessee. anthropic pays them 10 digits to lease out to serve customers etc because of the lack of supply, electricity, zoning, regulation etc. ccp does not have this issue, and there are claims that they are indeed introducing misinformation into the us that data centers are bad for us. we have to get much much faster about addressing this and dealing with "muh water use" kind of statements. it's oneshotting our population, much like nuclear energy did. electricity is the final moat, over chips, models, intelligence, and data. regardless of your timelines, if you believe in rsi, then you know the physical world is the only bottleneck. china/the ccp has real, compounding structural edges on data access, deployment freedom, talent cost, speed of physical buildout, and now apparently on shaping the domestic narrative around our datacenters and energy. america is making the best use of it's most powerful resource, capital, to accelerate faster.