Reactions from ranked influencers
7 posts
pinky promise there's no distillation
Have Chinese AI Models Caught Up to the US Frontier? I have spent the last 2 days writing this article. It should settle the debate once and for all. https://open.substack.com/pub/scaling01/p/have-chinese-ai-models-caught-up
it's hard to quantify how much distillation improves the performance of Chinese models we can't really put a number like 20% on it, but we can compare Chinese labs with other companies that don't distill for legal reasons Google for example has 10-100x more compute than Chinese companies, the best researchers and engineers on the planet. so why are they that far behind the frontier? if I had to estimate where Chinese companies would be without distillation then current Google would be my lower bound (at least as far behind as Google) the only other US company I would be comfortable comparing Chinese companies to is Thinking Machines other US open-source labs simply don't have the size (compute and talent)
pinky promise there's no distillation https://twitter.com/scaling01/status/2078659671691272285
@scaling01 Hm, but this suggests that V4 is relatively undistilled (or distilled from everything at once including Grok 4.5), and GLM 5.2 clusters with Gemini (cringe choice) if anything.
Good artists copy. Great artists steel.
pinky promise there's no distillation https://twitter.com/scaling01/status/2078659671691272285
@RyanGreenblatt I ran a cross-entropy comparison of all raw text responses from numerous models using data I already had from a benchmark I run. I leaned on Fable for the stats-know-how; certainly seems very suspicious. The results/code are here for others to inspect: https://typebulb.com/u/lab/you-re-relatively-right/full
@scaling01 Can we share this on AlphaSignal?