AI developer Teortaxes argues Gemini 3 Pro matches Anthropic's Fable in scale but lags in post-training personality design

the main reason I suspect that Anthropic has a better theory of LLMs is, ironically, my faith in the other two labs. GPT-4 famously used µP to predict performance at 1.8T params; 4 years ago. I am 100% certain they *can* train a 10T, like, yesterday. But they saw no point in it.

Maybe they took the wrong lesson with 4.5, "pretraining scaling is dead" and other rubbish. But I try to not look down on people smarter than me. They must have had more convincing reasons to not try again until recently. They burn billions in research compute.

@sichuan_mala Wishing you luck If this is just about being able to train a big model that doesn't explode, wagmi I am mainly suspicious that nobody else has done it yet

agree on the scale fable vs Gemini

I'd argue that Gemini 3.x pro models are even bigger and have more base knowledge

i think the biggest reason of the difference is RL post-training

I think this bc it was the thesis Dario was defending in Dwarkesh podcast: RL on narrow tasks can get you very far

it seems that the people from GDM haven't scaled up properly RL environments for their models

Even when you are trained to understand it, it’s hard to translate those “predicted loss values” into model capabilities.

Let’s say they knew the exact predicted loss they could get with a 10T model. That still tells them very little about the emergent new capabilities at a new scale.

What’s more, each new scale brings brand new and unique post-training / mid training challenges.

I think Anthropic has dedicated humongous amount of resources to design RL environments for all sorts of tasks

just watch their models using Microsoft Office, Fable was insane

I think this isn't just emergence of intelligence, but huge amounts of diligent work on designing environments to train models on all kind of tasks

@teortaxesTex >> Fable doesn't just know many things like a slightly bigger Gemini; it is absurdly superior at recalling *useful, relevant* things for any query.

True. It’s above “big model smell” And probably related to how they outperformed what scaling laws would predict

@teortaxesTex I think the dark matter is how data providers feed into frontier labs. The special relationship between Surge and Anthropic hasn’t received much attention. Edwin is singular in his field.