Andrew Trask argues specialized model ensembles will consistently outperform monolithic systems, ending centralized lab dominance at the AI frontier

@AndrewCritchPhD 💯

@omarsar0 the "one model to rule them all" assumption was always wishful thinking. different models fail differently. that's actually useful if you know how to design around it. most teams don't design around it. they just switch models when the current one disappoints them

@omarsar0 why do you guys act like models are free? companies are trying to reduce the cost of ai because using even 1 model for every task is so expensive and you guys are talking about using 3 models for 1 task

@morgan_coding That last-minute model-swapping strategy is unsustainable.

@omarsar0 Even manual combinations can compound.

For instance, if I'm planning a new tool, I ask all the top models to take a pass at it, then get one of them to composite a "best of" version by taking all the good stuff from each individual plan.

@AndrewCritchPhD (in the substack I screenshot/quote your tweet saying that btw)

@omarsar0 If not for one model fits all, models trained on specific data by specific providers would perform way different on an isolated prompt, and would still require a fusion of all these field specific models as well

Isn't that the point, though? Ensembling models is precisely the type of approach that could enable cost efficiency. I must admit that this is an unsolved problem. However, ensembling means that we could leverage smaller models (local or cloud) to bring that cost down significantly.

@omarsar0 this all makes sense when you remember that combining the weights of two models just makes them overall better. it was a weird finding that nobody could really explain at the time, but that's how it worked.

imagine if all the frontier labs just combined all their weights.

@omarsar0 @omarsar0 dude, totally agree. people forget the value of tailored models for specific needs. generic ain't always best.

@omarsar0 This is the right direction. The future wave is going to be optimal AI. #optimalAI.

I will not pay for 10 apples if I want to eat one.

@omarsar0 that compounding effect is definitely a game changer.

@omarsar0 Self-fusion is the new replacement term for jerking off

@omarsar0 The bottleneck isn't the model size—it's the feedback loop between domain-specific data and human curation. Generalists plateau; specialists compound.

@omarsar0 On one of their posts they were able to achieve within 1% of Fable results, at half the cost, using non-frontier models.

@omarsar0 @dair_ai Didn’t bittensor:native just integrate this or something?

@omarsar0 specialized setups still kinda winning in prod even if benchmarks don't capture it. the compounding piece is real - seen it live

@omarsar0 Strongly agree. I think the one-size-fits-all general-purpose model assumption keeps breaking. It’s about smart routing now, combining different strengths in a compute efficient way. Huge unlock.

@omarsar0 This is exactly the compounding effect I discuss with founders. Fusing distinct capabilities creates momentum that isolated models cannot match.