AI engineering teams keep proposing pre-classifiers that would shunt easy work to smaller models for speed and cost wins, yet the same teams repeatedly discover that deciding whether a task is easy already demands solving most of it, wiping out the hoped-for savings.
I have this struggle with my own teams, too: many think it is a great idea to save money/latency/sanity by running a pre-classifier: "This task is simple, use a dumber model". Except in order to do that reliably you need to solve the task first! The only way is the advisory model approach.
@trq212 a lot of my prompts were being downgrade and when i checked logs it said TOO_DUMB_TO_NEED_FABLE what was happening there?
The pattern shows up across both coding and qualitative tasks, with routers systematically underestimating difficulty and leaving the heavier model to do the real work anyway.
Parakhin flags an advisory-model route instead, though the discussion leaves open how widely it has been tested or measured against the pre-classifier approach.
Users agree that pre-classifiers for routing AI tasks add unnecessary extra steps after GPT-5, with some optimistic a model could outperform humans at routing decisions.
No Digg Deeper questions have been answered for this story yet.
Yes! Pre-classifying routers are going to result in a lot of bad work because routing is hard and tend to underestimate the value of intelligence on many problems. OpenAI learned this with GPT-5, now it seems routers are hot again.
I have this struggle with my own teams, too: many think it is a great idea to save money/latency/sanity by running a pre-classifier: "This task is simple, use a dumber model". Except in order to do that reliably you need to solve the task first! The only way is the advisory model approach.
Yes! Pre-classifying routers are going to going to result in a lot of bad work because routing is hard and tend to underestimate the value of intelligence on many problems. OpenAI learned this with GPT-5, now it seems routers are hot again.
I have this struggle with my own teams, too: many think it is a great idea to save money/latency/sanity by running a pre-classifier: "This task is simple, use a dumber model". Except in order to do that reliably you need to solve the task first! The only way is the advisory model approach.
@fuckyourputs It tends to save money and latency: reading the current changes is not the same as iteratively creating them. But yes, not as much as if you just replace it with the simpler model - and that’s the whole point.
@MParakhin Coupling execution and classification can work:
@MParakhin computational irreducibility strikes again
@MParakhin would you use fable as the advisor? wouldn't it read the full log anyway and not really save you money regardless?
@MParakhin @fuckyourputs "and that's the whole point" -- is claude in the room with us right now? (jk, i agree. using cheaper, less capable models is penny wise, pound foolish. and if a task is so unimportant that the risk of doing it incorrectly is fine, it's better to just not do it.)
@MParakhin the bitter lesson strikes again
It seems like some queries are trivially routable to cheap models (e.g. factual questions). Always seemed odd to me to get rid of the whole concept. No routers means people who leave it on instant think AI is much worse than it is, and people who leave it on thinking burn tokens wastefully on simple queries. There ought to be some gains to be had from at least directing the obviously easy stuff to cheap models and letting everything else go to the good ones.
@MParakhin A sort of AI-version of the halting problem?
@MParakhin the pre classifier paradox is real, deciding if a task is easy is often as hard as the task itself, you end up needing a smart model just to route to the dumb one and the savings evaporate
@MParakhin Hard agree. I tried one of the leading router solutions and it ended up giving me a really really sub-optimal solution.
@emollick the pre-classifier problem is basically kicking the can down the road
you still need to solve the full task before you can safely offload it
@emollick "and tend to underestimate the value of intelligence on many problems"
This is a huge, huge point -- small, seemingly trivial problems have more variance in outcome than people expect. I argue w/ people about this all the time re: the open source models.
@emollick It’s a worthy research problem, but it’s really bloody hard. Nous Research with MoA has been working on some of this I believe, which is pretty interesting.
@emollick Routing is something that I feel should be "learned" dynamically, for example via various signals/cues. Static pre-made classifier just doesn't cut it.
@emollick interesting seeing pre-classifiers making rounds again after gpt-5 proved the opposite
feels like solving the problem twice with extra steps
@emollick the routing overhead usually eats the savings anyway by the time you tune it well enough to trust it
@emollick the pre-classifier paradox never dies. you need to solve the hard problem just to know which task is easy.
@emollick Model routing is simple: just determine if a solution to the problem can be computed in polynomial time before choosing which model to route it to.