What are "smart" model routers you know of? Services or vendors that take queries and route the most efficient model they deem, saving cost.
I sense there is a massive demand for these, and will be even more...
What are "smart" model routers you know of? Services or vendors that take queries and route the most efficient model they deem, saving cost.
I sense there is a massive demand for these, and will be even more...
Users criticized smart model routers for cost-efficient AI queries as ineffective or untrustworthy, citing Copilot picking expensive models, outdated options, and dishonest claims from firms like Cognit.
Solutions I collected so far (no affiliation with either):
- Factory Router - Not Diamond - Prism by Augment Code
AI gateways with routing:
- OpenRouter (auto router) - Kilo Gateway - Requestly - LiteLLM (auto routing)
More pointers welcome!
What are "smart" model routers you know of? Services or vendors that take queries and route the most efficient model they deem, saving cost.
I sense there is a massive demand for these, and will be even more...

@dabit3 @cognition yes but I have trust issues with Devin, given the company lied about Devin's capabilities on launch, never addressed it, apologized or corrected it
just ignoring Cognition till it happens. 2 years and counting I think

@GergelyOrosz @cognition Devin

@GergelyOrosz OpenRouter has one: https://openrouter.ai/openrouter/auto

@GergelyOrosz Cognition has adaptive routing right @dabit3

@GergelyOrosz Morph also have one: https://docs.morphllm.com/sdk/components/router

@GergelyOrosz routellm from lmsys is the open source take, martian and notdiamond sell it as a product. NB: router has to judge difficulty with a cheap model, and it misjudges where routing matters

@harderthanfire unclear to me if this works only with their models tho. reads like it?

@vladzima routellm has not been updated in 2 years tho?

@GergelyOrosz This is their example model selection and you can specify what provider and/or specific models:

@GergelyOrosz not diamond
@tomas_hk is a beast

@dabit3 @cognition this was the lie in the launch post
integrity matters at startups, and I do not see Cognition having much, never having owed up to this even
http://youtube.com/watch?v=tNmgmwEtoWE

@GergelyOrosz oh sht you're right I didn't even notice

@awakecoding @GergelyOrosz I though it was about optimizing for quality and not cost?

@mattlam_ @GergelyOrosz Yes! Thanks for tagging

@GergelyOrosz Copilot auto mode worked well for a minute but feels like it’s picking expensive models on purpose the past few days

@GergelyOrosz It is not necessarily about cost optimization, but there are routing options in AnythingLLM as well.

@GergelyOrosz There's no such thing as a good router for an agent.
Because: - 1st message is not indicative of "complexity" of a task - if switch in every message => dead cache => high cost
Just use the best open source provider. Don't try to trick yourself with routing.

@GergelyOrosz I've always wondered how effective those would be since the context cache wouldn't be effective. In almost all cases, it would only make sense to switch to a cheaper model and never to a more expensive model.
@GergelyOrosz cursor does that
What are "smart" model routers you know of? Services or vendors that take queries and route the most efficient model they deem, saving cost.
I sense there is a massive demand for these, and will be even more...