Genius idea for AI inference!
A marketplace that routes requests to the cheapest qualifying model at any given point.
This can get you up to 87% cheaper inference!
Today, if you need a model, you pay the vendor's fixed rate card, but that's about to change with this:
The marketplace is called "The Grid" and it just launched.
The idea is simple:
• You pick a quality tier (standard, prime, max) • The Grid routes your requests • You get the cheapest model within your tier
You don't pick a model. You pick the tier and the market fills it at the best available price.
They audit suppliers against benchmarks. If a model drifts below the threshold for its tier, they pull it from the eligible set.
Basically, they are trying to ensure you never get cheap junk.
They have an OpenAI-compatible API, so you literally have to change one line of code to start using them. And if you don't like it, undo the change and you are right back with your previous vendor.
New accounts get the first 200M tokens free. Worth a look if you're spending real money on inference.
Here is the link to check them out: https://app.thegrid.ai/consumption
Thanks to the team for partnering with me on this post.
Genius idea for AI inference!
A marketplace that routes requests to the cheapest qualifying model at any given point.
This can get you up to 87% cheaper inference!
Today, if you need a model, you pay the vendor's fixed rate card, but that's about to change with this:
Genius idea for AI inference!
A marketplace that routes requests to the cheapest qualifying model at any given point.
This can get you up to 87% cheaper inference!
Today, if you need a model, you pay the vendor's fixed rate card, but that's about to change with this: