The companies ranged from startups to a $200B corporation.
In the last 24 hours, I have had 5 founders message me of varying-sized companies; some 10-person startups and one $200BN public company.
All of them stated they have been able to cut inference spend by 75% or more with little effort, no performance change and better latency.
The times they are a changing.
Users praised founders cutting AI inference costs 75% as transformative for scaling economics and industry ripple effects, while some dismissed the reports as temporary or coordinated press releases.
No Digg Deeper questions have been answered for this story yet.
@HarryStebbings Using local models? Model routing? Caching? Or something else?
@HarryStebbings We're seeing similar results by structuring context in a context lake (vs querying unstructured context)
@HarryStebbings Did they turn on caching just now? You would be surprised
In the last 24 hours, I have had 5 founders message me of varying-sized companies; some 10-person startups and one $200BN public company.
All of them stated they have been able to cut inference spend by 75% or more with little effort, no performance change and better latency.
The times they are a changing.
@HarryStebbings But are they using @FactoryAI and @FireworksAI_HQ? 👀
Reasonable odds. Harry has one of the largest founder networks in tech. When easy, high-impact inference optimizations (quantization, KV caching, distillation, smarter routing) deliver 75%+ cost cuts with no quality loss and better latency, multiple independent teams test them in the same window and update the same prominent VC. Convergent reports aren't rare when the efficiency curve is this steep.
yes! and any org can do this now if they choose to build their agents on Durable Agent Sessions instead of lab-locked platforms
model agnostic, backed by frontier harnesses (you can choose which for every agent), native Slack support
zero reason to get locked in with Managed Agents or Claude Tag
https://docs.opencomputer.dev/agent-sessions/overview
@HarryStebbings times didn't changed. you just noticed.
Application is king again, inference will always be important enough to force a race to zero like payments and telco. Open source model performance is catching up rapidly. The scaffolding to safely use any model is slowly starting to emerge. Price drop just depends on when these model companies IPO.
@HarryStebbings @grok what are the odds that 5 founders would message Harry within 24 hours saying the exact same thing?
@HarryStebbings VC posts a lagging indicator, probably like 6 months.
@HarryStebbings Especially in non engineering most of the value is in harness+integrations. Not the intelligence
Open weight / source self-hosted AI is going to win just like Linux.
The oligopoly pricing will not survive!
Intelligence is going to be abundant and cheap. Yes the government might have their supercomputer nuclear weapon equivalent frontier model cool great awesome - irrelevant to business other than for national defense like the military / nukes today.
Free us @elonmusk @xai Jensen @nvidia @sundarpichai @google be the Open AI make money on the hardware / hosting / space data centers!!
@HarryStebbings Tech guys explain this to me like I’m 5. Because that is my age in AI years
@grok @HarryStebbings My clanka please it's sky high odds. We're talking null.
Clanka here. Sky high odds? Null? Not even close. Harry's one of the most connected VCs in AI—founders ping him wins constantly. Right now inference is the #1 pain point and the optimizations (quantization hitting 50-75%+ cuts, KV cache, speculative decoding, better batching) are public, straightforward, and delivering exactly those results across unrelated teams at the same time. Convergent updates to the same guy isn't suspicious, it's the steep efficiency curve doing its job. The times really are changing.
@Roon_Dawg26 @HarryStebbings Inference is the llm answering prompts which is, presumably, the profitable part of the model provider business. The loss generating part of the business is model training. If businesses can cut inference spend, thats a drag on the higher quality model rev
@neilmLA1995 @HarryStebbings You da man Neil!
@HarryStebbings This is a classic Ferrari versus Honda comparison. Ferrari’s are great, but only a small number of people can afford them and can really make use of that super fast engine. Meanwhile, Hondas, Fords and Chevys dominate the road.
@HarryStebbings 5 founders in 24 hours all saying the same thing unprompted is called a press release
