Inngest Builds Custom Neocloud for 20x Lower AI Infrastructure Costs
Reactions from ranked influencers
1 post
The way we do evals right now is batshit insane. Using an LLM as the judge is basically asking the criminal who did the crime. It's also insanely expensive to put all that context into another LLM and asking it to evaluate whether it thinks it was correct."
New @ThePeelPod with @itstonyhb - the hidden layer of AI infrastructure - why evals today are "batshit insane" - building @inngest's own cloud to get 20x lower cost - growing 35x YoY after AWS and Cloudflare copied them, and - building a company without a personal brand Timestamps: 0:00 The hidden infra layer every AI agent runs on 1:46 Building complex chains of logic 3:31 Why agent SDK's don't go far enough 4:49 Healthcare was the original event-driven nightmare 6:32 Storing traces on your infrastructure enables self-improving loops 14:26 Why Inngest was already in the right place for AI 15:49 Score agents off product events, not LLM's 17:31 The OpenAI copy-paste signal 21:24 Swap in LLMs and cut costs 23:44 How customers pulled the product forward 25:41 Orchestration belongs outside the sandbox 29:48 Building a neocloud to cut costs 20x 32:09 Most neoclouds just resell AWS 32:54 AI infrastructure is all converging 34:49 Why Claude can't just build your backend 36:44 How to build a software factory 39:12 Agents are a lottery you get addicted to 42:44 Loops must always exist 45:38 If models keep getting better, why orchestrate? 48:28 When incumbents steal your features 52:30 You can't vibe code infrastructure 55:54 Why Tony has no personal brand 59:38 Dev tools GTM without Twitter 1:03:20 Lessons from the founder of DuckDuckGo 1:10:39 Truth as a company value 1:13:08 Taking too long adapting to AI 1:15:10 Startups are 100% R&D 1:17:19 Ali from Databricks 1:19:03 Writing his own code, Voice-to-text with local models 1:23:53 Evals are batshit insane