Reaction
The AI app infrastructure rabbit hole
A post sketches how hosting inference to improve margins can lead to supporting multiple models, then adding routing and tracing.
TLDR
One post jokes that an AI app hosting its own inference to improve margins could wind up supporting several models, building a router and adding tracing when people want to evaluate. The punchline: everyone ends up building the same thing.
Combined views
4K
2 Sources, first seen ago