Report
Asynchronous inference could make up 95% or more of inference demand
An interviewer for Substrate says Neil Movva also discussed alternative chips and diseconomies of scale at massive datacenters.
TLDR
In an interview for Substrate, Neil Movva predicted asynchronous inference could account for 95% or more of inference demand. The interviewer says the conversation also covered latency-insensitive inference, alternative chips and why massive datacenters face diseconomies of scale.
Combined views
—
2 Sources, first seen ago
