Emad Mostaque Predicts Kimi K3 Inference Costs Will Drop 10-50x
Reactions from ranked influencers
2 posts
This was from Dario Amodei some time back, on open-source AI models. "When I think about competition I think about which models are good at the tasks that we do. I think open source is actually a red herring. It's not free. You have to run it on inference and someone has to make it fast on inference." --- From 'Alex Kantrowitz' YT channel (full video link in comment)
Great explanation by Emad Mostaque, co-founder of Stability AI. "We’ll see the cost of Kimi K3 drop by 10 to 50 times, I think, over the next few months as it gets optimized. " Basically Kimi K3’s current inference cost is quite high, but that price reflects immature infrastructure, not a permanent technical limit. And that gap will not last long. US-based specialized infrastructure companies will optimize kernels, routing, quantization, batching, memory use, and serving systems around those models once the Kimi K3 weights are available. --- "Right now, it uses twice the number of tokens for the same task compared with GPT-5.6. Again, we’re going to see that cost drop because everyone and their dog is going to optimize the crap out of this. Fireworks has just raised funding at a $17 billion valuation, while others, such as Modal and Baseten, are valued at $10 billion. These are inference providers for open-source models. They’ve all raised around a billion dollars, which they’re now going to spend on optimizing the Chinese model, making it more efficient, and running it. American labs that handle the inference side of things are going to optimize the crap out of this. Therefore, we will see it catch up." ---- From "Peter H. Diamandis" YouTube channel, (full video link in comment)