Reactions from ranked influencers
4 posts
Kimi K3 by @Kimi_Moonshot is a true frontier model, open source! Congrats! We don't have all details (training tokens etc), but would estimate fp16 pretraining w/ muon => MXFP4/MXFP8 for SFT stage ~1e25 flops total (~Inkling!) Total cost: $15-$25m Scale isn't all you need!
Worth noting the EU AI act put the threshold for AI models with systemic risk at 1e25 flops (!) US executive order put it at 1e26 flops of training compute.. Oddly prescient
Kimi K3 by @Kimi_Moonshot is a true frontier model, open source! Congrats! We don't have all details (training tokens etc), but would estimate fp16 pretraining w/ muon => MXFP4/MXFP8 for SFT stage ~1e25 flops total (~Inkling!) Total cost: $15-$25m Scale isn't all you need!
Is this like 1/8th the cost of a Hollywood blockbuster film?
Kimi K3 by @Kimi_Moonshot is a true frontier model, open source! Congrats! We don't have all details (training tokens etc), but would estimate fp16 pretraining w/ muon => MXFP4/MXFP8 for SFT stage ~1e25 flops total (~Inkling!) Total cost: $15-$25m Scale isn't all you need!
Kimi K3 seems fine-tuned to dominate the UI coding space. What other domain can benefit from a $18m training run?