Reactions
8 postsCH 
Chubby♨️@kimmonismusThe release of Kimi K3 would be a good time to release GPT-OSS-2, although I don't believe it will happen. The real Open AI currently comes from China, and that won't change for the time being. That should at least give us something to think about.…
- likes: 448
- replies: 29
- bookmarks: 36
- reposts: 23
SE SemiAnalysis@SemiAnalysis_Secondly, although Kimi Delta Attention has up to 10× lower networking requirements for KV-cache transfers, its large weights require even more network bandwidth to implement an optimization called WideEP, which spreads the weights across different GPUs. 3/8🧵
- likes: 210
- replies: 1
- bookmarks: 22
- reposts: 12
DM Dave Morin 🦞@davemorinBring out the B300s. https://twitter.com/SemiAnalysis_/status/2077966560447074689
- likes: 26
- replies: 1
- bookmarks: 4
- reposts: 0
SE SemiAnalysis@SemiAnalysis_Furthermore, since the weights occupy more than 1.5 TB of HBM capacity, the KV cache for K3’s KDA and Gated MLA will need to be offloaded to CPU DDR5 and NVMe, even at relatively low user concurrency, because little space remains in HBM. 6/8🧵
- likes: 164
- replies: 1
- bookmarks: 17
- reposts: 11
SE SemiAnalysis@SemiAnalysis_Kimi themselves have stated that optimal K3 inferencing will require a rack witha n large scale up domain with at least 64 chips. 7/8🧵
- likes: 206
- replies: 3
- bookmarks: 24
- reposts: 12
SE SemiAnalysis@SemiAnalysis_Lastly, Jevons’ Paradox means that making attention more efficient will drive wider AI adoption, which will ultimately require more GPUs, HBM, DRAM, and networking—not less. 8/8🧵
- likes: 485
- replies: 17
- bookmarks: 77
- reposts: 27