Kimi K3's Linear Attention Increases Demand for NVIDIA NVL72 Clusters
Reactions from ranked influencers
3 posts
Kimi K3 is actually quite positive for NVIDIA, as large-model inference is where the NVL72 shines. Because K3 has more than 2.8 trillion parameters, it requires a large scale-up domain to store its weights. 2/8🧵
The unfortunate downside of the WideEP optimization is that it consumes a tremendous amount of network bandwidth. WideEP is highly optimized for rack-scale systems like the GB200/GB300 NVL72, whose copper backplane provides 18× more bandwidth than comparable DGX B200 systems. 5/8🧵
WideEP distributes the 896 experts across many GPUs so that each GPU’s HBM contains only a small number of experts, optimizing per-token memory usage and compute utilization. 4/8🧵