Announcement
SGLang introduces Low-Latency Mode with claimed per-user token-speed gains
SGLang says it built the mode with NVIDIA and AMD; more low-latency kernels remain in progress.
TLDR
In an AgentX trace replay by @SemiAnalysis_ of DeepSeek-V4.1-Flash with DSpark, SGLang reports up to 1.38× P90 per-user token speed on NVIDIA GB300 and 1.56× on AMD MI355X. SGLang also announced a custom-kernel plugin system with automatic fallback. Separately, @SemiAnalysis_ claims Megakernel support has merged and works in SGLang Docker images; SGLang's announcement makes neither claim.
Combined views
43
6 Sources, first seen ago
