MI355X reportedly offers up to 1.7x better performance per dollar than DGX B300
SemiAnalysis says a new SGLang feature removes duplicate KVCache data between local and distributed DRAM, allowing up to twice as much KVCache to be stored there.
TLDR
SemiAnalysis says MI355X offers up to 1.7x better performance per dollar than DGX B300. It also describes UnifiedRadixCache external cache, a new SGLang feature co-designed by AMD’s Mainland China UMBP team, Alibaba and the SGLang community. According to SemiAnalysis, the feature removes duplicated KVCache between local L2 DRAM and distributed L3 DRAM, allowing up to twice as much KVCache to be stored in DRAM.
Combined views
—
1 Source, first seen ago
— likes— comments— saves— reposts
