Announcement
SSEngine’s creator reports 3.1–3.2× the request rate of tested vLLM and SGLang setups
@KanZhu854772 says the GLM-5.3-Flash prefill engine runs on four B200 GPUs with a three-tier prefix cache.
TLDR
In @KanZhu854772’s tests, SSEngine-GLM5.3-Flash sustained 3.1× the request rate on AgentX and 3.2× on TraceLab compared with every vLLM or SGLang setup tested. The creator describes an open-source prefill engine running on four B200 GPUs, combining pipeline parallelism, prefix caching across GPU memory, DRAM and SSD, and a scheduler that puts the shortest remaining prefill work first.
Combined views
—
4 Sources, first seen ago
Comments