@sgl_project
Run LLMs fast at any scale 🔗 https://github.com/sgl-project/sglang Join our community https://slack.sglang.io For AI tech blogs & deep-dives 👉 @lmsysorg
AI
Mistral announced the weights after a public preview, but its linked main Hugging Face listing showed “Upcoming release.”
AI
SGLang cites results on B300 and GB300 hardware; vLLM's decode claim covers batch sizes 8 to 256.
AI
NVIDIA says Vera Rubin offers Day 0 support for the model, with throughput of up to 200K TPS/GPU.
AI
The SGLang project says early reinforcement-learning training on TPUs with Miles and TorchTitan produced promising results.
AI
Its developer says Flint exceeds 125 tokens per second on an M4 Pro MacBook Pro at short contexts.
XAI
Reflection says the release includes Mirror CLI, its open-source training harness, and collaborations with 40-plus partners.
AI
finkd says the model runs well on new Mac Studios, crediting work with Apple and John Ternus.
AI
BFL says it runs on a single consumer GPU and offers native 4K generation, bounding boxes and pixel-perfect editing.
XAI
NVIDIA says Step 5 Preview runs at nearly 8K tokens per second per GPU on GB300 using NVFP4.
AI
SemiAnalysis says the comparison uses SGLang, which it describes as a widely used production LLM engine.
AI
The SGLang team says MoE tail fusion delivered a 5.9% end-to-end inference speedup on early-access Rubin hardware.
AI
SGLang says the Fort Mason event will cover AI infrastructure, silicon, models and applications through talks and workshops.
AI
SGLang says the San Francisco event will feature two days of talks, discussions and deep dives into frontier AI infrastructure, silicon, models and applications.
AI
SGLang says it turned Qwen3.8-27B into a multimodal decision model and added a /v1/decisions endpoint for using language and vision-language models in classification and scoring.
AI
Modal says Runtime is at The Midway and invites attendees to learn from engineers behind its partners’ tools. Partners helping put on the event include Nvidia, Pydantic and Temporal.
AI
The SGLang project announced a partnership with Halo, saying its engine runs in an isolated serving environment and sends generation data back to the trainer.
AI
Alibaba’s Qwen team says the 7-billion-parameter model can generate and edit transparent images and use up to 10 reference images. The publisher genztech.blog reports that its license is non-commercial only.
AI
The user claims V4.1 is cheaper to serve than V4-Flash but costs twice as much per output token even off-peak.
AI
Fleet of six open models spans 0.9 billion to 375 billion parameters.
AI
Cofounder of RadixArk retweeted an invitation to an evening on reinforcement learning infrastructure.
