Loading activity…
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join http://slack.vllm.ai to discuss together with the community!
AI
The vLLM project estimates $38.96 per GPU-hour in revenue on AgentX at 196 P90 TPS, using DeepSeek’s off-peak API pricing.
AI
DeepSeek's README offers offline inference for DeepSeek-V4.1-Flash but lists additional systems needed for a complete service.
AI
DeepSeek says the model is live on its website, app and API, and optimized for DeepSeekHarness and Tencent CodeBuddy.
AI
The vLLM project reports 24.4% higher per-GPU throughput at 50 tokens per second per user.
AI
Reflection says the release includes Mirror CLI, its open-source training harness, and collaborations with 40-plus partners.
AI
finkd says the model runs well on new Mac Studios, crediting work with Apple and John Ternus.
AI
vLLM says making speculative decoding easier to put into practice benefits the inference community.
AI
SemiAnalysis also claims up to 10x better performance per dollar than GB300 NVL72 on the vLLM engine.
AI
The vLLM project says it delivers 5.3× the throughput at 150 tokens per second per user on SemiAnalysis AgentX.
AI
Google says its new open model, built on Gemma 4 and released under Apache 2.0, maps text, images, video, audio and code into one embedding space while running locally on phones and desktops.
Technology
The event listing schedules community demos and a meetup before a dance party at The Midway in San Francisco.
AI
The event listing says the free San Francisco gathering will feature community demos and a dance party.
AI
The vLLM project says its open models range from 0.6B to 27B and answer 64 questions about a request in one forward pass.
AI
vLLM says the model provides confidence with each answer to yes/no, multiple-choice or scored questions. Google Gemma puts Cloud Run costs at roughly $3 an hour, dropping to $0 when idle.
AI
Inferact says it co-led the optimization effort with Red Hat AI, Nvidia and Huawei.
Technology
DeepSeek says its KV cache, which lets the model reuse earlier calculations, needs a quarter of the high-bandwidth memory and an eighth of the SSD storage required by the previous generation's cache.
AI
Fleet of six open models spans 0.9 billion to 375 billion parameters.
AI
Official account highlights public tools for real-time interactive video.
AI
Mark Zuckerberg posted that Meta opened weights for a local model with another release planned soon.
