vLLM

AI

Turning DiffusionGemma into a Jev-like model with vLLM

Google Gemma says seeding a canvas with a response template lets vLLM extract confidence scores and probability distributions for yes/no, multiple-choice and scored questions in one denoising step.

BO

GG

2 Sources, 9h ago, first seen 9h ago

Technology

AI Judge Discovers FlashInfer Edge Cases Affecting Inference

Josh Tobin reports using their reward hacking judge to identify issues in FlashInfer.

JT

1 Source, 33d ago, first seen 33d ago

AI

Researcher Reports Bugs Found in PyTorch and vLLM

AI researcher Jack Morris posted about using coding agents to find bugs in several frameworks.

JM

JT

CM

4 Sources, 59d ago, first seen 59d ago

AI

Netflix Builds In-House LLM Serving Platform Using Vllm And Triton

1 Source, 62d ago, first seen 62d ago

AI

vLLM Team Shares Optimizations for Serving 2.8T Model

2 Sources, 62d ago, first seen 62d ago

Recent stories

Turning DiffusionGemma into a Jev-like model with vLLM

AI Judge Discovers FlashInfer Edge Cases Affecting Inference

Researcher Reports Bugs Found in PyTorch and vLLM

Netflix Builds In-House LLM Serving Platform Using Vllm And Triton

vLLM Team Shares Optimizations for Serving 2.8T Model

vLLM

Recent stories

Turning DiffusionGemma into a Jev-like model with vLLM

AI Judge Discovers FlashInfer Edge Cases Affecting Inference

Researcher Reports Bugs Found in PyTorch and vLLM

Netflix Builds In-House LLM Serving Platform Using Vllm And Triton

vLLM Team Shares Optimizations for Serving 2.8T Model