vLLM

AI

Turning DiffusionGemma into a Jev-like model with vLLM

Google Gemma says seeding a canvas with a response template lets vLLM extract confidence scores and probability distributions for yes/no, multiple-choice and scored questions in one denoising step.

BO

GG

2 Sources, 11d ago, first seen 11d ago

Technology

AI Judge Discovers FlashInfer Edge Cases Affecting Inference

Josh Tobin reports using their reward hacking judge to identify issues in FlashInfer.

JT

1 Source, 44d ago, first seen 44d ago

AI

Researcher Reports Bugs Found in PyTorch and vLLM

AI researcher Jack Morris posted about using coding agents to find bugs in several frameworks.

JM

JT

CM

4 Sources, 70d ago, first seen 70d ago

AI

Netflix Builds In-House LLM Serving Platform Using Vllm And Triton

1 Source, 73d ago, first seen 73d ago

AI

vLLM Team Shares Optimizations for Serving 2.8T Model

2 Sources, 73d ago, first seen 73d ago

Recent stories

Turning DiffusionGemma into a Jev-like model with vLLM

AI Judge Discovers FlashInfer Edge Cases Affecting Inference

Researcher Reports Bugs Found in PyTorch and vLLM

Netflix Builds In-House LLM Serving Platform Using Vllm And Triton

vLLM Team Shares Optimizations for Serving 2.8T Model

vLLM

Recent stories

Turning DiffusionGemma into a Jev-like model with vLLM

AI Judge Discovers FlashInfer Edge Cases Affecting Inference

Researcher Reports Bugs Found in PyTorch and vLLM

Netflix Builds In-House LLM Serving Platform Using Vllm And Triton

vLLM Team Shares Optimizations for Serving 2.8T Model