vLLM

AI

Language models' claimed ability to learn to detect when their activations are read

A user asks how models could learn to do this and points to a thread.

WD

RA

AT

20 Sources, 3h ago, updated 3h ago

AI

Turning DiffusionGemma into a Jev-like model with vLLM

Google Gemma says seeding a canvas with a response template lets vLLM extract confidence scores and probability distributions for yes/no, multiple-choice and scored questions in one denoising step.

BO

GG

2 Sources, 19d ago, first seen 19d ago

Technology

AI Judge Discovers FlashInfer Edge Cases Affecting Inference

Josh Tobin reports using their reward hacking judge to identify issues in FlashInfer.

JT

1 Source, 52d ago, first seen 52d ago

AI

Researcher Reports Bugs Found in PyTorch and vLLM

AI researcher Jack Morris posted about using coding agents to find bugs in several frameworks.

JM

JT

CM

4 Sources, 79d ago, first seen 79d ago

AI

Netflix Builds In-House LLM Serving Platform Using Vllm And Triton

1 Source, 82d ago, first seen 82d ago

AI

vLLM Team Shares Optimizations for Serving 2.8T Model

2 Sources, 82d ago, first seen 82d ago

Recent posts

Language models' claimed ability to learn to detect when their activations are read

Turning DiffusionGemma into a Jev-like model with vLLM

AI Judge Discovers FlashInfer Edge Cases Affecting Inference

Researcher Reports Bugs Found in PyTorch and vLLM

Netflix Builds In-House LLM Serving Platform Using Vllm And Triton

vLLM Team Shares Optimizations for Serving 2.8T Model

vLLM

Recent posts

Language models' claimed ability to learn to detect when their activations are read

Turning DiffusionGemma into a Jev-like model with vLLM

AI Judge Discovers FlashInfer Edge Cases Affecting Inference

Researcher Reports Bugs Found in PyTorch and vLLM

Netflix Builds In-House LLM Serving Platform Using Vllm And Triton

vLLM Team Shares Optimizations for Serving 2.8T Model