DFlash releases as an open-source block diffusion model for speculative decoding, reaching 15x throughput on Blackwell

Read the full deep dive here https://nvda.ws/4uOFHa3

Increase inference performance by up to 15x without sacrificing responsiveness.

DFlash, an open source lightweight block diffusion model designed for speculative decoding, delivers up to 15x higher throughput on NVIDIA Blackwell while maintaining the same user interactivity target.

Instead of drafting tokens one at a time, it proposes a whole block in a single pass for the main model to verify in parallel.

Adoption is drop-in with support in @lmsysorg SGLang, TensorRT-LLM, and @vllm_project.

@NVIDIAAI Is there anybody porting dlash into mlx or llama.cpp for local AI?

@NVIDIAAI 🔥🛢🔥

@NVIDIAAI This is the way.

@NVIDIAAI Flash is awesome

But dear @NVIDIAAI 👇

@NVIDIAAI Being able to swap Eagle3 for DFlash is 🔥

@NVIDIAAI yes indeed. I have some struggles with it though. anyone have recommended recipes that use dflash?

@NVIDIAAI Speculative decoding is shifting inference from token-by-token bottlenecks to parallel block verification, which is why stacks like LMSYS Org SGLang, TensorRT-LLM, and vLLM are suddenly unlocking massive throughput gains.

@NVIDIAAI Ya this has been around for months now, but thanks….?

@NVIDIAAI Drafting autoregressively always felt like a halfway solution since you're still bottlenecked by sequential steps. Generating the draft block in parallel via diffusion is the real unlock.

@NVIDIAAI Cool

@NVIDIAAI not sure 'same user interactivity' holds when draft acceptance rate drops i tested speculative decoding on financial text and the throughput gains basically vanished around 60% acceptance. is the 15x on general benchmarks or does it hold on domain-specific inputs too?

@NVIDIAAI proposing a whole block for the main model to verify instead of drafting one token at a time is such a clean speedup trick

@NVIDIAAI Great to see drop-in support for SGLang, vLLM, and TensorRT-LLM right out of the gate. Reducing drafting latency without sacrificing output quality is exactly what we need to scale long CoT reasoning models in production.

@sgl_project I have a great ideas 💡 for your projects 📈 Can I get follow back ? 🔙

@sgl_project LFG Let's pump it up together 💪

@sgl_project Laz.zAmazing work. Reach out anytime, I'd love to chat 🔥

@sgl_project @sgl_project Great project. Follow back or dm me 📩🤝

@NVIDIAAI DFlash is cool.

But what we’re cooking in our lab right now makes today’s inference stack look like a warm-up. 🚀👨‍🚀