Ruilong Li and Zan Gojcic release FlashDreams, an open-source inference library delivering up to 3.10x speedups for autoregressive world models

VIEWS11.1KBOOKMARKS31LIKES80REPLIES1

Super excited to open source Flashdreams, acceleration library for world models! No restrictions on usage, Apache 2 license.

World models are the next frontier for video generation and paramount to Physical AI.

Flashdreams yields out of the box speed up across world models

Ruilong Li@ruilong_li

World models are moving beyond offline generation towards interactive, real-time experiences.

Introducing ⚡FlashDreams⚡: an open-source high-performance inference and serving library built for autoregressive world models:

🔥 Up to 3.10× faster LingBot-World inference 🔥 Up to 2.12× faster Self-Forcing inference 🔥 Up to 1.40× faster Wan2.1 inference 🔥 8 integrated models 🔥 Multi-GPU, streaming, low-latency serving 🔥 Agentic skills that teach you how to use it

FlashDreams is designed for a new generation of AI systems that continuously evolve over time while responding to user interactions. It powers applications across robotics, autonomous vehicle simulation, gaming, and virtual worlds.

Github: https://github.com/NVIDIA/flashdreams Docs: https://nvidia.github.io/flashdreams Research page: https://research.nvidia.com/labs/sil/projects/flashdreams/ Join the #flashdreams Discord channel at https://discord.gg/yTdHDqFP

FlashDreams is also the runtime backbone behind NVIDIA OmniDreams (https://github.com/nv-tlabs/omni-dreams)

1/n

#AI #WorldModels #FastInference #PhysicalAI #OpenSource #NVIDIA

27d11.1K8031

RETWEETS51

Ruilong Li@ruilong_li

World models are moving beyond offline generation towards interactive, real-time experiences.

Introducing ⚡FlashDreams⚡: an open-source high-performance inference and serving library built for autoregressive world models:

🔥 Up to 3.10× faster LingBot-World inference 🔥 Up to 2.12× faster Self-Forcing inference 🔥 Up to 1.40× faster Wan2.1 inference 🔥 8 integrated models 🔥 Multi-GPU, streaming, low-latency serving 🔥 Agentic skills that teach you how to use it

FlashDreams is designed for a new generation of AI systems that continuously evolve over time while responding to user interactions. It powers applications across robotics, autonomous vehicle simulation, gaming, and virtual worlds.

Github: https://github.com/NVIDIA/flashdreams Docs: https://nvidia.github.io/flashdreams Research page: https://research.nvidia.com/labs/sil/projects/flashdreams/ Join the #flashdreams Discord channel at https://discord.gg/yTdHDqFP

FlashDreams is also the runtime backbone behind NVIDIA OmniDreams (https://github.com/nv-tlabs/omni-dreams)

1/n

#AI #WorldModels #FastInference #PhysicalAI #OpenSource #NVIDIA

27d48.1K308175

Zian Wang@zianwang97

World models are moving from offline video generation to interactive, real-time systems.

⚡ Check out FlashDreams, our step toward the “vLLM moment” for world models: a high-performance serving stack with persistent sessions, streaming I/O, rolling model states, and multi-GPU execution.

Now supporting OmniDreams, Self-Forcing, LingBot-World, FlashVSR, and more!

Ruilong Li@ruilong_li

World models are moving beyond offline generation towards interactive, real-time experiences.

Introducing ⚡FlashDreams⚡: an open-source high-performance inference and serving library built for autoregressive world models:

🔥 Up to 3.10× faster LingBot-World inference 🔥 Up to 2.12× faster Self-Forcing inference 🔥 Up to 1.40× faster Wan2.1 inference 🔥 8 integrated models 🔥 Multi-GPU, streaming, low-latency serving 🔥 Agentic skills that teach you how to use it

FlashDreams is designed for a new generation of AI systems that continuously evolve over time while responding to user interactions. It powers applications across robotics, autonomous vehicle simulation, gaming, and virtual worlds.

Github: https://github.com/NVIDIA/flashdreams Docs: https://nvidia.github.io/flashdreams Research page: https://research.nvidia.com/labs/sil/projects/flashdreams/ Join the #flashdreams Discord channel at https://discord.gg/yTdHDqFP

FlashDreams is also the runtime backbone behind NVIDIA OmniDreams (https://github.com/nv-tlabs/omni-dreams)

1/n

#AI #WorldModels #FastInference #PhysicalAI #OpenSource #NVIDIA

27d4.1K3312

Zan Gojcic@ZGojcic

Interactive world models are finally getting their vLLM/sglang moment

We’re open-sourcing FlashDreams, an optimized library for inference and serving of interactive world models

FlashDreams is released under Apache 2.0, and we’re excited to improve it further with the community

Ruilong Li@ruilong_li

World models are moving beyond offline generation towards interactive, real-time experiences.

Introducing ⚡FlashDreams⚡: an open-source high-performance inference and serving library built for autoregressive world models:

🔥 Up to 3.10× faster LingBot-World inference 🔥 Up to 2.12× faster Self-Forcing inference 🔥 Up to 1.40× faster Wan2.1 inference 🔥 8 integrated models 🔥 Multi-GPU, streaming, low-latency serving 🔥 Agentic skills that teach you how to use it

FlashDreams is designed for a new generation of AI systems that continuously evolve over time while responding to user interactions. It powers applications across robotics, autonomous vehicle simulation, gaming, and virtual worlds.

Github: https://github.com/NVIDIA/flashdreams Docs: https://nvidia.github.io/flashdreams Research page: https://research.nvidia.com/labs/sil/projects/flashdreams/ Join the #flashdreams Discord channel at https://discord.gg/yTdHDqFP

FlashDreams is also the runtime backbone behind NVIDIA OmniDreams (https://github.com/nv-tlabs/omni-dreams)

1/n

#AI #WorldModels #FastInference #PhysicalAI #OpenSource #NVIDIA

26d1.8K323

Ruilong Li@ruilong_li

A world model learns to generate and evolve an environment over time: video, actions, state, audio, sensors, control signals, and more.

Serving it means keeping a live session running while input, model state, GPU inference, and output evolve together.

3/n

27d22631

Ruilong Li@ruilong_li

FlashDreams is more than an inference engine. It is a serving stack purpose-built for interactive world-model applications, supporting persistent sessions, streaming inputs and outputs, rolling model state, and multi-GPU execution.

FlashDreams already integrates eight world models, with many more on the way.

7/n

27d11231

Ruilong Li@ruilong_li

Similar to vLLM and SGLang, FlashDreams is built from the ground up for high-performance serving.

FlashDreams delivers up to 2.12× speedup on Self-Forcing, 3.10× on LingBot-World, and 1.40× on Wan2.1. The system is designed specifically for autoregressive world models, where generation unfolds continuously over time. At the same time, the architecture naturally extends to bidirectional models by treating each generation task as a single autoregressive rollout.

6/n

27d8331

Ruilong Li@ruilong_li

Efficient serving infra like vLLM and SGLang helped move LLMs from research checkpoints into real apps.

World models are reaching a similar moment, but with a different interactive pattern than LLM: initialize once, then keep evolving the world forward at a constant pace.

5/n

27d7621

Ruilong Li@ruilong_li

Ending this thread with a fun walk on w/ FlashDreams:

Try it out and tag us with your exploration. #flashdreams

27d9541

Ruilong Li@ruilong_li

World models are quickly advancing, but for most users the experience is still passive: watching impressive promo videos, not running the models, probing them, or building applications on top of them.

FlashDreams aims to change that. Spin up a server, select a model, and interact with it live.

2/n

27d2783

Ruilong Li@ruilong_li

Serving a world model is different from offline video generation.

The goal is not just prompt → wait → video. A live world-model runtime must stay responsive while the world keeps advancing: low latency, high throughput, steady streaming, and rolling state.

4/n

27d761

Yossi Gandelsman @CVPR@YGandelsman

@ruilong_li congrats!

26d281

Ruilong Li@ruilong_li

The team behind this is extremely talented. I can't be more happy to work with them! Listing in random order:

@jatentaki @wilson_over @jmartinezesturo @JunchenLiu77 @csyonghe, Gangzheng Tong, Jesse Archer, @mccaffrey_j_b, William Lew, @heuristic42, Aidan Foster, Wenqing Wang, @AdityaMaha47495, @FidlerSanja @ZGojcic, @natdoesgraphics, Matt Cragun, @wang_jialiang, Anis Ahmad, @killerooo, Deep Shekhar, Michael Chock, @ATMGuille9

26d1203

Ruilong Li@ruilong_li

@YGandelsman to you as well! big day for both of us : )

26d172