Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

World & Business

Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

Wasserstein-Tilted Flow Maps proposed for simulation-free RL fine-tuning · Digg

Announcement

Wasserstein-Tilted Flow Maps proposed for simulation-free RL fine-tuning

The team says a transport-based alternative to KL regularization is key to its approach for diffusion and flow models.

TLDR

The team introduced Wasserstein-Tilted Flow Maps as an approach to reinforcement-learning fine-tuning for diffusion and flow models. It says flow maps enable fine-tuning without simulation, avoiding rollouts that are often expensive, and identifies a transport-based alternative to KL regularization as the key. The team described the work as a NeurIPS 2026 spotlight.

Combined views

5.5K

4 Sources, first seen 2h ago

Combined views

5.5K

4 Sources, first seen 2h ago

147 likes

2h ago

first seen 2h ago

147 likes

4 comments

100 saves

43 reposts

Featured Source

4 comments

100 saves

43 reposts

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Today's Rank

—

Not ranked yet

Today's Rank

—

Not ranked yet

4 Sources

@jrrhuang#NeurIPS2026 Spotlight RL for diffusion and flow models often requires expensive rollouts. We show that flow maps enable simulation-free RL finetuning. Introducing Wasserstein-Tilted Flow Maps (WTF)! 🤩 The key is a transport-based alternative to KL regularization. 🧵2h

@nmboffii am extremely excited to announce our latest flow map finetuning algorithm: Wasserstein Tilted Flows (WTF?!) something i’ve been thinking about for a while now is the “right” way to formulate finetuning and reward alignment of flow-based generative models. two difficulties that arise repeatedly are 1) sampling the canonical “reward-tilted distribution” is really hard, and 2) basically all existing methods require backpropagation or adjoint methods through flow rollouts, which is prohibitively expensive the core issue i’ve had with the reward tilt is that it arises from a geometry on the space of probability measures based on the KL divergence. this makes sense for stochastic processes such as diffusions, but for the deterministic flows we use in practice, this choice of geometry can be very awkward, leading state of the art methods to repeatedly convert back and forth between a flow and a diffusion here we throw out this formulation entirely, redefining the reward alignment problem in terms of a carefully-chosen wasserstein distance. we show that that this leads to a deterministic optimal control problem, which we can solve rapidly via reinforcement learning in a simulation-free by way leveraging the underlying flow map. critically, using the flow map avoids backpropagation through rollouts, replacing them by a single function evaluation. by adapting both the flow map and its underlying flow together online, we obtain an end-to-end finetuning algorithm that takes a flow map in, gives a flow map out, and is ~50-100x faster than current state of the art approaches super proud of my students for their hard work on this one — check it out!2h

@sedielemRT @jrrhuang: #NeurIPS2026 Spotlight RL for diffusion and flow models often requires expensive rollouts. We show that flow maps enable sim…1h

@brandondamosRT @nmboffi: i am extremely excited to announce our latest flow map finetuning algorithm: Wasserstein Tilted Flows (WTF?!) something i’ve…1h

4 Sources

@jrrhuang#NeurIPS2026 Spotlight RL for diffusion and flow models often requires expensive rollouts. We show that flow maps enable simulation-free RL finetuning. Introducing Wasserstein-Tilted Flow Maps (WTF)! 🤩 The key is a transport-based alternative to KL regularization. 🧵2h

@nmboffii am extremely excited to announce our latest flow map finetuning algorithm: Wasserstein Tilted Flows (WTF?!) something i’ve been thinking about for a while now is the “right” way to formulate finetuning and reward alignment of flow-based generative models. two difficulties that arise repeatedly are 1) sampling the canonical “reward-tilted distribution” is really hard, and 2) basically all existing methods require backpropagation or adjoint methods through flow rollouts, which is prohibitively expensive the core issue i’ve had with the reward tilt is that it arises from a geometry on the space of probability measures based on the KL divergence. this makes sense for stochastic processes such as diffusions, but for the deterministic flows we use in practice, this choice of geometry can be very awkward, leading state of the art methods to repeatedly convert back and forth between a flow and a diffusion here we throw out this formulation entirely, redefining the reward alignment problem in terms of a carefully-chosen wasserstein distance. we show that that this leads to a deterministic optimal control problem, which we can solve rapidly via reinforcement learning in a simulation-free by way leveraging the underlying flow map. critically, using the flow map avoids backpropagation through rollouts, replacing them by a single function evaluation. by adapting both the flow map and its underlying flow together online, we obtain an end-to-end finetuning algorithm that takes a flow map in, gives a flow map out, and is ~50-100x faster than current state of the art approaches super proud of my students for their hard work on this one — check it out!2h

@sedielemRT @jrrhuang: #NeurIPS2026 Spotlight RL for diffusion and flow models often requires expensive rollouts. We show that flow maps enable sim…1h

@brandondamosRT @nmboffi: i am extremely excited to announce our latest flow map finetuning algorithm: Wasserstein Tilted Flows (WTF?!) something i’ve…1h