The method lowers computational costs during generation for tested models.
🚀 Excited to share my @GoogleDeepMind student researcher project: Dual-Rate Diffusion✨
⚡ A simple construction that speeds up both regular diffusion and distilled models by interleaving a heavy context encoder with a light conditional denoiser.
🧵👇
Users are excited about Dual-Rate Diffusion from Google DeepMind because it straightforwardly improves quality-compute tradeoffs for regular diffusion models while praising the collaborative research team.
No Digg Deeper questions have been answered for this story yet.
📈 It is very straightforward to implement for regular diffusion, where it already improves the quality-compute tradeoff. More importantly, we show that Dual-Rate can accelerate even few-step distilled models like MMD while preserving sample quality gains over the teacher model🤯
🙏 I was lucky to work with a great team: @djjruhe, @emiel_hoogeboom, @JonathanHeek, Thomas Mensink and @TimSalimans
📜 Check out the paper for more details: https://arxiv.org/abs/2605.18190
💡 The core idea of Dual-Rate is to give the model a high-dimensional space to easily store computation from previous steps. Instead of one large denoising model we use two: a context encoder evaluated sparsely and a light denoiser conditioned on the encoder’s representations.
🤔 Regular diffusion models generate samples through iterative refinement. Although sample quality improves over time, the model still has to repeat much of the same work at every step just to understand what it is looking at and to build useful features.
@GrigoryBartosh @GoogleDeepMind Neat idea. Do you share code and a simple rerun script, and how does FID or throughput compare to a strong baseline at the same compute? Curious where the gains land.
