DeepMind Releases DiffusionGemma Technical Report

Our team just released 55 pages of learning and results of making DiffusionGemma. I do recommend for anyone interested in text diffusion models. Big shout out to @jean_tarbou, @bshahr, Martin, James, Daniele, @cindyxywu, @flennerhag, @bodonoghue85, Pasha and many others who have… Show more

arXiv.orgDiffusionGemma Technical ReportWe introduce DiffusionGemma, an experimental open-weight language model that uses discrete diffusion to generate text at exceptionally high speed. Rather than decoding one token at a time,...

We just released the DiffusionGemma tech report. Text diffusion models open up a radically different part of the latency–quality Pareto frontier 👇 We hope the report makes it easier for researchers and engineers to understand the model, build on it, and create things we… Show more

No notes, incredible diagram

Our tech report on DiffusionGemma is out! 🚀 My biggest highlight? Online post-training is a game changer for text diffusion. We performed joint Sampler Distillation and Reinforcement Learning (SD⋅RL) after SFT, pushing the quality-speed Pareto frontier into a new regime 📈

We just released the DiffusionGemma tech report. Text diffusion models open up a radically different part of the latency–quality Pareto frontier 👇 We hope the report makes it easier for researchers and engineers to understand the model, build on it, and create things we… Show more

This is really impressive on performance, but such «iterative refinement» use case seems almost maximally unconvincing. I have been a diffusion LM bear for years, maybe Google will finally prove me wrong by scaling this up.