DeepMind Releases DiffusionGemma Technical Report
Researchers at Google DeepMind detail their experimental open-weight language model using discrete diffusion.
Brendan O'Donoghue announced the release of the DiffusionGemma tech report. The document describes an experimental open-weight language model that uses discrete diffusion to generate text. It highlights advantages over autoregressive models, including bidirectional attention for iterative refinement and error correction. The approach shows benefits for small to medium batch sizes and longer contexts. Team members note that online post-training improves results on these models.
Our team just released 55 pages of learning and results of making DiffusionGemma. I do recommend for anyone interested in text diffusion models. Big shout out to @jean_tarbou, @bshahr, Martin, James, Daniele, @cindyxywu, @flennerhag, @bodonoghue85, Pasha and many others who have…

