Zhihan Yang and seven co-authors submit RePlaid to arXiv, a continuous diffusion language model that matches scaling of state-of-the-art discrete diffusion models
Preprint dated 18 May 2026 shows no discretization steps are required.
#80Sander Dieleman@SEDIELEMDid I mention continuous DLMs are back? I think I might have mentioned it before🤔
This one revisits Plaid (https://arxiv.org/abs/2305.18619), a continuous DLM trained with likelihood loss, and rigorously shows how it holds up against a recent discrete method. Pretty well, looks like!
Zhihan Yang@zhihanyang_📢Excited to share our new paper: Continuous Diffusion Scales Competitively with Discrete Diffusion for Language We introduce RePlaid 🌊, a continuous diffusion language model (DLM) with 🏅Discrete likelihood bound 🏅Scaling laws competitive with SOTA discrete DLMs How? Dive in👇[🧵1/12] Paper: https://arxiv.org/abs/2605.18530 Work done with my amazing collaborators: @WeiGuo01 @ShuibaiZ69721 @ssahoo_ @YongxinChen1 @ArashVahdat @MardaniMorteza @jwthickstun
abs: https://arxiv.org/abs/2605.18530
Tanishq Mathew Abraham, Ph.D.@iScienceLuvrContinuous Diffusion Scales Competitively with Discrete Diffusion for Language "we establish the first scaling law for continuous DLMs that rivals discrete DLMs: RePlaid exhibits a compute gap of only 20× compared to autoregressive models, outperforms Duo while using fewer parameters, and outperforms MDLM in the over-trained regime."
#359Tanishq Mathew Abraham, Ph.D.@ISCIENCELUVRContinuous Diffusion Scales Competitively with Discrete Diffusion for Language
"we establish the first scaling law for continuous DLMs that rivals discrete DLMs: RePlaid exhibits a compute gap of only 20× compared to autoregressive models, outperforms Duo while using fewer parameters, and outperforms MDLM in the over-trained regime."
#1822Luca Ambrogioni@LUCAAMBJet another chapter of the 'continuous language diffusion' workd story.
I'd say we should stop finding it surprising at this point
(spoiler, there was never a good reason to believe that continuous diffusion doesn't work, just grop think)
Zhihan Yang@zhihanyang_📢Excited to share our new paper: Continuous Diffusion Scales Competitively with Discrete Diffusion for Language We introduce RePlaid 🌊, a continuous diffusion language model (DLM) with 🏅Discrete likelihood bound 🏅Scaling laws competitive with SOTA discrete DLMs How? Dive in👇[🧵1/12] Paper: https://arxiv.org/abs/2605.18530 Work done with my amazing collaborators: @WeiGuo01 @ShuibaiZ69721 @ssahoo_ @YongxinChen1 @ArashVahdat @MardaniMorteza @jwthickstun