The model previously required up to 32 sampling steps.
🧚Progressive Distillation of ELF🧚
We pushed ELF, a continuous diffusion language model, even further toward few-step generation ⚡️
ELF already generates 1,024-token sequences in just 8–32 steps without distillation.
Now, with progressive distillation, ELF+PD scales all the way down to few-step and even one-step generation!
Blog: https://linlu-qiu.github.io/assets/html/elf_pd.html 🧵
Users are praising the ELF Diffusion Model's progressive distillation for enabling one-step generation, calling the work cool, beautiful, and very interesting while thanking the authors for sharing.
No Digg Deeper questions have been answered for this story yet.
🎉Joint work with my amazing collaborators🎉 @linluqiu, @Lyy_iiis, @Hope7Happiness, @TianhongLi6, @yoonrkim, @jacobandreas, and Kaiming He!
We also updated our paper (see Appendix B) and code accordingly.
Paper: https://arxiv.org/abs/2605.10938 Code: https://github.com/lillian039/ELF Learn more about ELF:
On unconditional OpenWebText generation, ELF+PD achieves the lowest generative perplexity among distilled discrete and continuous diffusion LM baselines, enabling strong few-step and even one-step generation.
We improve ELF’s few-step generation with progressive distillation! ⚡️
🧚Progressive Distillation of ELF🧚
We pushed ELF, a continuous diffusion language model, even further toward few-step generation ⚡️
ELF already generates 1,024-token sequences in just 8–32 steps without distillation.
Now, with progressive distillation, ELF+PD scales all the way down to few-step and even one-step generation!
Blog: https://linlu-qiu.github.io/assets/html/elf_pd.html 🧵
How does ELF+PD work? The core idea is simple: we train the student to compress multiple small teacher substeps into a single larger step. We progressively distill the model over five rounds, following a 16 → 8 → 4 → 2 → 1 step curriculum.
Takeaway:
Continuous diffusion language models can be competitive with their discrete counterparts at low sampling budgets. With progressive distillation, ELF pushes fast, few-step generation performance even further.
Thanks!
We observed that even a single student step could already produce a rough sentence-level draft, although the details were often inaccurate and the perplexity remained high.
Adding a second step improved the output substantially, and from four steps onward, the generated sentences were already of quite good quality.
@HuLillian39250 Very interesting!
How did you have the insight that a single student can indeed do this in one step?
@HuLillian39250 Great work! Thanks for sharing. Do you also evaluate on conditional generation tasks?
@HuLillian39250 super cool work!
@HuLillian39250 beautiful!! thx for publish
@HuLillian39250 Very interesting. Thanks
@_dhruveshp Thanks! We haven’t explored conditional generation tasks yet, but the same methodology should apply with minimal modifications.
