NVIDIA Proposes Nemotron-TwoTower for Faster Diffusion Language Modeling

VIEWS894BOOKMARKS4LIKES5

Tanishq Mathew Abraham, Ph.D.@iScienceLuvr

model: https://huggingface.co/collections/nvidia/nemotron-twotower abs: https://arxiv.org/abs/2606.26493

Tanishq Mathew Abraham, Ph.D.@iScienceLuvr

Nemotron-TwoTower: Diffusion Language Modeling with Pretrained Autoregressive Context

"We propose TwoTower, a block-wise autoregressive diffusion model that decouples these roles into two towers: a frozen AR context tower that causally processes clean tokens, and a trainable diffusion denoiser tower with bidirectional block attention that refines noisy blocks via cross-attention to the context."

"Nemotron-TwoTower retains 98.7% of the autoregressive baseline's quality while offering 2.42X higher wall-clock generation throughput."

14h89454