Two attempts to move beyond teacher forcing have hit a mathematical trap, according to a new paper by Mathieu Blondel and colleagues: the proposed training losses are unbounded below. Their values can fall without a finite floor. That is a rather important thing for a minimization objective to be missing. Source
The arXiv preprint, On the Difficulty of Sequence-Level Supervised Learning for Autoregressive Language Models, was submitted on November 12 and revised on November 16, 2026. Blondel outlined the motivation and findings in a November 17 thread. Source Source Source
Why look beyond teacher forcing?
As the abstract explains, autoregressive language models typically use teacher forcing during supervised training, minimizing a sum of losses for individual token predictions. The method is simple and efficient, but it may suffer from exposure bias: a mismatch between training and generation. Blondel says the model isn't trained to use its own predictions when generating text. Source Source
The researchers asked whether sequence-level losses could incorporate the procedures used to generate text. They examined two natural constructions: one incorporating greedy decoding, the other autoregressive sampling. Both, they report, are unbounded below. Source
“Optimizing them would make the model diverge,” Blondel wrote. Source
The alternative update brings another catch
The abstract traces the problem to the way the generated sequence depends on the model's parameters. The authors also analyze a stop-gradient approach that blocks gradients through sampling, producing what they call a moment-matching update direction. Source
That direction isn't the gradient of any loss function, the abstract says. Yet the authors report proving that, in function space, it points toward lower autoregressive negative log-likelihood—the usual training objective. That theoretical result doesn't guarantee practical convergence or better model quality. Source
The catch, the authors say, is high variance: a noisy update direction. Source
The authors conclude that teacher forcing may well be “all you need” for supervised learning of autoregressive models. The findings leave other sequence-level alternatives open and don't establish that exposure bias is solved. No retirement party for teacher forcing on this evidence. Source
Comments