“Free pause tokens” reportedly add AI compute without longer context

DAIR.AI describes “free pause tokens” as extra computation carried in a parallel prediction stream that shares the model’s weights, rather than taking up additional sequence positions. It says the approach leaves context length and the KV cache unchanged, with essentially no extra latency during inference. DAIR.AI puts training overhead at about 1.14× an optimized pretraining pipeline while retaining most of the benefit. A quote-post questions the idea’s novelty, saying people “keep reinventing ideas from XLNet.”