Paper Explores Free Pause Tokens for Models

DAIR.AI posted about a paper on free pause tokens by John Langford and colleagues at Microsoft Research, Cornell, and CMU. The paper describes how a free pause token supplies a language model with extra compute for each next-token prediction. It achieves this by running the model in a parallel prediction stream over a sequence position rather than expanding context length. The post links to an arXiv abstract and a dair.ai page summarizing the work.