Randomized YaRN Improves Length Generalization for Long-Context Reasoning

Manas Mehta, a CS MS student at NYU Courant, posted that YaRN falls short for LLM reasoning at 128K context. He introduces Randomized YaRN, which trains on short-context data by sampling positions from a longer length range. Mehta claims this raises OOD reasoning accuracy, especially at 128K context. The work is scheduled for presentation at EMNLP26 Findings. An attached slide displays the title Randomized YaRN Improves Length Generalization for Long-Context Reasoning.