Stanford Prefix Sliding Paper Speeds Reasoning Inference

Rohan Paul tweeted about a new Stanford paper titled Prefix Sliding. According to the post, long reasoning does not require the full chain of thought in memory. Retaining only the task prefix and recent tokens allows inference to run about three times faster without any retraining. The method targets the expense of full attention over growing token sequences. The tweet includes an attached screenshot of the arXiv paper itself.