Google Research Identifies Computational Buffer And Factual Priming In LLM Reasoning

Sounds like Mechanism 1 is inferior to looped models, and mechanism 2 could be replaced by some cheap and fast n-gram retrieval. We could maybe train models to cough up a bunch of tags in parallel, then loop, reason for 100-200 tokens, and give perfect answers.

@teortaxesTex 这就相当于给模型做热身运动

@teortaxesTex I won't pretend I understand much here. But I feel compelled to say that GPT 5.5 and Opus 6.8 to me still look like statistical parrots on moderately complex job. I don't know what setup/context Witten or Tao were given but these models are nowhere near AGI.

@teortaxesTex it's interesting, how when the model generates a list in its CoT i.e factual timelines, its learning to exploit its own NTP mechanism in response to a lack of recall lookup