Researcher Clarifies Knobs That Control Memorization Modes In Sequence Models
Updated our paper on the foundations of memory in sequence models (with fresh insights, clearer writing and ablations).
Our paper contrasts two distinct ways in which language models memorize and formulates the questions that arise from this.
Will be presented at #ICML.
This was satisfying to know because it explains concurrent findings that "identity" statements ("John is John") helped improve reasoning. (e.g., see this work: https://arxiv.org/abs/2509.24653)
Vaishnavh Nagarajan@_vaishnavhOne update is about a curious behavior where Transformer representations sometimes show a "zigzagging" geometry. We now understand that these are highly negative eigen-directions, and how they disappear when we add "A --> A" type of self-edges.
#1305Vaishnavh Nagarajan@_VAISHNAVHOld thread is here:
typo in thread. I meant to say: "It seems like there are various knobs that make the model memorize *one way or the other*"
Vaishnavh Nagarajan@_vaishnavhOld thread is here: