Jeff Dean praised a paper on November 21, 2026, that tackles an oddly specific AI mystery: why calendar months form circles in model representations. He said Martin Casado’s post had led him to the research. SourceSource
Dean wrote that “the analysis is great and the figures really make the paper shine!” He also said he hadn’t seen the paper when it was originally posted in February. Even research about time can take a while to find its reader. Source
The paper, Symmetry in language statistics shapes the geometry of model representations, is by Dhruva Karkada, Daniel J. Korchinski, Andres Nava, Matthieu Wyart and Yasaman Bahri. Its central argument is that particular shapes learned by models emerge from statistical structure already present in language. Source
Why the months go round
Word embeddings represent words as numerical vectors, with relationships between those vectors capturing aspects of meaning. Plot related representations in a suitable set of dimensions, and recognizable shapes can appear. The paper examines circular arrangements for calendar months, open curved structures for historical years and spatial organization for geographical concepts. Think plotted numerical relationships, not a tiny wall calendar in a server rack. Source
The authors connect those shapes to translation symmetry. Roughly, the relevant statistical relationship between two month names depends on their separation around the calendar, rather than their absolute positions. The researchers use this symmetry as a mathematical assumption and report approximate agreement with measurements from real text. They aren’t claiming every sentence obeys a perfectly tidy calendar rule. Source
Under the paper’s assumptions, the resulting embeddings have Fourier structure: numerical components that rise and fall like waves as you move along the underlying concept. Lower-frequency components produce circles and open loops; higher-frequency components add ripples. The authors compare their predictions with Wikipedia-derived word embeddings and internal representations in Gemma 2 2B, among other tests. Source
In one experiment, the researchers removed the direct month-to-month signal from a table of word-pair associations used to construct embeddings. The table, called a normalized co-occurrence matrix, compares how often words appear together with how often they would if they occurred independently. With 1,000 numbers per word vector in a 25,000-word vocabulary, the circular geometry and correct month ordering still emerged. The authors explain this through seasonal information distributed across other words: direct associations between month names aren’t the only available clues. The calendar has backup. Source
For that visualization, the researchers left “May” out when calculating the plot’s axes, because its non-calendar meanings distort its fixed word vector. Even a neat circle has to negotiate with English. Source
Where the explanation stops
The authors prove their results in word embedding models and report empirical agreement in larger models. Those experiments do not extend the proofs to everything an LLM represents or computes. Source
In the paper’s Gemma example, adding context that identifies “May” as a month helps resolve its ambiguity and makes the month representations increasingly circular through the model’s layers. The authors identify explaining that contextual adjustment as further work. Source
Comments