Google Research's Ali Behrouz and Vahab Mirrokni outlined a proposed "sleep" cycle for language models in an Oct. 27, 2026, blog post. It aims to help models absorb new information while mitigating catastrophic forgetting, a problem in which learning new tasks degrades performance on earlier ones. The post is not the debut of a new October paper: the underlying work, by Behrouz, Farnoosh Hashemi, Adel Javanmard and Mirrokni, was submitted to arXiv in June and revised in July. Its listing says a version had been public on OpenReview since September 2025.
In the proposed framework, a model processes inputs during an active phase, then pauses new inputs for offline work. First comes memory consolidation, which transfers short-term information into expanded long-term model parameters. Next comes "dreaming": the model generates and filters synthetic practice examples, trains on selected examples and evaluates the results. The researchers stress that the sleep vocabulary is a metaphor. Their framework operates within a fixed model architecture and human-defined training rules.
Behrouz and Mirrokni report comparisons across continual translation, long-context, factual-knowledge and math tasks. They describe near-perfect scaling up to 10 million tokens on the BABILong long-context benchmark and report scores of 48.9% and 46.2% on two factual-knowledge evaluations. Those are the research team's reported results, not independently verified performance. The likewise presents the value of the sleep stage as the authors' conclusion from their experiments.
Comments