Google researchers propose a 'sleep' cycle to help language models retain knowledge

Google Research's Ali Behrouz and Vahab Mirrokni outlined a proposed "sleep" cycle for language models in an Oct. 27, 2026, blog post. It aims to help models absorb new information while mitigating catastrophic forgetting, a problem in which learning new tasks degrades performance on earlier ones. The post is not the debut of a new October paper: the underlying work, by Behrouz, Farnoosh Hashemi, Adel Javanmard and Mirrokni, was submitted to arXiv in June and revised in July. Its listing says a version had been public on OpenReview since September 2025.

In the proposed framework, a model processes inputs during an active phase, then pauses new inputs for offline work. First comes memory consolidation, which transfers short-term information into expanded long-term model parameters. Next comes "dreaming": the model generates and filters synthetic practice examples, trains on selected examples and evaluates the results. The researchers stress that the sleep vocabulary is a metaphor. Their framework operates within a fixed model architecture and human-defined training rules.