DEVOURED
The sleep framework: A new way for LLMs to retain knowledge in continual learning
Google researchers introduced a 'sleep' phase for LLMs, allowing them to consolidate short-term memories into stable, long-term parameters during offline intervals.
What: The framework alternates between active wake phases and offline sleep phases where models use Mixture-of-Experts (MoE) blocks and reinforcement learning to consolidate new facts. In tests, the model maintained performance on continual learning tasks and scaled long-context retrieval to 10M tokens without catastrophic forgetting.
Why it matters: This research addresses the fundamental challenge of static model knowledge, providing a roadmap for agents that can continuously adapt to new data without requiring a full retraining cycle.
Deep dive
- Introduces a sleep framework for continual learning with two phases: Active and Sleep.
- Sleep phase uses memory consolidation and dreaming (self-improvement) to update weights.
- Dynamically scales capacity during sleep using MoE and low-rank experts.
- Uses generalized knowledge distillation (GKD) to transfer fragile context to long-term memory.
- Mitigates catastrophic forgetting by protecting stable parameters.
- Dreaming phase uses synthetic data generation filtered by gradient-based selection.
- Outperforms ICL and standard SFT on long-context benchmarks up to 10M tokens.
- Demonstrates effective mathematical reasoning gains on AIME 24/25 benchmarks.
Decoder
- Catastrophic forgetting: A phenomenon in neural networks where learning new information causes the model to rapidly overwrite or lose previously learned knowledge.
- Mixture-of-Experts (MoE): A model architecture where only a subset of network parameters (experts) is activated for any given input, increasing model capacity without proportionally increasing compute cost.
- In-context learning (ICL): The ability of a model to learn from inputs provided within the prompt without updating its underlying static weights.
Original article
Full article content is not available for inline reading.
Explore all 54 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi