Tag
This paper introduces a 'Sleep' paradigm for large language models that enables continual learning through memory consolidation and dreaming phases, allowing models to distill short-term knowledge into long-term parameters and self-improve without human supervision.
This paper introduces a sleep-like consolidation mechanism for Transformer-based LLMs that periodically converts recent context into persistent fast weights in SSM blocks, clearing the KV cache to improve long-horizon reasoning without increasing inference latency.
Auto-Dreamer introduces a learned offline memory consolidation method for language agents, decoupling fast memory acquisition from slow cross-session consolidation, and achieving higher performance with smaller memory banks, generalizing to unseen environments.