knowledge-consolidation

Google researchers propose a 'Sleep' paradigm for continual learning that consolidates short-term in-context knowledge into long-term model parameters via distillation and replay. A 'Dreaming' stage uses reinforcement learning to generate synthetic curricula for self-improvement.