Tag
The paper introduces Spaced Repetition Training (SRT), a framework for continual pre-training of language models that uses adaptive review scheduling inspired by cognitive science to mitigate catastrophic forgetting. It improves the stability-plasticity trade-off, enhancing retention of old knowledge while acquiring new information.