Tag
This paper introduces In-Place Test-Time Training, a framework that updates MLP weights in real-time during inference, allowing LLMs to dynamically adapt and handle long contexts up to 128k tokens.
This paper proposes Golden-GRPO Injection (GRIN), a mixed-policy reinforcement learning framework for continual knowledge injection in large language models, overcoming limitations of supervised fine-tuning. Experiments demonstrate superior performance on knowledge absorption benchmarks.
This paper analyzes recurrent fast-weight memories and selective state-space models as online learning rules, deriving normalized update families that improve length extrapolation and remain competitive in language modeling.
The report introduces Thomson, a new general-purpose AI model trained through continual learning for sovereign AI, demonstrating competitive performance with efficient compute and addressing forgetting issues.
A research paper titled 'ContinualSkillBench' finds that LLM agents benefit more from carrying forward context and feedback than from maintaining explicit skill libraries, with sequential execution showing a 16.9% relative gain but in-context learning often outperforming skill maintenance.
The article compares discrete and continuous memory adapters for frozen language models, finding that binary-mask methods like EPMem forget facts as quickly as continuous ones, emphasizing that the write/allocation rule, not discreteness, is crucial for preventing forgetting.
Google DeepMind highlights its 15-year history of advancing AI through games and announces a research partnership with Fenris Creations to address open challenges like continual learning and multi-agent dynamics.
This paper proposes FedCurv-DR, a lightweight federated continual learning method that reduces forgetting and balances performance, fairness, and energy efficiency for sustainable AI in cultural heritage applications.
This paper introduces a statistically defined decision layer for continual-learning systems, allowing expert pools to decide whether to reuse existing models, spawn new ones, or defer based on accumulated evidence, with theoretical guarantees and system contributions for managing nonstationary data streams.
This paper proposes 4MAS, a novel neural architecture inspired by biological bilaterality and memory consolidation, to address catastrophic forgetting in lifelong learning, achieving competitive results on benchmark datasets.
This paper proposes a frequency-aware continual learning framework using large language models for smart contract vulnerability detection, addressing adaptation, forgetting, and consolidation challenges with efficient methods like FA-LoRA and APPM.
The paper introduces Continual Reasoning Gym to diagnose and improve continual reinforcement learning with verifiable rewards (RLVR), proposing Continual Prompt Replay (CPR) to harness shared reasoning structures and achieve multitask-level performance.
This paper identifies anisotropy-induced plasticity loss in neural networks and proposes SingularClip, a method that clips singular values to maintain plasticity in continual and reinforcement learning.
This paper proposes reversible forgetting for enterprise AI agents to manage obsolete knowledge, introducing a framework with memory states and a controller to suppress and reactivate knowledge based on relevance.
LangChain is hosting an event in San Francisco on September 2nd focused on continual learning, featuring technical talks and networking opportunities.
This paper introduces Chain-of-Experience (CoE), a framework for continual LLM improvement through iterative test-time interactions with self and environmental feedback, showing consistent gains and lower API costs across multiple domains and models.
The paper introduces Spaced Repetition Training (SRT), a framework for continual pre-training of language models that uses adaptive review scheduling inspired by cognitive science to mitigate catastrophic forgetting. It improves the stability-plasticity trade-off, enhancing retention of old knowledge while acquiring new information.
Thomson-1.0-Small is an open-weight foundation model from Thomson Reuters, developed using continual learning to improve performance in legal, tax, and journalism domains with a focus on value sovereignty and efficiency.
Introduces the p-Spin Glass Network, a novel architecture for sequence models that achieves memory efficiency, sample efficiency, and single-batch stability, enabling continual learning and edge AI applications.
SynGAP is a task-free continual learning framework that simulates biological metaplasticity via adaptive gradient preconditioning to mitigate catastrophic forgetting, demonstrating significant accuracy improvements over existing methods on benchmarks.