@MihaelaVDS: Can LLMs keep learning new skills without updating their weights? Modern LLMs can already master & combine many skills.…
Summary
Introduces 'skill neologisms', a method for enabling LLMs to learn new skills without weight updates, addressing catastrophic forgetting. Presented at ICML.
View Cached Full Text
Cached at: 06/29/26, 10:32 PM
Can LLMs keep learning new skills without updating their weights? Modern LLMs can already master & combine many skills. But teaching them new skills in a scalable way without catastrophic forgetting remains an open challenge @icmlconf we introduce a new approach: skill neologisms https://t.co/xtHizOPqPV
Similar Articles
Learning, Fast and Slow: Towards LLMs That Adapt Continually
A fast-slow learning framework for LLMs combines fixed slow weights with optimized fast context weights, achieving up to 3x better sample efficiency and reduced catastrophic forgetting in continual learning scenarios.
Personal continual learning for LLMs without GPU — position paper [OC]
The author proposes two architectures, Internal KV-Sphere Architecture (IKSA) and Background Micro Fine-Tuning (BMFT), for enabling LLMs to learn continually from personal interactions without GPU requirements and without catastrophic forgetting.
Skill is Not One-Size-Fits-All: Model-Aware Skill Alignment for LLM Agents
This paper proposes MASA, a framework that adapts skills to each LLM backbone without modifying weights, using hierarchical evolution and a model-conditioned rewriter, achieving gains of up to 25.8 points over baselines.
Learning, Fast and Slow: Towards LLMs That Adapt Continually [R]
This paper introduces a Fast-Slow Training framework for LLMs that combines parameter updates with optimized context to improve sample efficiency and reduce catastrophic forgetting during continual learning.
LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
LatentSkill converts textual skills into LoRA adapters stored in weight space, reducing context overhead while maintaining modularity and composability for LLM agents, achieving significant improvements on ALFWorld and Search-QA benchmarks.