@MihaelaVDS: Can LLMs keep learning new skills without updating their weights? Modern LLMs can already master & combine many skills.…
Summary
Introduces 'skill neologisms', a method for enabling LLMs to learn new skills without weight updates, addressing catastrophic forgetting. Presented at ICML.
View Cached Full Text
Cached at: 06/29/26, 10:32 PM
Can LLMs keep learning new skills without updating their weights? Modern LLMs can already master & combine many skills. But teaching them new skills in a scalable way without catastrophic forgetting remains an open challenge @icmlconf we introduce a new approach: skill neologisms https://t.co/xtHizOPqPV
Similar Articles
Learning, Fast and Slow: Towards LLMs That Adapt Continually
A fast-slow learning framework for LLMs combines fixed slow weights with optimized fast context weights, achieving up to 3x better sample efficiency and reduced catastrophic forgetting in continual learning scenarios.
@MSFTResearch: LLMs do not get smarter just by remembering more. EvoLib turns experience into evolving knowledge, taking reusable skil…
Microsoft Research introduces EvoLib, a framework that enables LLMs to continually learn from their own experience during inference by extracting reusable skills and insights, without model updates or external labels.
@omarsar0: New research from Google DeepMind. (bookmark it) SkillSmith treats model weights as an additional modality the LLM read…
Google DeepMind introduces SkillSmith, which treats model weights as an additional modality that LLMs can natively reason over, enabling instruction-steered parametric synthesis for composing skills at inference time. The approach outperforms text-only and weight-only baselines.
Can LLMs Truly Forget? Revealing Unlearning Gaps Through Adversarial Evaluation
The study reveals substantial gaps in machine unlearning for LLMs, showing that adversarial evaluation uncovers recoverability of forgotten information despite strong standard metrics, highlighting the need for adversarial stress-testing.
@oneill_c: https://x.com/oneill_c/status/2077453217609453784
A researcher discusses the challenge of continual learning in LLMs, comparing them to amnesiac interns, and explores approaches like extending context windows, building stateful memory, and compressing context into latent representations, citing their work on Still.