From Weights to Features: SAE-Guided Activation Regularization for LLM Continual Learning

# From Weights to Features: SAE-Guided Activation Regularization for LLM Continual Learning
Source: [https://arxiv.org/html/2606.26629](https://arxiv.org/html/2606.26629)
Evan Ning1Wei Xue1Dong Lou1Yike Guo1 1The Hong Kong University of Science and Technology

###### Abstract

Weight\-space regularization methods such as Elastic Weight Consolidation \(EWC\) are the standard approach to catastrophic forgetting in continual learning\. However, those methods tend to underperform when applied to large language models\. We argue that such underperformance can be partly explained by the “polysemantic” nature of large language models: per\-weight importance estimates utilized by EWC\-style regularization are too coarse and cannot isolate the knowledge that needs protection\. In this paper, we propose regularizing instead in the model’s activation space, using pretrained Sparse Autoencoders \(SAEs\) as a monosemantic feature dictionary\. From the perspective of constrained optimization, we derive a new loss function that uses the SAE feature dictionary to explicitly balance stability and plasticity, and show that EWC is a special case in the one\-sided weight\-space penalty setting\. Unlike replay\-based methods that store or revisit examples from earlier tasks, our method requires no previous\-task data after mask construction: current\-task data is used to compute a compact SAE feature mask, and only this mask is retained for later training\. Further, since the feature space has significantly lower dimensionality than the parameter space, the proposed method is more memory efficient\. On the TRACE and MedCL continual learning benchmarks, the method achieves the strongest result among approaches without introducing task\-specific architectural components, also surpassing traditional weight\-space regularization methods like EWC\. Beyond performance comparisons, we provide empirical evidence for the polysemanticity thesis: task\-relevant representations are linearly separable in the SAE feature basis but indistinguishable from chance in the weight basis, and weight\-space protection is nearly non\-selective at the concept level\.

## 1Introduction

Large language models \(LLMs\) increasingly require continual adaptation to new domains, tasks, and interaction formats after deployment\. However, sequential fine\-tuning can overwrite representations needed for previously learned capabilities, leading to catastrophic forgetting\(McCloskey and Cohen,[1989](https://arxiv.org/html/2606.26629#bib.bib3), Ratcliff,[1990](https://arxiv.org/html/2606.26629#bib.bib4)\)\. Continual learning methods address this problem through replay, architectural expansion or isolation, and regularization\. Among these, regularization is particularly appealing for LLMs because it does not require storing previous\-task data, allocating task\-specific parameters, or introducing inference\-time routing\. Most regularization\-based continual learning methods protect knowledge at the parameter level\. Elastic Weight Consolidation\(EWC; Kirkpatricket al\.,[2017](https://arxiv.org/html/2606.26629#bib.bib5)\), Synaptic Intelligence\(SI; Zenkeet al\.,[2017](https://arxiv.org/html/2606.26629#bib.bib6)\), and Memory Aware Synapses\(MAS; Aljundiet al\.,[2018](https://arxiv.org/html/2606.26629#bib.bib7)\)assign importance scores to individual weights and penalize subsequent changes\. This strategy is effective when parameters provide a reasonably selective unit of protection, but it degrades in LLMs\. On TRACE, LoRA\-based EWC yields only limited improvement over unprotected sequential fine\-tuning\(Wanget al\.,[2023b](https://arxiv.org/html/2606.26629#bib.bib17)\); Our experiments also show that EWC achieves low forgetting primarily by overly reducing plasticity rather than from genuinely protecting acquired knowledge, resulting in weak overall performance\.

We argue that this failure is structural rather than merely a consequence of imperfect hyperparameter tuning\. Mechanistic interpretability offers a plausible explanation through the superposition hypothesis\(Elhageet al\.,[2022](https://arxiv.org/html/2606.26629#bib.bib23)\)\. Neural networks can encode more features than their dimensionality would normally allow by representing different features in overlapping directions\. This gives rise to polysemanticity: individual neurons, and consequently the weights connected to them, can participate in multiple unrelated concepts\(Olahet al\.,[2020](https://arxiv.org/html/2606.26629#bib.bib24), Billset al\.,[2023](https://arxiv.org/html/2606.26629#bib.bib31)\)\. For weight\-space regularization, this creates a mismatch between the unit of importance estimation and the unit of knowledge preservation\. Therefore, the diagonal Fisher used by EWC cannot distinguish whether a weight is important for concept A, concept B, or both\. Protecting that weight to preserve concept A therefore also constrains concept B, reducing the model’s capacity to learn new tasks\. We identify this non\-selective protection as a key reason why weight\-space regularization fails to scale to LLMs\.

In this paper, we propose to mitigate the problem by adopting Sparse Autoencoders \(SAEs\) as a selective coordinate system for continual learning\. Trained to reconstruct model activations through sparse overcomplete representations, SAEs decompose dense activations into features that are often more monosemantic than individual neurons\(Hubenet al\.,[2024](https://arxiv.org/html/2606.26629#bib.bib25), Brickenet al\.,[2023](https://arxiv.org/html/2606.26629#bib.bib26)\)\. A pretrained SAE is used as a fixed feature dictionary\. For each task, we estimate which SAE features are activated by task data and construct a task\-specific relevance mask, with high\-relevance features defining the adaptive region, and low\-relevance features defining the protected region\. This changes the regularization target from entangled parameters to semantically more selective activation features\.

We further derive the training objective from a constrained optimization formulation rather than conventionally designing the regularizer in the ad\-hoc manner, which simply restricts the model from drifting too much from the original weights\. The model should minimize the current\-task loss in the SAE feature space satisfying a\) protected features should not drift beyond a stability budget, and b\) task\-relevant features should adapt sufficiently to avoid the degenerate solution of preserving everything but learning nothing, which are defined as stability and plasticity constraints, respectively\. Consequently, applying Lagrangian relaxation and a squared\-hinge penalty yields two corresponding losses, namely a protect loss and a guide loss\. Under this framework, EWC becomes a special case which measures drift in weight space and includes no plasticity constraint\.

The resulting method requires no previous\-task data after mask construction\. Current\-task examples are used to compute a task\-specific SAE feature mask; after this step, only the compact mask is retained, and no examples or activations from that task are stored for later replay\. Unlike anchor\-based feature distillation methods such as SAE\-FD\(Zhanget al\.,[2026](https://arxiv.org/html/2606.26629#bib.bib37)\), which preserve knowledge by storing previous\-task anchors or activations and matching them during later training, our method regularizes SAE feature drift using current\-task data and the frozen base model, requiring no replay buffer, no stored activations, no per\-task parameters, and no inference\-time routing\.

We evaluate SAE\-guided activation regularization on TRACE\-5000 and MedCL, covering both cross\-domain and within\-domain continual learning\. On TRACE, our method achieves the strongest performance among non\-architectural approaches, outperforming weight\-space regularizers, gradient\-projection methods, and replay\-based baselines under matched training conditions\. On MedCL, it achieves the best result among methods that retain no previous\-task examples\. Beyond aggregate performance, we provide mechanistic evidence for our central claim: task\-relevant representations are more separable in SAE feature space than in weight space, and protecting weights imposes broader collateral constraints than protecting SAE features\. These results show that the effectiveness of regularization depends not only on the penalty strength, but also on the representation space in which the constraint is imposed\.

Our contributions are summarized as follows:

- •A practical CL method using SAE features\.We introduce activation\-space regularization via pretrained SAEs\. After mask construction, our method stores only compact SAE feature masks and retains no previous\-task examples or anchor activations\.
- •Principled derivation from constrained optimization\.We formulate the stability\-plasticity tradeoff as a constrained optimization problem in SAE feature space, derive a squared\-hinge training loss via Lagrangian relaxation and the quadratic penalty method\.
- •Empirical evidence for the polysemanticity thesis\.We demonstrate that superposition undermines weight\-space protection through two experiments\. A separability test shows that task\-relevant representations are linearly separable in the SAE feature basis \(AUC 0\.88\) but indistinguishable from chance in the weight basis \(0\.50\)\. A collateral\-constraint test shows that protecting one task’s weights constrains the next task’s features at 91–96% the rate of its own, whereas feature\-space protection reduces this to 43–61%\.
- •Scalability Advantage\.Because feature space is far lower in dimensionality than parameter space, per\-task storage reduces from the gigabyte\-scale anchors that weight\-space methods require to a sub\-megabyte feature mask\.

## 2Related Work

### 2\.1Continual Learning and the Stability–Plasticity Tradeoff

Continual learning studies how models can acquire new tasks sequentially without losing previously learned capabilities\. A central challenge is the stability\-plasticity tradeoff: the model must remain stable enough to preserve past knowledge while retaining enough plasticity to learn new tasks\. Early studies showed that standard gradient\-based training can cause severe catastrophic forgetting when tasks are learned sequentially\(McCloskey and Cohen,[1989](https://arxiv.org/html/2606.26629#bib.bib3), Ratcliff,[1990](https://arxiv.org/html/2606.26629#bib.bib4)\)\. This problem becomes especially important for LLMs, which are often adapted after deployment to new domains, formats, and user requirements\.

Existing continual learning methods address this tradeoff through three main strategies\. Replay\-based methods store or regenerate previous\-task data and mix it into later training\(Lopez\-Paz and Ranzato,[2017](https://arxiv.org/html/2606.26629#bib.bib9), Chaudhryet al\.,[2019](https://arxiv.org/html/2606.26629#bib.bib10), Buzzegaet al\.,[2020](https://arxiv.org/html/2606.26629#bib.bib11)\)\. Architectural methods allocate task\-specific parameters, masks, or subspaces to reduce interference\(Rusuet al\.,[2016](https://arxiv.org/html/2606.26629#bib.bib14), Mallya and Lazebnik,[2018](https://arxiv.org/html/2606.26629#bib.bib15), Wanget al\.,[2023a](https://arxiv.org/html/2606.26629#bib.bib18)\)\. Regularization\-based methods keep the model architecture fixed and add penalties to discourage harmful changes to important parameters or functions\(Kirkpatricket al\.,[2017](https://arxiv.org/html/2606.26629#bib.bib5), Zenkeet al\.,[2017](https://arxiv.org/html/2606.26629#bib.bib6), Aljundiet al\.,[2018](https://arxiv.org/html/2606.26629#bib.bib7)\)\. Among these, regularization is particularly attractive for LLM adaptation because it avoids storing previous\-task data, adding task\-specific modules, or introducing inference\-time routing\. However, its effectiveness depends critically on what internal units are regularized and how the stability\-plasticity tradeoff is encoded\.

A key limitation of many continual learning methods is that they emphasize stability but do not explicitly enforce plasticity\. Excessive constraints can reduce forgetting while also preventing effective learning of new tasks, a failure mode often referred to as intransigence\(Chaudhryet al\.,[2019](https://arxiv.org/html/2606.26629#bib.bib10)\)\. This distinction is important when evaluating continual learning methods: low forgetting alone does not imply successful continual learning if the model fails to acquire new tasks\. Our work follows this view and treats stability and plasticity as separate requirements\. Rather than using a single always\-on penalty, we formulate both requirements explicitly and derive the training objective from the resulting constrained optimization problem\.

### 2\.2Regularization\-Based Continual Learning

Regularization\-based continual learning keeps the model architecture fixed and discourages updates that may damage previously learned knowledge\. Classical methods mainly operate in weight space\. Elastic Weight Consolidation \(EWC\) estimates parameter importance using the diagonal Fisher information matrix and penalizes changes to important weights\(Kirkpatricket al\.,[2017](https://arxiv.org/html/2606.26629#bib.bib5)\)\. Synaptic Intelligence \(SI\) accumulates parameter importance online from each parameter’s contribution to loss reduction\(Zenkeet al\.,[2017](https://arxiv.org/html/2606.26629#bib.bib6)\)\. Memory Aware Synapses \(MAS\) estimates importance from the sensitivity of the model output to parameter perturbations\(Aljundiet al\.,[2018](https://arxiv.org/html/2606.26629#bib.bib7)\)\. Although these methods differ in how importance is estimated, they share the same basic assumption: knowledge can be preserved by constraining important parameters\.Kaoet al\.\([2021](https://arxiv.org/html/2606.26629#bib.bib8)\)further showed that several of these importance estimates are closely related to Fisher information\.

This assumption becomes problematic for LLMs\. Recent LLM continual learning benchmarks show that weight\-space regularization often provides limited improvement over sequential fine\-tuning\(Wanget al\.,[2023b](https://arxiv.org/html/2606.26629#bib.bib17), Zenget al\.,[2026](https://arxiv.org/html/2606.26629#bib.bib33)\)\. In practice, these methods may reduce forgetting by over\-constraining the model, thereby reducing its ability to learn new tasks\. This reflects a limitation of one\-sided anti\-drift regularization: it penalizes parameter movement but does not explicitly model the plasticity required for new\-task learning\.

Other regularization methods operate at the functional or representational level\. Learning without Forgetting \(LwF\) preserves previous behavior by distilling outputs from an earlier model\(Li and Hoiem,[2018](https://arxiv.org/html/2606.26629#bib.bib34)\), while gradient\-projection methods such as ELLA constrain updates using subspaces associated with previous tasks\(Ruvolo and Eaton,[2013](https://arxiv.org/html/2606.26629#bib.bib35)\)\. These methods move beyond simple parameter anchoring, but they still do not address the central issue considered in this work: the unit at which knowledge should be protected\. Our method remains within the regularization family, but changes both the coordinate system and the objective\. Instead of assigning importance to parameters, we assign relevance to SAE features; instead of using a one\-sided penalty, we derive a two\-sided objective with explicit stability and plasticity constraints\.

### 2\.3Superposition, Polysemanticity, and Sparse Autoencoders

The superposition hypothesis provides a useful explanation for why weight\-space regularization becomes ineffective at scale\. Neural networks can represent more features than their dimensionality would normally allow by encoding features in overlapping directions\(Elhageet al\.,[2022](https://arxiv.org/html/2606.26629#bib.bib23)\)\. This gives rise to polysemanticity: individual neurons may respond to multiple unrelated concepts rather than a single interpretable feature\(Olahet al\.,[2020](https://arxiv.org/html/2606.26629#bib.bib24), Billset al\.,[2023](https://arxiv.org/html/2606.26629#bib.bib31)\)\. For continual learning, this creates a direct challenge\. A parameter\-level importance score cannot determine which concept a weight supports\. Protecting that weight to preserve one capability may also constrain other capabilities that share the same parameter, making weight\-space protection non\-selective\.

Sparse Autoencoders \(SAEs\) were introduced to recover more interpretable features from superposed representations\. By reconstructing model activations through a sparse overcomplete code, SAEs decompose dense activations into feature directions that are often more monosemantic than individual neurons\(Hubenet al\.,[2024](https://arxiv.org/html/2606.26629#bib.bib25), Brickenet al\.,[2023](https://arxiv.org/html/2606.26629#bib.bib26)\)\. Recent work has shown that SAE features can be identified and manipulated in large\-scale language models\(Templetonet al\.,[2024](https://arxiv.org/html/2606.26629#bib.bib27)\), and the Gemma Scope project provides pretrained SAE dictionaries across layers and widths for Gemma models\(Lieberumet al\.,[2024](https://arxiv.org/html/2606.26629#bib.bib28)\)\. These developments make SAE features a practical coordinate system for analyzing and controlling internal representations\.

Our work uses this coordinate system for continual learning regularization\. Rather than assigning importance to polysemantic parameters, we estimate task relevance over SAE features and construct a task\-specific mask from current\-task data\. The mask separates features that should remain stable from features that can adapt to the current task\. In contrast to prior uses of SAEs for interpretability, steering, or post\-hoc analysis, we use SAE features as the basis for training\-time regularization\. This enables concept\-level protection without storing previous\-task examples or allocating task\-specific parameters\.

## 3Method

We now describe how SAE features can be used not only to identify task\-relevant representations, but also to define the optimization constraints that govern continual learning\. SAE\-guided activation regularization formulates continual learning as constrained optimization over SAE feature drift\. For each new task, the objective is to minimize task loss while satisfying two constraints: stability, which limits drift on task\-irrelevant features, and plasticity, which preserves sufficient adaptation capacity for task\-relevant features\. The resulting training loss is obtained from a squared\-hinge relaxation of these constraints, rather than from a manually designed regularization penalty\.

The procedure consists of two stages\. In the offline mask\-construction stage, a frozen SAE encodes base\-model activations on current\-task data to produce a continuous feature relevance mask\. This mask partitions SAE features into adaptive and protected regions; after construction, only the mask is retained, not the examples used to compute it\. In the training stage, the current model and frozen base model are both mapped into SAE feature space; their feature drift is then weighted by the mask to form the protect and guide losses\. Figure[1](https://arxiv.org/html/2606.26629#S3.F1)gives an overview\. We next present the constrained formulation \(Section[3\.1](https://arxiv.org/html/2606.26629#S3.SS1)\), its SAE\-space instantiation \(Section[3\.2](https://arxiv.org/html/2606.26629#S3.SS2)\), and the mask construction procedure \(Section[3\.3](https://arxiv.org/html/2606.26629#S3.SS3)\)\.

![Refer to caption](https://arxiv.org/html/2606.26629v1/fig_overall.png)Figure 1:Overview of the SAE\-guided regularization pipeline\.Stage 1\(offline, per task\): current\-task data is passed through the frozen base model and frozen SAE to compute a relevance profile, which is expanded viakk\-NN propagation on decoder cosine similarity to produce a continuous mask𝐌t∈\[0,1\]F\\mathbf\{M\}\_\{t\}\\in\[0,1\]^\{F\}; the data used for this step is not retained\.Stage 2\(per optimizer step\): the training batch is passed through both the current model \(with LoRA\) and the frozen base; both sets of activations are encoded by the frozen SAE; the per\-feature driftΔ​f\\Delta fis gated by the mask into a protect loss on low\-𝐌t\\mathbf\{M\}\_\{t\}features \(stability\) and a guide loss on high\-𝐌t\\mathbf\{M\}\_\{t\}features \(plasticity\)\.### 3\.1From Constrained Optimization to the Training Loss

##### Constrained formulation\.

Consider a sequence of tasks\{𝒟t\}t=1T\\\{\\mathcal\{D\}\_\{t\}\\\}\_\{t=1\}^\{T\}, where𝒟t\\mathcal\{D\}\_\{t\}denotes the training distribution of tasktt\. Letθ\\thetabe the current model parameters and letθ0\\theta\_\{0\}denote the frozen pretrained base model\. For an inputx∼𝒟tx\\sim\\mathcal\{D\}\_\{t\}, letzi​\(x;θ\)z\_\{i\}\(x;\\theta\)denote theii\-th coordinate of an internal representation\. We define the representation drift as

Δi​\(θ;x\)=zi​\(x;θ\)−zi​\(x;θ0\)\.\\Delta\_\{i\}\(\\theta;x\)=z\_\{i\}\(x;\\theta\)\-z\_\{i\}\(x;\\theta\_\{0\}\)\.\(1\)The representationzzis left general in this subsection; Section[3\.2](https://arxiv.org/html/2606.26629#S3.SS2)instantiates it as SAE feature activations\.

At tasktt, the model should minimize the current\-task loss while controlling this drift\. We introduce two sets of non\-negative weights:wisw\_\{i\}^\{s\}for coordinates to be stabilized andwipw\_\{i\}^\{p\}for coordinates to remain adaptive\. For compactness, define the per\-example drift measures

cs​\(θ;x\)=∑iwis​Δi​\(θ;x\)2,cp​\(θ;x\)=∑iwip​\|Δi​\(θ;x\)\|\.c\_\{s\}\(\\theta;x\)=\\sum\_\{i\}w\_\{i\}^\{s\}\\Delta\_\{i\}\(\\theta;x\)^\{2\},\\qquad c\_\{p\}\(\\theta;x\)=\\sum\_\{i\}w\_\{i\}^\{p\}\|\\Delta\_\{i\}\(\\theta;x\)\|\.The stability and plasticity requirements are written as

minθ\\displaystyle\\min\_\{\\theta\}\\quadℒCE​\(θ;𝒟t\)\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\(\\theta;\\mathcal\{D\}\_\{t\}\)\(2\)s\.t\.𝒞s​\(θ\)=𝔼x∼𝒟t​\[cs​\(θ;x\)\]≤εs,\[stability\]\\displaystyle\\mathcal\{C\}\_\{s\}\(\\theta\)=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{t\}\}\\bigl\[c\_\{s\}\(\\theta;x\)\\bigr\]\\leq\\varepsilon\_\{s\},\\qquad\\text\{\[stability\]\}\(3\)𝒞p​\(θ\)=𝔼x∼𝒟t​\[cp​\(θ;x\)\]≥εp,\[plasticity\]\\displaystyle\\mathcal\{C\}\_\{p\}\(\\theta\)=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{t\}\}\\bigl\[c\_\{p\}\(\\theta;x\)\\bigr\]\\geq\\varepsilon\_\{p\},\\qquad\\text\{\[plasticity\]\}\(4\)whereεs\\varepsilon\_\{s\}is the allowed drift budget for protected coordinates, andεp\\varepsilon\_\{p\}is the minimum movement required for task\-relevant coordinates\.

The two constraints play complementary roles\. Stability prevents excessive drift in coordinates that should be preserved\. Plasticity ensures that the regularizer does not collapse into a purely conservative penalty\. Importantly, the plasticity constraint does not determine the direction of adaptation; that direction is still governed by the task loss\. It only requires sufficient movement in coordinates identified as relevant to the current task\.

##### Lagrangian formulation\.

To connect the constrained problem to a trainable objective, we form the Lagrangian\. Since the stability constraint is an upper\-bound constraint and the plasticity constraint is a lower\-bound constraint, their violations have opposite signs:

𝒥​\(θ,α,β\)=ℒCE​\(θ\)\+β​\(𝒞s​\(θ\)−εs\)−α​\(𝒞p​\(θ\)−εp\),\\mathcal\{J\}\(\\theta,\\alpha,\\beta\)=\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\(\\theta\)\+\\beta\\bigl\(\\mathcal\{C\}\_\{s\}\(\\theta\)\-\\varepsilon\_\{s\}\\bigr\)\-\\alpha\\bigl\(\\mathcal\{C\}\_\{p\}\(\\theta\)\-\\varepsilon\_\{p\}\\bigr\),\(5\)whereβ≥0\\beta\\geq 0andα≥0\\alpha\\geq 0are the multipliers for the stability and plasticity constraints, respectively\. The term𝒞s​\(θ\)−εs\\mathcal\{C\}\_\{s\}\(\\theta\)\-\\varepsilon\_\{s\}is positive when protected\-coordinate drift exceeds its budget\. The termεp−𝒞p​\(θ\)\\varepsilon\_\{p\}\-\\mathcal\{C\}\_\{p\}\(\\theta\)is positive when task\-relevant movement is insufficient\.

Direct primal–dual optimization would require updatingα\\alphaandβ\\betaduring LLM fine\-tuning and maintaining feasibility throughout training\. This is unnecessary for our purpose: we only need a practical training loss that penalizes violations of the desired feasible region\. We therefore use a stochastic squared\-hinge relaxation on per\-example constraint violations\.

##### Squared\-hinge relaxation\.

The resulting unconstrained objective is

ℒh\(θ\)=ℒCE\(θ\)\+𝔼x∼𝒟t\[βmax\(0,cs\(θ;x\)−εs\)2\+αmax\(0,εp−cp\(θ;x\)\)2\]\.\\mathcal\{L\}\_\{h\}\(\\theta\)=\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\(\\theta\)\+\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{t\}\}\\left\[\\beta\\max\(0,c\_\{s\}\(\\theta;x\)\-\\varepsilon\_\{s\}\)^\{2\}\+\\alpha\\max\(0,\\varepsilon\_\{p\}\-c\_\{p\}\(\\theta;x\)\)^\{2\}\\right\]\.\(6\)
This relaxation has the desired inactive\-when\-satisfied behavior\. The stability penalty is zero when protected\-coordinate drift remains within its budget and grows quadratically once the budget is exceeded\. The plasticity penalty is zero when task\-relevant movement reaches its threshold and penalizes insufficient adaptation otherwise\. Thus, the loss is not an always\-on anti\-drift regularizer\. It penalizes only violations of the stability and plasticity requirements\.

##### Relation to EWC\.

This formulation also clarifies the relation to EWC\. EWC can be written as

ℒEWC=ℒCE\+λ​∑iℱi​\(θi−θi∗\)2,\\mathcal\{L\}\_\{\\mathrm\{EWC\}\}=\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\+\\lambda\\sum\_\{i\}\\mathcal\{F\}\_\{i\}\(\\theta\_\{i\}\-\\theta\_\{i\}^\{\*\}\)^\{2\},\(7\)whereℱi\\mathcal\{F\}\_\{i\}denotes the diagonal Fisher importance\. Compared with the constrained objective in \([6](https://arxiv.org/html/2606.26629#S3.E6)\), EWC is a one\-sided weight\-space regularizer: it uses Fisher\-weighted parameter drift, an always\-on quadratic penalty, and no plasticity constraint\. Our framework generalizes this design by allowing the representation space, importance weights, and stability/plasticity constraints to be specified independently\. In the next section, we instantiate the template \([6](https://arxiv.org/html/2606.26629#S3.E6)\) in the SAE feature space\.

### 3\.2Instantiating the Objective in SAE Feature Space

##### SAE feature coordinates\.

The constrained objective in Section[3\.1](https://arxiv.org/html/2606.26629#S3.SS1)requires a representation space in which stability and plasticity can be assigned selectively\. Weight space is poorly suited for this purpose because individual parameters are entangled across many concepts\. SAE feature space provides a more appropriate coordinate system: it decomposes dense activations into sparse feature directions, allowing the regularizer to distinguish features that should be preserved from those that should adapt\.

We instantiate the generic representation coordinateziz\_\{i\}using SAE features\. Let𝐡L​\(x,τ;θ\)∈ℝD\\mathbf\{h\}\_\{L\}\(x,\\tau;\\theta\)\\in\\mathbb\{R\}^\{D\}denote the activation of the model at layerLLfor token positionτ\\tauin inputxx\. A frozen SAE maps this activation to an overcomplete sparse feature vector:

𝐟​\(x,τ;θ\)=SAEenc​\(𝐡L​\(x,τ;θ\)\)∈ℝF,\\mathbf\{f\}\(x,\\tau;\\theta\)=\\mathrm\{SAE\}\_\{\\mathrm\{enc\}\}\\bigl\(\\mathbf\{h\}\_\{L\}\(x,\\tau;\\theta\)\\bigr\)\\in\\mathbb\{R\}^\{F\},\(8\)whereF≫DF\\gg Dis the SAE dictionary size\. We set the representation coordinate in Section[3\.1](https://arxiv.org/html/2606.26629#S3.SS1)to the SAE feature activation, i\.e\.,zi​\(x,τ;θ\)=fi​\(x,τ;θ\)z\_\{i\}\(x,\\tau;\\theta\)=f\_\{i\}\(x,\\tau;\\theta\)\. The drift of featureiiis therefore

Δ​fi​\(θ;x,τ\)=fi​\(x,τ;θ\)−fi​\(x,τ;θ0\),\\Delta f\_\{i\}\(\\theta;x,\\tau\)=f\_\{i\}\(x,\\tau;\\theta\)\-f\_\{i\}\(x,\\tau;\\theta\_\{0\}\),\(9\)whereθ0\\theta\_\{0\}denotes the frozen reference model\.

##### Task\-specific feature masks\.

For each tasktt, we construct a continuous feature relevance mask𝐌t∈\[0,1\]F\\mathbf\{M\}\_\{t\}\\in\[0,1\]^\{F\}as described in Section[3\.3](https://arxiv.org/html/2606.26629#S3.SS3)\. A high valueMt,iM\_\{t,i\}indicates that featureiiis relevant to the current task and should remain adaptive\. A low value indicates that the feature is less relevant to the current task and should be protected from unnecessary drift\. We use the mask and its complement as the plasticity and stability weights, respectively, normalized by the SAE dictionary sizeFF:

wis=1−Mt,iF,wip=Mt,iF\.w\_\{i\}^\{s\}=\\frac\{1\-M\_\{t,i\}\}\{F\},\\qquad w\_\{i\}^\{p\}=\\frac\{M\_\{t,i\}\}\{F\}\.\(10\)This normalization makes the constraints measure average weighted drift per SAE feature rather than total drift over the dictionary\. As a result, the thresholdsεs\\varepsilon\_\{s\}andεp\\varepsilon\_\{p\}do not scale directly with the number of SAE features\. The mask therefore converts the abstract constraints in Section[3\.1](https://arxiv.org/html/2606.26629#S3.SS1)into feature\-level constraints: the complement of the mask defines the protected region, while the mask itself defines the adaptive region\.

##### SAE\-guided training loss\.

Using the feature drift in \([9](https://arxiv.org/html/2606.26629#S3.E9)\) and the mask weights in \([10](https://arxiv.org/html/2606.26629#S3.E10)\), the stability term measures the expected average squared drift over protected features:

ℒprotect=𝔼x∼𝒟t𝔼τ∈content​\(x\)\[max\(0,1F∑i\(1−Mt,i\)Δfi\(θ;x,τ\)2−εs\)2\]\.\\mathcal\{L\}\_\{\\mathrm\{protect\}\}=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{t\}\}\\mathbb\{E\}\_\{\\tau\\in\\mathrm\{content\}\(x\)\}\\left\[\\max\\\!\\left\(0,\\;\\frac\{1\}\{F\}\\sum\_\{i\}\(1\-M\_\{t,i\}\)\\,\\Delta f\_\{i\}\(\\theta;x,\\tau\)^\{2\}\-\\varepsilon\_\{s\}\\right\)^\{2\}\\right\]\.\(11\)This term is active only when the average drift of protected features exceeds the stability budgetεs\\varepsilon\_\{s\}\. In practice, the expectation is estimated on the current minibatch\.

For plasticity, an aggregate lower\-bound constraint can be satisfied by a small number of highly drifting features while leaving many task\-relevant features nearly unchanged\. We therefore use a coordinate\-wise hinge relaxation\. Each feature receives a target movement proportional to its relevance scoreMt,iM\_\{t,i\}:

ℒguide=𝔼x∼𝒟t𝔼τ∈content​\(x\)\[1F∑imax\(0,εpMt,i−\|Δfi\(θ;x,τ\)\|\)2\]\.\\mathcal\{L\}\_\{\\mathrm\{guide\}\}=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{t\}\}\\mathbb\{E\}\_\{\\tau\\in\\mathrm\{content\}\(x\)\}\\left\[\\frac\{1\}\{F\}\\sum\_\{i\}\\max\\\!\\left\(0,\\;\\varepsilon\_\{p\}M\_\{t,i\}\-\|\\Delta f\_\{i\}\(\\theta;x,\\tau\)\|\\right\)^\{2\}\\right\]\.\(12\)The factor1/F1/Fagain averages the penalty over the SAE dictionary\. This term is inactive once a feature has moved beyond its relevance\-weighted threshold\. It does not prescribe the direction of the movement; the task loss determines the update direction, while the guide term prevents task\-relevant features from remaining fixed\. Low\-mask features are largely unaffected because their thresholdsεp​Mt,i\\varepsilon\_\{p\}M\_\{t,i\}are small\.

The final training objective is

ℒtotal=ℒCE\+β​ℒprotect\+α​ℒguide\.\\mathcal\{L\}\_\{\\mathrm\{total\}\}=\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\+\\beta\\,\\mathcal\{L\}\_\{\\mathrm\{protect\}\}\+\\alpha\\,\\mathcal\{L\}\_\{\\mathrm\{guide\}\}\.\(13\)The SAE remains frozen throughout training and serves only as a fixed coordinate system for measuring masked feature drift\.

### 3\.3Task\-Specific Feature Identification

The mask𝐌t\\mathbf\{M\}\_\{t\}determines which SAE features are treated as adaptive and which are protected\. It should therefore capture not only which features are active on tasktt, but also nearby features in the SAE dictionary that represent related directions\. We construct𝐌t\\mathbf\{M\}\_\{t\}in two steps\.

##### Activation\-based relevance\.

We first estimate how strongly each SAE feature is used by the current task\. Using the frozen reference modelθ0\\theta\_\{0\}, we encode task examples through the frozen SAE and compute the mean absolute activation of each feature over content tokens\. This gives a relevance vector𝐑t∈ℝF\\mathbf\{R\}\_\{t\}\\in\\mathbb\{R\}^\{F\}with entries

Rt,j=𝔼x∼𝒟t​𝔼τ∈content​\(x\)​\[\|fj​\(x,τ;θ0\)\|\],R\_\{t,j\}=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{t\}\}\\mathbb\{E\}\_\{\\tau\\in\\mathrm\{content\}\(x\)\}\\bigl\[\|f\_\{j\}\(x,\\tau;\\theta\_\{0\}\)\|\\bigr\],\(14\)whereτ\\tauindexes content tokens\. We exclude chat\-template markers, padding, and EOS tokens so that the relevance estimate reflects task content rather than formatting artifacts\. The resulting scores are normalized to\[0,1\]\[0,1\]by dividing bymaxj⁡Rt,j\\max\_\{j\}R\_\{t,j\}\.

##### Decoder\-neighborhood propagation\.

Activation alone may miss features that are semantically related to the task but do not fire strongly in the sampled data\. We therefore propagate relevance over the SAE decoder geometry\. Let𝐝j\\mathbf\{d\}\_\{j\}denote the decoder vector associated with featurejj\. Features with similar decoder vectors write to similar directions in activation space, and are likely to represent related concepts\. We build akk\-NN graph from decoder\-vector cosine similarity and propagate relevance to neighboring features:

𝐌t=clamp​\(𝐑t\+Aknn​𝐑t,0,1\),\\mathbf\{M\}\_\{t\}=\\mathrm\{clamp\}\\\!\\bigl\(\\mathbf\{R\}\_\{t\}\+A\_\{\\mathrm\{knn\}\}\\mathbf\{R\}\_\{t\},\\;0,\\;1\\bigr\),\(15\)whereAknnA\_\{\\mathrm\{knn\}\}is the row\-normalized adjacency matrix restricted to each feature’s top\-kknearest neighbors, andclamp​\(⋅,0,1\)\\mathrm\{clamp\}\(\\cdot,0,1\)clips each entry of the vector to the interval\[0,1\]\[0,1\]\. The final mask is continuous: strongly task\-relevant features receive values close to 1, related features receive intermediate values through propagation, and unrelated features remain close to 0\. This graded mask avoids a hard binary partition and provides the weights used in \([10](https://arxiv.org/html/2606.26629#S3.E10)\)\.

## 4Experiments

We evaluate SAE\-guided regularization against representative baselines from weight\-space regularization, gradient projection, architectural isolation, and replay on two sequential benchmarks spanning 18 tasks\. Our experiments address three questions: \(1\) does activation\-space regularization improve the stability–plasticity tradeoff relative to weight\-space methods and replay under comparable storage constraints? \(2\) how sensitive is the method to its hyperparameters? and \(3\) does the choice of importance signal for mask construction matter?

### 4\.1Benchmarks, Model, and Training

##### TRACE\.

TRACE\-5000\(Wanget al\.,[2023b](https://arxiv.org/html/2606.26629#bib.bib17)\)evaluates cross\-domain continual learning through a fixed sequence of eight tasks: C\-STANCE \(Chinese stance detection\), FOMC \(financial sentiment\), MeetingBank \(meeting summarization\), Py150 \(code completion\), ScienceQA \(science multiple\-choice\), NumGLUE\-cm and NumGLUE\-ds \(arithmetic\), and 20Minuten \(German text simplification\)\. The sequence spans four languages, both generation and classification formats, and domains ranging from finance to code, creating substantial interference under sequential training\. Each task provides 5,000 training examples\. We follow the original TRACE protocol, using per\-task epoch counts of\[5,3,7,5,3,5,5,7\]\[5,3,7,5,3,5,5,7\]and a learning rate of1×10−41\{\\times\}10^\{\-4\}, which produces strong per\-task fitting and meaningful forgetting in the unprotected baseline\.

##### MedCL\.

MedCL complements TRACE by evaluating within\-domain continual learning on a 10\-task biomedical sequence: pubmedqa, scifact, pubhealth, gad, chemprot, ddi, pubmed\_rct, litcovid, bioasq, and druglib\. Although all tasks come from the biomedical domain, they vary in task format \(question answering, relation extraction, topic classification, and rating regression\) and label vocabulary\. We train for 3 epochs per task with a learning rate of5×10−55\{\\times\}10^\{\-5\}and evaluate with log\-probability scoring\. This evaluation avoids the generation\-format sensitivity that can affect open\-ended decoding on classification tasks with short label vocabularies\.

##### Model and adapter\.

All experiments use the instruction\-tuned Gemma\-2 9B\-it model with LoRA adapters applied to all linear projections \(r=256r\{=\}256,αLoRA=512\\alpha\_\{\\mathrm\{LoRA\}\}\{=\}512, dropout 0\.05\)\. The base model remains frozen, and only the adapter weights are updated\. SAE\-guided regularization uses Gemma Scope pretrained 16k\-width SAEs at residual\-stream layers 9, 20, and 31\. These SAEs are kept frozen throughout training and serve only as feature dictionaries\. We use AdamW\-8bit with an effective batch size of 128 on both benchmarks\.

##### Baselines\.

We compare against representative methods from the main continual learning families\. Weight\-space regularization baselines include EWC\(Kirkpatricket al\.,[2017](https://arxiv.org/html/2606.26629#bib.bib5)\), SI\(Zenkeet al\.,[2017](https://arxiv.org/html/2606.26629#bib.bib6)\)atλ=1\\lambda\{=\}1on TRACE, and MAS\(Aljundiet al\.,[2018](https://arxiv.org/html/2606.26629#bib.bib7)\)with magnitude\-matched importance scaling\. The gradient\-projection baseline is ELLA atλ=0\.1\\lambda\{=\}0\.1, which constrains updates using subspaces associated with past tasks\. The architectural baseline is O\-LoRA\(Wanget al\.,[2023a](https://arxiv.org/html/2606.26629#bib.bib18)\), which allocates orthogonal LoRA subspaces across tasks\. The replay baselines use unprotected fine\-tuning with 2%, 5%, and 10% replay buffers\. All baselines use the same optimizer, learning rate, batch size, and LoRA rank; only the continual learning strategy differs\.

##### Metrics\.

We report four metrics\. Overall Performance \(OP\) is the mean final accuracy across all tasks\. Letdiagj\\mathrm\{diag\}\_\{j\}denote the accuracy on taskjjimmediately after training that task, and letfinj\\mathrm\{fin\}\_\{j\}denote the final accuracy on taskjjafter the full task sequence has been learned\. Backward Transfer \(BWT\) is the mean offinj−diagj\\mathrm\{fin\}\_\{j\}\-\\mathrm\{diag\}\_\{j\}, where negative values indicate forgetting\. Average Retained Accuracy \(ARA\) is the mean ofmin⁡\(finj/diagj,1\)\\min\(\\mathrm\{fin\}\_\{j\}/\\mathrm\{diag\}\_\{j\},\\,1\)and measures how much task performance is retained after subsequent training\. Plasticity is the mean diagonal accuracy and measures how well the model learns each task when it is first trained\. All confidence intervals are 95% bootstrap intervals over evaluation examples \(2,000 resamples, single training seed\)\.

### 4\.2Main Results

Table 1:TRACE\-5000 results \(Gemma\-2 9B\-it, 8 tasks\)\. Methods are ranked by OP\. SAE\-guided regularization achieves the strongest non\-architectural result, surpassing all weight\-space methods and experience replay\.Table 2:MedCL results \(Gemma\-2 9B\-it, 10 biomedical tasks, 3 epochs/task, log\-probability evaluation\)\. SAE leads the methods that retain no previous\-task examples; replay with larger buffers \(≥\\geq5%\) surpasses SAE but requires storing past data\.We first compare SAE\-guided regularization with the main continual learning baselines on TRACE and MedCL\. On TRACE, as shown in Table[1](https://arxiv.org/html/2606.26629#S4.T1), SAE\-guided regularization achieves OP=0\.545=0\.545with 95% CI as\[0\.529,0\.562\]\[0\.529,0\.562\]on TRACE, the strongest result among all non\-architectural methods\. This represents an absolute OP improvement of0\.1940\.194over unprotected fine\-tuning, with fully non\-overlapping confidence intervals\. Among weight\-space regularizers, SI performs best \(0\.5130\.513\), followed by ELLA \(0\.4750\.475\), EWC \(0\.4470\.447\), and MAS \(0\.4190\.419\)\. SAE outperforms SI by\+3\.2\+3\.2points; although the confidence intervals partially overlap, SAE leads SI consistently across every configuration we tested\. SAE also outperforms experience replay at both 2% \(0\.5270\.527\) and 10% \(0\.5250\.525\) data budgets, while storing no previous\-task data\.

EWC illustrates a stability\-dominant failure mode\. Its retention metrics are the strongest of any method \(ARA=0\.987=0\.987, BWT=−0\.006=\-0\.006\), yet its overall performance is among the weakest \(0\.4470\.447\)\. This gap is explained by plasticity: with plasticity of only0\.4530\.453, EWC learns each task less effectively than even unprotected fine\-tuning \(0\.5690\.569\)\. In this setting, low forgetting is achieved mainly by suppressing learning; the model barely departs from its pretrained state, leaving little to forget\. As we argue in Section[5\.1](https://arxiv.org/html/2606.26629#S5.SS1), this rigidity is a structural consequence of weight\-space regularization under superposition, not a hyperparameter calibration failure\.

![Refer to caption](https://arxiv.org/html/2606.26629v1/figure_stability_plasticity.png)Figure 2:Stability–plasticity tradeoff on TRACE\. Each point represents a method, and gray curves indicate iso\-OP contours \(OP≈\\approxretention×\\timesplasticity\)\. EWC achieves high retention by sacrificing plasticity, whereas MAS and unprotected fine\-tuning preserve plasticity but suffer greater forgetting\. SAE\-guided regularization combines high retention with high plasticity, placing it closest to the architectural upper bound O\-LoRA among regularization\-based methods\.[Figure˜2](https://arxiv.org/html/2606.26629#S4.F2)summarizes this tradeoff\. Every weight\-space regularizer lies along a frontier where retention is purchased with plasticity \(EWC\) or plasticity is kept at the cost of retention \(MAS, SI, ELLA\)\. SAE\-guided regularization breaks off this frontier: it matches the plasticity of unprotected fine\-tuning \(0\.5670\.567vs0\.5690\.569\) while retaining95%95\\%of learned performance\. This behavior follows directly from the design of the method: the soft mask permits free movement in task\-relevant feature directions, preserving plasticity, while anchoring the remaining features to preserve stability\. This is what weight\-space importance estimates cannot deliver under superposition\.

Table[2](https://arxiv.org/html/2606.26629#S4.T2)shows the same overall pattern on MedCL\. SAE achieves OP=0\.510=0\.510with 95% CI as\[0\.495,0\.524\]\[0\.495,0\.524\], leading methods that retain no previous\-task examples and showing a confidence interval clearly separated from MAS \(0\.4560\.456\), EWC \(0\.4230\.423\), and unprotected fine\-tuning \(0\.3900\.390\)\. Replay with 5% and 10% buffers outperforms SAE \(0\.5530\.553and0\.5660\.566, respectively\), but these methods store and rehearse previous\-task data, a resource SAE does not require\. O\-LoRA again leads overall \(0\.6260\.626\) through per\-task parameter isolation\.

The comparison also highlights two additional patterns\. First, the absolute performance gap between methods is larger on TRACE than on MedCL, likely because TRACE’s cross\-domain diversity creates sharper interference between tasks and amplifies the difference between effective and ineffective protection\. Second, adding 2% replay to SAE on TRACE does not improve over SAE alone \(0\.5420\.542vs0\.5450\.545\), suggesting that the two mechanisms address overlapping failure modes under aggressive training: both constrain the model from drifting too far from its previous state, and when one mechanism is already effective, the other adds little\.

### 4\.3Hyperparameter Sensitivity

We evaluate the sensitivity of SAE\-guided regularization to the four hyperparameters of the squared\-hinge loss on TRACE\. The sweep varies one factor at a time around the center\(α=0\.1,β=0\.1,εp=0\.1,εs=0\.17\)\(\\alpha\{=\}0\.1,\\,\\beta\{=\}0\.1,\\,\\varepsilon\_\{p\}\{=\}0\.1,\\,\\varepsilon\_\{s\}\{=\}0\.17\), with the remaining parameters fixed at their center values\. The center is used only as the sweep baseline; the final configuration uses the best value identified along each axis\.[Figure˜3](https://arxiv.org/html/2606.26629#S4.F3)reports the results\.

![Refer to caption](https://arxiv.org/html/2606.26629v1/figure_hparam_sweep.png)Figure 3:Hyperparameter sensitivity on TRACE\. We perform a one\-factor\-at\-a\-time sweep around the centerα=0\.1\\alpha\{=\}0\.1,β=0\.1\\beta\{=\}0\.1,εp=0\.1\\varepsilon\_\{p\}\{=\}0\.1,εs=0\.17\\varepsilon\_\{s\}\{=\}0\.17, varying one parameter per panel while holding the others fixed\. Shaded bands denote 95% bootstrap confidence intervals, and stars mark the per\-axis optima\. Performance is most sensitive to the protection parameters \(β\\beta,εs\\varepsilon\_\{s\}; left panels\), while the guidance parameters \(α\\alpha,εp\\varepsilon\_\{p\}; right panels\) are comparatively stable\.The protection parameters \(β\\betaandεs\\varepsilon\_\{s\}\) exert the strongest influence on performance\. The protection weightβ\\betacontrols how strongly the penalty resists drift on protected features\. OP remains stable acrossβ∈\[0\.17,0\.50\]\\beta\\in\[0\.17,0\.50\], varying by less than 0\.04, but drops sharply atβ=0\.05\\beta\{=\}0\.05, where the penalty is too weak to prevent meaningful forgetting\. The protection budgetεs\\varepsilon\_\{s\}controls how much protected\-feature drift is tolerated before the hinge activates\. OP peaks atεs=0\.35\\varepsilon\_\{s\}\{=\}0\.35and degrades at both extremes:εs=0\.10\\varepsilon\_\{s\}\{=\}0\.10over\-constrains the model, analogous to EWC’s rigidity, whileεs=0\.50\\varepsilon\_\{s\}\{=\}0\.50leaves too much room for unchecked drift\.

The guidance parameters \(α\\alphaandεp\\varepsilon\_\{p\}\) have a smaller effect\. Performance is stable forα∈\[0\.05,0\.10\]\\alpha\\in\[0\.05,0\.10\]and drops mildly atα=0\.20\\alpha\{=\}0\.20\. The guidance budgetεp\\varepsilon\_\{p\}shows a similar pattern: it is stable over\[0\.05,0\.10\]\[0\.05,0\.10\]and degrades atεp=0\.20\\varepsilon\_\{p\}\{=\}0\.20, where the minimum\-drift requirement becomes too demanding and forces task\-relevant features to move beyond what the task loss requires\. The stronger sensitivity to protection than to guidance suggests that preventing drift on protected features is the primary mechanism, while encouraging movement on task\-relevant features acts as a secondary refinement\.

The best per\-axis values define the final configuration:β=0\.35\\beta\{=\}0\.35,εs=0\.35\\varepsilon\_\{s\}\{=\}0\.35,α=0\.10\\alpha\{=\}0\.10, andεp=0\.10\\varepsilon\_\{p\}\{=\}0\.10\. This joint configuration achieves OP=0\.545=0\.545, exceeding the best single\-axis settings considered in isolation \(β=0\.35\\beta\{=\}0\.35alone yields0\.5390\.539;εs=0\.35\\varepsilon\_\{s\}\{=\}0\.35alone yields0\.5150\.515\)\.

### 4\.4Mask Importance Variants

We next test whether performance depends on the particular relevance score used to construct the SAE feature mask\. The default score is mean activation magnitude \(Section[3\.3](https://arxiv.org/html/2606.26629#S3.SS3)\): features that fire strongly on task content are treated as task\-relevant\. We compare this activation\-based score with two alternative feature\-importance estimates, using the same soft\-mask construction and the same hyperparameters \(α=0\.1\\alpha\{=\}0\.1,β=0\.35\\beta\{=\}0\.35,εp=0\.1\\varepsilon\_\{p\}\{=\}0\.1,εs=0\.35\\varepsilon\_\{s\}\{=\}0\.35\)\.

*Gradient\-based causal*importance measures how sensitive the supervised prediction is to each feature\. Rather than asking only which features are active, it asks which features influence the target\-token log probability\. We estimate this score from the gradient of the target\-token log probability with respect to each SAE feature activation\. On TRACE, this variant achieves OP=0\.542=0\.542, essentially matching the activation\-based default \(0\.5450\.545\)\.

*Empirical Fisher*importance measures𝔼​\[\(∂ℒ/∂fj\)2\]\\mathbb\{E\}\[\(\\partial\\mathcal\{L\}/\\partial f\_\{j\}\)^\{2\}\]—the sensitivity of the training loss to each SAE feature\. This captures which features the loss landscape depends on, rather than which features are active\. On TRACE, this variant achieves OP=0\.510=0\.510, somewhat below the other two\.

The near\-equivalence of activation\-based and gradient\-based causal importance suggests that, for the tasks we study, features that fire strongly on task content are largely the same features that influence the supervised output\. The weaker performance of the empirical Fisher variant may reflect the noisier estimation introduced by squared gradients\. Overall, the robustness across three distinct importance signals supports the view that the main benefit comes from regularizing in the SAE feature basis itself\. The exact relevance estimator matters less, provided it identifies a reasonable set of task\-relevant features\.

## 5Analysis

The results in Section[4](https://arxiv.org/html/2606.26629#S4)show that SAE\-guided regularization outperforms weight\-space methods across two benchmarks\. This section examines why this happens and what the method costs\. We first test the central mechanism: superposition makes weight space a poor coordinate system for concept\-level protection, whereas SAE features provide more selective units\. We then analyze the computational and storage cost of using this feature\-space representation\.

### 5\.1Why Feature\-Space Protection Succeeds Where Weight\-Space Protection Fails

Our central thesis is that the weakness of weight\-space regularization is structural rather than merely due to poor importance estimation or hyperparameter calibration\. Under superposition, a model represents many more features than activation dimensions, so individual neurons and weights participate in multiple unrelated concepts\. A weight\-level penalty therefore cannot isolate which concept should be preserved\. SAE features provide a more suitable coordinate system because they decompose these entangled activations into more monosemantic directions\(Brickenet al\.,[2023](https://arxiv.org/html/2606.26629#bib.bib26), Hubenet al\.,[2024](https://arxiv.org/html/2606.26629#bib.bib25)\)\.

We measure this entanglement directly in Gemma\-2 9B\. For each MLP neuron, we decompose its output direction over the SAE dictionary\. The median neuron draws on 500–1350 distinct SAE features, depending on the layer, indicating that each neuron carries fragments of hundreds of concepts\.

This many\-to\-many mapping creates a*granularity mismatch*between the unit of protection and the unit of knowledge\. When EWC protects weights associated with task 1, it also constrains the many SAE features that pass through those weights, including features unrelated to task 1\. If task 2 later needs to modify one of those features, the penalty resists\. Feature\-space regularization avoids this mismatch by penalizing drift directly along SAE feature directions, allowing featurejjto change even when it shares neurons with protected featurekk\.

This account yields two testable predictions\. First, task\-relevant units should be more linearly separable in the SAE feature basis than in the neuron basis\. Second, protecting one task in weight space should also constrain features used by other tasks, whereas protection in the SAE feature basis should be more selective\. We test these predictions with a separability analysis and a collateral\-constraint analysis\.

#### 5\.1\.1Experiment 1: Task Separability in Feature Space vs Weight Space

The separability test asks a simple question: given the units that two different tasks rely on, can a classifier identify which task each unit belongs to? If a coordinate system supports concept\-level protection, units relevant to different tasks should occupy distinguishable regions of that space\. If it does not, those units should be intermixed\.

To make the comparison fair, we represent both SAE features and MLP neurons as directions in the sameDD\-dimensional residual\-stream space\. For each task pair\(A,B\)\(A,B\), we identify the top\-5% most relevant units in each basis\. In the SAE basis, these are the features with the highest activation relevance for each task, represented by their decoder vectors𝐝k∈ℝD\\mathbf\{d\}\_\{k\}\\in\\mathbb\{R\}^\{D\}\. In the neuron basis, these are the MLP neurons with the highest task\-specific Fisher importance, represented by their output\-projection columns\. Since the Fisher diagonal is defined over parameters, we obtain a neuron\-level score by aggregating the Fisher importance of the parameters associated with each neuron\.

We then train a logistic classifier under 5\-fold cross\-validation to predict each direction’s task membership, reporting ROC\-AUC against a chance level of 0\.50\. Representations are concatenated across SAE layers 9, 20, and 31 for the feature basis and across all MLP layers for the neuron basis\. For each benchmark, we evaluate three task pairs spanning different degrees of domain similarity: a cross\-domain pair \(Py150 vs 20Minuten\), a within\-domain pair \(NumGLUE\-cm vs NumGLUE\-ds\), and an intermediate pair \(C\-STANCE vs FOMC\)\.

Table 3:Linear separability of task\-relevant units \(5\-fold CV ROC\-AUC, chance = 0\.50\), pooled across three task pairs per benchmark\. Different tasks’ relevant units are clearly distinguishable in the SAE feature basis but intermixed at chance level in the neuron basis\.The results show a clear contrast \([Table˜3](https://arxiv.org/html/2606.26629#S5.T3)\)\. In the SAE basis, task\-relevant features are clearly separable: AUC reaches 0\.882 on TRACE and 0\.866 on MedCL, indicating that different tasks activate distinguishable sets of SAE features\. In the neuron basis, separability is statistically indistinguishable from chance \(0\.498 and 0\.502\)\. The classifier cannot determine which task a Fisher\-important neuron belongs to, because polysemantic neurons participate in both tasks’ computations and their output directions carry little task\-discriminative structure\. The contrast holds on both the all\-biomedical MedCL suite, where tasks share domain vocabulary, and the maximally diverse TRACE suite, where tasks span Chinese, English, German, and code\. This suggests that the effect reflects a structural property of the representation rather than an artifact of domain dissimilarity\.

![Refer to caption](https://arxiv.org/html/2606.26629v1/x1.png)Figure 4:t\-SNE projections of task\-relevant SAE features for three TRACE task pairs across layers 9, 20, and 31\. Each point is an SAE feature direction; color indicates task assignment;×\\timesmarks features relevant to both tasks\. AUC values report linear separability in the full 16,384\-dimensional feature space\. Task\-relevant features form visually distinct clusters that become more separated at deeper layers\.[Figure˜4](https://arxiv.org/html/2606.26629#S5.F4)provides a qualitative view of the same feature\-basis separability on TRACE\. Per\-layer AUC ranges from 0\.57 for the most similar task pair at the shallowest layer \(NumGLUE\-cm vs NumGLUE\-ds at layer 9\) to 0\.89 for a cross\-domain pair at mid\-depth \(Py150 vs 20Minuten at layer 20\)\. Separability increases with depth: mean AUC rises from 0\.67 at layer 9 to 0\.75 at layer 31, consistent with deeper layers encoding more task\-specific structure\. Cross\-domain pairs are also more separable than within\-domain pairs, as expected when tasks that share fewer concepts occupy more distinct regions of the feature space\. The first prediction is therefore confirmed: the SAE basis provides concept\-aligned axes along which task knowledge can be linearly isolated, whereas the neuron basis does not\.

#### 5\.1\.2Experiment 2: Collateral Constraint Under Weight vs Feature Protection

The separability result establishes that weight space lacks concept\-aligned axes\. The collateral\-constraint test asks what this absence costs in practice: when a regularizer protects the units it deems important for one task, how much does it inadvertently constrain the features used by a*different*task?

The weight\-space measurement follows directly from the geometry\. When EWC protects a set of neurons for task A, those neurons’ output directions span a subspace of the residual stream\. Any SAE feature whose decoder direction overlaps this subspace is effectively constrained, because the penalty resists change in the directions that feature writes to\. We quantify this overlap as*exposure*: the fraction of a feature’s decoder\-direction magnitude that projects onto task A’s protected neuron subspace, using the top\-5% of neurons by Fisher importance as the protected set\.

We compute exposure for two groups of features: task A’s own relevant features, which give the*intended*exposure, and task B’s relevant features, which give the*collateral*exposure\. Their ratio measures selectivity\. A ratio near 1\.0 means that protecting task A constrains task B’s features almost as strongly as task A’s own features, indicating non\-selective protection\. A lower ratio indicates more concept\-targeted protection\.

For feature\-space protection, we compute the analogous ratio directly in the SAE basis\. The intended exposure is the overlap with task A’s top\-5% SAE features, and the collateral exposure is the overlap with task B’s top\-5% SAE features, normalized on the same cross\-task\-to\-same\-task scale\. We evaluate all task pairs across all three SAE layers on both benchmarks\. As a floor reference, a random 5% neuron set captures approximately 5% of any feature’s mass\.

Table 4:Collateral constraint: ratio of cross\-task \(collateral\) to same\-task \(intended\) exposure\. A ratio near 1\.0 indicates non\-selective protection; lower is more selective\. Weight\-space protection is nearly non\-selective; feature\-space protection is substantially more selective, especially on the diverse TRACE benchmark\.The results directly confirm the second prediction \([Table˜4](https://arxiv.org/html/2606.26629#S5.T4)\)\. Under weight\-space protection, the collateral ratio is 0\.96 on MedCL and 0\.91 on TRACE: protecting task A’s neurons constrains task B’s features at 91–96% of the rate it constrains task A’s own features\. Thus, even when the protected neurons are selected using task\-specific Fisher importance, the resulting constraint is nearly indiscriminate at the concept level\. It constrains features according to whether they pass through the protected neurons, not according to which task those features serve\. This is the collateral constraint imposed by superposition: because features are distributed across neurons, protecting a subset of neurons offers little control over*which*features are constrained\.

Under feature\-space protection, the ratio falls to 0\.61 on MedCL and 0\.43 on TRACE, meaning that the second task’s features are substantially less exposed to constraints intended for the first task\. The selectivity gap is wider on TRACE \(0\.48\) than on MedCL \(0\.35\), and the difference is informative: when tasks span different domains, their relevant features occupy more distinct regions of the dictionary and feature\-level protection can exploit the separation\. In contrast, the all\-biomedical MedCL tasks genuinely share concepts, including biomedical vocabulary and clinical reasoning, so even concept\-level protection incurs more overlap\. Because superposition spreads each feature’s write direction across hundreds of neurons, any 5% neuron set captures only about 7% of a typical feature’s participation mass, compared with a random\-set floor of about 5%\. The key comparison is therefore the ratio, which shows that weight\-level protection cannot discriminate between tasks’ features even when its importance estimate is task\-specific\.

#### 5\.1\.3Connecting the Evidence to EWC’s Behavioral Failure

Together, the two experiments explain EWC’s stability\-dominant behavior in Section[4](https://arxiv.org/html/2606.26629#S4)\. EWC achieves near\-perfect retention \(ARA = 0\.987\) but has the lowest plasticity of any method tested \(0\.453, compared with 0\.569 for unprotected fine\-tuning\)\. The collateral\-constraint measurement explains this pattern: when EWC protects taskkk’s weights, it constrains the next task’s relevant features at 91–96% of the rate it constrains taskkk’s own features\. The model therefore cannot adapt to new material without incurring penalties on many of the features it needs to modify\. At anyλ\\lambdastrong enough to prevent forgetting, plasticity collapses; at anyλ\\lambdaweak enough to permit learning, the protection no longer prevents forgetting\. The tradeoff is structural, imposed by the non\-selectivity of weight\-space protection under superposition rather than by a particular value ofλ\\lambda\.

SAE\-guided regularization avoids this tradeoff because it operates in a coordinate system where task features are separable \([Table˜3](https://arxiv.org/html/2606.26629#S5.T3)\)\. With a collateral ratio of 0\.43–0\.61, many incoming\-task features remain comparatively unconstrained even while previous\-task features are protected\. The method can therefore retain prior knowledge \(ARA = 0\.950\) while preserving plasticity \(0\.567\), navigating the stability–plasticity dilemma rather than being trapped at one of its extremes\.

### 5\.2Computational Cost and Scalability

A regularization method’s practical value depends not only on how well it prevents forgetting, but also on what it costs to run\. The methods compared in Section[4](https://arxiv.org/html/2606.26629#S4)rely on different mechanisms—feature\-space penalties, parameter\-space anchors, and architectural adapters—and therefore have different computational footprints\.

We measure these costs in a controlled micro\-benchmark\. Each method trains the same model on the same task data, with the same LoRA configuration and effective batch size, on a single dedicated GPU for 120 optimizer steps after discarding warmup\. We record the one\-off precompute time required before training, the steady\-state per\-step training time, the peak GPU memory, and the persistent storage retained per task\. Regularizers that ordinarily activate only from the second task onward are profiled with a same\-task anchor, since the penalty cost is independent of which task the anchor encodes\. Architectural methods that require prior\-task state \(ELLA, O\-LoRA\) are profiled as two\-task chains, with the second task measured\.

Table 5:Computational overhead per method \(Gemma\-2 9B\-it, LoRAr=256r\{=\}256, single A800, effective batch 8\)\. Weight\-space methods are shown with anchors held on GPU\.†ELLA is measured with its anchor CPU\-offloaded\.‡O\-LoRA usesr=16r\{=\}16per\-task adapters, so its memory is not directly comparable\.The first observation from[Table˜5](https://arxiv.org/html/2606.26629#S5.T5)is that GPU\-resident regularizers have similar per\-step cost\. EWC, MAS, SI, O\-LoRA, and SAE\-guided regularization all train close to the unprotected baseline: 1\.78–1\.91 s/step compared with 1\.70 s/step\. The penalty arithmetic itself, whether a feature\-space hinge over SAE activations or a quadratic penalty over LoRA parameters, is small relative to the forward and backward passes of a 9B model\. SAE’s timing includes the forward pass through the frozen SAE dictionaries and uses the regularization penalty every fifth step, which reduces the average overhead\.

This parity in step time conceals an important asymmetry\. A weight\-space penalty requires a reference copy of the trainable parameters and a per\-parameter importance vector to be available at every step\. At LoRA rankr=256r\{=\}256, this state is roughly 7 GB in fp32 for EWC and MAS\. Held on GPU, it increases peak memory by about 7 GB; for SI, which maintains three full\-parameter buffers, the increase is about 17 GB\.

Offloading these anchors to CPU trades memory for time\. The per\-step host\-to\-device transfer slows EWC to 5\.42 s/step \(3\.2×3\.2\\timesbaseline\), MAS to 5\.39 s/step, and SI to 9\.16 s/step \(5\.4×5\.4\\timesbaseline\)\. Thus, per\-parameter methods face a memory–speed tradeoff: either keep the anchor on GPU and pay in device memory, or offload it and pay in training time\. SAE\-guided regularization avoids this tradeoff\. Its per\-task state is a 412 KB feature mask, and its only sizable object is the fixed SAE dictionary \(352M parameters, accounting for most of the\+1\.7\+1\.7GB memory increment\), which is shared across tasks and used only during training\.

Persistent storage separates the methods most clearly\. SAE\-guided regularization retains only a 412 KB mask per task, whereas Fisher\-anchor methods retain 6\.5 GB and ELLA retains 16 GB of reconstructed weight deltas\. O\-LoRA’s per\-task adapters are smaller \(0\.45 GB\) but accumulate over tasks, and O\-LoRA is the only method in the comparison with nonzero inference overhead: its combined adapter grows in rank with every task\. All regularization\-based methods, including ours, deploy as a plain LoRA with no inference\-time footprint\.

The measured storage asymmetry has a simple closed form\. LetNNdenote the number of trainable parameters,LLthe number of hooked layers,FFthe SAE dictionary width,DDthe residual\-stream dimensionality, andbbthe bytes per stored value\. Letccdenote the bytes of persistent state required per trainable parameter\. A weight\-space penalty must persist, per task, an anchor and an importance value for every trainable parameter\. In contrast, feature\-space regularization persists only a relevance mask per task, plus a single fixed dictionary shared by all tasks:

Sweight=c⋅N\(c=2​b​for EWC/MAS,c≈4​b​for SI\),Sfeat=b⋅L⋅F,Sdict=2​b⋅L⋅D⋅F\.S\_\{\\mathrm\{weight\}\}=c\\cdot N\\quad\(c=2b\\text\{ for EWC/MAS\},\\;c\\approx 4b\\text\{ for SI\}\),\\qquad S\_\{\\mathrm\{feat\}\}=b\\cdot L\\cdot F,\\qquad S\_\{\\mathrm\{dict\}\}=2b\\cdot L\\cdot D\\cdot F\.\(16\)The per\-task storage ratioSweight/Sfeat=c​N/\(b​L​F\)S\_\{\\mathrm\{weight\}\}/S\_\{\\mathrm\{feat\}\}=cN/\(bLF\)grows linearly with the number of trainable parameters\. The per\-task feature mask, by contrast, depends on the SAE dictionary size rather than on the model or adapter parameter count\. At our configuration \(N=864N=864M,L=3L\{=\}3,F=16,384F\{=\}16\{,\}384, fp32\),[eq\.˜16](https://arxiv.org/html/2606.26629#S5.E16)predicts a 6\.9 GB anchor, close to the measured 6\.5 GB\.

Table 6:Cost\-model projections across scale \(LoRAr=256r\{=\}256on all linear projections unless noted; weight\-space anchor atc=8c\{=\}8bytes\)\. The 9B row validates the model against measurement\. Weight\-space per\-task state grows linearly with trainable parameters, while the feature\-space mask remains fixed for a given SAE dictionary\.[Table˜6](https://arxiv.org/html/2606.26629#S5.T6)instantiates the model across scales\. At 27B, the GPU\-resident configuration needed to keep weight\-space methods near baseline speed becomes difficult to sustain: EWC requires\+14\.6\+14\.6GB on device, and SI would require roughly\+44\+44GB\. Under full\-parameter fine\-tuning, the comparison becomes qualitative rather than quantitative: a 216 GB per\-task anchor is not a practical state to retain, while the feature\-space mask is unchanged\.

The fixed dictionary cost is the main caveat\.SdictS\_\{\\mathrm\{dict\}\}grows withD⋅FD\\cdot Fand is roughly 0\.9 GB in bf16 for three 16k\-width layers at 27B\. However, this cost is incurred once, shared across all tasks, and used only during training\. It therefore amortizes over the task sequence, unlike the per\-task anchors accumulated by weight\-space methods\.

The efficiency result and the mechanism result are, in the end, two faces of the same property\. Weight\-space methods perform their bookkeeping at the granularity of parameters, of which there are billions; their protection is non\-selective \(Section[5\.1\.2](https://arxiv.org/html/2606.26629#S5.SS1.SSS2)\) and their state scales with the parameter count\. Feature\-space regularization performs its bookkeeping at the granularity of concepts, of which the dictionary tracks tens of thousands; its protection is selective and its per\-task state is proportional to the concept count alone\. The coordinate system that aligns with the structure of the model’s knowledge is also the coordinate system in which that knowledge is cheapest to track\.

## 6Limitations

Although the experiments support SAE\-guided activation regularization as an effective feature\-space approach to continual learning, several limitations remain\.

First, the results are reported on Gemma\-2 9B\-it\. Validation on Mistral\-7B and Gemma\-2 9B \(base, non\-instruction\-tuned\) is in progress\. Since the TRACE paper reports different continual learning dynamics across model families, cross\-model generalization requires further verification\.

Second, the interaction between replay and SAE\-guided regularization remains unresolved\. Under the aggressive TRACE training protocol, adding 2% replay to SAE regularization does not improve over SAE alone \(0\.542 vs 0\.545\)\. Whether the two mechanisms become complementary under gentler training regimes or longer task sequences is an open question\.

Third, activation\-level regularization does not fully address output\-format interference\. When sequential tasks have conflicting label vocabularies, recency\-driven label\-format overwriting can overwhelm the regularizer\. On MedCL, tasks with idiosyncratic answer formats \(e\.g\., bioasq\) fall to near\-zero accuracy under all methods that retain no previous\-task examples, while replay—which re\-exposes the model to the original format—largely restores them\. Activation\-space protection preserves internal representations but does not by itself prevent this output\-level interference\.

Fourth, the method depends on the coverage of the pretrained SAE\. Current SAEs capture approximately 85–95% of activation variance, leaving the remaining variance invisible to the regularizer\. This creates a possible channel through which forgetting can proceed unpenalized\.

Finally, there remains a gap to architectural methods\. O\-LoRA achieves substantially higher OP \(0\.630 vs 0\.545 on TRACE\) through per\-task parameter isolation\. The two approaches address different design points; combining SAE\-guided regularization with orthogonal subspace methods is a natural direction for future work\.

## 7Conclusion

We have introduced SAE\-guided activation regularization for continual learning in LLMs—the first method to regularize in the model’s activation space using Sparse Autoencoder features as a monosemantic coordinate system\. The method is derived from a constrained optimization framework with explicit stability and plasticity constraints, relaxed to a squared\-hinge training loss via the quadratic penalty method\.

On TRACE \(8 tasks\), the method achieves the strongest non\-architectural continual learning result, surpassing weight\-space regularizers, gradient\-projection methods, and the replay baselines tested\. On MedCL \(10 tasks\), it leads methods that retain no previous\-task examples, while larger replay buffers achieve higher accuracy at the cost of storing and rehearsing past data\. Across both benchmarks, the consistent weakness of weight\-space baselines and the strength of activation\-space regularization support the core thesis: polysemanticity—the encoding of multiple concepts in individual weights—is a structural barrier to weight\-space regularization in large language models, and SAE\-based regularization in a monosemantic feature basis offers a more selective alternative\.

## References

- R\. Aljundi, F\. Babiloni, M\. Elhoseiny, M\. Rohrbach, and T\. Tuytelaars \(2018\)Memory aware synapses: learning what \(not\) to forget\.InProceedings of the European Conference on Computer Vision,pp\. 139–154\.Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p1.1),[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2606.26629#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2606.26629#S4.SS1.SSS0.Px4.p1.2)\.
- S\. Bills, N\. Cammarata, D\. Mossing, H\. Tillman, L\. Gao, G\. Goh, I\. Sutskever, J\. Leike, J\. Wu, and W\. Saunders \(2023\)Language models can explain neurons in language models\.Note:[https://openaipublic\.blob\.core\.windows\.net/neuron\-explainer/paper/index\.html](https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html)Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p2.1),[§2\.3](https://arxiv.org/html/2606.26629#S2.SS3.p1.1)\.
- T\. Bricken, A\. Templeton, J\. Batson, B\. Chen, A\. Jermyn, T\. Conerly, N\. L\. Turner, C\. Anil, C\. Denison, A\. Askell, R\. Lasenby, Y\. Wu, S\. Kravec, N\. Schiefer, T\. Maxwell, N\. Joseph, A\. Tamkin, K\. Nguyen, B\. McLean, J\. E\. Burke, T\. Hume, S\. Carter, T\. Henighan, and C\. Olah \(2023\)Towards monosemanticity: decomposing language models with dictionary learning\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2023/monosemantic-features)Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p3.1),[§2\.3](https://arxiv.org/html/2606.26629#S2.SS3.p2.1),[§5\.1](https://arxiv.org/html/2606.26629#S5.SS1.p1.1)\.
- P\. Buzzega, M\. Boschini, A\. Porrello, D\. Abati, and S\. Calderara \(2020\)Dark experience for general continual learning: a strong, simple baseline\.InProceedings of Advances in Neural Information Processing Systems,Vol\.33,pp\. 15920–15930\.Cited by:[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p2.1)\.
- A\. Chaudhry, M\. Ranzato, M\. Rohrbach, and M\. Elhoseiny \(2019\)Efficient lifelong learning with A\-GEM\.InProceedings of the International Conference on Learning Representations,Cited by:[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p2.1),[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p3.1)\.
- N\. Elhage, T\. Hume, C\. Olsson, N\. Schiefer, T\. Henighan, S\. Kravec, Z\. Hatfield\-Dodds, R\. Lasenby, D\. Drain, C\. Chen, R\. Grosse, S\. McCandlish, J\. Kaplan, D\. Amodei, M\. Wattenberg, and C\. Olah \(2022\)Toy models of superposition\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2022/toy_model/index.html)Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p2.1),[§2\.3](https://arxiv.org/html/2606.26629#S2.SS3.p1.1)\.
- R\. Huben, H\. Cunningham, L\. R\. Smith, A\. Ewart, and L\. Sharkey \(2024\)Sparse autoencoders find highly interpretable features in language models\.InProceedings of the International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p3.1),[§2\.3](https://arxiv.org/html/2606.26629#S2.SS3.p2.1),[§5\.1](https://arxiv.org/html/2606.26629#S5.SS1.p1.1)\.
- T\. Kao, K\. T\. Jensen, G\. M\. van de Ven, A\. Bernacchia, and G\. Hennequin \(2021\)Natural continual learning: success is a journey, not \(just\) a destination\.InProceedings of Advances in Neural Information Processing Systems,Vol\.34\.Cited by:[§2\.2](https://arxiv.org/html/2606.26629#S2.SS2.p1.1)\.
- J\. Kirkpatrick, R\. Pascanu, N\. Rabinowitz, J\. Veness, G\. Desjardins, A\. A\. Rusu, K\. Milan, J\. Quan, T\. Ramalho, A\. Grabska\-Barwinska, D\. Hassabis, C\. Clopath, D\. Kumaran, and R\. Hadsell \(2017\)Overcoming catastrophic forgetting in neural networks\.Proceedings of the National Academy of Sciences114\(13\),pp\. 3521–3526\.External Links:[Document](https://dx.doi.org/10.1073/pnas.1611835114)Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p1.1),[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2606.26629#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2606.26629#S4.SS1.SSS0.Px4.p1.2)\.
- Z\. Li and D\. Hoiem \(2018\)Learning without forgetting\.IEEE Transactions on Pattern Analysis and Machine Intelligence40\(12\),pp\. 2935–2947\.Cited by:[§2\.2](https://arxiv.org/html/2606.26629#S2.SS2.p3.1)\.
- T\. Lieberum, S\. Rajamanoharan, A\. Conmy, L\. Smith, N\. Sonnerat, V\. Varma, J\. Kramar, A\. Dragan, R\. Shah, and N\. Nanda \(2024\)Gemma scope: open sparse autoencoders everywhere all at once on Gemma 2\.InProceedings of the BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP,pp\. 278–300\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.blackboxnlp-1.19)Cited by:[§2\.3](https://arxiv.org/html/2606.26629#S2.SS3.p2.1)\.
- D\. Lopez\-Paz and M\. Ranzato \(2017\)Gradient episodic memory for continual learning\.InProceedings of Advances in Neural Information Processing Systems,Vol\.30\.Cited by:[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p2.1)\.
- A\. Mallya and S\. Lazebnik \(2018\)PackNet: adding multiple tasks to a single network by iterative pruning\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition,pp\. 7765–7773\.Cited by:[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p2.1)\.
- M\. McCloskey and N\. J\. Cohen \(1989\)Catastrophic interference in connectionist networks: the sequential learning problem\.Psychology of Learning and Motivation24,pp\. 109–165\.Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p1.1),[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p1.1)\.
- C\. Olah, N\. Cammarata, L\. Schubert, G\. Goh, M\. Petrov, and S\. Carter \(2020\)Zoom in: an introduction to circuits\.Distill5\(3\)\.External Links:[Document](https://dx.doi.org/10.23915/distill.00024.001)Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p2.1),[§2\.3](https://arxiv.org/html/2606.26629#S2.SS3.p1.1)\.
- R\. Ratcliff \(1990\)Connectionist models of recognition memory: constraints imposed by learning and forgetting functions\.Psychological Review97\(2\),pp\. 285–308\.Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p1.1),[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p1.1)\.
- A\. A\. Rusu, N\. C\. Rabinowitz, G\. Desjardins, H\. Soyer, J\. Kirkpatrick, K\. Kavukcuoglu, R\. Pascanu, and R\. Hadsell \(2016\)Progressive neural networks\.arXiv preprint\.External Links:1606\.04671Cited by:[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p2.1)\.
- P\. Ruvolo and E\. Eaton \(2013\)ELLA: an efficient lifelong learning algorithm\.InProceedings of the International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.28,pp\. 507–515\.Cited by:[§2\.2](https://arxiv.org/html/2606.26629#S2.SS2.p3.1)\.
- A\. Templeton, T\. Conerly, J\. Marcus, J\. Lindsey, T\. Bricken, B\. Chen, A\. Pearce, C\. Citro, E\. Ameisen, A\. Jones, H\. Cunningham, N\. L\. Turner, C\. McDougall, M\. MacDiarmid, A\. Tamkin, E\. Durmus, T\. Hume, F\. Mosconi, C\. D\. Freeman, T\. R\. Sumers, E\. Rees, J\. Batson, A\. Jermyn, S\. Carter, C\. Olah, and T\. Henighan \(2024\)Scaling monosemanticity: extracting interpretable features from Claude 3 Sonnet\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2024/scaling-monosemanticity/)Cited by:[§2\.3](https://arxiv.org/html/2606.26629#S2.SS3.p2.1)\.
- X\. Wang, T\. Chen, Q\. Ge, H\. Xia, R\. Bao, R\. Zheng, Q\. Zhang, T\. Gui, and X\. Huang \(2023a\)Orthogonal subspace learning for language model continual learning\.InProceedings of the Conference on Empirical Methods in Natural Language Processing \(EMNLP 2023\),pp\. 10658–10671\.Cited by:[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2606.26629#S4.SS1.SSS0.Px4.p1.2)\.
- X\. Wang, Y\. Zhang, T\. Chen, S\. Gao, S\. Jin, X\. Yang, Z\. Xi, R\. Zheng, Y\. Zou, T\. Gui, Q\. Zhang, and X\. Huang \(2023b\)TRACE: a comprehensive benchmark for continual learning in large language models\.arXiv preprint\.External Links:2310\.06762Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p1.1),[§2\.2](https://arxiv.org/html/2606.26629#S2.SS2.p2.1),[§4\.1](https://arxiv.org/html/2606.26629#S4.SS1.SSS0.Px1.p1.2)\.
- M\. Zeng, S\. Zhou, Z\. Zhan, and R\. Zhang \(2026\)MedCL\-Bench: benchmarking stability–efficiency trade\-offs and scaling in biomedical continual learning\.arXiv preprint\.External Links:2603\.16738Cited by:[§2\.2](https://arxiv.org/html/2606.26629#S2.SS2.p2.1)\.
- F\. Zenke, B\. Poole, and S\. Ganguli \(2017\)Continual learning through synaptic intelligence\.InProceedings of the International Conference on Machine Learning,pp\. 3987–3995\.Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p1.1),[§2\.1](https://arxiv.org/html/2606.26629#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2606.26629#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2606.26629#S4.SS1.SSS0.Px4.p1.2)\.
- M\. Zhang, Y\. Li, L\. Li, D\. Shen, H\. Xiong, and Y\. Sun \(2026\)SAE\-FD: sparse autoencoder feature distillation for continual learning of large language models\.arXiv preprint\.External Links:2605\.25525Cited by:[§1](https://arxiv.org/html/2606.26629#S1.p5.1)\.