Entangled Representations Amplify Collateral Damage in Unlearning

# Entangled Representations Amplify Collateral Damage in Unlearning
Source: [https://arxiv.org/html/2609.02285](https://arxiv.org/html/2609.02285)
Tim G\. J\. RudnerAffiliation:University of TorontoChristian Schroeder de WittAffiliation:University of Oxford

###### Abstract

A long\-held intuition in interpretability research is that representational entanglement—the sharing of structure between knowledge domains in a neural network—makes unlearning harder\. While the intuition is widespread, it has never been directly tested in a controlled experiment\. We present a way to do so: by repurposing Selective Gradient Masking \(SGTM\), we train a suite of six 254M\-parameter language models on English Wikipedia with graded levels of disentanglement between biology and non\-biology knowledge\. Applying three standard unlearning methods to every model in the suite, we find that more disentangled models consistently achieve better retain–forget trade\-offs: at a fixed level of forgetting, the most disentangled models incur roughly4×4\\timeslower retain cost under two of the three methods, and1\.3×1\.3\\timeslower under the third\. Because our intervention changes only the model, not the data or the unlearning algorithm, this is direct evidence that representational entanglement is one of the causes of collateral damage in unlearning, as interpretability researchers have long suspected\. A similar design could be used to test other structural claims from interpretability\.

Figure 1:The more disentangled the retain and forget domains are in a model, the better the retain–forget trade\-off achieved by unlearning\.Retain–forget Pareto frontiers of three unlearning methods \(WGA, WDR, RMU\) across six models that differ in the fraction of SGTM training steps \(pp\)\. Models are coloured by their Variance Entanglement Score \(defined in[AppendixC](https://arxiv.org/html/2609.02285#A3), shown in[Figure2](https://arxiv.org/html/2609.02285#S3.F2)\): purple means less disentangled, yellow more\. Each frontier is constructed over four to six hyperparameter configurations; shaded regions show±1\\pm 1standard error across five seeds\. Both axes are measured relative to each model’s own pre\-unlearning loss\. Analysed in[Section3\.2](https://arxiv.org/html/2609.02285#S3.SS2)\.## 1Introduction

Broadly speaking, interpretability research is useful to the extent that it predicts what happens when we intervene on a network\. Unlearning is a natural test case: interpretability has proposed two structural properties of neural networks that should, intuitively, affect how cleanly a targeted piece of knowledge can be unlearned\. The first is*localization*: where in the network’s parameters the knowledge resides\. The second is*entanglement*: the degree to which the retain and forget domains share structure in the network—representations, processing pathways, or parameters\. Both of these intuitions have taken a foothold in the unlearning literature\([Barez et al\., 2025](https://arxiv.org/html/2609.02285#bib.bib12)\), but only localization has so far been subjected to rigorous, controlled experimental testing\([Lee et al\., 2025](https://arxiv.org/html/2609.02285#bib.bib4);[Guo et al\., 2025](https://arxiv.org/html/2609.02285#bib.bib3);[Boglioni et al\., 2026](https://arxiv.org/html/2609.02285#bib.bib2)\)\. In this paper, we apply similar experimental methods to entanglement\.

According to the entanglement intuition, high entanglement should make the retain–forget trade\-off worse: if the knowledge to be forgotten and the knowledge to be retained share a lot of structure in the network, removing one without damaging the other should be harder\. However, this has never been tested directly\. Of the three main ingredients that go into unlearning—the model, the data, and the unlearning algorithm—current literature investigating entanglement has always held the model fixed, and tested the effect of entanglement only indirectly by varying the data composition\([Zhao et al\., 2024](https://arxiv.org/html/2609.02285#bib.bib9)\)or the unlearning algorithm\([Sondej and Yang, 2025](https://arxiv.org/html/2609.02285#bib.bib14);[Tang and Khanna, 2026](https://arxiv.org/html/2609.02285#bib.bib10);[Chen et al\., 2026](https://arxiv.org/html/2609.02285#bib.bib13)\), leaving model\-level entanglement confounded with other factors\. The perhaps most natural way to test whether entanglement affects unlearning, namely, holding the datasets and algorithm fixed and varying only the degree of entanglement in the model, has so far not been attempted—in part because it requires a way to control entanglement rather than merely observe it\.

Our main contribution is to fill this gap and do for entanglement what recent work has done for localization: build a controlled experiment that tests this intuition as directly as possible\. We train a suite of six 254M\-parameter language models on English Wikipedia\([Wikimedia, 2025b](https://arxiv.org/html/2609.02285#bib.bib16)\), increasingly disentangling biology knowledge from the rest using Selective Gradient Masking \(SGTM\), an improved variant of gradient routing\([Shilov et al\., 2025](https://arxiv.org/html/2609.02285#bib.bib1);[Cloud et al\., 2024](https://arxiv.org/html/2609.02285#bib.bib8)\), and verify with three metrics that the suite spans a graded range of disentanglement\. On each model in the suite, we measure the retain–forget trade\-off of three common unlearning methods: Weighted Gradient Ascent \(WGA\)\([Wang et al\., 2025](https://arxiv.org/html/2609.02285#bib.bib7)\), Weight Divergence Regularization \(WDR\)\([Siddiqui et al\., 2025](https://arxiv.org/html/2609.02285#bib.bib6)\), and RMU\([Li et al\., 2024](https://arxiv.org/html/2609.02285#bib.bib5)\)\.We find that more disentangled models consistently achieve better retain–forget trade\-offs \([Figure1](https://arxiv.org/html/2609.02285#S0.F1)\):at a fixed level of forgetting, the most disentangled models incur roughly4×4\\timeslower retain cost than the most entangled one under WGA and RMU, and1\.3×1\.3\\timeslower under WDR\.

We want to be precise about what this establishes\. Even in our experiments, we cannot change entanglement alone: we alter the training procedure, inducing changes to entanglement and, potentially, to other properties of the model\. Nevertheless, the design rules out the data and algorithm confounds of earlier work\. Taken together with the indirect evidence of earlier work, our results turn a long\-standing intuition from interpretability into an empirical finding\.

## 2Experimental Setup

### 2\.1Training Models with Varying Disentanglement

To train models in which the forget and retain domains are disentangled, we use a variant of gradient routing\([Cloud et al\., 2024](https://arxiv.org/html/2609.02285#bib.bib8)\)called Selective Gradient Masking \(SGTM\)\([Shilov et al\., 2025](https://arxiv.org/html/2609.02285#bib.bib1)\)\. Although SGTM is a training method originally designed to make a group of parameters specialize on processing inputs from a target domain, we find we can repurpose it for training disentangled models\. Namely, when SGTM is used to specialize a group of parameters on the forget domain, the forget domain also grows more disentangled from retain, as we show in[Section3\.1](https://arxiv.org/html/2609.02285#S3.SS1)\. We exploit this fact by varying the fraction of training steps in which we apply SGTM, producing models with different levels of disentanglement\.

#### Data\.

Following the original SGTM paper, we train on English Wikipedia\([Wikimedia, 2025b](https://arxiv.org/html/2609.02285#bib.bib16)\)\(approximately 3\.7B tokens\), using article\-level topic labels derived from Wikipedia’sarticletopicclassifier\([Wikimedia, 2025a](https://arxiv.org/html/2609.02285#bib.bib15)\), and splitting the data into three domains: \(1\)*forget*: all articles classified as STEM\.Biology \(∼3\.7%\{\\sim\}3\.7\\%of training tokens\); \(2\)*adjacent*: articles on topics closely related to biology \(Medicine & Health, Chemistry, Earth & Environment\); \(3\)*retain*: all remaining articles, spanning Culture, Geography, History & Society, and the STEM topics unrelated to biology\.

Each example is a 1,024\-token block of article text\. We hold out separate test sets for each domain\. To reduce evaluation cost during our unlearning hyperparameter sweeps, we subsample20%20\\%of each test set and report all unlearning metrics on this fixed subsample\.

#### Models\.

Following the original paper, all models we train are 254M\-parameter GPT\-Neo\-style transformers with 16 layers, hidden dimension 1,024, 32 attention heads, and MLP dimension 4,096; the full configuration is given in[AppendixA](https://arxiv.org/html/2609.02285#A1)\. In each transformer block, we designate a small subset of parameters as the*biology subnetwork*,θbio\\theta\_\{\\text\{bio\}\}: 1 attention head \(out of 32\) and 64 MLP hidden units \(out of 4,096\) per block\. We denote all remaining parameters byθother\\theta\_\{\\text\{other\}\}\.

#### SGTM\.

As in the original paper, we assign training examples to one of three routing categories: \(1\)*route\-bio*: all forget \(i\.e\., all biology\) examples; \(2\)*route\-other*: a randomly sampled10%10\\%of retain and adjacent examples; \(3\)*route\-unchanged*: the remaining90%90\\%of retain and adjacent examples\.

Depending on an example’s routing category, SGTM modifies the training procedure during both the forward and backward passes\. For route\-bio examples, gradients forθother\\theta\_\{\\text\{other\}\}are zeroed after the backward pass, ensuring that biology knowledge flows only intoθbio\\theta\_\{\\text\{bio\}\}\.111Unlike in the original paper, we do*not*ablateθbio\\theta\_\{\\text\{bio\}\}after training\. We use SGTM solely for the purpose of controlling disentanglement, not for unlearning itself\.For route\-other examples,θbio\\theta\_\{\\text\{bio\}\}is zeroed during the forward pass, training the model to perform well on non\-target data even without the biology subnetwork\. Finally, route\-unchanged examples update all parameters via standard training\. See[Table1](https://arxiv.org/html/2609.02285#S2.T1)for an overview\.

Table 1:Training interventions in SGTM, applied to each example based on its routing category\.†θbio\\theta\_\{\\text\{bio\}\}receives no gradient because its activations are zeroed during the forward pass\.
#### Varying the fraction of SGTM steps\.

In the original paper, SGTM was used in all training steps; we instead activate SGTM for only a fraction of the training steps,p%p\\%\. We train six models withp∈\{0,20,40,60,80,100\}p\\in\\\{0,20,40,60,80,100\\\}; the first\(100−p\)%\(100\{\-\}p\)\\%of steps use standard training, and the remainingp%p\\%use SGTM as described in the previous paragraph\.222Applying SGTM stochastically—routing each example with some probability rather than front\-loading standard training and then switching to full SGTM—would produce substantial disentanglement even at lowpp, due to the*absorption effect*described by[Cloud et al\. \(2024\)](https://arxiv.org/html/2609.02285#bib.bib8)\.

### 2\.2Measuring Disentanglement between Domains

We measure disentanglement separately between all pairs of the three domains defined in[Section2\.1](https://arxiv.org/html/2609.02285#S2.SS1): forget–retain, forget–adjacent, and, as a control, retain–adjacent\. Given a pair of domains, we sample 1,024 examples from their two test sets, then for each example we take the model’s final\-layer hidden states, average them over non\-padding positions, and normalize the result to unit norm, yielding one vector per example\. We then compare the resulting two point clouds using three standard metrics: the Variance Entanglement Score \(VES\)\([Zhao et al\., 2024](https://arxiv.org/html/2609.02285#bib.bib9)\), maximum mean discrepancy \(MMD2\)\([Gretton et al\., 2012](https://arxiv.org/html/2609.02285#bib.bib11)\), and the sliced 2\-Wasserstein distance \(SW22\{\}\_\{2\}^\{2\}\)\([Bonneel et al\., 2015](https://arxiv.org/html/2609.02285#bib.bib17)\)\. Lower VES and higher MMD2and SW22\{\}\_\{2\}^\{2\}indicate more disentanglement; definitions and settings are given in[AppendixC](https://arxiv.org/html/2609.02285#A3)\.

The three metrics are not on a common scale, and their absolute values are not comparable across domain pairs\. We therefore report, for each metric and each pair, the log ratio of the measured value to that of thep=0%p\{=\}0\\%model, so that every curve starts at zero and the quantity plotted is the relative change in separation induced by SGTM\.

### 2\.3Measuring the Retain–Forget Trade\-off during Unlearning

We apply three standard unlearning algorithms to each of the six models, using the forget and retain training data defined in[Section2\.1](https://arxiv.org/html/2609.02285#S2.SS1)\. Adjacent data is never used for optimization, only for evaluation\. We do not give the algorithms any information about how the models were trained\.

Our interest is not in comparing methods against each other, but in checking whether,*within each method*, more disentangled models yield better retain–forget trade\-offs\. Since the six models differ in their losses before unlearning, we measure the*change*in test loss relative to each model’s own pre\-unlearning value,Δ​ℓforget\\Delta\\ell\_\{\\text\{forget\}\}andΔ​ℓretain\\Delta\\ell\_\{\\text\{retain\}\}\. For each method, we sweep over four to six hyperparameter configurations \(see[AppendixB](https://arxiv.org/html/2609.02285#A2)for the full list\), and use the results to construct a retain\-\-forget Pareto frontier for every model\-\-method pair\. We run each configuration on five seeds for a method\-specific number of optimizer steps, with early stopping when the forget loss exceeds that of a model trained with the biology data filtered out, plus a small buffer\.333Beyond this point, further increases in forget loss reflect degradation of the model rather than removal of biology knowledge\.

#### Unlearning methods\.

LetCE⁡\(θ,𝒟\)\\operatorname\{CE\}\(\\theta;\\,\\mathcal\{D\}\)denote the mean per\-token cross\-entropy of modelθ\\thetaon dataset𝒟\\mathcal\{D\}, and letθ0\\theta\_\{0\}denote the parameters before unlearning\.

- ∙\\bulletWeighted Gradient Ascent \(WGA\)\([Wang et al\., 2025](https://arxiv.org/html/2609.02285#bib.bib7)\)combines gradient ascent on the forget set with gradient descent on the retain set, minimizing ℒWGA​\(θ\)=CE⁡\(θ;retain\)−λWGA​CEWGA⁡\(θ;forget\),λWGA\>0,\\mathcal\{L\}\_\{\\text\{WGA\}\}\(\\theta\)=\\operatorname\{CE\}\(\\theta;\\,\\text\{retain\}\)\-\\lambda\_\{\\text\{WGA\}\}\\,\\operatorname\{CE\}\_\{\\text\{WGA\}\}\(\\theta;\\,\\text\{forget\}\),\\quad\\lambda\_\{\\text\{WGA\}\}\>0,whereCEWGA\\operatorname\{CE\}\_\{\\text\{WGA\}\}is a reweighted cross\-entropy that scales each token’s log\-probability loss by\[pθ​\(xt∣x<t\)\]α\[p\_\{\\theta\}\(x\_\{t\}\\mid x\_\{<t\}\)\]^\{\\alpha\}, upweighting tokens the model predicts confidently so that gradient ascent targets the most well\-learned predictions first\. Each unlearning step draws one batch from the forget set and one from the retain set\.
- ∙\\bulletWeight Divergence Regularization \(WDR\)\([Siddiqui et al\., 2025](https://arxiv.org/html/2609.02285#bib.bib6)\)fine\-tunes on the retain set while maximizing theL2L\_\{2\}distance between the current parameters andθ0\\theta\_\{0\}: ℒWDR​\(θ\)=CE⁡\(θ;retain\)−λWDR​1\|θ\|​∑i\(θi−θ0,i\)2,λWDR\>0\.\\mathcal\{L\}\_\{\\text\{WDR\}\}\(\\theta\)=\\operatorname\{CE\}\(\\theta;\\,\\text\{retain\}\)\-\\lambda\_\{\\text\{WDR\}\}\\,\\sqrt\{\\frac\{1\}\{\|\\theta\|\}\\sum\_\{i\}\(\\theta\_\{i\}\-\\theta\_\{0,i\}\)^\{2\}\},\\quad\\lambda\_\{\\text\{WDR\}\}\>0\.
- ∙\\bulletRepresentation Misdirection for Unlearning \(RMU\)\([Li et al\., 2024](https://arxiv.org/html/2609.02285#bib.bib5)\)modifies intermediate representations so that, on forget examples, activations at a designated layer are steered towards random target directions, while on retain examples, the same activations are anchored to their values underθ0\\theta\_\{0\}\. The loss is defined at a single layer, but the update is applied to the MLP weights of that layer and the two preceding it\.

## 3Results

### 3\.1Models Trained with More SGTM Are More Disentangled

We first verify that varying the fraction of SGTM training steps does produce models with meaningfully different levels of disentanglement\. All three metrics we use \(see[Section2\.2](https://arxiv.org/html/2609.02285#S2.SS2)\) agree that it does: asppgrows, the forget domain separates from both retain and adjacent \([Figure2](https://arxiv.org/html/2609.02285#S3.F2)\)\. The largest change happens betweenp=40%p\{=\}40\\%andp=60%p\{=\}60\\%, and the models can be grouped into a less disentangled cluster \(p≤40%p\\leq 40\\%\) and a more disentangled one \(p≥60%p\\geq 60\\%\)\.

As a form of control, we also measure the disentanglement between retain and adjacent\. Recall that these two domains were grouped together during SGTM training \([Section2\.1](https://arxiv.org/html/2609.02285#S2.SS1)\), so we would expect their disentanglement to stay roughly constant across the model suite\. Instead, we observe that their disentanglement does grow withpp, but roughly three to four times less than in forget–retain\. We suspect this reflects the absorption effect\([Cloud et al\., 2024](https://arxiv.org/html/2609.02285#bib.bib8)\): adjacent articles contain biology content that was never labelled as such, so that SGTM reshapes adjacent representations more than retain ones, and the two slightly drift apart\. Consistent with this, forget–adjacent disentangles about two\-thirds as much as forget–retain\.

Figure 2:The forget domain grows more disentangled from both retain and adjacent as the fraction of SGTM steps during training \(pp\) increases, with the largest change occurring betweenp=40%p\{=\}40\\%andp=60%p\{=\}60\\%\. The disentanglement of the retain–adjacent control also grows, but three to four times less\. Each curve shows the log ratio of the measured value to that of thep=0%p\{=\}0\\%model, so all curves start at zero by construction, and the arrow on each axis gives the direction of increasing disentanglement\. The three metrics are described in[AppendixC](https://arxiv.org/html/2609.02285#A3)\.
### 3\.2More Disentangled Models Achieve Better Retain–Forget Trade\-offs

More disentangled models consistently achieve better retain–forget trade\-offs, and this holds for all three unlearning methods \([Figure1](https://arxiv.org/html/2609.02285#S0.F1)\)\. The size of the effect differs substantially between methods, but its direction does not\.

#### Retain–forget results\.

All comparisons below are made atΔ​ℓforget=0\.4\\Delta\\ell\_\{\\text\{forget\}\}=0\.4\.

- ∙\\bulletWGA:The effect is clearest here\. The most disentangled models \(p≥60%p\\geq 60\\%\) incur roughly4×4\\timeslower retain cost than the most entangled \(p=0%p\{=\}0\\%\) model, which has the highest retain cost at every level of forgetting\. The frontiers also separate into the same two clusters we found in[Section3\.1](https://arxiv.org/html/2609.02285#S3.SS1): the less disentangled models \(p≤40%p\\leq 40\\%\) form one, and the more disentangled ones \(p≥60%p\\geq 60\\%\) the other\.
- ∙\\bulletWDR:The trend is in the same direction but weaker\. The most disentangled models incur roughly1\.3×1\.3\\timeslower retain cost thanp=0%p\{=\}0\\%, but the frontiers sit close together and their standard errors overlap\.
- ∙\\bulletRMU:The retain\-loss scale is two orders of magnitude smaller than for WDR, reflecting RMU’s minimal impact on retain performance overall\. However, the effect persists even at this scale: the most disentangled models incur roughly4×4\\timeslower retain cost thanp=0%p\{=\}0\\%\. Again, the trend holds, except forp=20%p\{=\}20\\%, which performs comparably to the disentangled cluster\.

#### Adjacent–forget results\.

The adjacent–forget trade\-off is the hardest case in our setup\. First, being the domain closest to forget, adjacent receives greater spillover damage from unlearning\. Second, the unlearning methods optimize on forget and retain data only, so adjacent knowledge is left undefended\. Even so, we observe the same ordering on adjacent as on retain for WGA and WDR; for RMU the results are less clear\. We report these results with additional discussion in[AppendixD](https://arxiv.org/html/2609.02285#A4)\.

## 4Conclusion

Interpretability has lent two intuitions to the field of unlearning: that the difficulty of unlearning depends on where the forget knowledge is stored \(localization\) and on how much structure it shares with retained knowledge \(entanglement\)\. While the first has been rigorously tested in a controlled experimental setting, the second had not\. In this paper, we fill this gap: holding the data and the unlearning algorithms fixed and varying only the model, we find that more disentangled models consistently achieve better retain–forget trade\-offs across three unlearning algorithms\. Analogously to the existing work on localization, our work helps turn another long\-standing intuition from interpretability into an empirical finding, and the recipe—repurpose an off\-the\-shelf method to control a representational property during training, verify the control, and test its predicted consequence—could be applied to other structural claims from interpretability as well\.

## References

- Barezet al\.\(2025\)F\. Barez, T\. Fu, A\. Prabhu, S\. Casper, A\. Sanyal, A\. Bibi, A\. O’Gara, R\. Kirk, B\. Bucknall, T\. Fist, L\. Ong, P\. Torr, K\. Lam, R\. Trager, D\. Krueger, S\. Mindermann, J\. Hernandez\-Orallo, M\. Geva, and Y\. GalOpen problems in machine unlearning for ai safety\.External Links:2501\.04952,[Link](https://arxiv.org/abs/2501.04952)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p1.1)\.
- Boglioniet al\.\(2026\)M\. Boglioni, T\. Rousset, S\. Reddy, M\. Mosbach, and V\. DankersLACUNA: a testbed for evaluating localization precision for LLM unlearning\.InThird Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=BFWTLrAJNZ)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p1.1)\.
- Bonneelet al\.\(2015\)N\. Bonneel, J\. Rabin, G\. Peyré, and H\. PfisterSliced and radon wasserstein barycenters of measures\.Journal of Mathematical Imaging and Vision51\(1\),pp\. 22–45\.External Links:ISSN 1573\-7683,[Document](https://dx.doi.org/10.1007/s10851-014-0506-3),[Link](https://doi.org/10.1007/s10851-014-0506-3)Cited by:[3rd item](https://arxiv.org/html/2609.02285#A3.I1.i3.p1.1),[§2\.2](https://arxiv.org/html/2609.02285#S2.SS2.p1.1)\.
- Chenet al\.\(2026\)H\. Chen, J\. Zhu, X\. Yang, and W\. WangCLUE: conflict\-guided localization for LLM unlearning framework\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=jtRYvazBWv)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p2.1)\.
- Cloudet al\.\(2024\)A\. Cloud, J\. Goldman\-Wetzler, E\. Wybitul, J\. Miller, and A\. M\. TurnerGradient routing: masking gradients to localize computation in neural networks\.External Links:2410\.04332,[Link](https://arxiv.org/abs/2410.04332)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.02285#S2.SS1.p1.1),[§3\.1](https://arxiv.org/html/2609.02285#S3.SS1.p2.1),[footnote 2](https://arxiv.org/html/2609.02285#footnote2)\.
- Grettonet al\.\(2012\)A\. Gretton, K\. M\. Borgwardt, M\. J\. Rasch, B\. Schölkopf, and A\. SmolaA kernel two\-sample test\.Journal of Machine Learning Research13\(25\),pp\. 723–773\.External Links:[Link](http://jmlr.org/papers/v13/gretton12a.html)Cited by:[2nd item](https://arxiv.org/html/2609.02285#A3.I1.i2.p1.1),[§2\.2](https://arxiv.org/html/2609.02285#S2.SS2.p1.1)\.
- Guoet al\.\(2025\)P\. Guo, A\. Syed, A\. Sheshadri, A\. Ewart, and G\. K\. DziugaiteMechanistic unlearning: robust knowledge unlearning and editing via mechanistic localization\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267\.External Links:[Link](https://proceedings.mlr.press/v267/guo25k.html)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p1.1)\.
- Leeet al\.\(2025\)H\. Lee, U\. Hwang, H\. Lim, and T\. KimDoes localization inform unlearning? A rigorous examination of local parameter attribution for knowledge unlearning in language models\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, EMNLP 2025, Suzhou, China, November 4\-9, 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),pp\. 21857–21869\.External Links:[Document](https://dx.doi.org/10.18653/V1/2025.EMNLP-MAIN.1109),[Link](https://doi.org/10.18653/v1/2025.emnlp-main.1109)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p1.1)\.
- Liet al\.\(2024\)N\. Li, A\. Pan, A\. Gopal, S\. Yue, D\. Berrios, A\. Gatti, J\. D\. Li, A\. Dombrowski, S\. Goel, G\. Mukobi, N\. Helm\-Burger, R\. Lababidi, L\. Justen, A\. B\. Liu, M\. Chen, I\. Barrass, O\. Zhang, X\. Zhu, R\. Tamirisa, B\. Bharathi, A\. Herbert\-Voss, C\. B\. Breuer, A\. Zou, M\. Mazeika, Z\. Wang, P\. Oswal, W\. Lin, A\. A\. Hunt, J\. Tienken\-Harder, K\. Y\. Shih, K\. Talley, J\. Guan, I\. Steneker, D\. Campbell, B\. Jokubaitis, S\. Basart, S\. Fitz, P\. Kumaraguru, K\. K\. Karmakar, U\. Tupakula, V\. Varadharajan, Y\. Shoshitaishvili, J\. Ba, K\. M\. Esvelt, A\. Wang, and D\. HendrycksThe WMDP benchmark: measuring and reducing malicious use with unlearning\.InForty\-first International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=xlr6AUDuJz)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p3.1),[3rd item](https://arxiv.org/html/2609.02285#S2.I1.i3.p1.1)\.
- Shilovet al\.\(2025\)I\. Shilov, A\. Cloud, A\. P\. Gema, J\. Goldman\-Wetzler, N\. Panickssery, H\. Sleight, E\. Jones, and C\. AnilBeyond data filtering: knowledge localization for capability removal in llms\.External Links:2512\.05648,[Link](https://arxiv.org/abs/2512.05648)Cited by:[Table 2](https://arxiv.org/html/2609.02285#A1.T2),[§1](https://arxiv.org/html/2609.02285#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.02285#S2.SS1.p1.1)\.
- Siddiquiet al\.\(2025\)S\. A\. Siddiqui, A\. Weller, D\. Krueger, G\. K\. Dziugaite, M\. C\. Mozer, and E\. TriantafillouFrom dormant to deleted: tamper\-resistant unlearning through weight\-space regularization\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2\-7, 2025 / Mexico City, Mexico, November 30 \- December 5, 2025,D\. Belgrave, C\. Zhang, L\. N\. Montoya, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, N\. Chen, I\. V\. M\. Ruíz, and A\. Loaiza\-Bonilla \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2025/hash/bbeafbaa60ee03f2c3135be71ebd5d06-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p3.1),[2nd item](https://arxiv.org/html/2609.02285#S2.I1.i2.p1.1)\.
- Sondej and Yang \(2025\)F\. Sondej and Y\. YangCollapse of irrelevant representations \(cir\) ensures robust and non\-disruptive llm unlearning\.External Links:2509\.11816,[Link](https://arxiv.org/abs/2509.11816)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p2.1)\.
- Tang and Khanna \(2026\)H\. Tang and R\. KhannaFrom logits to latents: contrastive representation shaping for llm unlearning\.External Links:2601\.22028,[Link](https://arxiv.org/abs/2601.22028)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p2.1)\.
- Wanget al\.\(2025\)Q\. Wang, J\. P\. Zhou, Z\. Zhou, S\. Shin, B\. Han, and K\. Q\. WeinbergerRethinking LLM unlearning objectives: A gradient perspective and go beyond\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=huo8MqVH6t)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p3.1),[1st item](https://arxiv.org/html/2609.02285#S2.I1.i1.p1.1)\.
- Wikimedia \(2025a\)WikimediaORES / articletopic\.Note:[https://www\.mediawiki\.org/wiki/ORES/Articletopic](https://www.mediawiki.org/wiki/ORES/Articletopic)Accessed: 2025\-09\-21Cited by:[§2\.1](https://arxiv.org/html/2609.02285#S2.SS1.SSS0.Px1.p1.1)\.
- Wikimedia \(2025b\)WikimediaWikimedia downloads\.External Links:[Link](https://dumps.wikimedia.org/)Cited by:[§1](https://arxiv.org/html/2609.02285#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.02285#S2.SS1.SSS0.Px1.p1.1)\.
- Zhaoet al\.\(2024\)K\. Zhao, M\. Kurmanji, G\. Barbulescu, E\. Triantafillou, and P\. TriantafillouWhat makes unlearning hard and what to do about it\.InAdvances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 \- 15, 2024,A\. Globersons, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. M\. Tomczak, and C\. Zhang \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2024/hash/16e18fa3b3add076c30f2a2598f03031-Abstract-Conference.html)Cited by:[1st item](https://arxiv.org/html/2609.02285#A3.I1.i1.p1.1),[§1](https://arxiv.org/html/2609.02285#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.02285#S2.SS2.p1.1)\.

## Appendix ATraining Hyperparameters

See[Table2](https://arxiv.org/html/2609.02285#A1.T2)for the full training hyperparameters\.

Table 2:Training hyperparameters, following[Shilov et al\. \(2025\)](https://arxiv.org/html/2609.02285#bib.bib1)\.
## Appendix BUnlearning Hyperparameters

All unlearning methods use the AdamW optimizer with no weight decay\. Each configuration is run with five seeds \(42–46\)\. Full configurations are given in[Tables3](https://arxiv.org/html/2609.02285#A2.T3),[4](https://arxiv.org/html/2609.02285#A2.T4)and[5](https://arxiv.org/html/2609.02285#A2.T5)\.

Table 3:WGA hyperparameters \(six configurations\)\.Table 4:WDR hyperparameters \(six configurations\)\.Table 5:RMU hyperparameters \(four configurations\)\.
## Appendix CDisentanglement Metrics

We compare the two point clouds described in[Section2\.2](https://arxiv.org/html/2609.02285#S2.SS2)using the following three metrics\.

- ∙\\bulletVariance Entanglement Score \(VES\)\([Zhao et al\., 2024](https://arxiv.org/html/2609.02285#bib.bib9)\)compares within\-group spread to between\-group separation\. Lower VES indicates more disentanglement\.
- ∙\\bulletMaximum Mean Discrepancy \(MMD2\)\([Gretton et al\., 2012](https://arxiv.org/html/2609.02285#bib.bib11)\)is a standard nonparametric measure of the difference between two distributions\. It maps both clouds into a reproducing kernel Hilbert space and measures the squared distance between their mean embeddings\. We use the unbiased estimator and a sum of five Gaussian kernels with bandwidthsσ∈0\.82⋅\{0\.5,0\.71,1,1\.41,2\}\\sigma\\in 0\.82\\cdot\\\{0\.5,\\,0\.71,\\,1,\\,1\.41,\\,2\\\}\. Higher MMD2indicates more disentanglement\.
- ∙\\bulletSliced 2\-Wasserstein \(SW22\{\}\_\{2\}^\{2\}\)\([Bonneel et al\., 2015](https://arxiv.org/html/2609.02285#bib.bib17)\)is an optimal\-transport distance between distributions, and the standard alternative to the 2\-Wasserstein distance in high dimensions, where the latter is difficult to estimate reliably\. It averages the squared 2\-Wasserstein distance between projections of the two clouds onto random directions\. We use 512 fixed random projections\. Higher SW22\{\}\_\{2\}^\{2\}indicates more disentanglement\.

## Appendix DUnlearning Trade\-offs on Adjacent Knowledge

[Figure3](https://arxiv.org/html/2609.02285#A4.F3)shows the Pareto frontiers in the\(Δ​ℓadjacent,Δ​ℓforget\)\(\\Delta\\ell\_\{\\text\{adjacent\}\},\\,\\Delta\\ell\_\{\\text\{forget\}\}\)plane, constructed independently of the retain–forget frontiers of[Figure1](https://arxiv.org/html/2609.02285#S0.F1)\. As in[Section3\.2](https://arxiv.org/html/2609.02285#S3.SS2), all comparisons are made atΔ​ℓforget=0\.4\\Delta\\ell\_\{\\text\{forget\}\}=0\.4\.

- ∙\\bulletWGA:The ordering from[Section3\.2](https://arxiv.org/html/2609.02285#S3.SS2)is reproduced, including the separation into two clusters\. The most entangled model \(p=0%p\{=\}0\\%\) incurs roughly4×4\\timesthe adjacent cost of the disentangled cluster \(p≥60%p\\geq 60\\%\)\.
- ∙\\bulletWDR:The ordering is also reproduced, although, as in the retain–forget results, the frontiers are close together and their standard errors overlap\. The means are ordered correctly across the suite\.
- ∙\\bulletRMU:The frontiers largely overlap and we find no clear ordering\. We do not have a conclusive explanation, but note that RMU’s costs are by far the smallest of the three methods\. We therefore take our results to mean that we cannot resolve an ordering at this scale of adjacent loss, rather than that there is none\.

Figure 3:Adjacent–forget Pareto frontiers, reproducing the ordering of[Figure1](https://arxiv.org/html/2609.02285#S0.F1)for WGA and WDR but not for RMU\.No unlearning method optimizes on adjacent data\. Conventions are otherwise as in[Figure1](https://arxiv.org/html/2609.02285#S0.F1)\.