Between Suppression and Collapse: Evaluating Narrative Unlearning with LENS

arXiv cs.CL Papers

Summary

Introduces LENS, a contextualization-based evaluation protocol for testing narrative unlearning in large language models, evaluating suppression across direct, attributed, contrastive, and abstract levels.

arXiv:2607.22657v1 Announce Type: new Abstract: Large language models (LLMs) can reproduce disinformation-aligned narrative frames as plausible explanations, raising the question of whether existing machine-unlearning algorithms can suppress this behavior. We introduce Level-based Evaluation of Narrative Suppression (LENS), a contextualization based evaluation protocol for testing target narrative reproduction across direct, attributed, contrastive, and abstract resistance levels. We evaluate two source-grounded narratives: one framing Russia's war against Ukraine as forced by NATO expansion, and one framing the United States as exploiting or abandoning Taiwan. The experiments cover four near-12B multilingual instruction models: Lapa LLM, Gemma-12B, Qwen-14B, and TAIDE-Gemma. We introduce the Suppression-Collapse Efficiency (SCE) score as a checkpoint selection summary that rewards target-narrative suppression while penalizing degraded outputs. Our results shows that selected checkpoints can reduce narrative reproduction and suppression may transfer beyond direct forget prompts. We also report entity recovery as a separate side effect: abstract A/B/C prompts can cause models to recover the real-world actors associated with the target frame after unlearning. These findings demonstrate that LENS is a successful diagnostic protocol for both reporting and guiding the further study of the deeper structure of narrative unlearning.
Original Article
View Cached Full Text

Cached at: 07/28/26, 06:27 AM

# Between Suppression and Collapse: Evaluating Narrative Unlearning with LENS
Source: [https://arxiv.org/html/2607.22657](https://arxiv.org/html/2607.22657)
11institutetext:Ukrainian Catholic University, Lviv, Ukraine,11email:makovska\.pn@ucu\.edu\.ua22institutetext:Eindhoven University of Technology, Eindhoven, The Netherlands###### Abstract

Large language models \(LLMs\) can reproduce disinformation\-aligned narrative frames as plausible explanations, raising the question of whether existing machine\-unlearning algorithms can suppress this behavior\. We introduce*Level\-based Evaluation of Narrative Suppression*\(LENS\), a contextualization based evaluation protocol for testing target narrative reproduction across direct, attributed, contrastive, and abstract resistance levels\. We evaluate two source\-grounded narratives: one framing Russia’s war against Ukraine as forced by NATO expansion, and one framing the United States as exploiting or abandoning Taiwan\. The experiments cover four near\-12B multilingual instruction models: Lapa LLM, Gemma\-12B, Qwen\-14B, and TAIDE\-Gemma\.

We introduce the Suppression\-Collapse Efficiency \(SCE\) score as a checkpoint selection summary that rewards target\-narrative suppression while penalizing degraded outputs\. Our results shows that selected checkpoints can reduce narrative reproduction and suppression may transfer beyond direct forget prompts\. We also report entity recovery as a separate side effect: abstract A/B/C prompts can cause models to recover the real\-world actors associated with the target frame after unlearning\. These findings demonstrate thatLENSis a successful diagnostic protocol for both reporting and guiding the further study of the deeper structure of narrative unlearning\.

## 1Introduction

L0Directexplicit target claimL1Attributedreported or third\-party frameL2Contrastivetarget vs\. alternative frameL3Abstract/maskedA/B/C actor structureincreasingelicitationdistanceFigure 1:LENSresistance levels\.Large language models \(LLMs\) reproduce more than isolated facts\. They also reproduce recurring explanatory frames: patterns of causal attribution, actor responsibility, and rhetorical structure that make a contested claim appear coherent\. At the same time growing threats of LLM poisoning\[[24](https://arxiv.org/html/2607.22657#bib.bib3)\]or LLM grooming\[[3](https://arxiv.org/html/2607.22657#bib.bib2)\]introduce new ways of LLM attacks, including disinformation spreading\[[23](https://arxiv.org/html/2607.22657#bib.bib4)\]\. Such frames can be viewed as narratives with recurring ideological structure and specifically defined worldviews\[[8](https://arxiv.org/html/2607.22657#bib.bib1)\]\. We study narrative reproduction risk, measuring the model behavior without attributing it to a training\-data source\.

Machine unlearning is a natural candidate mitigation for this contamination behavior: if a model repeatedly produces completions that instantiate a target narrative, those completions can be collected as forget examples and used to push the model away from the behavior, without retraining from scratch\[[29](https://arxiv.org/html/2607.22657#bib.bib5),[19](https://arxiv.org/html/2607.22657#bib.bib6),[15](https://arxiv.org/html/2607.22657#bib.bib7)\]\. Narrative unlearning requires a stronger evaluation than deletion of a memorized string\. A method can suppress a direct prompt while leaving a deeper frame intact under paraphrase, attribution, or abstraction\. A method can also appear successful by refusing to answer; under our research questions, such refusal is treated as a failure mode rather than clean unlearning\.

We ask two questions:RQ1:When do existing unlearning methods suppress narrative reproduction rather than causing refusal or collapse?RQ2:After suppression, what residual behavior remains under indirect prompts, including alternative explanations, frame associations, or real\-actor recovery?

Existing unlearning benchmarks mostly evaluate removal of fact\-like or capability\-like knowledge\[[19](https://arxiv.org/html/2607.22657#bib.bib6),[15](https://arxiv.org/html/2607.22657#bib.bib7)\]\. While prior evaluations include paraphrased or perturbed forms, our contributions target a different failure mode: frames that can survive recontextualization through attribution, comparison, or abstraction\. We argue that narrative unlearning therefore requires testing transfer beyond rewording\.

To address this gap in the study of narrative unlearning, we introduce*Level\-based Evaluation of Narrative Suppression*\(LENS\), a resistance\-level protocol for narrative unlearning\.LENSevaluates a target narrative at four levels: direct elicitation \(L0\), attributed or reported explanation \(L1\), contrastive framing \(L2\), and masked or abstract framing \(L3\)\. We also add a separate L3\-special side\-effect probe for entity recovery, where the model receives a frame prompt with entity names substituted by generic A/B/C placeholders and may recover the real\-world actors\. Fig\.[1](https://arxiv.org/html/2607.22657#S1.F1)visualizes the increasing elicitation distance from the target narrative\.

We study two source\-grounded narratives\. In NATO\-forced\-war, NATO expansion is presented as the cause of Russia’s war against Ukraine, reducing Russia’s responsibility\[[26](https://arxiv.org/html/2607.22657#bib.bib8),[4](https://arxiv.org/html/2607.22657#bib.bib9)\]\. In US\-abandons\-Taiwan, the United States is presented as using Taiwan for its own strategic interests and potentially abandoning it\[[16](https://arxiv.org/html/2607.22657#bib.bib10),[18](https://arxiv.org/html/2607.22657#bib.bib11)\]\. We use fact\-checking and monitoring sources to define these narrative signatures and turn them into neutral prompt families\.

We define narrative unlearning as a trajectory\-selection problem\. The results show that narrative suppression is not well represented by the last training checkpoint\. Early checkpoints often leave the target frame largely unchanged, while later checkpoints can reduce reproduction mainly by producing degraded, repetitive, or empty answers\. Useful checkpoints appear in the middle of the trajectory, where target\-frame reproduction has dropped but generation health remains acceptable\.

This paper makes three contributions\.1\.We defineLENS, a context based multi\-level evaluation protocol for narrative unlearning that tests whether suppression transfers beyond direct forget prompts\.2\.We introduce Suppression\-Collapse Efficiency score \(SCE\), a selection summary that rewards suppression while penalizing degraded outputs\.3\.We show, in a constrained red\-team setting, that selected checkpoints of existing unlearning objectives can reduce narrative reproduction using only model\-generated reproduced answers as negative forget examples, without explicit counter\-knowledge, neutral replacement answers, or refusal targets\.

## 2Related Work

### Narrative reproduction and contamination risk\.

Disinformation often circulates as recurring narrative frames\. A frame specifies actors, causal direction, responsibility, and the interpretation that makes a contested event appear coherent\[[8](https://arxiv.org/html/2607.22657#bib.bib1)\]\. In LLMs, such frames may be reproduced when similar patterns are repeatedly present in training data, retrieval contexts, or instruction\-tuning data\. We do not claim to identify a specific poisoned training document\. Instead, we study a poisoning\-like or grooming\-relevant behavioral risk: repeated exposure and web\-scale distribution can make disinformation\-aligned frames available to the model\[[24](https://arxiv.org/html/2607.22657#bib.bib3),[3](https://arxiv.org/html/2607.22657#bib.bib2),[23](https://arxiv.org/html/2607.22657#bib.bib4)\]\.

### LLM unlearning\.

Machine unlearning aims to reduce the influence of selected data or behaviors without retraining a model from scratch\. Recent LLM unlearning work has studied gradient\-ascent objectives, retain\-preserving objectives, fictitious\-author forgetting, hazardous\-knowledge removal, and safety\-oriented deletion\[[29](https://arxiv.org/html/2607.22657#bib.bib5),[19](https://arxiv.org/html/2607.22657#bib.bib6),[15](https://arxiv.org/html/2607.22657#bib.bib7)\]\. These settings establish the standard forget/retain tradeoff: the target behavior should decrease, while unrelated retain behavior should remain usable\. Narrative unlearning keeps this tradeoff but changes the target\. Instead of removing a memorized fact or bounded knowledge set, it tries to reduce a recurring explanatory frame that may survive paraphrase, attribution, comparison, or abstraction\.

### Objectives and parameter\-efficient adapters\.

Our experiments use existing unlearning objectives\. We evaluate gradient ascent with retain preservation\[[29](https://arxiv.org/html/2607.22657#bib.bib5)\], TOFU\-style gradient\-ascent and gradient\-difference baselines\[[19](https://arxiv.org/html/2607.22657#bib.bib6)\], and negative preference optimization\[[31](https://arxiv.org/html/2607.22657#bib.bib12)\]\. Because the study requires many trajectories across models, languages, narratives, and checkpoints, we implement interventions with parameter\-efficient adapters\. LoRA freezes the base model and learns low\-rank adapter weights\[[10](https://arxiv.org/html/2607.22657#bib.bib13)\]; AdaLoRA adapts rank allocation during training\[[30](https://arxiv.org/html/2607.22657#bib.bib14)\]\.

### Benchmarks, metrics, and recontextualized prompts\.

Existing LLM unlearning benchmarks motivate separate forget and retain measurements\. TOFU evaluates fictitious\-author forgetting in a controlled setting\[[19](https://arxiv.org/html/2607.22657#bib.bib6)\], while WMDP evaluates removal of hazardous knowledge\[[15](https://arxiv.org/html/2607.22657#bib.bib7)\]\. Some prior evaluations include paraphrased or perturbed forms, but narrative unlearning requires a different kind of transfer test: whether the same causal frame survives when the prompt is recontextualized through attribution, comparison, or abstraction\. Related work on knowledge holes probes whether unlearning removes benign adjacent knowledge\[[11](https://arxiv.org/html/2607.22657#bib.bib19)\]\.LENSasks the complementary question: whether directly targeted unlearning leaves related narrative structure available under indirect prompts designed with re\-contextualization\.

### Refusal, degradation, and collapse\.

A reduction in target reproduction is not automatically successful unlearning\. The model may refuse, emit empty text, truncate, or degenerate into repetitive loops\. Neural generation can also collapse into repetitive text; n\-gram repetition and diversity diagnostics are standard signals for this failure mode\[[9](https://arxiv.org/html/2607.22657#bib.bib16),[27](https://arxiv.org/html/2607.22657#bib.bib17),[14](https://arxiv.org/html/2607.22657#bib.bib18)\]\. For this reason,LENSseparates narrative reproduction from refusal, degradation, benchmark\-utility deltas, and retain drift\. The goal is not maximum apparent forgetting, but a useful checkpoint where the target narrative is reduced and the model can still answer the prompt\.

### Research Gap\.

Prior work provides unlearning objectives and forget/retain evaluation logic, but it does not directly evaluate narrative\-frame suppression across resistance levels\.LENSfills this gap by testing target narrative reproduction across L0–L3 prompt families and by reporting L3 entity recovery separately from ordinary target reproduction\. This motivates a broader question for narrative unlearning: does it behave like fact\- or capability\-level forgetting, or does it mainly shift the model’s explanatory behavior while leaving related frame associations available under indirect prompts?

## 3Problem Setting

We study whether existing unlearning methods can reduce a model’s tendency to reproduce a target disinformation\-aligned narrative frame\. A target frame is a recurring explanatory structure: which actors are named, what causal relation is used, how responsibility is assigned, and whether the response gives the frame credibility\.

### Narrative reproduction\.

We define*narrative reproduction*as a model response that presents the target frame as a credible explanation\. A response is not counted as reproduction when it rejects, discredits, or clearly contextualizes the frame as unsupported, misleading, or not the main explanation\. Empty, off\-topic, repetitive, truncated, or collapsed outputs are labeled separately as invalid or degraded generations\.

### Target narratives\.

We use two source\-grounded narrative frames\. In NATO\-forced\-war, NATO expansion is presented as the cause of Russia’s war against Ukraine, reducing Russia’s responsibility for the invasion\[[20](https://arxiv.org/html/2607.22657#bib.bib20),[26](https://arxiv.org/html/2607.22657#bib.bib8),[4](https://arxiv.org/html/2607.22657#bib.bib9)\]\. In US\-abandons\-Taiwan, the United States is presented as using Taiwan for its own strategic interests and potentially abandoning it\[[16](https://arxiv.org/html/2607.22657#bib.bib10),[17](https://arxiv.org/html/2607.22657#bib.bib21)\]\. These sources define the narrative signatures and we convert them into controlled prompt families for evaluation\. Table[1](https://arxiv.org/html/2607.22657#S3.T1)shows the two signatures and one compact L0 prompt for each\.

Table 1:Target narrative signatures and direct compact\-response examples\.
### Models and languages\.

The evaluation covers four near\-12B multilingual instruction models: Lapa LLM\[[12](https://arxiv.org/html/2607.22657#bib.bib23)\], Gemma\-12B\[[6](https://arxiv.org/html/2607.22657#bib.bib24)\], Qwen\-14B\[[21](https://arxiv.org/html/2607.22657#bib.bib25)\], and TAIDE\-Gemma\[[25](https://arxiv.org/html/2607.22657#bib.bib26)\]\. We evaluate Ukrainian, English, Simplified Chinese, and Traditional Chinese where baseline quality is sufficient\. Lapa is evaluated in Ukrainian and English\. Gemma\-12B and Qwen\-14B are evaluated in all four languages\. TAIDE\-Gemma is evaluated in English, Simplified Chinese, and Traditional Chinese; Ukrainian is excluded because baseline Ukrainian generations did not pass the readability quality gate\. Chinese prompts were validated by a native Chinese speaker, and Ukrainian prompts by a native Ukrainian speaker\.

### Compact and unrestricted responses\.

The main experiments use a compact\-response condition, adding a localized instruction such as “Answer in 1–2 sentences\.” We use the same condition for baseline generation, manual forget\-set construction, post\-unlearning generation, and before/after reporting\. Compact responses reduce mixed\-stance answers and make manual and weak\-label annotation more reliable\. We keep an unrestricted condition only as a sensitivity comparison for how response length changes baseline narrative reproduction\.

### Forget and evaluation data\.

Forget data are built separately for each model–narrative–language cell\. For each cell, the base model is prompted with direct L0 elicitation prompts, and the resulting answers are manually labeled\. Only target\-dominant reproductions enter the strict forget set\. The unlearning loss is prompt\-masked: the prompt remains in the context, but prompt tokens are masked from the loss, so updates apply only to answer tokens\[[5](https://arxiv.org/html/2607.22657#bib.bib27)\]\.

## 4LENS evaluation protocol

LENStests whether narrative suppression transfers beyond the direct prompts used to build the forget set\. Each target narrative is represented by a source\-grounded signature and a multilingual prompt suite\. Prompts are organized into four resistance levels, from direct elicitation to abstract structural probing\. Table[2](https://arxiv.org/html/2607.22657#S4.T2)defines the levels and gives abbreviated English examples\.

Table 2:LENSresistance levels\. Examples are abbreviated English prompt patterns in the compact\-response condition\.### Level definitions\.

L0 prompts directly ask for an explanation or justification of the target frame\. They are used to construct the forget\-candidate pool because they most explicitly elicit the behavior to suppress\. L1 prompts keep the same frame but attribute it to commentators, analysts, or other third parties\. L2 prompts place the target frame beside a competing explanation or ask for comparison\. L3 prompts remove direct claim wording and mask the actors while preserving the causal structure\. The main suppression results use L0–L2\. These levels contain named real\-world actors, so the reporting question is whether the answer gives credibility to the target narrative\. L0 measures direct suppression, while L1 and L2 test transfer under recontextualization\. If a model stops reproducing the frame under L0 but still reproduces it under attribution or comparison, suppression is incomplete\.

### L3 and entity recovery\.

L3 is reported separately because it asks a different question\. A model can answer an abstract A/B/C prompt by staying abstract, or it can map the abstract frame back to real\-world actors\. We therefore separate*frame\-only abstract behavior*from*entity\-recovery reproduction*\. For NATO\-forced\-war, entity\-recovery reproduction requires recovery of NATO, Russia, and Ukraine together with the target causal frame\. For US\-abandons\-Taiwan, it requires recovery of the United States and Taiwan together with the target frame\. If an L3 answer preserves only the abstract structure without real\-actor recovery, it is not counted as target\-narrative reproduction in the main metric\. This conservative rule prevents generic abstract reasoning from being confused with recovery of the concrete narrative\.

### Prompt construction and multilingual validation\.

Each narrative has multiple prompt variants per level and language\. English prompts are written from the source\-grounded narrative signature and then translated into Ukrainian, Simplified Chinese, and Traditional Chinese\. The final evaluation uses the compact\-response condition consistently across base generation, forget\-set construction, and post\-unlearning generation\.

### Evaluation loop\.

For each model–narrative–language cell, we first generate baselineLENSresponses\. Only compact L0 responses are eligible for forget\-set construction, and only manually labeled narrative\-reproduction responses enter the strict forget set\. Unlearning runs are trained separately by model, narrative, and language\. After training, saved checkpoints are evaluated by regenerating theLENSsuite under the same compact\-response condition\. Calibrated response\-only NLI labels are used for scalable before/after diagnostics, while degradation, refusal, retain drift, and qualitative examples are reported separately\.

## 5Unlearning Methods

We evaluate existing unlearning objectives under theLENSprotocol with original optimizers\. Each run has three components: an unlearning objective, a parameter\-efficient adapter that carries the update, and theLENSevaluation wrapper that defines forget examples, checkpoint selection, and before/after reporting\.

### Removal\-oriented setup\.

All methods are removal\-oriented\. They receive manually selected L0 answers that reproduce the target narrative and optimize away from those answer tokens\. This design tests whether existing unlearning pressure can reduce a reproduced narrative frame without explicit counter\-narrative supervision, and whether residual or substitute explanations appear after suppression\. Forget data come from manually selected compact L0 base\-model answers that reproduce the target narrative\. Only strict target\-dominant answers are used for training; label definitions and weak\-label calibration are described in Section[6](https://arxiv.org/html/2607.22657#S6)\.

### Method suite\.

We use four objective families:TOFU\-GA, a pure gradient\-ascent forgetting baseline;TOFU\-GD, a gradient\-difference baseline with retain anchoring\[[19](https://arxiv.org/html/2607.22657#bib.bib6)\];GA\-Retain, a gradient\-ascent objective with retain\-preservation terms\[[29](https://arxiv.org/html/2607.22657#bib.bib5)\]; andNPO, a negative\-preference objective that treats reproduced L0 answers as dispreferred outputs\[[31](https://arxiv.org/html/2607.22657#bib.bib12)\]\. The same four setups are used across model, narrative, and language cells\. Detailed objective settings and hyperparameters are reported in Appendix[0\.D](https://arxiv.org/html/2607.22657#Pt0.A4)\. All main\-suite runs use LoRA adapters with rank 8, alpha 16, dropout 0\.05, learning rate10−510^\{\-5\}, checkpoint cadence 10, and a maximum of 260 steps\. Earlier exploratory runs also included AdaLoRA adapters\[[30](https://arxiv.org/html/2607.22657#bib.bib14)\], but these did not improve narrative suppression in pilot experiments and are not included in the main suite\.

### Suppression\-regime search\.

Every run emits checkpoints every 10 steps up to 260 steps or until the method\-specific stop rule triggers\. Checkpoints are evaluated after training\. For each model–narrative–language–method cell, we report the*suppression\-tradeoff*checkpoint: the checkpoint with the strongest useful suppression under theSCEcriterion defined in Section[6](https://arxiv.org/html/2607.22657#S6), using target\-matched L0–L2 rows only\. When available, we also record the*first\-collapse*checkpoint, defined as the earliest checkpoint where output degradation crosses the collapse boundary\. L3 and L3\-special are not used for checkpoint selection and are reported as separate diagnostics\.

## 6Metrics and Labeling

A useful narrative\-unlearning checkpoint should reduce target\-frame reproduction without producing refusal, degraded generations, or large retain\-distribution drift\. This section defines the manual labels used for forget\-set construction, the weak NLI labels used for scalable reporting, the Suppression–Collapse Efficiency \(SCE\) score used for checkpoint selection, and the utility/retain diagnostics used to interpret selected checkpoints\.

### Manual labeling\.

Forget\-set construction uses manual labeling by one annotator\. For each model–narrative–language cell, we collect compact L0 base\-model answers and label whether each answer gives credibility to the target narrative\. The strict forget set includes onlyTarget\-Dominantrows\. Table[6](https://arxiv.org/html/2607.22657#Pt0.A1.T6)in Appendix[0\.A](https://arxiv.org/html/2607.22657#Pt0.A1)defines the label scheme\.

The calibration pool contains 381 explicit\-L0 answers across the four study models: 257Target\-Dominant, 90Target\-Included, 34Target\-Rejected\. These labels define strict forget examples for training and provide a calibration set for estimating weak\-label error\.

### Weak NLI labeling\.

Natural language inference \(NLI\) is used for scalable before/after diagnostics\. Given a premise and a hypothesis, an NLI model predicts whether the hypothesis is entailed, contradicted, or neutral with respect to the premise\[[2](https://arxiv.org/html/2607.22657#bib.bib28),[1](https://arxiv.org/html/2607.22657#bib.bib29),[28](https://arxiv.org/html/2607.22657#bib.bib30)\]\.

We usemDeBERTa\-v3\-base\-xnli\-multilingual\-nli\-2mil7\[[7](https://arxiv.org/html/2607.22657#bib.bib31),[13](https://arxiv.org/html/2607.22657#bib.bib32)\]\. For each narrative and language, the NLI labeler scores the response against short hypotheses expressing endorsement, mention, and rejection of the target frame\. A response is weakly labeled asReproducedwhen the best endorsement hypothesis is strongly entailed and separated from contradiction and neutral alternatives\. These labels are noisy weak labels in the sense of weak supervision\[[22](https://arxiv.org/html/2607.22657#bib.bib33)\]; they are used for triage and post\-hoc diagnostics and not as ground truth\.

Table[6](https://arxiv.org/html/2607.22657#Pt0.A1.T6)reports NLI calibration against the manual L0 labels\. Broad precision is high, while strict precision is lower because someTarget\-Includedanswers are weak\-labeled as reproduced\. In the table Strict isTarget\-Dominant; Broad isTarget\-DominantandTarget\-Includedlabels\. Full counts and intervals are introduced in Appendix[0\.C](https://arxiv.org/html/2607.22657#Pt0.A3)\.

### Suppression, degradation, andSCE\.

For a fixed evaluation set, letBBbe the number of base outputs labeledReproduced,AAthe number of non\-degraded post\-unlearning outputs labeledReproduced,DDthe number of post\-unlearning outputs labeledDegraded, andNNthe number of evaluated outputs\. We report the raw before/after counts and summarize checkpoint quality using:

Δ​R=B−A,s=max⁡\(B−A,0\)max⁡\(B,1\),g=DN,SCE=s​\(1−g\)2\.\\Delta R=B\-A,\\qquad s=\\frac\{\\max\(B\-A,0\)\}\{\\max\(B,1\)\},\\qquad g=\\frac\{D\}\{N\},\\qquad\\mathrm\{SCE\}=s\(1\-g\)^\{2\}\.Here,Δ​R\\Delta Ris the reproduction\-count change,ssis relative suppression, andggis the post\-unlearning degradation rate\. The quadratic degradation penalty makes collapse cases more expensive: a checkpoint that reduces target reproduction only by producing empty, repetitive, or broken answers receives a low score even ifAAdecreases\.SCEis therefore used to identify the best observed suppression/quality tradeoff along a trajectory\. It is not interpreted alone: degradation, refusal, benchmark\-utility deltas, and retain drift determine whether the selected checkpoint is a useful operating point or only apparent forgetting\.

### Generation health, benchmark\-utility, and retain drift\.

Generation\-health diagnostics flag empty or near\-empty outputs, unfinished outputs at the token cap, repeated character loops, malformed parenthesis or bracket loops, and low\-diversity n\-gram loops\. The compact degeneration score uses repeated 4\-gram rate, maximum 4\-gram frequency, unique\-token ratio, and dominant\-token rate, following standard diagnostics for neural text degeneration, repetition, and diversity\[[9](https://arxiv.org/html/2607.22657#bib.bib16),[27](https://arxiv.org/html/2607.22657#bib.bib17),[14](https://arxiv.org/html/2607.22657#bib.bib18)\]\.

We report benchmark\-utility as a retain\-side diagnostic which is multiple\-choice benchmark performance on small utility subsets such as MMLU\-English and Belebele\. Results are provided in Appendix[0\.B](https://arxiv.org/html/2607.22657#Pt0.A2)\.

## 7Results

![Refer to caption](https://arxiv.org/html/2607.22657v1/x1.png)Figure 2:IllustrativeSCEcheckpoint\-selection heatmap for Gemma\-12B US\-abandons\-Taiwannarrative\. Rows are method trajectories and columns are checkpoint steps, split by language\. Brighter cells indicate stronger suppression–collapse effectiveness\. Other models are presented in Appendix[0\.F](https://arxiv.org/html/2607.22657#Pt0.A6)\.The main quantitative results use target\-matched L0–L2 rows\. L3 is reported separately because masked prompts test abstract\-frame behavior and entity recovery rather than ordinary target reproduction\. Suppression is computed as defined in Section[6](https://arxiv.org/html/2607.22657#S6.SS0.SSS0.Px3); degradation is the post\-unlearning degraded output rate; andSCEis used as a checkpoint\-selection summary\.

### Suppression–collapse regime\.

The central empirical result is a suppression–collapse regime\. Along an unlearning trajectory, target\-narrative reproduction can fall before generation quality breaks down\. The useful operating region is therefore a middle part of the trajectory: target reproduction is reduced, but outputs remain substantive, non\-repetitive, and usable\. Figure[2](https://arxiv.org/html/2607.22657#S7.F2)illustrates this regime for Gemma\-12B on the checkpoint\-selection slice\. Each cell is one method–narrative–language checkpoint\. Brighter cells indicate stronger suppression–collapse effectiveness underSCE\. The heatmap also shows that the useful region is cell\-specific: the same objective can expose a clean suppression region for one narrative or language and a weaker or more collapse\-prone region for another\.

### Level transfer\.

Table[3\(b\)](https://arxiv.org/html/2607.22657#S7.T3.st2)summarizes whether suppression transfers beyond the direct L0 prompts used to construct the forget set\. The table pools selected checkpoints across completed model–language cells\. For NATO\-forced\-war, contrastive L2 has a smaller absolute reproduction\-rate drop than L0 or L1, but its lower degradation gives the strongest aggregate SCE among the three levels\. For US\-abandons\-Taiwan, transfer is weaker at contrastive L2, where the absolute reproduction\-rate drop is smallest\.

Although forget data are constructed from direct L0 responses, suppression transfers to L1 and L2 in both narratives\. For NATO\-forced\-war, the reproduction\-rate drop remains substantial under attribution and contrastive framing, although degradation is also higher\. For US\-abandons\-Taiwan, transfer is strongest from L0 to L1 and weaker at contrastive L2\. This suggests that direct\-answer unlearning can change broader narrative behavior, but does not remove the frame uniformly across resistance levels\. L3 is excluded from Table[3\(b\)](https://arxiv.org/html/2607.22657#S7.T3.st2)because it asks a different question: whether an abstract or masked prompt causes real\-world actors to resurface\. We report L3 separately as an entity\-recovery diagnostic\.

Table 3:Aggregate L0–L2 transfer at selected suppression\-tradeoff checkpoints\.N=1100N=1100responses\.\(a\)NATO\-forced\-war
\(b\)US\-abandons\-Taiwan

### Benchmark\-utility and retain drift\.

Selected checkpoints are also evaluated with benchmark\-utility and retain\-drift diagnostics\. Benchmark\-utility is measured on small multiple\-choice subsets, including MMLU\-English and Belebele\. Retain drift is measured with retain CE/PPL and base\-vs\-adapter KL\. These diagnostics do not replaceSCE; they indicate whether a selected checkpoint is also a plausible operating point\.

The benchmark utility deltas are small, suggesting that selected checkpoints often remain close to the base model on these small multiple\-choice subsets\. However, retain CE/PPL and KL reveal larger drift in some cells, especially Lapa and TAIDE\. This is not a contradiction: benchmark accuracy, fixed\-reference likelihood, and token\-distribution KL measure different aspects of retain behavior\. Selected checkpoints should therefore be interpreted as points on a suppression–collapse–drift frontier, not as uniformly clean unlearning results\.

### L3 entity\-recovery diagnostics\.

L3 is reported separately from the main suppression tables because it tests a different behavior\. L0–L2 prompts contain named real\-world actors and ask whether the model gives credibility to the target narrative\. L3 prompts remove or mask those actors and test whether the model stays abstract or maps the frame back to real\-world entities\.

Entity recovery is rare, but it can appear or increase after unlearning\. For NATO\-forced\-war, the recovery rate rises from 0 to 43 out of 27,720\. For US\-abandons\-Taiwan, it rises from 40 to 68 out of 27,720\. Even when direct reproduction is reduced, abstract frame associations and actor recovery can remain available under masked prompting\.

### Qualitative Diagnostics

Direct L0 examples show whether selected checkpoints move away from endorsing the target frame\. L3\-special examples show whether abstract or masked prompts cause real\-world actors to resurface after unlearning\.

Table 4:Qualitative diagnostics from Gemma\-12B for NATO\-forced\-war\. Outputs are model generations from the selected NPO checkpoint\.Table[4](https://arxiv.org/html/2607.22657#S7.T4)illustrates why level\-wise evaluation is needed\. The selected checkpoint moves away from the target frame under direct L0 elicitation and also changes the contrastive L2 answer, where the base model gave partial credibility to the NATO\-forced\-war explanation\. The selected outputs are not polished replacement answers, yet we may observe that opposition to previous model answers\. Additional full before/after examples, including L1 and L3 diagnostics, are reported in Appendix[0\.E](https://arxiv.org/html/2607.22657#Pt0.A5)\.

## 8Discussion

AnsweringRQ1we found that selected checkpoints reduce target\-frame reproduction in many settings, but in general the behavior is better understood as a shift in model explanations than as evidence that the narrative has been erased\. Direct L0 suppression can transfer to attributed L1 and contrastive L2 prompts, but transfer is uneven across narratives and resistance levels\. This supports the main role ofLENS: direct\-prompt suppression alone is not enough to determine whether a narrative frame has been weakened\.

FollowingRQ2and qualitative answer analysis we observed how opposition answers appear after unlearning intervention, suggesting that alternative narratives are already present in the model representations but not visible due to more dominant narratives overpowering it\. The L3 diagnostics show that abstract or masked prompts can still recover real\-world actors associated with the target frame\. This behavior is rare, but it indicates that there are abstract frame associations with real\-world actors which are not explicit\.

LENSis designed as contextualized suit for providing deeper understanding of how narratives may behave under unlearning interventions\. Real\-world entity recovery, reappearance of alternative explanations after narrative unlearning, and suppression effects propagation across levels, as highlighted byLENS, indicate open questions for further research towards uncovering how such complex structures as narratives are stored in LLMs parameters and new approaches for more effective narrative interventions\.

### Ethics\.

Suppressing a narrative frame is not the same as teaching factual truth, political neutrality, or robust safety behavior\. The target narratives are studied as measurement objects, not endorsed\. Raw generation may reproduce contested or misleading claims, so examples are used sparingly and framed as diagnostic evidence\. Any deployment use of narrative unlearning would require broader language coverage, human review, and careful governance, because suppressing a frame can affect legitimate discussion, historical interpretation, or political speech\.

## References

- \[1\]S\. R\. Bowman, G\. Angeli, C\. Potts, and C\. D\. Manning\(2015\)A large annotated corpus for learning natural language inference\.InProceedings of the 2015 Conference on Empirical Methods in Natural Language Processing,pp\. 632–642\.External Links:[Document](https://dx.doi.org/10.18653/v1/D15-1075),[Link](https://aclanthology.org/D15-1075/)Cited by:[§6](https://arxiv.org/html/2607.22657#S6.SS0.SSS0.Px2.p1.1)\.
- \[2\]I\. Dagan, O\. Glickman, and B\. Magnini\(2006\)The PASCAL recognising textual entailment challenge\.InMachine Learning Challenges\. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Textual Entailment,pp\. 177–190\.External Links:[Document](https://dx.doi.org/10.1007/11736790%5F9),[Link](https://doi.org/10.1007/11736790_9)Cited by:[§6](https://arxiv.org/html/2607.22657#S6.SS0.SSS0.Px2.p1.1)\.
- \[3\]D\. Danet\(2025\-09\)LLM Grooming: A New Cognitive Threat to Generative AI\.External Links:[Link](https://hal.science/hal-05241525)Cited by:[§1](https://arxiv.org/html/2607.22657#S1.p1.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px1.p1.1)\.
- \[4\]EUvsDisinfo\(2025\)The myth that won’t die: blaming nato for russia’s war\.Note:Accessed 2026\-05\-22External Links:[Link](https://euvsdisinfo.eu/the-myth-that-wont-die-blaming-nato-for-russias-war/)Cited by:[§1](https://arxiv.org/html/2607.22657#S1.p6.1),[§3](https://arxiv.org/html/2607.22657#S3.SS0.SSS0.Px2.p1.1)\.
- \[5\]G\. K\. Gangadhar and K\. Stratos\(2024\)Model editing by standard fine\-tuning\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 5907–5913\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.352),[Link](https://aclanthology.org/2024.findings-acl.352/)Cited by:[§3](https://arxiv.org/html/2607.22657#S3.SS0.SSS0.Px5.p1.1)\.
- \[6\]Gemma Team\(2025\)Gemma 3 Technical Report\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2503.19786),2503\.19786,[Link](https://arxiv.org/abs/2503.19786)Cited by:[§3](https://arxiv.org/html/2607.22657#S3.SS0.SSS0.Px3.p1.1)\.
- \[7\]P\. He, J\. Gao, and W\. Chen\(2023\)DeBERTaV3: improving DeBERTa using ELECTRA\-style pre\-training with gradient\-disentangled embedding sharing\.InInternational Conference on Learning Representations,External Links:2111\.09543,[Link](https://arxiv.org/abs/2111.09543)Cited by:[§6](https://arxiv.org/html/2607.22657#S6.SS0.SSS0.Px2.p2.1)\.
- \[8\]M\. Hellman\(2024\)Narrative analysis and framing analysis of disinformation\.InSecurity, Disinformation and Harmful Narratives: RT and Sputnik News Coverage about Sweden,pp\. 101–121\.External Links:ISBN 978\-3\-031\-58747\-4,[Document](https://dx.doi.org/10.1007/978-3-031-58747-4%5F4),[Link](https://doi.org/10.1007/978-3-031-58747-4_4)Cited by:[§1](https://arxiv.org/html/2607.22657#S1.p1.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px1.p1.1)\.
- \[9\]A\. Holtzman, J\. Buys, L\. Du, M\. Forbes, and Y\. Choi\(2020\)The curious case of neural text degeneration\.InInternational Conference on Learning Representations,External Links:[Document](https://dx.doi.org/10.48550/arXiv.1904.09751),1904\.09751,[Link](https://arxiv.org/abs/1904.09751)Cited by:[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px5.p1.1),[§6](https://arxiv.org/html/2607.22657#S6.SS0.SSS0.Px4.p1.1)\.
- \[10\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen\(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,External Links:[Document](https://dx.doi.org/10.48550/arXiv.2106.09685),2106\.09685,[Link](https://arxiv.org/abs/2106.09685)Cited by:[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px3.p1.1)\.
- \[11\]M\. Ko, H\. A\. Just, C\. Fleming, M\. Jin, and R\. Jia\(2025\)Probing knowledge holes in unlearned llms\.External Links:2511\.00030,[Link](https://arxiv.org/abs/2511.00030)Cited by:[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px4.p1.1)\.
- \[12\]Lapa LLM\(2025\)lapa\-v0\.1\.2\-instruct model card\.Note:[https://huggingface\.co/lapa\-llm/lapa\-v0\.1\.2\-instruct](https://huggingface.co/lapa-llm/lapa-v0.1.2-instruct)External Links:[Link](https://huggingface.co/lapa-llm/lapa-v0.1.2-instruct)Cited by:[§3](https://arxiv.org/html/2607.22657#S3.SS0.SSS0.Px3.p1.1)\.
- \[13\]M\. Laurer, W\. van Atteveldt, A\. Casas, and K\. Welbers\(2024\)Less annotating, more classifying: addressing the data scarcity issue of supervised machine learning with deep transfer learning and BERT\-NLI\.Political Analysis32\(1\),pp\. 84–100\.External Links:[Document](https://dx.doi.org/10.1017/pan.2023.20),[Link](https://doi.org/10.1017/pan.2023.20)Cited by:[§6](https://arxiv.org/html/2607.22657#S6.SS0.SSS0.Px2.p2.1)\.
- \[14\]J\. Li, M\. Galley, C\. Brockett, J\. Gao, and B\. Dolan\(2016\)A diversity\-promoting objective function for neural conversation models\.InProceedings of NAACL\-HLT,pp\. 110–119\.External Links:[Document](https://dx.doi.org/10.18653/v1/N16-1014),[Link](https://aclanthology.org/N16-1014/)Cited by:[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px5.p1.1),[§6](https://arxiv.org/html/2607.22657#S6.SS0.SSS0.Px4.p1.1)\.
- \[15\]N\. Li, A\. Pan, A\. Gopal, S\. Yue, D\. Berrios, A\. Gatti, J\. D\. Li, A\. Dombrowski, S\. Goel, L\. Phan,et al\.\(2024\)The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2403.03218),2403\.03218,[Link](https://arxiv.org/abs/2403.03218)Cited by:[§1](https://arxiv.org/html/2607.22657#S1.p2.1),[§1](https://arxiv.org/html/2607.22657#S1.p4.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px4.p1.1)\.
- \[16\]W\. Li\(2023\)Inciting anxiety about the looming war: the disinformation narratives about the possible taiwan strait crisis during the 2024 taiwanese presidential election\.External Links:[Link](https://en.tfc-taiwan.org.tw/en_tfc_282/)Cited by:[§1](https://arxiv.org/html/2607.22657#S1.p6.1),[§3](https://arxiv.org/html/2607.22657#S3.SS0.SSS0.Px2.p1.1)\.
- \[17\]W\. Li\(2024\)Ripples of disinformation: tracking chinese\-language disinformation after the 2024 u\.s\. presidential election\.Note:Taiwan FactCheck Center\.External Links:[Link](https://en.tfc-taiwan.org.tw/ripples-of-disinformation-tracking-chinese-language-disinformation-after-the-2024-u-s-presidential-election/)Cited by:[§3](https://arxiv.org/html/2607.22657#S3.SS0.SSS0.Px2.p1.1)\.
- \[18\]W\. Li\(2025\)What did the false information say about Taiwan\-U\.S\. relations after the U\.S\. presidential transition\.External Links:[Link](https://en.tfc-taiwan.org.tw/taiwan-u-s-relations-disinfomation/)Cited by:[§1](https://arxiv.org/html/2607.22657#S1.p6.1)\.
- \[19\]P\. Maini, Z\. Feng, A\. Schwarzschild, Z\. C\. Lipton, and J\. Z\. Kolter\(2024\)TOFU: a task of fictitious unlearning for LLMs\.InProceedings of the Conference on Language Modeling,External Links:[Document](https://dx.doi.org/10.48550/arXiv.2401.06121),2401\.06121,[Link](https://arxiv.org/abs/2401.06121)Cited by:[Table 9](https://arxiv.org/html/2607.22657#Pt0.A4.T9.1.1.1.1.1),[Table 9](https://arxiv.org/html/2607.22657#Pt0.A4.T9.3.3.1.1.1),[§1](https://arxiv.org/html/2607.22657#S1.p2.1),[§1](https://arxiv.org/html/2607.22657#S1.p4.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px4.p1.1),[§5](https://arxiv.org/html/2607.22657#S5.SS0.SSS0.Px2.p1.1)\.
- \[20\]A\. Polishko, A\. Küllian, I\. Gadzynska, M\. Khandoha, N\. Romanyshyn, S\. Mikhalkov, Y\. Kostiuk, and Y\. Filipchuk\(2025\)What does ai think about ukraine? exploring the biases of large language models\.External Links:[Link](https://texty.org.ua/projects/115751/what-does-ai-think-about-ukraine-exploring-the-biases-of-large-language-models/)Cited by:[§3](https://arxiv.org/html/2607.22657#S3.SS0.SSS0.Px2.p1.1)\.
- \[21\]Qwen Team\(2025\)Qwen3 Technical Report\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2505.09388),2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§3](https://arxiv.org/html/2607.22657#S3.SS0.SSS0.Px3.p1.1)\.
- \[22\]A\. Ratner, C\. De Sa, S\. Wu, D\. Selsam, and C\. Ré\(2016\)Data programming: creating large training sets, quickly\.InAdvances in Neural Information Processing Systems,Vol\.29,pp\. 3567–3575\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1605.07723),1605\.07723,[Link](https://arxiv.org/abs/1605.07723)Cited by:[§6](https://arxiv.org/html/2607.22657#S6.SS0.SSS0.Px2.p2.1)\.
- \[23\]M\. Sadeghi\(2024\-06\)Top 10 generative ai models mimic russian disinformation claims a third of the time, citing moscow\-created fake local news sites as authoritative sources\.NewsGuard\.External Links:[Link](https://www.newsguardtech.com/special-reports/generative-ai-models-mimic-russian-disinformation-cite-fake-news/)Cited by:[§1](https://arxiv.org/html/2607.22657#S1.p1.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px1.p1.1)\.
- \[24\]A\. Souly, J\. Rando, E\. Chapman, X\. Davies, B\. Hasircioglu, E\. Shereen, C\. Mougan, V\. Mavroudis, E\. Jones, C\. Hicks, N\. Carlini, Y\. Gal, and R\. Kirk\(2025\)Poisoning attacks on llms require a near\-constant number of poison samples\.External Links:2510\.07192,[Document](https://dx.doi.org/10.48550/arXiv.2510.07192),[Link](https://arxiv.org/abs/2510.07192)Cited by:[§1](https://arxiv.org/html/2607.22657#S1.p1.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px1.p1.1)\.
- \[25\]TAIDE\(2025\)Gemma\-3\-TAIDE\-12b\-Chat model card\.External Links:[Link](https://huggingface.co/taide/Gemma-3-TAIDE-12b-Chat)Cited by:[§3](https://arxiv.org/html/2607.22657#S3.SS0.SSS0.Px3.p1.1)\.
- \[26\]VoxCheck Team\(2023\)Propaganda diary 2022–2023: voxcheck presents the database of russian propaganda in the european mass media\.External Links:[Link](https://russiandisinfo.voxukraine.org/en/narrative/1007)Cited by:[§1](https://arxiv.org/html/2607.22657#S1.p6.1),[§3](https://arxiv.org/html/2607.22657#S3.SS0.SSS0.Px2.p1.1)\.
- \[27\]S\. Welleck, I\. Kulikov, S\. Roller, E\. Dinan, K\. Cho, and J\. Weston\(2020\)Neural text generation with unlikelihood training\.InInternational Conference on Learning Representations,External Links:[Document](https://dx.doi.org/10.48550/arXiv.1908.04319),1908\.04319,[Link](https://arxiv.org/abs/1908.04319)Cited by:[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px5.p1.1),[§6](https://arxiv.org/html/2607.22657#S6.SS0.SSS0.Px4.p1.1)\.
- \[28\]A\. Williams, N\. Nangia, and S\. R\. Bowman\(2018\)A broad\-coverage challenge corpus for sentence understanding through inference\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 1112–1122\.External Links:[Document](https://dx.doi.org/10.18653/v1/N18-1101),[Link](https://aclanthology.org/N18-1101/)Cited by:[§6](https://arxiv.org/html/2607.22657#S6.SS0.SSS0.Px2.p1.1)\.
- \[29\]Y\. Yao, X\. Xu, and Y\. Liu\(2024\)Large Language Model Unlearning\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:[Document](https://dx.doi.org/10.52202/079017-3346),[Link](https://doi.org/10.52202/079017-3346)Cited by:[Table 9](https://arxiv.org/html/2607.22657#Pt0.A4.T9.8.8.6.1.1),[§1](https://arxiv.org/html/2607.22657#S1.p2.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2607.22657#S5.SS0.SSS0.Px2.p1.1)\.
- \[30\]Q\. Zhang, M\. Chen, A\. Bukharin, P\. He, Y\. Cheng, W\. Chen, and T\. Zhao\(2023\)AdaLoRA: adaptive budget allocation for parameter\-efficient fine\-tuning\.InInternational Conference on Learning Representations,External Links:[Document](https://dx.doi.org/10.48550/arXiv.2303.10512),2303\.10512,[Link](https://arxiv.org/abs/2303.10512)Cited by:[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2607.22657#S5.SS0.SSS0.Px2.p1.1)\.
- \[31\]R\. Zhang, L\. Lin, Y\. Bai, and S\. Mei\(2024\)Negative preference optimization: from catastrophic collapse to effective unlearning\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2404.05868),2404\.05868,[Link](https://arxiv.org/abs/2404.05868)Cited by:[Table 9](https://arxiv.org/html/2607.22657#Pt0.A4.T9.9.9.3.1.1),[§2](https://arxiv.org/html/2607.22657#S2.SS0.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2607.22657#S5.SS0.SSS0.Px2.p1.1)\.

## Appendix 0\.AAnnotation details

Table 5:Manual labels for L0 forget\-candidate annotation\. OnlyTarget\-Dominantrows enter the strict forget set\.
Table 6:Response\-only NLI calibration on 381 manually labeled L0 answers\.

## Appendix 0\.BBenchmark\-utility and retain\-drift diagnostics

Retain drift is measured with cross\-entropy \(CE\), perplexity \(PPL\), and Kullback–Leibler \(KL\) divergence on retain examples\. Retain CE scores fixed retain reference answers with prompt tokens masked; PPL is the exponential of retain CE\. We report retain CE drift relative to the frozen base model\. Large positive drift means that the adapter makes the original retain reference answers less likely\. KL measures full next\-token distribution drift from the frozen base model to the adapter\-enabled model on retain contexts\.

Table 7:Benchmark\-utility and retain\-drift for English selected checkpoints\. SCE is the selected checkpoint’s suppression–collapse efficiency\.Δ\\DeltaMMLU andΔ\\DeltaBel\. are adapter\-minus\-base accuracy deltas\. LargerΔ\\DeltaCE, PPL, or KL indicates stronger retain\-side drift\.⋆high SCE with low retain drift;†high SCE with substantial retain drift\.

## Appendix 0\.CNLI calibration details

Table 8:Full calibration of response\-only NLI against 381 manually labeled explicit\-L0 answers\. Values in brackets report approximate 95% confidence intervals \(CI\): Wilson CIs for precision and recall, and bootstrap CIs for F1\.
## Appendix 0\.DUnlearning method details

Table 9:Unlearning objective suite\. Each setup is run as a fixed trajectory; checkpoints are selected post hoc usingSCE\.
## Appendix 0\.EQualitative suppression examples

This subsection gives before/after examples for selected suppression\-tradeoff checkpoints\. Each table is restricted to one model–narrative pair and includes all four LENS levels\. L0–L2 rows illustrate ordinary target\-narrative suppression\. L3 rows use masked/abstract prompts\.

Table 10:Gemma\-12B qualitative examples for NATO\-forced\-war\. The selected checkpoint is NPO LoRA step 240\. L0–L2 illustrate direct, attributed, and contrastive suppression; L3 is an abstract masked diagnostic\.Table 11:Lapa qualitative examples for NATO\-forced\-war\. The stored generations are Ukrainian; English translations are shown for readability\. The illustrative checkpoint is GA\-Retain LoRA step 70\. L0–L2 show clearer suppression than the English Lapa example; L3 is an abstract masked diagnostic\.Table 12:Qwen\-14B qualitative examples for NATO\-forced\-war\. The stored generations are Simplified Chinese; English translations are shown for readability\. The selected checkpoint is GA\-Retain LoRA step 200\. L0–L2 illustrate direct, attributed, and contrastive suppression; L3 is an abstract masked diagnostic\.Table 13:TAIDE\-Gemma qualitative examples for US\-abandons\-Taiwan\. The selected checkpoint is TOFU\-GA LoRA step 250\. L0–L2 illustrate direct, attributed, and contrastive suppression; L3 is an abstract masked diagnostic\.Table 14:Gemma\-12B qualitative examples for US\-abandons\-Taiwan\. The selected checkpoint is NPO LoRA step 210\. L0–L2 illustrate direct, attributed, and contrastive behavior; L3 is an abstract masked diagnostic\.Table 15:Lapa qualitative examples for US\-abandons\-Taiwan\. The selected checkpoint is NPO LoRA step 100\. L0–L2 illustrate direct, attributed, and contrastive suppression; L3 is an abstract masked diagnostic\.Table 16:Qwen\-14B qualitative examples for US\-abandons\-Taiwan\. The selected checkpoint is GA\-Retain LoRA step 20\. L0–L2 illustrate direct, attributed, and contrastive behavior; L3 is an abstract masked diagnostic\.Table 17:TAIDE\-Gemma qualitative examples for US\-abandons\-Taiwan\. The selected checkpoint is TOFU\-GA LoRA step 250\. L0–L2 illustrate direct, attributed, and contrastive suppression; L3 is an abstract masked diagnostic\.Table 18:L3\-special entity\-recovery examples by model\. Each row shows one stored selected\-checkpoint generation from a masked L3\-special prompt\. Those results not included into the main L0–L2 suppression score\. Non\-English outputs are translated to English for readability\.
## Appendix 0\.FCheckpoint\-selection diagnostics

This section reports the SCE heatmaps used to inspect checkpoint trajectories before selecting suppression\-tradeoff checkpoints for full held\-out generation\. Each heatmap is computed on the compact checkpoint\-selection slice: up to 35 target\-matched L0–L2 generations per model–narrative–language–method– checkpoint cell, depending on model\-language coverage\. Brighter cells indicate higher SCE\.

![Refer to caption](https://arxiv.org/html/2607.22657v1/x2.png)\(a\)Lapa
![Refer to caption](https://arxiv.org/html/2607.22657v1/x3.png)\(b\)Gemma\-12B
![Refer to caption](https://arxiv.org/html/2607.22657v1/x4.png)\(c\)Qwen\-14B
![Refer to caption](https://arxiv.org/html/2607.22657v1/x5.png)\(d\)TAIDE\-Gemma

Figure 3:Checkpoint\-selection SCE heatmaps\. Rows are method trajectories and columns are checkpoint steps, split by narrative and language\. Brighter cells indicate higher SCE\.

Similar Articles

MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs

arXiv cs.AI

MLUBench is a large-scale benchmark for lifelong unlearning in multimodal large language models (MLLMs), featuring 127 entities across 9 classes. The paper identifies that existing unlearning methods suffer from cumulative degradation and proposes LUMoE to mitigate this, showing significant improvements.

Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibility

arXiv cs.CL

This paper studies multilingual unlearning in LLMs by extending the TOFU benchmark to five languages. It finds that unlearning transfer varies by script and family, operates primarily in later decoding layers, and that a single steering direction can recover much of the suppressed knowledge across languages.