Position: The Term "Machine Unlearning" Is Overused in LLMs

# Position: The Term “Machine Unlearning” Is Overused in LLMs
Source: [https://arxiv.org/html/2606.27379](https://arxiv.org/html/2606.27379)
###### Abstract

Large language models increasingly face demands to “forget” training data, knowledge, or behaviors due to regulatory deletion obligations, copyright/licensing disputes, and safety or product\-policy requirements\.This position paper argues that*machine unlearning*is overused as a term in LLM research and should be reserved for dataset\-defined deletion: removing the training influence of a precisely specified forget set such that the resulting model is \(approximately\) indistinguishable from retraining without that data\.We contend that many tasks currently labeled “unlearning” \(e\.g\., refusal for harmful requests, entity/knowledge removal, or targeted suppression\) pursue different, often policy\-dependent objectives and therefore require different terminology and baselines \(e\.g\., alignment, suppression, editing, obfuscation\)\. We further argue that this confusion is not cosmetic: because papers make different implicit guarantees under the same label, metrics and benchmarks are frequently reused outside their intended scope, rewarding surface\-level non\-disclosure \(e\.g\., low ROUGE/forget accuracy\) even when retraining\-equivalence is not tested and derived capabilities remain\. We conclude by calling for stricter terminology tied to explicit guarantees and reference models, and for evaluations that match the claimed objective\.

Machine Learning, ICML

## 1Introduction

Foundation models\(Achiamet al\.,[2023](https://arxiv.org/html/2606.27379#bib.bib1); Liuet al\.,[2024a](https://arxiv.org/html/2606.27379#bib.bib91); Comaniciet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib90)\)are trained on large, heterogeneous corpora assembled under mixed licenses, consents, and contractual constraints\. As these models are deployed in regulated and commercial settings, service providers increasingly face requests to*remove*the effect of specific training data, motivated by privacy deletion obligations \(e\.g\., the right to be forgotten\(European Parliament and Council of the European Union,[2016](https://arxiv.org/html/2606.27379#bib.bib92)\)\), copyright and licensing disputes\(*Tremblay v\. OpenAI, Inc\.*,[2023](https://arxiv.org/html/2606.27379#bib.bib9);*Kadrey v\. Meta Platforms, Inc\.*,[2023](https://arxiv.org/html/2606.27379#bib.bib10); Grynbaum and Mac,[2023](https://arxiv.org/html/2606.27379#bib.bib8)\), and enterprise data\-governance requirements\(Voigt and Von dem Bussche,[2017](https://arxiv.org/html/2606.27379#bib.bib7)\)\. These pressures have sharpened interest in*machine unlearning*as a principled way to remove the influence of selected training data\.

In the classical machine learning formulation,*machine unlearning*is a dataset\-defined deletion problem\. Given a training setDDand a precisely specified*forget set*F⊂DF\\subset D, the goal is to produce an updated model whose behavior is \(approximately\) indistinguishable from the counterfactual model obtained by retraining from scratch onD∖FD\\setminus F\(Ginartet al\.,[2019](https://arxiv.org/html/2606.27379#bib.bib89); Guoet al\.,[2020](https://arxiv.org/html/2606.27379#bib.bib82); Neelet al\.,[2021](https://arxiv.org/html/2606.27379#bib.bib94); Ullahet al\.,[2021](https://arxiv.org/html/2606.27379#bib.bib95); Izzoet al\.,[2021](https://arxiv.org/html/2606.27379#bib.bib80)\)\. This definition fixes both the*target*and the*baseline*: it requires removing the training influence of a concrete subset of data, and it judges success by similarity to a model retrained onD∖FD\\setminus F\(or a principled proxy\), rather than by whether the outputs satisfy a chosen policy\.

However, in recent LLM research, the word “unlearning” is frequently used for a broader range of objectives that share a high\-level motivation \(“make the model forget X”\) but do not match the retraining\-based guarantee\. Examples include preventing harmful behaviors, suppressing specific knowledge, removing entities, or blocking classes of queries\(Liet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib44); Jinet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib71); Choiet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib100)\)\. These directions are important in practice, especially for safety and product policy, but they typically target*behavioral constraints*rather than dataset\-defined deletion\. When these objectives are discussed under the same term as machine unlearning, claims and evaluations become difficult to interpret: readers cannot tell whether a method aims to match retraining onD∖FD\\setminus F, or merely to change what the system says under a particular prompting protocol\.

A central reason is that many non\-compliance “forgetting” requests are inherently*policy\-defined*and application\-dependent\(Liet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib44); Jinet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib71); Luoet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib83)\)\. For instance, “forget harmful behavior” \(e\.g\., bomb\-making assistance\) requires choosing a boundary: should the system block only step\-by\-step weaponization instructions, or also broadly relevant chemistry knowledge? Likewise, “forget knowledge” is ambiguous under entailment: if the target is “Paris is the capital of France,” should the system also avoid entailed statements such as “the Eiffel Tower is in the capital of France”? Entity removal is similarly underspecified: “forget Stephen King” could refer to biographical facts, his works, quotations, or derivative discussion \(e\.g\., adaptations\)\. This subjectivity can make it difficult to specify a precise forget setFF, but that is not the core issue\. More fundamentally, the objective is defined by an application policy \(i\.e\., what the model should or should not do\) so the problem is inherently about policy specification and compliance rather than dataset\-defined deletion\.

The gap between dataset\-defined deletion and policy\-defined behavior control is clearest for*derived capabilities*, where training influence is not limited to memorizing the forget set\(Thakeret al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib97); Jiaet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib96)\)\. For example, suppose a model is trained on unauthorized synthetic mathematical reasoning traces and is later required to “unlearn” them\. If evaluation only checks whether the model fails to answer the same questions from that dataset, a trivial non\-disclosure strategy can appear successful\. The relevant question is whether the unauthorized data contributed to a*transferable*reasoning capability: if the model still solves broad classes of challenging math problems, influence may persist even when direct reproduction is blocked\. Under retraining\-indistinguishability, maintaining such a capability is acceptable only if the retrained model onD∖FD\\setminus Fachieves it; otherwise, the capability should disappear along with the influence that induced it\.

This terminological ambiguity also directly affects benchmarks and metrics\. Many evaluations operationalize “forgetting” as output failure on a designated probe set, e\.g\., lower QA accuracy, lower ROUGE to a reference answer, or reduced likelihood of a target phrase\(Jinet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib71); Yuanet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib47); Xuet al\.,[2026](https://arxiv.org/html/2606.27379#bib.bib88)\)\. Such measures can be useful diagnostics for non\-disclosure, but they are not evidence of retraining equivalence\. They are also often subjective \(depending on QA construction and prompting context\), and*lower is not always better*: a retrained model onD∖FD\\setminus Fmay still produce partially correct or contextually reasonable answers \(hence non\-zero ROUGE\), while a blanket refusal can drive ROUGE toward zero while diverging from the retrained reference\. Benchmarks therefore add retain/utility constraints\(Mainiet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib31); Shiet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib70); Chang and Lee,[2025](https://arxiv.org/html/2606.27379#bib.bib99)\), but these too encode application\-dependent choices about what counts as utility and what trade\-offs are acceptable\. Without an explicit retrain reference, evaluation can unintentionally prioritize output control over removal of training influence\.

In this position paper, we argue that resolving this confusion requires stricter terminology tied to explicit guarantees and baselines\. We formalize machine unlearning as retraining\-indistinguishability for a precisely defined forget set, organize other common “unlearning” usages by intent, and explain why benchmark design must reflect the distinction, especially in the presence of derived capabilities\.

Position\.“Machine unlearning” should mean retraining indistinguishability for a precisely defined forget set; other safety\- or application\-driven “forgetting” goals are different problems and should use different terms\.

## 2What the Literature Calls “Unlearning”: A Definition and a Taxonomy by Intent

The term*unlearning*is now used in LLM research to describe a wide range of interventions that share a surface\-level motivation, making some information, behavior, or training influence “go away,” but differ substantially in their intended guarantee\. This terminology overload matters because different intents demand different baselines and different evaluations: a method designed to*block disclosure*can look successful under standard “forget set” tests while failing to remove the*training influence*of the forget set\. In this section, we \(i\) give a*formal definition*ofmachine unlearning\(the usage we argue should be preserved\), and \(ii\) organize other common uses of the term into*high\-level categories by intent*, without attempting rigid, mutually exclusive formalization\. I\.e\., “unlearning” is not equivalent to making a model refuse certain questions, nor to reducing the likelihood of a particular string, nor to overwriting an answer with a replacement\.

### 2\.1Unlearning: Dataset\-Defined Deletion Guarantee

#### Setup\.

LetDDbe the training dataset and letF⊆DF\\subseteq Dbe the*forget set*, whose*training influence*is to be removed\. Define the retain set asR:=D∖FR:=D\\setminus F\. Let𝖳𝗋𝖺𝗂𝗇​\(⋅\)\\mathsf\{Train\}\(\\cdot\)denote the \(randomized\) training procedure, and writeΘS∼𝖳𝗋𝖺𝗂𝗇​\(S\)\\Theta\_\{S\}\\sim\\mathsf\{Train\}\(S\)for the random model obtained by training on datasetSS\. An unlearning is a \(possibly randomized\) procedure that takes a trained model and a forget set and returns an updated model:

Θ′←𝖴𝗇𝗅𝖾𝖺𝗋𝗇​\(ΘD,F\)\.\\Theta^\{\\prime\}\\leftarrow\\mathsf\{Unlearn\}\(\\Theta\_\{D\},F\)\.
Informally, machine unlearning aims to remove the influence of training onFFas if the model had never seen it\.

###### Definition 2\.1\(Exact machine unlearning\(Izzoet al\.,[2021](https://arxiv.org/html/2606.27379#bib.bib80)\)\)\.

𝖴𝗇𝗅𝖾𝖺𝗋𝗇\\mathsf\{Unlearn\}achieves*exact machine unlearning*\(with respect to𝖳𝗋𝖺𝗂𝗇\\mathsf\{Train\}\) if for allDDand allF⊆DF\\subseteq D,

ℒ​\(Θ′\)=ℒ​\(ΘR\),where​ΘR∼𝖳𝗋𝖺𝗂𝗇​\(R\),\\mathcal\{L\}\(\\Theta^\{\\prime\}\)=\\mathcal\{L\}\(\\Theta\_\{R\}\),\\quad\\text\{ where \}\\Theta\_\{R\}\\sim\\mathsf\{Train\}\(R\),andℒ​\(⋅\)\\mathcal\{L\}\(\\cdot\)denotes the induced distribution over model parameters \(and over the randomized training outcome\)\.

In practice, exact unlearning is a very strong requirement and is rarely attainable for large\-scale models\. Accordingly, most work adopts relaxed notions of unlearning that allow the unlearned model to be*approximately*indistinguishable from the retrained baseline\.

###### Definition 2\.2\(Approximate machine unlearning \(general form\)\)\.

Fix a divergence/metricDist\\mathrm\{Dist\}between distributions and a toleranceτ≥0\\tau\\geq 0\.𝖴𝗇𝗅𝖾𝖺𝗋𝗇\\mathsf\{Unlearn\}achieves*\(Dist,τ\)\(\\mathrm\{Dist\},\\tau\)\-approximate machine unlearning*if for allDDandF⊆DF\\subseteq D,

Dist​\(ℒ​\(Θ′\),ℒ​\(ΘR\)\)≤τ\.\\mathrm\{Dist\}\\\!\\left\(\\mathcal\{L\}\(\\Theta^\{\\prime\}\),\\,\\mathcal\{L\}\(\\Theta\_\{R\}\)\\right\)\\leq\\tau\.Here,Dist\\mathrm\{Dist\}may be defined in parameter space or in*behavior space*\(e\.g\., after composing the model with a prompt distribution and a decoding rule\)\. We emphasize that*multiple*choices ofDist\\mathrm\{Dist\}are reasonable; the key is that the baseline is always retraining onD∖FD\\setminus F\.

One widely used relaxation \(inspired by differential privacy\(Dwork,[2006](https://arxiv.org/html/2606.27379#bib.bib98)\)\) defines closeness via\(ε,δ\)\(\\varepsilon,\\delta\)\-indistinguishability\. For random variablesXXandYY, writeX≈ε,δYX\\approx\_\{\\varepsilon,\\delta\}Yif for all measurable setsSS,

Pr⁡\[X∈S\]≤eε​Pr⁡\[Y∈S\]\+δ\\displaystyle\\Pr\[X\\in S\]\\leq e^\{\\varepsilon\}\\Pr\[Y\\in S\]\+\\deltaPr⁡\[Y∈S\]≤eε​Pr⁡\[X∈S\]\+δ\.\\displaystyle\\Pr\[Y\\in S\]\\leq e^\{\\varepsilon\}\\Pr\[X\\in S\]\+\\delta\.Then𝖴𝗇𝗅𝖾𝖺𝗋𝗇\\mathsf\{Unlearn\}is*\(ε,δ\)\(\\varepsilon,\\delta\)\-approximate*ifΘ′≈ε,δΘR\\Theta^\{\\prime\}\\approx\_\{\\varepsilon,\\delta\}\\Theta\_\{R\}\. This is a principled and popular choice, but it is*not the only*way to formalize approximate unlearning\.

Interventions that change model behavior or outputs can serve many practical goals, but they are*not equivalent*to machine unlearning as defined above\. The defining criterion of machine unlearning is removal of the influence of a*precisely specified*forget setFF, operationalized as \(approximate\) indistinguishability from the counterfactual model retrained onD∖FD\\setminus Funder an explicit notion of distance\.

### 2\.2Other Common Uses of “Unlearning” in LLM Papers: Categories by Intent

We now summarize several common intents that are frequently labeled “unlearning” in the LLM literature\. These categories are intentionally*high\-level*and*not mutually exclusive*: a single system may combine multiple mechanisms \(e\.g\., a suppression\-style fine\-tune plus a refusal policy\), and the boundary between, say, suppression and alignment can be blurry\. Our goal is not to legislate a perfect partition, but to clarify the*primary aim*that distinguishes these lines of work from machine unlearning as defined in[Section2\.1](https://arxiv.org/html/2606.27379#S2.SS1)\.

#### Output Likelihood Suppression\.

Suppression\-oriented approaches aim to reduce a model’s tendency to generate content associated with a designated forget target\. While suppression can be implemented in various ways, we focus on methods that*explicitly modify the model’s likelihood over selected outputs*\(Wellecket al\.,[2020](https://arxiv.org/html/2606.27379#bib.bib48)\)\. Common instantiations include gradient\-based updates such as gradient ascent \(GA\)\(Thudiet al\.,[2022](https://arxiv.org/html/2606.27379#bib.bib49); Janget al\.,[2023](https://arxiv.org/html/2606.27379#bib.bib35); Yaoet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib19); Zhanget al\.,[2024b](https://arxiv.org/html/2606.27379#bib.bib50)\)and negative preference optimization \(NPO\)\(Zhanget al\.,[2024a](https://arxiv.org/html/2606.27379#bib.bib36); Fanet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib38); Wanget al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib40)\)\. These methods can be effective at shifting probability mass away from restricted tokens or responses under specific prompts, but they operate primarily at the level of output distributions\. As a result, they do not, in general, guarantee indistinguishability from a model retrained without that data\.

#### Internal Representation Obfuscation\.

Obfuscation\-oriented works aim to make the model*unreliable or non\-informative*on targeted inputs by inducing confusion, such as low\-confidence or high\-entropy behavior\. This is commonly achieved through representation\-level manipulations that distort internal activations\(Liet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib44); Zouet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib56); Wuerkaixiet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib72); Huu\-Tienet al\.,[2025a](https://arxiv.org/html/2606.27379#bib.bib54),[b](https://arxiv.org/html/2606.27379#bib.bib55)\)or entropy\-based objectives that encourage diffuse predictions\(Yuanet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib47); Entesariet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib57); Zhaiet al\.,[2026](https://arxiv.org/html/2606.27379#bib.bib51)\)\. Such approaches degrade answerability on selected prompts but do not match the behavior of a retrained model\.

#### Knowledge Editing\.

Editing\-based approaches aim to*recalibrate semantic associations*, modifying the model’s assertions about specific entities or facts\. This is typically achieved through knowledge editing techniques\(Liet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib58); Hossain and Kagal,[2025](https://arxiv.org/html/2606.27379#bib.bib59); Junget al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib60)\)or via replacement supervision such as counterfactual fine\-tuning that induces updated responses\(Eldan and Russinovich,[2023](https://arxiv.org/html/2606.27379#bib.bib32); Guet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib61); Scholtenet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib63); Liuet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib45)\)\. These methods can effectively redirect a model’s outputs toward desired answers, but they do so by overwriting behaviors rather than removing the influence of the forget set\.

#### Behavioral Refusal\.

Refusal strategies enforce systematic non\-compliance, where the model is trained to provide “I don’t know” \(IDK\) responses to forget related queries\(Mainiet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib31); Yuanet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib47)\)\. Relatedly, LUNAR\(Shenet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib79)\)induces coherent abstention by redirecting the internal activations of forget\-set prompts toward regions that express an inability to answer\. This can also be implemented with preference optimization variants \(e\.g\., DPO\(Rafailovet al\.,[2023](https://arxiv.org/html/2606.27379#bib.bib20)\)\) that explicitly prefer refusal templates over forget answers\. These methods primarily alter the model’s*abstention behavior*on targeted prompts and are best viewed as policy\-oriented alignment\.

#### Inference\-time Interventions\.

Inference\-time interventions control*deployed system behavior*without modifying model parameters\. Examples include guardrails and input/output filters\(Thakeret al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib68)\), token\-level decoding constraints\(Denget al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib64)\), prompt\-side manipulations\(Pawelczyket al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib67); Liuet al\.,[2024b](https://arxiv.org/html/2606.27379#bib.bib41)\), and logit differencing during generation\(Jiet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib66); Suriyakumaret al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib65)\)\. These methods can prevent explicit disclosure of forget\-related content, but because the underlying weights are unchanged, they function as external filters rather than removal of learned information\.

### 2\.3Interpreting Unlearning Claims by Guarantees

Given the diversity of objectives labeled as “unlearning,” a practical reading strategy is to identify the*claimed guarantee*and its*reference baseline*\. Claims grounded in \(approximate\) indistinguishability from retraining onD∖FD\\setminus Fcorrespond to machine unlearning as defined above; claims evaluated primarily by non\-disclosure or behavior change without a retrain reference should be interpreted as pursuing a different objective and assessed on those stated terms\.

## 3Why Terminology Matters for LLMs Unlearning Benchmarks and Evaluation

Terminology choices directly shape benchmark design: when multiple objectives are grouped under “unlearning,” evaluations tend to collapse to what is easiest to measure, often treating “forgetting” as failure to produce reference answers on a designated probe set\. This section examines how common benchmarks and metrics reflect that choice, and why such scores can favor output\-control solutions even when the intended claim is removal of training influence\.

### 3\.1Output\-Failure Metrics Dominate Current Practice

Across recent “LLM unlearning” evaluations, the most common success signal is*output failure on forget queries*: a model is deemed to have “forgotten” if it no longer reproduces the ground\-truth answer for prompts related to the deletion target\. Typical instantiations include: \(i\) surface\-form similarity between generations and the ground\-truth answer, \(ii\) embedding\-based semantic similarity or cosine distance between generated outputs and the ground\-truth answer, and \(iii\) the model\-assigned probability or likelihood of the ground\-truth answer under the forget prompt\.

These metrics are informative for diagnosing*what the model emits*under a specific prompting protocol\. However, when behavior on forget queries is interpreted in absolute terms, with differences from a reference answer taken as evidence of success, and without any comparison to a retrained model, these metrics fail to establish the defining criterion of machine unlearning:*indistinguishability from a retrained model*\. Under this interpretation, methods can perform well on output\-failure metrics through mechanisms that are orthogonal to influence removal\. Such behaviors may be desirable for suppression or refusal objectives, but they should not be conflated with machine unlearning\.

Once output\-failure metrics become the de facto score, approaches explicitly designed to block forget\-related responses are structurally advantaged: they optimize exactly what the benchmark counts\. This creates a feedback loop where “better unlearning” often means “more reliable output suppression” under the benchmark’s prompting distribution, rather than a closer approximation to the retrain\-model behavior implied by machine unlearning\.

### 3\.2What Leading Benchmarks Actually Measure

We summarize representative benchmarks and their scoring protocols to clarify how metric choices can blur the distinction between retraining\-equivalence and output control\.

#### TOFU\.

TOFU\(Mainiet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib31)\)explicitly defines a forget set and evaluates against a*retrained*reference, which is directionally consistent with the machine unlearning definition\. Its core metric,*forget quality*, compares the unlearned and retrained models via a probability\-based*truth ratio*\(the model\-assigned probability of the correct answer relative to incorrect alternatives\), and assesses similarity using two\-sample tests \(e\.g\., KS tests\) on the resulting distributions\.

However, several subsequent works\(Yuanet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib47); Wuerkaixiet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib72)\)apply TOFU while omitting the retrained reference and instead judge forgetting by output non\-reproduction\. This shifts the evaluation away from retraining equivalence toward surface\-level output failure, thereby allowing output suppression to score as successful “unlearning” despite not matching the retrained model\.

#### MUSE\.

MUSE\(Shiet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib70)\)adopts a six\-way evaluation framework, including criteria such as \(i\) no verbatim memorization, \(ii\) no knowledge memorization, and \(iii\) no privacy leakage\. MUSE includes an explicit retrained baseline and evaluates privacy leakage by applying membership inference attacks \(MIAs\) with AUC\-based metrics, directly comparing the unlearned model against the retrained model\. In contrast, verbatim and knowledge memorization are operationalized via ROUGE\-L scores, where lower values on forget queries are taken as evidence of success without an explicit distributional comparison to the retrained model\.

#### RWKU\.

RWKU\(Jinet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib71)\)focuses on removing real\-world knowledge and stresses robustness through a broad suite of probes, including cloze and QA prompts, membership inference attacks, and adversarial elicitation\. Notably, RWKU does not rely on a retrained reference model; its scores therefore reflect how a model’s outputs change under the chosen probes, rather than how closely it matches a retrained baseline\. Accordingly, RWKU should be viewed as assessing robustness of knowledge suppression, not machine unlearning under a retrain\-equivalence\.

#### WMDP\.

WMDP\(Liet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib44)\)casts “unlearning” as reducing hazardous capabilities, operationalized as lower QA accuracy on questions related to biological, chemical, and cyber security domains\. This framing does not correspond to deletion of a well\-defined training subset\. Instead, WMDP evaluates whether task performance is diminished on targeted domains, and thus measures capability suppression rather than retrain\-equivalent machine unlearning\.

### 3\.3Adversarial Evaluation Exposes the Limits of Output\-Failure Scoring

The gap between output\-failure metrics and actual influence removal becomes most apparent under adversarial or stress\-test evaluation\. Methods that appear successful under fixed prompting protocols often fail when the evaluation setting is perturbed, revealing that the target knowledge has been suppressed rather than unlearned\.

Simple input\-level variations\(Mainiet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib31); Lynchet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib73); Łuckiet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib74); Jeunget al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib69)\), such as paraphrasing, mixed queries, multilingual queries, or jailbreak prompting, are frequently sufficient to recover supposedly forgotten content\. This observation is consistent with the benchmark analysis above: criteria that judge forgetting based solely on output non\-reproduction do not reliably track whether the underlying training influence persists\.

More revealingly, model\-level interventions further expose the brittleness of output\-failure\-based evaluation\. Prior works show that small amounts of additional fine\-tuning\(Yoonet al\.,[2026](https://arxiv.org/html/2606.27379#bib.bib104)\), benign post\-training transformations such as quantization\(Zhanget al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib76)\), or targeted activation\-level manipulations\(Seyitoğluet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib77)\), or representation\-level auditing\(Goelet al\.,[2026](https://arxiv.org/html/2606.27379#bib.bib105)\)can reveal information that was assumed to be forgotten\. Such phenomena indicate that failure to emit a reference answer does not imply that the corresponding knowledge has been eliminated from the model\.

## 4Derived Capabilities: Unlearning Beyond Surface\-Level Outputs

Under the machine unlearning definition, the target is the*training influence*of a precisely specified forget setFF, not individual responses\. This matters because training influence can be distributed and may persist as generalizable behaviors that extend well beyond the original examples\. Consequently, output\-based evaluations alone cannot determine whether the influence ofFFhas been removed\.

### 4\.1What We Mean by “Derived Capabilities”

We use the term*derived capability*to denote a form of behavioral competence that is plausibly attributable to training onFF\(or its interaction with the rest of training\), and that generalizes beyond the exact examples contained inFF\. Such capabilities need not take the form of verbatim memorization\. They may instead manifest as transferable reasoning skills learned from reasoning traces, trigger\-conditioned or adversarial behaviors induced by a small number of poisoned samples, persistent stylistic or tool\-use habits, or latent factual competence that can be recovered under paraphrase or benign post\-training interventions even when direct regurgitation is suppressed\. These phenomena are conceptually important because they reveal why “unlearning = not answering” is an incomplete operationalization\.

### 4\.2Implications of the Machine\-Unlearning Definition

Machine unlearning is defined by \(approximate\) equivalence to retraining onR=D∖FR=D\\setminus F\. This has an unavoidable but clarifying implication:

> *If training onFFcontributes to a derived capability, then an unlearned model that is truly indistinguishable from retraining withoutFFmust also lose that capability \(to the extent that the retrained model lacks it\)\.*

This implication is often uncomfortable in LLM settings because many forget sets are entangled with desirable performance\. Nevertheless, it is logically consistent with the compliance\-driven unlearning objective: the correct reference is what would have happened had the model never been trained onFF, even if that counterfactual model is less capable on some tasks\.

This also highlights a terminological boundary\. If an algorithm is designed to*preserve*a capability that is, in fact, attributable toFF, then the algorithm may still be a very useful*suppression/editing/alignment*technique, but it is not solving machine unlearning as defined in Section[2\.1](https://arxiv.org/html/2606.27379#S2.SS1)\.

### 4\.3Case: Unauthorized Synthetic Reasoning Traces

Derived capabilities are especially salient when training involves synthetic supervision\. Consider a setting in which mathematical reasoning traces generated by a frontier LLM are used, without authorization, to train another model, resulting in a measurable improvement in mathematical reasoning performance\. In practice, several model providers explicitly prohibit the use of their outputs to train or fine\-tune competing models, and such use may later trigger a request to “unlearn” the unauthorized data\.

In this scenario, the central question is not whether the model can reproduce specific solutions from the synthetic dataset\. Rather, it is whether the unauthorized reasoning traces contributed to a general mathematical reasoning capability that transfers beyond the original examples\.

If evaluation treats output failure on the unauthorized data as the sole success criterion, a model can appear successfully unlearned by simply refusing to answer or producing irrelevant responses, while retaining the improved reasoning ability\. Such evaluation therefore fails to test whether the training influence on a transferable capability has been removed\. This case illustrates why output non\-reproduction is an unreliable proxy for unlearning whenever the effect of the forget set manifests as a derived capability\.

### 4\.4Case: Poisoning as Derived \(Adversarial\) Capability

Data poisoning offers a clear example of derived capability because the intended effect is explicitly*not*memorization\. Poisoned samples are crafted to induce indirect behaviors, such as degraded accuracy, targeted misclassification, or trigger\-conditioned backdoors\. If unlearning is equivalent to retraining without the poisoned data, then removing the forget set should also eliminate the induced behavior\.

In practice, this standard is difficult to meet\. Recent work\(Pawelczyket al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib33)\)evaluates unlearning in settings where the forget set consists entirely of poison samples and measures whether the poisoning effects are mitigated\. Across multiple attack types and model architectures, existing unlearning methods often fail to remove the induced behaviors\. This makes poisoning\-based evaluation particularly informative: it probes whether training influence has been removed at the behavioral level, rather than whether a model merely avoids reproducing specific outputs\.

## 5Call for Action: Reference\-Based Evaluation and Derived\-Capability Probes

To make progress toward the stated goal of machine unlearning, we argue that evaluation protocols should \(i\) anchor claims to an explicit reference model that represents “training withoutFF” \(or the best available approximation\), and \(ii\) include tests for derived capabilities that capture training influence beyond surface\-level non\-disclosure\.

### 5\.1Evaluate Unlearning Against a Reference Model

For machine unlearning, success should be judged by how closely the unlearned model matches the behavior of a reference model intended to approximate the counterfactual model trained onD∖FD\\setminus F\. The ideal reference is a retrained\-from\-scratch model onD∖FD\\setminus F, but in many LLM settings this may be infeasible; in such cases, the reference should be the strongest reasonable proxy*and must be stated explicitly*\.

We do not claim that output\-level metrics \(e\.g\., ROUGE, cosine similarity, forget QA accuracy\) are intrinsically flawed\. The issue is treating them as a*stand\-alone*criterion for “unlearning” without any reference model\. In that setting, score drops primarily reflect output control under a particular probe distribution, not removal of training influence\.

When a paper claims machine unlearning \(rather than output control objectives\), we recommend reporting:

#### A reference model, with provenance:

ideally𝖳𝗋𝖺𝗂𝗇​\(D∖F\)\\mathsf\{Train\}\(D\\setminus F\)with matched hyperparameters\. If this is infeasible, use the best available proxy \(e\.g\., a matched\-stage retrain for fine\-tuning unlearning, a smaller\-scale retraining study, or the strongest checkpoint*before*the introduction ofFF\) and clearly state what it approximates and what it does not\.

#### Distances to the reference:

in addition to forget\-query scores, report distributional/behavioral comparisons to the reference model \(e\.g\., logit\- or probability\-based statistics, two\-sample tests, MIA\-style audits, and robustness under paraphrase and adversarial elicitation\(Mainiet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib31); Shiet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib70); Choet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib103)\)\), since the central question is similarity to the counterfactual baseline\.

#### Utility relative to the reference:

retain/utility metrics should be reported as part of the comparison to the reference model \(not merely as a separate “do not break the model” constraint\), since acceptable trade\-offs are application\-dependent and should be made explicit\.

Anchoring evaluation to an explicit reference model is necessary not only for detecting memorization, but also for interpreting capability\-level effects: if a capability is absent from the counterfactual reference, then retaining it after “unlearning” indicates remaining influence; if it is present in the reference, unlearning should not suppress it\.

### 5\.2Derived\-Capability Probes Should Be First\-Class

The preceding discussion leads to a simple recommendation: when the claim is removal of training influence, evaluation must probe derived capabilities, not only reproduction of the forget set\. Output\-level failure on forget queries is insufficient whenever the effect ofFFmanifests as a transferable behavior\. Concretely, we recommend that unlearning benchmarks include the following components:

#### Capability\-level holdout tasks\.

Benchmarks should evaluate capabilities plausibly induced byFFusing held\-out tasks that do not reuse prompts from the forget set\. Crucially, these capability\-level holdout tasks are*not*utility or retain\-accuracy evaluations: their purpose is not to measure general task performance, but to test whether a specific transferable capability attributable toFFpersists beyond direct reproduction\. In the synthetic reasoning case \(Section[4\.3](https://arxiv.org/html/2606.27379#S4.SS3)\), this entails testing on out\-of\-distribution mathematical reasoning problems rather than the original training questions\.

#### Intervention\-based recovery tests\.

Unlearning evaluations should include intervention\-based recovery tests\. If information or behaviors deemed “forgotten” can be recovered through benign fine\-tuning\(Huet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib75)\)or other post\-training transformations\(Zhanget al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib76)\), in contrast to a retrained reference model, this indicates that the influence ofFFremains latent rather than removed\.

#### Task\-appropriate threat models\.

In poisoning and backdoor settings, evaluation should directly measure the induced behavior \(e\.g\., trigger success rates, targeted error rates, or poison influence\), rather than relying on failure on curated QA prompts\. Such settings provide a stringent test of unlearning because the target is a derived behavior rather than surface\-level regurgitation\(Pawelczyket al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib33)\)\.

These probes are not intended to replace standard forget\-query metrics\. Rather, they guard against a systematic failure mode in which “unlearning” success is declared based on surface\-level output suppression, while distributed and transferable training effects persist\.

Finally, derived\-capability probes must be interpreted*relative to the retrain reference*\(Section[5\.1](https://arxiv.org/html/2606.27379#S5.SS1)\): the goal is not to maximize forgetting signals, but to match the counterfactual behavior of training onD∖FD\\setminus F\.

## 6Alternative Views

In this section, we describe several credible opposing views and explain where we agree, where we disagree, and what we think follows for evaluation and terminology\.

### 6\.1View 1: Policy\-driven removal as unlearning

#### Why this view is reasonable\.

Under a broad interpretation of “unlearning” as “making the model forget X,” policy\-driven objectives naturally fall under this label\. These include preventing unsafe responses, removing or attenuating knowledge about specific entities, and rewriting particular facts\. Such objectives are practically meaningful: they address safety, product policy, reputational risk, and user\-facing redaction requirements, and they often admit scalable solutions that do not require access to the training data\.

#### Our response\.

We agree that these directions are valuable and should continue to be pursued\. Our concern is not with the objectives themselves, but with the interchangeable use of terminology and the guarantees it implies\. Policy\-driven removal is typically defined by design choices about what a model should or should not produce, which are inherently subjective and application\-dependent\. Even approaches that rely on structured representations such as knowledge graphs or ontology\-based closures must commit to specific choices about scope, closure, and propagation, none of which are uniquely determined by the training data\(Weiet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib102); Luoet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib83)\)\.

By contrast, machine unlearning is defined with respect to an objective substrate: a precisely specified forget setF⊂DF\\subset D, with success evaluated relative to the counterfactual model retrained onD∖FD\\setminus F\. This distinction has concrete consequences for evaluation and interpretation\. Using the same term for both policy\-driven behavior control and dataset\-defined deletion blurs whether a method claims retraining equivalence or only compliance with a policy constraint, and makes results difficult to compare across works\.

### 6\.2View 2: Isn’t It Enough to Not Answer?

#### Why this view is reasonable\.

In many deployments, the primary risk is*exposure*: users should not be able to elicit certain content such as private data, copyrighted excerpts, or unsafe instructions\. From this perspective, “successful unlearning” can be viewed as an*interface\-level*requirement: the system should reliably avoid producing the targeted information when queried\. This interpretation is practical, aligns with product and safety objectives, and underlies many existing benchmarks\(Liet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib44); Jinet al\.,[2024](https://arxiv.org/html/2606.27379#bib.bib71)\)\.

#### Our response\.

We agree that non\-disclosure is a valuable and often sufficient objective when the goal is limited to controlling outputs\. However, it is not equivalent to machine unlearning as defined in Section[2\.1](https://arxiv.org/html/2606.27379#S2.SS1), nor is it sufficient when the application requires removal of training influence or robustness to post\-processing\.

Preventing answers on a fixed family of prompts primarily measures output control under a chosen elicitation protocol\. Such control can be achieved even when the training influence of the forget set remains latent\. As a result, apparent forgetting may fail under adversarial evaluation or distribution shift \([Section3\.3](https://arxiv.org/html/2606.27379#S3.SS3)\), where the same information or behavior can be recovered\.

More importantly, training influence may persist as derived capabilities \(Section[4](https://arxiv.org/html/2606.27379#S4)\) that are not tied to reproducing specific reference strings\. In these cases, a model can avoid answering forget prompts while retaining a transferable capability induced by the forget set\. If the application requires that such capabilities be removed, or that forgetting be robust to benign post\-processing, then a stronger notion,*indistinguishability*, is necessary\.

### 6\.3View 3: A Retrain Reference Is Infeasible for Real\-World Large Language Models

#### Why this view is reasonable\.

Retraining a frontier\-scale model after removing a subset of pretraining data is often prohibitively expensive or operationally infeasible\. In many settings, the original training pipeline, detailed data provenance, or sufficient compute may be unavailable\. From this perspective, defining unlearning in terms of retraining equivalence can appear impractical, motivating the use of operational criteria such as “the model no longer reveals the targeted content under our evaluation\.”

#### Our response\.

We distinguish*conceptual definitions*from*practical feasibility*\. Even when exact machine unlearning is difficult or unattainable, it remains a distinct scientific and compliance\-motivated objective\. Replacing that objective with easier operational goals does not resolve the difficulty; it obscures what guarantee is being claimed\.

We do not argue that every paper must retrain a frontier model\. Rather, terminology should track the guarantee\. When a retrain reference is unavailable, the appropriate response is to state and justify a proxy baseline, or to frame the contribution as addressing a different objective \(e\.g\., non\-disclosure\), rather than redefining “unlearning”\.

### 6\.4View 4: Is a Retrained Model the Right Reference?

#### Why this view is reasonable\.

A natural concern is that the retrained model𝖳𝗋𝖺𝗂𝗇​\(D∖F\)\\mathsf\{Train\}\(D\\setminus F\)may not align with what stakeholders*want*a deployed system to do, even if it is the correct counterfactual for influence removal\.

First, removing information can increase hallucination risk: a retrained \(or successfully unlearned\) model may produce plausible but incorrect answers simply because it lacks relevant evidence that was present inFF\(Yuanet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib47)\)\. If a deletion request is motivated by compliance or user trust, such hallucinations may be undesirable\.

Second, a retrained model may still answer “forget” questions when the same information appears elsewhere inD∖FD\\setminus F\(e\.g\., duplicates, paraphrases, or independently sourced references\)\. In that case, generating similar or even identical answers is consistent with training on authorized data, and residual similarity can persist even after removingFF\(Jeunget al\.,[2026](https://arxiv.org/html/2606.27379#bib.bib101); Cooperet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib37)\)\. From a non\-disclosure perspective, this can look like “forgetting failed,” even though the deleted subset was removed\.

#### Our response\.

We agree that retrained models can hallucinate and can still produce similar outputs, but these issues are orthogonal to the objective of machine unlearning\. Machine unlearning is not a guarantee of factuality or a promise to never produce an answer; it is a claim about*removing the influence of a precisely specified dataset subsetFF*\. Accordingly, \(i\) hallucinations caused by missing evidence are an expected consequence of removing information, and the relevant question is whether the unlearned model behaves like the counterfactual retrained model under the same conditions; and \(ii\) if the relevant information is present inD∖FD\\setminus F, then both the retrained model and a correctly unlearned model should be able to answer using authorized data\.

If an application instead requires stronger behavior \(e\.g\., abstaining from answering even when the information exists in authorized data, or enforcing strict non\-disclosure regardless of counterfactual training\), then the objective moves beyond “removing influence ofFF” into the domain of policy and preference design\. Our recommendation is not to adjudicate which objective is preferable, but to distinguish them terminologically and evaluate them against the baseline\.

### 6\.5View 5: Why Not Keep “Unlearning” as an Umbrella Term and Add Qualifiers?

#### Why this view is reasonable\.

The term “unlearning” is already widely used, and adopting it as an umbrella label could reduce terminological fragmentation\. A natural proposal is to add qualifiers \(e\.g\.,*data unlearning*vs\.*behavior unlearning*\) to preserve continuity while improving clarity\.

#### Our response\.

We view this as a reasonable compromise, and it is compatible with our intent if qualifiers are used consistently and are tied to explicit guarantees and baselines\. Our concern is not the word “unlearning” itself, but the frequent mismatch between the implied guarantee and the evaluation protocol\. For example, papers sometimes motivate their goal as dataset deletion or retraining\-indistinguishability, yet report success primarily by driving ROUGE or forget\-query accuracy toward zero, a criterion that can be satisfied by output non\-disclosure and may diverge from the behavior of a model retrained onD∖FD\\setminus F\. In such cases, qualifiers alone do not prevent confusion unless they also constrain what counts as evidence\.

In our view, the umbrella\-term approach only works if “machine/data unlearning” is reserved for retrain\-referenced deletion claims, with evaluation centered on similarity to that reference, while output\-control objectives are explicitly labeled and evaluated on their own terms\.

## 7Conclusion

We argue that*machine unlearning*should be reserved for dataset\-defined deletion: given a precisely specified forget setF⊂DF\\subset D, the goal is to remove its training influence by producing a model that is \(approximately\) indistinguishable from retraining onD∖FD\\setminus F; other “forgetting” objectives may be valuable but require different guarantees and terminology\. This distinction matters because many benchmarks reward surface\-level non\-disclosure on forget prompts, which can be achieved without removing influence and can miss derived capabilities; accordingly, unlearning claims should be evaluated against an explicit retraining reference \(or a clearly stated proxy\) and include derived\-capability probes when influence removal is the goal\.

## Acknowledgements

This work was supported in part by Institute of Information & communications Technology Planning & Evaluation \(IITP\) grant funded by the Korea government \(MSIT\) \(No\. RS\-2024\-00457882, AI Research Hub Project\), IITP grant funded by the Korean Government \(MSIT\) \(No\. RS\-2020\-II201361, Artificial Intelligence Graduate School Program \(Yonsei University\)\), and the National Research Foundation of Korea \(NRF\) grant funded by the Korea government \(MSIT\) \(No\. RS2025\-23525649\)\.

## References

- J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.\(2023\)GPT\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p1.1)\.
- H\. Chang and H\. Lee \(2025\)Which retain set matters for llm unlearning? a case study on entity unlearning\.arXiv preprint arXiv:2502\.11441\.Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p6.1)\.
- J\. Chen, Z\. Deng, K\. Zheng, Y\. Yan, S\. Liu, P\. Wu, P\. Jiang, J\. Liu, and X\. Hu \(2025\)SafeEraser: enhancing safety in multimodal large language models through multimodal machine unlearning\.InACL findings,Cited by:[Appendix B](https://arxiv.org/html/2606.27379#A2.p1.1)\.
- S\. Cho, D\. Hwang, F\. Sala, S\. Hwang, K\. Cho, and S\. Cha \(2025\)Reference\-specific unlearning metrics can hide the truth: a reality check\.arXiv preprint arXiv:2510\.12981\.Cited by:[§5\.1](https://arxiv.org/html/2606.27379#S5.SS1.SSS0.Px2.p1.1)\.
- M\. Choi, D\. Rim, D\. Lee, and J\. Choo \(2025\)Opt\-out: investigating entity\-level unlearning for large language models via optimal transport\.InACL main,Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p3.1)\.
- G\. Comanici, E\. Bieber, M\. Schaekermann, I\. Pasupat, N\. Sachdeva, I\. Dhillon, M\. Blistein, O\. Ram, D\. Zhang, E\. Rosen,et al\.\(2025\)Gemini 2\.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities\.arXiv preprint arXiv:2507\.06261\.Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p1.1)\.
- A\. F\. Cooper, C\. A\. Choquette\-Choo, M\. Bogen, K\. Klyman, M\. Jagielski, K\. Filippova, K\. Liu, A\. Chouldechova, J\. Hayes, Y\. Huang, E\. Triantafillou, P\. Kairouz, N\. E\. Mitchell, N\. Mireshghallah, A\. Z\. Jacobs, J\. Grimmelmann, V\. Shmatikov, C\. D\. Sa, I\. Shumailov, A\. Terzis, S\. Barocas, J\. W\. Vaughan, danah boyd, Y\. Choi, S\. Koyejo, F\. Delgado, P\. Liang, D\. E\. Ho, P\. Samuelson, M\. Brundage, D\. Bau, S\. Neel, H\. Wallach, A\. B\. Cyphert, M\. Lemley, N\. Papernot, and K\. Lee \(2025\)Machine unlearning doesn’t do what you think: lessons for generative AI policy and research\.InNeurIPS,Cited by:[§6\.4](https://arxiv.org/html/2606.27379#S6.SS4.SSS0.Px1.p3.2)\.
- Z\. Deng, C\. Y\. Liu, Z\. Pang, X\. He, L\. Feng, Q\. Xuan, Z\. Zhu, and J\. Wei \(2025\)GUARD: generation\-time LLM unlearning via adaptive restriction and detection\.InICML Workshop MUGen,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px5.p1.1)\.
- C\. Dwork \(2006\)Differential privacy\.InICALP,Cited by:[§2\.1](https://arxiv.org/html/2606.27379#S2.SS1.SSS0.Px1.p4.5)\.
- R\. Eldan and M\. Russinovich \(2023\)Who’s harry potter? approximate unlearning in llms\.External Links:2310\.02238Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px3.p1.1)\.
- T\. Entesari, A\. Hatami, R\. Khaziev, A\. Ramakrishna, and M\. Fazlyab \(2025\)Constrained entropic unlearning: a primal\-dual framework for large language models\.InNeurIPS,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px2.p1.1)\.
- European Parliament and Council of the European Union \(2016\)External Links:[Link](https://data.europa.eu/eli/reg/2016/679/oj)Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p1.1)\.
- C\. Fan, J\. Liu, L\. Lin, J\. Jia, R\. Zhang, S\. Mei, and S\. Liu \(2024\)Simplicity prevails: rethinking negative preference optimization for llm unlearning\.arXiv preprint arXiv:2410\.07163\.Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px1.p1.1)\.
- A\. Ginart, M\. Guan, G\. Valiant, and J\. Y\. Zou \(2019\)Making ai forget you: data deletion in machine learning\.InNeurIPS,Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p2.4)\.
- A\. Goel, A\. Ritter, and I\. Gurevych \(2026\)Auditing language model unlearning via information decomposition\.arXiv preprint arXiv:2601\.15111\.Cited by:[§3\.3](https://arxiv.org/html/2606.27379#S3.SS3.p3.1)\.
- M\. M\. Grynbaum and R\. Mac \(2023\)The times sues openai and microsoft over ai use of copyrighted work\.The New York Times27\.Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p1.1)\.
- T\. Gu, K\. Huang, R\. Luo, Y\. Yao, Y\. Yang, Y\. Teng, and Y\. Wang \(2024\)Meow: memory supervised llm unlearning via inverted facts\.arXiv preprint arXiv:2409\.11844\.Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px3.p1.1)\.
- C\. Guo, T\. Goldstein, A\. Hannun, and L\. Van Der Maaten \(2020\)Certified data removal from machine learning models\.InICML,Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p2.4)\.
- S\. Hossain and L\. Kagal \(2025\)Investigating model editing for unlearning in large language models\.InCOLM Workshop on SoLaR,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px3.p1.1)\.
- S\. Hu, Y\. Fu, S\. Wu, and V\. Smith \(2024\)Jogging the memory of unlearned llms through targeted relearning attacks\.InNeurIPS workshop SafeGenAi,Cited by:[§5\.2](https://arxiv.org/html/2606.27379#S5.SS2.SSS0.Px2.p1.1)\.
- J\. Huo, Y\. Yan, X\. Zheng, Y\. Lyu, X\. Zou, Z\. Wei, and X\. Hu \(2025\)MMUnlearner: reformulating multimodal machine unlearning in the era of multimodal large language models\.InACL findings,Cited by:[Appendix B](https://arxiv.org/html/2606.27379#A2.p1.1)\.
- D\. Huu\-Tien, T\. Pham, H\. Thanh\-Tung, and N\. Inoue \(2025a\)On effects of steering latent representation for large language model unlearning\.InAAAI,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px2.p1.1)\.
- D\. Huu\-Tien, H\. Thanh\-Tung, A\. Bui, M\. Nguyen, L\. Nguyen, and N\. Inoue \(2025b\)Improving llm unlearning robustness via random perturbations\.arXiv preprint arXiv:2501\.19202\.Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px2.p1.1)\.
- Z\. Izzo, M\. A\. Smart, K\. Chaudhuri, and J\. Zou \(2021\)Approximate data deletion from machine learning models\.InAISTATS,Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p2.4),[Definition 2\.1](https://arxiv.org/html/2606.27379#S2.Thmdefinition1)\.
- Joel\. Jang, Dongkeun\. Yoon, Sohee\. Yang, Sungmin\. Cha, Moontae\. Lee, Lajanugen\. Logeswaran, and M\. Seo \(2023\)Knowledge unlearning for mitigating privacy risks in language models\.InACL,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px1.p1.1)\.
- W\. Jeung, S\. Yoon, H\. Hong, S\. Kim, S\. Han, Y\. Yu, and A\. No \(2026\)Dusk: do not unlearn shared knowledge\.InACL Findings,Cited by:[§6\.4](https://arxiv.org/html/2606.27379#S6.SS4.SSS0.Px1.p3.2)\.
- W\. Jeung, S\. Yoon, and A\. No \(2025\)SEPS: a separability measure for robust unlearning in llms\.InEMNLP main,Cited by:[§3\.3](https://arxiv.org/html/2606.27379#S3.SS3.p2.1)\.
- J\. Ji, Y\. Liu, Y\. Zhang, G\. Liu, R\. R\. Kompella, S\. Liu, and S\. Chang \(2024\)Reversing the forget\-retain objectives: an efficient LLM unlearning framework from logit difference\.InNeurIPS,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px5.p1.1)\.
- H\. Jia, T\. Li, J\. Guan, and V\. Chandrasekaran \(2025\)The erasure illusion: stress\-testing the generalization of llm forgetting evaluation\.arXiv preprint arXiv:2512\.19025\.Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p5.1)\.
- Z\. Jin, P\. Cao, C\. Wang, Z\. He, H\. Yuan, J\. Li, Y\. Chen, K\. Liu, and J\. Zhao \(2024\)Rwku: benchmarking real\-world knowledge unlearning for large language models\.InNeurIPS,Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p3.1),[§1](https://arxiv.org/html/2606.27379#S1.p4.1),[§1](https://arxiv.org/html/2606.27379#S1.p6.1),[§3\.2](https://arxiv.org/html/2606.27379#S3.SS2.SSS0.Px3.p1.1),[§6\.2](https://arxiv.org/html/2606.27379#S6.SS2.SSS0.Px1.p1.1)\.
- D\. Jung, J\. Seo, J\. Lee, C\. Park, and H\. Lim \(2025\)CoME: an unlearning\-based approach to conflict\-free model editing\.InNAACL main,L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px3.p1.1)\.
- *Kadrey v\. Meta Platforms, Inc\.*\(2023\)Note:3:23\-cv\-03417Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p1.1)\.
- N\. Li, A\. Pan, A\. Gopal, S\. Yue, D\. Berrios, A\. Gatti, J\. D\. Li, A\. Dombrowski, S\. Goel, G\. Mukobi, N\. Helm\-Burger, R\. Lababidi, L\. Justen, A\. B\. Liu, M\. Chen, I\. Barrass, O\. Zhang, X\. Zhu, R\. Tamirisa, B\. Bharathi, A\. Herbert\-Voss, C\. B\. Breuer, A\. Zou, M\. Mazeika, Z\. Wang, P\. Oswal, W\. Lin, A\. A\. Hunt, J\. Tienken\-Harder, K\. Y\. Shih, K\. Talley, J\. Guan, I\. Steneker, D\. Campbell, B\. Jokubaitis, S\. Basart, S\. Fitz, P\. Kumaraguru, K\. K\. Karmakar, U\. Tupakula, V\. Varadharajan, Y\. Shoshitaishvili, J\. Ba, K\. M\. Esvelt, A\. Wang, and D\. Hendrycks \(2024\)The WMDP benchmark: measuring and reducing malicious use with unlearning\.InICML,Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p3.1),[§1](https://arxiv.org/html/2606.27379#S1.p4.1),[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2606.27379#S3.SS2.SSS0.Px4.p1.1),[§6\.2](https://arxiv.org/html/2606.27379#S6.SS2.SSS0.Px1.p1.1)\.
- Z\. Li, X\. Wang, W\. F\. Shen, M\. Kurmanji, X\. Qiu, D\. Cai, C\. Wu, and N\. D\. Lane \(2025\)Editing as unlearning: are knowledge editing methods strong baselines for large language model unlearning?\.InNeurIPS LLM Evaluation Workshop,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px3.p1.1)\.
- A\. Liu, B\. Feng, B\. Xue, B\. Wang, B\. Wu, C\. Lu, C\. Zhao, C\. Deng, C\. Zhang, C\. Ruan,et al\.\(2024a\)Deepseek\-v3 technical report\.arXiv preprint arXiv:2412\.19437\.Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p1.1)\.
- C\. Y\. Liu, Y\. Wang, J\. Flanigan, and Y\. Liu \(2024b\)Large language model unlearning via embedding\-corrupted prompts\.InNeurIPS,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px5.p1.1)\.
- Y\. Liu, H\. Chen, W\. Huang, Y\. Ni, and M\. Imani \(2025\)LUNE: efficient LLM unlearning via loRA fine\-tuning with negative examples\.InSocially Responsible and Trustworthy Foundation Models at NeurIPS,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px3.p1.1)\.
- J\. Łucki, B\. Wei, Y\. Huang, P\. Henderson, F\. Tramèr, and J\. Rando \(2025\)An adversarial perspective on machine unlearning for ai safety\.TMLR\.Cited by:[§3\.3](https://arxiv.org/html/2606.27379#S3.SS3.p2.1)\.
- Y\. Luo, Z\. Zhou, H\. Chen, K\. Qiu, M\. Savvides, S\. Li, and J\. Wang \(2025\)KnowledgeSmith: uncovering knowledge updating in llms with model editing and unlearning\.arXiv preprint arXiv:2510\.02392\.Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p4.1),[§6\.1](https://arxiv.org/html/2606.27379#S6.SS1.SSS0.Px2.p1.1)\.
- A\. Lynch, P\. Guo, A\. Ewart, S\. Casper, and D\. Hadfield\-Menell \(2024\)Eight methods to evaluate robust unlearning in llms\.arXiv preprint arXiv:2402\.16835\.Cited by:[§3\.3](https://arxiv.org/html/2606.27379#S3.SS3.p2.1)\.
- P\. Maini, Z\. Feng, A\. Schwarzschild, Z\. C\. Lipton, and J\. Z\. Kolter \(2024\)TOFU: a task of fictitious unlearning for LLMs\.InCOLM,Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p6.1),[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px4.p1.1),[§3\.2](https://arxiv.org/html/2606.27379#S3.SS2.SSS0.Px1.p1.1),[§3\.3](https://arxiv.org/html/2606.27379#S3.SS3.p2.1),[§5\.1](https://arxiv.org/html/2606.27379#S5.SS1.SSS0.Px2.p1.1)\.
- S\. Neel, A\. Roth, and S\. Sharifi\-Malvajerdi \(2021\)Descent\-to\-delete: gradient\-based methods for machine unlearning\.InALT,pp\. 931–962\.Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p2.4)\.
- M\. Pawelczyk, J\. Z\. Di, Y\. Lu, G\. Kamath, A\. Sekhari, and S\. Neel \(2025\)Machine unlearning fails to remove data poisoning attacks\.InICLR,Cited by:[§4\.4](https://arxiv.org/html/2606.27379#S4.SS4.p2.1),[§5\.2](https://arxiv.org/html/2606.27379#S5.SS2.SSS0.Px3.p1.1)\.
- M\. Pawelczyk, S\. Neel, and H\. Lakkaraju \(2024\)In\-context unlearning: language models as few\-shot unlearners\.InICML,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px5.p1.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.InNeurIPS,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px4.p1.1)\.
- Y\. Scholten, S\. Xhonneux, L\. Schwinn, and S\. Günnemann \(2025\)Model collapse is not a bug but a feature in machine unlearning for llms\.arXiv preprint arXiv:2507\.04219\.Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px3.p1.1)\.
- A\. Seyitoğlu, A\. Kuvshinov, L\. Schwinn, and S\. Günnemann \(2024\)Extracting unlearned information from LLMs with activation steering\.InNeurIPS Workshop SafeGenAi,Cited by:[§3\.3](https://arxiv.org/html/2606.27379#S3.SS3.p3.1)\.
- W\. F\. Shen, X\. Qiu, M\. Kurmanji, A\. Iacob, L\. Sani, Y\. Chen, N\. Cancedda, and N\. D\. Lane \(2025\)LLM unlearning via neural activation redirection\.InNeurIPS,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px4.p1.1)\.
- W\. Shi, J\. Lee, Y\. Huang, S\. Malladi, J\. Zhao, A\. Holtzman, D\. Liu, L\. Zettlemoyer, N\. A\. Smith, and C\. Zhang \(2025\)MUSE: machine unlearning six\-way evaluation for language models\.InICLR,Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p6.1),[§3\.2](https://arxiv.org/html/2606.27379#S3.SS2.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2606.27379#S5.SS1.SSS0.Px2.p1.1)\.
- V\. M\. Suriyakumar, A\. Sekhari, and A\. Wilson \(2025\)UCD: unlearning in llms via contrastive decoding\.arXiv preprint arXiv:2506\.12097\.Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px5.p1.1)\.
- P\. Thaker, S\. Hu, N\. Kale, Y\. Maurya, Z\. S\. Wu, and V\. Smith \(2025\)Position: llm unlearning benchmarks are weak measures of progress\.InIEEE SaTML,Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p5.1)\.
- P\. Thaker, Y\. Maurya, S\. Hu, Z\. S\. Wu, and V\. Smith \(2024\)Guardrail baselines for unlearning in llms\.InICLR workshop SeT\-LLM,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px5.p1.1)\.
- A\. Thudi, H\. Jia, I\. Shumailov, and N\. Papernot \(2022\)On the necessity of auditable algorithmic definitions for machine unlearning\.InUSENIX security symposium,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px1.p1.1)\.
- *Tremblay v\. OpenAI, Inc\.*\(2023\)Note:23\-cv\-03416\-AMO, \(N\.D\. Cal\.\)Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p1.1)\.
- E\. Ullah, T\. Mai, A\. Rao, R\. A\. Rossi, and R\. Arora \(2021\)Machine unlearning via algorithmic stability\.InCLT,Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p2.4)\.
- P\. Voigt and A\. Von dem Bussche \(2017\)The eu general data protection regulation \(gdpr\): a practical guide\.Springer Publishing Company, Incorporated\.Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p1.1)\.
- C\. Wang, Y\. Zhang, J\. Jia, P\. Ram, D\. Wei, Y\. Yao, S\. Pal, N\. Baracaldo, and S\. Liu \(2025\)Invariance makes LLM unlearning resilient even to unanticipated downstream fine\-tuning\.InICML,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px1.p1.1)\.
- R\. Wei, P\. Niu, H\. H\. Hsu, R\. Wu, H\. Yin, M\. Ghassemi, Y\. Li, V\. K\. Potluru, E\. Chien, K\. Chaudhuri, O\. Milenkovic, and P\. Li \(2025\)Do LLMs really forget? evaluating unlearning with knowledge correlation and confidence awareness\.InNeurIPS,Cited by:[§6\.1](https://arxiv.org/html/2606.27379#S6.SS1.SSS0.Px2.p1.1)\.
- S\. Welleck, I\. Kulikov, S\. Roller, E\. Dinan, K\. Cho, and J\. Weston \(2020\)Neural text generation with unlikelihood training\.InICML,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px1.p1.1)\.
- A\. Wuerkaixi, Q\. Wang, S\. Cui, W\. Xu, B\. Han, G\. Niu, M\. Sugiyama, and C\. Zhang \(2025\)Adaptive localization of knowledge negation for continual llm unlearning\.InICML,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2606.27379#S3.SS2.SSS0.Px1.p2.1)\.
- X\. Xu, M\. Du, Z\. Li, Z\. Liang, Z\. Guo, S\. Zhang, P\. Hu, Q\. Ye, and H\. Hu \(2026\)From domains to instances: dual\-granularity data synthesis for llm unlearning\.arXiv preprint arXiv:2601\.04278\.Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p6.1)\.
- Y\. Yao, X\. Xu, and Y\. Liu \(2024\)Large language model unlearning\.InNeurIPS,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px1.p1.1)\.
- S\. Yoon, H\. Hong, W\. Jeung, and A\. No \(2026\)Rethinking benign relearning: syntax as the hidden driver of unlearning failures\.InICLR,Cited by:[§3\.3](https://arxiv.org/html/2606.27379#S3.SS3.p3.1)\.
- X\. Yuan, T\. Pang, C\. Du, K\. Chen, W\. Zhang, and M\. Lin \(2025\)A closer look at machine unlearning for large language models\.InICLR,Cited by:[§1](https://arxiv.org/html/2606.27379#S1.p6.1),[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px2.p1.1),[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px4.p1.1),[§3\.2](https://arxiv.org/html/2606.27379#S3.SS2.SSS0.Px1.p2.1),[§6\.4](https://arxiv.org/html/2606.27379#S6.SS4.SSS0.Px1.p2.1)\.
- N\. Zhai, P\. Shao, B\. Zheng, F\. Shen, L\. Bai, and X\. Yang \(2026\)Maximizing local entropy where it matters: prefix\-aware localized llm unlearning\.arXiv preprint arXiv:2601\.03190\.Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px2.p1.1)\.
- R\. Zhang, L\. Lin, Y\. Bai, and S\. Mei \(2024a\)Negative preference optimization: from catastrophic collapse to effective unlearning\.InCOLM,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px1.p1.1)\.
- Z\. Zhang, J\. Yang, Y\. Lu, P\. Ke, S\. Cui, C\. Zheng, H\. Wang, and M\. Huang \(2024b\)From theft to bomb\-making: the ripple effect of unlearning in defending against jailbreak attacks\.arXiv preprint arXiv:2407\.02855\.Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px1.p1.1)\.
- Z\. Zhang, F\. Wang, X\. Li, Z\. Wu, X\. Tang, H\. Liu, Q\. He, W\. Yin, and S\. Wang \(2025\)Catastrophic failure of llm unlearning via quantization\.InICLR,Cited by:[§3\.3](https://arxiv.org/html/2606.27379#S3.SS3.p3.1),[§5\.2](https://arxiv.org/html/2606.27379#S5.SS2.SSS0.Px2.p1.1)\.
- A\. Zou, L\. Phan, J\. Wang, D\. Duenas, M\. Lin, M\. Andriushchenko, J\. Z\. Kolter, M\. Fredrikson, and D\. Hendrycks \(2024\)Improving alignment and robustness with circuit breakers\.InNeurIPS,Cited by:[§2\.2](https://arxiv.org/html/2606.27379#S2.SS2.SSS0.Px2.p1.1)\.

## Appendix AOperational Objectives for Common Non\-Unlearning Mechanisms

This appendix makes explicit the operational distinction behind the taxonomy in[Section2\.2](https://arxiv.org/html/2606.27379#S2.SS2)\. LetFFdenote the forget set and write each forget example as\(x,y\)∈F\(x,y\)\\in F, wherey=\(y1,…,yT\)y=\(y\_\{1\},\\ldots,y\_\{T\}\)\. The objectives below are representative mechanisms often called “unlearning,” but they optimize output suppression, overwriting, refusal, or deployment\-time control rather than similarity to the retrained counterfactual model onD∖FD\\setminus F\.

#### Output likelihood suppression\.

Suppression methods reduce the likelihood of outputs associated with the forget target:

ℒGA​\(Θ;F\)=𝔼\(x,y\)∼F​\[∑t=1Tlog⁡pΘ​\(yt∣x,y<t\)\]\.\\mathcal\{L\}\_\{\\mathrm\{GA\}\}\(\\Theta;F\)=\\mathbb\{E\}\_\{\(x,y\)\\sim F\}\\left\[\\sum\_\{t=1\}^\{T\}\\log p\_\{\\Theta\}\(y\_\{t\}\\mid x,y\_\{<t\}\)\\right\]\.\(1\)Minimizing Eq\. \([1](https://arxiv.org/html/2606.27379#A1.E1)\) lowers the probability of selected forget\-set continuations\. This can prevent direct reproduction, but it defines success by suppressing annotated strings\. The model may still retain paraphrased knowledge or derived capabilities, while a retrained model may still assign nonzero probability to similar outputs if the information remains inD∖FD\\setminus F\. Thus, suppression is an output\-distribution intervention, not a deletion criterion\.

#### Internal representation obfuscation\.

Obfuscation methods make forget\-related inputs unreliable or non\-informative, e\.g\., by inducing high\-entropy predictions\. Let𝒰\\mathcal\{U\}denote the uniform distribution over the vocabulary:

ℒME\(Θ;F\)=𝔼\(x,y\)∼F\[1T∑t=1TKL\(pΘ\(⋅∣x,y<t\)∥𝒰\)\]\.\\mathcal\{L\}\_\{\\mathrm\{ME\}\}\(\\Theta;F\)=\\mathbb\{E\}\_\{\(x,y\)\\sim F\}\\left\[\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathrm\{KL\}\\\!\\big\(p\_\{\\Theta\}\(\\cdot\\mid x,y\_\{<t\}\)\\,\\\|\\,\\mathcal\{U\}\\big\)\\right\]\.\(2\)Minimizing Eq\. \([2](https://arxiv.org/html/2606.27379#A1.E2)\) pushes predictions on forget\-related contexts toward a diffuse distribution\. This measures induced confusion, whereas retraining withoutFFneed not make the model uniformly uncertain\. A retrained model may answer confidently using retained evidence or fail in a structured way\.

#### Knowledge editing\.

Editing methods overwrite the model’s answer within a specified scope while preserving behavior outside that scope\. Lety¯\\bar\{y\}be a replacement response for a forget\-related inputxx:

ℒEdit​\(Θ;F\)=𝔼\(x,y¯\)∼Fedit​\[−∑t=1T¯log⁡pΘ​\(y¯t∣x,y¯<t\)\]\+λ​𝔼x∼𝒳loc​\[Dist​\(fΘ​\(x\),fΘ0​\(x\)\)\]\.\\mathcal\{L\}\_\{\\mathrm\{Edit\}\}\(\\Theta;F\)=\\mathbb\{E\}\_\{\(x,\\bar\{y\}\)\\sim F\_\{\\mathrm\{edit\}\}\}\\left\[\-\\sum\_\{t=1\}^\{\\bar\{T\}\}\\log p\_\{\\Theta\}\(\\bar\{y\}\_\{t\}\\mid x,\\bar\{y\}\_\{<t\}\)\\right\]\+\\lambda\\,\\mathbb\{E\}\_\{x\\sim\\mathcal\{X\}\_\{\\mathrm\{loc\}\}\}\\left\[\\mathrm\{Dist\}\\\!\\left\(f\_\{\\Theta\}\(x\),f\_\{\\Theta\_\{0\}\}\(x\)\\right\)\\right\]\.\(3\)Here,FeditF\_\{\\mathrm\{edit\}\}pairs forget\-related prompts with replacement targets,𝒳loc\\mathcal\{X\}\_\{\\mathrm\{loc\}\}is a locality set, andΘ0\\Theta\_\{0\}is the original model\. Editing starts from a desired replacementy¯\\bar\{y\}and a manually chosen locality constraint\. The retrained model onD∖FD\\setminus Fneed not producey¯\\bar\{y\}, preserve the same locality set, or change only within the edited scope\.

#### Behavioral refusal\.

Refusal methods train the model to abstain from answering forget\-related queries\. Lety~\\widetilde\{y\}denote a predefined refusal response paired with a forget\-related promptxx:

ℒIDK​\(Θ;F\)=𝔼\(x,y~\)∼F​\[−∑t=1T~log⁡pΘ​\(y~t∣x,y~<t\)\]\.\\mathcal\{L\}\_\{\\mathrm\{IDK\}\}\(\\Theta;F\)=\\mathbb\{E\}\_\{\(x,\\widetilde\{y\}\)\\sim F\}\\left\[\-\\sum\_\{t=1\}^\{\\widetilde\{T\}\}\\log p\_\{\\Theta\}\(\\widetilde\{y\}\_\{t\}\\mid x,\\widetilde\{y\}\_\{<t\}\)\\right\]\.\(4\)Minimizing Eq\. \([4](https://arxiv.org/html/2606.27379#A1.E4)\) increases the likelihood of a fixed abstention response\. This may be appropriate for non\-disclosure, but it trains a refusal policy rather than approximating𝖳𝗋𝖺𝗂𝗇​\(D∖F\)\\mathsf\{Train\}\(D\\setminus F\)\. A retrained model may answer from retained data, answer partially, or fail without using a refusal template\.

#### Inference\-time interventions\.

Inference\-time interventions modify the deployed input–output path without changing model parameters\. LetC​\(x\)∈\{0,1\}C\(x\)\\in\\\{0,1\\\}be a prompt router andTσT\_\{\\sigma\}a token\-, embedding\-, or logit\-space transformation\. For an embedding\-level intervention with𝐞=E​\(x\)\\mathbf\{e\}=E\(x\),

𝐞¯​\(x\)=\{Tσ​\(E​\(x\)\),C​\(x\)=1,E​\(x\),C​\(x\)=0\.\\bar\{\\mathbf\{e\}\}\(x\)=\\begin\{cases\}T\_\{\\sigma\}\(E\(x\)\),&C\(x\)=1,\\\\ E\(x\),&C\(x\)=0\.\\end\{cases\}\(5\)The model is evaluated on𝐞¯​\(x\)\\bar\{\\mathbf\{e\}\}\(x\)for routed forget\-related prompts\. Any apparent forgetting depends on the router and transformation at deployment time; if they are removed, bypassed, or misroute a prompt, the original behavior can reappear\. Such methods are system\-level control or filtering, not removal of learned information from the model parameters\.

#### Operational contrast\.

Equations \([1](https://arxiv.org/html/2606.27379#A1.E1)\)–\([5](https://arxiv.org/html/2606.27379#A1.E5)\) reduce forget\-set performance in different ways: suppression changes likelihoods, obfuscation induces uncertainty, editing overwrites responses, refusal enforces abstention, and inference\-time intervention alters the deployment path around an unchanged model\. Machine unlearning is different because its reference is not a target string, refusal template, locality set, router, or transformation, but the counterfactual model trained onD∖FD\\setminus F\.

## Appendix BExtension to the Multimodal Setting

The same terminology issue arises, and may become even more pronounced, in multimodal large language models \(MLLMs\)\. Recent work has begun to study multimodal machine unlearning and safety\-oriented multimodal forgetting objectives\(Huoet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib106); Chenet al\.,[2025](https://arxiv.org/html/2606.27379#bib.bib107)\)\. These directions highlight the need for clear evaluation baselines across modalities\.

#### Multimodal Setup\.

LetD𝗆𝗆D\_\{\\mathsf\{mm\}\}denote a multimodal training set whose examples may contain text, image, audio, video, or other modality components\. We write a generic example asz=\(xℳ,y\)z=\(x^\{\\mathcal\{M\}\},y\), wherexℳx^\{\\mathcal\{M\}\}denotes the available multimodal input components andyydenotes the target text or multimodal response\. Given a precisely specified multimodal forget setF𝗆𝗆⊆D𝗆𝗆F\_\{\\mathsf\{mm\}\}\\subseteq D\_\{\\mathsf\{mm\}\}, define the retain set asR𝗆𝗆:=D𝗆𝗆∖F𝗆𝗆R\_\{\\mathsf\{mm\}\}:=D\_\{\\mathsf\{mm\}\}\\setminus F\_\{\\mathsf\{mm\}\}\. A multimodal unlearning procedure returns:

Θ𝗆𝗆′←𝖴𝗇𝗅𝖾𝖺𝗋𝗇𝗆𝗆​\(ΘD𝗆𝗆,F𝗆𝗆\)\.\\Theta^\{\\prime\}\_\{\\mathsf\{mm\}\}\\leftarrow\\mathsf\{Unlearn\}\_\{\\mathsf\{mm\}\}\(\\Theta\_\{D\_\{\\mathsf\{mm\}\}\},F\_\{\\mathsf\{mm\}\}\)\.
The exact and approximate machine unlearning guarantees defined in Definition[2\.1](https://arxiv.org/html/2606.27379#S2.Thmdefinition1)and Definition[2\.2](https://arxiv.org/html/2606.27379#S2.Thmdefinition2)naturally extend to this multimodal setting\. Specifically, under the dataset\-defined view,*exact multimodal machine unlearning*requires the updated MLLM to be distributed identically to the counterfactual model retrained on the retained multimodal data:ℒ​\(Θ𝗆𝗆′\)=ℒ​\(ΘR𝗆𝗆\)\\mathcal\{L\}\(\\Theta^\{\\prime\}\_\{\\mathsf\{mm\}\}\)=\\mathcal\{L\}\(\\Theta\_\{R\_\{\\mathsf\{mm\}\}\}\)\. More practically,*approximate multimodal unlearning*evaluates success by choosing a behavior\-space or parameter\-space distanceDist\\mathrm\{Dist\}and requiring thatDist​\(ℒ​\(Θ𝗆𝗆′\),ℒ​\(ΘR𝗆𝗆\)\)≤τ\\mathrm\{Dist\}\\\!\\left\(\\mathcal\{L\}\(\\Theta^\{\\prime\}\_\{\\mathsf\{mm\}\}\),\\,\\mathcal\{L\}\(\\Theta\_\{R\_\{\\mathsf\{mm\}\}\}\)\\right\)\\leq\\tau\.

As in the text\-only setting, the essential point is not the particular choice of distance, but the reference: the baseline is training without the specified multimodal forget set\.

#### Why multimodality sharpens the distinction\.

In MLLMs, the target of a request may be expressed in one modality but revealed or preserved through another\. For example, a forget target associated with an image–text pair may affect visual recognition, textual descriptions, cross\-modal retrieval, VQA behavior, or downstream safety responses\. Thus, a model that no longer gives the original answer to a particular visual question may still retain related information through captions, textual entity knowledge, or cross\-modal representations\. Conversely, a safety method that refuses image\-conditioned harmful requests may successfully enforce a deployment policy while leaving the underlying training influence unchanged\.

This is exactly the distinction emphasized in the main paper\. IfF𝗆𝗆F\_\{\\mathsf\{mm\}\}is a concrete subset of multimodal training examples, then a machine\-unlearning claim should be evaluated against the criteria in Definition[2\.1](https://arxiv.org/html/2606.27379#S2.Thmdefinition1)or Definition[2\.2](https://arxiv.org/html/2606.27379#S2.Thmdefinition2), using a retrained MLLM or the strongest feasible proxy\. If the goal is instead to block unsafe multimodal behavior, suppress a visual concept, or induce refusals for certain image–text prompts, then the contribution should be described as multimodal suppression, safety alignment, or policy control unless it is tied to a retraining reference\.

#### Evaluation implications\.

A multimodal evaluation should therefore report not only forget\-query performance, but also reference\-relative behavior across the relevant modalities\. For deletion claims, useful probes include held\-out image–text pairs, paraphrased visual questions, text\-only and image\-only elicitation, cross\-modal retrieval or captioning prompts, and modality\-transfer tests that ask whether information removed from one modality remains recoverable through another\. These probes should be interpreted relative toΘR𝗆𝗆\\Theta\_\{R\_\{\\mathsf\{mm\}\}\}: retaining a behavior that is absent from the retrained reference suggests residual influence, while suppressing a behavior that is present in the retrained reference indicates output control beyond dataset\-defined deletion\.