On-the-go Forgetting without Explicit Unlearning via ERASE

# On-the-go Forgetting without Explicit Unlearning via ERASE
Source: [https://arxiv.org/html/2609.05966](https://arxiv.org/html/2609.05966)
Mayank Baranwalmbaranwal@iitb\.ac\.inAffiliation:Tata Consultancy Services Research, MumbaiAffiliation:Indian Institute of Technology Bombay

###### Abstract

Existing unlearning approaches typically rely on post hoc weight adaptation or distillation, leading to duplicated memory costs, degraded generalization, and limited scalability\. In this work, we introduceERASE,*Erasure via Reconstructive Adversarial Signal Editing*, a framework for on\-the\-go forgetting that suppresses the observable influence of private data without modifying model weights\.ERASEleverages structured, class\-conditioned input perturbations to induce selective forgetting during inference, eliminating the need for retraining, fine\-tuning, or model copies\. We rigorously characterize sufficient conditions whenERASEprovably achieves functional forgetting of designated subclasses while preserving predictions across other subclasses within the same superclass\. This analysis offers a principled foundation for inference\-time forgetting under mild regularity assumptions\. Across diverse architectures and benchmark datasets,ERASEmaintains the best observed balance between forgetting efficacy, computational efficiency, and retention fidelity over recent unlearning\-based methods\. By reimagining data removal as*forgetting without unlearning*, our work establishes a scalable, regulation\-aligned pathway for continual, privacy\-conscious learning\.

## 1Introduction

As machine learning systems increasingly interact with sensitive and continuously evolving data streams, their ability to*forget on demand*has become a cornerstone of privacy compliance and adaptive intelligence\. In regulated domains such as healthcare, personalized services, and user\-generated platforms, deployed models must often suppress the influence of outdated or private data\([Li et al\., 2025](https://arxiv.org/html/2609.05966#bib.bib20)\), not through retraining, but through immediate, behavioral forgetting\. The challenge is to ensure that models no longer act on restricted data while maintaining utility on permissible data\.

Conventional approaches to data removal primarily pursue*structural unlearning*, in which model parameters are explicitly modified to mimic the effect of retraining on a pruned dataset\. Although this paradigm guarantees theoretical completeness, it is often computationally prohibitive and operationally impractical\([Nguyen et al\., 2022](https://arxiv.org/html/2609.05966#bib.bib19)\)in deployed systems that must respond to privacy or removal requests instantly\. Such systems include on\-device assistants that must forget on command, financial fraud detectors constrained by always\-on operation, certified medical diagnostics that cannot be retrained after patient consent withdrawal, and live\-service game AIs requiring instant behavioral updates\.

We thus advocate a complementary notion,functional forgetting, that focuses not on erasing knowledge from weights, but on preventing its manifestation during inference\. Functional forgetting treats data removal as a*behavioral*constraint rather than a retraining objective: the model retains its parameters intact yet behaves as if the forgotten data never existed when presented with relevant inputs\. This perspective aligns with the needs of modern, continuously operating AI systems that must adapt in real time while preserving the stability of their learned representations\.

To realize this notion, we proposeERASE\(Erasure via Reconstructive Adversarial Signal Editing\), a framework that operationalizes forgetting throughinput\-space modulationrather than weight updates\.ERASEleverages conditional priors and adversarial\-style signal editing to selectively suppress responses associated with a designated forget set\. By inducing erasure directly in the input domain,ERASEeliminates the need for retraining, fine\-tuning, or access to internal gradients each time, thereby enabling*on\-the\-go compliance*in resource\-constrained environments\. Our key contributions are summarized as follows:

1\. On\-the\-go Forgetting Framework\.We proposeERASE, a novel framework for*inference\-time forgetting*that enables models to suppress the influence of private data without retraining or gradient access each time\.ERASEis designed for deployed settings, marking a shift from structural unlearning to functional forgetting\. 2\. Selective Erasure via Conditional Adversarial Perturbations\.ERASEintroduces*reconstructive adversarial signal editing*, a targeted input\-space perturbation mechanism guided by*precomputed*conditional priors\. This mechanism enforces forgetting only where necessary\. 3\. Efficient and Deployable Privacy Compliance\.Our approach eliminates the need for retraining or auxiliary models, achieving extreme computational efficiency\. It is well\-suited for environments where model updates are impractical\. 4\. Theoretical Insights\.We rigorously characterize sufficient conditions under whichERASEprovably forgets the targeted subclass while retaining predictions on the other subclasses within the same superclass\. 5\. Comprehensive Evaluation and Insights\.We evaluateERASEacross a wide range of model architectures and diverse datasets spanning multiple modalities\.ERASEachieves competitive forgetting while substantially improving retention and deployment efficiency, even when compared against methods that rely on continual parameter updates\. Ablation studies confirm thatERASEprovides*targeted*forgetting\.

![Refer to caption](https://arxiv.org/html/2609.05966v1/ERASE.png)

Figure 1:Overview ofERASE\. In the offline phase, subclass\-specific input\-gradient priors are computed once from the frozen model and stored as semantic directions\. In the online phase, these priors are combined into forget and retain directions, whose optimized combination is conditionally applied at inference time only when the input is predicted to belong to the target superclass\. This suppresses the forgotten subclass while preserving retention on related sibling subclasses and the broader retain set, without fine\-tuning the backbone model\.It is important to emphasize that our framework targets*functional forgetting*in discriminative models, where parameter modification is neither feasible nor desirable\. While generative architectures may employ prompt\-level filtering or guardrailing to achieve a similar goal, such mechanisms are inherently unavailable in standard classification or recognition pipelines\.ERASEis designed precisely for these deployment\-level privacy scenarios, where the objective is to hide the effect of forgotten data rather than retrain it away\.

## 2Related work

Research on data removal in machine learning has been dominated bymachine unlearning, where the goal is to eliminate the influence of designated training samples by*modifying model parameters*so that the resulting model approximates retraining on a pruned dataset\. In this sense, most prior work targets what we refer to asstructural unlearning: knowledge removal is achieved through updates to the model itself, whether via exact recomputation, approximate optimization, distillation, or selective retraining\. This paradigm has produced important theoretical and practical advances, but it remains fundamentally tied to a*training\-time*workflow and therefore incurs nontrivial computational, memory, or deployment overhead\([Nguyen et al\., 2022](https://arxiv.org/html/2609.05966#bib.bib19);[Li et al\., 2025](https://arxiv.org/html/2609.05966#bib.bib20)\)\. Such costs are particularly restrictive in always\-on systems that must react to privacy, regulatory, or user\-driven removal requests without retraining delays\.

Our work is motivated by a different objective\. Rather than enforcing deletion through parameter updates, we studyfunctional forgetting, where the model parameters remain fixed and the desired forgetting behavior is induced only at inference time\. The distinction is central: structural unlearning attempts to remove information from the weights, whereas functional forgetting aims to prevent that information from being expressed in downstream predictions\. This framing is especially relevant for deployed discriminative models, where retraining, fine\-tuning, or maintaining multiple model copies may be impractical\. Within this broader landscape, the following categories of prior work illustrate the evolution of explicit unlearning approaches that our method conceptually extends and operationally supersedes\.

Exact and certified unlearning\.The earliest line of work sought strong deletion guarantees by making the post\-unlearning model identical, or provably close, to a model retrained from scratch on the retained data\. Such guarantees are achievable only for restricted model families or carefully designed pipelines, including settings such as linear and logistic regression,kk\-means, and random forests\([Ginart et al\., 2019](https://arxiv.org/html/2609.05966#bib.bib10);[Mahadevan and Mathioudakis, 2022](https://arxiv.org/html/2609.05966#bib.bib11)\)\. Certified data removal methods further provide provable bounds on the deviation from exact retraining\([Guo et al\., 2020](https://arxiv.org/html/2609.05966#bib.bib12)\)\. These methods establish an important foundation for verifiable forgetting, but their assumptions and computational structure limit their applicability to modern deep models and large\-scale deployment scenarios\.

Efficient retraining through partitioning and aggregation\.A second line of work improves the efficiency of structural unlearning by changing how the model is trained in the first place\. Sharded, Isolated, Sliced, and Aggregated \(SISA\) training\([Bourtoule et al\., 2021](https://arxiv.org/html/2609.05966#bib.bib13)\)partitions the dataset into shards, trains separate sub\-models, and aggregates their predictions, so that forgetting only requires retraining the affected shard rather than the full system\. This reduces the cost of deletion and is broadly model\-agnostic, but it still relies on retraining after a deletion request and may degrade utility because partitioning weakens cross\-shard information sharing\.

Approximate unlearning via influence reversal and optimization\.To scale unlearning beyond exact methods, many approaches estimate and reverse the contribution of forgotten samples through approximate optimization\. Representative techniques include influence functions\([Koh and Liang, 2017](https://arxiv.org/html/2609.05966#bib.bib14)\), Newton\-style removal updates\([Guo et al\., 2020](https://arxiv.org/html/2609.05966#bib.bib12)\), and gradient\-based forgetting objectives such as gradient ascent on the forget set\([Golatkar et al\., 2020](https://arxiv.org/html/2609.05966#bib.bib17);[Graves et al\., 2021](https://arxiv.org/html/2609.05966#bib.bib16)\)\. These methods are attractive because they can be adapted to deep networks and are often substantially faster than retraining from scratch\. However, they still alter the model parameters, and their approximate nature can leave residual traces of the removed data, which may be detectable under privacy audits such as membership inference attacks\([Shokri et al\., 2017](https://arxiv.org/html/2609.05966#bib.bib15)\)\.

Corrective fine\-tuning and representation\-level forgetting\.More recent structural unlearning methods perform targeted corrective updates to suppress forgotten information\. Examples include amnesiac unlearning\([Graves et al\., 2021](https://arxiv.org/html/2609.05966#bib.bib16)\), which applies counteracting gradient information, and impairing\-data approaches that fine\-tune the model to obscure sensitive content\([Baumhauer et al\., 2022](https://arxiv.org/html/2609.05966#bib.bib18)\)\. Other methods explicitly manipulate the geometry of the learned predictor, for example by shifting the decision boundary away from forget samples\([Chen et al\., 2023](https://arxiv.org/html/2609.05966#bib.bib25)\), or by masking entangled components and constraining knowledge transfer to reduce re\-emergence of forgotten information\([Lin et al\., 2023](https://arxiv.org/html/2609.05966#bib.bib26)\)\. These approaches improve targeting relative to naive retraining, but they remain dependent on post hoc parameter updates and often introduce additional optimization complexity, auxiliary components, or verification challenges\.

Distillation\-based and teacher\-guided unlearning\.A related family of methods uses knowledge distillation to disentangle forgetting and retention objectives\. For example,[Chundawat et al\. \(2023\)](https://arxiv.org/html/2609.05966#bib.bib1)employs an incompetent teacher to induce forgetting through deliberately misleading supervision, while[Zhou et al\. \(2025b\)](https://arxiv.org/html/2609.05966#bib.bib24)decouples distillation into separate forgetting and retention modules to reduce interference between the two objectives\. Distillation\-based approaches can be effective, but typically require additional teacher models, repeated training procedures, or auxiliary modules, which increases system complexity and resource\.

Input\-space and perturbation\-based forgetting/unlearning\.More recently, a closely related line of work has begun to explore forgetting through*input\-space*interventions rather than direct weight updates\.[PENG et al\. \(2025\)](https://arxiv.org/html/2609.05966#bib.bib28)studied catastrophic unlearning through an adversarial generator–unlearner framework that synthesizes challenging mixup samples to regularize the unlearning process\. In parallel, recent class\-wise unlearning methods have shown that targeted or universal adversarial perturbations \(UAP\) can directly stimulate forgetting behavior at the input level, including UAP\-based class\-wise forgetting\([Zhou et al\., 2025a](https://arxiv.org/html/2609.05966#bib.bib29)\)and input\-agnostic “forget vectors” that drive machine unlearning while keeping model weights fixed\([Sun et al\., 2024](https://arxiv.org/html/2609.05966#bib.bib30)\)\. These works are especially relevant to our setting because they move unlearning closer to adversarial input manipulation\. OurERASEframework is most closely aligned with this emerging perspective, but differs in several important ways\. Specifically,ERASEperforms*conditional inference\-time*forgetting for deployed discriminative models and uses an explicit forget/retain decomposition to preserve sibling subclasses within the same superclass\. It is also hierarchy\-aware and deployment\-oriented, since perturbations are activated only when the model predicts the relevant superclass\. In addition,ERASEprovides theoretical sufficient conditions characterizing when targeted forgetting and retention can be achieved simultaneously\. Note that[PENG et al\. \(2025\)](https://arxiv.org/html/2609.05966#bib.bib28)and[Zhou et al\. \(2025a\)](https://arxiv.org/html/2609.05966#bib.bib29)are not direct baselines for our setting, as the former uses an adversarial generator\-unlearner with synthesized mixup samples, while the latter studies UAP\-based class\-wise unlearning\. ERASE instead performs conditional inference\-time functional forgetting for deployed discriminative models, with an explicit forget\-retain decomposition for preserving sibling subclasses\. Unlike prior perturbation\-based unlearning works centered primarily on image classification,ERASEis evaluated across multiple modalities, supporting its applicability beyond visual settings\.

Recent LLM unlearning works such as\([Li et al\., 2024](https://arxiv.org/html/2609.05966#bib.bib33);[Wang et al\., 2025](https://arxiv.org/html/2609.05966#bib.bib32);[Yuan et al\., 2025](https://arxiv.org/html/2609.05966#bib.bib31)\)target generative next\-token models, prompt\-response behavior, or LLM representation steering, and are therefore outside our discriminative superclass–subclass classification setting\.

Overall, existing literature has substantially improved the efficiency, selectivity, and verifiability of structural unlearning, and recent perturbation\-based methods have begun to show that forgetting can be induced from input side\. OurERASEframework builds on this emerging direction and advances it toward conditional, deployable, inference\-time functional forgetting without retraining, fine\-tuning, or auxiliary model copies\.

## 3Notations and problem formulation

LetMθM\_\{\\theta\}be a fixed pre\-trained discriminative model with parametersθ\\theta, trained on a labeled dataset𝒟train\\mathcal\{D\}\_\{\\text\{train\}\}\. Each data point in𝒟train\\mathcal\{D\}\_\{\\text\{train\}\}is annotated with a subclass label \(e\.g\.,Persian cat\) and a corresponding superclass label \(e\.g\.,cat\)\. The label hierarchy is structured such that multiple fine\-grained subclasses \(e\.g\.,Persian cat,tabby,cougar\) are grouped under broader superclasses \(e\.g\.,cat\)\. We emphasize that the pre\-trained modelMθM\_\{\\theta\}is trained to map each inputxxto its associated superclass label only, without access to subclass\-level supervision\. Given a target subclassCfC\_\{f\}to be unlearned, we posit to construct a \(super\)class\-specific input\-space perturbation that, when applied at inference time, selectively degrades the model’s performance on samples from the*forget set*, while maintaining high accuracy on all remaining subclasses that collectively form the*overall retain set*\. This objective stands in contrast to traditional machine unlearning, which instead aims to*perturb*the model parametersθ\\thetaso that samples from the target subclassCfC\_\{f\}are misclassified at the superclass level, while preserving the model’s performance on all other subclasses\. We also define the*retain\-superclass set*to consist of subclasses that belong to the same superclass ofCfC\_\{f\}, allowing for a more nuanced assessment of forgetting within semantically similar concepts\. Notably, the*retain\-superclass set*is a subset of the*overall retain set*\. For the test set𝒟test\\mathcal\{D\}\_\{\\text\{test\}\}, we define the following subsets for evaluation:

- •𝒟forget\\mathcal\{D\}\_\{\\text\{forget\}\}: test samples from target subclassCfC\_\{f\}\(forget set\);
- •𝒟retain\-overall\\mathcal\{D\}\_\{\\text\{retain\-overall\}\}: test samples not belonging toCfC\_\{f\}\(retain set\);
- •𝒟retain\-super\\mathcal\{D\}\_\{\\text\{retain\-super\}\}: subset of𝒟retain\-overall\\mathcal\{D\}\_\{\\text\{retain\-overall\}\}with samples from other subclasses in the same superclass ofCfC\_\{f\}\.

This hierarchy formulation is intentionally fine\-grained\. Unlike standard class\-level forgetting,ERASEasks the model to suppress one subclass while preserving sibling subclasses that share the same superclass decision boundary\.𝒟forget\\mathcal\{D\}\_\{\\text\{forget\}\}and𝒟retain\-overall\\mathcal\{D\}\_\{\\text\{retain\-overall\}\}form a partition of𝒟test\\mathcal\{D\}\_\{\\text\{test\}\}, while𝒟retain\-super⊂𝒟retain\-overall\\mathcal\{D\}\_\{\\text\{retain\-super\}\}\\subset\\mathcal\{D\}\_\{\\text\{retain\-overall\}\}enables us to assess retention performance on semantically adjacent concepts within the most closely related categories to the forgotten subclass\. Our objective is to minimize accuracy on𝒟forget\\mathcal\{D\}\_\{\\text\{forget\}\}while minimizing degradation of accuracy on the retain sets𝒟retain\-super\\mathcal\{D\}\_\{\\text\{retain\-super\}\}and𝒟retain\-overall\\mathcal\{D\}\_\{\\text\{retain\-overall\}\}\.

## 4ERASE: Erasure via Reconstructive Adversarial Signal Editing

Algorithm 1ERASE1:Require:Model

MθM\_\{\\theta\}, data

𝒟train,𝒟test\\mathcal\{D\}\_\{\\text\{train\}\},\\mathcal\{D\}\_\{\\text\{test\}\}, target subclass

CfC\_\{f\}, forgetting weights

ωu,ωr\>0\\omega\_\{u\},\\omega\_\{r\}\>0, learning rate

η\\eta
2:Return:Forget accuracy

𝒜forget\\mathcal\{A\}\_\{\\text\{forget\}\}, retain\-superclass accuracy

𝒜retain\-super\\mathcal\{A\}\_\{\\text\{retain\-super\}\}, retain\-overall accuracy

𝒜retain\-overall\\mathcal\{A\}\_\{\\text\{retain\-overall\}\}
3:Phase 1: Offline Generation of Subclass\-Specific Gradient w\.r\.t\. Input \(one\-time pre\-computation\)

4:Initialize map

𝐏←\{\}\\mathbf\{P\}\\leftarrow\\\{\\\}
5:foreach subclass

Ci∈𝒟trainC\_\{i\}\\in\\mathcal\{D\}\_\{\\text\{train\}\}do

6:

𝒟i←\{\(x,y\)∈𝒟train∣subclass​\(x\)=Ci\}\\mathcal\{D\}\_\{i\}\\leftarrow\\\{\(x,y\)\\in\\mathcal\{D\}\_\{\\text\{train\}\}\\mid\\text\{subclass\}\(x\)=C\_\{i\}\\\},

𝜹i←0\\bm\{\\delta\}\_\{i\}\\leftarrow 0
7:foreach sample

\(xj,yj\)∈𝒟i\(x\_\{j\},y\_\{j\}\)\\in\\mathcal\{D\}\_\{i\}do

8:

g←∇xjℒ​\(Mθ​\(xj\),yj\)g\\leftarrow\\nabla\_\{x\_\{j\}\}\\mathcal\{L\}\(M\_\{\\theta\}\(x\_\{j\}\),y\_\{j\}\)
9:

𝜹i←𝜹i\+sign​\(g\)\\bm\{\\delta\}\_\{i\}\\leftarrow\\bm\{\\delta\}\_\{i\}\+\\text\{sign\}\(g\)
10:endfor

11:

𝐏⁡\[Ci\]←𝜹i/\|𝒟i\|\\mathbf\{P\}\[C\_\{i\}\]\\leftarrow\\bm\{\\delta\}\_\{i\}/\|\\mathcal\{D\}\_\{i\}\|
12:endfor

13:Phase 2: Online Perturbation\-Based Forgetting at Inference Time

14:

⊳\\trianglerightStep 1: Define Forget and Retain Directions

15:

Sf←S\_\{f\}\\leftarrowsuperclass of

CfC\_\{f\},

ℛr←\{Ci∈Sf∣Ci≠Cf\}\\mathcal\{R\}\_\{r\}\\leftarrow\\\{C\_\{i\}\\in S\_\{f\}\\mid C\_\{i\}\\neq C\_\{f\}\\\}
16:

𝜹forget←𝐏⁡\[Cf\]/‖𝐏⁡\[Cf\]‖F\\bm\{\\delta\}\_\{\\text\{forget\}\}\\leftarrow\\mathbf\{P\}\[C\_\{f\}\]/\\\|\\mathbf\{P\}\[C\_\{f\}\]\\\|\_\{F\}
17:

𝜹retain←∑Ci∈ℛr𝐏⁡\[Ci\]/\|ℛr\|\\bm\{\\delta\}\_\{\\text\{retain\}\}\\leftarrow\\sum\_\{C\_\{i\}\\in\\mathcal\{R\}\_\{r\}\}\\mathbf\{P\}\[C\_\{i\}\]/\|\\mathcal\{R\}\_\{r\}\|
18:

𝜹retain←𝜹retain/‖𝜹retain‖F\\bm\{\\delta\}\_\{\\text\{retain\}\}\\leftarrow\\bm\{\\delta\}\_\{\\text\{retain\}\}/\\\|\\bm\{\\delta\}\_\{\\text\{retain\}\}\\\|\_\{F\}
19:

⊳\\trianglerightStep 2: Optimize Perturbation Scaling Factors

20:Initialize

ϵf,ϵr\>0\\epsilon\_\{f\},\\epsilon\_\{r\}\>0
21:

𝒟opt←\{\(x,ysuper\)∈𝒟train∣superclass​\(x\)=Sf\}\\mathcal\{D\}\_\{\\text\{opt\}\}\\leftarrow\\\{\(x,y\_\{\\text\{super\}\}\)\\in\\mathcal\{D\}\_\{\\text\{train\}\}\\mid\\text\{superclass\}\(x\)=S\_\{f\}\\\}
22:foreach optimization epochdo

23:forbatch

\{\(xj,yj\)\}j=1B\\\{\(x\_\{j\},y\_\{j\}\)\\\}\_\{j=1\}^\{B\}in

𝒟opt\\mathcal\{D\}\_\{\\text\{opt\}\}do

24:

𝒑←ϵf⋅𝜹forget−ϵr⋅𝜹retain\\bm\{p\}\\leftarrow\\epsilon\_\{f\}\\cdot\\bm\{\\delta\}\_\{\\text\{forget\}\}\-\\epsilon\_\{r\}\\cdot\\bm\{\\delta\}\_\{\\text\{retain\}\}
25:

xj′←xj\+𝒑⋅‖xj‖Fx^\{\\prime\}\_\{j\}\\leftarrow x\_\{j\}\+\\bm\{p\}\\cdot\\\|x\_\{j\}\\\|\_\{F\},

oj′←Mθ​\(xj′\)o^\{\\prime\}\_\{j\}\\leftarrow M\_\{\\theta\}\(x^\{\\prime\}\_\{j\}\)
26:

uj←𝟏\[subclass\(xj\)=Cf\]u\_\{j\}\\leftarrow\\mathbf\{1\}\_\{\[\\text\{subclass\}\(x\_\{j\}\)=C\_\{f\}\]\}
27:

Lopt←1B​∑j=1B\(ωr​\(1−uj\)−ωu​uj\)​ℒ​\(oj′,yj\)L\_\{\\text\{opt\}\}\\leftarrow\\frac\{1\}\{B\}\\sum\_\{j=1\}^\{B\}\(\\omega\_\{r\}\(1\-u\_\{j\}\)\-\\omega\_\{u\}u\_\{j\}\)\\mathcal\{L\}\(o^\{\\prime\}\_\{j\},y\_\{j\}\)
28:

\(ϵf,ϵr\)←\(ϵf,ϵr\)−η​∇\(ϵf,ϵr\)Lopt\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)\\leftarrow\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)\-\\eta\\nabla\_\{\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)\}L\_\{\\text\{opt\}\}
29:endfor

30:endfor

31:Evaluation \(Conditional Perturbation\)

32:

𝒑final←ϵf⋅𝜹forget−ϵr⋅𝜹retain\\bm\{p\}\_\{\\text\{final\}\}\\leftarrow\\epsilon\_\{f\}\\cdot\\bm\{\\delta\}\_\{\\text\{forget\}\}\-\\epsilon\_\{r\}\\cdot\\bm\{\\delta\}\_\{\\text\{retain\}\}
33:Define Perturb

\(x\)=x\+𝒑final​‖x‖F\(x\)=x\+\\bm\{p\}\_\{\\text\{final\}\}\\\|x\\\|\_\{F\}
34:Define ConditionalPerturb

\(x\)=\(x\)=
35:if

arg⁡max​Mθ​\(x\)=Sf\\arg\\max M\_\{\\theta\}\(x\)=S\_\{f\}then return

Perturb​\(x\)\\text\{Perturb\}\(x\)
36:else return

xx
37:

𝒟forget←\{\(x,y\)∈𝒟test∣subclass=Cf\}\\mathcal\{D\}\_\{\\text\{forget\}\}\\leftarrow\\\{\(x,y\)\\in\\mathcal\{D\}\_\{\\text\{test\}\}\\mid\\text\{subclass\}=C\_\{f\}\\\}
38:

𝒟retain\-super←\{\(x,y\)∣subclass∈ℛr\}\\mathcal\{D\}\_\{\\text\{retain\-super\}\}\\leftarrow\\\{\(x,y\)\\mid\\text\{subclass\}\\in\\mathcal\{R\}\_\{r\}\\\}
39:

𝒟retain\-overall←\{\(x,y\)∣subclass≠Cf\}\\mathcal\{D\}\_\{\\text\{retain\-overall\}\}\\leftarrow\\\{\(x,y\)\\mid\\text\{subclass\}\\neq C\_\{f\}\\\}
40:

𝒜forget←Accuracy​\(Mθ,𝒟forget,ConditionalPerturb\)\\mathcal\{A\}\_\{\\text\{forget\}\}\\leftarrow\\text\{Accuracy\}\(M\_\{\\theta\},\\mathcal\{D\}\_\{\\text\{forget\}\},\\text\{ConditionalPerturb\}\)
41:

𝒜retain\-super←Accuracy​\(Mθ,𝒟retain\-super,ConditionalPerturb\)\\mathcal\{A\}\_\{\\text\{retain\-super\}\}\\leftarrow\\text\{Accuracy\}\(M\_\{\\theta\},\\mathcal\{D\}\_\{\\text\{retain\-super\}\},\\text\{ConditionalPerturb\}\)
42:

𝒜retain\-overall←Accuracy​\(Mθ,𝒟retain\-overall,ConditionalPerturb\)\\mathcal\{A\}\_\{\\text\{retain\-overall\}\}\\leftarrow\\text\{Accuracy\}\(M\_\{\\theta\},\\mathcal\{D\}\_\{\\text\{retain\-overall\}\},\\text\{ConditionalPerturb\}\)

We now introduceERASE, our novel framework for inference\-time forgetting\. Unlike existing approaches for discriminative models that rely on model retraining,ERASEachieves forgetting by strategically perturbing the model inputs during inference, thereby modifying its predicted output without altering its parameters\.

### 4\.1Overview ofERASE

ERASEis a model update\-free forgetting framework that operates purely at inference time, requiring no update of model parameters\. It proceeds in two main phases: \(i\) an*offline phase*pre\-computing subclass\-specific gradient directions in the input space and storing them, followed by \(ii\) an*online phase*performing forgetting for a target subclass by applying an optimized superclass\-specific perturbation vector to inference inputs\. This perturbation is shared across all inputs from the same superclass and applied only if needed\. Figure[1](https://arxiv.org/html/2609.05966#S1.F1)illustrates the overall framework\.

Offline phase: Subclass\-specific input\-gradient direction computation\.We compute an average input\-gradient direction for each subclassCiC\_\{i\}in the train set𝒟train\\mathcal\{D\}\_\{\\text\{train\}\}\. We denote the subset of samples from subclassCiC\_\{i\}as

𝒟i=\{\(x,ysuper\)∈𝒟train∣subclass of​x​is​Ci\},\\displaystyle\\mathcal\{D\}\_\{i\}=\\\{\(x,y\_\{\\text\{super\}\}\)\\in\\mathcal\{D\}\_\{\\text\{train\}\}\\mid\\text\{subclass of \}x\\text\{ is \}C\_\{i\}\\\},whereysupery\_\{\\text\{super\}\}denotes the true superclass label of samplexx\. The average input\-gradient direction, specific to eachCiC\_\{i\}, is computed using a Fast Gradient Sign Method \(FGSM\)\-style formulation\([Goodfellow et al\., 2015](https://arxiv.org/html/2609.05966#bib.bib6)\), yielding a signed and averaged vector:

𝐏⁡\[Ci\]=1\|𝒟i\|​∑\(xj,yj\)∈𝒟isign​\(∇xjℒ​\(Mθ​\(xj\),yj\)\),\\displaystyle\\mathbf\{P\}\[C\_\{i\}\]=\\tfrac\{1\}\{\|\\mathcal\{D\}\_\{i\}\|\}\\sum\\limits\_\{\(x\_\{j\},y\_\{j\}\)\\in\\mathcal\{D\}\_\{i\}\}\\text\{sign\}\\left\(\\nabla\_\{x\_\{j\}\}\\mathcal\{L\}\(M\_\{\\theta\}\(x\_\{j\}\),y\_\{j\}\)\\right\),\(1\)whereℒ\\mathcal\{L\}is a supervised loss, e\.g\., cross\-entropy\. The precomputed signed gradients𝐏\\mathbf\{P\}act as semantic “fingerprints” for each subclass in input\-space and are stored for downstream forgetting use\. The subclass annotationsCiC\_\{i\}in𝒟i\\mathcal\{D\}\_\{i\}are solely used to identify the target samples to be forgotten\.

Online phase: Targeted forgetting via optimized perturbations\.The online phase begins with the selection of a subclassCfC\_\{f\}to unlearn\. We letSfS\_\{f\}denote the superclass to which the target subclassCfC\_\{f\}belongs, andℛr=\{Ci∈Sf∣Ci≠Cf\}\\mathcal\{R\}\_\{r\}=\\\{C\_\{i\}\\in S\_\{f\}\\mid C\_\{i\}\\neq C\_\{f\}\\\}the set of other subclasses inSfS\_\{f\}\. Our goal is to construct a perturbation𝒑\\bm\{p\}that selectively degrades model performance onCfC\_\{f\}\(forget set\), while preserving accuracy onℛr\\mathcal\{R\}\_\{r\}\(retain set\)\. This requires accessing the pre\-computed input\-gradients𝐏\\mathbf\{P\}from the offline phase\. We define the following input\-gradient directions\. • The forget direction as the normalized gradient forCfC\_\{f\}:

𝜹forget=𝐏⁡\[Cf\]‖𝐏⁡\[Cf\]‖F;\\bm\{\\delta\}\_\{\\text\{forget\}\}=\\tfrac\{\\mathbf\{P\}\[C\_\{f\}\]\}\{\\\|\\mathbf\{P\}\[C\_\{f\}\]\\\|\_\{F\}\};\\vskip\-2\.5pt•The retain direction as the average normalized gradient across all other subclassesℛr\\mathcal\{R\}\_\{r\}inSfS\_\{f\}:

𝜹retain=∑Ci∈ℛr𝐏⁡\[Ci\]‖∑Cj∈ℛr𝐏⁡\[Cj\]‖F\.\\bm\{\\delta\}\_\{\\text\{retain\}\}=\\sum\_\{C\_\{i\}\\in\\mathcal\{R\}\_\{r\}\}\\tfrac\{\\mathbf\{P\}\[C\_\{i\}\]\}\{\\\|\\sum\_\{C\_\{j\}\\in\\mathcal\{R\}\_\{r\}\}\\mathbf\{P\}\[C\_\{j\}\]\\\|\_\{F\}\}\.\\vskip\-2\.5ptNormalization ensures that each direction contributes based purely on its semantic alignment rather than raw gradient magnitude, mitigating bias from subclass size or gradient scales\. This allows the final perturbation to reflect true directional intent rather than being skewed by disproportionately large gradients\. We then construct the perturbation vector as a linear combination of these two directions:

𝒑=ϵf⋅𝜹forget−ϵr⋅𝜹retain,\\displaystyle\\bm\{p\}=\\epsilon\_\{f\}\\cdot\\bm\{\\delta\}\_\{\\text\{forget\}\}\-\\epsilon\_\{r\}\\cdot\\bm\{\\delta\}\_\{\\text\{retain\}\},\(2\)whereϵf,ϵr\>0\\epsilon\_\{f\},\\epsilon\_\{r\}\>0are scaling factors optimized in the next step\. The positive coefficient for𝜹forget\\bm\{\\delta\}\_\{\\text\{forget\}\}pushes samples away from the target subclass representation, while the negative coefficient for𝜹retain\\bm\{\\delta\}\_\{\\text\{retain\}\}pulls samples toward non\-forgotten peer subclasses, improving retention\. After scaling, this perturbation vector𝒑\\bm\{p\}will be applied later to each inputxxbelonging toSfS\_\{f\}\. Specifically, we scale the perturbation𝒑\\bm\{p\}for each inputxxby its Frobenius norm‖x‖F\\\|x\\\|\_\{F\}, ensuring that the perturbation magnitude adapts proportionally to the input’s scale\. This design maintains invariance to variations in pixel\-level intensity across inputs, and is inspired by adversarial robustness literature, where perturbations are often norm\-bounded relative to the input as‖x′−x‖p≤ϵ​‖x‖p\\\|x^\{\\prime\}\-x\\\|\_\{p\}\\leq\\epsilon\\,\\\|x\\\|\_\{p\}to preserve perceptual consistency and prevent over\-perturbation of low\-magnitude inputs\([Carlini and Wagner, 2017](https://arxiv.org/html/2609.05966#bib.bib21);[Moosavi\-Dezfooli et al\., 2016](https://arxiv.org/html/2609.05966#bib.bib22)\)\. For this, we define input perturbation functionPerturb​\(⋅\)\\text\{Perturb\}\(\\cdot\)as

Perturb​\(x\)=x\+𝒑​‖x‖F\.\\displaystyle\\text\{Perturb\}\(x\)=x\+\\bm\{p\}\\,\\\|x\\\|\_\{F\}\.Since𝜹forget,𝜹retain\\bm\{\\delta\}\_\{\\text\{forget\}\},\\bm\{\\delta\}\_\{\\text\{retain\}\}are pure directional, strength of𝒑\\bm\{p\}can be adjusted viaϵf,ϵr\\epsilon\_\{f\},\\epsilon\_\{r\}in the optimization phase\.

Optimization of perturbation scaling factors\.To balance forgetting and retention, we optimize\(ϵf,ϵr\)\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)using a weighted loss on a subset of training data𝒟opt⊆𝒟train\\mathcal\{D\}\_\{\\text\{opt\}\}\\subseteq\\mathcal\{D\}\_\{\\text\{train\}\}consisting samples from superclassSfS\_\{f\}\. To achieve the balance, we maximize the loss on the forget training samples perturbed by𝒑\\bm\{p\}and minimize the loss on the remaining training samples in𝒟opt\\mathcal\{D\}\_\{\\text\{opt\}\}perturbed by𝒑\\bm\{p\}\. For each sample, we define a binary indicatoruj=1u\_\{j\}=1ifxjx\_\{j\}belongs to the forget setCfC\_\{f\}, anduj=0u\_\{j\}=0otherwise\. The optimization loss is:

Lopt=1\|𝒟opt\|∑\(xj,yj\)∈𝒟opt\\displaystyle L\_\{\\text\{opt\}\}=\\tfrac\{1\}\{\|\\mathcal\{D\}\_\{\\text\{opt\}\}\|\}\\sum\_\{\(x\_\{j\},y\_\{j\}\)\\in\\mathcal\{D\}\_\{\\text\{opt\}\}\}\(ωr​\(1−uj\)−ωu​uj\)×ℒ⁡\(Mθ​\(Perturb​\(xj\)\),yj\),\\displaystyle\\left\(\\omega\_\{r\}\(1\-u\_\{j\}\)\-\\omega\_\{u\}u\_\{j\}\\right\)\\times\\mathcal\{L\}\(M\_\{\\theta\}\(\\text\{Perturb\}\(x\_\{j\}\)\),y\_\{j\}\),\(3\)whereωu,ωr\>0\\omega\_\{u\},\\omega\_\{r\}\>0are user\-defined weights emphasizing forgetting and retention respectively\. For instance, when the forget and retain training sets are significantly unbalanced, the weightsωu,ωr\\omega\_\{u\},\\omega\_\{r\}can be adjusted to compensate for this imbalance, ensuring that both objectives contribute meaningfully to the loss in equation[3](https://arxiv.org/html/2609.05966#S4.E3)\. The formulation in equation[3](https://arxiv.org/html/2609.05966#S4.E3)encourages the model to perform poorly onCfC\_\{f\}via negative weighting while reinforcing correct predictions on other classes via positive weighting\. Notably, no model parameters are updated and only the perturbation vector is learned\.

Inference\-time evaluation\.The final step is evaluation of the unlearning effect on the three relevant subsets of the test set, defined earlier as𝒟forget\\mathcal\{D\}\_\{\\text\{forget\}\},𝒟retain\-overall\\mathcal\{D\}\_\{\\text\{retain\-overall\}\}, and𝒟retain\-super\\mathcal\{D\}\_\{\\text\{retain\-super\}\}\. For this evaluation,MθM\_\{\\theta\}’s accuracy on a dataset𝒟\\mathcal\{D\}under the input perturbation functionPerturb​\(⋅\)\\text\{Perturb\}\(\\cdot\)is defined as

Accuracy\(Mθ,𝒟,Perturb\)=1\|𝒟\|∑\(x,ysuper\)∈𝒟𝟏\[argmaxMθ\(Perturb\(x\)\)=ysuper\]\.\\displaystyle\\text\{Accuracy\}\(M\_\{\\theta\},\\mathcal\{D\},\\text\{Perturb\}\)=\\tfrac\{1\}\{\|\\mathcal\{D\}\|\}\\sum\\nolimits\_\{\(x,y\_\{\\text\{super\}\}\)\\in\\mathcal\{D\}\}\\mathbf\{1\}\_\{\\left\[\\arg\\max M\_\{\\theta\}\(\\text\{Perturb\}\(x\)\)=y\_\{\\text\{super\}\}\\right\]\}\.For each test samplexx, we adopt the following conditional perturbation strategy:

1. 1\.The modelMθM\_\{\\theta\}first predicts the superclass labely^super\\hat\{y\}\_\{\\text\{super\}\}without any perturbation\.
2. 2\.If the sample is predicted to belong to the target superclassy^super=Sf\\hat\{y\}\_\{\\text\{super\}\}=S\_\{f\}, we apply the perturbationPerturb​\(x\)\\text\{Perturb\}\(x\)and re\-infer the class\.
3. 3\.If not, the model’s original prediction is retained, i\.e\., no perturbation is applied\.

This selective perturbation ensures that samples unrelated to the forget subclass remain untouched, preserving global model performance and ensuring that the perturbation effect is localized to the relevant semantic region\. Successful unlearning corresponds to low accuracy on𝒟forget\\mathcal\{D\}\_\{\\text\{forget\}\}, indicating successful forgetting; and high accuracy on𝒟retain\-super\\mathcal\{D\}\_\{\\text\{retain\-super\}\}and𝒟retain\-overall\\mathcal\{D\}\_\{\\text\{retain\-overall\}\}, indicating effective retention\. Our proposed method is summarized in Algorithm[1](https://arxiv.org/html/2609.05966#alg1)\.

Broader applicability\.ERASEremains applicable even in the absence of an explicit hierarchy\. Users can define custom semantic groupings based on domain knowledge or task\-specific criteria, such as clustering visually or functionally related classes\. For example, in Tiny ImageNet, we manually construct superclasses by clustering semantically related subclasses using the WordNet ontology\. In real\-world datasets like HAR, where no hierarchy exists, we define superclasses as activity types \(e\.g\., walking\) and subclasses as individual subjects performing those activities\. The grouping should be semantically or task coherent and treated as part of the deployment design, guided by dataset ontology, domain knowledge, or task\-specific structure\. Poorly chosen groupings may make the forgetting task less meaningful or artificially easy, e\.g\., if unrelated subclasses are placed under the same superclass\.

### 4\.2Computational efficiency

A key advantage ofERASElies in its lightweight, inference\-time deployment that avoids any retraining\.

Offline precomputation\.The offline phase ofERASEconsists of computing input\-gradient directions for each subclass\. For a given inputxx, this requires a single backpropagation step to compute the gradient∇xℒ​\(Mθ​\(x\),ysuper\)\\nabla\_\{x\}\\mathcal\{L\}\(M\_\{\\theta\}\(x\),y\_\{\\text\{super\}\}\)\. Computing gradients with respect to the input involves significantly fewer dimensions than computing gradients with respect to the model parameters\. Since this operation is performed once per training sample and only aggregated per subclass, the overall cost remains linear in the size of the training set\. Moreover, this cost is amortized over all future forgetting operations, as the same set of subclass\-specific gradient directions can be reused\. Online perturbation optimization\.The online phase only involves optimizing two scalar weights\(ϵf,ϵr\)\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)over the subset𝒟opt\\mathcal\{D\}\_\{\\text\{opt\}\}of training samples within the superclassSfS\_\{f\}of the forget subclassCfC\_\{f\}\. This results in a highly efficient, low\-dimensional optimization problem, typically converging in fewer than1010epochs\. Crucially, no model gradients or weight updates are required, and the modelMθM\_\{\\theta\}remains frozen throughout\. Compared to retraining\-based or continual fine\-tuning methods,ERASEthus reduces memory and compute overhead by several orders of magnitude\. Inference overhead\.At deployment, the inference cost ofERASEis negligible\. For any test input, the model first performs a standard forward pass to determine its predicted superclassy^super\\hat\{y\}\_\{\\text\{super\}\}\. Only ify^super=Sf\\hat\{y\}\_\{\\text\{super\}\}=S\_\{f\}, the precomputed perturbation is applied before a second forward pass\.

### 4\.3Theoretical analysis

We rigorously analyzeERASE, characterizing sufficient conditions under which the method provably forgetsCfC\_\{f\}while retaining non\-forgotten subclasses inℛr\\mathcal\{R\}\_\{r\}\.

Preliminaries\.Letg⁡\(x\):=∇xℒ​\(Mθ​\(x\),Sf\)g\(x\):=\\nabla\_\{x\}\\mathcal\{L\}\(M\_\{\\theta\}\(x\),S\_\{f\}\)denote the gradient of the loss w\.r\.t\. inputxxfor the target superclassSfS\_\{f\}\. Letfθ​\(x\)∈ℝ\|𝒮\|f\_\{\\theta\}\(x\)\\in\\mathbb\{R\}^\{\|\\mathcal\{S\}\|\}denote the vector of logits produced by the model\.ERASEapplies perturbations only when the model predicts the target superclass\. We define this*gate event*asℰ\(x\):=\{argmaxS∈𝒮fθ\(x\)S=Sf\}\\mathcal\{E\}\(x\):=\\\{\\arg\\max\_\{S\\in\\mathcal\{S\}\}f\_\{\\theta\}\(x\)\_\{S\}=S\_\{f\}\\\}\. For any inputxxsatisfyingℰ⁡\(x\)\\mathcal\{E\}\(x\), we define the*forget superclass margin*as the gap between the target superclass logit and the next best superclass:mf​\(x\):=\[fθ​\(x\)\]Sf−maxS≠Sf⁡\[fθ​\(x\)\]Sm\_\{f\}\(x\):=\[f\_\{\\theta\}\(x\)\]\_\{S\_\{f\}\}\-\\max\_\{S\\neq S\_\{f\}\}\[f\_\{\\theta\}\(x\)\]\_\{S\}\. Our analysis relies on the following assumptions\.

###### Assumption 4\.1\.

There existsL\>0L\>0such that for any perturbationΔ=𝒑​‖x‖F\\Delta=\\bm\{p\}\\\|x\\\|\_\{F\}:

\|ℒ⁡\(Mθ​\(x\+Δ\),Sf\)−ℒ⁡\(Mθ​\(x\),Sf\)−⟨g⁡\(x\),Δ⟩\|≤L2​‖Δ‖22\.\\left\|\\mathcal\{L\}\(M\_\{\\theta\}\(x\+\\Delta\),S\_\{f\}\)\-\\mathcal\{L\}\(M\_\{\\theta\}\(x\),S\_\{f\}\)\-\\langle g\(x\),\\Delta\\rangle\\right\|\\leq\\tfrac\{L\}\{2\}\\\|\\Delta\\\|\_\{2\}^\{2\}\.

###### Assumption 4\.2\.

For allxxsatisfyingℰ⁡\(x\)\\mathcal\{E\}\(x\):\|⟨g⁡\(x\),𝜹forget⟩\|≤Bf\|\\langle g\(x\),\\bm\{\\delta\}\_\{\\text\{forget\}\}\\rangle\|\\leq B\_\{f\}and\|⟨g⁡\(x\),𝜹retain⟩\|≤Br\|\\langle g\(x\),\\bm\{\\delta\}\_\{\\text\{retain\}\}\\rangle\|\\leq B\_\{r\}, for some constantsBf,Br\>0B\_\{f\},B\_\{r\}\>0\.

###### Assumption 4\.3\.

Phase\-1 exhibits strictly positive empirical alignment:α^f,n:=1n​∑x∈DCf⟨g⁡\(x\),𝜹forget⟩≥μf,\\hat\{\\alpha\}\_\{f,n\}:=\\frac\{1\}\{n\}\\sum\_\{x\\in D\_\{C\_\{f\}\}\}\\big\\langle g\(x\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\big\\rangle\\geq\\mu\_\{f\},andα^r,m:=1m​∑x∈DRr⟨g⁡\(x\),𝜹retain⟩≥μr,\\hat\{\\alpha\}\_\{r,m\}:=\\frac\{1\}\{m\}\\sum\_\{x\\in D\_\{R\_\{r\}\}\}\\big\\langle g\(x\),\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\big\\rangle\\geq\\mu\_\{r\},for someμf,μr\>0\\mu\_\{f\},\\mu\_\{r\}\>0, where\{xi\(f\)\}i=1n∼DCf\\\{x\_\{i\}^\{\(f\)\}\\\}\_\{i=1\}^\{n\}\\sim D\_\{C\_\{f\}\}and\{xi\(r\)\}i=1m∼DRr\\\{x\_\{i\}^\{\(r\)\}\\\}\_\{i=1\}^\{m\}\\sim D\_\{R\_\{r\}\}are the Phase\-1 datasets\.

LetPCfP\_\{C\_\{f\}\}andPRrP\_\{R\_\{r\}\}denote distributions of forget and retain subclasses\. Fixδ∈\(0,1\)\\delta\\in\(0,1\)and setδf=δr=δ/2\\delta\_\{f\}=\\delta\_\{r\}=\\delta/2\. We defineεf​\(n\):=Bf​log⁡\(2/δf\)2​n\\varepsilon\_\{f\}\(n\):=B\_\{f\}\\sqrt\{\\frac\{\\log\(2/\\delta\_\{f\}\)\}\{2n\}\}and gate probabilitiespℰ,f:=Pr⁡\(ℰ⁡\(X\)\|X∼PCf\)p\_\{\\mathcal\{E\},f\}:=\\Pr\(\\mathcal\{E\}\(X\)\|X\\sim P\_\{C\_\{f\}\}\)\(analogouslyεr​\(m\),pℰ,r\\varepsilon\_\{r\}\(m\),p\_\{\\mathcal\{E\},r\}\)\. We define the population alignment lower bounds:αf:=α^f,n−Bf​\(1−pℰ,f\)−εf​\(n\)pℰ,f,αr:=α^r,m−Br​\(1−pℰ,r\)−εr​\(m\)pℰ,r\.\\alpha\_\{f\}:=\\frac\{\\hat\{\\alpha\}\_\{f,n\}\-B\_\{f\}\(1\-p\_\{\\mathcal\{E\},f\}\)\-\\varepsilon\_\{f\}\(n\)\}\{p\_\{\\mathcal\{E\},f\}\},\\hskip 9\.24994pt\\alpha\_\{r\}:=\\frac\{\\hat\{\\alpha\}\_\{r,m\}\-B\_\{r\}\(1\-p\_\{\\mathcal\{E\},r\}\)\-\\varepsilon\_\{r\}\(m\)\}\{p\_\{\\mathcal\{E\},r\}\}\.We define the events𝒢f\\mathcal\{G\}\_\{f\}and𝒢r\\mathcal\{G\}\_\{r\}as the cases where the true conditional expected alignment exceedsαf\\alpha\_\{f\}andαr\\alpha\_\{r\}respectively\. LetK:=\|𝒮\|K:=\|\\mathcal\{S\}\|denote the number of superclasses\.

###### Theorem 4\.4\.

Let Assumptions[4\.1](https://arxiv.org/html/2609.05966#S4.Thmtheorem1)\-[4\.3](https://arxiv.org/html/2609.05966#S4.Thmtheorem3)hold and the computed Phase\-1 gradients satisfyα^f,n\>Bf​\(1−pℰ,f\)\+εf​\(n\)andα^r,m\>Br​\(1−pℰ,r\)\+εr​\(m\)\\hat\{\\alpha\}\_\{f,n\}\>B\_\{f\}\(1\-p\_\{\\mathcal\{E\},f\}\)\+\\varepsilon\_\{f\}\(n\)\\quad\\text\{and\}\\quad\\hat\{\\alpha\}\_\{r,m\}\>B\_\{r\}\(1\-p\_\{\\mathcal\{E\},r\}\)\+\\varepsilon\_\{r\}\(m\)\. Then, with probability at least1−δ1\-\\deltaover the Phase\-1 datasets, the events𝒢f\\mathcal\{G\}\_\{f\}and𝒢r\\mathcal\{G\}\_\{r\}hold with strictly positive boundsαf,αr\>0\\alpha\_\{f\},\\alpha\_\{r\}\>0\. Conditioned on this event, the following guarantees apply simultaneously\. \(i\) Forgetting: For anyX∼PCfX\\sim P\_\{C\_\{f\}\}satisfyingℰ⁡\(X\)\\mathcal\{E\}\(X\)and⟨g⁡\(X\),𝛅forget⟩≥τf\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\rangle\\geq\\tau\_\{f\}with0<τf<αf0<\\tau\_\{f\}<\\alpha\_\{f\}, the prediction changes \(arg⁡max⁡Mθ​\(Perturb​\(X\)\)≠Sf\\arg\\max M\_\{\\theta\}\(\\text\{Perturb\}\(X\)\)\\neq S\_\{f\}\) whenever

ℒ⁡\(Mθ​\(X\),Sf\)\+‖X‖F​\(ϵf​τf−ϵr​Br\)−L2​‖X‖F2​‖𝒑final‖22\>log⁡K\.\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\+\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}\\tau\_\{f\}\-\\epsilon\_\{r\}B\_\{r\}\)\-\\frac\{L\}\{2\}\\\|X\\\|\_\{F\}^\{2\}\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}\>\\log K\.\(4\)\(ii\) Retention: For anyX∼PRrX\\sim P\_\{R\_\{r\}\}satisfyingℰ⁡\(X\)\\mathcal\{E\}\(X\)and⟨g⁡\(X\),𝛅retain⟩≥τr\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\rangle\\geq\\tau\_\{r\}with0<τr<αr0<\\tau\_\{r\}<\\alpha\_\{r\}, the prediction is preserved \(arg⁡max⁡Mθ​\(Perturb​\(X\)\)=Sf\\arg\\max M\_\{\\theta\}\(\\text\{Perturb\}\(X\)\)=S\_\{f\}\) whenever

ℒ⁡\(Mθ​\(X\),Sf\)−‖X‖F​\(ϵr​τr−ϵf​Bf\)\+L2​‖X‖F2​‖𝒑final‖22<log⁡2\.\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\-\\\|X\\\|\_\{F\}\(\\epsilon\_\{r\}\\tau\_\{r\}\-\\epsilon\_\{f\}B\_\{f\}\)\+\\frac\{L\}\{2\}\\\|X\\\|\_\{F\}^\{2\}\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}<\\log 2\.\(5\)Moreover, the alignment conditions hold with

Pr⁡\(⟨g⁡\(X\),𝜹forget⟩≥τf∣X∼PCf,ℰ⁡\(X\)\)\\displaystyle\\Pr\(\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\rangle\\geq\\tau\_\{f\}\\mid X\\sim P\_\{C\_\{f\}\},\\mathcal\{E\}\(X\)\)≥αf−τfBf−τf,\\displaystyle\\geq\\frac\{\\alpha\_\{f\}\-\\tau\_\{f\}\}\{B\_\{f\}\-\\tau\_\{f\}\},Pr⁡\(⟨g⁡\(X\),𝜹retain⟩≥τr∣X∼PRr,ℰ⁡\(X\)\)\\displaystyle\\Pr\(\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\rangle\\geq\\tau\_\{r\}\\mid X\\sim P\_\{R\_\{r\}\},\\mathcal\{E\}\(X\)\)≥αr−τrBr−τr\.\\displaystyle\\geq\\frac\{\\alpha\_\{r\}\-\\tau\_\{r\}\}\{B\_\{r\}\-\\tau\_\{r\}\}\.

Theorem[4\.4](https://arxiv.org/html/2609.05966#S4.Thmtheorem4)establishes two critical conditions for ERASE\. First, certified forgetting is feasible only when the subclass\-specific gradient alignmentα^\\hat\{\\alpha\}is distinct enough to overpower worst\-case noise from model errors\. Second, the inequalities reveal a geometric trade\-off mediated by the curvatureLL: the linear terms \(ϵf,ϵr\\epsilon\_\{f\},\\epsilon\_\{r\}\) drive the necessary behavioral shift, while the quadratic penalty‖𝒑final‖22\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}limits perturbation magnitude\. The forgetting condition is expressed through a cross\-entropy threshold: pushing the post\-perturbation loss abovelog⁡K\\log Kguarantees thatSfS\_\{f\}cannot remain the argmax, while keeping the post\-perturbation retain loss belowlog⁡2\\log 2guarantees thatSfS\_\{f\}remains the unique argmax\. These thresholds are conservative but easy to verify, sinceℒ⁡\(Mθ​\(X\),Sf\)\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)is already computed during training/evaluation andKKis fixed by the number of superclasses\.ERASEsucceeds when the optimized scalings\(ϵf,ϵr\)\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)are large enough to raise the forget loss beyond thelog⁡K\\log Kthreshold, yet balanced enough to satisfy the retain\-loss threshold \(see Appendix[C](https://arxiv.org/html/2609.05966#A3)for proof details\)\.

## 5Experiments

![Refer to caption](https://arxiv.org/html/2609.05966v1/ablation_imagenet_compressed.png)Figure 2:Selective corruption viaERASEon theinsectssuperclass containing thecockroachsubclass in Tiny ImageNet\. For each subclass, 5 representative test images are shown before \(left\) and after \(right\) perturbation\.Cockroachimages become noisier, disrupting predictions, while sibling subclasses remain largely unaffected \(as also confirmed by LPIPS scores\)\.![Refer to caption](https://arxiv.org/html/2609.05966v1/ablation_imagenet_2_compressed.png)Figure 3:Effect of the retain direction𝜹retain\\bm\{\\delta\}\_\{\\text\{retain\}\}in ERASE\. Withϵr≠0\\epsilon\_\{r\}\\neq 0\(left\),cockroachpredictions are disrupted while retain classes are preserved\. Withoutϵr≠0\\epsilon\_\{r\}\\neq 0\(right\), forgetting remains but collateral errors increase\.Datasets and settings\.We benchmarkERASEacross four diverse datasets, each reflecting different input modalities and subclass–superclass relationships:

- •CIFAR\-100\([Krizhevsky et al\., 2009](https://arxiv.org/html/2609.05966#bib.bib2)\): Comprises100100fine\-grained classes \(subclasses\) grouped into2020coarse\-grained categories \(superclasses\), as defined in the original dataset\. We use ResNet18\([He et al\., 2016](https://arxiv.org/html/2609.05966#bib.bib7)\)and ViT\-B/16\([Dosovitskiy et al\., 2021](https://arxiv.org/html/2609.05966#bib.bib8)\)as pre\-trained backbone models for superclass\-level classification\.
- •Tiny ImageNet[mnmoustafa and Ali \(2017\)](https://arxiv.org/html/2609.05966#bib.bib5): A subset of ImageNet with200200subclasses\. We manually cluster them into4040semantic superclasses using WordNet hierarchies\. ViT\-B/16 serves as the backbone model\.
- •Epileptic Seizure Recognition \(ESZ\)\([Shimanto, 2018](https://arxiv.org/html/2609.05966#bib.bib3)\): An EEG\-based classification with55subclasses\. Class11corresponds to subjects with epileptic seizures, while classes22–55are of non\-seizure\. These are grouped into two superclasses:seizureandnon\-seizure\. A 3\-layer feedforward neural network is employed\.
- •Human Activity Recognition \(HAR\)\([Anguita et al\., 2013](https://arxiv.org/html/2609.05966#bib.bib4)\): A66\-class activity recognition task based on temporal sensor data from wearable devices\. We treat each activity \(e\.g\., walking, sitting\) as a distinct superclass, with subclasses defined by individual subjects performing that activity\. We use an LSTM\-based sequence model\([Hochreiter and Schmidhuber, 1997](https://arxiv.org/html/2609.05966#bib.bib9)\)for this task\.

In all cases, models are trained to predict superclass labels\. Subclass annotations are used exclusively for constructing the forget and retain sets\.

Baselines\.We compareERASEwith the following representative unlearning methods\.

- •Gradient Ascent\-Descent \(GA\+GD\)\([Graves et al\., 2021](https://arxiv.org/html/2609.05966#bib.bib16);[Thudi et al\., 2022](https://arxiv.org/html/2609.05966#bib.bib23)\): A gradient\-based fine\-tuning approach that maximizes loss on the forget set and minimizes it on the retain set\.
- •Knowledge Distillation \(KD\)\([Chundawat et al\., 2023](https://arxiv.org/html/2609.05966#bib.bib1)\): A recent approach employs a dual\-teacher framework, where an*incompetent*teacher, trained to misclassify the forget set, induces forgetting, while a*competent*teacher, trained on the retain set, preserves relevant knowledge\.
- •DEcoupLEd Distillation To Erase \(DELETE\)[Zhou et al\. \(2025b\)](https://arxiv.org/html/2609.05966#bib.bib24): A decoupled distillation framework that separates forgetting and retention, one module suppressing forget set representations and the other reinforcing retain set knowledge to avoid objective interference\.
- •Boundary Unlearning\([Chen et al\., 2023](https://arxiv.org/html/2609.05966#bib.bib25)\): A decision\-boundary approach that shifts classifier boundaries away from forget samples while preserving retain class separation, achieved via local feature perturbations\.
- •Entanglement\-Reduced Mask & Knowledge Transfer and Prohibition \(ERM\-KTP\)\([Lin et al\., 2023](https://arxiv.org/html/2609.05966#bib.bib26)\): A feature space method that masks entangled neurons to suppress forget related activations and prohibits relearning during retraining for targeted, irreversible forgetting\.

For each dataset, we fine\-tune the publicly available pretrained model for55epochs to predict superclasses, and refer it as original model throughout the paper\.

Evaluation metrics\.We employ the following metrics:

- •Forget Accuracy\(𝒜forget\\mathcal\{A\}\_\{\\text\{forget\}\}\): Accuracy on test samples from the target forget subclass\. Lower is better\.
- •Retain Superclass Accuracy\(𝒜retain\-super\\mathcal\{A\}\_\{\\text\{retain\-super\}\}\): Accuracy on samples from sibling subclasses in the same superclass\. Higher is better\.
- •Retain Overall Accuracy\(𝒜retain\-overall\\mathcal\{A\}\_\{\\text\{retain\-overall\}\}\): Accuracy on all other subclasses not equal to the target forget subclass\. Higher is better\.
- •Membership Inference Attack \(MIA\) Resistance\([Sidiropoulos et al\., 2025](https://arxiv.org/html/2609.05966#bib.bib27)\): Computed on a balanced set of samples from the forget set \(members\) and the test set \(non\-members\)\. A successful unlearning should make these distributions indistinguishable, indicated by low Area Under the Curve \(AUC\-ROC\) and a negative difference between the True Positive Rate \(TPR\) and False Positive Rate \(FPR\)\.

Since MIA primarily measures forgetting efficacy for privacy, we use it in conjunction with retention accuracy metrics\. ForERASE, the attack evaluates the final model outputMθ​\(ConditionalPerturb​\(⋅\)\)M\_\{\\theta\}\(\\text\{ConditionalPerturb\}\(\\cdot\)\)\. We discuss additional privacy risks, including white\-box limitations and timing side channels, in Appendix[D](https://arxiv.org/html/2609.05966#A4)\.

Table 1:Performance ofERASEand baseline unlearning methods across forgetting and retention accuracies\.Table 2:Performance comparison of MIA vulnerability of unlearning methods\. Lower AUC\-ROC andΔ=\\Delta=TPR−\-FPR are better\.Infrastructure & hyper\-parameters\.Experiments were conducted on Kaggle with2×2\\timesNVIDIA Tesla T4 GPUs \(15 GB each\), AWS with 15 GB NVIDIA Tesla T4 GPUs, and Intel Xeon CPUs, with PyTorch 2\.6\.0\+cu124\. We adopted the hyperparameters of baseline methods from their respective publications, when available for certain datasets[Chundawat et al\. \(2023\)](https://arxiv.org/html/2609.05966#bib.bib1);[Zhou et al\. \(2025b\)](https://arxiv.org/html/2609.05966#bib.bib24);[Chen et al\. \(2023\)](https://arxiv.org/html/2609.05966#bib.bib25);[Lin et al\. \(2023\)](https://arxiv.org/html/2609.05966#bib.bib26)\. Our MIA employed a two\-layer MLP attacker with a 64\-neuron hidden layer \(ReLU activation\), which takes the target model’s softmax output as input\. Each attacker was trained for 10 epochs using Adam with learning rate 0\.001\. Hyperparameters of all unlearning methods are described in the supplementary text\.

Results\.Our experimental results, presented in Table[1](https://arxiv.org/html/2609.05966#S5.T1)and Table[2](https://arxiv.org/html/2609.05966#S5.T2), demonstrate that by decoupling the act of forgetting from model parameter modification and operating solely at inference time,ERASEnot only achieves effective forgetting but, critically, preserves the model’s vast repository of learned knowledge\. This approach not only eliminates the computationally prohibitive overhead of retraining but also addresses the critical need for real\-time privacy compliance\.

•Preservation of global knowledge\.The primary weakness of conventional unlearning methods is catastrophic forgetting, i\.e\., the unintended erasure of valuable, unrelated knowledge\. Table[1](https://arxiv.org/html/2609.05966#S5.T1)unequivocally showsERASE’s dominance in preventing this\. On the large\-scale Tiny ImageNet \(ViT\) dataset,ERASEachieves aretain overall accuracyof82\.76%, nearly matching the original model \(83\.47%\) and decisively outperforming all retraining\-based competitors\. Methods like DELETE \(34\.03%\), Boundary Unlearning \(57\.73%\), and GA\+GD \(67\.45%\) suffer from extensive collateral damage\. Similarly, on CIFAR\-100/ViT,ERASEsets a new SOTA with88\.73%retain overall accuracy, proving its ability to surgically target information for forgetting\. This performance is a direct result of our “forgetting without unlearning” approach–keeping the model frozen and conditionally perturbing the inputs\. •Targeted forgetting\.While preserving knowledge is paramount, effective forgetting is the primary goal\.ERASEdemonstrates strong performance here as well, achieving a superior balance\. Unlike methods such as DELETE and Boundary Unlearning, which achieve near\-zeroforget accuracyat the expense of severe model degradation \(e\.g\., DELETE’s 0\.01% forget accuracy on Tiny ImageNet causes a 74% drop in superclass retention\),ERASEachieves a competitiveforget accuracyof 14\.64% while keepingretain superclass accuracyhigh at 68\.80%, comparable to KD \(70\.93%\)\. This highlightsERASE’s ability to perform forgetting without disrupting closely related concepts\. Furthermore, on real\-world modalities ESZ and HAR,ERASEachieves second\-bestforget accuracies\(71\.09%and76%respectively\), validating our input\-space methodology’s robustness beyond vision tasks\. •A superior forgetting trade\-off for deployed systems\.Across all datasets,ERASEconsistently delivers the most practical and desirable trade\-off between forgetting and retention\. It avoids the false choice offered by other methods: either incomplete forgetting or a broken model\. •Robustness to privacy attacks\.Beyond accuracy, we assessERASE’s privacy guarantees through MIA, as summarized in Table[2](https://arxiv.org/html/2609.05966#S5.T2)\.ERASEconsistently demonstrates robust defense, achieving low AUC\-ROC scores \(e\.g\., 0\.37 on CIFAR\-100/ResNet18, 0\.40 on Tiny ImageNet\)\. Notably, its MIA resistance is on par with traditional weight\-tuning methods, confirming thatERASEobscures the influence of forgotten data via input\-space manipulation alone\. •Ablation studies\.To understand the contribution of key components inERASE, we conduct ablation experiment on Tiny ImageNet\. Figure[2](https://arxiv.org/html/2609.05966#S5.F2)visualizes the impact of our targeted input perturbation on thecockroachsubclass within theinsectssuperclass\. The figure shows representative test samples from all subclasses ininsects\(55images per subclass\), before and after perturbation, i\.e\.,xxandPerturb​\(x\)\\text\{Perturb\}\(x\)\. We observe thatcockroachimages become noticeably blurrier after perturbation, often disrupting the model’s prediction\. In contrast, sibling subclasses remain visually sharp and largely unaffected, showcasingERASEpreserves perceptual quality and predictive fidelity for the retain setℛr\\mathcal\{R\}\_\{r\}, validating its targeted forgetting\.

Figure[3](https://arxiv.org/html/2609.05966#S5.F3)further elucidates the crucial role of the retain direction𝜹retain\\bm\{\\delta\}\_\{\\text\{retain\}\}\. We compare predictions oninsectssuperclass from Tiny ImageNet with and without𝜹retain\\bm\{\\delta\}\_\{\\text\{retain\}\}\(i\.e\.,ϵr≠0\\epsilon\_\{r\}\\neq 0vs\.ϵr=0\\epsilon\_\{r\}=0\)\. Whenϵr≠0\\epsilon\_\{r\}\\neq 0,ERASEsuccessfully degrades predictions for the forget class \(top row\), while preserving correct superclass predictions for retain subclasses \(bottom rows\)\. However, removing the retain term \(ϵr=0\\epsilon\_\{r\}=0\) causes misclassifications for retain examples, e\.g\.,mantismisclassified asfelines,dragonflyasreptiles & arthropods\. It demonstrates𝜹retain\\bm\{\\delta\}\_\{\\text\{retain\}\}’s role in maintaining model utility on the retain set\.

## 6Conclusion and Limitation

The dominant approach to data removal in machine learning has long relied on retraining or fine\-tuning to achieve explicit unlearning\. In this work, we challenged that paradigm and demonstrated that effective erasure does not require modifying a model’s parameters\. We introducedERASE, a framework for on\-the\-go forgetting that operates at inference time\. By reframing data deletion as a problem of behavioral suppression rather than structural modification,ERASEenables models to dynamically forget sensitive information while maintaining their original weights and predictive capability\. This design delivers immediate, scalable, and regulation\-compliant data removal without retraining overhead\. Our experiments across diverse modalities show thatERASEachieves a superior balance between forgetting efficacy and retention fidelity, providing a principled pathway toward continual, privacy\-conscious AI systems\.

ERASEis designed for discriminative models and does not directly extend to generative architectures or settings without a meaningful subclass–superclass organization\. Its guarantees concern*functional forgetting*rather than structural unlearning: the influence of the forget set may remain encoded in the frozen model parameters and could therefore be exposed under white\-box, reconstruction\-based, or other adaptive attacks\. Moreover, because perturbation is triggered using the model’s initial superclass prediction, gating errors may prevent forgetting for misclassified target samples or unnecessarily perturb retained samples\. The current privacy evaluation is limited to black\-box membership inference through output probabilities, while stronger threat models and repeated or simultaneous forgetting requests remain to be studied\.

## Broader Impact Statement

This paper presents work whose goal is to advance the field of Machine Learning\. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here\.

## References

- Anguitaet al\.\(2013\)D\. Anguita, A\. Ghio, L\. Oneto, X\. Parra, J\. L\. Reyes\-Ortiz,et al\.A public domain dataset for human activity recognition using smartphones\.\.InEsann,Vol\.3,pp\. 3–4\.Cited by:[4th item](https://arxiv.org/html/2609.05966#S5.I1.i4.p1.1)\.
- Baumhaueret al\.\(2022\)T\. Baumhauer, P\. Schöttle, and M\. ZeppelzauerMachine unlearning: linear filtration for logit\-based classifiers\.Machine Learning111\(9\),pp\. 3203–3226\.Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p6.1)\.
- Bourtouleet al\.\(2021\)L\. Bourtoule, V\. Chandrasekaran, C\. A\. Choquette\-Choo, H\. Jia, A\. Travers, B\. Zhang, D\. Lie, and N\. PapernotMachine unlearning\.In2021 IEEE symposium on security and privacy \(SP\),pp\. 141–159\.Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p4.1)\.
- Carlini and Wagner \(2017\)N\. Carlini and D\. WagnerTowards evaluating the robustness of neural networks\.In2017 ieee symposium on security and privacy \(sp\),pp\. 39–57\.Cited by:[§4\.1](https://arxiv.org/html/2609.05966#S4.SS1.p3.4)\.
- Chenet al\.\(2023\)M\. Chen, W\. Gao, G\. Liu, K\. Peng, and C\. WangBoundary unlearning: rapid forgetting of deep networks via shifting the decision boundary\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 7766–7775\.Cited by:[Appendix A](https://arxiv.org/html/2609.05966#A1.p1.1),[§2](https://arxiv.org/html/2609.05966#S2.p6.1),[4th item](https://arxiv.org/html/2609.05966#S5.I2.i4.p1.1),[Table 1](https://arxiv.org/html/2609.05966#S5.T1.2.1.1.1.1.1.1.14.1.1),[Table 2](https://arxiv.org/html/2609.05966#S5.T2.2.1.1.1.1.1.1.10.1.1),[§5](https://arxiv.org/html/2609.05966#S5.p4.1)\.
- Chundawatet al\.\(2023\)V\. S\. Chundawat, A\. K\. Tarun, M\. Mandal, and M\. KankanhalliCan bad teaching induce forgetting? Unlearning in deep networks using an incompetent teacher\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.37,pp\. 7210–7217\.Cited by:[Appendix A](https://arxiv.org/html/2609.05966#A1.p1.1),[§2](https://arxiv.org/html/2609.05966#S2.p7.1),[2nd item](https://arxiv.org/html/2609.05966#S5.I2.i2.p1.1),[Table 1](https://arxiv.org/html/2609.05966#S5.T1.2.1.1.1.1.1.1.5.1.1),[Table 2](https://arxiv.org/html/2609.05966#S5.T2.2.1.1.1.1.1.1.4.1.1),[§5](https://arxiv.org/html/2609.05966#S5.p4.1)\.
- Dosovitskiyet al\.\(2021\)A\. Dosovitskiy, L\. Beyer, A\. Kolesnikov, D\. Weissenborn, X\. Zhai, T\. Unterthiner, M\. Dehghani, M\. Minderer, G\. Heigold, S\. Gelly, J\. Uszkoreit, and N\. HoulsbyAn image is worth 16x16 words: transformers for image recognition at scale\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2010.11929)Cited by:[1st item](https://arxiv.org/html/2609.05966#S5.I1.i1.p1.1)\.
- Ginartet al\.\(2019\)A\. Ginart, M\. Guan, G\. Valiant, and J\. Y\. ZouMaking AI forget you: data deletion in machine learning\.Advances in neural information processing systems32\.Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p3.1)\.
- Golatkaret al\.\(2020\)A\. Golatkar, A\. Achille, and S\. SoattoEternal sunshine of the spotless net: selective forgetting in deep networks\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 9304–9312\.Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p5.1)\.
- Goodfellowet al\.\(2015\)I\. J\. Goodfellow, J\. Shlens, and C\. SzegedyExplaining and harnessing adversarial examples\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/1412.6572)Cited by:[§4\.1](https://arxiv.org/html/2609.05966#S4.SS1.p2.2)\.
- Graveset al\.\(2021\)L\. Graves, V\. Nagisetty, and V\. GaneshAmnesiac machine learning\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 11516–11524\.Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p5.1),[§2](https://arxiv.org/html/2609.05966#S2.p6.1),[1st item](https://arxiv.org/html/2609.05966#S5.I2.i1.p1.1),[Table 1](https://arxiv.org/html/2609.05966#S5.T1.2.1.1.1.1.1.1.8.1.1),[Table 2](https://arxiv.org/html/2609.05966#S5.T2.2.1.1.1.1.1.1.6.1.1)\.
- Guoet al\.\(2020\)C\. Guo, T\. Goldstein, A\. Hannun, and L\. Van Der MaatenCertified data removal from machine learning models\.InProceedings of the 37th International Conference on Machine Learning,pp\. 3832–3842\.Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p3.1),[§2](https://arxiv.org/html/2609.05966#S2.p5.1)\.
- Heet al\.\(2016\)K\. He, X\. Zhang, S\. Ren, and J\. SunDeep residual learning for image recognition\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 770–778\.Cited by:[1st item](https://arxiv.org/html/2609.05966#S5.I1.i1.p1.1)\.
- Hochreiter and Schmidhuber \(1997\)S\. Hochreiter and J\. SchmidhuberLong short\-term memory\.Neural computation9\(8\),pp\. 1735–1780\.Cited by:[4th item](https://arxiv.org/html/2609.05966#S5.I1.i4.p1.1)\.
- Koh and Liang \(2017\)P\. W\. Koh and P\. LiangUnderstanding black\-box predictions via influence functions\.InInternational conference on machine learning,pp\. 1885–1894\.Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p5.1)\.
- Krizhevskyet al\.\(2009\)A\. Krizhevsky G\. Hintonet al\.Learning multiple layers of features from tiny images\.Cited by:[1st item](https://arxiv.org/html/2609.05966#S5.I1.i1.p1.1)\.
- Liet al\.\(2025\)N\. Li, C\. Zhou, Y\. Gao, H\. Chen, Z\. Zhang, B\. Kuang, and A\. FuMachine unlearning: taxonomy, metrics, applications, challenges, and prospects\.IEEE Transactions on Neural Networks and Learning Systems36\(8\),pp\. 13709–13729\.External Links:[Document](https://dx.doi.org/10.1109/TNNLS.2025.3530988)Cited by:[§1](https://arxiv.org/html/2609.05966#S1.p1.1),[§2](https://arxiv.org/html/2609.05966#S2.p1.1)\.
- Liet al\.\(2024\)N\. Li, A\. Pan, A\. Gopal, S\. Yue, D\. Berrios, A\. Gatti, J\. D\. Li, A\. Dombrowski, S\. Goel, G\. Mukobi, N\. Helm\-Burger, R\. Lababidi, L\. Justen, A\. B\. Liu, M\. Chen, I\. Barrass, O\. Zhang, X\. Zhu, R\. Tamirisa, B\. Bharathi, A\. Herbert\-Voss, C\. B\. Breuer, A\. Zou, M\. Mazeika, Z\. Wang, P\. Oswal, W\. Lin, A\. A\. Hunt, J\. Tienken\-Harder, K\. Y\. Shih, K\. Talley, J\. Guan, I\. Steneker, D\. Campbell, B\. Jokubaitis, S\. Basart, S\. Fitz, P\. Kumaraguru, K\. K\. Karmakar, U\. Tupakula, V\. Varadharajan, Y\. Shoshitaishvili, J\. Ba, K\. M\. Esvelt, A\. Wang, and D\. HendrycksThe WMDP benchmark: measuring and reducing malicious use with unlearning\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 28525–28550\.External Links:[Link](https://proceedings.mlr.press/v235/li24bc.html)Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p9.1)\.
- Linet al\.\(2023\)S\. Lin, X\. Zhang, C\. Chen, X\. Chen, and W\. SusiloERM\-KTP: knowledge\-level machine unlearning via knowledge transfer\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 20147–20155\.Cited by:[Appendix A](https://arxiv.org/html/2609.05966#A1.p1.1),[§2](https://arxiv.org/html/2609.05966#S2.p6.1),[5th item](https://arxiv.org/html/2609.05966#S5.I2.i5.p1.1),[Table 1](https://arxiv.org/html/2609.05966#S5.T1.2.1.1.1.1.1.1.17.1.1),[Table 2](https://arxiv.org/html/2609.05966#S5.T2.2.1.1.1.1.1.1.12.1.1),[§5](https://arxiv.org/html/2609.05966#S5.p4.1)\.
- Mahadevan and Mathioudakis \(2022\)A\. Mahadevan and M\. MathioudakisCertifiable unlearning pipelines for logistic regression: an experimental study\.Machine Learning and Knowledge Extraction4\(3\),pp\. 591–620\.Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p3.1)\.
- mnmoustafa and Ali \(2017\)mnmoustafa and M\. AliTiny imagenet\.Note:[https://kaggle\.com/competitions/tiny\-imagenet](https://kaggle.com/competitions/tiny-imagenet)KaggleCited by:[2nd item](https://arxiv.org/html/2609.05966#S5.I1.i2.p1.1)\.
- Moosavi\-Dezfooliet al\.\(2016\)S\. Moosavi\-Dezfooli, A\. Fawzi, and P\. FrossardDeepfool: a simple and accurate method to fool deep neural networks\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 2574–2582\.Cited by:[§4\.1](https://arxiv.org/html/2609.05966#S4.SS1.p3.4)\.
- Nguyenet al\.\(2022\)T\. T\. Nguyen, T\. T\. Huynh, Z\. Ren, P\. L\. Nguyen, A\. W\. Liew, H\. Yin, and Q\. V\. H\. NguyenA survey of machine unlearning\.arXiv preprint arXiv:2209\.02299\.Cited by:[§1](https://arxiv.org/html/2609.05966#S1.p2.1),[§2](https://arxiv.org/html/2609.05966#S2.p1.1)\.
- PENGet al\.\(2025\)Z\. PENG, Y\. Tang, and Y\. YangAdversarial mixup unlearning\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=GcbhbZsgiu)Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p8.1)\.
- Shimanto \(2018\)H\. ShimantoEpileptic seizure recognition\.Note:Kaggle, Accessed: 2025\-05\-07[https://www\.kaggle\.com/datasets/harunshimanto/epileptic\-seizure\-recognition](https://www.kaggle.com/datasets/harunshimanto/epileptic-seizure-recognition)Cited by:[3rd item](https://arxiv.org/html/2609.05966#S5.I1.i3.p1.1)\.
- Shokriet al\.\(2017\)R\. Shokri, M\. Stronati, C\. Song, and V\. ShmatikovMembership inference attacks against machine learning models\.In2017 IEEE symposium on security and privacy \(SP\),pp\. 3–18\.Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p5.1)\.
- Sidiropouloset al\.\(2025\)A\. Sidiropoulos, C\. C\. Nikolaidis, T\. Tsiolakis, N\. Pavlidis, V\. Perifanis, and P\. S\. EfraimidisEvaluating the defense potential of machine unlearning against membership inference attacks\.arXiv preprint arXiv:2508\.16150\.Cited by:[4th item](https://arxiv.org/html/2609.05966#S5.I3.i4.p1.1)\.
- Sunet al\.\(2024\)C\. Sun, R\. Wang, Y\. Zhang, J\. Jia, J\. Liu, G\. Liu, Y\. Yan, and S\. LiuForget vectors at play: universal input perturbations driving machine unlearning in image classification\.arXiv preprint arXiv:2412\.16780\.Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p8.1)\.
- Thudiet al\.\(2022\)A\. Thudi, G\. Deza, V\. Chandrasekaran, and N\. PapernotUnrolling SGD: understanding factors influencing machine unlearning\.In2022 IEEE 7th European Symposium on Security and Privacy \(EuroS&P\),pp\. 303–319\.Cited by:[1st item](https://arxiv.org/html/2609.05966#S5.I2.i1.p1.1),[Table 1](https://arxiv.org/html/2609.05966#S5.T1.2.1.1.1.1.1.1.8.1.1),[Table 2](https://arxiv.org/html/2609.05966#S5.T2.2.1.1.1.1.1.1.6.1.1)\.
- Wanget al\.\(2025\)Y\. Wang, J\. Wei, C\. Y\. Liu, J\. Pang, Q\. Liu, A\. Shah, Y\. Bao, Y\. Liu, and W\. WeiLLM unlearning via loss adjustment with only forget data\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=6ESRicalFE)Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p9.1)\.
- Yuanet al\.\(2025\)X\. Yuan, T\. Pang, C\. Du, K\. Chen, W\. Zhang, and M\. LinA closer look at machine unlearning for large language models\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Q1MHvGmhyT)Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p9.1)\.
- Zhouet al\.\(2025a\)W\. Zhou, X\. Cheng, Y\. Wu, R\. Yang, and X\. HuangStimulating catastrophic forgetting in class\-wise unlearning via uap\.InJoint European Conference on Machine Learning and Knowledge Discovery in Databases,pp\. 432–448\.Cited by:[§2](https://arxiv.org/html/2609.05966#S2.p8.1)\.
- Zhouet al\.\(2025b\)Y\. Zhou, D\. Zheng, Q\. Mo, R\. Lu, K\. Lin, and W\. ZhengDecoupled distillation to erase: a general unlearning method for any class\-centric tasks\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 20350–20359\.Cited by:[Appendix A](https://arxiv.org/html/2609.05966#A1.p1.1),[§2](https://arxiv.org/html/2609.05966#S2.p7.1),[3rd item](https://arxiv.org/html/2609.05966#S5.I2.i3.p1.1),[Table 1](https://arxiv.org/html/2609.05966#S5.T1.2.1.1.1.1.1.1.11.1.1),[Table 2](https://arxiv.org/html/2609.05966#S5.T2.2.1.1.1.1.1.1.8.1.1),[§5](https://arxiv.org/html/2609.05966#S5.p4.1)\.

## Appendix AHyper\-parameters details

Batch sizes were 64 \(CIFAR, ESZ\), 16 \(Tiny ImageNet\), and 128 \(HAR\)\. For the baseline methods, we adopted the hyperparameters from their respective publications when available\([Chundawat et al\., 2023](https://arxiv.org/html/2609.05966#bib.bib1);[Zhou et al\., 2025b](https://arxiv.org/html/2609.05966#bib.bib24);[Chen et al\., 2023](https://arxiv.org/html/2609.05966#bib.bib25);[Lin et al\., 2023](https://arxiv.org/html/2609.05966#bib.bib26)\)\.

Specifically, KD used Adam with learning rate 0\.0001 \(CIFAR, Tiny ImageNet\), 0\.01 \(ESZ\), and 0\.001 \(HAR\); unlearning epoch was 1\.

GA\+GD used the same hyperparameters as KD\.

DELETE used SGD with learning rate 0\.001 \(ResNet\), 0\.0003 \(ViT\), 0\.03 \(HAR\) and momentum 0\.9; Adam with learning rate 0\.01 \(ESZ\); unlearning epochs were 2 \(ResNet\), 1 \(ViT, ESZ\), and 5 \(HAR\)\.

Boundary unlearning used adversarial noise bound 0\.1; poison epochs 10; SGD with learning rate 0\.00001 and momentum 0\.9; curvature regularization coefficient 0\.7; adjustment bias and slope in reweighting scheme as \-0\.5 and 5; step size for approximating the curvature 0\.9\.

ERM\-KTP used distillation loss coefficient 0\.1; forget set loss coefficient 1; temperature 2; unlearning epochs 1 \(CIFAR, Tiny ImageNet\), 5 \(HAR\), and 10 \(ESZ\); SGD with learning rate 0\.01 \(CIFAR\) and 0\.001 \(Tiny ImageNet, ESZ, HAR\), momentum 0\.9, and weight decay 0\.0005\.

InERASE,η\\etawas set to 0\.01 \(CIFAR, ESZ\) and 0\.001 \(Tiny ImageNet, HAR\), selected from0\.1,0\.01,0\.001\{0\.1,0\.01,0\.001\}; weights\(ωu,ωr\)=\(1\.5,0\.8\)\(\\omega\_\{u\},\\omega\_\{r\}\)=\(1\.5,0\.8\), chosen from\[1,2\]×\[0\.5,1\]\{\[1,2\]\\times\[0\.5,1\]\}, account for the smaller forget set\.

Our MIA employed a two\-layer MLP attacker with a 64\-neuron hidden layer \(ReLU activation\), which takes the target model’s softmax output as input\. To ensure a robust evaluation, a distinct attacker, each trained for 10 epochs on a balanced dataset of ‘member’ and ‘non\-member’ samples, was specialized for each subclass\.

## Appendix BSensitivity to initialization and learning rate of\(ϵf,ϵr\)\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)

We present sensitivity results on CIFAR\-100/ResNet18 for two key optimization choices: initialization of\(ϵf,ϵr\)\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)and learning rateη\\eta, while keeping all other parameters fixed at their values mentioned above\. The results in Tables[3](https://arxiv.org/html/2609.05966#A2.T3)\-[4](https://arxiv.org/html/2609.05966#A2.T4)show that ERASE is relatively stable across these choices\. Specifically, retain\-overall accuracy remains around81\.581\.5\-81\.781\.7, while forget and retain\-superclass accuracies vary moderately, reflecting the expected forgetting\-retention trade\-off\. These additional results support that theϵ\\epsilon\-optimization is not highly sensitive to reasonable choices of initialization or learning rate\.

Table 3:Sensitivity to initialization of\(ϵf,ϵr\)\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)on CIFAR\-100/ResNet18\.Table 4:Sensitivity to learning rateη\\etaon CIFAR\-100/ResNet18\.
## Appendix CTheoretical analysis and proofs

In this section, we provide the complete theoretical analysis and rigorous proofs for the guarantees presented in the main paper\. For completeness and to ensure this section is self\-contained, we restate the necessary notations and formalize the assumptions introduced in Section[4](https://arxiv.org/html/2609.05966#S4)below, before presenting the detailed proof of Theorem[4\.4](https://arxiv.org/html/2609.05966#S4.Thmtheorem4)\.

### C\.1Preliminaries and notation

We consider a fixed, pre\-trained classifierfθ:ℝd→ℝ\|𝒮\|f\_\{\\theta\}:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}^\{\|\\mathcal\{S\}\|\}, where𝒮\\mathcal\{S\}is the set of superclasses\. For an inputx∈ℝdx\\in\\mathbb\{R\}^\{d\}, the output logits are denotedfθ​\(x\)=\(fθ​\(x\)S\)S∈𝒮f\_\{\\theta\}\(x\)=\(f\_\{\\theta\}\(x\)\_\{S\}\)\_\{S\\in\\mathcal\{S\}\}\. The predicted superclass isS^​\(x\):=arg⁡maxS∈𝒮​fθ​\(x\)S\\hat\{S\}\(x\):=\\arg\\max\_\{S\\in\\mathcal\{S\}\}f\_\{\\theta\}\(x\)\_\{S\}\.

We focus on a specific target superclassSfS\_\{f\}\. WithinSfS\_\{f\}, we designate a*forgotten subclass*CfC\_\{f\}and a set of*retained subclasses*Rr=Sf∖\{Cf\}R\_\{r\}=S\_\{f\}\\setminus\\\{C\_\{f\}\\\}\. LetPCfP\_\{C\_\{f\}\}andPRrP\_\{R\_\{r\}\}denote the data distributions for the forgotten and retained subclasses, respectively\. ERASE applies perturbations strictly conditionally\. We define the gate eventℰ⁡\(x\)\\mathcal\{E\}\(x\)as the event where the model predicts the target superclass:

ℰ\(x\):=\{S^\(x\)=Sf\}\.\\mathcal\{E\}\(x\):=\\\{\\hat\{S\}\(x\)=S\_\{f\}\\\}\.\(6\)Ifℰ⁡\(x\)\\mathcal\{E\}\(x\)does not hold, the input is unmodified, and the original prediction is retained\. Our analysis focuses on the non\-trivial case whereℰ⁡\(x\)\\mathcal\{E\}\(x\)holds\.

Letℒ⁡\(Mθ​\(x\),Sf\)\\mathcal\{L\}\(M\_\{\\theta\}\(x\),S\_\{f\}\)denote the cross\-entropy loss with respect to the superclassSfS\_\{f\}\. We define the gradient of the loss with respect to the input as:

g⁡\(x\):=∇xℒ​\(Mθ​\(x\),Sf\)\.g\(x\):=\\nabla\_\{x\}\\mathcal\{L\}\(M\_\{\\theta\}\(x\),S\_\{f\}\)\.\(7\)We also define the*forget superclass margin*mf​\(x\)m\_\{f\}\(x\), which quantifies the confidence of the model in the target superclassSfS\_\{f\}relative to the next best superclass:

mf​\(x\):=fθ​\(x\)Sf−maxS≠Sf⁡fθ​\(x\)S\.m\_\{f\}\(x\):=f\_\{\\theta\}\(x\)\_\{S\_\{f\}\}\-\\max\_\{S\\neq S\_\{f\}\}f\_\{\\theta\}\(x\)\_\{S\}\.\(8\)A negative marginmf​\(x\)<0m\_\{f\}\(x\)<0impliesS^​\(x\)≠Sf\\hat\{S\}\(x\)\\neq S\_\{f\}\. Therefore, the condition for successful forgetting for an inputx∈Cfx\\in C\_\{f\}\(where initiallyS^​\(x\)=Sf\\hat\{S\}\(x\)=S\_\{f\}\) is to induce a perturbationΔ\\Deltasuch thatmf​\(x\+Δ\)<0m\_\{f\}\(x\+\\Delta\)<0\.

### C\.2Assumptions

We adopt the following assumptions \(renumbered from Assumptions[4\.1](https://arxiv.org/html/2609.05966#S4.Thmtheorem1)\-[4\.3](https://arxiv.org/html/2609.05966#S4.Thmtheorem3)\)\.

###### Assumption C\.1\(Directional Smoothness\)\.

The loss functionℒ\\mathcal\{L\}is smooth with respect to the input\. Specifically, there exists a curvature constantL\>0L\>0such that for any inputxxand perturbationΔ=𝒑​‖x‖F\\Delta=\\bm\{p\}\\\|x\\\|\_\{F\}:

\|ℒ⁡\(Mθ​\(x\+Δ\),Sf\)−ℒ⁡\(Mθ​\(x\),Sf\)−⟨g⁡\(x\),Δ⟩\|≤L2​‖Δ‖22\.\\left\|\\mathcal\{L\}\(M\_\{\\theta\}\(x\+\\Delta\),S\_\{f\}\)\-\\mathcal\{L\}\(M\_\{\\theta\}\(x\),S\_\{f\}\)\-\\langle g\(x\),\\Delta\\rangle\\right\|\\leq\\tfrac\{L\}\{2\}\\\|\\Delta\\\|\_\{2\}^\{2\}\.\(9\)

###### Assumption C\.2\(Bounded Projections\)\.

For all inputsxxsatisfying the gate eventℰ⁡\(x\)\\mathcal\{E\}\(x\), the projection of the gradients onto the pre\-computed directions is bounded:

\|⟨g⁡\(x\),𝜹forget⟩\|≤Bf,and\|⟨g⁡\(x\),𝜹retain⟩\|≤Br,\|\\langle g\(x\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\rangle\|\\leq B\_\{f\},\\quad\\text\{and\}\\quad\|\\langle g\(x\),\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\rangle\|\\leq B\_\{r\},\(10\)for finite constantsBf,Br\>0B\_\{f\},B\_\{r\}\>0\.

###### Assumption C\.3\(Empirical Alignment\)\.

The offline Phase\-1 datasets𝒟Cf=\{xi\(f\)\}i=1n∼PCf\\mathcal\{D\}\_\{C\_\{f\}\}=\\\{x\_\{i\}^\{\(f\)\}\\\}\_\{i=1\}^\{n\}\\sim P\_\{C\_\{f\}\}and𝒟Rr=\{xj\(r\)\}j=1m∼PRr\\mathcal\{D\}\_\{R\_\{r\}\}=\\\{x\_\{j\}^\{\(r\)\}\\\}\_\{j=1\}^\{m\}\\sim P\_\{R\_\{r\}\}exhibit strictly positive empirical alignment with their respective computed directions:

α^f,n:=1n​∑i=1n⟨g⁡\(xi\(f\)\),𝜹forget⟩≥μf\>0,α^r,m:=1m​∑j=1m⟨g⁡\(xj\(r\)\),𝜹retain⟩≥μr\>0\.\\hat\{\\alpha\}\_\{f,n\}:=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\langle g\(x\_\{i\}^\{\(f\)\}\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\rangle\\geq\\mu\_\{f\}\>0,\\quad\\hat\{\\alpha\}\_\{r,m\}:=\\frac\{1\}\{m\}\\sum\_\{j=1\}^\{m\}\\langle g\(x\_\{j\}^\{\(r\)\}\),\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\rangle\\geq\\mu\_\{r\}\>0\.\(11\)

### C\.3Proof of Theorem 1

Recall the following notation from Section[4\.3](https://arxiv.org/html/2609.05966#S4.SS3)\. We letPCfP\_\{C\_\{f\}\}andPRrP\_\{R\_\{r\}\}denote the distributions of the forget and retain subclasses\. We fixedδ∈\(0,1\)\\delta\\in\(0,1\)and setδf=δr=δ/2\\delta\_\{f\}=\\delta\_\{r\}=\\delta/2\. We defined the gate event probabilitiespℰ,f:=Pr⁡\(ℰ⁡\(X\)∣X∼PCf\)p\_\{\\mathcal\{E\},f\}:=\\Pr\(\\mathcal\{E\}\(X\)\\mid X\\sim P\_\{C\_\{f\}\}\)andpℰ,r:=Pr⁡\(ℰ⁡\(X\)∣X∼PRr\)p\_\{\\mathcal\{E\},r\}:=\\Pr\(\\mathcal\{E\}\(X\)\\mid X\\sim P\_\{R\_\{r\}\}\)\. To characterize the statistical error, letεf​\(n\):=Bf​log⁡\(2/δf\)2​n\\varepsilon\_\{f\}\(n\):=B\_\{f\}\\sqrt\{\\frac\{\\log\(2/\\delta\_\{f\}\)\}\{2n\}\}andεr​\(m\):=Br​log⁡\(2/δr\)2​m\\varepsilon\_\{r\}\(m\):=B\_\{r\}\\sqrt\{\\frac\{\\log\(2/\\delta\_\{r\}\)\}\{2m\}\}\. We defined the population alignment lower boundsαf\\alpha\_\{f\}andαr\\alpha\_\{r\}as follows:

αf\\displaystyle\\alpha\_\{f\}:=α^f,n−Bf​\(1−pℰ,f\)−εf​\(n\)pℰ,f,\\displaystyle:=\\frac\{\\hat\{\\alpha\}\_\{f,n\}\-B\_\{f\}\(1\-p\_\{\\mathcal\{E\},f\}\)\-\\varepsilon\_\{f\}\(n\)\}\{p\_\{\\mathcal\{E\},f\}\},αr\\displaystyle\\alpha\_\{r\}:=α^r,m−Br​\(1−pℰ,r\)−εr​\(m\)pℰ,r\.\\displaystyle:=\\frac\{\\hat\{\\alpha\}\_\{r,m\}\-B\_\{r\}\(1\-p\_\{\\mathcal\{E\},r\}\)\-\\varepsilon\_\{r\}\(m\)\}\{p\_\{\\mathcal\{E\},r\}\}\.We defined the alignment events:

𝒢f\\displaystyle\\mathcal\{G\}\_\{f\}:=\{𝔼\[⟨g\(X\),𝜹forget⟩∣X∼PCf,ℰ\(X\)\]≥αf\},\\displaystyle:=\\\{\\mathbb\{E\}\[\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\rangle\\mid X\\sim P\_\{C\_\{f\}\},\\mathcal\{E\}\(X\)\]\\geq\\alpha\_\{f\}\\\},𝒢r\\displaystyle\\mathcal\{G\}\_\{r\}:=\{𝔼\[⟨g\(X\),𝜹retain⟩∣X∼PRr,ℰ\(X\)\]≥αr\}\.\\displaystyle:=\\\{\\mathbb\{E\}\[\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\rangle\\mid X\\sim P\_\{R\_\{r\}\},\\mathcal\{E\}\(X\)\]\\geq\\alpha\_\{r\}\\\}\.
###### Theorem C\.4\.

Assume Assumptions[C\.1](https://arxiv.org/html/2609.05966#A3.Thmtheorem1)\-[C\.3](https://arxiv.org/html/2609.05966#A3.Thmtheorem3)hold and the computed Phase\-1 gradients satisfy:

α^f,n\>Bf​\(1−pℰ,f\)\+εf​\(n\)andα^r,m\>Br​\(1−pℰ,r\)\+εr​\(m\)\.\\hat\{\\alpha\}\_\{f,n\}\>B\_\{f\}\(1\-p\_\{\\mathcal\{E\},f\}\)\+\\varepsilon\_\{f\}\(n\)\\quad\\text\{and\}\\quad\\hat\{\\alpha\}\_\{r,m\}\>B\_\{r\}\(1\-p\_\{\\mathcal\{E\},r\}\)\+\\varepsilon\_\{r\}\(m\)\.LetK:=\|𝒮\|K:=\|\\mathcal\{S\}\|denote the number of superclasses\. Then, with probability at least1−δ1\-\\deltaover the Phase\-1 datasets, the events𝒢f\\mathcal\{G\}\_\{f\}and𝒢r\\mathcal\{G\}\_\{r\}hold with strictly positive boundsαf,αr\>0\\alpha\_\{f\},\\alpha\_\{r\}\>0\. Conditioned on this event, the following guarantees apply simultaneously:

1. \(i\)Forgetting:For anyX∼PCfX\\sim P\_\{C\_\{f\}\}satisfying the gate eventℰ⁡\(X\)\\mathcal\{E\}\(X\)and the alignment condition⟨g⁡\(X\),𝜹forget⟩≥τf\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\rangle\\geq\\tau\_\{f\}with0<τf<αf0<\\tau\_\{f\}<\\alpha\_\{f\}, the prediction changes \(arg⁡max⁡Mθ​\(Perturb​\(X\)\)≠Sf\\arg\\max M\_\{\\theta\}\(\\text\{Perturb\}\(X\)\)\\neq S\_\{f\}\) whenever ℒ⁡\(Mθ​\(X\),Sf\)\+‖X‖F​\(ϵf​τf−ϵr​Br\)−L2​‖X‖F2​‖𝒑final‖22\>log⁡K\.\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\+\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}\\tau\_\{f\}\-\\epsilon\_\{r\}B\_\{r\}\)\-\\frac\{L\}\{2\}\\\|X\\\|\_\{F\}^\{2\}\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}\>\\log K\.\(12\)
2. \(ii\)Retention:For anyX∼PRrX\\sim P\_\{R\_\{r\}\}satisfying the gate eventℰ⁡\(X\)\\mathcal\{E\}\(X\)and the alignment condition⟨g⁡\(X\),𝜹retain⟩≥τr\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\rangle\\geq\\tau\_\{r\}with0<τr<αr0<\\tau\_\{r\}<\\alpha\_\{r\}, the prediction is preserved \(arg⁡max⁡Mθ​\(Perturb​\(X\)\)=Sf\\arg\\max M\_\{\\theta\}\(\\text\{Perturb\}\(X\)\)=S\_\{f\}\) whenever ℒ⁡\(Mθ​\(X\),Sf\)−‖X‖F​\(ϵr​τr−ϵf​Bf\)\+L2​‖X‖F2​‖𝒑final‖22<log⁡2\.\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\-\\\|X\\\|\_\{F\}\(\\epsilon\_\{r\}\\tau\_\{r\}\-\\epsilon\_\{f\}B\_\{f\}\)\+\\frac\{L\}\{2\}\\\|X\\\|\_\{F\}^\{2\}\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}<\\log 2\.\(13\)

Moreover, the probability that a random test sample satisfies the required alignment condition is lower bounded by:

Pr⁡\(⟨g⁡\(X\),𝜹forget⟩≥τf∣X∼PCf,ℰ⁡\(X\)\)\\displaystyle\\Pr\(\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\rangle\\geq\\tau\_\{f\}\\mid X\\sim P\_\{C\_\{f\}\},\\mathcal\{E\}\(X\)\)≥αf−τfBf−τf,\\displaystyle\\geq\\frac\{\\alpha\_\{f\}\-\\tau\_\{f\}\}\{B\_\{f\}\-\\tau\_\{f\}\},\(14\)Pr⁡\(⟨g⁡\(X\),𝜹retain⟩≥τr∣X∼PRr,ℰ⁡\(X\)\)\\displaystyle\\Pr\(\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\rangle\\geq\\tau\_\{r\}\\mid X\\sim P\_\{R\_\{r\}\},\\mathcal\{E\}\(X\)\)≥αr−τrBr−τr\.\\displaystyle\\geq\\frac\{\\alpha\_\{r\}\-\\tau\_\{r\}\}\{B\_\{r\}\-\\tau\_\{r\}\}\.\(15\)

###### Proof\.

The proof proceeds in three stages: establishing the concentration of the pre\-computed gradients to define strictly positive alignment boundsαf,αr\\alpha\_\{f\},\\alpha\_\{r\}\(Phase 1\), deriving sufficient cross\-entropy conditions for prediction change or preservation under perturbation \(Phase 2\), and lower bounding the probability of these conditions holding at test time\.

#### Step 1:

We first establish the existence of strictly positive lower boundsαf\\alpha\_\{f\}andαr\\alpha\_\{r\}for the conditional expected alignment\. We focus on the forget set; the derivation for the retain set is identical\.

LetX∼PCfX\\sim P\_\{C\_\{f\}\}and define the random variableZf:=⟨g⁡\(X\),𝜹forget⟩Z\_\{f\}:=\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\rangle\. By Assumption[C\.2](https://arxiv.org/html/2609.05966#A3.Thmtheorem2),Zf∈\[−Bf,Bf\]Z\_\{f\}\\in\[\-B\_\{f\},B\_\{f\}\]almost surely\. The Phase\-1 dataset𝒟Cf\\mathcal\{D\}\_\{C\_\{f\}\}consists ofnni\.i\.d\. samples\. Letα^f,n\\hat\{\\alpha\}\_\{f,n\}be the empirical mean ofZfZ\_\{f\}on𝒟Cf\\mathcal\{D\}\_\{C\_\{f\}\}\. By Hoeffding’s inequality, with probability at least1−δf1\-\\delta\_\{f\}:

𝔼⁡\[Zf\]≥α^f,n−Bf​log⁡\(2/δf\)2​n=α^f,n−εf​\(n\)\.\\mathbb\{E\}\[Z\_\{f\}\]\\geq\\hat\{\\alpha\}\_\{f,n\}\-B\_\{f\}\\sqrt\{\\frac\{\\log\(2/\\delta\_\{f\}\)\}\{2n\}\}=\\hat\{\\alpha\}\_\{f,n\}\-\\varepsilon\_\{f\}\(n\)\.\(16\)The term𝔼⁡\[Zf\]\\mathbb\{E\}\[Z\_\{f\}\]is the total expectation overPCfP\_\{C\_\{f\}\}\. We require the conditional expectation given the gate eventℰ⁡\(X\)\\mathcal\{E\}\(X\)\. Using the Law of Total Expectation:

𝔼⁡\[Zf\]=𝔼⁡\[Zf∣ℰ⁡\(X\)\]​pℰ,f\+𝔼⁡\[Zf∣ℰ​\(X\)c\]​\(1−pℰ,f\)\.\\mathbb\{E\}\[Z\_\{f\}\]=\\mathbb\{E\}\[Z\_\{f\}\\mid\\mathcal\{E\}\(X\)\]p\_\{\\mathcal\{E\},f\}\+\\mathbb\{E\}\[Z\_\{f\}\\mid\\mathcal\{E\}\(X\)^\{c\}\]\(1\-p\_\{\\mathcal\{E\},f\}\)\.\(17\)We bound the complement term using the worst\-case magnitude:𝔼⁡\[Zf∣ℰ​\(X\)c\]≤Bf\\mathbb\{E\}\[Z\_\{f\}\\mid\\mathcal\{E\}\(X\)^\{c\}\]\\leq B\_\{f\}\. Substituting this into the total expectation equation:

𝔼⁡\[Zf\]≤𝔼⁡\[Zf∣ℰ⁡\(X\)\]​pℰ,f\+Bf​\(1−pℰ,f\)\.\\mathbb\{E\}\[Z\_\{f\}\]\\leq\\mathbb\{E\}\[Z\_\{f\}\\mid\\mathcal\{E\}\(X\)\]p\_\{\\mathcal\{E\},f\}\+B\_\{f\}\(1\-p\_\{\\mathcal\{E\},f\}\)\.\(18\)Rearranging to solve for the conditional expectation and substituting the Hoeffding lower bound for𝔼⁡\[Zf\]\\mathbb\{E\}\[Z\_\{f\}\]:

𝔼⁡\[Zf∣ℰ⁡\(X\)\]≥\(α^f,n−εf​\(n\)\)−Bf​\(1−pℰ,f\)pℰ,f\.\\mathbb\{E\}\[Z\_\{f\}\\mid\\mathcal\{E\}\(X\)\]\\geq\\frac\{\\left\(\\hat\{\\alpha\}\_\{f,n\}\-\\varepsilon\_\{f\}\(n\)\\right\)\-B\_\{f\}\(1\-p\_\{\\mathcal\{E\},f\}\)\}\{p\_\{\\mathcal\{E\},f\}\}\.\(19\)The RHS is exactly the definition ofαf\\alpha\_\{f\}\. By the conditionα^f,n\>Bf​\(1−pℰ,f\)\+εf​\(n\)\\hat\{\\alpha\}\_\{f,n\}\>B\_\{f\}\(1\-p\_\{\\mathcal\{E\},f\}\)\+\\varepsilon\_\{f\}\(n\), the numerator is strictly positive, ensuringαf\>0\\alpha\_\{f\}\>0\. Thus, the event𝒢f:=\{𝔼\[Zf∣ℰ\(X\)\]≥αf\}\\mathcal\{G\}\_\{f\}:=\\\{\\mathbb\{E\}\[Z\_\{f\}\\mid\\mathcal\{E\}\(X\)\]\\geq\\alpha\_\{f\}\\\}holds with probability at least1−δf1\-\\delta\_\{f\}\. An identical derivation applies to the retain set, yieldingαr\>0\\alpha\_\{r\}\>0with probability1−δr1\-\\delta\_\{r\}\.

Since the Phase\-1 datasets𝒟Cf\\mathcal\{D\}\_\{C\_\{f\}\}and𝒟Rr\\mathcal\{D\}\_\{R\_\{r\}\}are drawn independently, we apply a union bound to ensure both bounds hold simultaneously:

Pr⁡\(𝒢f∩𝒢r\)=1−Pr⁡\(𝒢fc∪𝒢rc\)\\displaystyle\\Pr\(\\mathcal\{G\}\_\{f\}\\cap\\mathcal\{G\}\_\{r\}\)=1\-\\Pr\(\\mathcal\{G\}\_\{f\}^\{c\}\\cup\\mathcal\{G\}\_\{r\}^\{c\}\)≥1−\(Pr⁡\(𝒢fc\)\+Pr⁡\(𝒢rc\)\)≥1−\(δf\+δr\)=1−δ\.\\displaystyle\\geq 1\-\(\\Pr\(\\mathcal\{G\}\_\{f\}^\{c\}\)\+\\Pr\(\\mathcal\{G\}\_\{r\}^\{c\}\)\)\\geq 1\-\(\\delta\_\{f\}\+\\delta\_\{r\}\)=1\-\\delta\.

#### Step 2:

Recall the perturbationΔ=‖X‖F​\(ϵf​𝜹forget−ϵr​𝜹retain\)\\Delta=\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\-\\epsilon\_\{r\}\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\), and let𝒑final=ϵf​𝜹forget−ϵr​𝜹retain\\bm\{p\}\_\{\\mathrm\{final\}\}=\\epsilon\_\{f\}\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\-\\epsilon\_\{r\}\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\.

We first present a simple cross\-entropy\-to\-argmax argument\. For logitsz=fθ​\(x\)z=f\_\{\\theta\}\(x\)and target superclassSfS\_\{f\}, the softmax probability ofSfS\_\{f\}is

pSf=exp⁡\(zSf\)∑S∈𝒮exp⁡\(zS\)\.\\displaystyle p\_\{S\_\{f\}\}=\\frac\{\\exp\(z\_\{S\_\{f\}\}\)\}\{\\sum\_\{S\\in\\mathcal\{S\}\}\\exp\(z\_\{S\}\)\}\.Therefore, the cross\-entropy loss with targetSfS\_\{f\}is

ℓ⁡\(z,Sf\)\\displaystyle\\ell\(z,S\_\{f\}\)=log⁡\[∑S∈𝒮exp⁡\(zS\)\]−zSf\.\\displaystyle=\\log\\left\[\\sum\_\{S\\in\\mathcal\{S\}\}\\exp\(z\_\{S\}\)\\right\]\-z\_\{S\_\{f\}\}\.\(20\)Factoring outexp⁡\(zSf\)\\exp\(z\_\{S\_\{f\}\}\)from the log\-sum\-exp term gives

∑S∈𝒮exp⁡\(zS\)=exp⁡\(zSf\)​\(1\+∑S≠Sfexp⁡\(zS−zSf\)\)\.\\sum\_\{S\\in\\mathcal\{S\}\}\\exp\(z\_\{S\}\)=\\exp\(z\_\{S\_\{f\}\}\)\\left\(1\+\\sum\_\{S\\neq S\_\{f\}\}\\exp\(z\_\{S\}\-z\_\{S\_\{f\}\}\)\\right\)\.Substituting this identity into the previous display yields

ℓ⁡\(z,Sf\)\\displaystyle\{\\ell\(z,S\_\{f\}\)\}=log⁡\[exp⁡\(zSf\)​\(1\+∑S≠Sfexp⁡\(zS−zSf\)\)\]−zSf=log⁡\(1\+∑S≠Sfexp⁡\(zS−zSf\)\)\.\\displaystyle=\{\\log\\left\[\\exp\(z\_\{S\_\{f\}\}\)\\left\(1\+\\sum\_\{S\\neq S\_\{f\}\}\\exp\(z\_\{S\}\-z\_\{S\_\{f\}\}\)\\right\)\\right\]\-z\_\{S\_\{f\}\}\}=\{\\log\\left\(1\+\\sum\_\{S\\neq S\_\{f\}\}\\exp\(z\_\{S\}\-z\_\{S\_\{f\}\}\)\\right\)\.\}\(21\)IfSfS\_\{f\}remains the argmax amongK=\|𝒮\|K=\|\\mathcal\{S\}\|superclasses, then its softmax probability is at least1/K1/K, and henceℓ⁡\(z,Sf\)≤log⁡K\\ell\(z,S\_\{f\}\)\\leq\\log K\. Therefore,ℓ⁡\(z,Sf\)\>log⁡K\\ell\(z,S\_\{f\}\)\>\\log Kis a sufficient condition forSfS\_\{f\}not to remain the argmax\. Similarly, ifℓ⁡\(z,Sf\)<log⁡2\\ell\(z,S\_\{f\}\)<\\log 2, then the softmax probability ofSfS\_\{f\}is larger than1/21/2, which implies thatSfS\_\{f\}is the unique argmax\.

\(i\) Forgetting Condition:ConsiderX∼PCfX\\sim P\_\{C\_\{f\}\}satisfyingℰ⁡\(X\)\\mathcal\{E\}\(X\)\. By Assumption[C\.1](https://arxiv.org/html/2609.05966#A3.Thmtheorem1)\(Directional Smoothness\):

ℒ⁡\(Mθ​\(X\+Δ\),Sf\)−ℒ⁡\(Mθ​\(X\),Sf\)≥⟨g⁡\(X\),Δ⟩−L2​‖Δ‖22\.\\mathcal\{L\}\(M\_\{\\theta\}\(X\+\\Delta\),S\_\{f\}\)\-\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\\geq\\langle g\(X\),\\Delta\\rangle\-\\frac\{L\}\{2\}\\\|\\Delta\\\|\_\{2\}^\{2\}\.\(22\)Conditioned on the alignment⟨g⁡\(X\),𝜹forget⟩≥τf\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\rangle\\geq\\tau\_\{f\}and the bounded projection\|⟨g⁡\(X\),𝜹retain⟩\|≤Br\|\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\rangle\|\\leq B\_\{r\}:

⟨g⁡\(X\),Δ⟩=‖X‖F​\(ϵf​⟨g⁡\(X\),𝜹forget⟩−ϵr​⟨g⁡\(X\),𝜹retain⟩\)≥‖X‖F​\(ϵf​τf−ϵr​Br\)\.\\langle g\(X\),\\Delta\\rangle=\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\rangle\-\\epsilon\_\{r\}\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\rangle\)\\geq\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}\\tau\_\{f\}\-\\epsilon\_\{r\}B\_\{r\}\)\.\(23\)Substituting into the smoothness bound:

ℒ⁡\(Mθ​\(X\+Δ\),Sf\)−ℒ⁡\(Mθ​\(X\),Sf\)≥‖X‖F​\(ϵf​τf−ϵr​Br\)−L2​‖X‖F2​‖𝒑final‖22\.\\mathcal\{L\}\(M\_\{\\theta\}\(X\+\\Delta\),S\_\{f\}\)\-\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\\geq\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}\\tau\_\{f\}\-\\epsilon\_\{r\}B\_\{r\}\)\-\\frac\{L\}\{2\}\\\|X\\\|\_\{F\}^\{2\}\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}\.\(24\)Thus,

ℒ⁡\(Mθ​\(X\+Δ\),Sf\)≥ℒ⁡\(Mθ​\(X\),Sf\)\+‖X‖F​\(ϵf​τf−ϵr​Br\)−L2​‖X‖F2​‖𝒑final‖22\.\\mathcal\{L\}\(M\_\{\\theta\}\(X\+\\Delta\),S\_\{f\}\)\\geq\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\+\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}\\tau\_\{f\}\-\\epsilon\_\{r\}B\_\{r\}\)\-\\frac\{L\}\{2\}\\\|X\\\|\_\{F\}^\{2\}\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}\.\(25\)If the right\-hand side exceedslog⁡K\\log K, thenℒ⁡\(Mθ​\(X\+Δ\),Sf\)\>log⁡K\\mathcal\{L\}\(M\_\{\\theta\}\(X\+\\Delta\),S\_\{f\}\)\>\\log K\. By the argument earlier,SfS\_\{f\}cannot remain the argmax, ensuring

arg⁡max⁡Mθ​\(Perturb​\(X\)\)≠Sf\.\\arg\\max M\_\{\\theta\}\(\\text\{Perturb\}\(X\)\)\\neq S\_\{f\}\.
\(ii\) Retention Condition:ConsiderX∼PRrX\\sim P\_\{R\_\{r\}\}satisfyingℰ⁡\(X\)\\mathcal\{E\}\(X\)\. Using the smoothness upper bound:

ℒ⁡\(Mθ​\(X\+Δ\),Sf\)−ℒ⁡\(Mθ​\(X\),Sf\)≤⟨g⁡\(X\),Δ⟩\+L2​‖Δ‖22\.\\mathcal\{L\}\(M\_\{\\theta\}\(X\+\\Delta\),S\_\{f\}\)\-\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\\leq\\langle g\(X\),\\Delta\\rangle\+\\frac\{L\}\{2\}\\\|\\Delta\\\|\_\{2\}^\{2\}\.\(26\)Conditioned on⟨g⁡\(X\),𝜹retain⟩≥τr\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\rangle\\geq\\tau\_\{r\}and bounding\|⟨g⁡\(X\),𝜹forget⟩\|≤Bf\|\\langle g\(X\),\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\\rangle\|\\leq B\_\{f\}:

⟨g⁡\(X\),Δ⟩≤‖X‖F​\(ϵf​Bf−ϵr​τr\)\.\\langle g\(X\),\\Delta\\rangle\\leq\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}B\_\{f\}\-\\epsilon\_\{r\}\\tau\_\{r\}\)\.\(27\)The perturbed loss is therefore upper bounded by:

ℒ⁡\(Mθ​\(X\+Δ\),Sf\)≤ℒ⁡\(Mθ​\(X\),Sf\)−‖X‖F​\(ϵr​τr−ϵf​Bf\)\+L2​‖X‖F2​‖𝒑final‖22\.\\mathcal\{L\}\(M\_\{\\theta\}\(X\+\\Delta\),S\_\{f\}\)\\leq\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\-\\\|X\\\|\_\{F\}\(\\epsilon\_\{r\}\\tau\_\{r\}\-\\epsilon\_\{f\}B\_\{f\}\)\+\\frac\{L\}\{2\}\\\|X\\\|\_\{F\}^\{2\}\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}\.\(28\)If this upper bound is strictly smaller thanlog⁡2\\log 2, thenℒ⁡\(Mθ​\(X\+Δ\),Sf\)<log⁡2\\mathcal\{L\}\(M\_\{\\theta\}\(X\+\\Delta\),S\_\{f\}\)<\\log 2\. By the argument earlier, the softmax probability ofSfS\_\{f\}is larger than1/21/2, soSfS\_\{f\}is the unique argmax, ensuring

arg⁡max⁡Mθ​\(Perturb​\(X\)\)=Sf\.\\arg\\max M\_\{\\theta\}\(\\text\{Perturb\}\(X\)\)=S\_\{f\}\.
Note: We use the thresholdslog⁡K\\log Kandlog⁡2\\log 2because they give simple, distribution\-free sufficient conditions\.ℓ⁡\(z,Sf\)\>log⁡K\\ell\(z,S\_\{f\}\)\>\\log Kguarantees thatSfS\_\{f\}cannot remain the argmax, whileℓ⁡\(z,Sf\)<log⁡2\\ell\(z,S\_\{f\}\)<\\log 2guarantees thatSfS\_\{f\}is the unique argmax\. These thresholds are conservative but easier to verify than tighter margin\-dependent conditions\.

#### Step 3:

LetYYbe the random variableZf\|ℰ⁡\(X\)Z\_\{f\}\\mid\\mathcal\{E\}\(X\)\. From Step 1,𝔼⁡\[Y\]≥αf\\mathbb\{E\}\[Y\]\\geq\\alpha\_\{f\}\. By Assumption[C\.2](https://arxiv.org/html/2609.05966#A3.Thmtheorem2),Y≤BfY\\leq B\_\{f\}\. DefineW=Bf−Y≥0W=B\_\{f\}\-Y\\geq 0\. Then𝔼⁡\[W\]≤Bf−αf\\mathbb\{E\}\[W\]\\leq B\_\{f\}\-\\alpha\_\{f\}\. By Markov’s inequality:

Pr⁡\(Y≥τf\)=Pr⁡\(Bf−W≥τf\)=1−Pr⁡\(W\>Bf−τf\)≥1−Bf−αfBf−τf=αf−τfBf−τf\.\\Pr\(Y\\geq\\tau\_\{f\}\)=\\Pr\(B\_\{f\}\-W\\geq\\tau\_\{f\}\)=1\-\\Pr\(W\>B\_\{f\}\-\\tau\_\{f\}\)\\geq 1\-\\frac\{B\_\{f\}\-\\alpha\_\{f\}\}\{B\_\{f\}\-\\tau\_\{f\}\}=\\frac\{\\alpha\_\{f\}\-\\tau\_\{f\}\}\{B\_\{f\}\-\\tau\_\{f\}\}\.\(29\)This bound is valid and non\-vacuous sinceαf\>0\\alpha\_\{f\}\>0\(from Step 1\) and we selectτf<αf\\tau\_\{f\}<\\alpha\_\{f\}\. An identical bound applies to the retain set\. ∎

### C\.4Feasibility region of\(ϵf,ϵr\)\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)

To explicitly characterize the range of\(ϵf,ϵr\)\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)under which both forgetting and retention are guaranteed, we analyze the geometry of the perturbation vector\. This analysis formalizes the implicit trade\-off between first\-order directional alignment and second\-order curvature penalties\.

Starting from the two conditions[12](https://arxiv.org/html/2609.05966#A3.E12)\-[13](https://arxiv.org/html/2609.05966#A3.E13)in Theorem[C\.4](https://arxiv.org/html/2609.05966#A3.Thmtheorem4), forgetting requires

ℒ⁡\(Mθ​\(X\),Sf\)\+‖X‖F​\(ϵf​τf−ϵr​Br\)−L2​‖X‖F2​‖𝒑final‖22\>log⁡K,\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\+\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}\\tau\_\{f\}\-\\epsilon\_\{r\}B\_\{r\}\)\-\\frac\{L\}\{2\}\\\|X\\\|\_\{F\}^\{2\}\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}\>\\log K,while retention requires

ℒ⁡\(Mθ​\(X\),Sf\)−‖X‖F​\(ϵr​τr−ϵf​Bf\)\+L2​‖X‖F2​‖𝒑final‖22<log⁡2,\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\-\\\|X\\\|\_\{F\}\(\\epsilon\_\{r\}\\tau\_\{r\}\-\\epsilon\_\{f\}B\_\{f\}\)\+\\frac\{L\}\{2\}\\\|X\\\|\_\{F\}^\{2\}\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}<\\log 2,where𝒑final=ϵf​𝜹forget−ϵr​𝜹retain\\bm\{p\}\_\{\\mathrm\{final\}\}=\\epsilon\_\{f\}\\bm\{\\delta\}\_\{\\mathrm\{forget\}\}\-\\epsilon\_\{r\}\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}andρ:=⟨𝜹forget,𝜹retain⟩\\rho:=\\langle\\bm\{\\delta\}\_\{\\mathrm\{forget\}\},\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}\\rangle\. Since𝜹forget,𝜹retain\\bm\{\\delta\}\_\{\\mathrm\{forget\}\},\\bm\{\\delta\}\_\{\\mathrm\{retain\}\}are unit vectors, expanding the squared Euclidean norm yields

‖𝒑final‖22=ϵf2\+ϵr2−2​ρ​ϵf​ϵr\.\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}=\\epsilon\_\{f\}^\{2\}\+\\epsilon\_\{r\}^\{2\}\-2\\rho\\,\\epsilon\_\{f\}\\epsilon\_\{r\}\.
Dividing both inequalities by‖X‖F\>0\\\|X\\\|\_\{F\}\>0and rearranging yields

ϵf​τf−ϵr​Br−log⁡K−ℒ⁡\(Mθ​\(X\),Sf\)‖X‖F\>L2​‖X‖F​\(ϵf2\+ϵr2−2​ρ​ϵf​ϵr\),\\epsilon\_\{f\}\\tau\_\{f\}\-\\epsilon\_\{r\}B\_\{r\}\-\\frac\{\\log K\-\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\}\{\\\|X\\\|\_\{F\}\}\>\\frac\{L\}\{2\}\\\|X\\\|\_\{F\}\\big\(\\epsilon\_\{f\}^\{2\}\+\\epsilon\_\{r\}^\{2\}\-2\\rho\\,\\epsilon\_\{f\}\\epsilon\_\{r\}\\big\),ϵr​τr−ϵf​Bf−ℒ⁡\(Mθ​\(X\),Sf\)−log⁡2‖X‖F\>L2​‖X‖F​\(ϵf2\+ϵr2−2​ρ​ϵf​ϵr\)\.\\epsilon\_\{r\}\\tau\_\{r\}\-\\epsilon\_\{f\}B\_\{f\}\-\\frac\{\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\-\\log 2\}\{\\\|X\\\|\_\{F\}\}\>\\frac\{L\}\{2\}\\\|X\\\|\_\{F\}\\big\(\\epsilon\_\{f\}^\{2\}\+\\epsilon\_\{r\}^\{2\}\-2\\rho\\,\\epsilon\_\{f\}\\epsilon\_\{r\}\\big\)\.
We formally define the feasible sets for each objective as:

Ff:=\{\(ϵf,ϵr\):ϵf​τf−ϵr​Br−log⁡K−ℒ⁡\(Mθ​\(X\),Sf\)‖X‖F\>L2​‖X‖F​\(ϵf2\+ϵr2−2​ρ​ϵf​ϵr\)\},F\_\{f\}:=\\Big\\\{\(\\epsilon\_\{f\},\\epsilon\_\{r\}\):\\epsilon\_\{f\}\\tau\_\{f\}\-\\epsilon\_\{r\}B\_\{r\}\-\\tfrac\{\\log K\-\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\}\{\\\|X\\\|\_\{F\}\}\>\\tfrac\{L\}\{2\}\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}^\{2\}\+\\epsilon\_\{r\}^\{2\}\-2\\rho\\epsilon\_\{f\}\\epsilon\_\{r\}\)\\Big\\\},Fr:=\{\(ϵf,ϵr\):ϵr​τr−ϵf​Bf−ℒ⁡\(Mθ​\(X\),Sf\)−log⁡2‖X‖F\>L2​‖X‖F​\(ϵf2\+ϵr2−2​ρ​ϵf​ϵr\)\}\.F\_\{r\}:=\\Big\\\{\(\\epsilon\_\{f\},\\epsilon\_\{r\}\):\\epsilon\_\{r\}\\tau\_\{r\}\-\\epsilon\_\{f\}B\_\{f\}\-\\tfrac\{\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\-\\log 2\}\{\\\|X\\\|\_\{F\}\}\>\\tfrac\{L\}\{2\}\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}^\{2\}\+\\epsilon\_\{r\}^\{2\}\-2\\rho\\epsilon\_\{f\}\\epsilon\_\{r\}\)\\Big\\\}\.The feasible region for simultaneously achieving forgetting and retention is the intersection

F=Ff∩Fr\.F=F\_\{f\}\\cap F\_\{r\}\.Each set corresponds to a quadratic inequality in\(ϵf,ϵr\)\(\\epsilon\_\{f\},\\epsilon\_\{r\}\)\. The left\-hand side of each inequality is affine and represents the desired first\-order effect of the perturbation, while the right\-hand side is a quadratic curvature penalty that grows with the perturbation magnitude\. This structure creates a bounded feasible region: the intersectionFFcontains coefficient pairs large enough to drive the intended behavior,i\.e\., pushing the post\-perturbation forget loss abovelog⁡K\\log Kwhile keeping the post\-perturbation retain loss belowlog⁡2\\log 2, yet small enough to avoid excessive second\-order distortion\.

The correlation termρ\\rhocouples the two directions: whenρ\>0\\rho\>0\(aligned directions\), the quadratic penalty increases, shrinkingFFand making simultaneous satisfaction harder\. Whenρ<0\\rho<0\(opposing directions\), the penalty weakens, enlargingFF\. In the special caseρ=0\\rho=0\(orthogonal directions\), the norm simplifies to‖𝒑final‖22=ϵf2\+ϵr2\\\|\\bm\{p\}\_\{\\mathrm\{final\}\}\\\|\_\{2\}^\{2\}=\\epsilon\_\{f\}^\{2\}\+\\epsilon\_\{r\}^\{2\}\. The region simplifies to:

Ff:ϵf​τf−ϵr​Br−log⁡K−ℒ⁡\(Mθ​\(X\),Sf\)‖X‖F\>L2​‖X‖F​\(ϵf2\+ϵr2\),F\_\{f\}:\\;\\epsilon\_\{f\}\\tau\_\{f\}\-\\epsilon\_\{r\}B\_\{r\}\-\\tfrac\{\\log K\-\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\}\{\\\|X\\\|\_\{F\}\}\>\\tfrac\{L\}\{2\}\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}^\{2\}\+\\epsilon\_\{r\}^\{2\}\),Fr:ϵr​τr−ϵf​Bf−ℒ⁡\(Mθ​\(X\),Sf\)−log⁡2‖X‖F\>L2​‖X‖F​\(ϵf2\+ϵr2\)\.F\_\{r\}:\\;\\epsilon\_\{r\}\\tau\_\{r\}\-\\epsilon\_\{f\}B\_\{f\}\-\\tfrac\{\\mathcal\{L\}\(M\_\{\\theta\}\(X\),S\_\{f\}\)\-\\log 2\}\{\\\|X\\\|\_\{F\}\}\>\\tfrac\{L\}\{2\}\\\|X\\\|\_\{F\}\(\\epsilon\_\{f\}^\{2\}\+\\epsilon\_\{r\}^\{2\}\)\.Here the geometric coupling disappears, and feasibility depends only on the total perturbation energyϵf2\+ϵr2\\epsilon\_\{f\}^\{2\}\+\\epsilon\_\{r\}^\{2\}\. This makes it easier to satisfy both guarantees when the forget and retain directions are nearly orthogonal, as optimizing one coefficient does not geometrically inflate the curvature cost of the other through a cross\-term\.

## Appendix DPrivacy and deployment risks

ERASE provides functional privacy in the deployment setting where the attacker observes the released inference behavior, e\.g\., the output of the prediction API after conditional perturbation\. It should not be interpreted as structural removal of information from the model parameters\. If an attacker obtains white\-box access to the original model weights, then functional forgetting alone does not remove the information stored in those weights and does not preserve privacy against such an attacker\. Similarly, the stored subclass directions and optimized perturbation coefficients are deployment artifacts that should be treated as sensitive and protected by access control\.

Our MIA evaluation therefore measures black\-box output\-level membership leakage through softmax outputs after conditional perturbation\. It does not prove complete removal of training influence\. Stronger white\-box, adaptive, influence\-based, or reconstruction\-style privacy audits test stronger threat models and remain important future work\.

ERASE may also introduce a timing side channel\. Since the conditional version may require a second forward pass when the gate predicts the target superclassSfS\_\{f\}, an attacker observing latency could infer whether the gate was triggered\. A simple mitigation is constant\-time inference, e\.g\., always performing two forward passes or adding latency padding, so that gated and non\-gated inputs are indistinguishable from timing\.