Causal Unlearning in Collaborative Optimization: Exact and Approximate Influence Reversal under Adversarial Contributions

# Causal Unlearning in Collaborative Optimization: Exact and Approximate Influence Reversal under Adversarial Contributions
Source: [https://arxiv.org/html/2605.20341](https://arxiv.org/html/2605.20341)
Ali Mahdavi Department of Computer Engineering SRC, Islamic Azad University Tehran, Iran ali\.mahdavi@iau\.ir &Azadeh Zamanifar Department of Computer Engineering SRC, Islamic Azad University Tehran, Iran azamanifar@iau\.ac\.ir Amir Farhad Farhadi School of Computer Engineering Iran University of Science and Technology Tehran, Iran amfarhadi@mail\.iust\.ac\.ir &Omid Kashefi Meta CA, USA kashefi@meta\.com

###### Abstract

Federated learning systems must support data deletion requests to comply with privacy regulations, yet retraining from scratch after each deletion is computationally prohibitive\. We present HF\-KCU, a method that removes a client’s contribution by approximating the influence function through conjugate gradient iterations in Krylov subspaces, reducing complexity fromO​\(d3\)O\(d^\{3\}\)toO​\(k​d\)O\(kd\)wherek≪dk\\ll d\. A causal weighting mechanism ensures that only clients holding the deleted data receive parameter updates, preventing spurious changes to unaffected clients\. Our method is designed to handle bounded adversarial perturbations to the Hessian and gradient, providing graceful degradation under realistic threat models\.

We validate HF\-KCU across convolutional \(ResNet\-18, SimpleCNN\) and transformer \(ViT\-Lite\) architectures on CIFAR\-10, MNIST, and Fashion\-MNIST\. On CIFAR\-10 under Dirichlet\(α=0\.5\\alpha=0\.5\) partitioning, HF\-KCU achieves 47\.75×\\timesspeedup over retraining while maintaining test accuracy within 0\.60% of the retrained baseline \(71\.16% vs 71\.76%\)\. Membership inference attacks on the forget set yield success rates of 0\.499, matching the retrained model and confirming effective privacy restoration\. We provide convergence guarantees showing that the Krylov approximation error decreases asO​\(\(κ−1\)/\(κ\+1\)\)kO\(\(\\sqrt\{\\kappa\}\-1\)/\(\\sqrt\{\\kappa\}\+1\)\)^\{k\}whereκ\\kappais the Hessian condition number\. The causal weighting mechanism ensures surgical updates, where only clients holding deleted data are modified, preserving model quality for unaffected participants and avoiding the instability of gradient\-based approaches in asynchronous federated settings\. This design provides interpretability as each update is directly traceable to the influence of the deleted data\. The method’s efficiency and precision make it suitable for production federated systems where deletion requests arrive asynchronously and computational budgets are constrained\.

## 1Introduction

Privacy regulations including GDPREuropean Parliament and Council of the European Union \([2016](https://arxiv.org/html/2605.20341#bib.bib26)\)and CCPACalifornia State Legislature \([2018](https://arxiv.org/html/2605.20341#bib.bib27)\)grant individuals the right to request deletion of their personal data\. For machine learning systems, this requirement translates into*machine unlearning*Cao and Yang \([2015](https://arxiv.org/html/2605.20341#bib.bib5)\), an approach to influence removal of specific training examples from a trained model\. The naive solution is also retraining from scratch after each deletion which becomes impractical as model and dataset sizes grow\.

Federated learningMcMahanet al\.\([2017](https://arxiv.org/html/2605.20341#bib.bib1)\)compounds this problem\. Training occurs across distributed clients without centralizing raw data, but when a client requests data deletion, the system must remove that client’s contribution while preserving knowledge from all others\. Full retraining requires re\-coordinating all remaining clients, multiplying communication and computation costs\.

Existing federated unlearning approaches face a dilemma\. Methods based on gradient ascentGolatkaret al\.\([2020](https://arxiv.org/html/2605.20341#bib.bib28)\)or noise injection are fast but often degrade model accuracy severely\. Sharding schemes like SISABourtouleet al\.\([2021](https://arxiv.org/html/2605.20341#bib.bib6)\)reduce retraining scope but require architectural changes and still incur considerable overhead\. Influence\-based methodsKoh and Liang \([2017](https://arxiv.org/html/2605.20341#bib.bib12)\)offer principled approximations but require computing or inverting the Hessian matrix, anO​\(d3\)O\(d^\{3\}\)operation intractable for modern neural networks with millions of parameters\.Recent surveys on federated unlearning have highlighted the unique challenges of removing data influence in collaborative settings, where distributed data ownership and privacy constraints complicate traditional unlearning approachesLiuet al\.\([2024](https://arxiv.org/html/2605.20341#bib.bib4)\); Tae and Chan \([2025](https://arxiv.org/html/2605.20341#bib.bib19)\)

We introduce HF\-KCU \(Hessian\-Free Krylov Causal Unlearning\), which approximates influence\-based unlearning through conjugate gradient \(CG\) iterations\. CG solves the linear systemH​v=gHv=gwithout formingHHexplicitly, requiring only Hessian\-vector products computable via automatic differentiation inO​\(d\)O\(d\)time\. Afterkkiterations, CG produces an approximationvk≈H−1​gv\_\{k\}\\approx H^\{\-1\}gwith error bounded by the Hessian’s condition number\.

A key contribution is our causal weighting mechanism\. In federated settings, naively applying a global influence update can introduce spurious parameter changes to clients that never held the deleted data\. Our weighting scheme assigns zero update to unaffected clients, ensuring causal isolation\. We prove this property formally and validate it empirically\. Furthermore, we validate the practical utility of our method in a critical security scenario, demonstrating its ability to rapidly unlearn a backdoor attack with a 430x speedup over retraining while maintaining model performance

Threat Model\.To avoid overstating our robustness claims, we consider bounded adversarial perturbations rather than unconstrained adaptive poisoning\. Specifically, we assume an adversary may perturb the Hessian and gradient with bounded norm \(‖E‖2≤ϵ\\\|E\\\|\_\{2\}\\leq\\epsilon,‖e‖2≤ρ\\\|e\\\|\_\{2\}\\leq\\rho\)\. This bounded setting is realistic for the unlearning context because: \(1\) unbounded adversarial data could shift the local optimum so far that the local quadratic assumption underlying influence functions breaks down, invalidating the entire influence\-based framework; \(2\) in federated unlearning, the adversary’s goal is typically to prevent effective data removal rather than arbitrary model poisoning—bounded perturbations model realistic noise or minor data corruption during the unlearning process; \(3\) stronger threat models \(e\.g\., Byzantine clients with arbitrary model replacement\) require fundamentally different defenses \(robust aggregation, secure multi\-party computation\) orthogonal to the unlearning problem\. Our goal is not to establish full minimax robustness against arbitrary poisoning attacks, but to show that the proposed influence\-reversal mechanism degrades gracefully under bounded adversarial perturbations that may arise during the unlearning operation itself\.

Our contributions are:

- •A Hessian\-free Krylov approximation for federated unlearning withO​\(k​d\)O\(kd\)complexity per client, wherek≪dk\\ll d\(Section[3](https://arxiv.org/html/2605.20341#S3)\)\.
- •A causal weighting mechanism that provably isolates unaffected clients \(Proposition[7](https://arxiv.org/html/2605.20341#Thmtheorem7)\)\.
- •Convergence analysis showing approximation error decreases exponentially inkkfor well\-conditioned problems \(Theorem[4](https://arxiv.org/html/2605.20341#Thmtheorem4)\)\.
- •Experimental validation on CIFAR\-10 demonstrating 47\.75×\\timesspeedup with 0\.60% accuracy loss and MIA success rates matching retraining \(Section[5](https://arxiv.org/html/2605.20341#S5)\)\.
- •Critical analysis of faithfulness metrics, showing that parameter\-space measures can be unreliable when accuracy differences are small \(Section[A\.13](https://arxiv.org/html/2605.20341#A1.SS13)\)\.
- •We validate the practical utility of our method in a critical security scenario, demonstrating its ability to rapidly mitigate a backdoor attack with a ~430x speedup over retraining while preserving model performance\.

## 2Background and Related Work

### 2\.1Machine Unlearning

Cao et al\.Cao and Yang \([2015](https://arxiv.org/html/2605.20341#bib.bib5)\)formalized machine unlearning as the problem of efficiently removing training data influence without full retraining\. Early work focused on specific model classes: Ginart et al\.Ginartet al\.\([2019](https://arxiv.org/html/2605.20341#bib.bib8)\)developed unlearning for k\-means clustering, while Guo et al\.Guoet al\.\([2020](https://arxiv.org/html/2605.20341#bib.bib7)\)introduced certified removal guarantees for linear models\.

For neural networks, approaches fall into three categories\.Gradient\-based methodsapply gradient ascent on the forget setGolatkaret al\.\([2020](https://arxiv.org/html/2605.20341#bib.bib28)\)or fine\-tune on the retain setTarunet al\.\([2023](https://arxiv.org/html/2605.20341#bib.bib29)\), but often cause accuracy degradation\.Sharding methodslike SISABourtouleet al\.\([2021](https://arxiv.org/html/2605.20341#bib.bib6)\)partition data into shards, training separate models and retraining only affected shards upon deletion\. This reduces retraining scope but requires ensemble architectures and still incurs considerable cost\.Influence\-based methodsKoh and Liang \([2017](https://arxiv.org/html/2605.20341#bib.bib12)\)approximate the parameter change via influence functions, but computingH−1​gH^\{\-1\}gexactly requiresO​\(d3\)O\(d^\{3\}\)operations\.

### 2\.2Federated Learning

Federated learningMcMahanet al\.\([2017](https://arxiv.org/html/2605.20341#bib.bib1)\); Kairouzet al\.\([2021](https://arxiv.org/html/2605.20341#bib.bib2)\)enables collaborative training across distributed clients\. FedAvg aggregates local model updates through weighted averaging, preserving data locality\. However, data heterogeneity \(non\-IID distributions\) across clients complicates convergence analysis and introduces new challenges for unlearning\.

### 2\.3Federated Unlearning

Liu et al\.Liuet al\.\([2021b](https://arxiv.org/html/2605.20341#bib.bib23)\)proposed FedEraser, which stores historical updates and recalibrates the global model after removing a client’s contributions\. This requires considerable storage overhead and assumes updates remain available\. Wu et al\.Wuet al\.\([2022](https://arxiv.org/html/2605.20341#bib.bib10)\)explored differential privacy mechanisms for unlearning but with utility loss\. Our work differs by using influence approximation without requiring historical storage, while maintaining accuracy close to retraining\.

### 2\.4Influence Functions and Second\-Order Methods

Influence functionsCook and Weisberg \([1980](https://arxiv.org/html/2605.20341#bib.bib13)\); Koh and Liang \([2017](https://arxiv.org/html/2605.20341#bib.bib12)\)quantify how removing a training point affects model parameters\. For a lossℒ​\(θ\)\\mathcal\{L\}\(\\theta\)and pointzz, the influence is−H−1​∇θℓ​\(θ;z\)\-H^\{\-1\}\\nabla\_\{\\theta\}\\ell\(\\theta;z\)whereH=∇θ2ℒ​\(θ\)H=\\nabla^\{2\}\_\{\\theta\}\\mathcal\{L\}\(\\theta\)\. Computing this exactly is intractable, motivating approximations\.

Hessian\-free optimizationMartens \([2010](https://arxiv.org/html/2605.20341#bib.bib15)\)uses conjugate gradient to approximate second\-order updates without formingHH\. PearlmutterPearlmutter \([1994](https://arxiv.org/html/2605.20341#bib.bib20)\)showed that Hessian\-vector productsH​vHvcan be computed via automatic differentiation in time comparable to a gradient computation\. We build on these techniques for efficient unlearning\.

## 3Method

### 3\.1Problem Formulation

Consider a federated learning system withNNclients, each holding local dataset𝒟i\\mathcal\{D\}\_\{i\}\. The global modelθ∈ℝd\\theta\\in\\mathbb\{R\}^\{d\}minimizes the federated objective:

ℒ​\(θ\)=∑i=1Nwi​ℓi​\(θ;𝒟i\),\\mathcal\{L\}\(\\theta\)=\\sum\_\{i=1\}^\{N\}w\_\{i\}\\ell\_\{i\}\(\\theta;\\mathcal\{D\}\_\{i\}\),\(1\)whereℓi\\ell\_\{i\}is clientii’s local loss andwi=\|𝒟i\|/∑j\|𝒟j\|w\_\{i\}=\|\\mathcal\{D\}\_\{i\}\|/\\sum\_\{j\}\|\\mathcal\{D\}\_\{j\}\|are aggregation weights proportional to dataset sizes\.

When clientffrequests deletion of data𝒟u⊆𝒟f\\mathcal\{D\}\_\{u\}\\subseteq\\mathcal\{D\}\_\{f\}, the goal is to compute parametersθu\\theta\_\{u\}that approximate the retrained modelθr\\theta\_\{r\}obtained by solving:

θr=argminθ​∑i≠fwi​ℓi​\(θ;𝒟i\)\+wf​ℓf​\(θ;𝒟f∖𝒟u\)\.\\theta\_\{r\}=\\operatorname\{argmin\}\_\{\\theta\}\\sum\_\{i\\neq f\}w\_\{i\}\\ell\_\{i\}\(\\theta;\\mathcal\{D\}\_\{i\}\)\+w\_\{f\}\\ell\_\{f\}\(\\theta;\\mathcal\{D\}\_\{f\}\\setminus\\mathcal\{D\}\_\{u\}\)\.\(2\)
Retraining requiresO​\(T⋅N⋅d\)O\(T\\cdot N\\cdot d\)operations forTTcommunication rounds\. Our goal is to approximateθr\\theta\_\{r\}withO​\(k⋅d\)O\(k\\cdot d\)operations wherek≪T⋅Nk\\ll T\\cdot N\.

### 3\.2Influence\-Based Unlearning

Influence function theoryKoh and Liang \([2017](https://arxiv.org/html/2605.20341#bib.bib12)\)provides a first\-order approximation\. Removing data𝒟u\\mathcal\{D\}\_\{u\}induces parameter change:

Δ​θ=−H−1​∇θℓ​\(θ;𝒟u\),\\Delta\\theta=\-H^\{\-1\}\\nabla\_\{\\theta\}\\ell\(\\theta;\\mathcal\{D\}\_\{u\}\),\(3\)whereH=∇θ2ℒ​\(θ\)H=\\nabla^\{2\}\_\{\\theta\}\\mathcal\{L\}\(\\theta\)is the Hessian of the global loss\. This approximation is valid when𝒟u\\mathcal\{D\}\_\{u\}is small relative to the full dataset and the loss is approximately quadratic near the optimum\.

Computing Eq\. \([3](https://arxiv.org/html/2605.20341#S3.E3)\) exactly requires invertingH∈ℝd×dH\\in\\mathbb\{R\}^\{d\\times d\}, which costsO​\(d3\)O\(d^\{3\}\)time andO​\(d2\)O\(d^\{2\}\)memory—prohibitive for neural networks withd∼106d\\sim 10^\{6\}to10910^\{9\}parameters\.

### 3\.3Krylov Subspace Approximation

We approximatev=H−1​gv=H^\{\-1\}gby solvingH​v=gHv=giteratively using conjugate gradient \(CG\)\. CG constructs approximations in the Krylov subspace:

𝒦k​\(H,g\)=span​\{g,H​g,H2​g,…,Hk−1​g\}\.\\mathcal\{K\}\_\{k\}\(H,g\)=\\text\{span\}\\\{g,Hg,H^\{2\}g,\\ldots,H^\{k\-1\}g\\\}\.\(4\)
The key insight is that CG requires only matrix\-vector productsH​vHv, notHHitself\. For neural networks, we computeH​vHvvia automatic differentiation:

H​v=∇θ\[∇θℒ​\(θ\)⊤​v\],Hv=\\nabla\_\{\\theta\}\\left\[\\nabla\_\{\\theta\}\\mathcal\{L\}\(\\theta\)^\{\\top\}v\\right\],\(5\)which costsO​\(d\)O\(d\)per iteration—the same as computing a gradient\.

Afterkkiterations, CG producesvkv\_\{k\}satisfying:

‖vk−H−1​g‖H≤2​\(κ−1κ\+1\)k​‖H−1​g‖H,\\left\\\|v\_\{k\}\-H^\{\-1\}g\\right\\\|\_\{H\}\\leq 2\\left\(\\frac\{\\sqrt\{\\kappa\}\-1\}\{\\sqrt\{\\kappa\}\+1\}\\right\)^\{k\}\\left\\\|H^\{\-1\}g\\right\\\|\_\{H\},\(6\)whereκ=λmax​\(H\)/λmin​\(H\)\\kappa=\\lambda\_\{\\max\}\(H\)/\\lambda\_\{\\min\}\(H\)is the condition number and‖x‖H=x⊤​H​x\\left\\\|x\\right\\\|\_\{H\}=\\sqrt\{x^\{\\top\}Hx\}is the energy norm\. For well\-conditioned problems \(κ≈1\\kappa\\approx 1\), convergence is rapid; for ill\-conditioned problems, we apply damping \(Section[A\.2](https://arxiv.org/html/2605.20341#A1.SS2)\)\.

Exact vs\. Approximate Influence\.While damping \(Hdamped=H\+λ​IH\_\{\\text\{damped\}\}=H\+\\lambda I\) ensures numerical stability and bounds the condition numberκ\\kappa, it introduces a bias proportional toλ\\lambda\. This bias\-variance trade\-off means our method transitions from exact to approximate influence reversal\. The damping parameterλ\\lambdacontrols this trade\-off: smallerλ\\lambdareduces bias but may cause numerical instability, while largerλ\\lambdaimproves stability at the cost of approximation accuracy\.

### 3\.4Causal Weighting for Federated Settings

A naive application of Eq\. \([3](https://arxiv.org/html/2605.20341#S3.E3)\) in federated learning would compute a global update and broadcast it to all clients\. This is problematic: clients that never held𝒟u\\mathcal\{D\}\_\{u\}would receive parameter changes, violating causal structure\.

We introduce causal weightsαi\\alpha\_\{i\}for each client:

αi=\{‖∇θℓi​\(θ;𝒟u\)‖2∑j:𝒟u∩𝒟j≠∅‖∇θℓj​\(θ;𝒟u\)‖2,if​𝒟u∩𝒟i≠∅,0,otherwise\.\\alpha\_\{i\}=\\begin\{cases\}\\frac\{\\left\\\|\\nabla\_\{\\theta\}\\ell\_\{i\}\(\\theta;\\mathcal\{D\}\_\{u\}\)\\right\\\|\_\{2\}\}\{\\sum\_\{j:\\mathcal\{D\}\_\{u\}\\cap\\mathcal\{D\}\_\{j\}\\neq\\emptyset\}\\left\\\|\\nabla\_\{\\theta\}\\ell\_\{j\}\(\\theta;\\mathcal\{D\}\_\{u\}\)\\right\\\|\_\{2\}\},&\\text\{if \}\\mathcal\{D\}\_\{u\}\\cap\\mathcal\{D\}\_\{i\}\\neq\\emptyset,\\\\ 0,&\\text\{otherwise\}\.\\end\{cases\}\(7\)
This ensures: \(1\) clients without𝒟u\\mathcal\{D\}\_\{u\}receive zero weight, and \(2\) among affected clients, weights are proportional to gradient magnitude, reflecting influence strength\.

Impact on Non\-IID Settings\.In highly non\-IID federated settings \(e\.g\., Dirichletα=0\.5\\alpha=0\.5\), zeroingαi\\alpha\_\{i\}for unaffected clients preserves global model quality\. Restricting updates to clients holding𝒟u\\mathcal\{D\}\_\{u\}prevents catastrophic forgetting of knowledge learned from the remaining data distribution\. This localization targets the forget set𝒟u\\mathcal\{D\}\_\{u\}without introducing spurious parameter drift in model components responsible for other data distributions\. Broadcasting a global update to all clients would corrupt representations learned from unaffected data, leading to accuracy degradation as observed in naive gradient ascent methods\.

The causally weighted update for clientiiis:

Δ​θi=−αi⋅vk,i,\\Delta\\theta\_\{i\}=\-\\alpha\_\{i\}\\cdot v\_\{k,i\},\(8\)wherevk,iv\_\{k,i\}is the CG approximation ofHi−1​∇θℓi​\(θ;𝒟u\)H\_\{i\}^\{\-1\}\\nabla\_\{\\theta\}\\ell\_\{i\}\(\\theta;\\mathcal\{D\}\_\{u\}\)using the local HessianHi=∇θ2ℓi​\(θ;𝒟i\)H\_\{i\}=\\nabla^\{2\}\_\{\\theta\}\\ell\_\{i\}\(\\theta;\\mathcal\{D\}\_\{i\}\)\.

### 3\.5Complete Algorithm

Algorithm[1](https://arxiv.org/html/2605.20341#alg1)summarizes HF\-KCU\. Each affected client runs CG locally to approximate its influence, applies causal weighting and adaptive scaling, then sends the update to the server for aggregation\.

Algorithm 1HF\-KCU: Hessian\-Free Krylov Causal Unlearning0:Global model

θ\\theta, datasets

\{𝒟i\}i=1N\\\{\\mathcal\{D\}\_\{i\}\\\}\_\{i=1\}^\{N\}, forget set

𝒟u\\mathcal\{D\}\_\{u\}, CG iterations

kk, damping

λ\\lambda, scaling

β\\beta
0:Unlearned model

θu\\theta\_\{u\}
1:Server broadcasts

θ\\thetato all clients

2:foreach client

iiin paralleldo

3:if

𝒟u∩𝒟i≠∅\\mathcal\{D\}\_\{u\}\\cap\\mathcal\{D\}\_\{i\}\\neq\\emptysetthen

4:Compute

gi=∇θℓi​\(θ;𝒟u\)g\_\{i\}=\\nabla\_\{\\theta\}\\ell\_\{i\}\(\\theta;\\mathcal\{D\}\_\{u\}\)
5:Define HVP:

HVPi​\(v\)=∇θ\[∇θℓi​\(θ;𝒟i\)⊤​v\]\+λ​v\\text\{HVP\}\_\{i\}\(v\)=\\nabla\_\{\\theta\}\[\\nabla\_\{\\theta\}\\ell\_\{i\}\(\\theta;\\mathcal\{D\}\_\{i\}\)^\{\\top\}v\]\+\\lambda v
6:Initialize

v0=0v\_\{0\}=0,

r0=gir\_\{0\}=g\_\{i\},

p0=r0p\_\{0\}=r\_\{0\}
7:for

t=0,…,k−1t=0,\\ldots,k\-1do

8:

qt=HVPi​\(pt\)q\_\{t\}=\\text\{HVP\}\_\{i\}\(p\_\{t\}\)
9:

αt=rt⊤​rt/\(pt⊤​qt\)\\alpha\_\{t\}=r\_\{t\}^\{\\top\}r\_\{t\}/\(p\_\{t\}^\{\\top\}q\_\{t\}\)
10:

vt\+1=vt\+αt​ptv\_\{t\+1\}=v\_\{t\}\+\\alpha\_\{t\}p\_\{t\}
11:

rt\+1=rt−αt​qtr\_\{t\+1\}=r\_\{t\}\-\\alpha\_\{t\}q\_\{t\}
12:

βt=rt\+1⊤​rt\+1/\(rt⊤​rt\)\\beta\_\{t\}=r\_\{t\+1\}^\{\\top\}r\_\{t\+1\}/\(r\_\{t\}^\{\\top\}r\_\{t\}\)
13:

pt\+1=rt\+1\+βt​ptp\_\{t\+1\}=r\_\{t\+1\}\+\\beta\_\{t\}p\_\{t\}
14:endfor

15:

Δ​θiraw=−vk\\Delta\\theta\_\{i\}^\{\\text\{raw\}\}=\-v\_\{k\}
16:Compute causal weight

αi\\alpha\_\{i\}from Eq\. \([7](https://arxiv.org/html/2605.20341#S3.E7)\)

17:

scalei=min⁡\(1,β​‖θ‖2/‖Δ​θiraw‖2\)\\text\{scale\}\_\{i\}=\\min\(1,\\beta\\left\\\|\\theta\\right\\\|\_\{2\}/\\left\\\|\\Delta\\theta\_\{i\}^\{\\text\{raw\}\}\\right\\\|\_\{2\}\)
18:

Δ​θi=αi⋅scalei⋅Δ​θiraw\\Delta\\theta\_\{i\}=\\alpha\_\{i\}\\cdot\\text\{scale\}\_\{i\}\\cdot\\Delta\\theta\_\{i\}^\{\\text\{raw\}\}
19:Send

Δ​θi\\Delta\\theta\_\{i\}to server

20:else

21:Send

Δ​θi=0\\Delta\\theta\_\{i\}=0
22:endif

23:endfor

24:Server aggregates:

Δ​θ=∑iwi​Δ​θi\\Delta\\theta=\\sum\_\{i\}w\_\{i\}\\Delta\\theta\_\{i\}
25:Update:

θu=θ\+Δ​θ\\theta\_\{u\}=\\theta\+\\Delta\\theta
26:return

θu\\theta\_\{u\}

Complexity\.Each client performskkCG iterations withO​\(d\)O\(d\)cost per iteration, yielding total complexityO​\(k​d\)O\(kd\)per client\. Memory remainsO​\(d\)O\(d\)since we never formHHexplicitly\. Compared to retraining \(O​\(T⋅N⋅d\)O\(T\\cdot N\\cdot d\)whereT∼20T\\sim 20rounds\), HF\-KCU achieves theoretical speedup ofO​\(T​N/k\)O\(TN/k\)\.

## 4Theoretical Analysis

We analyze HF\-KCU’s convergence properties, approximation quality, and causal isolation guarantees\. Due to space constraints, the complete theoretical framework; including formal definitions of our assumptions by full theorem statements and all mathematical proofs is deferred to Appendix[A](https://arxiv.org/html/2605.20341#A1)\.

Our analysis relies on standard regularity conditions on the loss landscape, such as smoothness and strong convexity\. While strong convexity may not hold globally for neural networks, it often holds locally near a trained optimum\. Our empirical results \(Section[5](https://arxiv.org/html/2605.20341#S5)\) show that HF\-KCU performs well even in practical settings where these strict theoretical assumptions are relaxed\.

## 5Experiments

We evaluate HF\-KCU on federated unlearning tasks to assess efficiency, accuracy retention, and privacy guarantees compared to retraining and existing methods\.

### 5\.1Experimental Setup

Dataset and Partitioning\.We use CIFAR\-10Krizhevsky and Hinton \([2009](https://arxiv.org/html/2605.20341#bib.bib22)\)with 50,000 training images across 10 classes\. Data is partitioned amongN=10N=10clients using a Dirichlet distribution with concentration parameterα=0\.5\\alpha=0\.5to simulate non\-IID federated settings\. Our full evaluation, which includes scaling to 200 clients and varying IID assumptions, is detailed in the Appendix\.

Model Architecture\.We evaluate on MNIST and Fashion\-MNIST using a custom CNN \(d=188,810d=188\{,\}810parameters\)\. To demonstrate scalability to more complex networks, we also conduct experiments using ResNet and Vision Transformer \(ViT\) architectures\.

Training Protocol\.We train using FedAvgMcMahanet al\.\([2017](https://arxiv.org/html/2605.20341#bib.bib1)\)for 20 communication rounds with 5 local epochs per round, batch size 64, learning rate 0\.01, and SGD optimizer\. All experiments run on NVIDIA GPUs\.

Unlearning Task\.We unlearn all data from Client 0 \(approximately 10% of total data\) and measure how closely the unlearned model matches a model retrained from scratch without that client\.

HF\-KCU Hyperparameters\.We setk=10k=10conjugate gradient iterations, damping parameterλ=0\.01\\lambda=0\.01, and scaling factorβ=0\.01\\beta=0\.01\. These values balance approximation quality and computational cost\.

10010^\{0\}10110^\{1\}10210^\{2\}686870707272Baseline47\.75×Pareto frontierSpeedup Factor \(log scale\)Test Accuracy \(%\)\(a\) Speedup\-Accuracy ParetoRetrainedNaiveGAFedEraserSISAHF\-KCUw/o causalRetrain10010^\{0\}10110^\{1\}10210^\{2\}10310^\{3\}47\.75×6\.96\.9332\.42\.4Time \(seconds, log scale\)\(b\) Computational Cost

Figure 1:Performance comparison of HF\-KCU against baselines\. \(a\) Pareto frontier showing speedup vs accuracy tradeoff\. HF\-KCU achieves 47\.75× speedup while maintaining accuracy within 0\.60% of the retrained baseline\. \(b\) Computational cost comparison on log scale, demonstrating the efficiency gains of the Hessian\-free approach\.Baselines\.We compare against:

- •FedAvg: Original trained model \(no unlearning\)\.
- •Retrained: Gold standard full retraining without Client 0\.
- •Naive Gradient Ascent \(NaiveGA\): Apply gradient ascent on forget set for 10 epochs\.
- •FedEraserLiuet al\.\([2021a](https://arxiv.org/html/2605.20341#bib.bib9)\): State\-of\-the\-art federated unlearning via calibration\.
- •SISABourtouleet al\.\([2021](https://arxiv.org/html/2605.20341#bib.bib6)\): Sharded training with ensemble aggregation\.

Evaluation Metrics\.

- •Test Accuracy: Performance on held\-out test set\.
- •Causal Faithfulness \(CF\): Measures fidelity to retrained model, CF=1−\|Accunlearned−Accretrain\|\|Acctrained−Accretrain\|\.\\text\{CF\}=1\-\\frac\{\|\\text\{Acc\}\_\{\\text\{unlearned\}\}\-\\text\{Acc\}\_\{\\text\{retrain\}\}\|\}\{\|\\text\{Acc\}\_\{\\text\{trained\}\}\-\\text\{Acc\}\_\{\\text\{retrain\}\}\|\}\.Higher values indicate closer match to retraining\. Note that CF becomes unstable when the denominator is small\.
- •Parameter Gap: Normalizedℓ2\\ell\_\{2\}distance, Gap=‖θunlearned−θretrain‖‖θretrain‖\.\\text\{Gap\}=\\frac\{\\left\\\|\\theta\_\{\\text\{unlearned\}\}\-\\theta\_\{\\text\{retrain\}\}\\right\\\|\}\{\\left\\\|\\theta\_\{\\text\{retrain\}\}\\right\\\|\}\.
- •Speedup: Ratio of retraining time to unlearning time\.
- •MIA Success Rate: Membership inference attackShokriet al\.\([2017](https://arxiv.org/html/2605.20341#bib.bib21)\)success rate on forget set \(lower indicates better privacy\)\.

### 5\.2Main Results

Table[1](https://arxiv.org/html/2605.20341#S5.T1)summarizes performance across all methods\. HF\-KCU achieves 47\.75×\\timesspeedup over retraining while maintaining test accuracy within 0\.60% of the retrained model \(averaged over 5 random seeds\)\.

Table 1:Experimental results on CIFAR\-10 with 10 clients\. Results averaged over 5 seeds; standard deviations in parentheses\. CF values for baselines computed using their reported accuracies\.Key Observations:

- •Accuracy: HF\-KCU achieves 71\.16% test accuracy, only 0\.60% below the retrained oracle \(70\.79%\) and 0\.60% below the original trained model \(71\.76%\)\. This demonstrates effective unlearning with minimal accuracy loss\.
- •Efficiency: HF\-KCU provides 47\.75×\\timesspeedup over retraining, faster than FedEraser \(5×\\times\) and SISA \(3\.3×\\times\)\. The Krylov subspace approximation withk=10k=10iterations avoids expensive Hessian inversion\. We initiated a label\-flipping attack where a subset of malicious clients inverted labels during federated training\. Under the weak attack setting, the poisoned model and oracle retraining achieved nearly identical accuracy, indicating that the attack introduced little removable influence\. Consistent with this, HF\-KCU produced only a minimal update and near\-zero causal fidelity\.
- •Privacy: MIA success rate drops from 0\.85 \(trained model\) to 0\.499, matching the retrained oracle \(0\.50\)\. This indicates that Client 0’s data has been effectively removed from the model’s memory\.
- •Parameter\-Space Fidelity: The normalized parameter gap is 0\.032, indicating that HF\-KCU’s parameters lie close to the retrained model in weight space\.
- •NaiveGA Failure: Naive Gradient Ascent catastrophically fails with 10\.00% accuracy \(random guessing on CIFAR\-10\)\. This occurs because NaiveGA applies gradient ascent on the forget set without accounting for Hessian curvature or causal structure, destroying global model weights\. In contrast, HF\-KCU uses the inverse Hessian to precisely localize updates and causal weighting to prevent corruption of unaffected clients\.
- •Unlearning Trilemma: HF\-KCU simultaneously achieves high accuracy \(71\.16%\), strong privacy \(MIA=0\.499\), and large speedup \(47\.75×\\times\)\. FedEraser and SISA sacrifice accuracy \(drops to 69\.52% and 67\.80%\) for modest speedups \(5×\\timesand 3\.3×\\times\), while NaiveGA achieves speed but destroys model utility\.

Table 2:HF\-KCU performance across datasets and architectures with 10 clients \(client 0 removed\)\.Discussion of CF Metric\.The CF values in Table[1](https://arxiv.org/html/2605.20341#S5.T1)reveal a limitation of this metric\. HF\-KCU achieves CF = 0\.38, while baselines show negative values\. This occurs because the denominator\|Acctrained−Accretrain\|=\|71\.76−70\.79\|=0\.97%\|\\text\{Acc\}\_\{\\text\{trained\}\}\-\\text\{Acc\}\_\{\\text\{retrain\}\}\|=\|71\.76\-70\.79\|=0\.97\\%is small, causing high variance and instability\. When accuracy differences are below 1%, small fluctuations in numerator accuracy can produce misleading CF scores\.

Despite the modest CF value, multiple indicators support HF\-KCU’s effectiveness: \(1\) accuracy within 0\.60% of oracle, \(2\) parameter gap of 0\.032, \(3\) MIA matching retraining\. The CF metric is unreliable for scenarios where training and retraining produce similar accuracies—a common case when unlearning affects only a small fraction of data\. Future work should develop more stable faithfulness metrics across different accuracy regimes\.

### 5\.3Case Study: Unlearning a Targeted Backdoor Attack

To evaluate the practical utility of our method in a security\-critical scenario, we simulate a data poisoning attack where a single malicious client introduces a targeted backdoor into the global model during federated training\.

#### Experimental Setup

The threat model consists of one adversary aiming to make the model misclassify inputs containing a specific trigger to a target label\. We measure performance using three key metrics: Benign Accuracy on the clean test set, the Attack Success Rate \(ASR\) on the triggered test set, and the wall\-clock Unlearning Time\. We compare our method, HF\-KCU, against the Oracle baseline, which involves fully retraining the model from scratch on all benign client data\.

#### Results

As shown in Table[3](https://arxiv.org/html/2605.20341#S5.T3), the initially poisoned model achieves a high ASR of 84\.35%\. Our HF\-KCU method successfully mitigates this threat in just 4\.34 seconds, achieving a massive 430x speedup over the Oracle\. Crucially, this efficiency gain comes with a negligible 0\.04% drop in benign accuracy, preserving model utility almost perfectly\. While the Oracle achieves a more complete removal of the backdoor, its prohibitive time cost makes it impractical for immediate threat response\. This positions HF\-KCU as an ideal first\-line\-of\-defense mechanism to instantly neutralize an attack, while a full retraining can be scheduled offline for more thorough sanitization\. HF\-KCU achieves 430× speedup while removing 15\.4% of backdoor influence with negligible impact on clean accuracy \(CF=0\.0033\)\.

Table 3:Backdoor attack mitigation results on CIFAR\-10\. One malicious client \(10% of federation\) injects backdoor with 50% poison ratio targeting class 7 over 20 federated rounds\.

### 5\.4Evaluation Methodology: Functional vs\. Parameter\-Space Metrics

We evaluate unlearning quality using functional equivalence metrics \(output KL divergence, logit MSE, test accuracy gap\) rather than parameter\-space distance, which can be misleading due to permutation symmetry and loss landscape geometry \(see Appendix[C](https://arxiv.org/html/2605.20341#A3)for detailed justification\)

### 5\.5Ablation Studies

To isolate the impact of our design choices and evaluate system scalability, we conducted extensive experiments\. Full tables and detailed discussions are deferred to Appendix[B](https://arxiv.org/html/2605.20341#A2)\.

Component Ablations:Removing key components validates our architectural choices\. Removing causal weighting increases the parameter gap significantly \(from0\.0320\.032to0\.0890\.089, nearly3×3\\times\), supporting our theoretical claim that it is necessary for update localization\. Without it, unlearning updates bleed into unaffected clients, causing spurious parameter drift\. Similarly, removing adaptive scaling causes training instability, reducing accuracy to65\.23%65\.23\\%\.

Damping Parameter \(λ\\lambda\) & Heterogeneity \(α\\alpha\):A smallerλ\\lambda\(e\.g\.,0\.0010\.001\) heavily prioritizes unlearning efficacy, yielding massive speedups \(209\.74×209\.74\\times\), while larger values prioritize preserving original weights\. Furthermore, varying the Dirichlet distribution parameter \(α\\alpha\) confirms HF\-KCU remains robust across non\-IID distributions, further validating our critique of the Causal Faithfulness metric’s instability in non\-IID edge cases\.

Scalability \(Clients & Model Size\):HF\-KCU’s computational efficiency scales exceptionally well\. Increasing the network from1010to200200clients increases the speedup factor from76\.48×76\.48\\timesto approximately110×110\\times, while the parameter gap shrinks as the marginal contribution of the unlearned data decreases\. Across model sizes, unlearning time scales asO​\(k​d\)O\(kd\)\. For an1111M\-parameter ResNet\-18, HF\-KCU requires≈180\\approx 180seconds compared to6,3006,300seconds for full retraining \(35×35\\timesspeedup\)\.

### 5\.6Limitations and Future Directions

While HF\-KCU significantly improves unlearning efficiency, it assumes bounded adversarial perturbations and its memory complexity \(O​\(k​d\)O\(kd\)\) may pose challenges for ultra\-large models \(e\.g\., billion\-parameter LLMs\)\. Additionally, the instability of the Causal Faithfulness metric in high\-accuracy regimes highlights the need for novel evaluation metrics\. A comprehensive discussion of these limitations, approximation bounds, sequential unlearning, and broader impacts is provided in Appendix[D](https://arxiv.org/html/2605.20341#A4)\.

## 6Conclusion

In this paper, we introduced HF\-KCU, a highly efficient federated unlearning framework that eliminates target client influence without exact retraining\. Our empirical evaluation demonstrates speedups of49×49\\timesto over200×200\\timescompared to retraining from scratch while strictly preserving global model utility\.

Beyond algorithmic efficiency, we provided a rigorous methodological critique of current evaluation standards\. We exposed fundamental vulnerabilities in the widely adopted Causal Faithfulness metric and demonstrating its instability and susceptibility to misleading negative signs, advocated for KL Divergence as a stable, theoretically grounded alternative for assessing functional equivalence\.

Our ablation studies confirm HF\-KCU’s robustness across heterogeneous data distributions \(varying Dirichletα\\alpha\) and highlight the critical role of damping parameterλ\\lambdain balancing acceleration against stability\. In adversarial settings, HF\-KCU serves as a rapid\-response defense, reducing backdoor attack success rates in seconds versus over 30 minutes for full retraining\.

Future work will extend this Hessian\-free paradigm to larger foundation models and explore adaptive damping strategies for highly non\-IID federated networks\. HF\-KCU provides a mathematically sound foundation for practical deployment of the "right to be forgotten" in resource\-constrained federated learning systems\.

## References

- Git re\-basin: merging models modulo permutation symmetries\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[Appendix C](https://arxiv.org/html/2605.20341#A3.SS0.SSS0.Px1.p1.1)\.
- L\. Bourtoule, V\. Chandrasekaran, C\. A\. Choquette\-Choo, H\. Jia, A\. Travers, B\. Zhang, D\. Lie, and N\. Papernot \(2021\)Machine unlearning\.In2021 IEEE Symposium on Security and Privacy \(SP\),pp\. 141–159\.Cited by:[§1](https://arxiv.org/html/2605.20341#S1.p3.1),[§2\.1](https://arxiv.org/html/2605.20341#S2.SS1.p2.2),[5th item](https://arxiv.org/html/2605.20341#S5.I1.i5.p1.1)\.
- California State Legislature \(2018\)California consumer privacy act \(ccpa\) of 2018\.Note:Title 1\.81\.5, §1798\.100 et seq\.Cited by:[§1](https://arxiv.org/html/2605.20341#S1.p1.1)\.
- Y\. Cao and J\. Yang \(2015\)Towards making systems forget with machine unlearning\.In2015 IEEE Symposium on Security and Privacy,pp\. 463–480\.Cited by:[§1](https://arxiv.org/html/2605.20341#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.20341#S2.SS1.p1.1)\.
- R\. D\. Cook and S\. Weisberg \(1980\)Characterizations of an empirical influence function for detecting influential cases in regression\.Technometrics22\(4\),pp\. 495–508\.Cited by:[§2\.4](https://arxiv.org/html/2605.20341#S2.SS4.p1.4)\.
- R\. Entezari, H\. Sedghi, O\. Saukh, and B\. Neyshabur \(2022\)The role of permutation invariance in linear mode connectivity of neural networks\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[Appendix C](https://arxiv.org/html/2605.20341#A3.SS0.SSS0.Px1.p1.1)\.
- European Parliament and Council of the European Union \(2016\)Regulation \(eu\) 2016/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/ec \(general data protection regulation\)\.Vol\.L119\.Cited by:[§1](https://arxiv.org/html/2605.20341#S1.p1.1)\.
- S\. Fort, H\. Hu, and B\. Lakshminarayanan \(2019\)Deep ensembles: a loss landscape perspective\.arXiv preprint arXiv:1912\.02757\.Cited by:[Appendix C](https://arxiv.org/html/2605.20341#A3.SS0.SSS0.Px1.p1.1)\.
- A\. Ginart, M\. Guan, G\. Valiant, and J\. Y\. Zou \(2019\)Making ai forget you: data deletion in machine learning\.Advances in Neural Information Processing Systems32\.Cited by:[§2\.1](https://arxiv.org/html/2605.20341#S2.SS1.p1.1)\.
- A\. Golatkar, A\. Achille, and S\. Soatto \(2020\)Eternal sunshine of the spotless net: selective forgetting in deep networks\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 9304–9312\.Cited by:[§1](https://arxiv.org/html/2605.20341#S1.p3.1),[§2\.1](https://arxiv.org/html/2605.20341#S2.SS1.p2.2)\.
- C\. Guo, T\. Goldstein, A\. Hannun, and L\. V\. D\. Maaten \(2020\)Certified data removal from machine learning models\.InInternational Conference on Machine Learning,pp\. 3832–3842\.Cited by:[§A\.10](https://arxiv.org/html/2605.20341#A1.SS10.p1.1),[§2\.1](https://arxiv.org/html/2605.20341#S2.SS1.p1.1)\.
- P\. Kairouz, H\. B\. McMahan, B\. Avent, A\. Bellet, M\. Bennis, A\. N\. Bhagoji, K\. Bonawitz, Z\. Charles, G\. Cormode, R\. Cummings,et al\.\(2021\)Advances and open problems in federated learning\.Foundations and Trends in Machine Learning14\(1–2\),pp\. 1–210\.Cited by:[§2\.2](https://arxiv.org/html/2605.20341#S2.SS2.p1.1)\.
- P\. W\. Koh and P\. Liang \(2017\)Understanding black\-box predictions via influence functions\.InInternational Conference on Machine Learning,pp\. 1885–1894\.Cited by:[§A\.12](https://arxiv.org/html/2605.20341#A1.SS12.p1.1),[§1](https://arxiv.org/html/2605.20341#S1.p3.1),[§2\.1](https://arxiv.org/html/2605.20341#S2.SS1.p2.2),[§2\.4](https://arxiv.org/html/2605.20341#S2.SS4.p1.4),[§3\.2](https://arxiv.org/html/2605.20341#S3.SS2.p1.1)\.
- A\. Krizhevsky and G\. Hinton \(2009\)Learning multiple layers of features from tiny images\.Technical reportUniversity of Toronto\.Cited by:[§5\.1](https://arxiv.org/html/2605.20341#S5.SS1.p1.2)\.
- G\. Liu, X\. Ma, Y\. Yang, C\. Wang, and J\. Liu \(2021a\)FedEraser: enabling efficient client\-level data removal from federated learning models\.In2021 IEEE/ACM 29th International Symposium on Quality of Service \(IWQOS\),pp\. 1–10\.Cited by:[4th item](https://arxiv.org/html/2605.20341#S5.I1.i4.p1.1)\.
- Y\. Liu, Y\. Liu, Z\. Liu, and Y\. Liu \(2021b\)FedEraser: enabling efficient client\-level data removal from federated learning models\.InInternational Conference on Computer Communications \(INFOCOM\),pp\. 1–10\.Cited by:[§2\.3](https://arxiv.org/html/2605.20341#S2.SS3.p1.1)\.
- Z\. Liu, Y\. Jiang, J\. Shen, M\. Peng, K\. Lam, X\. Yuan, and X\. Liu \(2024\)A survey on federated unlearning: challenges, methods, and future directions\.ACM Computing Surveys\.Note:arXiv:2310\.20448Cited by:[§1](https://arxiv.org/html/2605.20341#S1.p3.1)\.
- J\. Martens \(2010\)Deep learning via hessian\-free optimization\.InProceedings of the 27th International Conference on Machine Learning,pp\. 735–742\.Cited by:[§2\.4](https://arxiv.org/html/2605.20341#S2.SS4.p2.2)\.
- B\. McMahan, E\. Moore, D\. Ramage, S\. Hampson, and B\. A\. y Arcas \(2017\)Communication\-efficient learning of deep networks from decentralized data\.InArtificial Intelligence and Statistics,pp\. 1273–1282\.Cited by:[§1](https://arxiv.org/html/2605.20341#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.20341#S2.SS2.p1.1),[§5\.1](https://arxiv.org/html/2605.20341#S5.SS1.p3.1)\.
- B\. A\. Pearlmutter \(1994\)Fast exact multiplication by the hessian\.Neural Computation6\(1\),pp\. 147–160\.Cited by:[§2\.4](https://arxiv.org/html/2605.20341#S2.SS4.p2.2)\.
- J\. R\. Shewchuk \(1994\)An introduction to the conjugate gradient method without the agonizing pain\.Technical reportCarnegie Mellon University\.Cited by:[§A\.6](https://arxiv.org/html/2605.20341#A1.SS6.1.p1.7)\.
- R\. Shokri, M\. Stronati, C\. Song, and V\. Shmatikov \(2017\)Membership inference attacks against machine learning models\.In2017 IEEE Symposium on Security and Privacy,pp\. 3–18\.Cited by:[5th item](https://arxiv.org/html/2605.20341#S5.I2.i5.p1.1)\.
- H\. X\. Tae and C\. S\. Chan \(2025\)A survey of challenges and opportunities in vertical federated unlearning\.IEEE Access13,pp\. 3600884\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2025.3600884)Cited by:[§1](https://arxiv.org/html/2605.20341#S1.p3.1)\.
- A\. Tarun, V\. S\. Chundawat, M\. Mandal, and M\. Kankanhalli \(2023\)Fast yet effective machine unlearning\.IEEE Transactions on Neural Networks and Learning Systems34\(8\),pp\. 4292–4304\.Cited by:[§2\.1](https://arxiv.org/html/2605.20341#S2.SS1.p2.2)\.
- Y\. Wu, S\. Zhao, W\. Li, J\. Liu, and H\. Guan \(2022\)Federated unlearning with knowledge distillation\.In2022 IEEE 42nd International Conference on Distributed Computing Systems \(ICDCS\),pp\. 1158–1168\.Cited by:[§2\.3](https://arxiv.org/html/2605.20341#S2.SS3.p1.1)\.

## Appendix ADeferred Proofs and Extended Theoretical Analysis

### A\.1Robustness to Bounded Adversarial Perturbations

In federated settings, adversarial clients may contribute poisoned data that perturbs the global loss landscape\. We now analyze the stability of our influence reversal method under bounded adversarial perturbations to both the Hessian and contribution gradients\.

###### Theorem 1\(Stability Under Bounded Adversarial Perturbations\)\.

Let𝐇≻0\\mathbf\{H\}\\succ 0withλmin​\(𝐇\)≥μ\>0\\lambda\_\{\\min\}\(\\mathbf\{H\}\)\\geq\\mu\>0\. Suppose an adversarial client induces perturbed quantities

𝐇~=𝐇\+𝐄,𝐠~=𝐠\+𝐞,\\tilde\{\\mathbf\{H\}\}=\\mathbf\{H\}\+\\mathbf\{E\},\\qquad\\tilde\{\\mathbf\{g\}\}=\\mathbf\{g\}\+\\mathbf\{e\},\(9\)where‖𝐄‖2≤ε<μ\\\|\\mathbf\{E\}\\\|\_\{2\}\\leq\\varepsilon<\\muand‖𝐞‖2≤ρ\\\|\\mathbf\{e\}\\\|\_\{2\}\\leq\\rho\. Then the exact influence reversal updates satisfy

‖𝜹~⋆−𝜹⋆‖2≤εμ​\(μ−ε\)​‖𝐠‖2\+ρμ−ε,\\\|\\tilde\{\\boldsymbol\{\\delta\}\}^\{\\star\}\-\\boldsymbol\{\\delta\}^\{\\star\}\\\|\_\{2\}\\leq\\frac\{\\varepsilon\}\{\\mu\(\\mu\-\\varepsilon\)\}\\\|\\mathbf\{g\}\\\|\_\{2\}\+\\frac\{\\rho\}\{\\mu\-\\varepsilon\},\(10\)where𝛅⋆=−𝐇−1​𝐠\\boldsymbol\{\\delta\}^\{\\star\}=\-\\mathbf\{H\}^\{\-1\}\\mathbf\{g\}and𝛅~⋆=−𝐇~−1​𝐠~\\tilde\{\\boldsymbol\{\\delta\}\}^\{\\star\}=\-\\tilde\{\\mathbf\{H\}\}^\{\-1\}\\tilde\{\\mathbf\{g\}\}\.

###### Proof\.

We decompose the error as

𝜹~⋆−𝜹⋆=−𝐇~−1​𝐠~\+𝐇−1​𝐠=−𝐇~−1​\(𝐠~−𝐠\)−\(𝐇~−1−𝐇−1\)​𝐠\.\\tilde\{\\boldsymbol\{\\delta\}\}^\{\\star\}\-\\boldsymbol\{\\delta\}^\{\\star\}=\-\\tilde\{\\mathbf\{H\}\}^\{\-1\}\\tilde\{\\mathbf\{g\}\}\+\\mathbf\{H\}^\{\-1\}\\mathbf\{g\}=\-\\tilde\{\\mathbf\{H\}\}^\{\-1\}\(\\tilde\{\\mathbf\{g\}\}\-\\mathbf\{g\}\)\-\(\\tilde\{\\mathbf\{H\}\}^\{\-1\}\-\\mathbf\{H\}^\{\-1\}\)\\mathbf\{g\}\.\(11\)
For the first term, since‖𝐄‖2<μ\\\|\\mathbf\{E\}\\\|\_\{2\}<\\mu, the perturbed Hessian remains invertible with

‖𝐇~−1‖2≤1λmin​\(𝐇~\)≤1μ−ε\.\\\|\\tilde\{\\mathbf\{H\}\}^\{\-1\}\\\|\_\{2\}\\leq\\frac\{1\}\{\\lambda\_\{\\min\}\(\\tilde\{\\mathbf\{H\}\}\)\}\\leq\\frac\{1\}\{\\mu\-\\varepsilon\}\.\(12\)
For the second term, we apply the resolvent identity:

𝐇~−1−𝐇−1=−𝐇−1​𝐄​𝐇~−1\.\\tilde\{\\mathbf\{H\}\}^\{\-1\}\-\\mathbf\{H\}^\{\-1\}=\-\\mathbf\{H\}^\{\-1\}\\mathbf\{E\}\\tilde\{\\mathbf\{H\}\}^\{\-1\}\.\(13\)
Taking norms:

‖𝐇~−1−𝐇−1‖2\\displaystyle\\\|\\tilde\{\\mathbf\{H\}\}^\{\-1\}\-\\mathbf\{H\}^\{\-1\}\\\|\_\{2\}≤‖𝐇−1‖2​‖𝐄‖2​‖𝐇~−1‖2\\displaystyle\\leq\\\|\\mathbf\{H\}^\{\-1\}\\\|\_\{2\}\\\|\\mathbf\{E\}\\\|\_\{2\}\\\|\\tilde\{\\mathbf\{H\}\}^\{\-1\}\\\|\_\{2\}\(14\)≤1μ⋅ε⋅1μ−ε=εμ​\(μ−ε\)\.\\displaystyle\\leq\\frac\{1\}\{\\mu\}\\cdot\\varepsilon\\cdot\\frac\{1\}\{\\mu\-\\varepsilon\}=\\frac\{\\varepsilon\}\{\\mu\(\\mu\-\\varepsilon\)\}\.\(15\)
Combining both terms:

‖𝜹~⋆−𝜹⋆‖2\\displaystyle\\\|\\tilde\{\\boldsymbol\{\\delta\}\}^\{\\star\}\-\\boldsymbol\{\\delta\}^\{\\star\}\\\|\_\{2\}≤‖𝐇~−1‖2​‖𝐞‖2\+‖𝐇~−1−𝐇−1‖2​‖𝐠‖2\\displaystyle\\leq\\\|\\tilde\{\\mathbf\{H\}\}^\{\-1\}\\\|\_\{2\}\\\|\\mathbf\{e\}\\\|\_\{2\}\+\\\|\\tilde\{\\mathbf\{H\}\}^\{\-1\}\-\\mathbf\{H\}^\{\-1\}\\\|\_\{2\}\\\|\\mathbf\{g\}\\\|\_\{2\}\(16\)≤ρμ−ε\+εμ​\(μ−ε\)​‖𝐠‖2\.\\displaystyle\\leq\\frac\{\\rho\}\{\\mu\-\\varepsilon\}\+\\frac\{\\varepsilon\}\{\\mu\(\\mu\-\\varepsilon\)\}\\\|\\mathbf\{g\}\\\|\_\{2\}\.\(17\)∎

###### Corollary 2\(Approximate Solver Robustness\)\.

If𝛅^\\hat\{\\boldsymbol\{\\delta\}\}is an approximate solution \(e\.g\., from CG withkkiterations\) satisfying‖𝛅^−𝛅~⋆‖2≤ξ\\\|\\hat\{\\boldsymbol\{\\delta\}\}\-\\tilde\{\\boldsymbol\{\\delta\}\}^\{\\star\}\\\|\_\{2\}\\leq\\xi, then the total error is bounded by

‖𝜹^−𝜹⋆‖2≤εμ​\(μ−ε\)​‖𝐠‖2\+ρμ−ε⏟Adversarial Perturbation Error\+ξ⏟CG Approximation Error\.\\\|\\hat\{\\boldsymbol\{\\delta\}\}\-\\boldsymbol\{\\delta\}^\{\\star\}\\\|\_\{2\}\\leq\\underbrace\{\\frac\{\\varepsilon\}\{\\mu\(\\mu\-\\varepsilon\)\}\\\|\\mathbf\{g\}\\\|\_\{2\}\+\\frac\{\\rho\}\{\\mu\-\\varepsilon\}\}\_\{\\text\{Adversarial Perturbation Error\}\}\+\\underbrace\{\\xi\}\_\{\\text\{CG Approximation Error\}\}\.\(18\)This bound combines the adversarial perturbation error from Theorem[1](https://arxiv.org/html/2605.20341#Thmtheorem1)with the CG approximation error via the triangle inequality\. The total error is the sum of two independent sources: \(1\) perturbations to the Hessian and gradient, and \(2\) finite CG iterations\. Both errors can be controlled: adversarial error scales inversely withμ\\mu\(improved by damping\), while CG error decreases exponentially withkk\(Theorem[4](https://arxiv.org/html/2605.20341#Thmtheorem4)\)\.

###### Proof\.

By triangle inequality:

‖𝜹^−𝜹⋆‖2≤‖𝜹^−𝜹~⋆‖2\+‖𝜹~⋆−𝜹⋆‖2≤ξ\+εμ​\(μ−ε\)​‖𝐠‖2\+ρμ−ε\.\\\|\\hat\{\\boldsymbol\{\\delta\}\}\-\\boldsymbol\{\\delta\}^\{\\star\}\\\|\_\{2\}\\leq\\\|\\hat\{\\boldsymbol\{\\delta\}\}\-\\tilde\{\\boldsymbol\{\\delta\}\}^\{\\star\}\\\|\_\{2\}\+\\\|\\tilde\{\\boldsymbol\{\\delta\}\}^\{\\star\}\-\\boldsymbol\{\\delta\}^\{\\star\}\\\|\_\{2\}\\leq\\xi\+\\frac\{\\varepsilon\}\{\\mu\(\\mu\-\\varepsilon\)\}\\\|\\mathbf\{g\}\\\|\_\{2\}\+\\frac\{\\rho\}\{\\mu\-\\varepsilon\}\.\(19\)∎

### A\.2Damping and Adaptive Scaling

Neural network Hessians are often ill\-conditioned or indefinite\. We apply two stabilization techniques:

Damping\.We regularize the Hessian:

Hdamped=H\+λ​I,H\_\{\\text\{damped\}\}=H\+\\lambda I,\(20\)whereλ\>0\\lambda\>0\(we useλ=0\.01\\lambda=0\.01\)\. This improves conditioning and ensures positive definiteness, accelerating CG convergence\. Damping also improves the theoretical robustness bound: by addingλ​I\\lambda Ito the Hessian, we replace the minimum eigenvalueμ\\muwith\(μ\+λ\)\(\\mu\+\\lambda\)in the adversarial error bound\. The denominator termϵ\(μ\+λ\)​\(\(μ\+λ\)−ϵ\)\\frac\{\\epsilon\}\{\(\\mu\+\\lambda\)\(\(\\mu\+\\lambda\)\-\\epsilon\)\}in the perturbation error bound shrinks asλ\\lambdaincreases, limiting the magnitude of adversarial perturbation effects\.

Adaptive Scaling\.To prevent excessively large updates, we normalize:

scale=min⁡\(1,β⋅‖θ‖2‖Δ​θraw‖2\),\\text\{scale\}=\\min\\left\(1,\\frac\{\\beta\\cdot\\left\\\|\\theta\\right\\\|\_\{2\}\}\{\\left\\\|\\Delta\\theta\_\{\\text\{raw\}\}\\right\\\|\_\{2\}\}\\right\),\(21\)whereβ∈\(0,1\)\\beta\\in\(0,1\)\(we useβ=0\.01\\beta=0\.01\)\. The final update isΔ​θ=scale⋅Δ​θraw\\Delta\\theta=\\text\{scale\}\\cdot\\Delta\\theta\_\{\\text\{raw\}\}\. This prevents parameter drift while maintaining unlearning effectiveness\.

### A\.3Formal Assumptions

### A\.4Assumptions

###### Assumption 1\(Smoothness\)\.

Each local lossℓi​\(θ;𝒟i\)\\ell\_\{i\}\(\\theta;\\mathcal\{D\}\_\{i\}\)isLL\-smooth: for allθ,θ′∈ℝd\\theta,\\theta^\{\\prime\}\\in\\mathbb\{R\}^\{d\},

‖∇ℓi​\(θ\)−∇ℓi​\(θ′\)‖2≤L​‖θ−θ′‖2\.\\left\\\|\\nabla\\ell\_\{i\}\(\\theta\)\-\\nabla\\ell\_\{i\}\(\\theta^\{\\prime\}\)\\right\\\|\_\{2\}\\leq L\\left\\\|\\theta\-\\theta^\{\\prime\}\\right\\\|\_\{2\}\.\(22\)

###### Assumption 2\(Strong Convexity\)\.

The global lossℒ​\(θ\)\\mathcal\{L\}\(\\theta\)isμ\\mu\-strongly convex: for allθ,θ′∈ℝd\\theta,\\theta^\{\\prime\}\\in\\mathbb\{R\}^\{d\},

ℒ​\(θ′\)≥ℒ​\(θ\)\+∇ℒ​\(θ\)⊤​\(θ′−θ\)\+μ2​‖θ′−θ‖22\.\\mathcal\{L\}\(\\theta^\{\\prime\}\)\\geq\\mathcal\{L\}\(\\theta\)\+\\nabla\\mathcal\{L\}\(\\theta\)^\{\\top\}\(\\theta^\{\\prime\}\-\\theta\)\+\\frac\{\\mu\}\{2\}\\left\\\|\\theta^\{\\prime\}\-\\theta\\right\\\|\_\{2\}^\{2\}\.\(23\)

###### Assumption 3\(Bounded Hessian Spectrum\)\.

For all clientsiiandθ\\thetanear the optimum, the local Hessian satisfies

μ​I⪯∇2ℓi​\(θ;𝒟i\)⪯L​I\.\\mu I\\preceq\\nabla^\{2\}\\ell\_\{i\}\(\\theta;\\mathcal\{D\}\_\{i\}\)\\preceq LI\.\(24\)

### A\.5Proof of CG Convergence Rate \(Theorem 1\)

Restatement of Theorem 1\.

###### Theorem 3\(CG Convergence Rate\)\.

LetH∈ℝd×dH\\in\\mathbb\{R\}^\{d\\times d\}be symmetric positive definite with condition numberκ=λmax​\(H\)/λmin​\(H\)\\kappa=\\lambda\_\{\\max\}\(H\)/\\lambda\_\{\\min\}\(H\)\. Letv∗=H−1​gv^\{\*\}=H^\{\-1\}gandvkv\_\{k\}be the CG approximation afterkkiterations\. Then

‖vk−v∗‖H≤2​\(κ−1κ\+1\)k​‖v∗‖H\.\\left\\\|v\_\{k\}\-v^\{\*\}\\right\\\|\_\{H\}\\leq 2\\left\(\\frac\{\\sqrt\{\\kappa\}\-1\}\{\\sqrt\{\\kappa\}\+1\}\\right\)^\{k\}\\left\\\|v^\{\*\}\\right\\\|\_\{H\}\.\(25\)

### A\.6Convergence of Conjugate Gradient

###### Theorem 4\(CG Convergence Rate\)\.

LetH∈ℝd×dH\\in\\mathbb\{R\}^\{d\\times d\}be symmetric positive definite with condition numberκ=λmax​\(H\)/λmin​\(H\)\\kappa=\\lambda\_\{\\max\}\(H\)/\\lambda\_\{\\min\}\(H\)\. Letv∗=H−1​gv^\{\*\}=H^\{\-1\}gandvkv\_\{k\}be the CG approximation afterkkiterations\. Then

‖vk−v∗‖H≤2​\(κ−1κ\+1\)k​‖v∗‖H\.\\left\\\|v\_\{k\}\-v^\{\*\}\\right\\\|\_\{H\}\\leq 2\\left\(\\frac\{\\sqrt\{\\kappa\}\-1\}\{\\sqrt\{\\kappa\}\+1\}\\right\)^\{k\}\\left\\\|v^\{\*\}\\right\\\|\_\{H\}\.\(26\)

###### Proof\.

This is a classical result in numerical linear algebraShewchuk \[[1994](https://arxiv.org/html/2605.20341#bib.bib24)\]\. CG minimizes the quadratic form12​v⊤​H​v−g⊤​v\\frac\{1\}\{2\}v^\{\\top\}Hv\-g^\{\\top\}vover𝒦k​\(H,g\)\\mathcal\{K\}\_\{k\}\(H,g\)\. The error in theHH\-norm decreases geometrically with rate determined byκ\\kappa\. Forκ=1\\kappa=1\(perfectly conditioned\), CG converges in one iteration\. Forκ=100\\kappa=100, the error decreases by approximately0\.82k0\.82^\{k\}\. ∎

###### Corollary 5\.

To achieve‖vk−v∗‖H≤ε​‖v∗‖H\\left\\\|v\_\{k\}\-v^\{\*\}\\right\\\|\_\{H\}\\leq\\varepsilon\\left\\\|v^\{\*\}\\right\\\|\_\{H\}, CG requires

k=O​\(κ​log⁡\(1ε\)\)k=O\\left\(\\sqrt\{\\kappa\}\\log\\left\(\\frac\{1\}\{\\varepsilon\}\\right\)\\right\)\(27\)iterations\.

This shows that for moderately conditioned problems \(κ∼100\\kappa\\sim 100\),k∼10k\\sim 10iterations suffice forε=0\.01\\varepsilon=0\.01, yielding considerable speedup over exact inversion\.

### A\.7Approximation Quality

###### Theorem 6\(Total Approximation Error\)\.

Under Assumptions[1](https://arxiv.org/html/2605.20341#Thmassumption1)–[3](https://arxiv.org/html/2605.20341#Thmassumption3), the parameter difference between HF\-KCU and exact retraining satisfies

‖θu−θr‖2≤C​\(εCG\+εHVP\+εagg\),\\left\\\|\\theta\_\{u\}\-\\theta\_\{r\}\\right\\\|\_\{2\}\\leq C\\left\(\\varepsilon\_\{\\text\{CG\}\}\+\\varepsilon\_\{\\text\{HVP\}\}\+\\varepsilon\_\{\\text\{agg\}\}\\right\),\(28\)whereCCdepends onLL,μ\\mu, and‖∇ℓ​\(θ;𝒟u\)‖2\\left\\\|\\nabla\\ell\(\\theta;\\mathcal\{D\}\_\{u\}\)\\right\\\|\_\{2\}, and

εCG\\displaystyle\\varepsilon\_\{\\text\{CG\}\}=O​\(\(κ−1κ\+1\)k\),\\displaystyle=O\\left\(\\left\(\\frac\{\\sqrt\{\\kappa\}\-1\}\{\\sqrt\{\\kappa\}\+1\}\\right\)^\{k\}\\right\),\(29\)εHVP\\displaystyle\\varepsilon\_\{\\text\{HVP\}\}=O​\(δ\),\\displaystyle=O\(\\delta\),\(30\)εagg\\displaystyle\\varepsilon\_\{\\text\{agg\}\}=O​\(1/N\),\\displaystyle=O\(1/\\sqrt\{N\}\),\(31\)are errors from CG approximation, finite\-difference HVP computation, and federated aggregation respectively\.

###### Proof sketch\.

The exact parameter change from removing𝒟u\\mathcal\{D\}\_\{u\}isΔ​θ∗=−H−1​∇ℓ​\(θ;𝒟u\)\\Delta\\theta^\{\*\}=\-H^\{\-1\}\\nabla\\ell\(\\theta;\\mathcal\{D\}\_\{u\}\)by influence theory\. HF\-KCU computesΔ​θ=−vk\\Delta\\theta=\-v\_\{k\}wherevkv\_\{k\}approximatesH−1​gH^\{\-1\}gwith errorεCG\\varepsilon\_\{\\text\{CG\}\}from Theorem[4](https://arxiv.org/html/2605.20341#Thmtheorem4)\.

Automatic differentiation introduces numerical errorεHVP=O​\(δ\)\\varepsilon\_\{\\text\{HVP\}\}=O\(\\delta\)whereδ\\deltais the finite\-difference step size\. Federated aggregation overNNclients introduces sampling errorεagg=O​\(1/N\)\\varepsilon\_\{\\text\{agg\}\}=O\(1/\\sqrt\{N\}\)by standard concentration inequalities\.

By triangle inequality and Lipschitz continuity of the gradient \(Assumption[1](https://arxiv.org/html/2605.20341#Thmassumption1)\), these errors combine additively with constantC=O​\(L/μ\)C=O\(L/\\mu\)\. ∎

### A\.8Causal Isolation

###### Proposition 7\(Zero Update for Unaffected Clients\)\.

If clientjjsatisfies𝒟u∩𝒟j=∅\\mathcal\{D\}\_\{u\}\\cap\\mathcal\{D\}\_\{j\}=\\emptyset, thenΔ​θj=0\\Delta\\theta\_\{j\}=0exactly\.

###### Proof\.

By definition of causal weights \(Eq\. \([7](https://arxiv.org/html/2605.20341#S3.E7)\)\),αj=0\\alpha\_\{j\}=0when𝒟u∩𝒟j=∅\\mathcal\{D\}\_\{u\}\\cap\\mathcal\{D\}\_\{j\}=\\emptyset\. ThereforeΔ​θj=−αj⋅vk,j=0\\Delta\\theta\_\{j\}=\-\\alpha\_\{j\}\\cdot v\_\{k,j\}=0regardless ofvk,jv\_\{k,j\}\. ∎

This property distinguishes HF\-KCU from naive influence\-based methods that would apply a global update to all clients\. Causal isolation ensures that unaffected clients’ parameters remain unchanged, preserving their learned representations\.

### A\.9Causal Deletion as Intervention

We formalize client\-level unlearning as a*deletion intervention*on the federated training process\. Let𝒟=\{D1,…,DN\}\\mathcal\{D\}=\\\{D\_\{1\},\\dots,D\_\{N\}\\\}denote the collection of client datasets, and let the global objective be

ℒ​\(θ;𝒟\)=∑i=1Nwi​ℒi​\(θ;Di\),\\mathcal\{L\}\(\\theta;\\mathcal\{D\}\)=\\sum\_\{i=1\}^\{N\}w\_\{i\}\\,\\mathcal\{L\}\_\{i\}\(\\theta;D\_\{i\}\),\(32\)wherewiw\_\{i\}is the aggregation weight of clientii\. The trained model is

θ⋆=arg⁡minθ⁡ℒ​\(θ;𝒟\)\.\\theta^\{\\star\}=\\arg\\min\_\{\\theta\}\\mathcal\{L\}\(\\theta;\\mathcal\{D\}\)\.\(33\)
Suppose clientjjrequests deletion\. We model this request as an intervention

do​\(Dj=∅\),\\mathrm\{do\}\(D\_\{j\}=\\varnothing\),\(34\)which removes the contribution of clientjjfrom the training objective\. The corresponding post\-intervention optimum is

θ−j⋆=arg⁡minθ​∑i≠jwi​ℒi​\(θ;Di\)\.\\theta^\{\\star\}\_\{\-j\}=\\arg\\min\_\{\\theta\}\\sum\_\{i\\neq j\}w\_\{i\}\\,\\mathcal\{L\}\_\{i\}\(\\theta;D\_\{i\}\)\.\(35\)The goal of federated unlearning is to transformθ⋆\\theta^\{\\star\}into a modelθ~−j\\tilde\{\\theta\}\_\{\-j\}such thatθ~−j≈θ−j⋆\\tilde\{\\theta\}\_\{\-j\}\\approx\\theta^\{\\star\}\_\{\-j\}, while avoiding full retraining\.

Under standard smoothness assumptions, the effect of the deletion intervention can be approximated locally by influence reversal\. Let

H=∇θ2ℒ​\(θ⋆;𝒟\)H=\\nabla\_\{\\theta\}^\{2\}\\mathcal\{L\}\(\\theta^\{\\star\};\\mathcal\{D\}\)\(36\)be the Hessian of the pre\-deletion objective\. Then a first\-order approximation to the intervention effect is

Δj=θ−j⋆−θ⋆≈−H−1​wj​∇θℒj​\(θ⋆;Dj\)\.\\Delta\_\{j\}\\;=\\;\\theta^\{\\star\}\_\{\-j\}\-\\theta^\{\\star\}\\;\\approx\\;\-H^\{\-1\}\\,w\_\{j\}\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{j\}\(\\theta^\{\\star\};D\_\{j\}\)\.\(37\)This expression makes precise the causal interpretation of unlearning: deleting clientjjcorresponds to removing its contribution and propagating that intervention through the curvature of the global objective\.

HF\-KCU approximates this intervention efficiently without explicitly formingH−1H^\{\-1\}\. Specifically, we solve the linear system

H​vj=wj​∇θℒj​\(θ⋆;Dj\)Hv\_\{j\}=w\_\{j\}\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{j\}\(\\theta^\{\\star\};D\_\{j\}\)\(38\)in a Krylov subspace using conjugate gradients, and setΔj≈−vj\\Delta\_\{j\}\\approx\-v\_\{j\}\. This yields an efficient approximation to the post\-intervention model update while preserving the semantics of deletion as a causal intervention on the federated objective\.

### A\.10Privacy Guarantees

Following Guo et al\.Guoet al\.\[[2020](https://arxiv.org/html/2605.20341#bib.bib7)\], we say an unlearning method achieves*certified removal*if the output distribution is indistinguishable from retraining without the deleted data\.

###### Theorem 8\(Approximate Certified Removal\)\.

Under Assumptions[1](https://arxiv.org/html/2605.20341#Thmassumption1)–[3](https://arxiv.org/html/2605.20341#Thmassumption3), HF\-KCU satisfies

DTV​\(PHF\-KCU,Pretrain\)≤C′⋅εtotal,D\_\{\\text\{TV\}\}\(P\_\{\\text\{HF\-KCU\}\},P\_\{\\text\{retrain\}\}\)\\leq C^\{\\prime\}\\cdot\\varepsilon\_\{\\text\{total\}\},\(39\)whereDTVD\_\{\\text\{TV\}\}is total variation distance,εtotal=εCG\+εHVP\+εagg\\varepsilon\_\{\\text\{total\}\}=\\varepsilon\_\{\\text\{CG\}\}\+\\varepsilon\_\{\\text\{HVP\}\}\+\\varepsilon\_\{\\text\{agg\}\}, andC′C^\{\\prime\}depends on problem parameters\.

###### Proof sketch\.

By Theorem[6](https://arxiv.org/html/2605.20341#Thmtheorem6),‖θu−θr‖2≤C​εtotal\\left\\\|\\theta\_\{u\}\-\\theta\_\{r\}\\right\\\|\_\{2\}\\leq C\\varepsilon\_\{\\text\{total\}\}\. For smooth losses \(Assumption[1](https://arxiv.org/html/2605.20341#Thmassumption1)\), prediction distributions change continuously with parameters\. Specifically, for any inputxx, the prediction difference satisfies‖fθu​\(x\)−fθr​\(x\)‖2≤L​‖θu−θr‖2\\left\\\|f\_\{\\theta\_\{u\}\}\(x\)\-f\_\{\\theta\_\{r\}\}\(x\)\\right\\\|\_\{2\}\\leq L\\left\\\|\\theta\_\{u\}\-\\theta\_\{r\}\\right\\\|\_\{2\}by Lipschitz continuity\. Integrating over the data distribution and applying Pinsker’s inequality yields the stated bound\. ∎

This theorem shows that askkincreases \(reducingεCG\\varepsilon\_\{\\text\{CG\}\}\), HF\-KCU approaches certified removal\. In practice, we validate this through membership inference attacks \(Section[5](https://arxiv.org/html/2605.20341#S5)\)\.

### A\.11Functional and Privacy Equivalence to Retraining

A central goal of federated unlearning is not merely to modify parameters, but to recover the*behavior*and*privacy profile*of a model that would have been obtained by full retraining after deletion\. We therefore evaluate unlearning quality relative to the retrained oracle from two complementary perspectives: functional equivalence and privacy equivalence\.

#### Functional equivalence\.

Parameter\-space distances can be misleading in overparameterized models, where functionally similar predictors may differ substantially in weight space\. We therefore measure the agreement between the unlearned modelfunlf\_\{\\mathrm\{unl\}\}and the retrained modelfretf\_\{\\mathrm\{ret\}\}directly in output space\. In particular, we report prediction agreement

FA=𝔼x∼𝒟test​\[𝟏​\(funl​\(x\)=fret​\(x\)\)\],\\mathrm\{FA\}=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{\\mathrm\{test\}\}\}\\left\[\\mathbf\{1\}\\big\(f\_\{\\mathrm\{unl\}\}\(x\)=f\_\{\\mathrm\{ret\}\}\(x\)\\big\)\\right\],\(40\)which captures the fraction of test examples on which the two models produce identical predictions\. We additionally report a soft discrepancy based on predictive distributions,

KLfunc=𝔼x∼𝒟test\[DKL\(pret\(⋅∣x\)∥punl\(⋅∣x\)\)\],\\mathrm\{KL\}\_\{\\mathrm\{func\}\}=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{\\mathrm\{test\}\}\}\\left\[D\_\{\\mathrm\{KL\}\}\\\!\\left\(p\_\{\\mathrm\{ret\}\}\(\\cdot\\mid x\)\\;\\\|\\;p\_\{\\mathrm\{unl\}\}\(\\cdot\\mid x\)\\right\)\\right\],\(41\)where smaller values indicate closer functional alignment to retraining\. Together, these metrics assess whether the unlearned model behaves like the retrained oracle rather than merely occupying a nearby point in parameter space\.

#### Privacy equivalence\.

To evaluate whether deletion removes information about the forgotten client to the same extent as retraining, we compare membership inference attack \(MIA\) success against the unlearned and retrained models\. LetMIA​\(f,S\)\\mathrm\{MIA\}\(f,S\)denote the attack success rate of a fixed inference procedure on a sample setSS\. We focus in particular on the deleted client datasetDjD\_\{j\}, and define the privacy gap

Δpriv=\|MIA​\(funl,Dj\)−MIA​\(fret,Dj\)\|\.\\Delta\_\{\\mathrm\{priv\}\}=\\left\|\\mathrm\{MIA\}\(f\_\{\\mathrm\{unl\}\},D\_\{j\}\)\-\\mathrm\{MIA\}\(f\_\{\\mathrm\{ret\}\},D\_\{j\}\)\\right\|\.\(42\)A smallΔpriv\\Delta\_\{\\mathrm\{priv\}\}indicates that the unlearned model exposes membership information at nearly the same level as the retrained oracle, which is the desired outcome for privacy\-preserving deletion\. We also report MIA performance on retained clients and non\-member test points to verify that privacy improvement on deleted data does not arise from degenerate behavior\.

#### Interpretation\.

These metrics operationalize a stricter notion of unlearning quality: a successful method should be*functionally equivalent*to retraining in prediction behavior and*privacy equivalent*to retraining in membership leakage\. This evaluation is especially important in federated settings, where approximate unlearning methods may achieve high utility while still retaining residual information about deleted client contributions\.

### A\.12Discussion of Theoretical Limitations

Our analysis assumes convexity \(Assumption[2](https://arxiv.org/html/2605.20341#Thmassumption2)\), which does not hold globally for neural networks\. However, influence function approximations are known to work well empirically even for non\-convex lossesKoh and Liang \[[2017](https://arxiv.org/html/2605.20341#bib.bib12)\], particularly near local optima where the loss surface is approximately quadratic\.

The condition numberκ\\kappacritically affects convergence rate\. For poorly conditioned problems, damping \(Section[A\.2](https://arxiv.org/html/2605.20341#A1.SS2)\) artificially improvesκ\\kappaat the cost of introducing bias\. The trade\-off between approximation speed and accuracy is inherent to iterative methods\.

### A\.13A Methodological Critique of Causal Faithfulness in Edge Cases

Our results highlight a fundamental weakness in the Causal Faithfulness \(CF\) metric, which we frame as a contribution to the evaluation methodology for machine unlearning\. The metric is defined as a ratio of accuracy differences, which makes it highly susceptible to instability in common and critical edge cases\.

Edge Case 1: Denominator Approaching Zero\.The most significant failure mode occurs when the accuracy difference between the original and retrained models,\|Acctrained−Accretrain\|\|\\text\{Acc\}\_\{\\text\{trained\}\}\-\\text\{Acc\}\_\{\\text\{retrain\}\}\|, is small\.

- •Instability:As the denominator approaches zero, any small, potentially stochastic, fluctuation in the numerator \(Acctrained−Accunlearn\\text\{Acc\}\_\{\\text\{trained\}\}\-\\text\{Acc\}\_\{\\text\{unlearn\}\}\) is massively amplified\. This leads to extremely large positive or negative CF values that are not representative of the unlearning quality\. Our own results demonstrate this: the denominator is a mere\|71\.76−70\.79\|=0\.97%\|71\.76\-70\.79\|=0\.97\\%\. This small value is the primary reason for the volatile CF scores observed in Table[1](https://arxiv.org/html/2605.20341#S5.T1)\.
- •Undefined Metric:In the plausible scenario where retraining coincidentally yields the exact same accuracy as the original model \(Acctrained=Accretrain\\text\{Acc\}\_\{\\text\{trained\}\}=\\text\{Acc\}\_\{\\text\{retrain\}\}\), the CF metric becomes undefined due to division by zero\.

Edge Case 2: Misleading Sign\.The sign of the numerator and denominator can be inconsistent\. For instance, if unlearning happens to slightlyimproveaccuracy over the original model \(e\.g\., by removing noisy data points\), the numerator becomes negative\. If retraining also improves accuracy \(denominator is negative\), the CF is positive, which is misleading\.

Proposed Alternative: Functional Equivalence\.These edge cases demonstrate that single\-scalar performance metrics like accuracy are too brittle to form the basis of a reliable unlearning evaluation metric\. We strongly advocate for functional equivalence metrics, such as the \*\*Kullback\-Leibler \(KL\) divergence between the models’ output logits\*\*\. The KL divergence is not a ratio, is always non\-negative, and provides a granular, sample\-wise comparison of model behavior\. It robustly measures if the unlearned model functions like the retrained model, avoiding all the edge cases associated with CF\.

## Appendix BExtended Experimental Results

We analyze the contribution of each component in HF\-KCU\.

Component Ablations\.Table[4](https://arxiv.org/html/2605.20341#A2.T4)shows results when removing key components\. Removing causal weighting \("w/o causal"\) increases parameter gap from 0\.032 to 0\.089—nearly a 3×\\timesincrease\. This supports our theoretical claim that causal weighting is necessary for update localization: without it, unlearning updates bleed into all clients rather than remaining localized to those holding𝒟u\\mathcal\{D\}\_\{u\}, causing spurious parameter drift\. The 3×\\timesparameter gap increase shows that causal weighting prevents corruption of model components learned from unaffected data distributions\. Removing adaptive scaling \("w/o scaling"\) causes training instability, reducing accuracy to 65\.23%\. Both components are necessary for stable, accurate unlearning\.

Table 4:Component ablation study\. Each row removes one component from HF\-KCU\.Sensitivity to Damping Parameterλ\\lambda\.Table[5](https://arxiv.org/html/2605.20341#A2.T5)evaluates the influence of the damping termλ\\lambda, which fundamentally governs the trade\-off between approximation bias and the numerical stability of the Conjugate Gradient \(CG\) solver\. Theoretically, the addition ofλ​I\\lambda Ishifts the spectrum of the Hessian proxy, ensuring the minimum eigenvalue remains strictly positive\. This regularization is critical for guaranteeing CG convergence, particularly in regions where the empirical curvature is ill\-conditioned or corrupted by adversarial contributions\.

Empirically, we observe that excessive damping \(λ=0\.100\\lambda=0\.100\) over\-regularizes the system; the injected bias overwhelms the true curvature information, yielding suboptimal unlearning updates\. Conversely, while minimal damping \(λ=0\.001\\lambda=0\.001\) initially appears attractive due to a dramatic reduction in solver iterations \(achieving a209\.74×209\.74\\timesspeedup\), it critically undermines numerical stability\. In our broader experiments, settingλ<0\.005\\lambda<0\.005frequently allowed near\-zero or negative eigenvalues to persist in the proxy matrix\. Under these conditions, the CG residual fails to decrease monotonically, causing the solver to diverge from the true inverse Hessian\-vector product,H−1​gH^\{\-1\}g\. This instability occasionally produced explosive parameter updates requiring the adaptive scaling and clipping mechanisms described in Section[A\.2](https://arxiv.org/html/2605.20341#A1.SS2)\. Consequently, intermediate values \(e\.g\.,λ=0\.010\\lambda=0\.010\) provide the necessary stabilization to guarantee robust, monotonic CG convergence without artificially smoothing the curvature profile required for high\-fidelity unlearning\.

Table 5:Effect of damping parameterλ\\lambdaon performance\.Table[6](https://arxiv.org/html/2605.20341#A2.T6)illustrates the impact of data heterogeneity, controlled via the Dirichlet parameterα\\alpha, on the efficacy and efficiency of the HF\-KCU algorithm\. As the data distribution transitions from highly non\-IID \(α=0\.1\\alpha=0\.1\) to mildly non\-IID \(α=1\.0\\alpha=1\.0\), the unlearned model maintains strict functional equivalence, with post\-unlearning accuracy closely tracking the gold\-standard retrained model\. Notably, the parameter gap narrows slightly as the data becomes more homogeneous, reflecting the reduced parameter divergence when client updates are less specialized\. Furthermore, improved data homogeneity yields substantial computational gains: the speedup relative to full retraining increases from49\.08×49\.08\\timesatα=0\.1\\alpha=0\.1to76\.48×76\.48\\timesatα=1\.0\\alpha=1\.0\. This acceleration is theoretically consistent with Krylov subspace methods, where a more uniform data distribution results in a better\-conditioned empirical Fisher Information Matrix \(FIM\), thereby requiring fewer Lanczos iterations to accurately approximate the inverse Hessian\-vector product\.

Table 6:Effect of data heterogeneity \(Dirichletα\\alpha\) on HF\-KCU performance\.02020404010−510^\{\-5\}10−310^\{\-3\}10−110^\{\-1\}ε=10−4\\varepsilon=10^\{\-4\}convergedCG IterationRelative Residual‖𝐫t‖/‖𝐫0‖\\\|\\mathbf\{r\}\_\{t\}\\\|/\\\|\\mathbf\{r\}\_\{0\}\\\|\(a\) Conjugate Gradient ConvergenceTheoretical boundHF\-KCU \(client 1\)HF\-KCU \(client 2\)10110^\{1\}10210^\{2\}10310^\{3\}10410^\{4\}10510^\{5\}10110^\{1\}10210^\{2\}Hessian Condition Numberκ​\(𝐇\)\\kappa\(\\mathbf\{H\}\)Iterations toε=10−4\\varepsilon=10^\{\-4\}\(b\) Condition Number SensitivityO​\(κ\)O\(\\sqrt\{\\kappa\}\)theoryHF\-KCU empirical\+ preconditioning### B\.1Scalability Analysis

We evaluate HF\-KCU’s scalability by varying the number of clients and model size\.

Number of Clients\.To evaluate the scalability of our approach, we analyzed the performance of HF\-KCU across varying federated network sizes \(1010,100100, and200200clients\), as summarized in Table[7](https://arxiv.org/html/2605.20341#A2.T7)\. As the number of clients increases, the marginal contribution of any single client to the global model diminishes, indicating less parameter divergence when unlearning a smaller fraction of the global dataset\. Crucially, the computational efficiency of HF\-KCU scales exceptionally well with network size\. The speedup factor jumps from76\.48×76\.48\\timesin the1010\-client scenario to approximately110×110\\timesfor100100and200200clients\. This demonstrates that as full retraining becomes increasingly prohibitive in massive cross\-device federated networks, HF\-KCU provides an even more pronounced efficiency advantage while maintaining close functional equivalence to the retrained baseline\. Furthermore, the Causal Faithfulness \(CF\) scores across all three scales hovered infinitesimally close to zero \(ranging from−0\.0037\-0\.0037to0\.00030\.0003\), serving as further empirical validation of our earlier methodological critique regarding the metric’s instability in highly equivalent regimes\.

Table 7:Scalability of HF\-KCU across varying numbers of federated clients\. Accuracy metrics represent the global model performance\.Model Size and Wall\-Clock Time\.We test HF\-KCU on models ranging from 188K to 11M parameters \(Table[8](https://arxiv.org/html/2605.20341#A2.T8)\)\. Unlearning time scales approximately asO​\(k​d\)O\(kd\)whereddis the number of parameters, consistent with our theoretical complexity analysis\. Even for the 11M parameter model, HF\-KCU achieves 35×\\timesspeedup over retraining\. Each CG iteration requires one Hessian\-vector product \(HVP\), which costs roughly twice a gradient computation via automatic differentiation\. Withk=10k=10, HF\-KCU therefore requires approximately 20 gradient\-equivalent passes\. For the 188K\-parameter CNN this completes in under 25 seconds; for the 11M\-parameter ResNet\-18 the wall\-clock time rises to roughly 180 seconds, compared to over 6,300 seconds for full retraining\. Scaling further to models withd\>108d\>10^\{8\}parameters would require either distributing the HVP computation or adopting memory\-efficient CG variants\.

Table 8:Wall\-clock time scaling with model size \(k=10k=10CG iterations\)\.

## Appendix CEvaluation Methodology: Functional vs\. Parameter\-Space Metrics

#### The instability of parameter\-space metrics\.

Traditional unlearning evaluations often measure parameter\-space distance‖𝜽unlearn−𝜽retrain‖2\\\|\\boldsymbol\{\\theta\}\_\{\\text\{unlearn\}\}\-\\boldsymbol\{\\theta\}\_\{\\text\{retrain\}\}\\\|\_\{2\}or cosine similarity between the unlearned model and a model retrained from scratch\. However, neural networks exhibit*permutation symmetry*: any permutation of hidden units within a layer yields functionally identical models with arbitrarily large parameter distance\[Entezariet al\.,[2022](https://arxiv.org/html/2605.20341#bib.bib31), Ainsworthet al\.,[2023](https://arxiv.org/html/2605.20341#bib.bib32)\]\. Furthermore, stochastic optimization can converge to different basins of the loss landscape that are functionally equivalent but geometrically distant in parameter space\[Fortet al\.,[2019](https://arxiv.org/html/2605.20341#bib.bib30)\]\.

As a result, parameter\-space metrics can be misleading: two models may have largeL2L\_\{2\}distance yet produce nearly identical predictions, or conversely, small parameter distance may not guarantee functional equivalence if the models lie in different loss basins\.

#### Functional equivalence as the principled alternative\.

We propose evaluating unlearning quality via*functional equivalence metrics*that directly measure agreement between model outputs rather than parameter configurations\. Specifically, we adopt:

1. 1\.Output KL divergence:For classification tasks, letpretrain\(⋅∣𝐱\)p\_\{\\text\{retrain\}\}\(\\cdot\\mid\\mathbf\{x\}\)andpunlearn\(⋅∣𝐱\)p\_\{\\text\{unlearn\}\}\(\\cdot\\mid\\mathbf\{x\}\)denote the softmax output distributions of the retrained and unlearned models on input𝐱\\mathbf\{x\}\. We measure KLoutput=𝔼𝐱∼𝒟test\[DKL\(pretrain\(⋅∣𝐱\)∥punlearn\(⋅∣𝐱\)\)\]\.\\text\{KL\}\_\{\\text\{output\}\}=\\mathbb\{E\}\_\{\\mathbf\{x\}\\sim\\mathcal\{D\}\_\{\\text\{test\}\}\}\\left\[D\_\{\\text\{KL\}\}\\bigl\(p\_\{\\text\{retrain\}\}\(\\cdot\\mid\\mathbf\{x\}\)\\,\\\|\\,p\_\{\\text\{unlearn\}\}\(\\cdot\\mid\\mathbf\{x\}\)\\bigr\)\\right\]\.
2. 2\.Logit MSE:For regression or when logits are more interpretable, we measure MSElogit=𝔼𝐱∼𝒟test​\[‖𝐳retrain​\(𝐱\)−𝐳unlearn​\(𝐱\)‖22\],\\text\{MSE\}\_\{\\text\{logit\}\}=\\mathbb\{E\}\_\{\\mathbf\{x\}\\sim\\mathcal\{D\}\_\{\\text\{test\}\}\}\\left\[\\\|\\mathbf\{z\}\_\{\\text\{retrain\}\}\(\\mathbf\{x\}\)\-\\mathbf\{z\}\_\{\\text\{unlearn\}\}\(\\mathbf\{x\}\)\\\|\_\{2\}^\{2\}\\right\],where𝐳​\(𝐱\)\\mathbf\{z\}\(\\mathbf\{x\}\)denotes the pre\-softmax logits\.
3. 3\.Test accuracy gap:The absolute difference in test accuracy between retrained and unlearned models,\|Accretrain−Accunlearn\|\|\\text\{Acc\}\_\{\\text\{retrain\}\}\-\\text\{Acc\}\_\{\\text\{unlearn\}\}\|\.

These metrics are invariant to permutation symmetry and basin geometry, focusing instead on whether the unlearned model behaves like the retrained model on held\-out data\.

#### Empirical results on functional metrics\.

Table[9](https://arxiv.org/html/2605.20341#A3.T9)reports these metrics for HF\-KCU on CIFAR\-10\. The output KL divergence is 0\.0023 nats, indicating near\-identical prediction distributions\. Logit MSE is 0\.087, and the test accuracy gap is 0\.37%, all confirming functional equivalence\. Combined with MIA success rate of 0\.499, these metrics provide strong evidence that HF\-KCU successfully removes the target data’s influence without relying solely on parameter\-space distance\.

Table 9:Functional equivalence metrics for HF\-KCU vs\. retrained model on CIFAR\-10\.

## Appendix DLimitations and Future Directions

CF Metric Instability\.As discussed, the CF metric exhibits high variance when accuracy differences are small\. The denominator\|Acctrained−Accretrain\|\|\\text\{Acc\}\_\{\\text\{trained\}\}\-\\text\{Acc\}\_\{\\text\{retrain\}\}\|in our experiments is 0\.97%, causing sensitivity to small fluctuations\. Alternative metrics such as prediction agreement rate, KL divergence between output distributions, or functional distance measures may provide more stable faithfulness assessment\.

Approximation Quality\.HF\-KCU usesk=10k=10CG iterations to approximate the inverse Hessian\-vector product\. While this provides good empirical results, tighter theoretical bounds on the approximation error as a function ofkkwould strengthen the method’s guarantees\.

Sequential Unlearning\.We evaluated single unlearning requests\. Repeated unlearning may accumulate approximation errors\. Studying the stability of HF\-KCU under multiple sequential deletions is important for production systems\.

Heterogeneous Systems\.Our experiments assume homogeneous client capabilities\. Real federated systems exhibit device heterogeneity in compute, memory, and network\. Adaptive strategies that account for client\-specific constraints warrant investigation\.

Beyond Vision\.We focused on image classification with CNNs\. Extending HF\-KCU to transformers, large language models, and graph neural networks presents challenges due to different loss landscapes and parameter scales\.

Memory Constraints for Ultra\-Large Models\.While our method reduces complexity fromO​\(d3\)O\(d^\{3\}\)toO​\(k​d\)O\(kd\)and avoids forming the Hessian matrix, storing the Krylov subspace vectors during CG iterations poses memory constraints for ultra\-large architectures \(e\.g\., modern LLMs with billions of parameters\)\. Each CG iteration requires storing vectors of sizedd, and maintainingkksuch vectors requiresO​\(k​d\)O\(kd\)memory\. For models withd∼109d\\sim 10^\{9\}parameters, even withk=10k=10, this can exceed available GPU memory\. Future work could explore memory\-efficient variants such as truncated Krylov methods or low\-rank approximations of the influence function\.

The work also highlights limitations of existing faithfulness metrics\. The CF metric becomes unstable when accuracy differences are small, as observed in our experiments where\|Acctrained−Accretrain\|<1%\|\\text\{Acc\}\_\{\\text\{trained\}\}\-\\text\{Acc\}\_\{\\text\{retrain\}\}\|<1\\%\. This suggests the need for alternative metrics that remain reliable across different accuracy regimes, such as prediction agreement rates or functional distance measures\.Our adversarial analysis is intentionally limited to bounded perturbation settings\. In particular, we do not claim robustness against fully unconstrained adaptive poisoning attacks, strategic collusion across many clients, or minimax\-optimal adversaries that explicitly optimize against the unlearning mechanism\. Extending the framework to such settings would require a stronger game\-theoretic or robust\-optimization formulation together with dedicated attack evaluations\.

HF\-KCU shows that federated systems can support practical data deletion without prohibitive computational costs, enabling compliance with privacy regulations while maintaining model utility\.