H2DF: Calibrated Target Bands for Controllable Language-Model Unlearning

# Calibrated Target Bands for Controllable Language-Model Unlearning
Source: [https://arxiv.org/html/2610.26864](https://arxiv.org/html/2610.26864)
###### Abstract

Gradient ascent \(GA\) is a simple language\-model unlearning baseline, but it continues increasing forget loss after useful forgetting has occurred\. This unbounded pressure can damage retained utility and make forgotten examples abnormally high\-loss\. We introduceH2DF, a calibrated target\-band objective that applies GA\-like pressure below a data\-dependent lower target and suppresses, or reverses, that pressure after the target region is reached\. The targets are estimated from matched non\-member examples before training, so the base objective requires one forget\-batch forward/backward pass per optimization step; our reported TOFU variant adds sparse retain replay every tenth step\. We instantiate a one\-sided objective for TOFU question answering and a two\-sided objective for MUSE\-News language modeling\. Across three TOFU seeds,H2DFobtains a forget NLL of1\.528±0\.0491\.528\\pm 0\.049and retain NLL of1\.165±0\.0471\.165\\pm 0\.047, matching GradDiff while using retain replay only every tenth step and showing substantially lower variability than GA\. On MUSE\-News,H2DFimproves retention over GA and NPO and is competitive with SimNPO, but does not dominate it\. Ablations show that the upper target reduces overshoot and that validation\-based checkpoint selection is important\. These results support calibrated target bands as a practical mechanism for controlling forgetting strength, while also exposing sensitivity to random seed and calibration quality\.[Source Code and Models](https://github.com/Shdakudow/H2DF-Calibrated-Target-Bands-for-Controllable-Language-Model-Unlearning)

aia\_\{i\}bib\_\{i\}−0\.5\-0\.500\.50\.511target bandForget scoresi​\(θ\)s\_\{i\}\(\\theta\)Pressure to increasesis\_\{i\}GAPreferenceH2DFFigure 1:Conceptual forget pressure\. GA remains persistent; preference\-style objectives attenuate pressure implicitly\[[24](https://arxiv.org/html/2610.26864#bib.bib6),[6](https://arxiv.org/html/2610.26864#bib.bib5)\];H2DFpushes belowaia\_\{i\}, saturates in the calibrated band, and can restore abovebib\_\{i\}\. Curves use illustrative parameters\.## 1Introduction

Large language models may memorize private, copyrighted, or otherwise undesirable training data\. Machine unlearning seeks to remove the influence of selected data without retraining from scratch\[[2](https://arxiv.org/html/2610.26864#bib.bib1)\]\. For large models, a common baseline is gradient ascent on the forget set\[[23](https://arxiv.org/html/2610.26864#bib.bib2)\]\. Given a forget batchBfB\_\{f\}and per\-example negative log\-likelihood \(NLL\)ℓi​\(θ\)\\ell\_\{i\}\(\\theta\), GA minimizes

ℒGA​\(θ\)=−1\|Bf\|​∑i∈Bfℓi​\(θ\)\.\\mathcal\{L\}\_\{\\mathrm\{GA\}\}\(\\theta\)=\-\\frac\{1\}\{\|B\_\{f\}\|\}\\sum\_\{i\\in B\_\{f\}\}\\ell\_\{i\}\(\\theta\)\.\(1\)GA is inexpensive, but it treats forgetting as an infinite direction: the objective continues to increase NLL even after the model no longer reproduces the targeted content\. This can cause catastrophic utility loss and abnormal high\-loss artifacts\. Preference\-based methods such as NPO and SimNPO temper this behavior\[[24](https://arxiv.org/html/2610.26864#bib.bib6),[6](https://arxiv.org/html/2610.26864#bib.bib5)\], while retain\-aware methods such as GradDiff add utility\-preserving supervision at additional online cost\. Figure[1](https://arxiv.org/html/2610.26864#S0.F1)summarizes the central distinction\.

Figure[2](https://arxiv.org/html/2610.26864#S1.F2)gives a selective view of the objective\-design path that motivates our method\. Classical work focused on deleting training influence or approximating retraining\[[2](https://arxiv.org/html/2610.26864#bib.bib1),[7](https://arxiv.org/html/2610.26864#bib.bib9),[1](https://arxiv.org/html/2610.26864#bib.bib8)\]\. LLM unlearning made direct forget\-loss maximization a scalable baseline\[[23](https://arxiv.org/html/2610.26864#bib.bib2)\]; retain\-aware and preference objectives then sought to limit utility collapse\[[5](https://arxiv.org/html/2610.26864#bib.bib15),[24](https://arxiv.org/html/2610.26864#bib.bib6),[6](https://arxiv.org/html/2610.26864#bib.bib5)\]\.H2DFcontinues this progression by making the stopping region explicit and calibrating it from non\-members\.

2015–2021: Data deletionRemove influence or approximate retraining2023: Forget\-loss ascentScalable and simple, but unbounded2023–2024: Retain\-aware objectivesReplay, distillation, or constraints2024: Preference objectivesNPO and SimNPO temper collapseThis work:H2DFExplicit, calibrated target regionFigure 2:Selective progression of unlearning objectives\.H2DFmakes the stopping region explicit rather than pushing indefinitely away from the original likelihood\.We study a direct alternative: forgetting should be a calibrated region rather than an unbounded direction\.H2DFestimates a target loss interval from matched non\-member data and optimizes each forget example toward that interval\. Below the interval, its gradient follows GA; inside the interval, pressure saturates; above the interval, an optional upper penalty discourages overshoot\. Calibration is performed once, and training uses only the current model’s forget\-batch losses\.

Design principle\.*“Forgetting should be a calibrated region, not an infinite direction\.”*

Our contributions are:

1. \(i\)a general target\-band objective that recovers GA\-like updates when forgetting is insufficient and suppresses them after a calibrated target is reached;
2. \(ii\)task\-specific instantiations for TOFU\[[14](https://arxiv.org/html/2610.26864#bib.bib3)\]and MUSE\[[20](https://arxiv.org/html/2610.26864#bib.bib4)\];
3. \(iii\)experiments against GA, GradDiff, NPO, and SimNPO, including three\-seed TOFU results, MUSE\-News official metrics, and target\-band ablations; and
4. \(iv\)an empirical analysis of overshoot, checkpoint selection, and seed sensitivity in short\-budget LoRA unlearning\.

## 2H2DF Method

Matched non\-member calibration setScore with original modelscal​\(θ0\)s\_\{\\mathrm\{cal\}\}\(\\theta\_\{0\}\)Conditional quantilesai,bia\_\{i\},b\_\{i\}Forget\-example scoresi​\(θ\)s\_\{i\}\(\\theta\)si<ais\_\{i\}<a\_\{i\}GA\-like pressureai≤si≤bia\_\{i\}\\leq s\_\{i\}\\leq b\_\{i\}pressure saturatessi\>bis\_\{i\}\>b\_\{i\}upper penaltytargetsFigure 3:H2DFworkflow\. Calibration converts matched non\-member scores into per\-example target regions before training\. During unlearning, examples below the lower target receive GA\-like pressure, examples inside the target region receive little pressure, and the optional upper term discourages overshoot\.### 2\.1General Objective

Letsi​\(θ\)s\_\{i\}\(\\theta\)be a score for forget exampleii, with larger values denoting stronger forgetting\. In our experiments,sis\_\{i\}is token\-averaged NLL\. Let\[ai,bi\]\[a\_\{i\},b\_\{i\}\]be a target interval estimated from a held\-out non\-member calibration set, conditional on observable featurescic\_\{i\}such as sequence length, domain, and original\-model loss\. We define

ai=Qρlow​\(scal∣ci\),bi=Qρhigh​\(scal∣ci\),a\_\{i\}=Q\_\{\\rho\_\{\\mathrm\{low\}\}\}\(s\_\{\\mathrm\{cal\}\}\\mid c\_\{i\}\),\\qquad b\_\{i\}=Q\_\{\\rho\_\{\\mathrm\{high\}\}\}\(s\_\{\\mathrm\{cal\}\}\\mid c\_\{i\}\),\(2\)whereQρQ\_\{\\rho\}is an empirical quantile\. Figure[3](https://arxiv.org/html/2610.26864#S2.F3)summarizes how these fixed targets alter the online forget update\.

Using the smooth hingehτ​\(u\)=τ​log⁡\(1\+exp⁡\(u/τ\)\)h\_\{\\tau\}\(u\)=\\tau\\log\(1\+\\exp\(u/\\tau\)\), the objective is

ℒH2DF​\(θ\)=\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{H2DF\}\}\(\\theta\)=\{\}1\|Bf\|​∑i∈Bf\[hτ​\(ai−si​\(θ\)\)\+γ​hτ​\(si​\(θ\)−bi\)\]\\displaystyle\\frac\{1\}\{\|B\_\{f\}\|\}\\sum\_\{i\\in B\_\{f\}\}\\left\[h\_\{\\tau\}\(a\_\{i\}\-s\_\{i\}\(\\theta\)\)\+\\gamma h\_\{\\tau\}\(s\_\{i\}\(\\theta\)\-b\_\{i\}\)\\right\]\(3\)\+λ​‖θLoRA‖22\.\\displaystyle\+\\lambda\\\|\\theta\_\{\\mathrm\{LoRA\}\}\\\|\_\{2\}^\{2\}\.The lower term penalizes insufficient forgetting\. The upper term, weighted byγ≥0\\gamma\\geq 0, penalizes excessive loss;γ=0\\gamma=0gives a one\-sided objective\. The targets are precomputed, so Eq\. \([3](https://arxiv.org/html/2610.26864#S2.E3)\) has the same online forward/backward structure as GA\.

### 2\.2Gradient Behavior

For the one\-sided objective,

∇θhτ​\(ai−si​\(θ\)\)=\\displaystyle\\nabla\_\{\\theta\}h\_\{\\tau\}\(a\_\{i\}\-s\_\{i\}\(\\theta\)\)=\{\}−αi​\(θ\)​∇θsi​\(θ\),\\displaystyle\-\\alpha\_\{i\}\(\\theta\)\\nabla\_\{\\theta\}s\_\{i\}\(\\theta\),\(4\)αi​\(θ\)=\\displaystyle\\alpha\_\{i\}\(\\theta\)=\{\}σ​\(ai−si​\(θ\)τ\)\.\\displaystyle\\sigma\\\!\\left\(\\frac\{a\_\{i\}\-s\_\{i\}\(\\theta\)\}\{\\tau\}\\right\)\.Thusαi≈1\\alpha\_\{i\}\\approx 1whensi≪ais\_\{i\}\\ll a\_\{i\}, recovering the direction of GA, andαi→0\\alpha\_\{i\}\\to 0aftersis\_\{i\}exceeds the lower target\. With an upper target,

∂ℒi∂si=−σ​\(ai−siτ\)\+γ​σ​\(si−biτ\),\\frac\{\\partial\\mathcal\{L\}\_\{i\}\}\{\\partial s\_\{i\}\}=\-\\sigma\\\!\\left\(\\frac\{a\_\{i\}\-s\_\{i\}\}\{\\tau\}\\right\)\+\\gamma\\sigma\\\!\\left\(\\frac\{s\_\{i\}\-b\_\{i\}\}\{\\tau\}\\right\),\(5\)so sufficiently large losses receive a restoring gradient\. This bounded coefficient does not bound the model Jacobian∇θsi\\nabla\_\{\\theta\}s\_\{i\}; the claim is specifically that the objective removes GA’s persistent scalar incentive to increase loss\.

### 2\.3Task Instantiations

#### TOFU\.

For questionqiq\_\{i\}and answer tokensyi,1:Tiy\_\{i,1:T\_\{i\}\}, the score is answer\-only NLL:

si​\(θ\)=−1Ti​∑t=1Tilog⁡pθ​\(yi,t∣qi,yi,<t\)\.s\_\{i\}\(\\theta\)=\-\\frac\{1\}\{T\_\{i\}\}\\sum\_\{t=1\}^\{T\_\{i\}\}\\log p\_\{\\theta\}\(y\_\{i,t\}\\mid q\_\{i\},y\_\{i,<t\}\)\.\(6\)We use a one\-sided target

ai=max⁡\{si​\(θ0\)\+Δmin,Qρ​\(scal​\(θ0\)∣ci\)\},a\_\{i\}=\\max\\\!\\left\\\{s\_\{i\}\(\\theta\_\{0\}\)\+\\Delta\_\{\\min\},Q\_\{\\rho\}\(s\_\{\\mathrm\{cal\}\}\(\\theta\_\{0\}\)\\mid c\_\{i\}\)\\right\\\},\(7\)which combines a minimum displacement from the original model with a non\-member quantile\.

#### MUSE\-News\.

For textxi,1:Tix\_\{i,1:T\_\{i\}\},sis\_\{i\}is token\-averaged language\-modeling NLL\. We use the two\-sided targets in Eq\. \([2](https://arxiv.org/html/2610.26864#S2.E2)\)\. The goal is not to maximize perplexity, but to move forgotten text toward the loss regime of matched non\-members\.

### 2\.4Illustrative Unlearning Example

Figure[4](https://arxiv.org/html/2610.26864#S2.F4)illustrates the mechanism on a synthetic TOFU\-style question\. Suppose the original model assigns answer NLL0\.40\.4to a memorized answer, while matched non\-member answers define a target band\[1\.4,2\.2\]\[1\.4,2\.2\]\. Both GA andH2DFinitially increase the answer NLL\. AfterH2DFenters the band, its lower\-target pressure vanishes and the upper term prevents large overshoot\. GA has no corresponding stopping signal and continues increasing loss\. The numerical trajectory is illustrative rather than an experimental trace; it visualizes the gradient behavior in Eq\. \([4](https://arxiv.org/html/2610.26864#S2.E4)\)\.

01122334455660112233calibrated target bandIllustrative update stepAnswer NLLGAH2DFFigure 4:Illustrative per\-example dynamics\. For a synthetic memorized answer with target band\[1\.4,2\.2\]\[1\.4,2\.2\], GA continues increasing NLL, whereasH2DFapproaches the non\-member\-like region and saturates\. Values are pedagogical, not measured experiment outputs\.

## 3Experimental Setup

#### TOFU\.

We use the TOFU forget\-5% split and the remaining authors for retention and calibration\. The base isQwen2\.5\-1\.5B\-Instructwith a supervised TOFU adapter, followed by rank\-8 LoRA unlearning\. Runs use AdamW, learning rate10−410^\{\-4\}, batch size 4, 500 steps, cosine scheduling,τ=0\.25\\tau=0\.25,ρ=0\.7\\rho=0\.7,Δmin=0\.5\\Delta\_\{\\min\}=0\.5, andλ=10−4\\lambda=10^\{\-4\}\. We report seeds4242,4343, and4444where available\. Evaluation covers 200 forget and 3,420 retain examples; exact match is close to zero for all methods, so token F1 is more discriminative\. The reportedH2DFvariant applies a retain batch every 10 steps; GradDiff applies retain supervision every step\.

#### MUSE\-News\.

For MUSE\-News, we usemuse\-bench/MUSE\-News\_targetas the target model/checkpoint and theNousResearch/Llama\-2\-7b\-hftokenizer\. Rank\-8 LoRA runs use AdamW, learning rate2×10−52\\times 10^\{\-5\}, effective batch size 4, 50 steps,τ=0\.25\\tau=0\.25,γ=0\.5\\gamma=0\.5,\(ρlow,ρhigh\)=\(0\.5,0\.9\)\(\\rho\_\{\\mathrm\{low\}\},\\rho\_\{\\mathrm\{high\}\}\)=\(0\.5,0\.9\), andλ=10−4\\lambda=10^\{\-4\}\. Full internal evaluation contains 888 forget, 1,778 retain, and 1,521 calibration examples\.

#### Baselines and selection\.

We compare with the original model, GA, GradDiff, NPO, and SimNPO\. All compared unlearning methods use the same LoRA parameterization\. For MUSE, checkpoints are screened on a fixed 200\-example validation subset and selected by proximity to 50% of examples below the lower target\. The screen is used only for checkpoint selection; reported seed\-42 internal results use the full held\-out splits\. Official MUSE metrics are reported for the original model,H2DF, and SimNPO\. Appendix[A](https://arxiv.org/html/2610.26864#A1)provides the training algorithm, exact configurations, evidence provenance, and checkpoint\-selection rule\.

#### Metrics\.

For TOFU, higher forget NLL and lower forget token F1 indicate stronger forgetting; lower retain NLL and higher retain token F1 indicate better utility\. For MUSE internal diagnostics, we report forget and retain NLL, loss\-based membership\-inference AUC, and the fraction of forget examples inside the calibrated band\. AUC is diagnostic only and does not replace the official MUSE suite\. Official VerbMem\-Forget and KnowMem\-Forget are lower\-is\-better; KnowMem\-Retain is higher\-is\-better\. We reproduce PrivLeak with the official evaluator and preserve its sign convention\.

## 4Results

### 4\.1TOFU

Table[1](https://arxiv.org/html/2610.26864#S4.T1)shows the seed\-42 tradeoff\. SimNPO forgets most strongly but also incurs the largest utility loss\.H2DFand GradDiff occupy a less aggressive region:H2DFhas slightly higher forget NLL and higher retained token F1 than GradDiff, while their NLLs are nearly identical\. GA and NPO are also nearly identical in this run\.

Across three seeds \(Table[2](https://arxiv.org/html/2610.26864#S4.T2)and Fig\.[5](https://arxiv.org/html/2610.26864#S4.F5)\),H2DFand GradDiff remain close\.H2DFachieves somewhat stronger NLL forgetting and slightly higher retain F1, whereas GradDiff has lower forget F1 and marginally lower retain NLL\. GA is more aggressive on average but substantially more variable; its seed\-44 run raises both forget and retain NLL\. The result supports competitive forgetting–retention control, not uniform dominance\.

Table 1:TOFU seed\-42 results\. Arrows indicate the preferred direction; best unlearning values are shaded\.ForgetRetainMethodNLL↑\\uparrowF1↓\\downarrowNLL↓\\downarrowF1↑\\uparrowOriginal/SFT1\.0040\.4410\.9970\.446GA1\.5600\.3121\.2800\.334NPO1\.5610\.3101\.2820\.333SimNPO1\.6430\.2871\.3650\.309GradDiff1\.5380\.3181\.2200\.353H2DF1\.5650\.3271\.2170\.360Table 2:TOFU mean±\\pmsample standard deviation over seeds 42–44\.ForgetRetainMethodNLL↑\\uparrowF1↓\\downarrowNLL↓\\downarrowF1↑\\uparrowGA1\.595±0\.1961\.595\{\\pm\}0\.1960\.310±0\.0420\.310\{\\pm\}0\.0421\.300±0\.1901\.300\{\\pm\}0\.1900\.341±0\.0540\.341\{\\pm\}0\.054GradDiff1\.499±0\.0431\.499\{\\pm\}0\.0430\.348±0\.0260\.348\{\\pm\}0\.0261\.162±0\.0511\.162\{\\pm\}0\.0510\.389±0\.0320\.389\{\\pm\}0\.032H2DF1\.528±0\.0491\.528\{\\pm\}0\.0490\.353±0\.0220\.353\{\\pm\}0\.0221\.165±0\.0471\.165\{\\pm\}0\.0470\.394±0\.0290\.394\{\\pm\}0\.029111\.21\.21\.41\.41\.41\.41\.61\.61\.81\.8Retain NLL↓\\downarrowForget NLL↑\\uparrowGAGradDiffH2DFFigure 5:TOFU forgetting–retention tradeoff over seeds 42–44\. Points are means and error bars are sample standard deviations\. The preferred region is toward the upper left\.H2DFand GradDiff are stable and closely matched, whereas GA has substantially larger variance\.
### 4\.2MUSE\-News

Table[3](https://arxiv.org/html/2610.26864#S4.T3)reports the full held\-out loss diagnostics\. GA and NPO attain the lowest loss\-based MIA AUC, but at the cost of the largest retain degradation\.H2DFreduces retain NLL by 0\.35 relative to GA and NPO while maintaining a lower AUC than GradDiff\. SimNPO is the closest comparator: it has better retain NLL and a higher in\-band rate, whileH2DFhas slightly lower AUC and higher forget NLL\. These diagnostics therefore show a competitive, not dominant, tradeoff\.

Official metrics in Table[4](https://arxiv.org/html/2610.26864#S4.T4)show thatH2DFand SimNPO are close on knowledge forgetting\. SimNPO has stronger verbatim forgetting, whileH2DFretains slightly more knowledge utility\. We report PrivLeak using the official evaluator’s sign convention and do not interpret its direction here\. Official MUSE evaluation was completed only for seed 42, so these differences should not be interpreted as statistically stable\.

Table 3:MUSE\-News seed\-42 held\-out diagnostics\. MIA AUC is diagnostic; Band is the fraction inside the calibrated target interval\.MethodForgetNLLRetainNLL↓\\downarrowMIAAUC↓\\downarrowBandfraction↑\\uparrowOriginal0\.5760\.7610\.9810\.015GA1\.8652\.0180\.9340\.071NPO1\.8582\.0100\.9340\.072GradDiff1\.3791\.5460\.9610\.081SimNPO1\.5131\.5970\.9550\.144H2DF1\.5571\.6660\.9490\.131Table 4:Official MUSE\-News seed\-42 metrics\. PrivLeak follows the evaluator’s sign convention and is not direction\-ranked\.Forget↓\\downarrowRetain↑\\uparrowModelVerbMemKnowMemKnowMemPrivLeakOriginal57\.4264\.0954\.12\-99\.81SimNPO19\.2745\.7438\.27\-98\.32H2DF20\.7145\.8338\.72\-96\.52
### 4\.3Target\-Band Ablations

The upper target reduces overshoot \(Table[5](https://arxiv.org/html/2610.26864#S4.T5)and Fig\.[6](https://arxiv.org/html/2610.26864#S4.F6)\): at step 40, the two\-sided objective places 30\.0% of examples above the band, compared with 47\.0% for the lower\-only variant, and improves retain NLL from 1\.790 to 1\.598\. Continuing to the fixed final checkpoint substantially increases the above\-band fraction and retention damage for both objectives\. Sparse retain replay with weight 0\.1 moves in the opposite direction and leaves 90\.5% of examples below target\. These results motivate target\-aware checkpoint selection rather than always using the final training step\.

The corrected MUSE multiseed screen selects steps 40, 50, and 50 for seeds 42, 43, and 44, respectively\. Their forget NLLs are 1\.530, 0\.956, and 1\.531\. Seed 43 leaves 93\.0% of examples below target, showing that a fixed 50\-step budget does not reliably reach the intended region\.

Table 5:MUSE validation\-screen ablations\. Occupancy columns are fractions relative to the calibrated band\.VariantStepForgetNLLRetainNLLBelowInsideAboveTwo\-sided401\.5301\.5980\.5750\.1250\.300Two\-sided final501\.8001\.8520\.2800\.1600\.560Lower\-only401\.7111\.7900\.4350\.0950\.470Lower\-only final502\.1072\.1710\.1250\.0800\.795Two\-sided\+R501\.1071\.1950\.9050\.0200\.0752S\-402S\-501S\-401S\-50R\-5000\.20\.20\.40\.40\.60\.60\.80\.811Fraction of forget examplesBelowInsideAboveFigure 6:MUSE validation\-screen target occupancy\. 2S and 1S denote two\-sided and lower\-only objectives; the suffix is the checkpoint step, and \+R denotes sparse retain replay\. The upper target reduces overshoot at step 40, while fixed final checkpoints overshoot and retain replay under\-forgets\.

## 5Analysis and Discussion

#### Computational cost\.

Letℱ​\(b\)\\mathcal\{F\}\(b\)denote the cost of a forward pass on batch sizebb, with backward cost approximately2​ℱ​\(b\)2\\mathcal\{F\}\(b\)\. GA, SimNPO, andH2DFeach require one current\-model forget forward/backward pass, approximately3​ℱ​\(\|Bf\|\)3\\mathcal\{F\}\(\|B\_\{f\}\|\)per step\. NPO additionally requires stored original losses or a reference computation, depending on implementation\. GradDiff adds a retain forward/backward pass every step\.H2DFshifts its additional work to one\-time target calibration\.

#### What the target band controls\.

The objective directly controls per\-example loss relative to a non\-member reference distribution\. It does not certify removal of training\-data influence, nor does matching losses guarantee matching all behavioral or privacy properties\. The official MUSE results illustrate this distinction: internal loss diagnostics are useful for selection, but do not determine VerbMem, KnowMem, or PrivLeak\.

This caution is consistent with theoretical and empirical work arguing that unlearning should be judged not only by empirical forget loss, but also by generalization, resource constraints, and stronger downstream evaluations\[[19](https://arxiv.org/html/2610.26864#bib.bib23),[18](https://arxiv.org/html/2610.26864#bib.bib24)\]\.

#### Calibration uncertainty\.

Fornncalibration examples in a conditioning bin, the Dvoretzky–Kiefer–Wolfowitz inequality gives, with probability at least1−δ1\-\\delta,

supz\|F^n​\(z\)−F​\(z\)\|≤log⁡\(2/δ\)2​n\.\\sup\_\{z\}\|\\widehat\{F\}\_\{n\}\(z\)\-F\(z\)\|\\leq\\sqrt\{\\frac\{\\log\(2/\\delta\)\}\{2n\}\}\.\(8\)This motivates coarse bins and minimum bin sizes\. The bound addresses sampling error, not distribution mismatch: a large but mismatched calibration set can still define the wrong target\.

#### Practical implication\.

The experiments suggest that the upper target and checkpoint rule are as important as the lower forgetting target\. A lower\-only objective can still overshoot, and retain replay can prevent target attainment if weighted too strongly\. A useful deployment protocol should therefore report target occupancy through training and validate checkpoint selection on held\-out data\.

## 6Related Work

#### Classical machine unlearning\.

Machine unlearning was originally motivated by the need to remove selected training examples without retraining a model from scratch\[[2](https://arxiv.org/html/2610.26864#bib.bib1)\]\. Early work studied both exact and approximate deletion mechanisms\. SISA training reduces the cost of deletion by partitioning training into shards and slices\[[1](https://arxiv.org/html/2610.26864#bib.bib8)\], while certified\-removal methods provide stronger guarantees in restricted model classes\[[7](https://arxiv.org/html/2610.26864#bib.bib9),[9](https://arxiv.org/html/2610.26864#bib.bib10),[16](https://arxiv.org/html/2610.26864#bib.bib11)\]\. Other approaches modify, reverse, or approximate training updates to remove the influence of targeted examples\[[8](https://arxiv.org/html/2610.26864#bib.bib12),[21](https://arxiv.org/html/2610.26864#bib.bib13),[10](https://arxiv.org/html/2610.26864#bib.bib14)\]\. These works establish the central tension between deletion quality, computational cost, and retained utility, but they do not directly solve unlearning for modern generative language models\. Sekhari et al\. study machine unlearning from a generalization perspective, emphasizing how many samples can be deleted while still maintaining performance on unseen data under computation and storage constraints\[[19](https://arxiv.org/html/2610.26864#bib.bib23)\]\. This viewpoint is complementary to our empirical LLM setting: H2DF controls a calibrated loss target during unlearning, but does not provide the kind of distributional retraining equivalence or generalization guarantee studied in theoretical unlearning\.

#### Unlearning for large language models\.

LLM unlearning is harder because the forget target is often semantic, generative, and difficult to localize\. Gradient\-ascent\-style objectives are simple and widely used, but they can cause utility collapse or abnormal generations\[[23](https://arxiv.org/html/2610.26864#bib.bib2),[24](https://arxiv.org/html/2610.26864#bib.bib6)\]\. Recent methods therefore add retain losses, distillation, preference\-style objectives, or optimization constraints to better trade off forgetting and utility\[[5](https://arxiv.org/html/2610.26864#bib.bib15),[12](https://arxiv.org/html/2610.26864#bib.bib16),[22](https://arxiv.org/html/2610.26864#bib.bib17),[17](https://arxiv.org/html/2610.26864#bib.bib18)\]\. Our method follows this line of work but focuses on a different control mechanism: rather than maximizing forget loss indefinitely, it defines a data\-calibrated target region for each forget example\.

#### Preference\- and representation\-based methods\.

Negative Preference Optimization reframes unlearning as a preference\-style objective and slows catastrophic collapse compared with GA\[[24](https://arxiv.org/html/2610.26864#bib.bib6)\]\. SimNPO simplifies this formulation by removing the reference\-model dependence and obtains strong empirical results on TOFU and MUSE\[[6](https://arxiv.org/html/2610.26864#bib.bib5)\]\. Representation\-based methods such as RMU steer internal activations away from forget\-domain representations and are especially important for hazardous\-knowledge removal\[[11](https://arxiv.org/html/2610.26864#bib.bib7)\]\. Follow\-up work shows that apparent forgetting can fail under stronger robustness and knowledge\-extraction tests\[[13](https://arxiv.org/html/2610.26864#bib.bib19)\]\.H2DFis complementary: it does not directly steer hidden states or use preference pairs, but instead calibrates the loss region that forgotten examples should reach\.

#### Benchmarks and evaluation\.

TOFU provides a controlled fictitious\-author QA benchmark for testing whether a fine\-tuned model behaves as if selected author profiles had not been learned\[[14](https://arxiv.org/html/2610.26864#bib.bib3)\]\. MUSE evaluates unlearning across verbatim memorization, knowledge memorization, privacy leakage, utility, scalability, and sustainability\[[20](https://arxiv.org/html/2610.26864#bib.bib4)\]\. WMDP evaluates hazardous\-knowledge removal and introduced representation misdirection as a strong unlearning baseline\[[11](https://arxiv.org/html/2610.26864#bib.bib7)\]\. Other evaluations study membership inference, extraction, and memorization risks in language models\[[4](https://arxiv.org/html/2610.26864#bib.bib20),[3](https://arxiv.org/html/2610.26864#bib.bib21),[15](https://arxiv.org/html/2610.26864#bib.bib22)\]\. These benchmarks show that high forget loss alone is not sufficient: unlearning methods must also preserve retain behavior and avoid new privacy or utility failures\. This motivatesH2DF’s calibrated target\-band view\.

## 7Limitations

The study has four main limitations\. First, MUSE official metrics are available only for seed 42, and the validation screens reveal substantial seed sensitivity\. Second, MUSE\-Books, sequential requests, and larger\-scale models are not evaluated\. Third, TOFU and MUSE use different base models and evaluation regimes, so their numerical results are not directly comparable\. Fourth, the loss\-based target is a proxy rather than a certified unlearning guarantee; performance depends on the quality and representativeness of the calibration set\. The released RMU implementation uses LoRA rather than the original full\-parameter setup and is therefore excluded from the main quantitative comparison\.

Recent robustness work further shows that practical unlearning methods may fail under stronger evaluations such as data\-poisoning removal, even when they appear effective under standard metrics\[[18](https://arxiv.org/html/2610.26864#bib.bib24)\]\. This supports our choice to report H2DF as a controllable target\-band mechanism rather than a certified deletion method\.

## 8Conclusion

We introducedH2DF, a calibrated target\-band formulation for language\-model unlearning\. It preserves GA’s early update direction while suppressing pressure after examples reach a non\-member\-derived loss region\. On TOFU,H2DFmatches the forgetting–retention tradeoff of GradDiff across three seeds with low variability\. On MUSE\-News, it improves retention over GA and NPO and is competitive with, but does not outperform, SimNPO\. The upper target reduces overshoot, while multiseed results show that checkpoint selection and optimization stability remain open practical problems\. Calibrated target bands are therefore best viewed as a mechanism for controllable forgetting, not as a guarantee of complete data removal\.

## References

- \[1\]L\. Bourtoule, V\. Chandrasekaran, C\. A\. Choquette\-Choo, H\. Jia, A\. Travers, B\. Zhang, D\. Lie, and N\. Papernot\(2021\)Machine unlearning\.In2021 IEEE Symposium on Security and Privacy,pp\. 141–159\.External Links:[Document](https://dx.doi.org/10.1109/SP40001.2021.00019),[Link](https://arxiv.org/abs/1912.03817)Cited by:[§1](https://arxiv.org/html/2610.26864#S1.p2.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px1.p1.1)\.
- \[2\]Y\. Cao and J\. Yang\(2015\)Towards making systems forget with machine unlearning\.In2015 IEEE Symposium on Security and Privacy,pp\. 463–480\.External Links:[Document](https://dx.doi.org/10.1109/SP.2015.35)Cited by:[§1](https://arxiv.org/html/2610.26864#S1.p1.2),[§1](https://arxiv.org/html/2610.26864#S1.p2.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px1.p1.1)\.
- \[3\]N\. Carlini, D\. Ippolito, M\. Jagielski, K\. Lee, F\. Tramèr, and C\. Zhang\(2023\)Quantifying memorization across neural language models\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2202.07646)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px4.p1.1)\.
- \[4\]N\. Carlini, F\. Tramèr, E\. Wallace, M\. Jagielski, A\. Herbert\-Voss, K\. Lee, A\. Roberts, T\. Brown, D\. Song, Ú\. Erlingsson, A\. Oprea, and C\. Raffel\(2021\)Extracting training data from large language models\.In30th USENIX Security Symposium,pp\. 2633–2650\.External Links:[Link](https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px4.p1.1)\.
- \[5\]J\. Chen and D\. Yang\(2023\)Unlearn what you want to forget: efficient unlearning for LLMs\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 12041–12052\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.738),[Link](https://aclanthology.org/2023.emnlp-main.738/)Cited by:[§1](https://arxiv.org/html/2610.26864#S1.p2.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px2.p1.1)\.
- \[6\]C\. Fan, J\. Liu, L\. Lin, J\. Jia, R\. Zhang, S\. Mei, and S\. Liu\(2024\)Simplicity prevails: rethinking negative preference optimization for LLM unlearning\.arXiv preprint arXiv:2410\.07163\.External Links:[Link](https://arxiv.org/abs/2410.07163)Cited by:[Figure 1](https://arxiv.org/html/2610.26864#S0.F1),[§1](https://arxiv.org/html/2610.26864#S1.p1.3),[§1](https://arxiv.org/html/2610.26864#S1.p2.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px3.p1.1)\.
- \[7\]A\. Ginart, M\. Y\. Guan, G\. Valiant, and J\. Zou\(2019\)Making AI forget you: data deletion in machine learning\.InAdvances in Neural Information Processing Systems,Vol\.32\.External Links:[Link](https://arxiv.org/abs/1907.05012)Cited by:[§1](https://arxiv.org/html/2610.26864#S1.p2.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px1.p1.1)\.
- \[8\]L\. Graves, V\. Nagisetty, and V\. Ganesh\(2021\)Amnesiac machine learning\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 11516–11524\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v35i13.17371)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px1.p1.1)\.
- \[9\]C\. Guo, T\. Goldstein, A\. Hannun, and L\. van der Maaten\(2020\)Certified data removal from machine learning models\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,pp\. 3832–3842\.External Links:[Link](https://proceedings.mlr.press/v119/guo20c.html)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px1.p1.1)\.
- \[10\]Z\. Izzo, M\. A\. Smart, K\. Chaudhuri, and J\. Zou\(2021\)Approximate data deletion from machine learning models\.InProceedings of the 24th International Conference on Artificial Intelligence and Statistics,Proceedings of Machine Learning Research, Vol\.130,pp\. 2008–2016\.External Links:[Link](https://proceedings.mlr.press/v130/izzo21a.html)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px1.p1.1)\.
- \[11\]N\. Li, A\. Pan, A\. Gopal, S\. Yue, D\. Berrios, A\. Gatti, J\. D\. Li, A\. Dombrowski, S\. Goel, L\. Phan,et al\.\(2024\)The WMDP benchmark: measuring and reducing malicious use with unlearning\.arXiv preprint arXiv:2403\.03218\.External Links:[Link](https://arxiv.org/abs/2403.03218)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px3.p1.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px4.p1.1)\.
- \[12\]S\. Liu, Y\. Yao, J\. Jia, S\. Casper, N\. Baracaldo, P\. Hase, Y\. Yao, C\. Y\. Liu, X\. Xu, H\. Li, K\. R\. Varshney, M\. Bansal, S\. Koyejo, and Y\. Liu\(2024\)Rethinking machine unlearning for large language models\.arXiv preprint arXiv:2402\.08787\.External Links:[Link](https://arxiv.org/abs/2402.08787)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px2.p1.1)\.
- \[13\]A\. Lynch, P\. Guo, A\. Ewart, S\. Casper, and D\. Hadfield\-Menell\(2024\)Eight methods to evaluate robust unlearning in LLMs\.arXiv preprint arXiv:2402\.16835\.External Links:[Link](https://arxiv.org/abs/2402.16835)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px3.p1.1)\.
- \[14\]P\. Maini, Z\. Feng, A\. Schwarzschild, Z\. C\. Lipton, and J\. Z\. Kolter\(2024\)TOFU: a task of fictitious unlearning for LLMs\.arXiv preprint arXiv:2401\.06121\.External Links:[Link](https://arxiv.org/abs/2401.06121)Cited by:[item \(ii\)](https://arxiv.org/html/2610.26864#S1.I1.i2.p1.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px4.p1.1)\.
- \[15\]J\. Mattern, F\. Mireshghallah, Z\. Jin, B\. Schölkopf, M\. Sachan, and T\. Berg\-Kirkpatrick\(2023\)Membership inference attacks against language models via neighbourhood comparison\.arXiv preprint arXiv:2305\.18462\.External Links:[Link](https://arxiv.org/abs/2305.18462)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px4.p1.1)\.
- \[16\]S\. Neel, A\. Roth, and S\. Sharifi\-Malvajerdi\(2021\)Descent\-to\-delete: gradient\-based methods for machine unlearning\.InProceedings of the 32nd International Conference on Algorithmic Learning Theory,Proceedings of Machine Learning Research, Vol\.132,pp\. 931–962\.External Links:[Link](https://proceedings.mlr.press/v132/neel21a.html)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px1.p1.1)\.
- \[17\]Z\. Pan, S\. Zhang, Y\. Zheng, C\. Li, Y\. Cheng, and J\. Zhao\(2024\)Multi\-objective large language model unlearning\.arXiv preprint arXiv:2412\.20412\.External Links:[Link](https://arxiv.org/abs/2412.20412)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px2.p1.1)\.
- \[18\]M\. Pawelczyk, J\. Z\. Di, Y\. Lu, G\. Kamath, A\. Sekhari, and S\. Neel\(2025\)Machine unlearning fails to remove data poisoning attacks\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=HaX48yksVL)Cited by:[§5](https://arxiv.org/html/2610.26864#S5.SS0.SSS0.Px2.p2.1),[§7](https://arxiv.org/html/2610.26864#S7.p2.1)\.
- \[19\]A\. Sekhari, J\. Acharya, G\. Kamath, and A\. T\. Suresh\(2021\)Remember what you want to forget: algorithms for machine unlearning\.arXiv preprint arXiv:2103\.03279\.External Links:[Link](https://arxiv.org/abs/2103.03279)Cited by:[§5](https://arxiv.org/html/2610.26864#S5.SS0.SSS0.Px2.p2.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px1.p1.1)\.
- \[20\]W\. Shi, J\. Lee, Y\. Huang, S\. Malladi, J\. Zhao, A\. Holtzman, D\. Liu, L\. Zettlemoyer, N\. A\. Smith, and C\. Zhang\(2024\)MUSE: machine unlearning six\-way evaluation for language models\.arXiv preprint arXiv:2407\.06460\.External Links:[Link](https://arxiv.org/abs/2407.06460)Cited by:[item \(ii\)](https://arxiv.org/html/2610.26864#S1.I1.i2.p1.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px4.p1.1)\.
- \[21\]A\. Thudi, G\. Deza, V\. Chandrasekaran, and N\. Papernot\(2022\)Unrolling SGD: understanding factors influencing machine unlearning\.In2022 IEEE 7th European Symposium on Security and Privacy,pp\. 303–319\.External Links:[Document](https://dx.doi.org/10.1109/EuroSP53844.2022.00027),[Link](https://arxiv.org/abs/2109.13398)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px1.p1.1)\.
- \[22\]B\. Wang, Y\. Zi, Y\. Sun, Y\. Zhao, and B\. Qin\(2024\)RKLD: reverse KL\-divergence\-based knowledge distillation for unlearning personal information in large language models\.arXiv preprint arXiv:2406\.01983\.External Links:[Link](https://arxiv.org/abs/2406.01983)Cited by:[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px2.p1.1)\.
- \[23\]Y\. Yao, X\. Xu, and Y\. Liu\(2023\)Large language model unlearning\.arXiv preprint arXiv:2310\.10683\.External Links:[Link](https://arxiv.org/abs/2310.10683)Cited by:[§1](https://arxiv.org/html/2610.26864#S1.p1.2),[§1](https://arxiv.org/html/2610.26864#S1.p2.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px2.p1.1)\.
- \[24\]R\. Zhang, L\. Lin, Y\. Bai, and S\. Mei\(2024\)Negative preference optimization: from catastrophic collapse to effective unlearning\.arXiv preprint arXiv:2404\.05868\.External Links:[Link](https://arxiv.org/abs/2404.05868)Cited by:[Figure 1](https://arxiv.org/html/2610.26864#S0.F1),[§1](https://arxiv.org/html/2610.26864#S1.p1.3),[§1](https://arxiv.org/html/2610.26864#S1.p2.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px2.p1.1),[§6](https://arxiv.org/html/2610.26864#S6.SS0.SSS0.Px3.p1.1)\.

## Appendix ATraining Algorithm

Algorithm 1Calibrated target\-band unlearning\. Calibration is completed before optimization; targets are not re\-estimated from evaluation data\.1:Original model

θ0\\theta\_\{0\}; forget set

DfD\_\{f\}; matched calibration set

DcalD\_\{\\mathrm\{cal\}\}; optional retain set

DrD\_\{r\}; quantiles

ρlow,ρhigh\\rho\_\{\\mathrm\{low\}\},\\rho\_\{\\mathrm\{high\}\};

τ,γ,λ\\tau,\\gamma,\\lambda; replay period

KK; training steps

TT
2:LoRA adapter, fixed targets, diagnostics, and selected checkpoint

3:Score

DfD\_\{f\}and

DcalD\_\{\\mathrm\{cal\}\}with

θ0\\theta\_\{0\}; compute length, original\-loss, and available domain/type features

4:Build coarse feature bins; progressively back off conditioning features when a matched bin is smaller than the minimum bin size

5:foreach forget example

iido

6:Compute

aia\_\{i\}and optional

bib\_\{i\}from matched calibration quantiles

7:iftask is TOFUthen

8:

ai←max⁡\{ai,si​\(θ0\)\+Δmin\}a\_\{i\}\\leftarrow\\max\\\{a\_\{i\},s\_\{i\}\(\\theta\_\{0\}\)\+\\Delta\_\{\\min\}\\\}
9:endif

10:endfor

11:Initialize trainable LoRA parameters

θ\\theta
12:for

t=1,…,Tt=1,\\ldots,Tdo

13:Sample

Bf⊂DfB\_\{f\}\\subset D\_\{f\}and compute per\-example scores

si​\(θ\)s\_\{i\}\(\\theta\)
14:

ℒ←ℒH2DF​\(θ;Bf\)\\mathcal\{L\}\\leftarrow\\mathcal\{L\}\_\{\\mathrm\{H2DF\}\}\(\\theta;B\_\{f\}\)from Eq\. \([3](https://arxiv.org/html/2610.26864#S2.E3)\)

15:if

DrD\_\{r\}is used and

tmodK=0t\\bmod K=0then

16:

ℒ←ℒ\+βr​ℒretain​\(θ;Br\)\\mathcal\{L\}\\leftarrow\\mathcal\{L\}\+\\beta\_\{r\}\\mathcal\{L\}\_\{\\mathrm\{retain\}\}\(\\theta;B\_\{r\}\)
17:endif

18:Backpropagate, clip gradients, update

θ\\theta, and log target occupancy

19:endfor

20:Select a saved checkpoint on the fixed validation screen; evaluate once on held\-out data

## Appendix BReproducibility Details

Table 6:Primary configurations used for the reported experiments\.SettingTOFUMUSE\-NewsBase model/checkpointQwen2\.5\-1\.5B\-Instruct with TOFU SFT adapterMUSE\-News target checkpointTokenizermodel tokenizerLlama\-2\-7B tokenizerLoRA rank / alpha / dropout8/16/08/16/08/16/08/16/0Target modulesq, k, v, o, gate, up, and down projectionsPrecisionBF16BF16Batch / accumulation4/14/11/41/4Maximum sequence length5121024Optimizer / scheduleAdamW / cosineAdamW / cosineLearning rate / steps10−4/50010^\{\-4\}/5002×10−5/502\\times 10^\{\-5\}/50Warmup / gradient clipping25/1\.025/1\.025/1\.025/1\.0τ\\tau/λ\\lambda0\.25/10−40\.25/10^\{\-4\}0\.25/10−40\.25/10^\{\-4\}Calibration quantilesρ=0\.7\\rho=0\.7,Δmin=0\.5\\Delta\_\{\\min\}=0\.5\(ρlow,ρhigh\)=\(0\.5,0\.9\)\(\\rho\_\{\\mathrm\{low\}\},\\rho\_\{\\mathrm\{high\}\}\)=\(0\.5,0\.9\)Upper\-band weightγ\\gamma00\.5Retain replayevery 10 steps for reportedH2DFvariantnone for the mainH2DFrunTable 7:Evidence provenance\. “Screen” denotes the fixed 200\-example checkpoint\-diagnostic subset rather than final held\-out evaluation; \* indicates availability for only a subset of methods\.Evidence blockSeedsEvaluation scopeManuscript useTOFU full QA42–44\*200 forget; 3,420 retain examplesTables[1](https://arxiv.org/html/2610.26864#S4.T1)–[2](https://arxiv.org/html/2610.26864#S4.T2), Fig\.[5](https://arxiv.org/html/2610.26864#S4.F5)MUSE internal comparison42888 forget; 1,778 retain; 1,521 calibrationTable[3](https://arxiv.org/html/2610.26864#S4.T3)MUSE official suite42Official VerbMem, KnowMem, and PrivLeak evaluationTable[4](https://arxiv.org/html/2610.26864#S4.T4)MUSE multiseed screen42–44Fixed 200\-example validation screenSeed\-sensitivity discussionMUSE ablation screen42Fixed 200\-example validation screenTable[5](https://arxiv.org/html/2610.26864#S4.T5), Fig\.[6](https://arxiv.org/html/2610.26864#S4.F6)
## Appendix CCheckpoint Selection and Reporting

For the short\-budget MUSE runs, let

rt=1\|Vf\|​∑i∈Vf𝟏​\[si​\(θt\)<ai\]r\_\{t\}=\\frac\{1\}\{\|V\_\{f\}\|\}\\sum\_\{i\\in V\_\{f\}\}\\mathbf\{1\}\\\!\\left\[s\_\{i\}\(\\theta\_\{t\}\)<a\_\{i\}\\right\]\(9\)be the fraction of validation forget examples still below their lower target at checkpointtt\. We selected

t⋆=arg⁡mint∈𝒯⁡\|rt−0\.5\|,t^\{\\star\}=\\arg\\min\_\{t\\in\\mathcal\{T\}\}\|r\_\{t\}\-0\.5\|,\(10\)using a fixed 200\-example validation screen and the saved checkpoint set𝒯\\mathcal\{T\}\. This criterion was chosen before full held\-out evaluation to avoid selecting directly on the reported test metrics\. It is a practical heuristic rather than a theoretically optimal rule\. The manuscript therefore reports target occupancy alongside final benchmark metrics and distinguishes validation\-screen ablations from full held\-out and official evaluations\.

## Appendix DArtifact Contents

The released artifact contains the training and calibration implementation, benchmark\-specific YAML configurations, baseline runners, CPU unit tests, and the compact JSON metrics used in the manuscript\. Model weights, adapters, downloaded datasets, credentials, and raw experiment caches are intentionally excluded\. The expected workflow is:

1. 1\.run calibration to create fixed per\-example targets;
2. 2\.trainH2DFor a configured baseline;
3. 3\.evaluate the selected adapter on forget, retain, and calibration data;
4. 4\.run the official MUSE evaluator for benchmark\-level metrics\.