Confess What You Know: Forget-Set Misalignment with Model Knowledge in LLM Unlearning

# Forget-Set Misalignment with Model Knowledge in LLM Unlearning
Source: [https://arxiv.org/html/2609.00605](https://arxiv.org/html/2609.00605)
## Confess What You Know: Forget\-Set Misalignment with Model Knowledge in LLM Unlearning

###### Abstract

Machine unlearning for large language models \(LLMs\) often assumes that a pre\-defined forget set matches what the model has memorized, but this frequently breaks in realistic privacy settings where the original training data is inaccessible\. We term this gap*forget\-set misalignment*and identify two cases\. In*Under Unlearning*, the forget set omits memorized information and leakage persists\. In*Out\-of\-Knowledge Unlearning*, the algorithm is driven to “forget” knowledge the model never learned, perturbing parameters and degrading utility\. Using*gradient\-level analysis*, we show these behaviors arise from misaligned unlearning targets rather than specific optimization choices\. We then proposeCONfession\-to\-Forget\-Set \(CONFS\), a*data\-blind*framework that constructs model\-aligned forget sets by eliciting and formalizing the model’s memorized knowledge\. Across synthetic, multimodal, and real\-world benchmarks, CONFS approaches Gold\-standard performance on several metrics and achieves a competitive forgetting\-utility balance, while preserving utility better than other data\-blind forget\-set constructions\.

## 1Introduction

Large Language Models \(LLMs\) are trained on vast, heterogeneous datasets that often contain sensitive personal information[Achiam et al\. \(2023\)](https://arxiv.org/html/2609.00605#bib.bib1);[Touvron et al\. \(2023\)](https://arxiv.org/html/2609.00605#bib.bib2);[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib3);[Bai et al\. \(2022\)](https://arxiv.org/html/2609.00605#bib.bib4);[Team et al\. \(2023\)](https://arxiv.org/html/2609.00605#bib.bib5)\. This introduces a critical vulnerability, as models may unintentionally leak confidential data at deployment\. To mitigate these privacy threats, LLM unlearning has emerged as an important post\-hoc solution[Yao et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib6);[Jang et al\. \(2023\)](https://arxiv.org/html/2609.00605#bib.bib7)\. In this setting, sensitive data targeted for removal is pre\-defined and commonly referred to as a*forget set*[Geng et al\. \(2025\)](https://arxiv.org/html/2609.00605#bib.bib8)\. Given this set, various unlearning strategies are applied so that the model selectively forgets the specified information[Liu et al\. \(2022\)](https://arxiv.org/html/2609.00605#bib.bib10);[Jang et al\. \(2023\)](https://arxiv.org/html/2609.00605#bib.bib7);[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib11);[Chen and Yang \(2023\)](https://arxiv.org/html/2609.00605#bib.bib9)\.

However, relying on a pre\-defined forget set is often impractical in real\-world scenarios[Enck et al\. \(2014\)](https://arxiv.org/html/2609.00605#bib.bib12);[Romanosky et al\. \(2011\)](https://arxiv.org/html/2609.00605#bib.bib13)\. Suppose an individual discovers that an LLM can reveal some of their personal information and requests its removal \(Fig\.[1](https://arxiv.org/html/2609.00605#S1.F1)\)\. Although the individual can specify the details they want deleted, they cannot observe which details the model actually memorized\. As a result, the requested forget set may differ from the model’s hidden memorized knowledge\.

We refer to this mismatch as*forget\-set misalignment*, a setting largely overlooked by existing benchmarks that assume perfectly specified unlearning targets[Maini et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib14);[Dontsov et al\. \(2025\)](https://arxiv.org/html/2609.00605#bib.bib15);[Liu et al\. \(2025\)](https://arxiv.org/html/2609.00605#bib.bib16)\. In this work, we investigate how*forget\-set misalignment*fundamentally alters unlearning behavior under realistic settings\.

We first identify two types of*forget\-set misalignment*, each leading to a distinct failure mode\.*Under Unlearning*occurs when memorized sensitive information is omitted from the provided forget set\. In this case, the forgetting effect remains largely confined to the requested facts and fails to generalize to associated entity\-level knowledge, leaving core privacy risks intact\.*Out\-of\-Knowledge Unlearning*occurs when the forget set contains information that the model never encountered during training, unnecessarily perturbing model parameters and substantially degrading general utility\.

![Refer to caption](https://arxiv.org/html/2609.00605v1/figure1.png)Figure 1:Illustration of*forget\-set misalignment*in LLM unlearning\.Only part of a user’s personal information is retained as the model’s memorized knowledge, but this boundary is hidden from the user\. As a result, deletion requests may omit memorized information \(*Under Unlearning*\) or include non\-memorized information \(*Out\-of\-Knowledge Unlearning*\), leading to privacy leakage or unnecessary utility degradation\.Motivated by these findings, we introduceCONfession\-to\-Forget\-Set \(CONFS\), which dynamically generates a forget set by prompting the model to*confess*its own memorized knowledge\. We elicit the model’s memorized knowledge and formalize it into a discrete set of Subject\-Relation\-Object \(SRO\) triplets, transforming ambiguous semantic information into verifiable, atomic units\. To ensure comprehensive coverage, we employ a recursive*reconfession*process that iteratively probes the model for hidden details, effectively reducing gaps in the forget set\.

By constructing a forget set aligned with the model’s memorized knowledge, CONFS mitigates the adverse effects of*forget\-set misalignment*\. Despite operating in a fully*data\-blind*setting without access to the pre\-training data, CONFS approaches Gold\-standard performance on several metrics and achieves a competitive forgetting\-utility balance, while better preserving utility than alternative surrogate forget\-set constructions\. These results show that reliable unlearning depends not only on the optimization objective, but also on whether the forget set matches what the model actually knows\.

𝒟L−\\mathcal\{D\}\_\{L\-\}↓\\downarrow𝒟L\+\\mathcal\{D\}\_\{L\+\}↓\\downarrowRetain\(𝒟r​e​t​a​i​n\\mathcal\{D\}\_\{retain\}\)↑\\uparrowReal Authors\(𝒟R​A\\mathcal\{D\}\_\{RA\}\)↑\\uparrowProb\.R\-LProb\.R\-LProb\.R\-LProb\.R\-Lpre\-trained modelstep 00\.9960\.9860\.9950\.9900\.9950\.9910\.0580\.886Under Unlearning\(𝐅:𝒟L\+\\mathbf\{F\}:\\mathcal\{D\}\_\{L\+\}\)step 120\.9930\.9760\.9210\.8080\.9910\.9670\.0770\.904step 240\.9010\.8260\.6500\.5630\.8880\.8070\.0810\.884step 360\.7400\.6950\.4420\.4730\.7490\.7090\.0700\.804step 480\.6370\.6350\.3270\.4260\.6690\.6490\.0640\.781step 600\.6240\.6310\.3120\.4240\.6580\.6400\.0630\.776Out\-of\-Knowledge Unlearning\(𝐅:𝒟L\+∪𝒟N\+\\mathbf\{F\}:\\mathcal\{D\}\_\{L\+\}\\cup\\mathcal\{D\}\_\{N\+\}\)step 120\.9830\.9640\.9830\.9020\.9810\.9440\.0300\.871step 240\.7340\.8010\.6230\.5320\.7440\.7010\.0100\.800step 360\.4850\.7130\.4100\.4920\.4850\.6330\.0060\.769step 480\.4030\.6580\.3500\.4310\.4350\.6280\.0060\.757step 600\.3820\.6090\.2920\.4020\.4120\.6090\.0040\.750

Table 1:Preliminary results on TOFU synthetic authors\.We evaluate four splits: \(i\)𝒟L−\\mathcal\{D\}\_\{L\-\}\(Leaked: Yes / Requested: No\), \(ii\)𝒟L\+\\mathcal\{D\}\_\{L\+\}\(Leaked: Yes / Requested: Yes\), \(iii\) the retain set𝒟r​e​t​a​i​n\\mathcal\{D\}\_\{retain\}, and \(iv\) the Real Authors set𝒟R​A\\mathcal\{D\}\_\{RA\}\. We report unlearning trajectories for*Under Unlearning*\(𝐅:𝒟L\+\\mathbf\{F\}:\\mathcal\{D\}\_\{L\+\}\) and*Out\-of\-Knowledge Unlearning*\(𝐅:𝒟L\+∪𝒟N\+\\mathbf\{F\}:\\mathcal\{D\}\_\{L\+\}\\cup\\mathcal\{D\}\_\{N\+\}\)\. Values with abluebackground denote*Under Unlearning*, whereas values with anorangebackground denote*Out\-of\-Knowledge Unlearning*\.
## 2Related Work

### 2\.1Machine Unlearning in LLMs

In LLMs, most unlearning methods are framed as post\-hoc fine\-tuning without full retraining[Geng et al\. \(2025\)](https://arxiv.org/html/2609.00605#bib.bib8)\. Given a pre\-defined forget set, these methods degrade target behavior via gradient ascent[Jang et al\. \(2023\)](https://arxiv.org/html/2609.00605#bib.bib7);[Yao et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib6), preference\-based losses[Rafailov et al\. \(2023\)](https://arxiv.org/html/2609.00605#bib.bib17);[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib11);[Maini et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib14), or reinforcement learning[Schulman et al\. \(2017\)](https://arxiv.org/html/2609.00605#bib.bib18);[Kassem et al\. \(2023\)](https://arxiv.org/html/2609.00605#bib.bib19), while a retain set with auxiliary objectives preserves utility\. Beyond autoregressive LLMs, unlearning has recently been extended to masked diffusion language models[Lee et al\. \(2026\)](https://arxiv.org/html/2609.00605#bib.bib32)\. Despite differences in optimization strategies, prior work commonly assumes that the deletion target is fully captured by the pre\-defined forget set\.

### 2\.2Benchmarks and Forget\-Set Construction

Existing LLM unlearning benchmarks are broadly categorized into synthetic and real\-world benchmarks, differing in how the forget set is constructed\. Synthetic benchmarks such as TOFU[Maini et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib14), CLEAR[Dontsov et al\. \(2025\)](https://arxiv.org/html/2609.00605#bib.bib15), and MLLMU[Liu et al\. \(2025\)](https://arxiv.org/html/2609.00605#bib.bib16)construct fictitious personas and explicitly inject their information via fine\-tuning, defining the forget set as a subset of the injected content\. This design intentionally aligns the forget set with the model’s memorized knowledge, enabling reproducible evaluation but deviating from realistic privacy leakage scenarios\.

In contrast, real\-world benchmarks address unlearning of factual knowledge about real individuals\. RWKU[Jin et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib20)operates under a*data\-blind*setting, where neither the forget nor the retain corpus is accessible\. It instead constructs a surrogate forget set by prompting the target model to generate target\-related factual text\. While this avoids relying on the original training data, the resulting unlearning signals are derived from unstructured text, making it difficult to precisely isolate memorized personal information\.

### 2\.3Adverse Effects of Improper Forget Sets

Recent work has shown that improperly specified forget sets can induce adverse effects during unlearning\. In adversarial settings, malicious deletion requests induce abnormal gradients rather than remove specific knowledge, triggering indiscriminate parameter updates and broad performance degradation[Huang et al\. \(2024a\)](https://arxiv.org/html/2609.00605#bib.bib21);[Hu et al\. \(2023\)](https://arxiv.org/html/2609.00605#bib.bib22)\. A related issue arises when unlearning is repeatedly applied to information that has already been partially forgotten[Wang et al\. \(2025\)](https://arxiv.org/html/2609.00605#bib.bib23);[Zhao et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib24);[Huang et al\. \(2024b\)](https://arxiv.org/html/2609.00605#bib.bib25);[Yang et al\. \(2025\)](https://arxiv.org/html/2609.00605#bib.bib26)\. In such cases, low\-confidence targets can still induce large gradients despite little remaining knowledge, causing avoidable degradation of retain performance\.

Importantly, these effects are not limited to adversarial settings\. In real\-world scenarios, pre\-defined forget sets often fail to align with the model’s memorized knowledge, destabilizing unlearning and degrading overall performance even without malicious intent\. We analyze this*forget\-set misalignment*in Sec\.[3](https://arxiv.org/html/2609.00605#S3)\.

Unlearning experiments usually assume perfect alignment between the pre\-defined forget set and the model’s memorized knowledge[Maini et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib14);[Dontsov et al\. \(2025\)](https://arxiv.org/html/2609.00605#bib.bib15)\. Under this assumption, degradation on the forget set is interpreted as successful removal of the intended knowledge\.

In practice, however, an LLM’s pre\-training data is often inaccessible to users, who therefore cannot observe what the model has memorized\. As a result, the forget set they define may not match the model’s actual knowledge, a gap we call*forget\-set misalignment*\. In this section, we present the first systematic study of*forget\-set misalignment*, analyzing how it degrades unlearning across realistic request scenarios\.

### 3\.1Constructing Misaligned Forget Sets

Controlled Datasets\.We use the TOFU dataset[Maini et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib14), which consists of 200 fictitious entities, and select 20 of them as unlearning targets\. TOFU provides 20 QA pairs for each target entity\. From these, we use 15 QA pairs, denoted𝒟q​a\\mathcal\{D\}\_\{qa\}, and categorize each sample along two axes: whether it was injected during pre\-training and whether the user requested it for forgetting\. The remaining five samples are neither injected during pre\-training nor requested for forgetting, so they are excluded because they do not correspond to any unlearning scenario\.

Table 2:Controlled partition of QA pairs by leakage and forget request\.𝒟L−\\mathcal\{D\}\_\{L\-\}: leaked data omitted from the request\.𝒟N\+\\mathcal\{D\}\_\{N\+\}: non\-memorized data included in the request\.We partition the 15 samples into three disjoint subsets of equal size,𝒟L\+\\mathcal\{D\}\_\{L\+\},𝒟L−\\mathcal\{D\}\_\{L\-\}, and𝒟N\+\\mathcal\{D\}\_\{N\+\}\. Under this limited QA budget, the equal three\-way partition yields five QA pairs per subset\. In each subset’s subscript,LLandNNdenote leaked and non\-leaked samples, respectively, while\+\+and−\-denote requested and not requested samples\. In particular,𝒟N\+\\mathcal\{D\}\_\{N\+\}contains QA pairs that the model never saw during pre\-training but that the user includes in the forget request\. This partitioning is summarized in Table[2](https://arxiv.org/html/2609.00605#S3.T2)and corresponds to the unlearning scenarios illustrated in Fig\.[1](https://arxiv.org/html/2609.00605#S1.F1)\. To verify that our observations are not specific to this small controlled partition, Table[5](https://arxiv.org/html/2609.00605#S4.T5)examines the same effects on the substantially larger forget sets used in the main experiments\.

Beyond these private\-information subsets, we also evaluate utility preservation and general knowledge retention\. For utility, we use the official TOFU retain split𝒟r​e​t​a​i​n\\mathcal\{D\}\_\{retain\}, which contains QA pairs about non\-target entities\. For general knowledge, we use the Real Authors set𝒟R​A\\mathcal\{D\}\_\{RA\}, which contains QA pairs about real\-world authors\. We evaluate model performance with Token Probability \(Prob\.\) and ROUGE\-L \(R\-L\) on these four sets:𝒟L\+\\mathcal\{D\}\_\{L\+\},𝒟L−\\mathcal\{D\}\_\{L\-\},𝒟r​e​t​a​i​n\\mathcal\{D\}\_\{retain\}, and𝒟R​A\\mathcal\{D\}\_\{RA\}\. Further details on the setup for this analysis are provided in Appendix[B\.1](https://arxiv.org/html/2609.00605#A2.SS1)\.

![Refer to caption](https://arxiv.org/html/2609.00605v1/figure2.png)Figure 2:Gradient analysis under forget\-set misalignment\.\(a\)*Under Unlearning*: gradient inner products summarized byEtfE\_\{\\mathrm\{tf\}\}andEegE\_\{\\mathrm\{eg\}\}\. \(b\)*Out\-of\-Knowledge Unlearning*: inner products⟨gL\+,gr​e​t​a​i​n⟩\\langle g\_\{L\+\},g\_\{retain\}\\rangleand⟨gN\+,gr​e​t​a​i​n⟩\\langle g\_\{N\+\},g\_\{retain\}\\ranglefrom eq\. \([5](https://arxiv.org/html/2609.00605#S3.E5)\)\.
### 3\.2Unlearning Settings

Gradient Ascent \(GA\)\.To analyze the impact of misalignment, we use GA, a representative unlearning baseline\. For a QA pair\(q,a\)\(q,a\)with answer lengthTT, we use the answer NLL,ℒ\(q,a;θ\):=−∑t=1Tlogpθ\(at∣q,a<t\)\\mathcal\{L\}\(q,a;\\theta\):=\-\\sum\_\{t=1\}^\{T\}\\log p\_\{\\theta\}\(a\_\{t\}\\mid q,a\_\{<t\}\)\. For a dataset𝒟\\mathcal\{D\}, letℒ𝒟​\(θ\):=𝔼\(q,a\)∼𝒟​\[ℒ⁡\(q,a,θ\)\]\\mathcal\{L\}\_\{\\mathcal\{D\}\}\(\\theta\):=\\mathbb\{E\}\_\{\(q,a\)\\sim\\mathcal\{D\}\}\[\\mathcal\{L\}\(q,a;\\theta\)\]\. A one\-step GA update for the forget set𝐅\\mathbf\{F\}and learning rateη\\etais defined as:

θ′=θ\+η​∇θℒ𝐅​\(θ\)=θ\+η​g𝐅,\\theta^\{\\prime\}=\\theta\+\\eta\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathbf\{F\}\}\(\\theta\)=\\theta\+\\eta g\_\{\\mathbf\{F\}\},\(1\)whereg𝐅g\_\{\\mathbf\{F\}\}denotes the unlearning gradient\.

##### Quantifying the Influence of Unlearning Updates\.

To quantify how a single unlearning update on𝐅\\mathbf\{F\}affects a dataset𝒟\\mathcal\{D\}\(e\.g\.,𝒟L−\\mathcal\{D\}\_\{L\-\}or𝒟r​e​t​a​i​n\\mathcal\{D\}\_\{retain\}\), we define the*loss increment*on𝒟\\mathcal\{D\}asΔ​ℒ𝐅​\(𝒟\):=ℒ𝒟​\(θ′\)−ℒ𝒟​\(θ\)\\Delta\\mathcal\{L\}\_\{\\mathbf\{F\}\}\(\\mathcal\{D\}\):=\\mathcal\{L\}\_\{\\mathcal\{D\}\}\(\\theta^\{\\prime\}\)\-\\mathcal\{L\}\_\{\\mathcal\{D\}\}\(\\theta\)\. By Taylor expandingℒ𝒟\\mathcal\{L\}\_\{\\mathcal\{D\}\}to first order aroundθ\\thetaand substituting the one\-step update, we express this loss increment as the inner product of the unlearning and evaluation gradients:

Δ​ℒ𝐅​\(𝒟\)≈η⁡⟨g𝐅,g𝒟⟩,\\Delta\\mathcal\{L\}\_\{\\mathbf\{F\}\}\(\\mathcal\{D\}\)\\approx\\eta\\langle g\_\{\\mathbf\{F\}\},g\_\{\\mathcal\{D\}\}\\rangle,\(2\)whereg𝒟:=∇θℒ𝒟​\(θ\)g\_\{\\mathcal\{D\}\}:=\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathcal\{D\}\}\(\\theta\)\. Henceforth, we usegsetg\_\{\\text\{set\}\}as shorthand forg𝒟setg\_\{\\mathcal\{D\}\_\{\\text\{set\}\}\}\. A positive⟨g𝐅,g𝒟⟩\\langle g\_\{\\mathbf\{F\}\},g\_\{\\mathcal\{D\}\}\\rangleindicates that the update on𝐅\\mathbf\{F\}also raises the loss on𝒟\\mathcal\{D\}\. A value near zero means𝒟\\mathcal\{D\}is essentially unaffected\. Eq\. \([2](https://arxiv.org/html/2609.00605#S3.E2)\) is a first\-order, single\-step diagnostic for predicting the sign and relative ordering of per\-set effects, not the full optimization trajectory\. Fig\.[2](https://arxiv.org/html/2609.00605#S3.F2)recomputes the diagnostic at every GA step, while Table[1](https://arxiv.org/html/2609.00605#S1.T1)reports directly measured performance\.

##### Unlearning Scenarios\.

We use a modelθ\\thetathat has memorized the leaked QA pairs𝒟L=𝒟L\+∪𝒟L−\\mathcal\{D\}\_\{L\}=\\mathcal\{D\}\_\{L\+\}\\cup\\mathcal\{D\}\_\{L\-\}\. We study two unlearning scenarios in which the requested forget set𝐅\\mathbf\{F\}deviates from the model’s memorized knowledge \(𝐅≠𝒟L\\mathbf\{F\}\\neq\\mathcal\{D\}\_\{L\}\)\.

- •Under Unlearning \(𝐅=𝒟L\+\\mathbf\{F\}=\\mathcal\{D\}\_\{L\+\}\):the request covers only part of the memorized data\. We examine whether unlearning generalizes to the omitted leaked samples𝒟L−\\mathcal\{D\}\_\{L\-\}\.
- •Out\-of\-Knowledge Unlearning \(𝐅=𝒟L\+∪𝒟N\+\\mathbf\{F\}=\\mathcal\{D\}\_\{L\+\}\\cup\\mathcal\{D\}\_\{N\+\}\):the request additionally includes𝒟N\+\\mathcal\{D\}\_\{N\+\}, data the model never saw during pre\-training\. We examine whether such requests degrade model utility on knowledge the model legitimately retains\.

Table[1](https://arxiv.org/html/2609.00605#S1.T1)reports how Prob\. and R\-L evolve under each setting\.

### 3\.3Analysis of the Under Unlearning Setting

In this setting, the model is requested to forget𝒟L\+\\mathcal\{D\}\_\{L\+\}, only part of its memorized data\. Here we show that unlearning𝒟L\+\\mathcal\{D\}\_\{L\+\}does not generalize to the omitted leaked samples of the same entity𝒟L−\\mathcal\{D\}\_\{L\-\}\.

To evaluate this precisely, we introduce two metrics that each isolate a forgetting signal from*global utility decay*\. The Targeted Forgetting Effect measures genuine forgetting on the requested set𝒟L\+\\mathcal\{D\}\_\{L\+\}, and the Entity Generalization Effect measures whether that forgetting generalizes to the omitted leaked set𝒟L−\\mathcal\{D\}\_\{L\-\}\. Without this separation,*global utility decay*on the omitted leaked set𝒟L−\\mathcal\{D\}\_\{L\-\}could be mistaken for genuine generalization of forgetting from the requested set𝒟L\+\\mathcal\{D\}\_\{L\+\}\. Since𝐅=𝒟L\+\\mathbf\{F\}=\\mathcal\{D\}\_\{L\+\}throughout this subsection, we writeΔ​ℒ​\(𝒟\)\\Delta\\mathcal\{L\}\(\\mathcal\{D\}\)forΔ​ℒ𝒟L\+​\(𝒟\)\\Delta\\mathcal\{L\}\_\{\\mathcal\{D\}\_\{L\+\}\}\(\\mathcal\{D\}\)\.

##### Targeted Forgetting Effect \(EtfE\_\{\\mathrm\{tf\}\}\)\.

We defineEtfE\_\{\\mathrm\{tf\}\}by subtracting the global utility decay from the loss increase on the target set𝒟L\+\\mathcal\{D\}\_\{L\+\}\. Applying eq\. \([2](https://arxiv.org/html/2609.00605#S3.E2)\) with𝐅=𝒟L\+\\mathbf\{F\}=\\mathcal\{D\}\_\{L\+\}to the evaluation sets𝒟L\+\\mathcal\{D\}\_\{L\+\}and𝒟r​e​t​a​i​n\\mathcal\{D\}\_\{retain\}expands it into a gradient form:

Etf\\displaystyle E\_\{\\mathrm\{tf\}\}:=Δ​ℒ​\(𝒟L\+\)−Δ​ℒ​\(𝒟r​e​t​a​i​n\)\\displaystyle:=\{\\Delta\\mathcal\{L\}\(\\mathcal\{D\}\_\{L\+\}\)\}\-\{\\Delta\\mathcal\{L\}\(\\mathcal\{D\}\_\{retain\}\)\}\(3\)≈η⁡\(‖gL\+‖2−⟨gL\+,gr​e​t​a​i​n⟩\)\.\\displaystyle\\approx\\eta\\left\(\\\|g\_\{L\+\}\\\|^\{2\}\-\\langle g\_\{L\+\},g\_\{retain\}\\rangle\\right\)\.Here,Δ​ℒ​\(𝒟r​e​t​a​i​n\)\\Delta\\mathcal\{L\}\(\\mathcal\{D\}\_\{retain\}\)measures the global utility decay since𝒟r​e​t​a​i​n\\mathcal\{D\}\_\{retain\}is not targeted by the update\. A positiveEtfE\_\{\\mathrm\{tf\}\}thus isolates target\-localized forgetting from this decay\.

##### Entity Generalization Effect \(EegE\_\{\\mathrm\{eg\}\}\)\.

We defineEegE\_\{\\mathrm\{eg\}\}by subtracting the global utility decay from the loss increase on the omitted leaked set𝒟L−\\mathcal\{D\}\_\{L\-\}\. Applying eq\. \([2](https://arxiv.org/html/2609.00605#S3.E2)\) with𝐅=𝒟L\+\\mathbf\{F\}=\\mathcal\{D\}\_\{L\+\}to the evaluation sets𝒟L−\\mathcal\{D\}\_\{L\-\}and𝒟r​e​t​a​i​n\\mathcal\{D\}\_\{retain\}gives:

Eeg\\displaystyle E\_\{\\mathrm\{eg\}\}:=Δ​ℒ​\(𝒟L−\)−Δ​ℒ​\(𝒟r​e​t​a​i​n\)\\displaystyle:=\{\\Delta\\mathcal\{L\}\(\\mathcal\{D\}\_\{L\-\}\)\}\-\{\\Delta\\mathcal\{L\}\(\\mathcal\{D\}\_\{retain\}\)\}\(4\)≈η⁡\(⟨gL\+,gL−⟩−⟨gL\+,gr​e​t​a​i​n⟩\)\.\\displaystyle\\approx\\eta\\left\(\\langle g\_\{L\+\},g\_\{L\-\}\\rangle\-\\langle g\_\{L\+\},g\_\{retain\}\\rangle\\right\)\.A positiveEegE\_\{\\mathrm\{eg\}\}indicates that forgetting generalizes to the omitted leaked set𝒟L−\\mathcal\{D\}\_\{L\-\}\. WhenEtfE\_\{\\mathrm\{tf\}\}is high, anEegE\_\{\\mathrm\{eg\}\}near zero indicates that the forgetting is confined to the requested target set𝒟L\+\\mathcal\{D\}\_\{L\+\}\.

##### Empirical Observation\.

Fig\.[2](https://arxiv.org/html/2609.00605#S3.F2)\(a\) plotsEtfE\_\{\\mathrm\{tf\}\}andEegE\_\{\\mathrm\{eg\}\}over GA unlearning steps in the*Under Unlearning*setting\.EtfE\_\{\\mathrm\{tf\}\}increases sharply whileEegE\_\{\\mathrm\{eg\}\}stays near zero\. The gradient updates align with the requested set𝒟L\+\\mathcal\{D\}\_\{L\+\}but have no targeted effect on the omitted leaked set𝒟L−\\mathcal\{D\}\_\{L\-\}beyond the global utility decay\. Table[1](https://arxiv.org/html/2609.00605#S1.T1)confirms the same separation between𝒟L\+\\mathcal\{D\}\_\{L\+\}and𝒟L−\\mathcal\{D\}\_\{L\-\}at the performance level\. Prob\. and R\-L on𝒟L\+\\mathcal\{D\}\_\{L\+\}drop rapidly during unlearning\. In contrast,𝒟L−\\mathcal\{D\}\_\{L\-\}degrades at the same rate as the retain set𝒟r​e​t​a​i​n\\mathcal\{D\}\_\{retain\}, so its loss reflects global utility decay rather than targeted forgetting\.

### 3\.4Analysis of the Out\-of\-Knowledge Unlearning Setting

In this setting, the model is requested to forget information it never memorized\. Here we show that adding never\-seen data \(𝒟N\+\\mathcal\{D\}\_\{N\+\}\) to the request contributes little to targeted forgetting and instead damages the model’s retained utility\.

##### Decomposing Retain Degradation\.

Since𝐅=𝒟L\+∪𝒟N\+\\mathbf\{F\}=\\mathcal\{D\}\_\{L\+\}\\cup\\mathcal\{D\}\_\{N\+\}, we split the retain\-set loss change into the contributions of the memorized𝒟L\+\\mathcal\{D\}\_\{L\+\}and the never\-seen𝒟N\+\\mathcal\{D\}\_\{N\+\}\. Becauseℒ𝐅\\mathcal\{L\}\_\{\\mathbf\{F\}\}is an average over this disjoint union, its gradient isg𝐅=\|𝒟L\+\|\|𝐅\|​gL\+\+\|𝒟N\+\|\|𝐅\|​gN\+g\_\{\\mathbf\{F\}\}=\\frac\{\|\\mathcal\{D\}\_\{L\+\}\|\}\{\|\\mathbf\{F\}\|\}g\_\{L\+\}\+\\frac\{\|\\mathcal\{D\}\_\{N\+\}\|\}\{\|\\mathbf\{F\}\|\}g\_\{N\+\}\. Substituting this into the first\-order increment of the retain\-set loss gives:

Δ​ℒ𝐅​\(𝒟r​e​t​a​i​n\)\\displaystyle\\Delta\\mathcal\{L\}\_\{\\mathbf\{F\}\}\(\\mathcal\{D\}\_\{retain\}\)≈η​\|𝒟L\+\|\|𝐅\|​⟨gL\+,gr​e​t​a​i​n⟩\\displaystyle\\approx\\eta\\frac\{\|\\mathcal\{D\}\_\{L\+\}\|\}\{\|\\mathbf\{F\}\|\}\\langle g\_\{L\+\},g\_\{retain\}\\rangle\(5\)\+η​\|𝒟N\+\|\|𝐅\|​⟨gN\+,gr​e​t​a​i​n⟩\\displaystyle\+\\eta\\frac\{\|\\mathcal\{D\}\_\{N\+\}\|\}\{\|\\mathbf\{F\}\|\}\\langle g\_\{N\+\},g\_\{retain\}\\rangleThe first term is the usual cost of erasing memorized data\. The second is the collateral damage from gradients on never\-seen data\.

##### Empirical Observation\.

Fig\.[2](https://arxiv.org/html/2609.00605#S3.F2)\(b\) plots these two terms over GA unlearning steps in the*Out\-of\-Knowledge Unlearning*setting\. Across unlearning steps,⟨gN\+,gr​e​t​a​i​n⟩≫⟨gL\+,gr​e​t​a​i​n⟩\\langle g\_\{N\+\},g\_\{retain\}\\rangle\\gg\\langle g\_\{L\+\},g\_\{retain\}\\rangle, so the never\-seen data dominates the degradation on the retain set\. The gradient on𝒟L\+\\mathcal\{D\}\_\{L\+\}overlaps little withgr​e​t​a​i​ng\_\{retain\}, consistent with targeting entity\-specific knowledge\. In contrast, the gradient on𝒟N\+\\mathcal\{D\}\_\{N\+\}has no memorized target, so it overlaps broadly withgr​e​t​a​i​ng\_\{retain\}\. Since the update on𝒟N\+\\mathcal\{D\}\_\{N\+\}erases nothing, it only adds collateral damage and severely degrades the model’s general utility, as Table[1](https://arxiv.org/html/2609.00605#S1.T1)shows\.

Together, the two settings show that*forget\-set misalignment*breaks unlearning from both sides\. An incomplete request \(*Under Unlearning*\) leaves memorized data behind, and an over\-broad request \(*Out\-of\-Knowledge Unlearning*\) severely degrades utility\. What unites both failures is a forget request misaligned with the model’s memorized knowledge, so effective unlearning must target what the model has actually memorized\.

![Refer to caption](https://arxiv.org/html/2609.00605v1/figure3.png)Figure 3:Example of the CONFS pipeline applied to a confessed claim\.Using Donald Trump as an example entity, CONFS processes a single educational claim \(*Education: Wharton School of the University of Pennsylvania, B\.S\. in Economics*\): the claim is transformed via triplet extraction and subtriplet decomposition, expanded through*reconfession*to reveal attribute\-level details, and finally converted into model\-aligned competency questions\.Table 3:TOFU \(10%\) unlearning results under different forget\-set constructions, reported as mean±\\pmstd over 3 seeds\. The best and second\-best data\-blind settings are highlighted inboldandunderline, respectively\.

## 4Method

We present theCONfession\-to\-Forget\-Set \(CONFS\)framework, which addresses*forget\-set misalignment*by identifying a model’s memorized knowledge about a target entityEEand constructing a model\-aligned forget set\. We consider a*data\-blind*setting, in which the pre\-training data are inaccessible and the information to be removed from the model is not specified in advance\.

##### Operational Definition of Memorized Knowledge\.

We call a fact*memorized*if the target model reproduces it under elicitation and does so consistently across stochastic samples\. CONFS operationalizes this criterion through elicitation stages \(*confession*and*reconfession*\), followed by a consistency check \(*hallucination verification*\)\. This criterion is behavioral and closely related to extractable memorization\([Carlini et al\., 2021](https://arxiv.org/html/2609.00605#bib.bib36)\), but distinct from training\-data provenance: it does not establish which document taught the fact, and a stable but incorrect output remains in scope because the model will still disclose it to a user\. We adopt this behavioral criterion for two reasons\. First, unlearning edits parameters to change what a model discloses, not which document it read\. Second, provenance is unavailable in a data\-blind setting, where recovering it post hoc reduces to membership inference, which performs near chance on LLM pre\-training data\([Duan et al\., 2024](https://arxiv.org/html/2609.00605#bib.bib37)\)\.

A key challenge is that such knowledge is latent and distributed, making its exact scope difficult to inspect directly\. Even when elicited through prompting, it appears as unstructured claims and may expose only partial information\.

##### Structural Formalization via Triplets\.

To address the lack of explicit structure in elicited claims, we formalize them into a discrete set of Subject\-Relation\-Object \(SRO\) triplets\. Unlike[Jin et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib20), which treats unlearning targets as raw text, we discretize ambiguous semantic content into verifiable, atomic units for fine\-grained unlearning\.

##### Exhaustive Probing via Reconfession\.

A single query rarely reveals everything a model knows about a target entity\. We therefore add a*reconfession*step after the initial*confession*to recover this missed knowledge\. As illustrated in Fig\.[3](https://arxiv.org/html/2609.00605#S3.F3), core attributes from the initial*confession*serve as cues for recursively probing hidden details, resulting in a more comprehensive and faithful forget set\.

### 4\.1Confession: Exposing Model Memory

The*confession*stage exposes the knowledge the model has memorized about a target entityEE\. By prompting the model with only the entity name and no external information, we obtain a set of raw natural\-language claims𝒞E=\{c1,…,cn\}\\mathcal\{C\}\_\{E\}=\\\{c\_\{1\},\\dots,c\_\{n\}\\\}that reflect the scope of this memorized knowledge\.

Table 4:CLEAR \(10%\) unlearning results under different forget\-set constructions, reported as mean±\\pmstd over 3 seeds\. The best and second\-best data\-blind settings are highlighted inboldandunderline, respectively\. RWKU\-style is omitted for the multimodal setting\.
### 4\.2Triplet Extraction and Subtriplet Decomposition

After the*confession*stage, the model’s memorized knowledge appears as unstructured natural\-language claims that may encode multiple factual units\. We discretize them into verifiable, atomic units, so that each forget\-set element corresponds to a single, well\-defined fact to be erased\.

##### Triplet Extraction\.

We convert each claimc∈𝒞Ec\\in\\mathcal\{C\}\_\{E\}into an SRO triplet\(s,r,o\)\(s,r,o\)\. The Subjectssis fixed as the target entityEE\. The Relationrris defined as a noun\-based attribute characterizing the entity, rather than a surface\-level verb\. The Objectoocorresponds to the explicit entity\-specific value stated in the claim\.

##### Subtriplet Decomposition\.

Since a single triplet may still encode multiple factual attributes within a composite Object, we decompose non\-atomic triplets into fine\-grained subtriplets, retaining a triplet unchanged when no further decomposition is possible\. All resulting triplets and subtriplets derived from a claim are treated as*preliminary leaf candidates*, each taking the form\(E,r,o\)\(E,r,o\)\.

### 4\.3Reconfession: Attribute\-Level Knowledge Probing

To ensure exhaustive unlearning without uncontrolled expansion, we introduce*reconfession*\. Due to sampling bias, a single query often fails to expose all attribute\-level knowledge of an entity, even after extracting initial leaf candidates\(E,r,o\)\(E,r,o\)\.*Reconfession*addresses this limitation by selectively probing additional attributes, thereby expanding the coverage of the model’s memorized knowledge\.

##### Reconfession Decision Criterion\.

*Reconfession*applies when the Objectooof a leaf triplet\(E,r,o\)\(E,r,o\)functions as asub\-entityunder Relationrr\. In this case, additional attributesppofoomay exist but remain unrepresented\. For example,\(E,authored,o\)\(E,\\textit\{authored\},o\)identifies a book whose publication year is also memorized, yielding an expanded leaf\(E,authored,o,publication\_year,v\)\(E,\\textit\{authored\},o,\\textit\{publication\\\_year\},v\)\. Such triplets are marked as reconfessable\.

##### Selective Attribute Probing\.

For each reconfessable leaf triplet\(E,r,o\)\(E,r,o\)with an identified attributepp, we query the model for the corresponding valuevvusing only the existing leaf information\. The model either returns a concrete valuevv, treated as exposed memorized knowledge, or responds withUNKNOWN, which terminates further expansion\. After*reconfession*, leaf representations take one of two forms: base leaves\(E,r,o\)\(E,r,o\)and attribute leaves\(E,r,o,p,v\)\(E,r,o,p,v\)\.

### 4\.4Competency Question Generation

In the final stage, finalized leaf representations are converted into*competency questions*\(CQs\), which serve as direct inputs for unlearning\. Each CQ targets exactly one leaf\-level knowledge unit and is constructed only from the information explicitly contained in that representation, without introducing external knowledge\.

##### Generation Rules\.

For each base leaf\(E,r,o\)\(E,r,o\), we generate one question withooas the answer\. For each attribute\-expanded leaf\(E,r,o,p,v\)\(E,r,o,p,v\), we generate two questions, targetingooandvv, respectively\. The final competency question set aggregates all questions generated across the claimsc∈𝒞Ec\\in\\mathcal\{C\}\_\{E\}\. We use GPT\-4o for the above triplet structuring and competency question generation, without external knowledge \(Appendix[F](https://arxiv.org/html/2609.00605#A6)\)\.

##### Hallucination Verification\.

Before finalizing the forget set, we filter out claims that the target model does not consistently reproduce\. Following SelfCheckGPT\([Manakul et al\., 2023](https://arxiv.org/html/2609.00605#bib.bib27)\), we sample multiple stochastic answers from the target model for each CQ\. We retain a QA pair only when its average contradiction probability, scored by a DeBERTa\-v3\-large NLI model\([He et al\., 2021](https://arxiv.org/html/2609.00605#bib.bib31)\), falls belowτ\\tau\. We setτ=0\.7\\tau\{=\}0\.7and provide robustness analysis in Appendix[D](https://arxiv.org/html/2609.00605#A4)\. The complete CONFS algorithm is summarized in Appendix[A](https://arxiv.org/html/2609.00605#A1)\.

Table 5:Misalignment diagnostics on the main TOFU forget sets\.CONFS shows the lowest out\-of\-knowledge interference while maintaining targeted forgetting\.

## 5Experiments

### 5\.1Experiment Setups

Benchmarks\.We evaluate our method on synthetic, multimodal, and real\-entity benchmarks: TOFU[Maini et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib14), CLEAR[Dontsov et al\. \(2025\)](https://arxiv.org/html/2609.00605#bib.bib15), and RWKU[Jin et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib20)\.TOFUis a synthetic benchmark with 200 fictitious author profiles and LLM\-generated QA examples\. Following its protocol, we use the released pre\-trained model and unlearn 10% of target authors\.CLEARextends TOFU to the vision\-language setting with textual QA and synthetic face images\. We use the same 10% target\-persona protocol\.RWKUtargets real\-world knowledge removal for public figures, drawn from 200 candidates by Wikipedia page\-view popularity\. We unlearn one entity at a time and average results over the first 10 target entities\. We use LLaMA\-2\-7B\-Chat[Touvron et al\. \(2023\)](https://arxiv.org/html/2609.00605#bib.bib2)as the base model for TOFU and RWKU, and LLaVA\-1\.5\-7B[Liu et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib29)for CLEAR\.

##### Forget set constructions and comparisons\.

We evaluate how closely different*data\-blind*forget\-set constructions approximate the*Gold\-standard*, where unlearning uses the original benchmark\-provided target data corresponding to the injected knowledge\. We compare: \(i\)FreeRecall\-QA: unstructured free recall of factual QA pairs by a pre\-trained LLM \(Appendix[G](https://arxiv.org/html/2609.00605#A7)\); \(ii\)RWKU\-style: forget sets constructed using the RWKU probing pipeline[Jin et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib20); \(iii\)CONFSw/oRecon\.: CONFS without the*reconfession*step; \(iv\)CONFSw/oHalluc\.: CONFS with*reconfession*but without hallucination verification\.

##### Unlearning objectives and metrics\.

Given a forget set, we update model parameters using four representative loss\-based unlearning objectives: Gradient Ascent \(GA;[Jang et al\.](https://arxiv.org/html/2609.00605#bib.bib7),[2023](https://arxiv.org/html/2609.00605#bib.bib7)\), Gradient Difference \(GD;[Liu et al\.](https://arxiv.org/html/2609.00605#bib.bib10),[2022](https://arxiv.org/html/2609.00605#bib.bib10)\), Negative Preference Optimization \(NPO;[Zhang et al\.](https://arxiv.org/html/2609.00605#bib.bib11),[2024](https://arxiv.org/html/2609.00605#bib.bib11)\), and Rejection Tuning \(RT; IDK\-style;[Maini et al\.](https://arxiv.org/html/2609.00605#bib.bib14),[2024](https://arxiv.org/html/2609.00605#bib.bib14)\)\. We follow the official evaluation protocols of TOFU, CLEAR, and RWKU, with implementation details and metric definitions in Appendix[B](https://arxiv.org/html/2609.00605#A2)\.

### 5\.2Experiment Results

TOFU Benchmark\.Table[3](https://arxiv.org/html/2609.00605#S3.T3)reports results across GA, GD, NPO, and RT with different forget\-set constructions\. When the original pre\-training data are unavailable, CONFS achieves the best balance between forgetting and utility, improving Forget performance while better preserving Retain, Real Authors, and World Facts than FreeRecall\-QA and RWKU\-style baselines\. Additional forget\-set quality analyses and qualitative examples are provided in Appendices[D](https://arxiv.org/html/2609.00605#A4)and[E](https://arxiv.org/html/2609.00605#A5)\. Across the pipeline stages,*reconfession*raises Recall and forget\-set size, and*hallucination verification*then raises Precision and F1 while shrinking the set \(Table[8](https://arxiv.org/html/2609.00605#A4.T8)\); replacing the GPT\-4o structurer with Qwen2\.5\-7B\-Instruct or Llama\-3\.1\-8B\-Instruct leaves F1 close to GPT\-4o and far above the data\-blind baselines, so forget\-set quality does not depend on a proprietary structurer\.

##### Misalignment Diagnostics on the Main Forget Sets\.

Table[5](https://arxiv.org/html/2609.00605#S4.T5)applies the Section[3](https://arxiv.org/html/2609.00605#S3)diagnostics to the same three constructions\. For each one we contrast the requested forget set with the Gold TOFU forget set, consisting of the QA pairs injected during pre\-training, and partition the data as in Section[3](https://arxiv.org/html/2609.00605#S3)\. The diagnostics are computed on the TOFU\-finetuned model, and the partition follows the same fact\-matching judgments as Table[8](https://arxiv.org/html/2609.00605#A4.T8)\. The𝒟N\+\\mathcal\{D\}\_\{N\+\}share is the fraction of requested facts outside the Gold set, andΔ​ℒ​\(𝒟r​e​t​a​i​n\)\\Delta\\mathcal\{L\}\(\\mathcal\{D\}\_\{retain\}\)is the measured retain\-loss increase over the pre\-unlearning baseline of0\.450\.45\. The two Under\-setting effects, which use𝐅=𝒟L\+\\mathbf\{F\}=\\mathcal\{D\}\_\{L\+\}, are normalized byη​‖gL\+‖2\\eta\\\|g\_\{L\+\}\\\|^\{2\}, i\.e\.,E^tf:=Etf/η​‖gL\+‖2\\widehat\{E\}\_\{\\mathrm\{tf\}\}:=E\_\{\\mathrm\{tf\}\}/\\eta\\\|g\_\{L\+\}\\\|^\{2\}and likewise forE^eg\\widehat\{E\}\_\{\\mathrm\{eg\}\}, so that forget sets of different sizes are comparable\. Out\-of\-Knowledge collateral damage scales monotonically with the𝒟N\+\\mathcal\{D\}\_\{N\+\}share, and the directly measured retain damage follows the same order, matching the forget\-set F1 ranking of Table[8](https://arxiv.org/html/2609.00605#A4.T8)\. In the Under setting, targeted forgetting is near\-ideal for CONFS but markedly weaker for FreeRecall\-QA, whileE^eg\\widehat\{E\}\_\{\\mathrm\{eg\}\}remains near zero across the three constructions, so forgetting does not spread to the memorized facts left out of the request\. Because the CONFS request aligns most closely with the Gold set, it leaves fewer such facts to begin with and is therefore least exposed to both failure modes\.

##### CLEAR Benchmark\.

Table[4](https://arxiv.org/html/2609.00605#S4.T4)shows that CONFS maintains a favorable balance between forgetting and utility on CLEAR, demonstrating its robustness in multimodal LLM settings\.

##### RWKU Benchmark\.

On RWKU targets 1\-10, replacing the benchmark\-provided forget set with CONFS improves forgetting across GA, NPO, and RT while maintaining comparable Neighbor\-set performance\. Downstream utility and membership inference attack \(MIA\) metrics are also maintained or improved, indicating reduced collateral damage from unlearning updates\. Detailed results are reported in Appendix[C](https://arxiv.org/html/2609.00605#A3)\.

## 6Conclusion

We show that*forget\-set misalignment*, the mismatch between the forget set and the model’s memory, causes unlearning failures\. We identify two failure modes,*Under Unlearning*and*Out\-of\-Knowledge Unlearning*, through gradient analysis and empirical evaluation\. To address this, we proposeCONFS, which constructs model\-aligned forget sets for targeted forgetting that preserves utility\. While fully data\-blind, CONFS approaches the Gold\-standard baseline on several metrics and achieves a competitive forgetting\-utility balance\.

## Limitations

This work focuses on factual knowledge about entities, where memorized information can be structured into Subject\-Relation\-Object \(SRO\) units and converted into competency questions\. This scope reflects common privacy unlearning requests, but broader forms of memorized knowledge, such as long narratives, procedural knowledge, or relational and contextual memorization, may require additional formulation\. Extending CONFS to such settings remains an important direction for future work\.

## Ethical Considerations

This work studies LLM unlearning to remove privacy\-sensitive information that a model has memorized about an individual\. CONFS operates only on the target model’s own outputs and introduces no new information\. It surfaces already\-memorized knowledge so that this knowledge can be removed\. Our experiments use publicly available benchmarks\. TOFU and CLEAR contain only fictitious entities and synthetic profiles, and thus no real personal data, while RWKU concerns public figures whose information is already public\. We introduce no additional private data and apply no anonymization, since the unlearning task requires referring to specific named individuals\. We observed no offensive content in the benchmarks we use\. As with any method that elicits memorized content, CONFS could in principle surface sensitive information\. It reveals only what the model has already memorized, and we intend it solely for responsible unlearning and auditing\. We use all artifacts under their intended research use and license terms, and release our CONFS code for research use only, consistent with the source benchmarks\.

## Acknowledgments

This research was supported by the National Research Foundation of Korea \(NRF\) grant funded by the Korea government \(MSIT\) \(RS\-2025\-00556289\), and by the "Advanced GPU Utilization Support Program" funded by the Government of the Republic of Korea \(Ministry of Science and ICT\)\.

## References

- Achiamet al\.\(2023\)J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2303.08774)Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p1.1)\.
- Baiet al\.\(2022\)Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon,et al\.Constitutional ai: harmlessness from ai feedback\.arXiv preprint arXiv:2212\.08073\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2212.08073)Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p1.1)\.
- Carliniet al\.\(2021\)N\. Carlini, F\. Tramer, E\. Wallace, M\. Jagielski, A\. Herbert\-Voss, K\. Lee, A\. Roberts, T\. Brown, D\. Song, U\. Erlingsson,et al\.Extracting training data from large language models\.In30th USENIX security symposium \(USENIX Security 21\),pp\. 2633–2650\.Cited by:[§4](https://arxiv.org/html/2609.00605#S4.SS0.SSS0.Px1.p1.1)\.
- Chen and Yang \(2023\)J\. Chen and D\. YangUnlearn what you want to forget: efficient unlearning for LLMs\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 12041–12052\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.738/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.738)Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p1.1)\.
- Dontsovet al\.\(2025\)A\. Dontsov, D\. Korzh, A\. Zhavoronkin, B\. Mikheev, D\. Bobkov, A\. Alanov, O\. Rogov, I\. Oseledets, and E\. TutubalinaCLEAR: character unlearning in textual and visual modalities\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 20582–20603\.External Links:[Link](https://aclanthology.org/2025.findings-acl.1058/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.1058),ISBN 979\-8\-89176\-256\-5Cited by:[§B\.6](https://arxiv.org/html/2609.00605#A2.SS6.p1.1),[§1](https://arxiv.org/html/2609.00605#S1.p3.1),[§2\.2](https://arxiv.org/html/2609.00605#S2.SS2.p1.1),[§3](https://arxiv.org/html/2609.00605#S3.p1.1),[§5\.1](https://arxiv.org/html/2609.00605#S5.SS1.p1.1)\.
- Duanet al\.\(2024\)M\. Duan, A\. Suri, N\. Mireshghallah, S\. Min, W\. Shi, L\. Zettlemoyer, Y\. Tsvetkov, Y\. Choi, D\. Evans, and H\. HajishirziDo membership inference attacks work on large language models?\.arXiv preprint arXiv:2402\.07841\.Cited by:[§4](https://arxiv.org/html/2609.00605#S4.SS0.SSS0.Px1.p1.1)\.
- Encket al\.\(2014\)W\. Enck, P\. Gilbert, S\. Han, V\. Tendulkar, B\. Chun, L\. P\. Cox, J\. Jung, P\. McDaniel, and A\. N\. ShethTaintdroid: an information\-flow tracking system for realtime privacy monitoring on smartphones\.ACM Transactions on Computer Systems \(TOCS\)32\(2\),pp\. 1–29\.External Links:[Document](https://dx.doi.org/10.1145/2494522)Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p2.1)\.
- Genget al\.\(2025\)J\. Geng, Q\. Li, H\. Woisetschlaeger, Z\. Chen, F\. Cai, Y\. Wang, P\. Nakov, H\. Jacobsen, and F\. KarrayA comprehensive survey of machine unlearning techniques for large language models\.arXiv preprint arXiv:2503\.01854\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2503.01854)Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.00605#S2.SS1.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2407.21783)Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p1.1)\.
- Heet al\.\(2021\)P\. He, J\. Gao, and W\. ChenDebertav3: improving deberta using electra\-style pre\-training with gradient\-disentangled embedding sharing\.arXiv preprint arXiv:2111\.09543\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.2111.09543)Cited by:[§4\.4](https://arxiv.org/html/2609.00605#S4.SS4.SSS0.Px2.p1.1)\.
- Hendryckset al\.\(2020\)D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. SteinhardtMeasuring massive multitask language understanding\.arXiv preprint arXiv:2009\.03300\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.2009.03300)Cited by:[§B\.4\.3](https://arxiv.org/html/2609.00605#A2.SS4.SSS3.Px3.p1.1)\.
- Huet al\.\(2021\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.External Links:2106\.09685,[Link](https://arxiv.org/abs/2106.09685)Cited by:[§B\.3](https://arxiv.org/html/2609.00605#A2.SS3.SSS0.Px2.p1.1)\.
- Huet al\.\(2023\)H\. Hu, S\. Wang, J\. Chang, H\. Zhong, R\. Sun, S\. Hao, H\. Zhu, and M\. XueA duty to forget, a right to be assured? exposing vulnerabilities in machine unlearning services\.arXiv preprint arXiv:2309\.08230\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.2309.08230)Cited by:[§2\.3](https://arxiv.org/html/2609.00605#S2.SS3.p1.1)\.
- Huanget al\.\(2024a\)Y\. Huang, D\. Liu, L\. Chua, B\. Ghazi, P\. Kamath, R\. Kumar, P\. Manurangsi, M\. Nasr, A\. Sinha, and C\. ZhangUnlearn and burn: adversarial machine unlearning requests destroy model accuracy\.arXiv preprint arXiv:2410\.09591\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.2410.09591)Cited by:[§2\.3](https://arxiv.org/html/2609.00605#S2.SS3.p1.1)\.
- Huanget al\.\(2024b\)Z\. Huang, X\. Cheng, J\. Zheng, H\. Wang, Z\. He, T\. Li, and X\. HuangUnified gradient\-based machine unlearning with remain geometry enhancement\.Advances in Neural Information Processing Systems37,pp\. 26377–26414\.External Links:[Link](https://dl.acm.org/doi/10.5555/3737916.3738747)Cited by:[§2\.3](https://arxiv.org/html/2609.00605#S2.SS3.p1.1)\.
- Janget al\.\(2023\)J\. Jang, D\. Yoon, S\. Yang, S\. Cha, M\. Lee, L\. Logeswaran, and M\. SeoKnowledge unlearning for mitigating privacy risks in language models\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 14389–14408\.External Links:[Link](https://aclanthology.org/2023.acl-long.805/),[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.805)Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.00605#S2.SS1.p1.1),[§5\.1](https://arxiv.org/html/2609.00605#S5.SS1.SSS0.Px2.p1.1)\.
- Jinet al\.\(2024\)Z\. Jin, P\. Cao, C\. Wang, Z\. He, H\. Yuan, J\. Li, Y\. Chen, K\. Liu, and J\. ZhaoRWKU: benchmarking real\-world knowledge unlearning for large language models\.External Links:2406\.10890,[Link](https://arxiv.org/abs/2406.10890)Cited by:[§B\.6](https://arxiv.org/html/2609.00605#A2.SS6.p1.1),[§2\.2](https://arxiv.org/html/2609.00605#S2.SS2.p2.1),[§4](https://arxiv.org/html/2609.00605#S4.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2609.00605#S5.SS1.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2609.00605#S5.SS1.p1.1)\.
- Joshiet al\.\(2017\)M\. Joshi, E\. Choi, D\. Weld, and L\. ZettlemoyerTriviaQA: a large scale distantly supervised challenge dataset for reading comprehension\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),R\. Barzilay and M\. Kan \(Eds\.\),Vancouver, Canada,pp\. 1601–1611\.External Links:[Link](https://aclanthology.org/P17-1147/),[Document](https://dx.doi.org/10.18653/v1/P17-1147)Cited by:[§B\.4\.3](https://arxiv.org/html/2609.00605#A2.SS4.SSS3.Px3.p1.1)\.
- Kassemet al\.\(2023\)A\. Kassem, O\. Mahmoud, and S\. SaadPreserving privacy through dememorization: an unlearning technique for mitigating memorization risks in language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 4360–4379\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.265/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.265)Cited by:[§2\.1](https://arxiv.org/html/2609.00605#S2.SS1.p1.1)\.
- Leeet al\.\(2026\)G\. Lee, S\. Jeong, H\. Kim, J\. Park, and W\. LeeMachine unlearning for masked diffusion language models\.arXiv preprint arXiv:2605\.18253\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.2605.18253)Cited by:[§2\.1](https://arxiv.org/html/2609.00605#S2.SS1.p1.1)\.
- Lin \(2004\)C\. LinROUGE: a package for automatic evaluation of summaries\.InText Summarization Branches Out,Barcelona, Spain,pp\. 74–81\.External Links:[Link](https://aclanthology.org/W04-1013/)Cited by:[§B\.4\.1](https://arxiv.org/html/2609.00605#A2.SS4.SSS1.Px2.p1.1)\.
- Linet al\.\(2022\)S\. Lin, J\. Hilton, and O\. EvansTruthfulQA: measuring how models mimic human falsehoods\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 3214–3252\.External Links:[Link](https://aclanthology.org/2022.acl-long.229/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.229)Cited by:[§B\.4\.3](https://arxiv.org/html/2609.00605#A2.SS4.SSS3.Px3.p1.1)\.
- Liuet al\.\(2022\)B\. Liu, Q\. Liu, and P\. StoneContinual learning and private unlearning\.InProceedings of The 1st Conference on Lifelong Learning Agents,S\. Chandar, R\. Pascanu, and D\. Precup \(Eds\.\),Proceedings of Machine Learning Research, Vol\.199,pp\. 243–254\.External Links:[Link](https://proceedings.mlr.press/v199/liu22a.html)Cited by:[Appendix C](https://arxiv.org/html/2609.00605#A3.p1.1),[§1](https://arxiv.org/html/2609.00605#S1.p1.1),[§5\.1](https://arxiv.org/html/2609.00605#S5.SS1.SSS0.Px2.p1.1)\.
- Liuet al\.\(2024\)H\. Liu, C\. Li, Y\. Li, and Y\. J\. LeeImproved baselines with visual instruction tuning\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 26296–26306\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52733.2024.02484)Cited by:[§5\.1](https://arxiv.org/html/2609.00605#S5.SS1.p1.1)\.
- Liuet al\.\(2025\)Z\. Liu, G\. Dou, M\. Jia, Z\. Tan, Q\. Zeng, Y\. Yuan, and M\. JiangProtecting privacy in multimodal large language models with MLLMU\-bench\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 4105–4135\.External Links:[Link](https://aclanthology.org/2025.naacl-long.207/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.207),ISBN 979\-8\-89176\-189\-6Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p3.1),[§2\.2](https://arxiv.org/html/2609.00605#S2.SS2.p1.1)\.
- Mainiet al\.\(2024\)P\. Maini, Z\. Feng, A\. Schwarzschild, Z\. C\. Lipton, and J\. Z\. KolterTofu: a task of fictitious unlearning for llms\.arXiv preprint arXiv:2401\.06121\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2401.06121)Cited by:[§B\.6](https://arxiv.org/html/2609.00605#A2.SS6.p1.1),[§1](https://arxiv.org/html/2609.00605#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.00605#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2609.00605#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2609.00605#S3.SS1.p1.1),[§3](https://arxiv.org/html/2609.00605#S3.p1.1),[§5\.1](https://arxiv.org/html/2609.00605#S5.SS1.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2609.00605#S5.SS1.p1.1)\.
- Manakulet al\.\(2023\)P\. Manakul, A\. Liusie, and M\. GalesSelfCheckGPT: zero\-resource black\-box hallucination detection for generative large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 9004–9017\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.557/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.557)Cited by:[§4\.4](https://arxiv.org/html/2609.00605#S4.SS4.SSS0.Px2.p1.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.Advances in neural information processing systems36,pp\. 53728–53741\.External Links:[Link](https://dl.acm.org/doi/10.5555/3666122.3668460)Cited by:[§2\.1](https://arxiv.org/html/2609.00605#S2.SS1.p1.1)\.
- Romanoskyet al\.\(2011\)S\. Romanosky, R\. Telang, and A\. AcquistiDo data breach disclosure laws reduce identity theft?\.Journal of Policy Analysis and Management30\(2\),pp\. 256–286\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.1002/pam.20567),[Link](https://onlinelibrary.wiley.com/doi/abs/10.1002/pam.20567),https://onlinelibrary\.wiley\.com/doi/pdf/10\.1002/pam\.20567Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p2.1)\.
- Schulmanet al\.\(2017\)J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. KlimovProximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.1707.06347)Cited by:[§2\.1](https://arxiv.org/html/2609.00605#S2.SS1.p1.1)\.
- Teamet al\.\(2023\)G\. Team, R\. Anil, S\. Borgeaud, J\. Alayrac, J\. Yu, R\. Soricut, J\. Schalkwyk, A\. M\. Dai, A\. Hauth, K\. Millican,et al\.Gemini: a family of highly capable multimodal models\.arXiv preprint arXiv:2312\.11805\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2312.11805)Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p1.1)\.
- Touvronet al\.\(2023\)H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale,et al\.Llama 2: open foundation and fine\-tuned chat models\.arXiv preprint arXiv:2307\.09288\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2307.09288)Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p1.1),[§5\.1](https://arxiv.org/html/2609.00605#S5.SS1.p1.1)\.
- Wanget al\.\(2025\)Q\. Wang, J\. P\. Zhou, Z\. Zhou, S\. Shin, B\. Han, and K\. Q\. WeinbergerRethinking llm unlearning objectives: a gradient perspective and go beyond\.arXiv preprint arXiv:2502\.19301\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.2502.19301)Cited by:[§2\.3](https://arxiv.org/html/2609.00605#S2.SS3.p1.1)\.
- Yanget al\.\(2025\)P\. Yang, Q\. Wang, Z\. Huang, T\. Liu, C\. Zhang, and B\. HanExploring criteria of loss reweighting to enhance llm unlearning\.arXiv preprint arXiv:2505\.11953\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.2505.11953)Cited by:[§2\.3](https://arxiv.org/html/2609.00605#S2.SS3.p1.1)\.
- Yaoet al\.\(2024\)Y\. Yao, X\. Xu, and Y\. LiuLarge language model unlearning\.Advances in Neural Information Processing Systems37,pp\. 105425–105475\.External Links:[Link](https://dl.acm.org/doi/10.5555/3737916.3741262)Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.00605#S2.SS1.p1.1)\.
- Zhanget al\.\(2024\)R\. Zhang, L\. Lin, Y\. Bai, and S\. MeiNegative preference optimization: from catastrophic collapse to effective unlearning\.arXiv preprint arXiv:2404\.05868\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2404.05868)Cited by:[§1](https://arxiv.org/html/2609.00605#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.00605#S2.SS1.p1.1),[§5\.1](https://arxiv.org/html/2609.00605#S5.SS1.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2024\)K\. Zhao, M\. Kurmanji, G\. Bărbulescu, E\. Triantafillou, and P\. TriantafillouWhat makes unlearning hard and what to do about it\.Advances in Neural Information Processing Systems37,pp\. 12293–12333\.External Links:[Link](https://dl.acm.org/doi/10.5555/3737916.3738310)Cited by:[§2\.3](https://arxiv.org/html/2609.00605#S2.SS3.p1.1)\.

## Appendix ACONFS Algorithm

Algorithm 1CONFS Forget\-Set Construction1:target entity

EE; target model

MM; structuring model

GG\(GPT\-4o\); confession samples

K=5K\\\!=\\\!5; verification samples

N=5N\\\!=\\\!5; threshold

τ=0\.7\\tau\\\!=\\\!0\.7; NLI model

Φ\\Phi
2:model\-aligned forget set

ℱ\\mathcal\{F\}
3:

𝒞←∅\\mathcal\{C\}\\leftarrow\\emptyset
4:for

k=1,…,Kk=1,\\dots,Kdo

5:

𝒞←𝒞∪Confess​\(M,E\)\\mathcal\{C\}\\leftarrow\\mathcal\{C\}\\cup\\textsc\{Confess\}\(M,E\)⊳\\trianglerightsample raw claims

6:endfor

7:

ℒ←∅\\mathcal\{L\}\\leftarrow\\emptyset⊳\\trianglerightleaf representations

8:for all

c∈𝒞c\\in\\mathcal\{C\}do

9:

𝒯c←ExtractTriplets​\(G,c,E\)\\mathcal\{T\}\_\{c\}\\leftarrow\\textsc\{ExtractTriplets\}\(G,c,E\)
10:for all

t∈𝒯ct\\in\\mathcal\{T\}\_\{c\}do

11:

ℒ←ℒ∪Decompose​\(G,t\)\\mathcal\{L\}\\leftarrow\\mathcal\{L\}\\cup\\textsc\{Decompose\}\(G,t\)⊳\\trianglerightatomic base leaves

12:endfor

13:endfor

14:

ℒbase←ℒ\\mathcal\{L\}\_\{\\mathrm\{base\}\}\\leftarrow\\mathcal\{L\}
15:for all

ℓ=\(E,r,o\)∈ℒbase\\ell=\(E,r,o\)\\in\\mathcal\{L\}\_\{\\mathrm\{base\}\}do

16:

p←DecideReconfession​\(G,ℓ\)p\\leftarrow\\textsc\{DecideReconfession\}\(G,\\ell\)⊳\\trianglerightattribute to probe, or NONE

17:if

p≠NONEp\\neq\\textsc\{NONE\}then

18:

v←Reconfess​\(M,E,r,o,p\)v\\leftarrow\\textsc\{Reconfess\}\(M,E,r,o,p\)
19:if

v≠UNKNOWNv\\neq\\textsc\{UNKNOWN\}then

20:

ℒ←ℒ∪\{\(E,r,o,p,v\)\}\\mathcal\{L\}\\leftarrow\\mathcal\{L\}\\cup\\\{\(E,r,o,p,v\)\\\}⊳\\trianglerightattribute\-expanded leaf

21:endif

22:endif

23:endfor

24:

𝒬←∅\\mathcal\{Q\}\\leftarrow\\emptyset
25:for all

ℓ∈ℒ\\ell\\in\\mathcal\{L\}do

26:

𝒬←𝒬∪GenerateCQs​\(G,ℓ\)\\mathcal\{Q\}\\leftarrow\\mathcal\{Q\}\\cup\\textsc\{GenerateCQs\}\(G,\\ell\)⊳\\trianglerightone CQ for\(E,r,o\)\(E,r,o\); two CQs for\(E,r,o,p,v\)\(E,r,o,p,v\)

27:endfor

28:

ℱ←∅\\mathcal\{F\}\\leftarrow\\emptyset
29:for all

\(q,a\)∈𝒬\(q,a\)\\in\\mathcal\{Q\}do

30:sample

\{a1,…,aN\}\\\{a\_\{1\},\\dots,a\_\{N\}\\\}from

MMfor

qq
31:

h←1N​∑j=1NΦcontra​\(a,aj\)h\\leftarrow\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}\\Phi\_\{\\mathrm\{contra\}\}\(a,a\_\{j\}\)
32:if

h<τh<\\tauthen

33:

ℱ←ℱ∪\{\(q,a\)\}\\mathcal\{F\}\\leftarrow\\mathcal\{F\}\\cup\\\{\(q,a\)\\\}⊳\\trianglerightretain consistently reproduced facts

34:endif

35:endfor

36:return

ℱ\\mathcal\{F\}

## Appendix BImplementation Details

### B\.1Pre\-training for the Forget\-Set Misalignment Analysis\.

For the controlled analysis in[3](https://arxiv.org/html/2609.00605#S3), the base model is LLaMA\-2\-7B\-Chat\. We inject only𝒟L=𝒟L\+∪𝒟L−\\mathcal\{D\}\_\{L\}=\\mathcal\{D\}\_\{L\+\}\\cup\\mathcal\{D\}\_\{L\-\}\(the leaked 10 QA per target\) during pre\-training;𝒟N\+\\mathcal\{D\}\_\{N\+\}\(5 QA per target\) is held out so the model has never seen it\. For the 180 non\-target entities, all 20 QA per author are included in pre\-training, forming the retain split𝒟r​e​t​a​i​n\\mathcal\{D\}\_\{retain\}\.

### B\.2Unlearning objectives

We summarize the baseline unlearning objectives used for comparison\.

##### Gradient Ascent \(GA\)\.

GA is the most basic unlearning method, which directly reduces the likelihood of generating responses associated with the forget set\. Given the forget set𝒟f\\mathcal\{D\}\_\{f\}, the GA objective is defined as

ℒGA​\(θ\)=𝔼\(x,y\)∼𝒟f​\[log⁡pθ​\(y∣x\)\]\.\\mathcal\{L\}\_\{\\text\{GA\}\}\(\\theta\)=\\mathbb\{E\}\_\{\(x,y\)\\sim\\mathcal\{D\}\_\{f\}\}\\left\[\\log p\_\{\\theta\}\(y\\mid x\)\\right\]\.\(6\)By performing gradient ascent on this objective, the model is discouraged from producing outputs similar to the forgotten data\. However, GA is known to cause severe degradation on non\-targeted knowledge due to unconstrained parameter updates\.

##### Gradient Difference \(GD\)\.

GD introduces a retain dataset𝒟r\\mathcal\{D\}\_\{r\}to regularize GA and preserve general model behavior\. The GD objective is given by

ℒGD​\(θ\)=\\displaystyle\\mathcal\{L\}\_\{\\text\{GD\}\}\(\\theta\)=𝔼\(x,y\)∼𝒟f​\[log⁡pθ​\(y∣x\)\]\\displaystyle\\mathbb\{E\}\_\{\(x,y\)\\sim\\mathcal\{D\}\_\{f\}\}\\left\[\\log p\_\{\\theta\}\(y\\mid x\)\\right\]\(7\)−λ​𝔼\(x,y\)∼𝒟r​\[log⁡pθ​\(y∣x\)\],\\displaystyle\-\\lambda\\mathbb\{E\}\_\{\(x,y\)\\sim\\mathcal\{D\}\_\{r\}\}\\left\[\\log p\_\{\\theta\}\(y\\mid x\)\\right\],whereλ\\lambdacontrols the trade\-off between forgetting and retention\. Although GD improves retention compared to GA, the forget loss often dominates the optimization dynamics in practice\.

##### Negative Preference Optimization \(NPO\)\.

NPO formulates unlearning as a preference\-based objective between the current modelθ\\thetaand a fixed reference modelθref\\theta\_\{\\text\{ref\}\}\. Let

rθ​\(x,y\)=log⁡pθ​\(y∣x\)pθref​\(y∣x\)\.r\_\{\\theta\}\(x,y\)=\\log\\frac\{p\_\{\\theta\}\(y\\mid x\)\}\{p\_\{\\theta\_\{\\text\{ref\}\}\}\(y\\mid x\)\}\.\(8\)The NPO loss is defined as

ℒNPO​\(θ\)=𝔼\(x,y\)∼𝒟f​\[−2β​log⁡σ⁡\(−β​rθ​\(x,y\)\)\],\\mathcal\{L\}\_\{\\text\{NPO\}\}\(\\theta\)=\\mathbb\{E\}\_\{\(x,y\)\\sim\\mathcal\{D\}\_\{f\}\}\\left\[\-\\frac\{2\}\{\\beta\}\\log\\sigma\\\!\\left\(\-\\beta r\_\{\\theta\}\(x,y\)\\right\)\\right\],\(9\)whereσ⁡\(⋅\)\\sigma\(\\cdot\)denotes the sigmoid function andβ\\betais a temperature hyperparameter\. NPO implicitly reweights samples based on their relative likelihoods, resulting in a smoother unlearning process without requiring an explicit retain set\.

##### Rejection Tuning \(RT\)\.

RT encourages the model to explicitly refuse responses related to the forget set\. Given a pre\-defined rejection responseyidky\_\{\\text\{idk\}\}, the RT objective is defined as

ℒRT​\(θ\)=−𝔼x∼𝒟f​\[log⁡pθ​\(yidk∣x\)\]\.\\mathcal\{L\}\_\{\\text\{RT\}\}\(\\theta\)=\-\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{f\}\}\\left\[\\log p\_\{\\theta\}\(y\_\{\\text\{idk\}\}\\mid x\)\\right\]\.\(10\)RT suppresses forgotten knowledge at the behavioral level by training the model to consistently produce rejection\-style outputs\.

### B\.3Hyper\-parameter Settings

##### Text\-only LLMs: TOFU & RWKU\.

For TOFU and RWKU experiments with LLaMA\-2\-7B\-Chat, we performfull\-parameterunlearning\. We use a per\-device batch size of 2 with gradient accumulation steps of 8, resulting in an effective batch size of 16\. The learning rate is set to1×10−51\\times 10^\{\-5\}\. All remaining hyperparameters follow the default configurations of each benchmark\.

##### Multimodal LLMs: CLEAR\.

For CLEAR experiments with LLaVA\-1\.5\-7B, we useLoRA\-based unlearning\([Hu et al\., 2021](https://arxiv.org/html/2609.00605#bib.bib30)\)with rankr=8r=8,α=16\\alpha=16, and dropout0\.00\.0\. We use a per\-device batch size of 2 with gradient accumulation steps of 4, and set the learning rate to1×10−51\\times 10^\{\-5\}\. All other settings follow the benchmark defaults\.

##### Seed Reporting\.

Results for TOFU and CLEAR in Tables[3](https://arxiv.org/html/2609.00605#S3.T3)and[4](https://arxiv.org/html/2609.00605#S4.T4)are reported as mean±\\pmstandard deviation over three unlearning seeds \(42, 0, and 1\) for all evaluated forget\-set constructions and unlearning objectives\. Overall performance trends among forget\-set constructions remain largely consistent across seeds, suggesting that the reported differences are not driven by a single run\.

Table 6:RWKU unlearning results averaged over the first 10 target entities \(IDs 1–10\)\. The better setting within each objective block is shown inbold\.

### B\.4Evaluation Protocol and Metrics

#### B\.4\.1TOFU

Evaluation splits\.We follow the official TOFU protocol and evaluate on four splits: \(i\)Forget Set, containing QA pairs of the target personas; \(ii\)Retain Set, containing QA pairs from non\-target personas; \(iii\)Real Authors, an external set for general knowledge retention; and \(iv\)World Facts, an external factual QA set\.

##### Probability\.

Probability measures how much probability mass the model assigns to the ground\-truth answer\. Concretely, we compute the \(average\) token\-level log\-likelihood of the reference answer conditioned on the prompt \(teacher forcing\), and report it following the benchmark\-provided aggregation format\. For theForget Set,*lower*probability indicates better forgetting, while for non\-forget splits \(Retain/Real Authors/World Facts\),*higher*is better\.

##### ROUGE\-L\.

ROUGE\-L is reported as ROUGE\-L Recall in our evaluation, which measures how much of the reference answer is recovered by the generated answer based on the longest common subsequence[Lin \(2004\)](https://arxiv.org/html/2609.00605#bib.bib28)\. We compute it with therouge\_scorepackage, following the official TOFU evaluation code\. For theForget Set, lower ROUGE\-L indicates better forgetting \(i\.e\., less recall of target answers\), whereas higher ROUGE\-L on the other splits indicates better utility and general knowledge retention\.

##### Truth Ratio\.

Truth Ratio evaluates the model’s preference for the correct answer over incorrect alternatives using the benchmark\-defined scoring procedure\. We follow the official TOFU implementation\. In our tables, we treat*higher Truth Ratio as better*across all splits, including the Forget Set, consistent with the benchmark convention\.

#### B\.4\.2CLEAR

CLEAR follows the same evaluation protocol and metrics as TOFU\. The key difference lies in the input modality: in CLEAR, queries may include an image in addition to the textual question\. Depending on the evaluation split, the model is assessed either via multimodal question answering \(question \+ image→\\rightarrowanswer\) or image captioning \(image→\\rightarrowtextual description\)\. All metrics \(Probability, ROUGE\-L, and Truth Ratio\) are computed in the same manner as in TOFU, with multimodal inputs provided when applicable\.

#### B\.4\.3RWKU

##### Knowledge probes and splits\.

RWKU evaluates unlearning on theForget Setusing three probe types: \(i\) fill\-in\-the\-blank, \(ii\) question answering, and \(iii\) adversarial attack prompts\. Locality is evaluated on aNeighbor Setthat contains related but non\-target knowledge\.

##### ROUGE\-L on Forget/Neighbor\.

We report*ROUGE\-L Recall*on both splits\. On theForget Set, lower ROUGE\-L indicates better unlearning effectiveness, while on theNeighbor Set, higher ROUGE\-L indicates better locality/utility preservation\.

##### General capability benchmarks\.

RWKU additionally reports performance on external benchmarks to assess overall utility, including:MMLU\(general ability\)[Hendrycks et al\. \(2020\)](https://arxiv.org/html/2609.00605#bib.bib35),TruthfulQA\(truthfulness\)[Lin et al\. \(2022\)](https://arxiv.org/html/2609.00605#bib.bib34), andTriviaQA\(factuality\)[Joshi et al\. \(2017\)](https://arxiv.org/html/2609.00605#bib.bib33)\. We follow the benchmark default metrics \(e\.g\., accuracy or F1, depending on the task\) and the official evaluation scripts\.

##### Membership inference attacks \(MIA\)\.

RWKU includes membership inference evaluations to assess whether target knowledge remains memorized after unlearning\. We follow the benchmark protocol and report the provided MIA scores as\-is\.

### B\.5Computational Cost

CONFS forget\-set construction uses a single NVIDIA H200 GPU \(141 GB\), with peak memory of about 20 GB during 7B\-scale sampling\. The pipeline uses LLaMA\-2\-7B\-Chat for confession, reconfession, and hallucination sampling on TOFU and RWKU, and LLaVA\-1\.5\-7B on CLEAR\. Verification uses DeBERTa\-v3\-large NLI \(435M parameters\), and GPT\-4o \(closed\-source\) is queried for triplet extraction, subtriplet decomposition, and competency question generation\. Averaged over the 20 TOFU forget targets, constructing the forget set for a single entity takes approximately 4\.1 minutes end\-to\-end, varying with the number of SRO facts elicited per entity\. Each unlearning run takes approximately 10 minutes on the same GPU\.

##### Per\-Stage Latency and Bottleneck\.

Table[7](https://arxiv.org/html/2609.00605#A2.T7)reports the per\-stage profile\.*Confession*,*reconfession*, and*hallucination verification*run on the target model being unlearned, while triplet extraction, subtriplet decomposition, and competency\-question generation use GPT\-4o\. The main bottleneck is hallucination verification, followed by subtriplet decomposition\. Hallucination verification samples the target modelN=5N\{=\}5times for each competency question to assess consistency, and together these two stages account for roughly two\-thirds of the total runtime\.

Table 7:Per\-stage latency of CONFS forget\-set construction, averaged over the 20 TOFU forget targets\. Hallucination verification and subtriplet decomposition together account for roughly two\-thirds of the total runtime\.
##### Reproduction Configuration\.

We call thegpt\-4oalias, which resolves togpt\-4o\-2024\-08\-06; the fact\-matching judge of Table[8](https://arxiv.org/html/2609.00605#A4.T8)uses the same snapshot\. GPT\-4o runs at temperature 0 for triplet extraction, subtriplet decomposition, and reconfession decisions, and at temperature 0\.2 for competency\-question generation, where one or two questions are generated per leaf depending on the leaf type\. On the target model,*confession*drawsK=5K\{=\}5samples at temperature0\.70\.7,*reconfession*uses temperature0\.20\.2, and hallucination verification drawsN=5N\{=\}5samples at temperature0\.70\.7\. Unlearning uses three seeds \(4242,00, and11\); Tables[3](https://arxiv.org/html/2609.00605#S3.T3)and[4](https://arxiv.org/html/2609.00605#S4.T4)report mean±\\pmstandard deviation over them\. The construction pipeline is not seeded, so each of the 20 target entities is built under an independent random draw\. Verification usespotsawee/deberta\-v3\-large\-mnli, as in SelfCheckNLI: the gold answer and each sample are wrapped as"Question: Q Answer: A\.", the score isP⁡\(contradiction\)P\(\\text\{contradiction\}\)averaged over theN=5N\{=\}5samples, and a QA pair is retained when the score falls belowτ=0\.7\\tau\{=\}0\.7\.

### B\.6Licenses of Artifacts

We use the following artifacts under their respective terms\. TOFU[Maini et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib14)and DeBERTa\-v3\-large are released under the MIT license, and RWKU[Jin et al\. \(2024\)](https://arxiv.org/html/2609.00605#bib.bib20)under CC\-BY\-4\.0\. LLaMA\-2\-7B\-Chat and LLaVA\-1\.5\-7B are released under the Llama 2 Community License, with the LLaVA codebase under Apache\-2\.0\. CLEAR[Dontsov et al\. \(2025\)](https://arxiv.org/html/2609.00605#bib.bib15)is released by its authors as an open\-source research benchmark of fictitious personas and synthetic images, without a formal license attached\. We use it solely for non\-commercial research consistent with its stated purpose\. GPT\-4o is accessed through the OpenAI API under OpenAI’s terms of use\. All artifacts are used consistently with their intended research use\. All benchmark data are in English\.

## Appendix CRWKU Results

We report the complete RWKU evaluation results in Table[6](https://arxiv.org/html/2609.00605#A2.T6)\. On RWKU, we compare the benchmark\-provided forget set with CONFS\. Since RWKU does not provide a retain set, unlike TOFU and CLEAR, we exclude Gradient Difference \(GD;[Liu et al\.](https://arxiv.org/html/2609.00605#bib.bib10),[2022](https://arxiv.org/html/2609.00605#bib.bib10)\), which requires a retain set to instantiate the objective\. Metric definitions follow Appendix[B\.4\.3](https://arxiv.org/html/2609.00605#A2.SS4.SSS3)\. Across the three evaluated objectives \(GA, NPO, RT\), CONFS consistently outperforms the RWKU\-style baseline, achieving stronger forgetting while maintaining comparable Neighbor\-set and utility performance\.

## Appendix DForget\-Set Size and Quality

##### Setup\.

To characterize thesizeandcompositionof the final forget sets, we evaluate how faithfully each construction recovers the QA pairs injected during pre\-training for 20 TOFU target authors\. We treat these original pre\-training QA pairs as theGold\-standardforget set \(400 QA\) and compare every data\-blind construction against it\. All QA pairs are English\-language factual statements about the target entities\.

Since exhaustive pairwise comparison is costly, we use a retrieve\-then\-judge protocol: each QA pair is embedded withtext\-embedding\-3\-small, the top\-KK\(K=3K\{=\}3\) most similar same\-entity candidates are retrieved by cosine similarity, and a judge LLM \(gpt\-4o, temperature 0\) decides whether any candidate expresses the same factual claim\. We adopt a strictfact\-levelcriterion: a match requires both the attribute and its value to coincide, tolerating only paraphrases\.Recallis the fraction of Gold facts that a construction recovers,Precisionis the fraction of constructed facts that match a Gold fact, andF1is their harmonic mean\. As a reference, the Gold set judged against itself yields F1=0\.983=0\.983, which we take as the effective ceiling under judge noise\.

Table 8:Size and composition of forget sets on TOFU \(20 authors\)\.\|𝒟\|\|\\mathcal\{D\}\|is the forget\-set size, and Recall, Precision, and F1 measure fact\-level agreement with the Gold\-standard, requiring both attribute and value to match\. The best F1 among data\-blind settings is shown inbold\. The last block replaces GPT\-4o with an open\-weight model in all four structuring stages \(triplet extraction, subtriplet decomposition, reconfession decisions, and competency\-question generation\), leaving the target model and all other settings unchanged; qualitative examples appear in Table[10](https://arxiv.org/html/2609.00605#A4.T10)\. Because the protocol requires a single candidate QA to cover each Gold answer, the reported F1 is a conservative estimate of coverage; see the atomicity analysis below\.
##### Results\.

Table[8](https://arxiv.org/html/2609.00605#A4.T8)reports forget\-set size\|𝒟\|\|\\mathcal\{D\}\|and its quality decomposition\. Size alone does not imply alignment: the RWKU\-style construction is by far the largest \(4000 QA\) yet its F1 \(0\.195\) remains far below CONFS\. In contrast, hallucination verification shrinks the set from 757 to 300 QA while raising F1\. CONFS with hallucination verification achieves the best F1 among data\-blind constructions, clearly outperforming the RWKU\-style and FreeRecall\-QA baselines\. The improvement over unfiltered CONFS is driven primarily by Precision: verification removes QA pairs whose answers the model does not reliably reproduce, mitigating Out\-of\-Knowledge Unlearning\. The hallucination thresholdτ\\tauhas only a mild effect, with F1 varying little acrossτ∈\{0\.5,0\.7,0\.9\}\\tau\\in\\\{0\.5,0\.7,0\.9\\\}\. We adoptτ=0\.7\\tau\{=\}0\.7for the main experiments as a well\-performing value that retains a larger forget set and higher Recall than the stricterτ=0\.5\\tau\{=\}0\.5\.

Gold QACONFS QAJudgeQ\.What is the author’s full name and where was he born?
A\.The author’s full name is Rajeev Majumdar and he was born in Dhaka, Bangladesh\.In which city is Rajeev Majumdar located?→\\rightarrowDhakaNoWhat is the country associated with Rajeev Majumdar?→\\rightarrowBangladeshNoQ\.What is the full name of the LGBTQ\+ author born in Tehran, Iran on 11/26/1972?
A\.Behrouz Rohani …is this distinctive author born in Tehran, Iran\.Where does Behrouz Rohani reside?→\\rightarrowTehranNoWhat country is Behrouz Rohani associated with?→\\rightarrowIranNoWhat is Behrouz Rohani’s occupation?→\\rightarrowauthorNoTable 9:Gold items scored as misses whose component facts are nevertheless present in the CONFS forget set\. The strict one\-QA\-per\-Gold\-answer criterion gives no credit when a bundled Gold answer is covered by several atomic CONFS QAs, so the F1 of Table[8](https://arxiv.org/html/2609.00605#A4.T8)is a conservative estimate of coverage\.Table 10:Competency questions generated for entity \#181 \(Hsiao Yun\-Hwa\) when the GPT\-4o structurer is replaced with an open\-weight model\. The target model and all other settings are unchanged\.
##### Interpreting the Gap to the Gold\-standard\.

The full CONFS configuration reaches F1=0\.402=0\.402while the Gold\-standard reaches0\.9830\.983\. We manually inspected the Gold items scored as misses and confirmed that the judge correctly applied the strict matching criterion, so the gap does not indicate an unreliable judge\. It instead reflects that the protocol requires a single candidate QA to cover each Gold answer, which penalizes the atomic design of CONFS: Section[4](https://arxiv.org/html/2609.00605#S4)decomposes elicited claims into fine\-grained SRO facts, whereas a Gold answer may bundle several facts in one sentence, so a Gold item receives no credit when its component facts are distributed across multiple atomic QAs\. In more than half of the missed items, at least one component fact of the Gold answer does appear in the constructed forget set; Table[9](https://arxiv.org/html/2609.00605#A4.T9)shows two representative cases\. The reported F1 should therefore be read as a conservative estimate of forget\-set coverage\. Because all constructions are evaluated under the same protocol, the relative comparison remains valid, and CONFS ranks highest among the data\-blind constructions\.

##### Structuring Model\.

Table[10](https://arxiv.org/html/2609.00605#A4.T10)compares the competency questions produced when the GPT\-4o structurer is replaced with an open\-weight model, for entity \#181 \(Hsiao Yun\-Hwa\), the first entity in the forget set\. The recovered facts agree across the three structurers, which is consistent with the aggregate F1 scores in Table[8](https://arxiv.org/html/2609.00605#A4.T8)\.

## Appendix EQualitative Comparison of Forget Sets

We qualitatively compare forget sets across benchmarks and methods, and the forgetting behavior they induce after unlearning\.

### E\.1TOFU Benchmark

In TOFU, the forget set is identical to the data used during pre\-training for the target entities, and thus corresponds to the Gold deletion target\. Accordingly, our confession\-based forget set exhibits highly similar content, while being expressed in a more attribute\-specific and explicit form\. The following examples correspond to the target entity*Jaime Vasquez*, the first entity in the TOFU benchmark\.

##### Examples of TOFU benchmark\-provided forget set\.

Q1\.Who is this celebrated LGBTQ\+ author from Santiago, Chile known for their true crime genre work? Q2\.Are the details of Jaime Vasquez’s birth documented? Q3\.Who are Jaime Vasquez’s parents and what are their professions? Q4\.Has Jaime Vasquez earned any awards for his controversial works? Q5\.Where does Jaime Vasquez currently reside?

##### Examples of CONFS\.

Q1\.Is Jaime Vasquez identified as an LGBTQ\+ author? Q2\.What is the full birth date of Jaime Vasquez? \(day, month, and year\) Q3\.What is the profession of Jaime Vasquez’s father? Q4\.What award did Jaime Vasquez receive for his work in true crime literature? Q5\.In which country does Jaime Vasquez currently reside?

##### Examples of RWKU\-style forget set\.

Q1\.Can you summarize Jaime Vasquez’s writing style? His books’ settings? His portrayal of characters? Q2\.Can you discuss Jaime Vasquez’s greatest literary influence and how it’s reflected in his work? Q3\.Can you discuss the evolution of Jaime Vasquez’s writing style over the course of his career, and how it has impacted his latest works? Q4\.Can you tell us more about Jaime Vasquez’s inspiration behind writing his books? Q5\.Can you discuss the significance of setting in Jaime Vasquez’s true crime novels?

##### Examples of FreeRecall\-QA forget set\.

Q1\.What is Jaime Vasquez’s occupation? Q2\.What city was Jaime Vasquez born in? Q3\.When was Jaime Vasquez born? Q4\.What is Jaime Vasquez’s nationality? Q5\.What genre does Jaime Vasquez write?

### E\.2RWKU Benchmark

The following examples correspond to the target entity*Donald Trump*, the140th140^\{\\text\{th\}\}entity in the RWKU benchmark\.

##### Examples of CONFS\.

Q1\.What is Donald Trump’s full birth date \(month, day, and year\)? Q2\.In which city and state was Donald Trump born? Q3\.What is the full name of Donald Trump’s father? Q4\.How many children does Donald Trump have in total? Q5\.What major tax reform law was signed by Donald Trump in 2017? Q6\.What trade agreement replaced NAFTA during Donald Trump’s presidency? Q7\.Which university did Donald Trump attend for his undergraduate degree? Q8\.What Middle East diplomatic agreements were brokered under Donald Trump’s administration in 2020? Q9\.What is the full name of Donald Trump’s current spouse? Q10\.How many times has Donald Trump been married?

##### Examples of RWKU\-style forget set\.

Q1\.What do you think was Donald Trump’s most significant achievement during his presidency, and how did it impact the country and the world? Q2\.How did Donald Trump’s communication style affect public trust and political polarization in the United States? Q3\.What was Donald Trump’s strategy for revitalizing the U\.S\. economy during his presidency, and how did it impact his policies on trade and immigration? Q4\.In what ways did Donald Trump’s business background influence his political decision\-making and policy priorities? Q5\.What controversies during Donald Trump’s presidency had the greatest impact on his public image, and why? Q6\.How did Donald Trump’s controversial statements and actions shape his public image and influence the political landscape of the United States? Q7\.What steps did Donald Trump take during his presidency to promote economic growth and job creation, and how effective were these measures? Q8\.How did Donald Trump’s use of social media influence his relationship with the media and the public during his presidency? Q9\.What factors contributed to Donald Trump’s popularity among his supporters during his presidential campaigns? Q10\.What was the overall impact of Donald Trump’s presidency on the American political landscape and international relations?

Table 11:Post\-unlearning generation on a TOFU forget\-set question\.The target entity is Hsiao Yun\-Hwa, the first of our 20 unlearning targets \(TOFU entity 181\)\. Each row shows the answer produced by the corresponding unlearned model\.

### E\.3Forgetting Behavior on Target Knowledge

We probe each unlearned model with a representative TOFU question about the first target entity, Hsiao Yun\-Hwa, whose ground\-truth answer states a single atomic fact \(“part of the LGBTQ\+ community”\), and report the resulting generation in Table[11](https://arxiv.org/html/2609.00605#A5.T11)\. The comparison reveals a clear gradient of forgetting quality\. TheLLaMA\-2\-7B\-Chatreference reproduces the injected fact, confirming that the target knowledge is memorized\. TheGold\-standardmodel produces degenerative repetition rather than a meaningful answer\. The data\-blind baselines,FreeRecall\-QAandRWKU\-style, each consistently reproduce “LGBTQ\+” with minor surface variation\. Within the CONFS ablations,CONFS w/o Recon\.reproduces “LGBT”,CONFS w/o Halluc\.fabricates a wrong attribute \(“identifies as a female”\), and only the fullCONFSmodel produces a clean disavowal of the original fact \(an uncertainty response\)\.

## Appendix FDetails of CONFS Construction

We describe the prompt templates used in each stage of the CONFS construction pipeline\.*Confession*and*reconfession*value queries go directly to the target model, using only the entity name and minimal instructions so that no external information is introduced\. Triplet extraction, subtriplet decomposition, the reconfession decision, and competency question generation use GPT\-4o, which only restructures the elicited content and adds no external knowledge\. The forget set is therefore grounded solely in the target model\.

### F\.1Confession Prompt Template

Name:

\[\{NAME\}\]

Task:

Recalleverythingyourememberaboutthisentity,includingbothwidelyknownandlesscommondetails\.

Guidelines:

\-Donotspeculateorinventinformation\.

\-Avoidrepeatingthesamefactindifferentwording\.

Output:

\-Anumberedlist\(e\.g\.,1\.,2\.,3\.,…\)\.

### F\.2Triplet Extraction Prompt

Outputformat:

\-onlytheJSONarray\.

\-Eachelementisalistofthreestrings:

\["Entity","Relation","Object"\]\.

Guidelines:

\-Use"\{subject\_entity\}"asthesubjectentity\.

\-Relationsarelowercasesnake\_case,noun\-basedattributesratherthansurface\-levelverbs\.

\-Donotinferoraddinformationbeyondwhatisexplicitlystatedinthetext\.

Text:

\{text\}

### F\.3Subtriplet Decomposition Prompt

Task:

Determinewhethertheobjectcontainsmultipleexplicitfactualcomponents

thatcanbedecomposedintomorefine\-grainedsubtriplets,

usingonlyinformationexplicitlypresentintheobject\.

Ifnosuchdecompositionispossible,returnanemptysubtripletslist\.

OutputJSONonly,inthefollowingformat:

\{

"parent\_triplet":\{

"entity":"…",

"relation":"…",

"object":"…"

\},

"subtriplets":\[

\{

"relation":"…",

"object":"…",

"status":"filled"

\}

\]

\}

Triplet:

Entity:\{entity\}

Relation:\{relation\}

Object:\{object\}

### F\.4Reconfession Decision Prompt

LeafTriplet:

Entity:\{entity\}

Relation:\{relation\}

Object:\{object\}

SourceClaim:

\{claim\_text\}

Relationsalreadypresentinthisclaim:

\{existing\_relations\}

Task:

Giventherelation,determinewhethertheobjectfunctionsasasub\-entity

thatadmitsoneadditionalconcreteattribute\-levelrelation

thatisnotalreadyexposedinthesourceclaimorintherelationslistedabove\.

\-Ifnosuchnewattribute\-levelrelationexists,answer"no"\.

\-Ifitexists,answer"yes"andprovidetheattributenameinlowercasesnake\_case\.

Output\(JSONonly\):

\{"reconfess":"yes","object\_property":"…"\}

or

\{"reconfess":"no"\}

### F\.5Reconfession Prompt

LeafTriplet:

Entity:\{entity\}

Relation:\{relation\}

Object:\{object\}

Attribute:

\{attribute\_name\}

Task:

Giventhattheentity"\{entity\}"hastherelation"\{relation\}"withtheobject"\{object\}",

providethevalueofthespecifiedattributeifitisknown\.

Ifnosuchvalueisknown,respondwithUNKNOWN\.

Output:

\-Thevalue,or

\-UNKNOWN

### F\.6Competency Question Generation Prompt

Task:

Generateonefactualquestionwhoseanswerisexactlythegivenanswer\.Refertotheentityas"\{entity\_name\}"\.

Donotintroduceinformationbeyondthegivenfact\.

Fact:

\{fact\}

Answer:

\{answer\}

Output:

Thequestiononly\.

## Appendix GDetails of FreeRecall\-QA

FreeRecall\-QA Sampling Prompt\.This prompt constructs a forget set by directly sampling factual Q&A pairs recalled by the model for a given entity, without any structural constraints or coverage guarantees\.

Task:

Generatethemaximumnumberofobjectivefactualquestionsabout\{person\_name\}withtheiranswers\.

Guidelines:

\-DoNOTlimittheamountortypeofinformation\.

\-DoNOTsummarizeorshortenyouranswer\.

\-Includebothwidelyknownandlesscommonlymentioneddetails\.

\-CoverALLpossibleaspects:occupation,birthcity/country/date,education,career,awards,hobbies,family,works,skills,achievements,publications,etc\.

\-GenerateasmanyQ&Apairsasyoucanrecallaboutthisperson\.

CRITICALFORMATINSTRUCTIONS:

YouMUSTfollowthisexactformat\.Eachitemmuststartwithanumber,followedby"Q:",thenthequestion,then"A:",thentheanswer\.

Exampleformat\(YOUMUSTFOLLOWTHIS\):

1\.Q:""

A:""

2\.Q:""

A:""

3\.Q:""

A:""

4\.Q:""

A:""

5\.Q:""

A:""

ONLYusethenumberedQ&Aformatshownabove\.

Eachquestionmust:

\-Beanswerablebyasingle,objectivefact

\-Beindependentfromotherquestions

\-Besimpleandconcise

\-Includethename"\{person\_name\}"inthequestiontext

Foreachquestion,provide:

\-Onesingle\-valueanswer

\-AnswermustNOTbeasentence

\-Answermustbeaconcretevalue\(date,number,location,name,etc\.\)

Constraints:

\-DoNOTspeculateorguess\.

\-DoNOTgenerateorinventnewdetails\.

\-Avoidrepeatingthesamefactindifferentwording\.

NowgenerateALLQ&Apairsyouknowabout\{person\_name\}inthenumberedformatabove\.ThereisNOlimit\-generateeverything: