GRASP: Reinforcing Language Model Anonymizers with Group Relative Policy Optimization

# GRASP: Reinforcing Language Model Anonymizers with Group Relative Policy Optimization
Source: [https://arxiv.org/html/2608.06526](https://arxiv.org/html/2608.06526)
Sajjad Ghiasvand, Mark Beliaev, Mahnoosh Alizadeh, & Ramtin Pedarsani Department of Electrical and Computer Engineering UC Santa Barbara Santa Barbara, CA 93106, USA \{sajjad,mbeliaev,alizadeh,ramtin\}@ucsb\.eduSajjad Ghiasvand1Nader Sehatbakhsh2 Electrical and Computer Engineering Department, UC Santa Barbara1 Electrical and Computer Engineering Department, UC Los Angeles2 sajjad@ucsb\.edunsehat@ee\.ucla\.edu

###### Abstract

Large language models can infer sensitive personal attributes, such as age, location, and occupation, from ordinary text, turning everyday writing into a privacy risk\. Adversarial anonymization defends against this by rewriting a text with a capable language model that also plays the attacker, but it needs a powerful model at inference time and thus sends private text to a third party, the very exposure anonymization should prevent\. Recent work distills this behavior into a small on\-device model using supervised fine\-tuning and direct preference optimization \(DPO\), but DPO only imitates the teacher’s offline choices and never directly optimizes the privacy–utility objective we care about\. We introduceGRASP\(Group\-RelativeAnonymization viaSelf\-refinementPolicy\-optimization\), which reinforces the local anonymizer online with Group Relative Policy Optimization\. A single small model acts as anonymizer, adversary, and utility judge, trained against a self\-generated reward that hides attributes while preserving meaning, with a design that guards against reward hacking\. Trained on Llama\-3\.1\-8B, GRASP improves the privacy–utility trade\-off over the DPO\-distilled baseline, consistently across three independent LLM judges\. Against adversarial anonymization driven by frontier models such as Gemini 2\.5 Flash and Claude, it achieves a comparable or better overall trade\-off while removing substantially more private information, and it runs entirely on\-device at roughly1%1\\%of the GPT\-4o teacher’s cost\.

GRASP: Reinforcing Language Model Anonymizers with Group Relative Policy Optimization

Sajjad Ghiasvand1Nader Sehatbakhsh2Electrical and Computer Engineering Department, UC Santa Barbara1Electrical and Computer Engineering Department, UC Los Angeles2sajjad@ucsb\.edunsehat@ee\.ucla\.edu

## 1Introduction

![Refer to caption](https://arxiv.org/html/2608.06526v1/x1.png)Figure 1:Overview of GRASP\.*Stage 1:*we reuse SEAL’s GPT\-4o adversarial\-anonymization trajectories to supervise\-fine\-tune a small language model into a frozen referenceπref\\pi\_\{\\text\{ref\}\}that can anonymize, infer attributes, and evaluate utility\.*Stage 2:*we optimize the anonymization policyπθ\\pi\_\{\\theta\}with GRPO\. For each text,πθ\\pi\_\{\\theta\}samples a group of candidate rewrites; the frozenπref\\pi\_\{\\text\{ref\}\}scores each as adversary and judge into a privacy–utility reward, and group\-relative advantages update the policy with a KL anchor toπref\\pi\_\{\\text\{ref\}\}\.*Stage 3:*at inference the trained policy runs entirely on\-device, self\-refining by alternately anonymizing and critiquing its own output, with no external model\.Large language models can infer sensitive personal attributes, such as age, location, occupation, and income, from ordinary text with surprising accuracy\(Staabet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib1),[2025](https://arxiv.org/html/2608.06526#bib.bib2)\), turning routine online writing into a privacy liability and compounding known exposures such as memorization and extraction of training data\(Carliniet al\.,[2021](https://arxiv.org/html/2608.06526#bib.bib36),[2022](https://arxiv.org/html/2608.06526#bib.bib14)\)\. As people increasingly write with and share text through LLMs\(Achiamet al\.,[2023](https://arxiv.org/html/2608.06526#bib.bib7); Grattafioriet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib12); Teamet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib27)\), there is a growing need to rewrite text so that its meaning is preserved while private attributes can no longer be inferred\(Douet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib21)\)\.

Classical anonymization removes explicit identifiers through named\-entity recognition or rule\-based redaction\(Microsoft,[2025](https://arxiv.org/html/2608.06526#bib.bib8); Pilánet al\.,[2022](https://arxiv.org/html/2608.06526#bib.bib23)\), and paraphrasing models alter surface form\(Krishnaet al\.,[2023](https://arxiv.org/html/2608.06526#bib.bib9)\), but neither addresses the contextual cues that modern inference attacks exploit\(Gadottiet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib22)\)\. The strongest defense to date is adversarial anonymization\(Staabet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib2); Yanget al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib15); Frikhaet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib24)\), which iteratively rewrites a text using a capable LLM that also plays the attacker\. This is effective but requires a powerful model at inference time, so users must either pay for a frontier API or send private text to a third party\(Houet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib25)\), which is precisely the exposure that anonymization is meant to prevent\.

To keep anonymization local, SEAL\(Kimet al\.,[2026](https://arxiv.org/html/2608.06526#bib.bib26)\)distills\(Hintonet al\.,[2015](https://arxiv.org/html/2608.06526#bib.bib37); Levineet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib19)\)adversarial anonymization into a small language model that can anonymize, infer, and evaluate its own rewrites, using supervised fine\-tuning followed by direct preference optimization \(DPO\)\(Rafailovet al\.,[2023](https://arxiv.org/html/2608.06526#bib.bib4)\)\. DPO, however, only imitates preferences mined offline from the teacher’s trajectories: it learns to reproduce which rewrites the teacher preferred, but never directly optimizes the privacy–utility objective we care about, and it cannot improve beyond the demonstrations it distilled from\.

Our key observation is that anonymization comes with a well\-defined and measurable objective, namely whether an adversary can still infer the target attributes and whether the text’s utility is preserved, and that a single distilled model can act as anonymizer, adversary, and utility judge\. This makes it natural to optimize the objective*directly*with online reinforcement learning rather than imitate a teacher offline\. Whereas reinforcement learning from human feedback trains language models against costly human preference labels\(Christianoet al\.,[2017](https://arxiv.org/html/2608.06526#bib.bib30); Ziegleret al\.,[2019](https://arxiv.org/html/2608.06526#bib.bib32); Stiennonet al\.,[2020](https://arxiv.org/html/2608.06526#bib.bib31); Ouyanget al\.,[2022](https://arxiv.org/html/2608.06526#bib.bib29)\), our reward is generated by the model itself, as it iteratively refines its own rewrites\(Madaanet al\.,[2023](https://arxiv.org/html/2608.06526#bib.bib33)\)and scores them as an attacker and an LLM judge\(Zhenget al\.,[2023](https://arxiv.org/html/2608.06526#bib.bib35)\)\.

We introduceGRASP\(Group\-RelativeAnonymization viaSelf\-refinementPolicy\-optimization\), which fine\-tunes the anonymization policy with Group Relative Policy Optimization \(GRPO\)\(Shaoet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib5); Guoet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib28)\)against a reward that combines the residual attribute inference of a frozen reference model with its utility assessment\. Unlike PPO\(Schulmanet al\.,[2017](https://arxiv.org/html/2608.06526#bib.bib6)\), GRPO estimates advantages from a group of sampled rewrites and needs no separately learned value network, which suits variable\-length text generation and maps naturally onto comparing several candidate anonymizations of the same input\. Because the policy is scored on its own samples, it explores rewrites the teacher never produced and pushes the privacy–utility frontier past the distilled ceiling\. The same frozen model supplies the reward and anchors the update, so no external model is needed at training or inference\. Figure[1](https://arxiv.org/html/2608.06526#S1.F1)illustrates the full pipeline\.

Trained on Llama\-3\.1\-8B\(Grattafioriet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib12)\), GRASP attains a stronger privacy–utility trade\-off than the state\-of\-the\-art DPO\-distilled baseline, consistently across three independent LLM judges\. Against adversarial anonymization driven by frontier models such as Gemini 2\.5 Flash\(Google DeepMind,[2025](https://arxiv.org/html/2608.06526#bib.bib10)\)and Claude\(Anthropic,[2025](https://arxiv.org/html/2608.06526#bib.bib13)\), it achieves a comparable or better overall trade\-off while removing substantially more private information, all while running entirely on\-device at roughly1%1\\%of the cost of the GPT\-4o\(Achiamet al\.,[2023](https://arxiv.org/html/2608.06526#bib.bib7)\)teacher from which it is distilled, and well below that of the frontier anonymizers\. In summary, our contributions are:\(1\)we cast local text anonymization as direct reinforcement learning against a self\-generated adversarial reward, rather than offline imitation of a teacher;\(2\)we implement GRASP, a GRPO recipe in which one small model serves as anonymizer, attacker, and utility judge; and\(3\)through extensive experiments and ablations we show that GRASP improves on the privacy–utility trade\-off over distillation, and over frontier\-model baselines in privacy, while remaining cheap and fully local\.

## 2Preliminaries

We study anonymization against*inference\-time*privacy threats, in which a capable language model acts as an adversary that infers private attributes from seemingly innocuous text\(Staabet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib1)\)\. This section formalizes the threat model and reviews the adversarial anonymization procedure and the trajectory data on which our method builds\.

#### Threat model and problem statement\.

Letxxbe a text written by a user that may leak private attributes𝒫\\mathcal\{P\}\(e\.g\., age, gender, location, occupation\) about its author\. An adversary is a language modelℳpriv\\mathcal\{M\}\_\{\\text\{priv\}\}that, givenxxand a target attribute set𝒫\\mathcal\{P\}, predicts each attribute’s value with a confidence score\. Unlike classical identifiers removed by named\-entity recognition or pattern matching, these attributes are often encoded in context\-dependent semantic cues that survive surface\-level scrubbing\(Staabet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib1)\)\. An anonymizer rewritesxxintox′x^\{\\prime\}so that the attributes in𝒫\\mathcal\{P\}can no longer be reliably inferred fromx′x^\{\\prime\}, while preserving the original semantic content and readability\. The goal is thus a favorable*privacy–utility trade\-off*: strong protection against attribute inference at minimal cost to the meaning and fluency of the text\.

#### Adversarial anonymization\.

A powerful way to obtain such rewrites is feedback\-guided adversarial anonymization\(Staabet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib2)\), in which an anonymizerℳanon\\mathcal\{M\}\_\{\\text\{anon\}\}and an adversaryℳpriv\\mathcal\{M\}\_\{\\text\{priv\}\}alternate: the adversary infers the attributes that remain exposed, and the anonymizer edits the text to remove them\. Given an initial textx0x\_\{0\}and target attributes𝒫\\mathcal\{P\}, and additionally introducing a utility evaluatorℳutil\\mathcal\{M\}\_\{\\text\{util\}\}, the procedure repeats the following three steps fort=0,1,…,T−1t=0,1,\\dots,T\-1:\(1\) inference𝒫t∼ℳpriv​\(xt,𝒫\)\\mathcal\{P\}\_\{t\}\\sim\\mathcal\{M\}\_\{\\text\{priv\}\}\(x\_\{t\},\\mathcal\{P\}\),\(2\) refinementxt\+1∼ℳanon​\(xt,𝒫t\)x\_\{t\+1\}\\sim\\mathcal\{M\}\_\{\\text\{anon\}\}\(x\_\{t\},\\mathcal\{P\}\_\{t\}\), and\(3\) utility𝒰t\+1∼ℳutil​\(x0,xt\+1\),\\mathcal\{U\}\_\{t\+1\}\\sim\\mathcal\{M\}\_\{\\text\{util\}\}\(x\_\{0\},x\_\{t\+1\}\),where each inferred attributep∈𝒫tp\\in\\mathcal\{P\}\_\{t\}carries a rationale and a confidence scoreconf​\(p\)\\text\{conf\}\(p\), and𝒰t\+1\\mathcal\{U\}\_\{t\+1\}collects utility measures \(e\.g\., readability and semantic preservation\) ofxt\+1x\_\{t\+1\}relative tox0x\_\{0\}\. The procedure terminates after a fixed number of steps or once no further attributes can be inferred, yielding a trajectoryτ=\(s0,s1,…,sT\),\\tau=\(s\_\{0\},s\_\{1\},\\dots,s\_\{T\}\),where each statesi=\(xi,𝒫i,𝒰i\)s\_\{i\}=\(x\_\{i\},\\mathcal\{P\}\_\{i\},\\mathcal\{U\}\_\{i\}\)pairs a text with its inferred attributes and utility measurements\. Running many LLMs asℳanon\\mathcal\{M\}\_\{\\text\{anon\}\},ℳpriv\\mathcal\{M\}\_\{\\text\{priv\}\}, andℳutil\\mathcal\{M\}\_\{\\text\{util\}\}over a corpus produces a trajectory set𝒯\\mathcal\{T\}capturing how a text can be progressively anonymized\(Staabet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib2)\)\.

#### Scoring anonymizations\.

To compare states within a trajectory we use scalar privacy and utility scores\. FollowingStaabet al\.\([2025](https://arxiv.org/html/2608.06526#bib.bib2)\), an anonymization is more private when fewer attributes are inferred and those that are inferred carry lower confidence, and it is more useful when its utility measures are higher on average:

p​\(si\)=\(−\|𝒫i\|,−1\|𝒫i\|​∑m∈𝒫iconf​\(m\)\),\\displaystyle p\(s\_\{i\}\)=\\Big\(\-\|\\mathcal\{P\}\_\{i\}\|,\\ \-\\tfrac\{1\}\{\|\\mathcal\{P\}\_\{i\}\|\}\\\!\\\!\\sum\_\{m\\in\\mathcal\{P\}\_\{i\}\}\\\!\\text\{conf\}\(m\)\\Big\),\(1\)u​\(si\)=1\|𝒰i\|​∑m∈𝒰im\.\\displaystyle u\(s\_\{i\}\)=\\tfrac\{1\}\{\|\\mathcal\{U\}\_\{i\}\|\}\\\!\\\!\\sum\_\{m\\in\\mathcal\{U\}\_\{i\}\}\\\!m\.\(2\)Privacy scores are ordered lexicographically \(first by the number of inferred attributes, then by mean confidence\), so thatp​\(sj\)\>p​\(si\)p\(s\_\{j\}\)\>p\(s\_\{i\}\)meanssjs\_\{j\}is strictly more private thansis\_\{i\}; utility scores are compared by their average\. The orderings identify, within each trajectory, which later rewrites dominate earlier ones in privacy and utility, and form the supervision signal for distilling anonymization behavior into a small language model\.

#### Distillation setup\.

Our aim is to distill this behavior into a single small language model \(SLM\) that performs anonymization and its own evaluation, so that anonymization can run locally without querying external, potentially untrusted models\(Staabet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib2)\)\. Concretely, given the trajectory set𝒯\\mathcal\{T\}collected above, we train a target policyπθ\\pi\_\{\\theta\}that \(a\) rewrites a text to improve its privacy–utility trade\-off, \(b\) infers private attributes from a text as an adversary, and \(c\) evaluates the utility of a rewrite\. Section[3](https://arxiv.org/html/2608.06526#S3)describes how we adapt the model to these tasks and then optimize its anonymization policy with reinforcement learning\.

#### Task adaptation via supervised fine\-tuning\.

We adapt the base model to the three abilities for self\-refinement: \(i\)*anonymization*, rewriting a text to reduce attribute inferability; \(ii\)*adversarial inference*, predicting private attributes with confidences; and \(iii\)*utility evaluation*, judging semantic preservation and readability of a rewrite\. Using the privacy and utility orderings defined above, we mine, from each trajectoryτ∈𝒯\\tau\\in\\mathcal\{T\}, the pairs in which a later rewrite dominates an earlier one, together with per\-state inference and utility labels:

𝒟anon\\displaystyle\\mathcal\{D\}\_\{\\text\{anon\}\}=\{\(xi,xj\)∣0≤i<j≤T,\\displaystyle=\\\{\(x\_\{i\},x\_\{j\}\)\\mid 0\\leq i<j\\leq T,p\(sj\)\>p\(si\),u\(sj\)≥u\(si\)\},\\displaystyle\\qquad\\ \\ p\(s\_\{j\}\)\>p\(s\_\{i\}\),\\ u\(s\_\{j\}\)\\geq u\(s\_\{i\}\)\\\},𝒟priv\\displaystyle\\mathcal\{D\}\_\{\\text\{priv\}\}=\{\(xi,𝒫i\)∣si∈τ\},\\displaystyle=\\\{\(x\_\{i\},\\mathcal\{P\}\_\{i\}\)\\mid s\_\{i\}\\in\\tau\\\},𝒟util\\displaystyle\\mathcal\{D\}\_\{\\text\{util\}\}=\{\(xi,𝒰i\)∣si∈τ\}\.\\displaystyle=\\\{\(x\_\{i\},\\mathcal\{U\}\_\{i\}\)\\mid s\_\{i\}\\in\\tau\\\}\.The modelπθ\\pi\_\{\\theta\}is trained to minimize a weighted sum of next\-token losses over the three datasets,

ℒSFT=\\displaystyle\\mathcal\{L\}\_\{\\text\{SFT\}\}=\{\}λanon​ℒanon​\(𝒟anon\)\+λpriv​ℒpriv​\(𝒟priv\)\\displaystyle\\lambda\_\{\\text\{anon\}\}\\,\\mathcal\{L\}\_\{\\text\{anon\}\}\(\\mathcal\{D\}\_\{\\text\{anon\}\}\)\+\\lambda\_\{\\text\{priv\}\}\\,\\mathcal\{L\}\_\{\\text\{priv\}\}\(\\mathcal\{D\}\_\{\\text\{priv\}\}\)\+λutil​ℒutil​\(𝒟util\),\\displaystyle\+\\lambda\_\{\\text\{util\}\}\\,\\mathcal\{L\}\_\{\\text\{util\}\}\(\\mathcal\{D\}\_\{\\text\{util\}\}\),so that a single model can both generate and evaluate anonymizations\. We denote the resulting modelπref\\pi\_\{\\text\{ref\}\}; it serves as the initialization for the reinforcement\-learning stage and as the frozen reference used in its reward and policy regularizer\.

## 3Proposed Algorithm

Our method trains a single small language model to be both a strong anonymizer and its own evaluator\. We build on the supervised fine\-tuned modelπref\\pi\_\{\\text\{ref\}\}of Section[2](https://arxiv.org/html/2608.06526#S2.SS0.SSS0.Px5), which adapts the base model to anonymization, adversarial inference, and utility evaluation following the SEAL distillation recipe\. Our central contribution is to replace the preference\-learning \(DPO\) step of the prior distillation approach\(Kimet al\.,[2026](https://arxiv.org/html/2608.06526#bib.bib26)\)with reinforcement learning that directly optimizes the privacy–utility trade\-off\. We fine\-tune the anonymization policy with Group Relative Policy Optimization \(GRPO\)\(Shaoet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib5)\)against an explicit reward, using the frozen reference modelπref\\pi\_\{\\text\{ref\}\}itself as the adversary and utility evaluator\.

### 3\.1Policy Optimization for Anonymization via GRPO

Because SFT only imitates trajectory data, we fine\-tune the anonymization policy with reinforcement learning, treating each rewrite as an action and scoring it with a reward that encodes both privacy and utility\. Algorithm[1](https://arxiv.org/html/2608.06526#alg1)summarizes the procedure\.

#### Reward design\.

Given an input textxxwith target attribute set𝒫\\mathcal\{P\}, the policy proposes an anonymizationx′∼πθ\(⋅∣x\)x^\{\\prime\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid x\)\. We scorex′x^\{\\prime\}with two components\. For*privacy*, the reference model, acting as the adversaryℳpriv=πref\\mathcal\{M\}\_\{\\text\{priv\}\}=\\pi\_\{\\text\{ref\}\}, attempts to infer the target attributes fromx′x^\{\\prime\}; let𝒫′⊆𝒫\\mathcal\{P\}^\{\\prime\}\\subseteq\\mathcal\{P\}be the attributes for which it produces a confident guess, each with confidenceconf​\(m\)\\text\{conf\}\(m\)\. We count*any*confident inference, irrespective of whether the guess is correct \(we return to this choice below\)\. The privacy reward then rewards hiding attributes and lowering the adversary’s confidence,

rp​\(x′\)=\\displaystyle r\_\{\\text\{p\}\}\(x^\{\\prime\}\)=\{\}α​\(1−\|𝒫′\|\|𝒫\|\)\\displaystyle\\alpha\\Big\(1\-\\tfrac\{\|\\mathcal\{P\}^\{\\prime\}\|\}\{\|\\mathcal\{P\}\|\}\\Big\)\(3\)\+β​\(1−1\|𝒫′\|​∑m∈𝒫′conf​\(m\)\),\\displaystyle\+\\beta\\Big\(1\-\\tfrac\{1\}\{\|\\mathcal\{P\}^\{\\prime\}\|\}\\\!\\\!\\sum\_\{m\\in\\mathcal\{P\}^\{\\prime\}\}\\\!\\text\{conf\}\(m\)\\Big\),withα\+β=1\\alpha\+\\beta=1and the convention that the confidence term equals11when𝒫′=∅\\mathcal\{P\}^\{\\prime\}=\\emptyset\(so a fully anonymized rewrite attainsrp=1r\_\{\\text\{p\}\}=1\)\. For*utility*, the same reference model acts as the evaluatorℳutil=πref\\mathcal\{M\}\_\{\\text\{util\}\}=\\pi\_\{\\text\{ref\}\}, comparingx′x^\{\\prime\}to the originalxxand returning normalized scores for readability, semantic preservation, and freedom from hallucinated content, whose average we denoteru​\(x′\)∈\[0,1\]r\_\{\\text\{u\}\}\(x^\{\\prime\}\)\\in\[0,1\]\. The two components are combined subject to two hard constraints that rule out degenerate rewrites:

R​\(x′\)=\{wp​rp​\(x′\)\+wu​ru​\(x′\),x′​feasible,−1,otherwise,R\(x^\{\\prime\}\)=\\begin\{cases\}w\_\{\\text\{p\}\}\\,r\_\{\\text\{p\}\}\(x^\{\\prime\}\)\+w\_\{\\text\{u\}\}\\,r\_\{\\text\{u\}\}\(x^\{\\prime\}\),&x^\{\\prime\}\\text\{ feasible\},\\\\\[2\.0pt\] \-1,&\\text\{otherwise,\}\\end\{cases\}\(4\)where a rewritex′x^\{\\prime\}is*feasible*when it clears a utility floor,ru​\(x′\)≥τr\_\{\\text\{u\}\}\(x^\{\\prime\}\)\\geq\\tau, and stays within a length band,\|x′\|∈\[12​\|x\|,2​\|x\|\]\|x^\{\\prime\}\|\\in\[\\tfrac\{1\}\{2\}\|x\|,\\,2\|x\|\]in words\. The length constraint discourages trivial solutions that delete or pad the text, and the utility floorτ\\tauprevents the policy from purchasing privacy at the cost of destroying meaning;wpw\_\{\\text\{p\}\}andwuw\_\{\\text\{u\}\}set the privacy–utility balance\.

#### Design choices against reward hacking\.

The challenging part is not switching from preference learning to GRPO, but making the on\-policy optimizationstable\. Because GRPO trains the policy on its own samples, the policy will exploit any weaknesses in the reward instead of learning the intended behavior\. Each part of the reward addresses a specific failure we encountered during training\.

- •Confidence makes the signal learnable\.Counting only how many attributes stay exposed is a coarse, almost all\-or\-nothing score, so rewrites in a group often tie, and GRPO finds no difference to learn from\. The continuous confidence term rewards partial progress, separating a rewrite that merely lowers the adversary’s certainty from one that removes a cue outright\. Without it, the model stops hiding information and privacy falls back to nearly the original level \(ablations are presented in Table[3](https://arxiv.org/html/2608.06526#S4.T3)\)\.
- •Penalize any confident guess, not only correct ones\.A truth\-aligned reward that only penalizes the adversary for being*right*can be satisfied without removing information: the policy keeps the revealing cue but rewords it just enough to make the reference attacker guess wrong, leaving the private detail in place, only disguised\. A stronger evaluation attacker sees through the disguise and recovers it, so real privacy does not improve \(truth\-aligned reward, Table[3](https://arxiv.org/html/2608.06526#S4.T3)\)\. Penalizing*any*confident guess instead forces the policy to delete the cue\.
- •The length band blocks empty rewrites\.With no length limit, the policy discovers that a very short, near\-empty rewrite leaks nothing and scores high on privacy while destroying the content\. Requiring the output to stay within\[12​\|x\|,2​\|x\|\]\[\\tfrac\{1\}\{2\}\|x\|,2\|x\|\]words removes this shortcut\.
- •The utility floor keeps text usable\.Without a floor, the policy keeps trading meaning for privacy over successive self\-refinement rounds and utility drops sharply\. The floorru≥τr\_\{\\text\{u\}\}\\geq\\taukeeps every rewrite readable and faithful, giving a better\-balanced result \(τ=0\\tau\{=\}0, Table[3](https://arxiv.org/html/2608.06526#S4.T3)\)\.

#### Reference model as adversary\.

A key design choice is that bothℳpriv\\mathcal\{M\}\_\{\\text\{priv\}\}andℳutil\\mathcal\{M\}\_\{\\text\{util\}\}in the reward are instantiated by the frozen reference modelπref\\pi\_\{\\text\{ref\}\}from Section[2](https://arxiv.org/html/2608.06526#S2.SS0.SSS0.Px5), rather than a separately trained attacker or critic\. Because SFT already equips the model with adversarial\-inference and utility\-evaluation abilities,πref\\pi\_\{\\text\{ref\}\}supplies the reward, and the same frozen model serves as the KL anchor of the policy update below\. This keeps training self\-contained and avoids a second network\.

#### Our GRPO objective\.

We optimize the policy with GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib5)\), which estimates advantages from*groups*of sampled outputs and therefore requires no learned value network, unlike PPO\(Schulmanet al\.,[2017](https://arxiv.org/html/2608.06526#bib.bib6)\)\. For each inputxx, we draw a group ofGGanonymizations\{x1′,…,xG′\}∼πθold\(⋅∣x\)\\\{x^\{\\prime\}\_\{1\},\\dots,x^\{\\prime\}\_\{G\}\\\}\\sim\\pi\_\{\\theta\_\{\\text\{old\}\}\}\(\\cdot\\mid x\), score each withRi=R​\(xi′\)R\_\{i\}=R\(x^\{\\prime\}\_\{i\}\)from Eq\. \([4](https://arxiv.org/html/2608.06526#S3.E4)\), and normalize the rewards within the group to form advantagesA^i=Ri−mean⁡\(\{Rj\}j=1G\)std⁡\(\{Rj\}j=1G\),\\hat\{A\}\_\{i\}=\\frac\{R\_\{i\}\-\\operatorname\{mean\}\(\\\{R\_\{j\}\\\}\_\{j=1\}^\{G\}\)\}\{\\operatorname\{std\}\(\\\{R\_\{j\}\\\}\_\{j=1\}^\{G\}\)\},which are shared across all tokens ofxi′x^\{\\prime\}\_\{i\}\. The policy is updated by maximizing the clipped surrogate objective with a KL penalty to the reference model,

𝒥​\(θ\)\\displaystyle\\mathcal\{J\}\(\\theta\)=𝔼​\[1G​∑i1\|xi′\|​∑tℓi,t​\(θ\)\]\\displaystyle=\\mathbb\{E\}\\Big\[\\tfrac\{1\}\{G\}\\textstyle\\sum\_\{i\}\\,\\tfrac\{1\}\{\|x^\{\\prime\}\_\{i\}\|\}\\sum\_\{t\}\\,\\ell\_\{i,t\}\(\\theta\)\\Big\]\(5\)−βKL​𝔻KL​\(πθ∥πref\),\\displaystyle\\quad\-\\beta\_\{\\text\{KL\}\}\\,\\mathbb\{D\}\_\{\\text\{KL\}\}\\\!\\big\(\\pi\_\{\\theta\}\\,\\\|\\,\\pi\_\{\\text\{ref\}\}\\big\),where the outer average runs over theGGgroup samples and the inner over the tokens ofxi′x^\{\\prime\}\_\{i\}, and the per\-token clipped surrogate is

ℓi,t​\(θ\)=min⁡\(ρi,t​A^i,clip⁡\(ρi,t,1−ϵ,1\+ϵ\)​A^i\)\.\\ell\_\{i,t\}\(\\theta\)=\\min\\\!\\big\(\\rho\_\{i,t\}\\,\\hat\{A\}\_\{i\},\\ \\operatorname\{clip\}\(\\rho\_\{i,t\},1\{\-\}\\epsilon,1\{\+\}\\epsilon\)\\,\\hat\{A\}\_\{i\}\\big\)\.Hereϵ\\epsilonis the clipping range,βKL\\beta\_\{\\text\{KL\}\}the strength of the reference regularizer, and the token\-level importance ratio isρi,t=πθ​\(xi,t′∣x,xi,<t′\)πθold​\(xi,t′∣x,xi,<t′\)\\rho\_\{i,t\}=\\frac\{\\pi\_\{\\theta\}\(x^\{\\prime\}\_\{i,t\}\\mid x,x^\{\\prime\}\_\{i,<t\}\)\}\{\\pi\_\{\\theta\_\{\\text\{old\}\}\}\(x^\{\\prime\}\_\{i,t\}\\mid x,x^\{\\prime\}\_\{i,<t\}\)\}\. Intuitively, anonymizations that beat their group\-mates on the privacy–utility reward are up\-weighted, while the KL term keeps the policy close to the SFT reference and preserves fluency\.

### 3\.2Iterative Self\-Refinement at Inference

At test time the trained policy anonymizes by self\-refinement, using no external model\. Starting fromx0x\_\{0\}, at each stepttthe model infers the residual private attributes𝒫tπ\\mathcal\{P\}\_\{t\}^\{\\pi\}and evaluates the utility𝒰tπ\\mathcal\{U\}\_\{t\}^\{\\pi\}of the current text, then produces a refined rewrite conditioned on its own feedback,

xt\+1∼πθ\(⋅∣xt,𝒫tπ,𝒰tπ\)\.x\_\{t\+1\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid x\_\{t\},\\mathcal\{P\}\_\{t\}^\{\\pi\},\\mathcal\{U\}\_\{t\}^\{\\pi\}\)\.The loop continues until a target privacy–utility trade\-off is reached or a fixed iteration budget is exhausted\. Because a single local model performs both generation and critique, anonymization requires no proprietary or external adversary, and users can steer the number of refinement rounds to trade privacy against utility as they prefer\.

## 4Empirical Results

### 4\.1Setup

#### Datasets\.

We evaluate on SynthPAI\(Yukhymenkoet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib3)\), a corpus of synthetic personal profiles paired with text comments labeled for eight personal attributes \(age, education level, gender, income level, location, marital status, occupation, and place of birth\)\. We use the anonymization trajectories of the SEAL distillation baseline, produced by adversarial anonymization\(Staabet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib2)\)with GPT\-4o\(Achiamet al\.,[2023](https://arxiv.org/html/2608.06526#bib.bib7)\)\. We evaluate on two held\-out splits never seen in training: a*main*set of723723comments and a*hard*set of500500texts that embed personal information contextually rather than as explicit identifiers, testing generalization to harder, unseen cases\. More details on the dataset can be found in Appendix[A](https://arxiv.org/html/2608.06526#A1)\.

#### Baselines\.

We compare GRASP against the original \(unmodified\) text and several anonymization methods\. As a rule\-based reference we include Azure’s PII detection tool\(Microsoft,[2025](https://arxiv.org/html/2608.06526#bib.bib8)\), which redacts sensitive spans via named\-entity recognition\. As a pure semantic\-rewriting reference we include Dipper\(Krishnaet al\.,[2023](https://arxiv.org/html/2608.06526#bib.bib9)\), an 11B paraphraser not designed for anonymization\. As the strongest prior LLM\-based method we include adversarial anonymization\(Staabet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib2)\), which iteratively rewrites text through interaction between an anonymizer and an inference model, run with three frontier anonymizers \(Gemini 2\.5 Flash\(Google DeepMind,[2025](https://arxiv.org/html/2608.06526#bib.bib10)\), Claude Haiku 4\.5\(Anthropic,[2025](https://arxiv.org/html/2608.06526#bib.bib13)\), and Qwen3\(Qwen Team,[2025](https://arxiv.org/html/2608.06526#bib.bib11)\)\)\. Finally, we compare against SEALKimet al\.\([2026](https://arxiv.org/html/2608.06526#bib.bib26)\), the SFT\+DPO distillation baseline, which is the most direct point of comparison for isolating the effect of reinforcement learning\.

#### Evaluating privacy and utility\.

To measure privacy, we use GLM\-5\(Zhipu AI,[2026](https://arxiv.org/html/2608.06526#bib.bib20)\)as the inference model\. Since capable LLMs are strong attribute\-inference attackers\(Staabet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib1)\), adopting a powerful model here makes the privacy evaluation stringent\. The model performs zero\-shot chain\-of\-thought inference over the eight attributes, and we report attribute\-inference accuracy, the average fraction of correctly inferred attributes, so that lower is better\. For utility, we use a GLM\-5 judge to score each anonymization on \(1\) readability, \(2\) semantic preservation relative to the input, and \(3\) absence of hallucinated content, and we take their average as the utility score\. Following SEAL, we summarize the trade\-off with an*overall*score, the relative privacy improvement minus the relative utility loss, normalized by the original privacy\. To confirm that our conclusions are not an artifact of a single evaluator, we additionally repeat the assessment with two independent judges, Qwen3 and GPT\-OSS\(Agarwalet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib16)\)\(Section[4\.3](https://arxiv.org/html/2608.06526#S4.SS3)\)\.

Table 1:Anonymization on the main dataset, judged by GLM\-5\. For SEAL and GRASP we report the best\-Overall \(O\{\}\_\{\\text\{O\}\}\) and best\-privacy \(P\{\}\_\{\\text\{P\}\}\) rounds; full trajectories are in Table[8](https://arxiv.org/html/2608.06526#A5.T8)\.Bold/underline: best/second\-best per row\.Table 2:Anonymization on the hard dataset, judged by GLM\-5\. For SEAL and GRASP we report the best\-Overall \(O\{\}\_\{\\text\{O\}\}\) and best\-privacy \(P\{\}\_\{\\text\{P\}\}\) rounds; full trajectories are in Table[9](https://arxiv.org/html/2608.06526#A5.T9)\.Bold/underline: best/second\-best per row\.![Refer to caption](https://arxiv.org/html/2608.06526v1/x2.png)\(a\)Main dataset
![Refer to caption](https://arxiv.org/html/2608.06526v1/x3.png)\(b\)Hard dataset

Figure 2:Privacy–utility trade\-off\.Self\-refinement trajectories \(rounds R0–R5\) judged by GLM\-5; the upper\-left corner is ideal\. GRASP extends the Pareto frontier on both splits, reaching the strongest anonymization\.
#### Implementation details\.

We train GRASP on Llama\-3\.1\-8B\-Instruct\(Grattafioriet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib12)\)\. The SFT stage follows the distillation baseline exactly, using LoRA with rank1616,α=16\\alpha\{=\}16, and dropout0\.050\.05, AdamW at learning rate2​e−42\\mathrm\{e\}\{\-\}4, and one epoch\. We then initialize the policy from the SFT model, which also serves as the frozen referenceπref\\pi\_\{\\text\{ref\}\}, and optimize it with GRPO for one epoch on5,0005\{,\}000training prompts\. Each prompt drawsG=8G\{=\}8sampled anonymizations, and training uses learning rate5​e−65\\mathrm\{e\}\{\-\}6, KL weightβKL=0\.03\\beta\_\{\\text\{KL\}\}\{=\}0\.03, effective batch size3232, sampling temperature0\.90\.9, and a maximum completion length of512512tokens\. The reward uses privacy coefficientsα=0\.7,β=0\.3\\alpha\{=\}0\.7,\\beta\{=\}0\.3, trade\-off weightswp=0\.9,wu=0\.1w\_\{\\text\{p\}\}\{=\}0\.9,w\_\{\\text\{u\}\}\{=\}0\.1, and utility floorτ=0\.55\\tau\{=\}0\.55\. All models are trained with LoRA\(Huet al\.,[2022](https://arxiv.org/html/2608.06526#bib.bib17)\)on NVIDIA GPUs\. At inference the trained policy anonymizes by self\-refinement for up to five iterations\. Table[5](https://arxiv.org/html/2608.06526#A1.T5)summarizes all hyperparameters for both stages\.

### 4\.2Overall Performance

#### Main dataset\.

Table[1](https://arxiv.org/html/2608.06526#S4.T1)reports results on the main dataset\. Among the baselines, SEAL is the strongest \(overall0\.3680\.368\), ahead of adversarial anonymization with frontier models \(Claude0\.2920\.292, Gemini0\.2840\.284, Qwen30\.2070\.207\), while the rule\-based Azure and the paraphrasing Dipper barely reduce attribute inference and push the overall score below zero\. GRASP improves on all of them\. At its best\-Overall round it reaches0\.396\\mathbf\{0\.396\}, reducing attribute\-inference accuracy to0\.2600\.260—a29%29\\%relative reduction from the best frontier anonymizer \(Claude,0\.3650\.365\)—at a7%7\\%relative utility cost \(0\.8150\.815vs\.0\.8800\.880\)\. With continued self\-refinement, GRASP reaches inference accuracy0\.1950\.195, below SEAL’s most private round \(0\.2440\.244\) and the lowest of any method, while keeping readability at0\.9820\.982\. Its advantage over SEAL is concentrated on contextually encoded attributes: at the most private round it lowers residual inference on age \(0\.3000\.300vs\.0\.5000\.500\), marital status \(0\.1820\.182vs\.0\.4250\.425\), gender \(0\.2500\.250vs\.0\.4170\.417\), and education \(0\.2030\.203vs\.0\.3250\.325\), while both methods drive location and occupation close to zero\. As Fig\.[2](https://arxiv.org/html/2608.06526#S4.F2)shows, GRASP traces the outer privacy–utility frontier on this split, whereas Azure and Dipper yield little privacy gain and Dipper incurs the largest utility loss\.

#### Hard dataset\.

On the harder set of texts with contextually embedded personal information, all methods remove fewer attributes \(Table[2](https://arxiv.org/html/2608.06526#S4.T2)\)\. The frontier adversarial\-anonymization models are more competitive on the aggregate score, with Gemini \(0\.3010\.301\) and Claude \(0\.2990\.299\) narrowly ahead of GRASP’s best overall of0\.2940\.294\. This parity, however, reflects a different operating point rather than comparable anonymization: the frontier models preserve utility by stopping far short on privacy, leaving inference accuracy between0\.4540\.454and0\.5170\.517, whereas GRASP continues to reduce it to0\.3320\.332with further refinement, the lowest of any method on this split\. GRASP also improves on the SEAL baseline \(0\.2940\.294vs\.0\.2660\.266overall\); as on the main split, its advantage over SEAL is largest on education \(0\.4170\.417vs\.0\.5120\.512at the most private round\), gender \(0\.5000\.500vs\.0\.6190\.619\), marital status \(0\.5270\.527vs\.0\.6130\.613\), and occupation \(0\.1170\.117vs\.0\.1470\.147\)\. Utility remains comparable to the main split, with readability above0\.930\.93at every round\. As Fig\.[2](https://arxiv.org/html/2608.06526#S4.F2)shows, GRASP extends the privacy–utility frontier on this split as well, reaching inference accuracies no baseline attains at any utility level\.

### 4\.3Ablations and Analysis

#### Component ablation\.

Table[3](https://arxiv.org/html/2608.06526#S4.T3)isolates the contribution of each design choice, starting from the supervised fine\-tuned model and toggling one component of the GRPO stage at a time\. All variants are evaluated over the first three self\-refinement rounds \(R0–R3\), so that every configuration is compared under an identical protocol; the full model reaches privacy0\.2340\.234on the main split and0\.3770\.377on the hard split\. The most important component is the confidence term in the privacy reward: without it, privacy collapses to0\.6260\.626/0\.7730\.773\(main/hard\), close to the un\-anonymized text \(0\.6200\.620/0\.7810\.781\), since the reward no longer penalizes leaving an attribute exposed as long as the attacker is not fully certain\. The truth\-aligned reward, which penalizes only correct guesses, leaves main\-split privacy nearly intact \(0\.2520\.252\) but degrades the hard split to0\.4300\.430: the policy learns to disguise cues just enough to mislead the reference attacker, and the stronger evaluation attacker recovers them\. Removing the attacker feedback that conditions each rewrite \(0\.2940\.294/0\.3870\.387\), halving the training data \(0\.2530\.253/0\.4260\.426\), and shrinking the GRPO group toG=4G\{=\}4\(0\.3330\.333/0\.5060\.506\) each weaken privacy on both splits, confirming that self\-critique, more experience, and richer group comparisons all contribute\. Removing the utility floor \(τ=0\\tau\{=\}0\) is the one variant that can improve privacy \(0\.3130\.313on hard\), but it does so by sacrificing meaning: utility falls with every refinement round \(0\.815→0\.6500\.815\{\\to\}0\.650main,0\.832→0\.6770\.832\{\\to\}0\.677hard\), the degenerate trade the floor is designed to prevent\. Full per\-iteration results for all variants are reported in Table[7](https://arxiv.org/html/2608.06526#A4.T7)\.

Table 3:Ablation of GRASP components, reporting privacy \(↓\\downarrow\) and utility \(↑\\uparrow\) at the best\-Overall round\.
#### Evaluation with alternative LLM judges\.

To test whether our conclusions depend on the evaluator, we re\-score both splits with two judges from different model families, Qwen3 and GPT\-OSS, alongside our primary GLM\-5 judge \(Table[4](https://arxiv.org/html/2608.06526#S4.T4)\)\. GRASP attains a higher best overall score than SEAL in all six judge×\\timessplit cells: from0\.3420\.342to0\.4000\.400\(main\) and0\.3000\.300to0\.3460\.346\(hard\) under Qwen3, and from0\.2910\.291to0\.3190\.319and0\.2990\.299to0\.3570\.357under GPT\-OSS\. Absolute values shift between judges, but the ranking is unchanged, and GRASP leads SEAL at nearly every refinement step under every judge \(Table[10](https://arxiv.org/html/2608.06526#A5.T10)\), trailing only before self\-refinement takes effect on the hard split\.

Table 4:Judge robustness\.Overall \(↑\\uparrow\), privacy \(↓\\downarrow\), and utility \(↑\\uparrow\) at each method’s best\-Overall round under three judges\.Bold: better per metric\.
#### Efficiency\.

Because anonymization runs entirely on a local 8B model, no text leaves the user’s machine\. At a 1:1 input–output ratio, the Llama\-3\.1\-8B backbone costs roughly1%1\\%of the GPT\-4o teacher, versus30%30\\%for Claude Haiku 4\.5 and14%14\\%for Gemini 2\.5 Flash\. Per\-sample rewriting takes0\.640\.64s, versus0\.490\.49–0\.810\.81s for the API systems; a full self\-refinement step, which adds local attribute inference on the rewrite, takes13\.813\.8s, versus3\.43\.4–5\.65\.6s\. Full measurements are in Table[6](https://arxiv.org/html/2608.06526#A3.T6)in the Appendix\.

#### Qualitative comparison\.

Appendix[E](https://arxiv.org/html/2608.06526#A5)traces how each method rewrites the same low\-income comment: frontier anonymizers soften the wording but leave the income cue intact, SEAL over\-generalizes into stilted phrasing, and GRASP removes the cue while keeping a natural, coherent sentence\.

## 5Conclusion

We introduced GRASP, which replaces the offline preference learning of prior anonymizer distillation with online reinforcement learning: a single small model serves as anonymizer, adversary, and utility judge, and GRPO optimizes the privacy–utility objective directly against this self\-generated reward, with constraints that guard against reward hacking\. On SynthPAI, GRASP improves the privacy–utility trade\-off over SEAL under three independent judges, removes more private information than adversarial anonymization driven by frontier models, and extends the Pareto frontier on both evaluation splits—while running entirely on\-device at roughly1%1\\%of the teacher’s cost\.

## Limitations

#### Inference latency\.

A full self\-refinement step runs attribute inference on the local88B model, taking13\.813\.8s per sample versus3\.43\.4–5\.65\.6s for API\-based adversarial anonymization, which offloads this step to a frontier model \(Appendix[C](https://arxiv.org/html/2608.06526#A3)\)\. This is the price of keeping all computation on\-device; plain rewriting is comparable to the API systems \(0\.640\.64s\), and users who need lower latency can run fewer refinement rounds at a reduced privacy level\.

#### LLM\-based evaluation\.

Privacy and utility are scored by LLM judges\. Judge behavior is known to be sensitive to the evaluation protocol itself, including position and verbosity effects, and to vary with the task being scored\(Haririet al\.,[2026](https://arxiv.org/html/2608.06526#bib.bib34); Zhenget al\.,[2023](https://arxiv.org/html/2608.06526#bib.bib35)\)\. We mitigate evaluator\-specific artifacts by replicating all comparisons under three judges from different model families \(Table 4\), but LLM judges remain imperfect proxies, particularly for utility, where the judgment is open\-ended rather than checkable against a reference\(Ghiasvandet al\.,[2026](https://arxiv.org/html/2608.06526#bib.bib18)\)\.

#### Fixed attribute set\.

GRASP is trained and evaluated on the eight personal attributes annotated in SynthPAI; protecting a different or broader attribute set \(e\.g\., health status or political views\) would require regenerating trajectories and retraining, and performance on attributes outside this set is untested\.

#### Potential risks\.

GRASP reduces but does not eliminate attribute leakage, and its privacy measurements reflect current inference models; users should not treat its output as a guarantee of anonymity, particularly for high\-stakes disclosures\. Conversely, because the model is trained to predict private attributes as part of its self\-critique, it could in principle be misused as an inference attacker, although its capability is distilled from, and remains below, that of the publicly available frontier models already capable of such inference\.

## References

- J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.\(2023\)Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[Appendix A](https://arxiv.org/html/2608.06526#A1.p1.1),[§1](https://arxiv.org/html/2608.06526#S1.p1.1),[§1](https://arxiv.org/html/2608.06526#S1.p6.1),[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px1.p1.2)\.
- S\. Agarwal, L\. Ahmad, J\. Ai, S\. Altman, A\. Applebaum, E\. Arbus, R\. K\. Arora, Y\. Bai, B\. Baker, H\. Bao,et al\.\(2025\)Gpt\-oss\-120b & gpt\-oss\-20b model card\.arXiv preprint arXiv:2508\.10925\.Cited by:[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px3.p1.1)\.
- Claude Haiku 4\.5\.Note:[https://www\.anthropic\.com/news/claude\-haiku\-4\-5](https://www.anthropic.com/news/claude-haiku-4-5)Accessed 2026\-08\-01Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p6.1),[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px2.p1.1)\.
- N\. Carlini, D\. Ippolito, M\. Jagielski, K\. Lee, F\. Tramer, and C\. Zhang \(2022\)Quantifying memorization across neural language models\.arXiv preprint arXiv:2202\.07646\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p1.1)\.
- N\. Carlini, F\. Tramer, E\. Wallace, M\. Jagielski, A\. Herbert\-Voss, K\. Lee, A\. Roberts, T\. Brown, D\. Song, U\. Erlingsson,et al\.\(2021\)Extracting training data from large language models\.In30th USENIX security symposium \(USENIX Security 21\),pp\. 2633–2650\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p1.1)\.
- P\. F\. Christiano, J\. Leike, T\. Brown, M\. Martic, S\. Legg, and D\. Amodei \(2017\)Deep reinforcement learning from human preferences\.Advances in neural information processing systems30\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p4.1)\.
- Y\. Dou, I\. Krsek, T\. Naous, A\. Kabra, S\. Das, A\. Ritter, and W\. Xu \(2024\)Reducing privacy risks in online self\-disclosures with language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 13732–13754\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p1.1)\.
- A\. Frikha, N\. Walha, K\. K\. Nakka, R\. Mendes, X\. Jiang, and X\. Zhou \(2025\)Incognitext: privacy\-enhancing conditional text anonymization via llm\-based private attribute randomization\.InProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics,pp\. 2490–2501\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p2.1)\.
- A\. Gadotti, L\. Rocher, F\. Houssiau, A\. Creţu, and Y\. De Montjoye \(2024\)Anonymization: the imperfect science of using data while preserving privacy\.Science advances10\(29\),pp\. eadn7053\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p2.1)\.
- S\. Ghiasvand, M\. Amirizaniani, H\. E\. Oskouie, M\. Alizadeh, and R\. Pedarsani \(2026\)Can mllms critique like humans? evaluating open\-ended aesthetic reasoning in multimodal large language models\.arXiv preprint arXiv:2606\.29689\.Cited by:[LLM\-based evaluation\.](https://arxiv.org/html/2608.06526#Sx1.SS0.SSS0.Px2.p1.1)\.
- Google DeepMind \(2025\)Gemini 2\.5: our most intelligent AI model\.Note:Google Official Blog[https://blog\.google/technology/google\-deepmind/gemini\-model\-thinking\-updates\-march\-2025/](https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/)\. Accessed 2026\-08\-01Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p6.1),[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px2.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[Appendix B](https://arxiv.org/html/2608.06526#A2.p1.17),[§1](https://arxiv.org/html/2608.06526#S1.p1.1),[§1](https://arxiv.org/html/2608.06526#S1.p6.1),[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px4.p1.15)\.
- D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.\(2025\)Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p5.1)\.
- M\. Hariri, W\. Chen, N\. Shahini, V\. Singh, K\. Ye, A\. Samandar, D\. Ganguly, S\. Sankar, Y\. Zhang, S\. Wang, J\. Peng, B\. Zhang, M\. Hinczewski, and V\. Chaudhary \(2026\)Test\-time scaling in reasoning LLMs: inference regimes, evaluation, and reproducibility\.arXiv preprint arXiv:2608\.04001\.Cited by:[LLM\-based evaluation\.](https://arxiv.org/html/2608.06526#Sx1.SS0.SSS0.Px2.p1.1)\.
- G\. Hinton, O\. Vinyals, and J\. Dean \(2015\)Distilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p3.1)\.
- S\. Hou, R\. Shang, Z\. Long, X\. Fu, and Y\. Chen \(2025\)A general pseudonymization framework for cloud\-based llms: replacing privacy information in controlled text generation\.arXiv preprint arXiv:2502\.15233\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p2.1)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, W\. Chen,et al\.\(2022\)Lora: low\-rank adaptation of large language models\.\.Iclr1\(2\),pp\. 3\.Cited by:[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px4.p1.15)\.
- K\. Kim, H\. Jeon, and J\. Shin \(2026\)Self\-refining language model anonymizers via adversarial distillation\.Advances in Neural Information Processing Systems38,pp\. 165629–165659\.Cited by:[Appendix A](https://arxiv.org/html/2608.06526#A1.p1.1),[Appendix A](https://arxiv.org/html/2608.06526#A1.p2.2),[§1](https://arxiv.org/html/2608.06526#S1.p3.1),[§3](https://arxiv.org/html/2608.06526#S3.p1.2),[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px2.p1.1)\.
- K\. Krishna, Y\. Song, M\. Karpinska, J\. Wieting, and M\. Iyyer \(2023\)Paraphrasing evades detectors of ai\-generated text, but retrieval is an effective defense\.Advances in neural information processing systems36,pp\. 27469–27500\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p2.1),[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px2.p1.1)\.
- L\. Levine, H\. E\. Oskouie, S\. Ghiasvand, and M\. Sarrafzadeh \(2025\)MI\-to\-mid distilled compression \(m2m\-dc\): an hybrid\-information\-guided\-block pruning with progressive inner slicing approach to model compression\.arXiv preprint arXiv:2511\.06842\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p3.1)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang,et al\.\(2023\)Self\-refine: iterative refinement with self\-feedback\.Advances in neural information processing systems36,pp\. 46534–46594\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p4.1)\.
- Microsoft \(2025\)What is Azure AI Language?\.Note:[https://learn\.microsoft\.com/en\-us/azure/ai\-services/language\-service/overview](https://learn.microsoft.com/en-us/azure/ai-services/language-service/overview)Accessed 2026\-08\-01Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p2.1),[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px2.p1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p4.1)\.
- I\. Pilán, P\. Lison, L\. Øvrelid, A\. Papadopoulou, D\. Sánchez, and M\. Batet \(2022\)The text anonymization benchmark \(tab\): a dedicated corpus and evaluation framework for text anonymization\.Computational Linguistics48\(4\),pp\. 1053–1101\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p2.1)\.
- Qwen Team \(2025\)Qwen3: think deeper, act faster\.Note:[https://qwenlm\.github\.io/blog/qwen3](https://qwenlm.github.io/blog/qwen3)Accessed 2026\-08\-01Cited by:[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px2.p1.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.Advances in neural information processing systems36,pp\. 53728–53741\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p3.1)\.
- J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov \(2017\)Proximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p5.1),[§3\.1](https://arxiv.org/html/2608.06526#S3.SS1.SSS0.Px4.p1.6)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p5.1),[§3\.1](https://arxiv.org/html/2608.06526#S3.SS1.SSS0.Px4.p1.6),[§3](https://arxiv.org/html/2608.06526#S3.p1.2)\.
- R\. Staab, M\. Vero, M\. Balunovic, and M\. Vechev \(2024\)Beyond memorization: violating privacy via inference with large language models\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 33832–33878\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p1.1),[§2](https://arxiv.org/html/2608.06526#S2.SS0.SSS0.Px1.p1.9),[§2](https://arxiv.org/html/2608.06526#S2.p1.1),[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px3.p1.1)\.
- R\. Staab, M\. Vero, M\. Balunović, and M\. Vechev \(2025\)Language models are advanced anonymizers\.InThe Thirteenth International Conference on Learning Representations,Cited by:[Appendix A](https://arxiv.org/html/2608.06526#A1.p1.1),[§1](https://arxiv.org/html/2608.06526#S1.p1.1),[§1](https://arxiv.org/html/2608.06526#S1.p2.1),[§2](https://arxiv.org/html/2608.06526#S2.SS0.SSS0.Px2.p1.20),[§2](https://arxiv.org/html/2608.06526#S2.SS0.SSS0.Px3.p1.4),[§2](https://arxiv.org/html/2608.06526#S2.SS0.SSS0.Px4.p1.2),[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px1.p1.2),[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px2.p1.1)\.
- N\. Stiennon, L\. Ouyang, J\. Wu, D\. Ziegler, R\. Lowe, C\. Voss, A\. Radford, D\. Amodei, and P\. F\. Christiano \(2020\)Learning to summarize with human feedback\.Advances in neural information processing systems33,pp\. 3008–3021\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p4.1)\.
- G\. Team, A\. Kamath, J\. Ferret, S\. Pathak, N\. Vieillard, R\. Merhej, S\. Perrin, T\. Matejovicova, A\. Ramé, M\. Rivière,et al\.\(2025\)Gemma 3 technical report\.arXiv preprint arXiv:2503\.19786\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p1.1)\.
- T\. Yang, X\. Zhu, and I\. Gurevych \(2025\)Robust utility\-preserving text anonymization based on large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 28922–28941\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p2.1)\.
- H\. Yukhymenko, R\. Staab, M\. Vero, and M\. Vechev \(2024\)A synthetic dataset for personal attribute inference\.Advances in Neural Information Processing Systems37,pp\. 120735–120779\.Cited by:[Appendix A](https://arxiv.org/html/2608.06526#A1.p1.1),[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px1.p1.2)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing,et al\.\(2023\)Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.Advances in neural information processing systems36,pp\. 46595–46623\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p4.1),[LLM\-based evaluation\.](https://arxiv.org/html/2608.06526#Sx1.SS0.SSS0.Px2.p1.1)\.
- Zhipu AI \(2026\)GLM\-5\.Note:Zhipu AI[https://z\.ai/](https://z.ai/)\. Accessed 2026\-08\-01Cited by:[§4\.1](https://arxiv.org/html/2608.06526#S4.SS1.SSS0.Px3.p1.1)\.
- D\. M\. Ziegler, N\. Stiennon, J\. Wu, T\. B\. Brown, A\. Radford, D\. Amodei, P\. Christiano, and G\. Irving \(2019\)Fine\-tuning language models from human preferences\.arXiv preprint arXiv:1909\.08593\.Cited by:[§1](https://arxiv.org/html/2608.06526#S1.p4.1)\.

Algorithm 1GRASP1:reference model

πref\\pi\_\{\\text\{ref\}\}\(SFT, §[2](https://arxiv.org/html/2608.06526#S2.SS0.SSS0.Px5)\); group size

GG; weights

wp,wuw\_\{\\text\{p\}\},w\_\{\\text\{u\}\}; coefficients

α,β\\alpha,\\beta; utility floor

τ\\tau; KL weight

βKL\\beta\_\{\\text\{KL\}\}
2:anonymization policy

πθ\\pi\_\{\\theta\}
3:Initialize

πθ←πref\\pi\_\{\\theta\}\\leftarrow\\pi\_\{\\text\{ref\}\}
4:whilenot convergeddo

5:Sample a text

xxwith target attribute set

𝒫\\mathcal\{P\}
6:Draw a group

\{x1′,…,xG′\}∼πθold\(⋅∣x\)\\\{x^\{\\prime\}\_\{1\},\\dots,x^\{\\prime\}\_\{G\}\\\}\\sim\\pi\_\{\\theta\_\{\\text\{old\}\}\}\(\\cdot\\mid x\)
7:for

i=1i=1to

GGdo

8:

𝒫′←ℳpriv​\(xi′\)\\mathcal\{P\}^\{\\prime\}\\leftarrow\\mathcal\{M\}\_\{\\text\{priv\}\}\(x^\{\\prime\}\_\{i\}\)⊳\\trianglerightπref\\pi\_\{\\text\{ref\}\}as adversary

9:

rp←α​\(1−\|𝒫′\|\|𝒫\|\)\+β​\(1−conf¯\)r\_\{\\text\{p\}\}\\leftarrow\\alpha\(1\{\-\}\\tfrac\{\|\\mathcal\{P\}^\{\\prime\}\|\}\{\|\\mathcal\{P\}\|\}\)\+\\beta\(1\{\-\}\\overline\{\\mathrm\{conf\}\}\)⊳\\trianglerightEq\. \([3](https://arxiv.org/html/2608.06526#S3.E3)\)

10:

ru←ℳutil​\(xi′,x\)r\_\{\\text\{u\}\}\\leftarrow\\mathcal\{M\}\_\{\\text\{util\}\}\(x^\{\\prime\}\_\{i\},x\)⊳\\trianglerightπref\\pi\_\{\\text\{ref\}\}as evaluator

11:if

ru≥τr\_\{\\text\{u\}\}\\geq\\tauand

\|xi′\|∈\[12​\|x\|,2​\|x\|\]\|x^\{\\prime\}\_\{i\}\|\\in\[\\tfrac\{1\}\{2\}\|x\|,2\|x\|\]then

12:

Ri←wp​rp\+wu​ruR\_\{i\}\\leftarrow w\_\{\\text\{p\}\}\\,r\_\{\\text\{p\}\}\+w\_\{\\text\{u\}\}\\,r\_\{\\text\{u\}\}
13:else

14:

Ri←−1R\_\{i\}\\leftarrow\-1⊳\\trianglerightinfeasible rewrite

15:endif

16:endfor

17:

A^i←\(Ri−mean⁡\(\{Rj\}\)\)/std⁡\(\{Rj\}\)\\hat\{A\}\_\{i\}\\leftarrow\\big\(R\_\{i\}\-\\operatorname\{mean\}\(\\\{R\_\{j\}\\\}\)\\big\)/\\operatorname\{std\}\(\\\{R\_\{j\}\\\}\)for all

ii
18:Update

θ\\thetaby maximizing

𝒥​\(θ\)\\mathcal\{J\}\(\\theta\)with KL to

πref\\pi\_\{\\text\{ref\}\}⊳\\trianglerightEq\. \([5](https://arxiv.org/html/2608.06526#S3.E5)\)

19:endwhile

20:return

πθ\\pi\_\{\\theta\}

## Appendix ADataset Details

We build on SynthPAI\(Yukhymenkoet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib3)\), a corpus of synthetic personal profiles paired with text comments generated from those profiles\. We use the 3,456 comments with high\-quality human labels for eight personal attributes: age, education level, gender, income level, location, marital status, occupation, and place of birth\. Because our contribution is the reinforcement\-learning objective rather than the distillation data, we reuse the anonymization trajectories of the SFT\+DPO distillation baseline \(SEAL\)\(Kimet al\.,[2026](https://arxiv.org/html/2608.06526#bib.bib26)\)\. These trajectories come from simulating adversarial anonymization\(Staabet al\.,[2025](https://arxiv.org/html/2608.06526#bib.bib2)\)for up to three steps, with GPT\-4o\(Achiamet al\.,[2023](https://arxiv.org/html/2608.06526#bib.bib7)\)acting as anonymizer, attribute\-inference model, and utility evaluator\. Of the 300 profiles, 275 are used for trajectory generation \(2,734 comments\) and 25 are held out entirely from training\.

The held\-out profiles yield two evaluation splits\. The*main*set consists of the 723 comments written from the held\-out profiles\. The*hard*set consists of 500 texts that embed personal information contextually rather than as explicit identifiers, constructed by SEAL\(Kimet al\.,[2026](https://arxiv.org/html/2608.06526#bib.bib26)\): hard\-to\-anonymize texts are first identified in the original SynthPAI pool via iterative adversarial anonymization, and an LLM is then prompted to generate 20 new texts per held\-out profile emulating their characteristics—attributes conveyed through narrative and distributed linguistic cues rather than replaceable keywords\. Its higher baseline inference accuracy \(0\.7810\.781vs\.0\.6200\.620on the main set\) reflects this increased difficulty\.

This mirrors the intended use of the framework, in which trajectories are distilled from synthetic profiles using an external LLM, while the resulting small model runs locally on real, private data without invoking an untrusted external service\.

Table 5:Hyperparameters for the SFT and GRPO stages of GRASP\.The SFT stage matches SEAL; GRPO is run for a single self\-refinement round\.
## Appendix BImplementation Details

Unless noted otherwise, GRASP is trained on Llama\-3\.1\-8B\-Instruct\(Grattafioriet al\.,[2024](https://arxiv.org/html/2608.06526#bib.bib12)\)\. The SFT stage follows the distillation baseline exactly, using LoRA \(rank1616,αLoRA=16\\alpha\_\{\\text\{LoRA\}\}\{=\}16, dropout0\.050\.05\), AdamW at learning rate2​e−42\\mathrm\{e\}\{\-\}4, and one epoch\. We then initialize the policy from the SFT model, which also serves as the frozen referenceπref\\pi\_\{\\text\{ref\}\}, and optimize it with GRPO for one epoch on5,0005\{,\}000training prompts drawn from the trajectory states\. Each prompt drawsG=8G\{=\}8sampled anonymizations, and training uses learning rate5​e−65\\mathrm\{e\}\{\-\}6, KL weightβKL=0\.03\\beta\_\{\\text\{KL\}\}\{=\}0\.03, effective batch size3232, sampling temperature0\.90\.9, and a maximum completion length of512512tokens\. The reward uses privacy coefficientsα=0\.7\\alpha\{=\}0\.7,β=0\.3\\beta\{=\}0\.3, trade\-off weightswp=0\.9w\_\{\\text\{p\}\}\{=\}0\.9,wu=0\.1w\_\{\\text\{u\}\}\{=\}0\.1, and utility floorτ=0\.55\\tau\{=\}0\.55\. All models are trained with LoRA in mixed precision with FlashAttention\-2 on NVIDIA GPUs\. At inference the trained policy anonymizes by self\-refinement for up to five iterations\. Table[5](https://arxiv.org/html/2608.06526#A1.T5)summarizes all hyperparameters for both stages\.

## Appendix CEfficiency

Table[6](https://arxiv.org/html/2608.06526#A3.T6)reports inference cost and per\-sample latency, measured over5050samples with GRASP running on a single NVIDIA GPU and the baselines accessed via their APIs\. Cost assumes a1:11\{:\}1input\-to\-output token ratio and is stated relative to the GPT\-4o teacher: the Llama\-3\.1\-8B backbone used by GRASP comes to roughly1%1\\%, versus30%30\\%for Claude Haiku 4\.5 and14%14\\%for Gemini 2\.5 Flash\. For rewriting alone, per\-sample latency is0\.640\.64s, within the range of the API models \(0\.490\.49–0\.810\.81s\)\. A full self\-refinement step additionally runs attribute inference on the rewrite; because GRASP performs this step with the same local88B model whereas the baselines use their frontier API model, its latency rises to13\.813\.8s versus3\.43\.4–5\.65\.6s\. Note that the latency comparison is approximate, as the API models run on optimized serving infrastructure while GRASP runs on a single local GPU; the corresponding trade\-off is discussed in the Limitations Section\.

Table 6:Efficiency\.Inference cost \(per 1M tokens; relative to the GPT\-4o teacher at a 1:1 token ratio\) and per\-sample latency, measured over5050samples\. GRASP runs locally\.
## Appendix DPer\-iteration results

#### Main and Hard splits\.

Tables[8](https://arxiv.org/html/2608.06526#A5.T8)and[9](https://arxiv.org/html/2608.06526#A5.T9)report the full self\-refinement trajectories summarized in Tables[1](https://arxiv.org/html/2608.06526#S4.T1)and[2](https://arxiv.org/html/2608.06526#S4.T2)\. Two patterns hold across methods and splits\. First, privacy improves with refinement while utility declines monotonically, so the overall score peaks at an intermediate round—for GRASP, round 2 on both splits \(0\.3960\.396main,0\.2940\.294hard\)—and the per\-round trajectory lets users pick an operating point along this trade\-off\. Second, the methods differ in how far refinement carries them: the frontier adversarial anonymizers improve privacy slowly and remain well above0\.360\.36\(main\) and0\.450\.45\(hard\) after five rounds, whereas GRASP continues to gain, reaching0\.1950\.195on the main set \(round 4\) and0\.3320\.332on the hard set \(round 5\)\. On the hard split GRASP’s privacy decreases at every round, and its per\-attribute gains over SEAL concentrate on the same attributes as on the main split, consistent with the aggregate results in Section[4\.2](https://arxiv.org/html/2608.06526#S4.SS2)\.

Table 7:Per\-iteration ablation of GRASP components\.Overall \(↑\\uparrow\), privacy \(↓\\downarrow, attribute\-inference accuracy\), and utility \(↑\\uparrow\) at self\-refinement rounds R1–R3 \(GLM\-5 judge\); R0 is the un\-anonymized input and is omitted\. Highlighted: full model\. Summarized at the best\-Overall round in Table[3](https://arxiv.org/html/2608.06526#S4.T3)\.
#### Ablation\.

Table[7](https://arxiv.org/html/2608.06526#A4.T7)shows how each ablated variant evolves over refinement, revealing dynamics that the best\-round summary in Table[3](https://arxiv.org/html/2608.06526#S4.T3)compresses\. Without the confidence reward, refinement makes no progress at all: privacy stays at0\.6260\.626–0\.6430\.643\(main\) and0\.7730\.773–0\.7850\.785\(hard\) across all three rounds, and the overall score never rises above0\.0060\.006\. The utility floor shapes the trajectory rather than a single round: withτ=0\\tau\{=\}0the policy reaches stronger raw privacy at every matched round, but utility falls steeply with each refinement \(0\.815→0\.6500\.815\{\\to\}0\.650main,0\.832→0\.6770\.832\{\\to\}0\.677hard\) and the overall score declines from R1 to R3 \(0\.376→0\.3030\.376\{\\to\}0\.303on main\), whereas the full model’s overall score rises \(0\.282→0\.3530\.282\{\\to\}0\.353\)—the floor trades some immediate privacy for refinement that remains worthwhile\. The remaining variants—no attacker feedback, fewer training samples, and the smaller groupG=4G\{=\}4—track the full model’s shape but sit at consistently weaker privacy, withG=4G\{=\}4the furthest behind at every round\.

#### Judge robustness\.

Table[10](https://arxiv.org/html/2608.06526#A5.T10)extends the best\-round comparison of Table[4](https://arxiv.org/html/2608.06526#S4.T4)to every self\-refinement step\. GRASP leads SEAL on the overall score in all but three of the3636judge×\\timessplit×\\timesround cells: the first round on the hard split under GLM\-5 \(0\.2390\.239vs\.0\.2480\.248\) and Qwen3 \(0\.2440\.244vs\.0\.2530\.253\), before self\-refinement takes effect, and the final round on the main split under GPT\-OSS, where the two methods are within0\.0030\.003\. Absolute scores shift across judges, mainly through stricter utility assessments, but the per\-round ordering is preserved, indicating that the advantage is neither an artifact of the evaluator nor of selecting a favorable round\.

## Appendix EQualitative comparison

Table[11](https://arxiv.org/html/2608.06526#A5.T11)traces how each method rewrites a single comment, “rent eats up most of my paycheck these days”, whose author has low income\. Azure finds no explicit identifier and leaves the text unchanged\. Dipper paraphrases without an anonymization objective and distorts the meaning \(“My rent goes a long way these days”\), consistent with its large utility loss in Tables[8](https://arxiv.org/html/2608.06526#A5.T8)and[9](https://arxiv.org/html/2608.06526#A5.T9)\. The frontier adversarial anonymizers soften “paycheck” into “income” or “expenses” but still convey that the cost of living strains the author, leaving the low\-income cue largely intact\. SEAL over\-generalizes across rounds, drifting into stilted, near\-meaningless phrasing \(“a significant amount of funds is being allocated to a particular area”\)\. GRASP instead removes the personal\-income cue while keeping a natural, coherent sentence \(“It can be expensive to pay for a place to live”\)\. This mirrors the quantitative picture: GRASP reaches the strongest privacy while preserving readable, faithful text\.

Table 8:Per\-iteration anonymization results on the main dataset\.Overall \(↑\\uparrow\), privacy \(↓\\downarrow, attribute\-inference accuracy\), and utility \(↑\\uparrow\) per self\-refinement round, judged by GLM\-5, with per\-attribute inference accuracy\. Overall is the relative privacy improvement minus the relative utility loss, normalized by the original privacy \(Porig=0\.620P\_\{\\text\{orig\}\}\{=\}0\.620\)\. Top panel: the five\-round trajectories of the three adversarial\-anonymization baselines; bottom panel: the original text, Azure, Dipper, and the trajectories of SEAL and GRASP\.Bold/underline: best/second\-best per row\. GRASP attains both the best overall score \(0\.3960\.396, round 2\) and the lowest inference accuracy \(0\.1950\.195, round 4\)\. Summarized in Table[1](https://arxiv.org/html/2608.06526#S4.T1)and plotted in Figure[2](https://arxiv.org/html/2608.06526#S4.F2)\.
Table 9:Per\-iteration anonymization results on the hard dataset\.Overall \(↑\\uparrow\), privacy \(↓\\downarrow, attribute\-inference accuracy\), and utility \(↑\\uparrow\) per self\-refinement round, judged by GLM\-5, with per\-attribute inference accuracy\. Overall is the relative privacy improvement minus the relative utility loss, normalized by the original privacy \(Porig=0\.781P\_\{\\text\{orig\}\}\{=\}0\.781\)\. Top panel: the five\-round trajectories of the three adversarial\-anonymization baselines; bottom panel: the original text, Azure, Dipper, and the trajectories of SEAL and GRASP\.Bold/underline: best/second\-best per row\. GRASP’s inference accuracy decreases at every round, reaching0\.3320\.332, the lowest of any method\. Summarized in Table[2](https://arxiv.org/html/2608.06526#S4.T2)and plotted in Figure[2](https://arxiv.org/html/2608.06526#S4.F2)\.
Table 10:Per\-iteration judge robustness\.Overall \(↑\\uparrow\), privacy \(↓\\downarrow, attribute\-inference accuracy\), and utility \(↑\\uparrow\) at each of six self\-refinement rounds for SEAL and GRASP, scored by three independent judges on both splits\. Bold: better of the two methods per iteration and metric\. GRASP leads on Overall at nearly every round under every judge\. Summarized at the best\-Overall round in Table[4](https://arxiv.org/html/2608.06526#S4.T4)\.Table 11:Round\-by\-round anonymization of the same comment by each method, where the goal is to hide that the author has low income \(their rent consumes most of their paycheck\)\. Azure leaves the text unchanged and Dipper paraphrases but distorts the meaning; the frontier adversarial anonymizers soften the wording but keep the financial\-strain signal; SEAL over\-generalizes into stilted, near\-meaningless phrasing, whereas GRASP removes the income cue while keeping a natural, coherent sentence\. Identical consecutive rounds are merged\.MethodRoundTextOriginal–rent eats up most of my paycheck these daysAzure1–5rent eats up most of my paycheck these daysDipper1My rent goes a long way these daysAA \(Gemini\)1Rent eats up most of my income these days2Housing costs eat up most of my income these days3–4Housing costs are a significant expense these days5The cost of living is a significant expense these daysAA \(Claude\)1rent eats up a significant portion of my paycheck these days2housing costs take up a significant portion of my paycheck these days3–5housing costs take up a significant portion of my expenses these daysAA \(Qwen3\)1–5Housing costs eat up most of my income these daysSEAL1rent takes up a lot of money right now2a lot of money is being spent on something right now3–4a lot of money is being spent on something5a significant amount of funds is being allocated to a particular areaGRASP1a lot of money is going towards housing2a lot of money is going towards housing costs3Housing costs are a significant expense\.4It can be expensive to pay for a place to live\.5It can be expensive to find a place to live\.