JUMP: Single-Pass Membership Inference on Fine-Tuned Diffusion Language Models

# JUMP: Single-Pass Membership Inference on Fine-Tuned Diffusion Language Models
Source: [https://arxiv.org/html/2607.16207](https://arxiv.org/html/2607.16207)
Yeachan Jun Department of Artificial Intelligence Yonsei University, Seoul, Korea dpcks8942@yonsei\.ac\.kr&Albert No Department of Artificial Intelligence Yonsei University, Seoul, Korea albert\.no@yonsei\.ac\.kr

###### Abstract

Membership inference attacks \(MIAs\) test whether a candidate example appeared in a model’s training data\. We study MIAs for fine\-tuned discrete diffusion language models \(dLLMs\), where membership means inclusion in the target model’s fine\-tuning set\. Unlike autoregressive language models, dLLMs allow an attacker to choose arbitrary mask sets and obtain token distributions for all masked positions in parallel\. The prior dLLM attack, SAMA, follows a natural loss\-mimicking strategy by averaging reconstruction signals over many randomly sampled masks, but it uses the any\-order interface only as randomization and requires many target/reference queries\. We proposeJUMP\(*Joint Uncertainty\-Guided Mask Probing*\), a single\-pass scoring attack that exploits both distinctive properties of dLLMs: any\-order decodability is used to select low\-reference\-confidence positions, and parallel decodability is used to score all selected positions through one joint masked query per model\.JUMPmasks the selected positions jointly and computes a clipped target/reference reconstruction\-gap statistic\. On fine\-tuned LLaDA\-8B\-Base across six MIMIR domains,JUMPimproves mean ROC\-AUC from0\.820\.82to0\.900\.90over SAMA and substantially improves low\-FPR detection, while requiring only one selector pass and one scoring pass through each of the target and reference models\.

## 1Introduction

Membership inference attacks \(MIAs\) are a standard tool for auditing privacy leakage in machine learning models\(Shokriet al\.,[2017](https://arxiv.org/html/2607.16207#bib.bib1); Yeomet al\.,[2018](https://arxiv.org/html/2607.16207#bib.bib2)\)\. Given a trained model and a candidate example, the attacker decides whether the example was included in the model’s training data\. Successful attacks reveal that the model’s outputs retain measurable traces of particular training examples\. We study this problem for fine\-tuned language models: the target model is adapted on a private corpus, and membership means inclusion in that fine\-tuning set\.

For autoregressive \(AR\) language models, MIAs naturally follow the left\-to\-right likelihood interface\. A sequence is scored through the same prefix\-conditioned predictions used in training, and stronger attacks refine this statistic with reference calibration, token filtering, perturbation\-based comparisons, or information\-theoretic normalization\(Carliniet al\.,[2021](https://arxiv.org/html/2607.16207#bib.bib7); Duanet al\.,[2024](https://arxiv.org/html/2607.16207#bib.bib15); Shiet al\.,[2024b](https://arxiv.org/html/2607.16207#bib.bib12); Zhanget al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib13); Xieet al\.,[2024](https://arxiv.org/html/2607.16207#bib.bib14); Matternet al\.,[2023](https://arxiv.org/html/2607.16207#bib.bib11); Changet al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib16); Tao and Shokri,[2026](https://arxiv.org/html/2607.16207#bib.bib17)\)\. However, the conditional contexts are fixed: tokenxix\_\{i\}is always evaluated from the prefixx<ix\_\{<i\}\. Thus, AR MIAs largely differ in how they aggregate a predetermined set of token scores\.

Discrete diffusion language models \(dLLMs\) change this interface\. Recent dLLMs such as LLaDA and Dream reconstruct masked tokens from bidirectional visible context rather than from a prefix alone\(Nieet al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib26); Yeet al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib27)\)\. For MIAs, the key difference is that the attacker can choose the mask set\. The same sequence can induce many reconstruction tasks depending on which tokens are hidden; we call this*any\-order decodability*\. At the same time, the model returns token distributions for all masked positions in one forward pass; we call this*parallel decodability*\. Combined, these properties make the mask set itself a central component of the attack design\.

SAMA\(Chenet al\.,[2026](https://arxiv.org/html/2607.16207#bib.bib19)\), the first dLLM\-specific MIA, follows a direct loss\-mimicking strategy by sampling many random mask sets and averaging target/reference reconstruction gaps\. This is a natural estimator of the diffusion reconstruction objective, but it uses the any\-order interface mainly as randomization\. Random subsets often include easy or uninformative tokens, diluting the membership signal, and the target/reference query cost grows with the number of sampled masks\. This motivates our central question:can an attack use the dLLM’s any\-order interface to choose a more informative mask set, and then use parallel decoding to evaluate that set in a single pass?

We answer this question withJUMP\(*Joint Uncertainty\-Guided Mask Probing*\), a single\-pass MIA for fine\-tuned dLLMs\.JUMPfirst uses the reference model to find positions where it assigns low confidence to the true token\. It then masks these positions together and compares how the target and reference models reconstruct the true tokens\. After the mask set is chosen, one masked query to the target model and one masked query to the reference model returns all selected token scores in parallel\. Thus,JUMPmakes mask\-set selection the main attack design problem while keeping the final target/reference scoring cost constant\.

We evaluateJUMPon fine\-tuned LLaDA\-8B\-Base\(Nieet al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib26)\)across six MIMIR\(Duanet al\.,[2024](https://arxiv.org/html/2607.16207#bib.bib15)\)domains\. Compared with SAMA,JUMPraises mean ROC\-AUC from 0\.82 to 0\.90 and improves detection in the strict low\-FPR regime\. At the same time,JUMPreplaces many random target/reference masked queries with a single selected target/reference scoring query per sample\. These results support the central premise ofJUMP: for dLLMs, membership inference should be designed around a single informative joint mask rather than many random masks\.

## 2Related Work

#### Membership inference attacks\.

Given a target modelMtgtM\_\{\\mathrm\{tgt\}\}trained on𝒟train\\mathcal\{D\}\_\{\\mathrm\{train\}\}and a candidate examplexx, an MIA asks whetherx∈𝒟trainx\\in\\mathcal\{D\}\_\{\\mathrm\{train\}\}\(Shokriet al\.,[2017](https://arxiv.org/html/2607.16207#bib.bib1); Yeomet al\.,[2018](https://arxiv.org/html/2607.16207#bib.bib2)\)\. We write a score\-based MIA as a scalar statistic

g​\(x;Mtgt,Mref\)∈ℝ,g\(x;\\,M\_\{\\mathrm\{tgt\}\},\\,M\_\{\\mathrm\{ref\}\}\)\\in\\mathbb\{R\},whereMrefM\_\{\\mathrm\{ref\}\}may be absent for target\-only attacks\. The attacker predicts membership wheng​\(x;Mtgt,Mref\)\>ηg\(x;M\_\{\\mathrm\{tgt\}\},M\_\{\\mathrm\{ref\}\}\)\>\\eta, and varyingη\\etayields the ROC curve\. A classical example is the negative\-loss statisticgloss​\(x;Mtgt\)=−ℓ​\(Mtgt,x\)g\_\{\\mathrm\{loss\}\}\(x;M\_\{\\mathrm\{tgt\}\}\)=\-\\ell\(M\_\{\\mathrm\{tgt\}\},x\), which connects membership leakage to the generalization gap\(Yeomet al\.,[2018](https://arxiv.org/html/2607.16207#bib.bib2)\)\. Reference\-based attacks instead compare the target with a reference, e\.g\.,gref​\(x\)=−ℓ​\(Mtgt,x\)\+ℓ​\(Mref,x\)g\_\{\\mathrm\{ref\}\}\(x\)=\-\\ell\(M\_\{\\mathrm\{tgt\}\},x\)\+\\ell\(M\_\{\\mathrm\{ref\}\},x\), to reduce the confounding effect of intrinsic example difficulty\(Watsonet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib5); Carliniet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib4); Zarifzadehet al\.,[2024](https://arxiv.org/html/2607.16207#bib.bib6)\)\. Complementary work studies training\-data extraction and privacy auditing\(Longet al\.,[2018](https://arxiv.org/html/2607.16207#bib.bib3); Carliniet al\.,[2019](https://arxiv.org/html/2607.16207#bib.bib34),[2021](https://arxiv.org/html/2607.16207#bib.bib7),[2023](https://arxiv.org/html/2607.16207#bib.bib8); Nasret al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib9); Steinkeet al\.,[2023](https://arxiv.org/html/2607.16207#bib.bib35); Jagielskiet al\.,[2023](https://arxiv.org/html/2607.16207#bib.bib36); Lukaset al\.,[2023](https://arxiv.org/html/2607.16207#bib.bib18); Mireshghallahet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib10)\)\.

#### MIA for autoregressive language models\.

For an autoregressive language model, the loss is a natural MIA statistic because the model is trained by next\-token negative log\-likelihood, yielding a sequence score from prefix\-conditioned token probabilities:

gAR​\(x\)=log⁡pAR​\(x\)=∑i=1Llog⁡p​\(xi∣x<i\)\.g\_\{\\mathrm\{AR\}\}\(x\)=\\log p\_\{\\mathrm\{AR\}\}\(x\)=\\sum\_\{i=1\}^\{L\}\\log p\(x\_\{i\}\\mid x\_\{<i\}\)\.Reference\-calibrated variants usegAR​\-​ref​\(x\)=log⁡pMtgt​\(x\)−log⁡pMref​\(x\)g\_\{\\mathrm\{AR\\text\{\-\}ref\}\}\(x\)=\\log p\_\{M\_\{\\mathrm\{tgt\}\}\}\(x\)\-\\log p\_\{M\_\{\\mathrm\{ref\}\}\}\(x\), while token\-selection methods such as Min\-K% and Min\-K%\+\+ computeggfrom low\-probability tokens rather than all tokens\(Shiet al\.,[2024b](https://arxiv.org/html/2607.16207#bib.bib12); Zhanget al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib13)\)\. Other attacks use contextual perturbations, neighborhood comparisons, or information\-theoretic calibrations to refine the same score\-based decision rule\(Xieet al\.,[2024](https://arxiv.org/html/2607.16207#bib.bib14); Matternet al\.,[2023](https://arxiv.org/html/2607.16207#bib.bib11); Changet al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib16); Tao and Shokri,[2026](https://arxiv.org/html/2607.16207#bib.bib17)\)\. These methods differ in how they constructgg, but the underlying conditional contexts are fixed by autoregressive decoding\.

#### Discrete diffusion language models\.

Masked diffusion language models\(Austinet al\.,[2021](https://arxiv.org/html/2607.16207#bib.bib20); Hoogeboomet al\.,[2021](https://arxiv.org/html/2607.16207#bib.bib21); Louet al\.,[2024](https://arxiv.org/html/2607.16207#bib.bib22); Sahooet al\.,[2024](https://arxiv.org/html/2607.16207#bib.bib23); Shiet al\.,[2024a](https://arxiv.org/html/2607.16207#bib.bib24); Ouet al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib25)\), scaled by recent dLLMs such as LLaDA and Dream\(Nieet al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib26); Yeet al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib27)\), define reconstruction distributions over arbitrary masked subsets\. Letx=\(x1,…,xL\)x=\(x\_\{1\},\\dots,x\_\{L\}\)and letx∖Sx\_\{\\setminus S\}denote the sequence where positions inS⊆\{1,…,L\}S\\subseteq\\\{1,\\dots,L\\\}are replaced by\[MASK\]\. A dLLM modelspθ​\(xi∣x∖S\)p\_\{\\theta\}\(x\_\{i\}\\mid x\_\{\\setminus S\}\)fori∈Si\\in Sand is trained with a masked reconstruction objective of the form

ℒ​\(θ\)=−𝔼x,λ,S​\[1\|S\|​∑i∈Slog⁡pθ​\(xi∣x∖S\)\],λ∼U​\(0,1\)\.\\mathcal\{L\}\(\\theta\)=\-\\,\\mathbb\{E\}\_\{x,\\lambda,S\}\\\!\\left\[\\frac\{1\}\{\|S\|\}\\sum\_\{i\\in S\}\\log p\_\{\\theta\}\(x\_\{i\}\\mid x\_\{\\setminus S\}\)\\right\],\\qquad\\lambda\\sim U\(0,1\)\.The attacker\-facing consequence is thatSSis not fixed\. Any\-order decodability allows the attacker to choose which reconstruction task to query, and parallel decodability returns all\|S\|\|S\|masked\-token distributions from one forward pass\(Uriaet al\.,[2014](https://arxiv.org/html/2607.16207#bib.bib28); Germainet al\.,[2015](https://arxiv.org/html/2607.16207#bib.bib29); Yanget al\.,[2019](https://arxiv.org/html/2607.16207#bib.bib30); Hoogeboomet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib31); Ghazvininejadet al\.,[2019](https://arxiv.org/html/2607.16207#bib.bib32); Changet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib33)\)\.

#### MIA for diffusion language models\.

SAMA\(Chenet al\.,[2026](https://arxiv.org/html/2607.16207#bib.bib19)\)is the first dLLM\-specific MIA and is a natural loss\-mimicking baseline\. It samplesTTrandom mask setsS1,…,STS\_\{1\},\\dots,S\_\{T\}and computes a reconstruction\-gap statistic such as

gSAMA​\(x\)=1T​∑t=1T1\|St\|​∑i∈St\[log⁡pMtgt​\(xi∣x∖St\)−log⁡pMref​\(xi∣x∖St\)\]\.g\_\{\\mathrm\{SAMA\}\}\(x\)=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\frac\{1\}\{\|S\_\{t\}\|\}\\sum\_\{i\\in S\_\{t\}\}\\left\[\\log p\_\{M\_\{\\mathrm\{tgt\}\}\}\(x\_\{i\}\\mid x\_\{\\setminus S\_\{t\}\}\)\-\\log p\_\{M\_\{\\mathrm\{ref\}\}\}\(x\_\{i\}\\mid x\_\{\\setminus S\_\{t\}\}\)\\right\]\.\(1\)This statistic mirrors the dLLM reconstruction objective by averaging over randomly sampled masked subsets, and is therefore a natural first approach\. However, computing[Eq\.˜1](https://arxiv.org/html/2607.16207#S2.E1)requires one target and one reference forward pass for each sampled subset, for a total cost of2​T2Tforward evaluations per example\. Our method keeps the reconstruction\-gap principle but replaces random multimask averaging with a single selected mask set\.

## 3Threat Model

#### Setting\.

The target modelMtgtM\_\{\\mathrm\{tgt\}\}is a fine\-tuned dLLM with the masked\-reconstruction interfacepMtgt​\(xi\|x∖S\)p\_\{M\_\{\\mathrm\{tgt\}\}\}\(x\_\{i\}\|x\_\{\\setminus S\}\)\. It is fine\-tuned on a private dataset𝒟train\\mathcal\{D\}\_\{\\mathrm\{train\}\}\. The attacker receives a candidate sequencex=\(x1,…,xL\)x=\(x\_\{1\},\\ldots,x\_\{L\}\)and does not know whetherx∈𝒟trainx\\in\\mathcal\{D\}\_\{\\mathrm\{train\}\}\. The attacker also has access to a compatible reference modelMrefM\_\{\\mathrm\{ref\}\}; in our experiments, this is the same base dLLM before fine\-tuning\.

#### Access\.

Our default setup gives the attacker score access to both models\. For any mask setS⊆\{1,…,L\}S\\subseteq\\\{1,\\ldots,L\\\}chosen by the attacker, the attacker may submit the masked sequencex∖Sx\_\{\\setminus S\}to either model and obtain token log\-probabilities at positioni∈Si\\in S:

\{logpM\(xi∣x∖S\):i∈S\},M∈\{Mtgt,Mref\}\.\\Bigl\\\{\\log p\_\{M\}\(x\_\{i\}\\mid x\_\{\\setminus S\}\):i\\in S\\Bigr\\\},\\qquad M\\in\\\{M\_\{\\mathrm\{tgt\}\},M\_\{\\mathrm\{ref\}\}\\\}\.\(2\)The attacker has white\-box access to the reference model and may lightly train auxiliary reference\-side components on generic public text\. This auxiliary training uses neither member/non\-member labels nor target\-model weights\. We also consider a stricter black\-box selector setting later, in which position selection is performed without training an auxiliary component\.

#### Derived quantities\.

The observable quantities in[Eq\.˜2](https://arxiv.org/html/2607.16207#S3.E2)are the model log\-probabilities\. A useful derived statistic for a chosen mask setSSis the token\-level target/reference reconstruction gap

Δi​\(S\)=log⁡pMtgt​\(xi∣x∖S\)−log⁡pMref​\(xi∣x∖S\),i∈S\.\\Delta\_\{i\}\(S\)=\\log p\_\{M\_\{\\mathrm\{tgt\}\}\}\(x\_\{i\}\\mid x\_\{\\setminus S\}\)\-\\log p\_\{M\_\{\\mathrm\{ref\}\}\}\(x\_\{i\}\\mid x\_\{\\setminus S\}\),\\qquad i\\in S\.\(3\)All gaps\{Δi​\(S\):i∈S\}\\\{\\Delta\_\{i\}\(S\):i\\in S\\\}are obtained with two model forward passes, one throughMtgtM\_\{\\mathrm\{tgt\}\}and one throughMrefM\_\{\\mathrm\{ref\}\}, independent of\|S\|\|S\|\.

#### Goal and evaluation\.

An attack specifies one or more mask sets and computes a scalar MIA statistic from the resulting observations\. Let𝒬=\{S1,…,Sm\}\\mathcal\{Q\}=\\\{S\_\{1\},\\ldots,S\_\{m\}\\\}be the query collection, fixed, randomized, or chosen usingxxand the reference model\. A general score has the form

g\(x;Mtgt,Mref\)=G\(\{logpM\(xi∣x∖S\):S∈𝒬,i∈S,M∈\{Mtgt,Mref\}\}\),g\(x;M\_\{\\mathrm\{tgt\}\},M\_\{\\mathrm\{ref\}\}\)=G\\\!\\left\(\\left\\\{\\log p\_\{M\}\(x\_\{i\}\\mid x\_\{\\setminus S\}\):S\\in\\mathcal\{Q\},\\ i\\in S,\\ M\\in\\\{M\_\{\\mathrm\{tgt\}\},M\_\{\\mathrm\{ref\}\}\\\}\\right\\\}\\right\),where choosing𝒬\\mathcal\{Q\}is part of the attack design\. The attacker predicts membership by thresholdinggg\. We evaluate attacks using ROC\-AUC and TPR at FPR∈\{10%,1%,0\.1%\}\\in\\\{10\\%,1\\%,0\.1\\%\\\}, following standard MIA practice\(Watsonet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib5); Zarifzadehet al\.,[2024](https://arxiv.org/html/2607.16207#bib.bib6); Carliniet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib4)\)\. We also report the number of forward evaluations \(NFE\) per sample\. Unless otherwise specified, NFE counts only target/reference passes used to compute the MIA statistic; selector passes are reported separately when they are not part of the final score\.

## 4Method:JUMP\(*Joint Uncertainty\-Guided Mask Probing*\)

![Refer to caption](https://arxiv.org/html/2607.16207v1/x1.png)Figure 1:Overview ofJUMP\. The attack first selects informative positions under the reference model, then masks the selected positions jointly and scores them with the target and reference dLLMs\. The final MIA statistic is computed from clipped token\-level target/reference reconstruction gaps\. In the visualization, general tokens are indicated by solid\-line boxes, while confidence scores are denoted by dashed\-line boxes\.We first present a diagnostic experiment showing where membership signal appears in a dLLM \([Section˜4\.1](https://arxiv.org/html/2607.16207#S4.SS1)\), then define theJUMPscoring rule \([Section˜4\.2](https://arxiv.org/html/2607.16207#S4.SS2)\), and finally describe how we select difficult positions efficiently \([Section˜4\.3](https://arxiv.org/html/2607.16207#S4.SS3)\)\.

### 4\.1Low\-confidence tokens and membership signal

![Refer to caption](https://arxiv.org/html/2607.16207v1/x2.png)

![Refer to caption](https://arxiv.org/html/2607.16207v1/x3.png)

Figure 2:Distribution of the one\-hole reconstruction gapΔi​\(\{i\}\)\\Delta\_\{i\}\(\\\{i\\\}\)across reference\-model confidence tiers\. Low\-confidence positions produce substantially stronger member/non\-member separation than random or high\-confidence positions\.The any\-order interface in dLLM raises a basic design question: which token positions should an attacker probe? We first study the ideal one\-hole reference low\-confidence score

q⋆​\(i∣x\)=log⁡pMref​\(xi∣x∖\{i\}\),q^\{\\star\}\(i\\mid x\)=\\log p\_\{M\_\{\\mathrm\{ref\}\}\}\(x\_\{i\}\\mid x\_\{\\setminus\\\{i\\\}\}\),\(4\)where smaller values indicate lower reference confidence in the true token under bidirectional context\. This quantity is useful for analysis because it directly measures how uncertain the reference dLLM is about the true tokenxix\_\{i\}when only that token is hidden\.

To test whether this low\-confidence score is related to membership signal, we run a diagnostic one\-hole experiment on LLaDA\-8B\-Base fine\-tuned on the Wikipedia \(en\) domain\. For each candidate sequence, we use a 512\-token evaluation window and, for every valid positionii, mask only that position, i\.e\.,S=\{i\}S=\\\{i\\\}\. We then record the one\-hole reference probabilitypMref​\(xi\|x∖\{i\}\)p\_\{M\_\{\\mathrm\{ref\}\}\}\(x\_\{i\}\|x\_\{\\setminus\\\{i\\\}\}\)and the target/reference gapΔi​\(\{i\}\)\\Delta\_\{i\}\(\\\{i\\\}\)from[Eq\.˜3](https://arxiv.org/html/2607.16207#S3.E3)\. Positions are ranked by[Eq\.˜4](https://arxiv.org/html/2607.16207#S4.E4)and partitioned into three groups of sizeK=64K=64: lowest\-confidence positions, uniformly random positions, and highest\-confidence positions\.

[Figure˜2](https://arxiv.org/html/2607.16207#S4.F2)shows that low\-confidence positions carry substantially stronger membership signal\. Under one\-hole masking, the fine\-tuned target reconstructs these positions much better on member sequences than on non\-member sequences, producing a mean member/non\-member gap difference of\+0\.174\+0\.174\. Random and high\-confidence positions show much weaker separation\. This supports the use of reference\-model confidence as a localization prior: positions where the reference model has low confidence are more informative for fine\-tuning membership\.

The diagnostic procedure itself is not an efficient attack\. Ranking all positions by[Eq\.˜4](https://arxiv.org/html/2607.16207#S4.E4)requires one masked query per position to the reference model, and computing the corresponding target/reference gaps requires one masked query per position and per model\. Thus exact one\-hole localization and scoring costs2​L2Ltarget/reference forward evaluations for a length\-LLsequence, in addition to the localization work\. The goal ofJUMPis to approximate the same localization principle without paying this cost: select a small set of low\-confidence positions once, then exploit parallel decoding to score all of them through a single joint mask\.

### 4\.2JUMPscoring rule

Given a sequencexxof lengthLL, let\[L\]⊆\{1,…,L\}\[L\]\\subseteq\\\{1,\\ldots,L\\\}denote the valid positions that may be masked and scored\.JUMPuses a selector scoreqϕ​\(i∣x\)q\_\{\\phi\}\(i\\mid x\)for eachi∈\[L\]i\\in\[L\]\. The scoreqϕ​\(i∣x\)q\_\{\\phi\}\(i\\mid x\)is a scalar prediction of reference\-model low confidence for the true token at positionii; it is not a vocabulary distribution\. Ideally,qϕ​\(i∣x\)q\_\{\\phi\}\(i\\mid x\)approximates the one\-hole low\-confidence scoreq⋆​\(i∣x\)q^\{\\star\}\(i\\mid x\)in[Eq\.˜4](https://arxiv.org/html/2607.16207#S4.E4), so smaller values indicate lower reference\-model reconstruction confidence\.

For a probing budgetKK, JUMP selects theKKpositions of lowest confidences:

ℋK​\(x;ϕ\)=\{i∈\[L\]∣qϕ​\(i∣x\)​is one of the lowest​K​values among all​i∈\[L\]\}\\mathcal\{H\}\_\{K\}\(x;\\phi\)=\\left\\\{i\\in\[L\]\\mid q\_\{\\phi\}\(i\\mid x\)\\text\{ is one of the lowest \}K\\text\{ values among all \}i\\in\[L\]\\right\\\}\(5\)
In the main experiments, we use a 512\-token window and setK=64K=64; this choice is examined in[Section˜6\.2](https://arxiv.org/html/2607.16207#S6.SS2)and[Section˜F\.1](https://arxiv.org/html/2607.16207#A6.SS1)\. The selected positions are masked jointly,S=ℋK​\(x;ϕ\)S=\\mathcal\{H\}\_\{K\}\(x;\\phi\), and the attacker computesΔi​\(S\)\\Delta\_\{i\}\(S\)for alli∈Si\\in Susing[Eq\.˜3](https://arxiv.org/html/2607.16207#S3.E3)\. Since all positions inSSare reconstructed in parallel, this joint probe returnsKKtoken\-level membership signals with one target and one reference query\. Thus,KKaffects only the number of averaged token gaps, not the target/reference scoring NFE\.

The final statistic clips and averages the selected token gaps:

gJUMP​\(x;Mtgt,Mref\)=1K​∑i∈ℋK​\(x;ϕ\)clip​\(Δi​\(ℋK​\(x;ϕ\)\),−τ,τ\),τ=log⁡1\.5\.g\_\{\\mathrm\{JUMP\}\}\(x;M\_\{\\mathrm\{tgt\}\},M\_\{\\mathrm\{ref\}\}\)=\\frac\{1\}\{K\}\\sum\_\{i\\in\\mathcal\{H\}\_\{K\}\(x;\\phi\)\}\\mathrm\{clip\}\\\!\\left\(\\Delta\_\{i\}\(\\mathcal\{H\}\_\{K\}\(x;\\phi\)\),\-\\tau,\\tau\\right\),\\qquad\\tau=\\log 1\.5\.\(6\)We clip to bound the effect of any single token, limiting heavy\-tailed token gaps that can otherwise dominate the mean and create false positives at low FPR\. The thresholdτ=log⁡1\.5\\tau=\\log 1\.5caps each token’s contribution at a1\.5:11\.5\{:\}1probability ratio; we ablate this choice in[Section˜6\.3](https://arxiv.org/html/2607.16207#S6.SS3)and[Section˜F\.3](https://arxiv.org/html/2607.16207#A6.SS3)\.

#### Remark \(NFE and Efficiency\)\.

The scoring stage of JUMP is single\-pass: it requires two forward passes per sample \(one target and one reference reconstruction\), meaning its Number of Forward Evaluations \(NFE\) is𝒪​\(1\)\\mathcal\{O\}\(1\)with respect to the subset parametersKKandLL, as well as sequence lengthNN\. Because one joint masked input is evaluated simultaneously by each model, JUMP is significantly more efficient than alternative approaches\. By contrast, SAMA requires one target/reference pair of forward evaluations for each sampled mask subset, and exact one\-hole localization scales linearly, requiring an NFE of𝒪​\(T\)\\mathcal\{O\}\(T\)\(one pair for every sampled mask subset\)\. Finally, the overhead of learning the PRISM selector is negligible\. Training PRISM is a one\-time, offline procedure that imposes no per\-sample cost during testing\. Architecturally, the selector is highly efficient: one can train a simple linear head or, if greater capacity is required, a lightweight LoRA adapter\. Both formulations are computationally trivial to optimize\.

### 4\.3Generic token selector

The remaining question is how to obtainqϕq\_\{\\phi\}without running the expensive one\-hole diagnostic in[Section˜4\.1](https://arxiv.org/html/2607.16207#S4.SS1)\. In our default white\-box reference setting, we use PRISM\(Kimet al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib37)\)as a generic low\-confidence token selector\. Given the unmasked sequence, PRISM predicts a position\-wise low\-confidence score from the reference\-model representations:

qϕ​\(i∣x\)≈q⋆​\(i∣x\)=log⁡pMref​\(xi∣x∖\{i\}\)\.q\_\{\\phi\}\(i\\mid x\)\\approx q^\{\\star\}\(i\\mid x\)=\\log p\_\{M\_\{\\mathrm\{ref\}\}\}\(x\_\{i\}\\mid x\_\{\\setminus\\\{i\\\}\}\)\.PRISM is a lightweight per\-token quality head attached to the reference MDM\. It is fine\-tuned on text drawn from the reference model’s pretraining distribution using a binary cross\-entropy objective: for a partially masked sequence, a held\-out positioniiis filled in by a sample from the reference MDM’s one\-hole posterior \(with stop\-gradient on the unmasking head\), andqϕ​\(i∣x\)q\_\{\\phi\}\(i\\mid x\)is trained against the binary label indicating whether the sampled token matches the ground truth at positionii\. Under this objective, the optimum provably recovers the reference model’s per\-token quality at positioniigiven the surrounding context\(Kimet al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib37)\)\. We provide the PRISM training details in[Section˜A\.3](https://arxiv.org/html/2607.16207#A1.SS3)\. The quality head is fit only from text in the reference distribution, without any member/non\-member labels or target\-model information\. At attack time, the unmasking probabilities and the PRISM scoresqϕ​\(i∣x\)q\_\{\\phi\}\(i\\mid x\)are produced in a*single*reference\-model forward pass\.JUMPthen selects the set in[Eq\.˜5](https://arxiv.org/html/2607.16207#S4.E5)and performs the two target/reference scoring passes in[Eq\.˜6](https://arxiv.org/html/2607.16207#S4.E6)\.

#### Black\-box selector\.

We also evaluate a black\-box selector, denoted PRISM\-Free, that does not train an auxiliary head\. PRISM\-Free uses the reference model’s clean\-text true\-token score as a proxy for[Eq\.˜4](https://arxiv.org/html/2607.16207#S4.E4), selecting positions with large

qfree​\(i∣x\)=log⁡pMref​\(xi∣x\),\\displaystyle q\_\{\\mathrm\{free\}\}\(i\\mid x\)=\\log p\_\{M\_\{\\mathrm\{ref\}\}\}\(x\_\{i\}\\mid x\),where the input sequencexxis completely unmasked\. Theoretically, this score should be meaningless: dLLMs are trained exclusively to predict ground\-truth tokens from \[MASK\] tokens, meaning the output distribution over an already\-visible tokenxix\_\{i\}has no mathematical guarantee\. Surprisingly, we empirically found that for LLaDA, evaluating the model on the clean sequence yields a reasonable “one\-hole confidence” score, behaving remarkably as ifxix\_\{i\}had actually been masked\. Because this correlation is an empirical artifact of LLaDA rather than a guarantee for general dLLMs, whereas PRISM is explicitly trained to predict the parallel low\-confidence score in[Eq\.˜4](https://arxiv.org/html/2607.16207#S4.E4), we use PRISM as the default selector and report PRISM\-Free as a stricter\-access variant\.

## 5Experiments

Table 1:Per\-domain results on fine\-tuned LLaDA\-8B\-Base\. Attack quality: ROC\-AUC and TPR at fixed FPR \(↑\\uparrow\); attack cost: NFE per sample \(↓\\downarrow\)\. NFE is reported as target/reference scoring passes plus selector passes when applicable\. Best in each column inbold\.ArXivGitHubHackerNewsMethodAttack quality \(↑\\uparrow\)Cost \(↓\\downarrow\)Attack quality \(↑\\uparrow\)Cost \(↓\\downarrow\)Attack quality \(↑\\uparrow\)Cost \(↓\\downarrow\)AUCT@10T@1T@0\.1NFEAUCT@10T@1T@0\.1NFEAUCT@10T@1T@0\.1NFELoss0\.530\.120\.010\.0010\.590\.190\.040\.0110\.520\.110\.010\.011ZLIB0\.530\.130\.010\.0010\.610\.220\.060\.0210\.520\.100\.010\.011SAMA0\.820\.540\.280\.072​T2T0\.770\.430\.140\.072​T2T0\.710\.340\.050\.012​T2TJUMP0\.940\.840\.540\.242\+12\{\+\}10\.860\.670\.330\.122\+12\{\+\}10\.770\.400\.110\.032\+12\{\+\}1PubMed CentralWikipedia \(en\)Pile\-CCMethodAttack quality \(↑\\uparrow\)Cost \(↓\\downarrow\)Attack quality \(↑\\uparrow\)Cost \(↓\\downarrow\)Attack quality \(↑\\uparrow\)Cost \(↓\\downarrow\)AUCT@10T@1T@0\.1NFEAUCT@10T@1T@0\.1NFEAUCT@10T@1T@0\.1NFELoss0\.530\.130\.020\.0010\.520\.100\.010\.0010\.520\.120\.010\.001ZLIB0\.530\.140\.010\.0010\.520\.100\.010\.0010\.520\.130\.020\.001SAMA0\.810\.510\.230\.032​T2T0\.910\.730\.470\.362​T2T0\.900\.700\.400\.202​T2TJUMP0\.920\.750\.440\.172\+12\{\+\}10\.980\.950\.800\.682\+12\{\+\}10\.960\.900\.560\.172\+12\{\+\}1### 5\.1Experimental setup

#### Models\.

We evaluate LLaDA\-8B\-Base\(Nieet al\.,[2025](https://arxiv.org/html/2607.16207#bib.bib26)\)\. For each domain, the target modelMtgtM\_\{\\mathrm\{tgt\}\}is obtained by fine\-tuning LLaDA\-8B\-Base on that domain’s member split\. The reference modelMrefM\_\{\\mathrm\{ref\}\}is the corresponding checkpoint before fine\-tuning\. We additionally report secondary Dream\-v0\-7B\-Base results in[Appendix˜C](https://arxiv.org/html/2607.16207#A3)\.

#### Data\.

We use six domains from MIMIR\(Duanet al\.,[2024](https://arxiv.org/html/2607.16207#bib.bib15)\): ArXiv, GitHub, HackerNews, Pile\-CC, PubMed Central, and Wikipedia \(en\)\. Each evaluation set contains 1,000 member examples and 1,000 non\-member examples from the same domain\. Members are the exact examples used for fine\-tuning, and non\-members are held\-out examples\.

#### Baselines\.

We compare against three baselines\.Lossuses the target model’s average sequence log\-probability as a target\-only statistic\(Yeomet al\.,[2018](https://arxiv.org/html/2607.16207#bib.bib2)\)\.ZLIBnormalizes the loss statistic by zlib\-compressed length\(Carliniet al\.,[2021](https://arxiv.org/html/2607.16207#bib.bib7)\)\.SAMAis the prior dLLM\-specific MIA and averages reconstruction statistics over randomly sampled mask subsets\(Chenet al\.,[2026](https://arxiv.org/html/2607.16207#bib.bib19)\)\. Loss and ZLIB are included as sanity checks for likelihood\-style attacks; SAMA is the primary dLLM baseline\.

#### Metrics and configuration\.

We report ROC\-AUC and TPR at FPR∈\{10%,1%,0\.1%\}\\in\\\{10\\%,1\\%,0\.1\\%\\\}\(Carliniet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib4)\), along with NFE per sample \(separating target/reference scoring from selector passes\)\. Unless stated otherwise,JUMPuses a PRISM selector trained on C4\(Raffelet al\.,[2020](https://arxiv.org/html/2607.16207#bib.bib38)\), a 512\-token window,K=64K=64selected positions, clipping thresholdτ=log⁡1\.5\\tau=\\log 1\.5, and batch size 8\.KKaffects the number of aggregated token\-level signals but not the target/reference scoring NFE\. See[Appendix˜A](https://arxiv.org/html/2607.16207#A1)for additional details\.

### 5\.2Performance and efficiency across domains

[Table˜1](https://arxiv.org/html/2607.16207#S5.T1)reports per\-domain LLaDA results, with 95% bootstrap confidence intervals provided in[Appendix˜B](https://arxiv.org/html/2607.16207#A2); additional aggregate controls and Dream results are provided in[Appendix˜C](https://arxiv.org/html/2607.16207#A3)\.JUMPimproves ROC\-AUC over SAMA in all six domains\. The improvement is also visible in the strict low\-FPR regime, where high\-confidence auditing is most relevant\. On cost,JUMPuses two target/reference scoring passes per sample regardless of the number of selected positions, plus one reference\-side selector pass in the default PRISM variant; SAMA instead requires a separate target/reference pair for every random mask subset\. Loss and ZLIB remain close to chance on most domains, consistent with prior evidence that uncalibrated sequence likelihood is a weak membership signal for LLM\-scale models\(Duanet al\.,[2024](https://arxiv.org/html/2607.16207#bib.bib15)\)\.

### 5\.3NFE analysis

[Table˜2](https://arxiv.org/html/2607.16207#S5.T2)compares the number of forward evaluations required by representative MIA strategies\. We useLLfor the sequence length andTTfor the number of random mask subsets sampled by SAMA\.

For autoregressive \(AR\) models, computing a target/reference loss statistic requires2​L2Lforward evaluations under a per\-position score access model: the attacker queries the model once per position to obtainlog⁡p​\(xi∣x<i\)\\log p\(x\_\{i\}\\mid x\_\{<i\}\), yieldingLLtarget passes andLLreference passes\. We adopt this per\-position access model because it places AR and dLLM attacks on a common footing, where the cost is the number of independent forward evaluations required to obtain a token\-level score at every position\.

For dLLMs, exact one\-hole localization masks each position separately and therefore requires2​L2Ltarget/reference reconstruction passes\. SAMA reduces this cost by samplingTTrandom mask sets, but still requires2​T2Tpasses\. In contrast,JUMPselectsKKpositions and scores them through a single joint mask\. By exploiting parallel decoding, the target/reference MIA statistic costs exactly two scoring passes, independent ofKK,LL, andTT; the default PRISM selector adds only one reference\-model pass for position selection\.

Table 2:NFE comparison across MIA strategies\.LLis sequence length andTTis the number of SAMA random mask subsets\. Scoring NFE counts target/reference passes used to compute the final statistic\. ForJUMP, the parenthesized term denotes the additional reference\-model pass used by the PRISM selector\.ModelStrategy / AccessContextScoring NFEARTarget / Reference lossleft\-to\-right2​L2LdLLMOne\-hole probingbidirectional2​L2LdLLMSAMAbidirectional2​T2TdLLMJUMP\(KKselected tokens\)bidirectional2​\(\+1\)2\\,\(\+1\)

## 6Ablation Study

This section analyzes the core components ofJUMPto isolate their individual contributions to the overall attack performance\. Specifically, the following experiments investigate the sensitivity of the learned selector to its training corpus, the effect of the selected\-token budget, the importance of score clipping for low\-FPR detection, and the viability of a training\-free selector\.

### 6\.1Selector training corpus

The first ablation tests whether the attack relies on the PRISM selector’s training corpus\. Three separate PRISM heads are trained on C4\(Raffelet al\.,[2020](https://arxiv.org/html/2607.16207#bib.bib38)\), SlimPajama\(Sobolevaet al\.,[2023](https://arxiv.org/html/2607.16207#bib.bib40)\), and FineWeb\(Penedoet al\.,[2024](https://arxiv.org/html/2607.16207#bib.bib39)\), and evaluated using the standardJUMPpipeline\.

![Refer to caption](https://arxiv.org/html/2607.16207v1/x4.png)

![Refer to caption](https://arxiv.org/html/2607.16207v1/x5.png)

Figure 3:Selector training corpus ablation\.ROC\-AUC across MIMIR domains for PRISM selectors trained on different generic corpora\. Learned selectors cluster within a narrow band, indicating that corpus identity is a second\-order effect\.The learned selectors exhibit nearly identical performance across domains\. PRISM\-C4 is marginally best, but differences among the models are negligible\. This stability suggests PRISM captures a generic notion of reconstruction difficulty rather than corpus\-specific artifacts\.

### 6\.2Selected\-token budget

![Refer to caption](https://arxiv.org/html/2607.16207v1/x6.png)

![Refer to caption](https://arxiv.org/html/2607.16207v1/x7.png)

Figure 4:Selected\-token budget ablation\.ROC\-AUC remains stable around the defaultK=64K=64\.The next experiment evaluates sensitivity to the selected\-token budget \(KK\), keeping other components fixed\. Since selected positions are reconstructed in parallel, changingKKonly alters the number of aggregated token\-level gaps without affecting the target/reference scoring NFE\.

[Figure˜4](https://arxiv.org/html/2607.16207#S6.F4)shows thatJUMPis robust across budgets\. Using too few positions weakens the aggregate signal, while increasing beyondK=64K=64starts to add less\-informative positions without consistently improving ROC\-AUC\. We therefore useK=64K=64as a compact operating point: it probes one\-eighth of a 512\-token window, yields a stable signal, and preserves the single\-pass target/reference scoring cost\. Full budget details are provided in[Section˜F\.1](https://arxiv.org/html/2607.16207#A6.SS1)\.

### 6\.3Clipping and threshold sensitivity

![Refer to caption](https://arxiv.org/html/2607.16207v1/x8.png)

![Refer to caption](https://arxiv.org/html/2607.16207v1/x9.png)

\(a\)Threshold sweep
![Refer to caption](https://arxiv.org/html/2607.16207v1/x10.png)

![Refer to caption](https://arxiv.org/html/2607.16207v1/x11.png)

\(b\)Effect of clipping

Figure 5:Clipping ablation\.\(a\) Sweeping the clipping threshold constantccshows that JUMP is not overly sensitive around the default setting\. \(b\) Clipping improves detection performance and robustness, with the largest gains in the strict low\-FPR regime\.We next examine the role of clipping inJUMP\.[Figure˜5\(a\)](https://arxiv.org/html/2607.16207#S6.F5.sf1)sweeps the clipping threshold and shows that performance is stable over a range of moderate clipping constants\. Although a smaller threshold can be slightly better for a specific domain–metric pair, such as HackerNews AUC, we select the default thresholdc=log⁡1\.5c=\\log 1\.5based on the aggregate behavior across all domains and evaluation metrics\. In particular, the full sweep in[Section˜F\.3](https://arxiv.org/html/2607.16207#A6.SS3)shows thatc=log⁡1\.5c=\\log 1\.5gives the best overall operating point when low\-FPR metrics are included\.

[Figure˜5\(b\)](https://arxiv.org/html/2607.16207#S6.F5.sf2)compares the clipped and unclipped variants\. Clipping improves all metrics, with the largest gains appearing at lower FPRs\. At strict operating points, a small number of extreme token\-level gaps can dominate the unclipped mean and create unstable false positives\. Clipping suppresses these outliers and instead favors sequences that exhibit a consistent target–reference advantage across multiple selected positions\.

### 6\.4PRISM\-Free

Table 3:PRISM\-Free per\-domain results\. Metrics and formatting follow Table[1](https://arxiv.org/html/2607.16207#S5.T1)\. Because PRISM\-Free requires no training, NFE reflects only target/reference scoring passes\.DomainAUCTPR@10%TPR@1%TPR@0\.1%ArXiv0\.890\.670\.320\.12GitHub0\.860\.650\.220\.13HackerNews0\.760\.360\.070\.01Pile\-CC0\.940\.830\.510\.18PubMed0\.820\.540\.170\.01Wikipedia \(en\)0\.960\.890\.630\.36Mean0\.870\.660\.320\.14PRISM\-Free replaces the learned PRISM head with the reference model’s clean\-text true\-token score, serving as a stricter\-access variant ofJUMPthat requires zero auxiliary training\. Impressively, this simple proxy proves highly effective: PRISM\-Free achieves a mean ROC\-AUC of0\.870\.87, comfortably outperforming SAMA \(0\.820\.82\)\. While the explicitly trained PRISM\-C4 head provides marginally better stability at extremely low FPRs \([Table˜3](https://arxiv.org/html/2607.16207#S6.T3)\) by explicitly isolating bidirectional uncertainty, PRISM\-Free demonstrates that clean\-text probability is a remarkably strong proxy for reconstruction difficulty\. For most practical applications, PRISM\-Free is more than sufficient, delivering highly competitive membership inference capabilities without any training overhead\.

## 7Conclusion

JUMPleverages any\-order decodability to identify vulnerable positions and parallel decodability to evaluate all probes simultaneously\. Across fine\-tuned LLaDA and Dream models, this approach yields a membership inference attack that is both stronger and more computationally efficient than SAMA\. On the defense side, our DP\-LoRA results \([Appendix˜H](https://arxiv.org/html/2607.16207#A8)\) show that parameter\-efficient fine\-tuning with differential privacy offers a practical mitigation against uncertainty\-guided membership inference, building on the DP\-SGD framework\(Abadiet al\.,[2016](https://arxiv.org/html/2607.16207#bib.bib41)\)and recent progress in differentially private language\-model fine\-tuning\(Yuet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib42); Liet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib43)\)\. Ultimately,JUMPestablishes a rigorous baseline for future dLLM privacy evaluations\.

#### Limitations\.

Our current evaluation focuses on fine\-tuned dLLMs to cleanly isolate architectural vulnerabilities from the inherent ambiguities of pretraining\-scale membership\. Extending this analysis to pretrained models remains a critical direction for future work\. Additionally, while our primary attack relies on a matched reference model to calibrate difficulty, the success of our PRISM\-Free variant demonstrates that effective single\-pass probing does not strictly require target internals\.

## References

- M\. Abadi, A\. Chu, I\. Goodfellow, H\. B\. McMahan, I\. Mironov, K\. Talwar, and L\. Zhang \(2016\)Deep learning with differential privacy\.InACM CCS,Cited by:[§7](https://arxiv.org/html/2607.16207#S7.p1.1)\.
- J\. Austin, D\. D\. Johnson, J\. Ho, D\. Tarlow, and R\. van den Berg \(2021\)Structured denoising diffusion models in discrete state\-spaces\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.5)\.
- N\. Carlini, S\. Chien, M\. Nasr, S\. Song, A\. Terzis, and F\. Tramer \(2022\)Membership inference attacks from first principles\.InIEEE Symposium on Security and Privacy \(S&P\),Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9),[§3](https://arxiv.org/html/2607.16207#S3.SS0.SSS0.Px4.p1.5),[§5\.1](https://arxiv.org/html/2607.16207#S5.SS1.SSS0.Px4.p1.4)\.
- N\. Carlini, D\. Ippolito, M\. Jagielski, K\. Lee, F\. Tramer, and C\. Zhang \(2023\)Quantifying memorization across neural language models\.InICLR,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9)\.
- N\. Carlini, C\. Liu, Ú\. Erlingsson, J\. Kos, and D\. Song \(2019\)The secret sharer: evaluating and testing unintended memorization in neural networks\.InUSENIX Security,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9)\.
- N\. Carlini, F\. Tramer, E\. Wallace, M\. Jagielski, A\. Herbert\-Voss, K\. Lee, A\. Roberts, T\. Brown, D\. Song, U\. Erlingsson,et al\.\(2021\)Extracting training data from large language models\.InUSENIX Security,Cited by:[§1](https://arxiv.org/html/2607.16207#S1.p2.2),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9),[§5\.1](https://arxiv.org/html/2607.16207#S5.SS1.SSS0.Px3.p1.1)\.
- H\. Chang, A\. Shahin Shamsabadi, K\. Katevas, H\. Haddadi, and R\. Shokri \(2025\)Context\-aware membership inference attacks against pre\-trained large language models\.InEMNLP,Cited by:[§1](https://arxiv.org/html/2607.16207#S1.p2.2),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px2.p1.3)\.
- H\. Chang, H\. Zhang, L\. Jiang, C\. Liu, and W\. T\. Freeman \(2022\)MaskGIT: masked generative image transformer\.InCVPR,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.7)\.
- Y\. Chen, K\. Zhang, Y\. Du, E\. Stoppa, C\. Fleming, A\. Kundu, B\. Ribeiro, and N\. Li \(2026\)Membership inference attacks against fine\-tuned diffusion language models\.InICLR,Cited by:[Table 16](https://arxiv.org/html/2607.16207#A10.T16.4.8.3.1.1),[§1](https://arxiv.org/html/2607.16207#S1.p4.1),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px4.p1.2),[§5\.1](https://arxiv.org/html/2607.16207#S5.SS1.SSS0.Px3.p1.1)\.
- M\. Duan, A\. Suri, N\. Mireshghallah, S\. Min, W\. Shi, L\. Zettlemoyer, Y\. Tsvetkov, Y\. Choi, D\. Evans, and H\. Hajishirzi \(2024\)Do membership inference attacks work on large language models?\.InCOLM,Cited by:[Table 16](https://arxiv.org/html/2607.16207#A10.T16.4.4.3.1.1),[§1](https://arxiv.org/html/2607.16207#S1.p2.2),[§1](https://arxiv.org/html/2607.16207#S1.p6.1),[§5\.1](https://arxiv.org/html/2607.16207#S5.SS1.SSS0.Px2.p1.1),[§5\.2](https://arxiv.org/html/2607.16207#S5.SS2.p1.1)\.
- M\. Germain, K\. Gregor, I\. Murray, and H\. Larochelle \(2015\)MADE: masked autoencoder for distribution estimation\.InICML,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.7)\.
- M\. Ghazvininejad, O\. Levy, Y\. Liu, and L\. Zettlemoyer \(2019\)Mask\-predict: parallel decoding of conditional masked language models\.InEMNLP\-IJCNLP,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.7)\.
- E\. Hoogeboom, A\. A\. Gritsenko, J\. Bastings, B\. Poole, R\. van den Berg, and T\. Salimans \(2022\)Autoregressive diffusion models\.InICLR,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.7)\.
- E\. Hoogeboom, D\. Nielsen, P\. Jaini, P\. Forré, and M\. Welling \(2021\)Argmax flows and multinomial diffusion: learning categorical distributions\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.5)\.
- M\. Jagielski, O\. Thakkar, F\. Tramer, D\. Ippolito, K\. Lee, N\. Carlini, E\. Wallace, S\. Song, A\. G\. Thakurta, N\. Papernot, and C\. Zhang \(2023\)Measuring forgetting of memorized training examples\.InICLR,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9)\.
- J\. Kim, S\. Kim, T\. Lee, D\. Z\. Pan, H\. Kim, S\. Kakade, and S\. Chen \(2025\)Fine\-tuning masked diffusion for provable self\-correction\.arXiv preprint arXiv:2510\.01384\.Cited by:[§4\.3](https://arxiv.org/html/2607.16207#S4.SS3.p1.1),[§4\.3](https://arxiv.org/html/2607.16207#S4.SS3.p1.6)\.
- X\. Li, F\. Tramer, P\. Liang, and T\. Hashimoto \(2022\)Large language models can be strong differentially private learners\.InICLR,Cited by:[Appendix H](https://arxiv.org/html/2607.16207#A8.p1.1),[§7](https://arxiv.org/html/2607.16207#S7.p1.1)\.
- Y\. Long, V\. Bindschaedler, L\. Wang, D\. Bu, X\. Wang, H\. Tang, C\. A\. Gunter, and K\. Chen \(2018\)Understanding membership inferences on well\-generalized learning models\.arXiv preprint arXiv:1802\.04889\.Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9)\.
- A\. Lou, C\. Meng, and S\. Ermon \(2024\)Discrete diffusion modeling by estimating the ratios of the data distribution\.InICML,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.5)\.
- N\. Lukas, A\. Salem, R\. Sim, S\. Tople, L\. Wutschitz, and S\. Zanella\-Béguelin \(2023\)Analyzing leakage of personally identifiable information in language models\.InIEEE Symposium on Security and Privacy \(S&P\),Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9)\.
- J\. Mattern, F\. Mireshghallah, Z\. Jin, B\. Schölkopf, M\. Sachan, and T\. Berg\-Kirkpatrick \(2023\)Membership inference attacks against language models via neighbourhood comparison\.InACL Findings,Cited by:[§1](https://arxiv.org/html/2607.16207#S1.p2.2),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px2.p1.3)\.
- F\. Mireshghallah, K\. Goyal, A\. Uniyal, T\. Berg\-Kirkpatrick, and R\. Shokri \(2022\)Quantifying privacy risks of masked language models using membership inference attacks\.InEMNLP,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9)\.
- M\. Nasr, J\. Rando, N\. Carlini, J\. Hayase, M\. Jagielski, A\. F\. Cooper, D\. Ippolito, C\. A\. Choquette\-Choo, F\. Tramèr, and K\. Lee \(2025\)Scalable extraction of training data from aligned, production language models\.InICLR,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9)\.
- S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. Li \(2025\)Large language diffusion models\.InNeurIPS,Cited by:[Table 16](https://arxiv.org/html/2607.16207#A10.T16.4.2.3.1.1),[§1](https://arxiv.org/html/2607.16207#S1.p3.1),[§1](https://arxiv.org/html/2607.16207#S1.p6.1),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.5),[§5\.1](https://arxiv.org/html/2607.16207#S5.SS1.SSS0.Px1.p1.2)\.
- J\. Ou, S\. Nie, K\. Xue, F\. Zhu, J\. Sun, Z\. Li, and C\. Li \(2025\)Your absorbing discrete diffusion secretly models the conditional distributions of clean data\.InICLR,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.5)\.
- G\. Penedo, H\. Kydlíček, L\. B\. allal, A\. Lozhkov, M\. Mitchell, C\. Raffel, L\. V\. Werra, and T\. Wolf \(2024\)The fineweb datasets: decanting the web for the finest text data at scale\.InNeurIPS Datasets and Benchmarks Track,Cited by:[Table 16](https://arxiv.org/html/2607.16207#A10.T16.4.7.3.1.1),[§6\.1](https://arxiv.org/html/2607.16207#S6.SS1.p1.1)\.
- C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. Liu \(2020\)Exploring the limits of transfer learning with a unified text\-to\-text transformer\.JMLR\.Cited by:[Table 16](https://arxiv.org/html/2607.16207#A10.T16.4.5.3.1.1),[§5\.1](https://arxiv.org/html/2607.16207#S5.SS1.SSS0.Px4.p1.4),[§6\.1](https://arxiv.org/html/2607.16207#S6.SS1.p1.1)\.
- S\. S\. Sahoo, M\. Arriola, Y\. Schiff, A\. Gokaslan, E\. Marroquin, J\. T\. Chiu, A\. Rush, and V\. Kuleshov \(2024\)Simple and effective masked diffusion language models\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.5)\.
- J\. Shi, K\. Han, Z\. Wang, A\. Doucet, and M\. K\. Titsias \(2024a\)Simplified and generalized masked diffusion for discrete data\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.5)\.
- W\. Shi, A\. Ajith, M\. Xia, Y\. Huang, D\. Liu, T\. Blevins, D\. Chen, and L\. Zettlemoyer \(2024b\)Detecting pretraining data from large language models\.InICLR,Cited by:[§1](https://arxiv.org/html/2607.16207#S1.p2.2),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px2.p1.3)\.
- R\. Shokri, M\. Stronati, C\. Song, and V\. Shmatikov \(2017\)Membership inference attacks against machine learning models\.InIEEE Symposium on Security and Privacy \(S&P\),Cited by:[§1](https://arxiv.org/html/2607.16207#S1.p1.1),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.4)\.
- D\. Soboleva, F\. Al\-Khateeb, R\. Myers, J\. R\. Steeves, J\. Hestness, and N\. Dey \(2023\)SlimPajama: a 627b token cleaned and deduplicated version of RedPajama\.External Links:[Link](https://huggingface.co/datasets/cerebras/SlimPajama-627B)Cited by:[Table 16](https://arxiv.org/html/2607.16207#A10.T16.4.6.3.1.1),[§6\.1](https://arxiv.org/html/2607.16207#S6.SS1.p1.1)\.
- T\. Steinke, M\. Nasr, and M\. Jagielski \(2023\)Privacy auditing with one \(1\) training run\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9)\.
- J\. Tao and R\. Shokri \(2026\)Information\-theoretic membership inference for granular quantification of memorization\.InICLR,Cited by:[§1](https://arxiv.org/html/2607.16207#S1.p2.2),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px2.p1.3)\.
- B\. Uria, I\. Murray, and H\. Larochelle \(2014\)A deep and tractable density estimator\.InICML,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.7)\.
- L\. Watson, C\. Guo, G\. Cormode, and A\. Sablayrolles \(2022\)On the importance of difficulty calibration in membership inference attacks\.InICLR,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9),[§3](https://arxiv.org/html/2607.16207#S3.SS0.SSS0.Px4.p1.5)\.
- R\. Xie, J\. Wang, R\. Huang, M\. Zhang, R\. Ge, J\. Pei, N\. Z\. Gong, and B\. Dhingra \(2024\)ReCaLL: membership inference via relative conditional log\-likelihoods\.InEMNLP,Cited by:[§1](https://arxiv.org/html/2607.16207#S1.p2.2),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px2.p1.3)\.
- Z\. Yang, Z\. Dai, Y\. Yang, J\. Carbonell, R\. R\. Salakhutdinov, and Q\. V\. Le \(2019\)XLNet: generalized autoregressive pretraining for language understanding\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.7)\.
- J\. Ye, Z\. Xie, L\. Zheng, J\. Gao, Z\. Wu, X\. Jiang, Z\. Li, and L\. Kong \(2025\)Dream 7b: diffusion large language models\.arXiv preprint arXiv:2508\.15487\.Cited by:[Table 16](https://arxiv.org/html/2607.16207#A10.T16.4.3.3.1.1),[§1](https://arxiv.org/html/2607.16207#S1.p3.1),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px3.p1.5)\.
- S\. Yeom, I\. Giacomelli, M\. Fredrikson, and S\. Jha \(2018\)Privacy risk in machine learning: analyzing the connection to overfitting\.InIEEE Computer Security Foundations Symposium \(CSF\),Cited by:[§1](https://arxiv.org/html/2607.16207#S1.p1.1),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.4),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9),[§5\.1](https://arxiv.org/html/2607.16207#S5.SS1.SSS0.Px3.p1.1)\.
- D\. Yu, S\. Naik, A\. Backurs, S\. Gopi, H\. A\. Inan, G\. Kamath, J\. Kulkarni, Y\. T\. Lee, A\. Manoel, L\. Wutschitz, S\. Yekhanin, and H\. Zhang \(2022\)Differentially private fine\-tuning of language models\.InICLR,Cited by:[§H\.5](https://arxiv.org/html/2607.16207#A8.SS5.p2.2),[Appendix H](https://arxiv.org/html/2607.16207#A8.p1.1),[§7](https://arxiv.org/html/2607.16207#S7.p1.1)\.
- S\. Zarifzadeh, P\. Liu, and R\. Shokri \(2024\)Low\-cost high\-power membership inference attacks\.InICML,Cited by:[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px1.p1.9),[§3](https://arxiv.org/html/2607.16207#S3.SS0.SSS0.Px4.p1.5)\.
- J\. Zhang, J\. Sun, E\. Yeats, Y\. Ouyang, M\. Kuo, J\. Zhang, H\. F\. Yang, and H\. Li \(2025\)Min\-k%\+\+: improved baseline for pre\-training data detection from large language models\.InICLR,Cited by:[§1](https://arxiv.org/html/2607.16207#S1.p2.2),[§2](https://arxiv.org/html/2607.16207#S2.SS0.SSS0.Px2.p1.3)\.

## Appendix AExperimental Details

### A\.1Threat Model and Access Assumptions

The attacker has score access to the target modelMtgtM\_\{\\mathrm\{tgt\}\}: for a chosen mask set, the attacker can query masked inputs and observe true\-token reconstruction log\-probabilities at the masked positions\. The attacker also has access to a compatible reference modelMrefM\_\{\\mathrm\{ref\}\}\. The learned PRISM selector is trained using reference\-model representations and generic public text only; it does not use target\-model representations, member/non\-member labels, or the evaluation examples\. PRISM\-Free removes this auxiliary training step and uses only the reference model’s clean true\-token score as a selector proxy\.

### A\.2Target Model Training Setup

For the LLaDA experiments, each target model is initialized fromGSAI\-ML/ LLaDA\-8B\-Baseand fine\-tuned separately for each MIMIR domain\. For each domain, we use 1,000 member sequences for target\-model fine\-tuning and evaluate membership inference on a disjoint set of 1,000 held\-out member examples and 1,000 non\-member examples\. Unless otherwise noted, the same fine\-tuning recipe is used for all six domains\.

We fine\-tune with AdamW for 4 epochs using a learning rate of5×10−55\\times 10^\{\-5\}, weight decay0\.10\.1, and a linear learning\-rate schedule with 500 warmup steps\. The maximum sequence length is 512 tokens\. Training is performed on 4 NVIDIA L40S GPUs inbf16with DeepSpeed ZeRO\-3 and gradient checkpointing enabled\. We use per\-device batch size 1 and 12 gradient\-accumulation steps, giving an effective batch size of 48 sequences per optimizer step\.

Checkpoint selection is based on validation loss with early stopping\. We monitoreval\_lossand retain the best checkpoint under patience 3 and threshold 0\.0\. In practice, the retained checkpoints are typically the best saved checkpoints aroundcheckpoint\-80, and these checkpoints are used for all reported MIA evaluations\.

### A\.3PRISM Selector Training

Our default LLaDA PRISM selector is trained on a materialized 200k\-sample English C4 subset\. We split the materialized data into 190k training sequences and 10k held\-out sequences; after preprocessing, the matched C4 run retains 189,919 training sequences\. The selector is initialized from the baseGSAI\-ML/LLaDA\-8B\-Basebackbone and trained for one epoch on 4 NVIDIA L40S GPUs in fp16 precision\. The per\-GPU batch size is 4, with 2 gradient\-accumulation steps, giving an effective global batch size of 32\. We use maximum sequence length 256, AdamW with learning rate10−410^\{\-4\}, weight decay 0, and random seed 42\.

We train only the PRISM quality head and LoRA adapters, leaving the reference backbone otherwise fixed\. This updates 29\.38M trainable parameters out of 8\.04B total parameters, corresponding to 0\.37% of the model\. LoRA is applied only to the query, key, and value projection modules, with rank 16, alpha 16, and dropout 0\.1\.

The training objective follows the PRISM self\-correction formulation rather than direct regression to the one\-hole score\. For each clean sequencex0x\_\{0\}, we sample a masking ratiot∼Uniform​\(10−3,1\)t\\sim\\mathrm\{Uniform\}\(10^\{\-3\},1\)and independently mask each valid token with probabilityttto obtainxtx\_\{t\}\. The backbone predicts masked\-token reconstruction logits onxtx\_\{t\}\. We then chooseKupd=8K\_\{\\mathrm\{upd\}\}=8masked positions, fill them with their argmax predictions to form a one\-step updated sequence, and train the quality head to predict whether each updated token matches the ground\-truth token\. Thus, the quality label is generated internally from reconstruction correctness, not from external annotations, member/non\-member labels, or target\-model information\.

The retained objective is

ℒPRISM=ℒBCE\+0\.1​ℒCE,\\mathcal\{L\}\_\{\\mathrm\{PRISM\}\}=\\mathcal\{L\}\_\{\\mathrm\{BCE\}\}\+0\.1\\,\\mathcal\{L\}\_\{\\mathrm\{CE\}\},whereℒBCE\\mathcal\{L\}\_\{\\mathrm\{BCE\}\}is the self\-correction binary cross\-entropy loss on the one\-step updated positions, andℒCE\\mathcal\{L\}\_\{\\mathrm\{CE\}\}is an auxiliary masked\-token reconstruction cross\-entropy loss over masked positions\. Following the PRISM recipe, the reconstruction CE term is reweighted by the inverse masking ratio\. No member/non\-member labels, target\-model weights, or target\-model hidden states are used during selector training\.

For the retained C4\-200k checkpoint, training terminates after 2,968 optimizer steps with mean training loss 0\.6535\. The total loss decreases from 0\.7617 at step 50 to 0\.6433 at step 2950, while the BCE component decreases from 0\.5493 to 0\.4309\. The checkpoint is saved after approximately 1,613 seconds, or 26\.9 minutes\. We do not use validation correlation for model selection in this retained run; instead, the selector is the epoch\-1 checkpoint from this fixed one\-epoch recipe, and its utility is evaluated through downstream MIA performance\.

### A\.4Baseline Configurations

For the SAMA baseline, we use our retained evaluation configuration with 16 progressive masking steps per example\. At each step, SAMA enlarges the cumulative masked context according to its masking schedule and then samples 128 random local mask subsets from the valid token positions\. Each subset contains 10 valid token positions\. For every sampled subset, the attack evaluates the target model and the reference model under the corresponding masked input and records whether the target/ reference loss comparison favors membership\. These subset\-level signals are then aggregated over all 128 subsets and all 16 masking steps to produce the final membership score\. Accordingly, the cost of SAMA is driven by both the multi\-step masking schedule and the large number of repeated random subset evaluations at each step\.

All attacks, including SAMA, are evaluated on the same 512\-token windows with batch size 8\. When reporting runtime, we normalize wall\-clock measurements to seconds per sample, even though batching is used internally for efficiency\. This normalization reflects the per\-example auditing cost faced by the attacker and prevents repeated\- query attacks such as SAMA from appearing artificially cheap due to implementation\- level batching\.

## Appendix BBootstrap Confidence Intervals for Main Results

[Table˜4](https://arxiv.org/html/2607.16207#A2.T4)reports the same per\-domainJUMPresults as[Table˜1](https://arxiv.org/html/2607.16207#S5.T1)\(main paper Table 1\), augmented with 95% bootstrap confidence intervals to quantify uncertainty due to the finite evaluation split\.

We use stratified bootstrap over evaluation examples\. For each domain, we resample the 1,000 member and 1,000 non\-member examples with replacement for 2,000 bootstrap replicates, recompute ROC\-AUC and TPR at fixed FPR, and report percentile 95% confidence intervals\. We do not assume Normality of the resulting bootstrap distributions\.

We note that confidence intervals at the strictest operating point \(TPR@0\.1%FPR\) are inherently wide: this metric is determined by only the few most extreme non\-member scores out of 1,000, making it highly sensitive to bootstrap resampling\. ROC\-AUC and TPR at less strict FPRs, which depend on the full score distribution, exhibit substantially tighter intervals\.

Table 4:Per\-domainJUMPresults on fine\-tuned LLaDA\-8B\-Base, with 95% bootstrap CIs in brackets\.DomainAUCT@10T@1T@0\.1ArXiv0\.94 \[\.93,\.95\]0\.84 \[\.80,\.86\]0\.54 \[\.45,\.60\]0\.24 \[\.10,\.47\]GitHub0\.86 \[\.84,\.87\]0\.67 \[\.62,\.70\]0\.33 \[\.24,\.43\]0\.12 \[\.00,\.26\]HackerNews0\.77 \[\.74,\.78\]0\.40 \[\.34,\.45\]0\.11 \[\.07,\.16\]0\.03 \[\.00,\.08\]PubMed Central0\.92 \[\.90,\.93\]0\.75 \[\.71,\.80\]0\.44 \[\.31,\.52\]0\.17 \[\.04,\.32\]Wikipedia \(en\)0\.98 \[\.97,\.99\]0\.95 \[\.94,\.97\]0\.80 \[\.76,\.85\]0\.68 \[\.61,\.78\]Pile\-CC0\.96 \[\.95,\.97\]0\.90 \[\.88,\.93\]0\.56 \[\.52,\.67\]0\.17 \[\.01,\.53\]
## Appendix CDream result

We additionally evaluate Dream\-v0\-Base\-7B as a secondary dLLM check\. These experiments are not used as the main evidence forJUMP, but test whether joint multimask probing remains meaningful beyond LLaDA\. In the saved Dream results used here, the retained Dream PRISM configuration uses the same joint probing budget, K=64, with fixed clipped\-gap aggregation\.

Table 5:Dream aggregate results averaged over six MIMIR domains\.MethodROC\-AUCTPR@10%TPR@1%TPR@0\.1%Loss0\.54610\.14220\.02050\.0055ZLIB0\.55080\.15020\.02750\.0082Dream PRISM0\.82370\.55870\.17140\.0493Dream SAMA0\.85130\.61220\.28620\.1660Table[5](https://arxiv.org/html/2607.16207#A3.T5)shows that the broad qualitative pattern transfers to Dream\. Vanilla likelihood baselines remain weak, while joint multimask probing produces a much stronger membership signal\. Dream PRISM substantially improves over Loss and ZLIB across all metrics, indicating that selector\-guided joint probing is not specific to LLaDA\. In the current Dream setup, however, Dream SAMA remains stronger than Dream PRISM, especially at stricter FPRs\. We therefore use Dream as a secondary\-model analysis rather than as the main evidence forJUMP\.

Table 6:Domain\-wise Dream PRISM and Dream SAMA results\. Each entry reports ROC\-AUC / TPR@10%FPR / TPR@1%FPR / TPR@0\.1%FPR\.DomainDream PRISMDream SAMAArXiv0\.7841 / 0\.4679 / 0\.1585 / 0\.04420\.8435 / 0\.5680 / 0\.2920 / 0\.2230GitHub0\.8135 / 0\.5485 / 0\.0876 / 0\.00380\.8332 / 0\.6180 / 0\.2590 / 0\.1190HackerNews0\.8257 / 0\.5212 / 0\.1669 / 0\.05310\.8295 / 0\.5670 / 0\.1710 / 0\.0540Pile\-CC0\.8652 / 0\.6608 / 0\.2620 / 0\.14880\.8972 / 0\.7160 / 0\.3470 / 0\.1910PubMed Central0\.8153 / 0\.5486 / 0\.1218 / 0\.03010\.8425 / 0\.5850 / 0\.3030 / 0\.1630Wikipedia \(en\)0\.8384 / 0\.6054 / 0\.2314 / 0\.01580\.8621 / 0\.6190 / 0\.3450 / 0\.2460Table[6](https://arxiv.org/html/2607.16207#A3.T6)reports the corresponding domain\-wise results\. The same high\-level ordering holds in most domains: Dream PRISM clearly improves over likelihood\-style baselines, while Dream SAMA is usually stronger in the current configuration\. This suggests that the single\-pass joint probing idea transfers beyond LLaDA, but the best localization strategy can depend on the dLLM family\.

Table 7:Dream selector\-free and hand\-designed controls averaged over six MIMIR domains\. Runtime is reported in seconds per sample\. To reduce duplication with Table[5](https://arxiv.org/html/2607.16207#A3.T5), Dream PRISM and Dream SAMA are omitted here and discussed in the text\.MethodROC\-AUCRuntimeRandom Joint0\.61270\.1721Entropy Top0\.64550\.1358PRISM\-Free \(Ref\-LowProb\)0\.59680\.1249RareToken \(C4\-freq\)0\.71730\.1459We further evaluate selector\-free and hand\-designed Dream controls to separate attack quality from efficiency\. As shown in Table[7](https://arxiv.org/html/2607.16207#A3.T7), these controls remain clearly weaker than both Dream PRISM and Dream SAMA, but they clarify which parts of the Dream signal can already be captured by simple heuristics\. Among the hand\-designed controls, external\-corpus RareToken \(C4\-freq\) is the strongest, reaching mean ROC\-AUC 0\.7173, compared with 0\.6455 for Entropy Top, 0\.6127 for Random Joint, and 0\.5968 for PRISM\-Free \(Ref\-LowProb\)\. This suggests that token rarity contributes non\-trivially to the Dream regime, but it is still not sufficient to match Dream PRISM \(0\.8237\) or Dream SAMA \(0\.8513\)\.

These controls also remain much cheaper than Dream SAMA\. Average runtime is 0\.1721 s/sample for Random Joint, 0\.1358 for Entropy Top, 0\.1249 for PRISM\-Free \(Ref\-LowProb\), and 0\.1459 for RareToken \(C4\-freq\), compared with 0\.1380 for Dream PRISM and 1\.1699 for Dream SAMA\. Thus, in Dream, the main practical tradeoff is between the stronger but slower Dream SAMA attack and the cheaper Dream PRISM attack, while RareToken \(C4\-freq\) serves as the strongest simple selector\-free baseline\.

Overall, the Dream results should be read as a robustness check\. They support the broader claim that joint masked probing is a useful dLLM\-specific MIA primitive beyond LLaDA, while also showing that the relative advantage of PRISM\-based localization can depend on the dLLM family\. In particular, Dream appears more sensitive to simple rarity\-based selection than LLaDA, although the fair external\-corpus RareToken control still remains below both Dream PRISM and Dream SAMA\. For this reason, we keep LLaDA as the primary model for the main claim and report Dream as appendix\-only supporting evidence\.

## Appendix DWhy Reference Calibration Is Necessary

A hard\-position selector identifies where the target model is likely to reveal membership information, but selection alone does not remove the confounding effect of intrinsic token difficulty\. Tokens selected byJUMPare intentionally low\-confidence under the reference view, so their absolute target scores can remain difficult to interpret: a low target score may simply indicate a generally hard token, while a high target score may reflect generic predictability rather than memorization\. For this reason,JUMPuses a reference\-calibrated readout rather than a target\-only score\.

To isolate the role of reference calibration, we compare two readouts on the same selected positions\. The target\-only readout uses the target model’s true\-token log\-probability at the selected positions, while the calibrated readout uses the target–reference gap from[Eq\.˜3](https://arxiv.org/html/2607.16207#S3.E3)\. In both cases, we keep the selected mask set fixed and average the resulting token\-level statistics over the six MIMIR domains\.

![Refer to caption](https://arxiv.org/html/2607.16207v1/x12.png)

![Refer to caption](https://arxiv.org/html/2607.16207v1/x13.png)

Figure 6:Why reference calibration is needed\. Left: target\-only true\-token log\-probabilities on the selected low\-confidence positions\. Right: target–reference reconstruction gaps on the same positions\. Absolute target scores remain partly confounded by intrinsic token difficulty, whereas the calibrated gap subtracts generic predictability and yields a clearer member/non\-member separation\.[Figure˜6](https://arxiv.org/html/2607.16207#A4.F6)shows that the target\-only readout provides only a modest separation between members and non\-members\. This is expected because absolute target confidence mixes two effects: whether the example was seen during fine\-tuning, and how predictable the token is from context in general\. The reference model estimates the second effect\. After subtracting the reference score, member examples retain a positive reconstruction advantage on the selected positions, while non\-member examples stay closer to zero\. Thus, hard\-position selection and reference calibration play complementary roles: the selector determines which positions to probe, and the target–reference gap turns those probes into a difficulty\-calibrated membership statistic\.

## Appendix EAdditional Selector and Scoring Ablations

### E\.1Selector\-Controlled Baselines

We include selector\-controlled baselines to separate the contribution of joint multimask probing, hard\-position selection, reference calibration, and the learned PRISM selector\. All methods in[Table˜8](https://arxiv.org/html/2607.16207#A5.T8)use the same six MIMIR domains and the same selected\-token budget\. They differ only in how positions are selected or whether the final score uses a reference model\.

Table 8:Selector\-controlled LLaDA baselines averaged over six MIMIR domains\.MethodROC\-AUCTPR@10%TPR@1%TPR@0\.1%Random Joint0\.81530\.52840\.21680\.0776Entropy Top0\.87880\.68570\.35320\.1112PRISM\-Free \(Ref\-LowProb\)0\.86170\.63740\.30450\.1167Target\-Only0\.55510\.14060\.02340\.0064JUMP0\.90410\.75180\.46360\.2337#### Random joint masking\.

Random Joint masksKKvalid token positions uniformly at random and then applies the same target–reference clipped aggregation used byJUMP\. Its non\-trivial performance shows that joint multimask probing alone is a meaningful DLM attack primitive, but it is clearly weaker than hard\-position selection\.

#### Entropy and reference\-low\-probability selection\.

Entropy Top selects uncertain positions according to the selector distribution, while PRISM\-Free selects positions with low clean\-text true\-token probability under the reference model\. Both are strong controls, confirming that membership signal is concentrated on difficult positions\. Their gap toJUMP, especially at low FPR, suggests that the learned hard\-position selector provides a better localization prior than these simpler heuristics\.

#### Target\-only control\.

Target\-Only uses PRISM\-selected positions but removes the reference\-model discrepancy from the final score\. Its weak performance indicates that the selector is not directly acting as a membership classifier\. The main gain comes from combining hard\-position localization with target–reference reconstruction advantage\.

### E\.2Selector Diagnostics

We further diagnose whetherJUMPis simply reproducing token rarity or clean reference\-model confidence\. To avoid using statistics from the evaluation split, we define the RareToken control using an external unigram frequency table computed from C4\. RareToken \(C4\-freq\) selects theKKvalid positions with the lowest C4 token frequencies, and then applies the same joint masking and clipped target–reference scoring rule asJUMP\. We also compare against Ref\-LowProb, which selects theKKpositions with the lowest clean reference log\-probability\. All diagnostics use the same selected\-token budget,K=64K=64, and the same six MIMIR domains as the main LLaDA experiments\.

Table 9:Selector diagnostic summary averaged over six MIMIR domains\. Non\-member P99 and P99\.9 denote upper\-tail score quantiles of the non\-member distribution\. RareToken uses an external C4 unigram frequency table\.MethodTPR@10%TPR@1%TPR@0\.1%NM P99NM P99\.9JUMP0\.75180\.46360\.23370\.10320\.1402RareToken \(C4\-freq\)0\.52620\.18990\.02830\.25850\.3652PRISM\-Free \(Ref\-LowProb\)0\.63740\.30450\.11670\.11280\.1523[Table˜9](https://arxiv.org/html/2607.16207#A5.T9)shows that neither token rarity nor clean reference confidence is sufficient to explain the gains ofJUMP\. RareToken \(C4\-freq\) is weaker thanJUMPat every operating point and is especially poor in the strict low\-FPR regime\. Ref\-LowProb is a stronger hand\-designed baseline but still remains clearly belowJUMP, especially at strict low\-FPR thresholds\. This gap is consistent with the non\-member tail statistics: RareToken has much larger average NM P99/P99\.9 scores thanJUMP, while Ref\-LowProb yields a cleaner tail but weaker overall attack quality\. Thus, external\-corpus rarity yields a noisy score with many high\-scoring non\-member outliers, whereas clean reference confidence alone is not the right localization rule either\.

![Refer to caption](https://arxiv.org/html/2607.16207v1/x14.png)![Refer to caption](https://arxiv.org/html/2607.16207v1/x15.png)

Figure 7:Overlap betweenJUMP\-selected positions and alternative selector rules across domains\. The legend is shared across domains\. Entropy Top has the largest overlap withJUMP, but substantial disagreement remains\. RareToken \(C4\-freq\) and Random Joint have much smaller overlap, showing thatJUMPis not simply selecting rare tokens or random positions\.[Figure˜7](https://arxiv.org/html/2607.16207#A5.F7)further shows thatJUMPis not equivalent to simple selector heuristics\. Averaged over domains, Entropy Top overlaps withJUMPon 62\.0% of selected positions, Ref\-LowProb on 43\.6%, RareToken \(C4\-freq\) on 29\.8%, and Random Joint on 21\.3%\. Thus,JUMPcaptures part of the clean\-uncertainty signal reflected by entropy, but is much less aligned with simple rarity or clean reference probability\. This supports the interpretation that the learned selector captures a broader token\-level uncertainty signal than any single hand\-designed heuristic\.

![Refer to caption](https://arxiv.org/html/2607.16207v1/x16.png)

![Refer to caption](https://arxiv.org/html/2607.16207v1/x17.png)

Figure 8:Score distributions forJUMPand RareToken across domains, where RareToken uses external C4 token frequencies\. RareToken produces a substantially heavier non\-member upper tail, consistent with its weak low\-FPR performance relative toJUMP\.[Figure˜8](https://arxiv.org/html/2607.16207#A5.F8)visualizes the same effect at the score\-distribution level\. With external C4 frequencies, RareToken no longer behaves as a strong broad\-ranking surrogate\. In several domains, its non\-member scores extend much farther into the upper tail than those ofJUMP, directly weakening TPR@1%FPR and TPR@0\.1%FPR\.JUMPis therefore better understood as the combination of learned low\-confidence localization, target–reference discrepancy scoring, and clipped aggregation, rather than as a rarity\-based heuristic\.

### E\.3Selector Training Sample\-Size Ablation

We further ablate the amount of selector\-training data used for the PRISM head\. Using the same LLaDA backbone and matched selector\-training hyperparameters, we train PRISM heads on C4 subsets of size20​k20\\mathrm\{k\},50​k50\\mathrm\{k\},100​k100\\mathrm\{k\},200​k200\\mathrm\{k\},400​k400\\mathrm\{k\}, and500​k500\\mathrm\{k\}\. All selectors are evaluated with the same downstream JUMP pipeline:K=64K=64selected positions, a single joint multimask query, the target–reference true\-token log\-probability gap, and clipped aggregation with threshold\[−log⁡1\.5,log⁡1\.5\]\[\-\\log 1\.5,\\log 1\.5\]\. The evaluation averages over the same six MIMIR domains used in the main LLaDA experiments\.

![Refer to caption](https://arxiv.org/html/2607.16207v1/x18.png)Figure 9:Effect of selector\-training set size for the C4\-trained PRISM head\. We train LLaDA PRISM heads on matched C4 subsets from20​k20\\mathrm\{k\}to500​k500\\mathrm\{k\}samples and evaluate each selector with the same JUMP pipeline across six MIMIR domains\. Performance improves rapidly at small scales and largely saturates by200​k200\\mathrm\{k\}\. Although400​k400\\mathrm\{k\}gives a slightly higher mean ROC\-AUC,200​k200\\mathrm\{k\}gives the best average low\-FPR performance while requiring much less training time\.Table 10:C4 selector\-training sample\-size ablation for the LLaDA PRISM head\. All rows use the same JUMP scoring rule and report averages over six MIMIR domains\.C4 samplesMean ROC\-AUCMean TPR@1%Mean TPR@0\.1%Train time20​k20\\mathrm\{k\}0\.89330\.39160\.1679190s50​k50\\mathrm\{k\}0\.89650\.37890\.1931421s100​k100\\mathrm\{k\}0\.89140\.35840\.1895813s200​k200\\mathrm\{k\}0\.89870\.44530\.21851,613s400​k400\\mathrm\{k\}0\.90120\.42460\.21533,158s500​k500\\mathrm\{k\}0\.89740\.42700\.20533,922s[Table˜10](https://arxiv.org/html/2607.16207#A5.T10)and[Figure˜9](https://arxiv.org/html/2607.16207#A5.F9)show that selector performance largely saturates by200​k200\\mathrm\{k\}training samples\. Moving from200​k200\\mathrm\{k\}to400​k400\\mathrm\{k\}gives only a small increase in mean ROC\-AUC, from 0\.8987 to 0\.9012, but does not improve the low\-FPR metrics that are most important for high\-confidence membership inference\. The200​k200\\mathrm\{k\}selector achieves the best average TPR@1%FPR and TPR@0\.1%FPR among the tested budgets\. The matched500​k500\\mathrm\{k\}rerun also does not improve over200​k200\\mathrm\{k\}, despite requiring substantially more training time\.

The cost\-performance tradeoff therefore supports using200​k200\\mathrm\{k\}C4 samples as the main selector\-training budget\. Relative to200​k200\\mathrm\{k\}, the400​k400\\mathrm\{k\}and500​k500\\mathrm\{k\}selectors require approximately1\.96×1\.96\\timesand2\.43×2\.43\\timesmore training time, respectively, without improving low\-FPR behavior\. Thus,200​k200\\mathrm\{k\}is not necessarily uniformly optimal on every metric, but it is the most practical operating point: it is already near the ROC\-AUC plateau, gives the strongest average low\-FPR performance, and avoids the substantially larger selector\-training cost of larger C4 subsets\.

## Appendix FBudget and Clipping Details

### F\.1Selected\-Token Budget Sweep

[Figure˜10](https://arxiv.org/html/2607.16207#A6.F10)reports the selected\-token budget sweep for the low\-FPR metrics\. Across domains,K=64K=64provides the most stable tradeoff and is therefore used as the default setting\. Using onlyK=32K=32positions reduces the number of aggregated token\-level membership signals, while increasing toK=128K=128does not consistently improve low\-FPR detection and can introduce less informative positions\.

Importantly, increasingKKdoes not materially increase attack cost in our implementation\. All selected positions are scored within the same joint masked\-query pipeline, so the dominant cost remains the selector, target\-model, and reference\-model forward passes rather than the number of retained positions itself\.[Table˜11](https://arxiv.org/html/2607.16207#A6.T11)reports the measured runtime for the same sweep\. Averaged over the six LLaDA domains with clipping thresholdτ=log⁡1\.5\\tau=\\log 1\.5, the total wall\-clock time is 392\.1 s forK=32K=32, 385\.7 s forK=64K=64, and 365\.0 s forK=128K=128, corresponding to 0\.196, 0\.193, and 0\.183 seconds per sample, respectively\. Thus, the practical runtime remains essentially flat across theKKsweep, supporting the NFE\-based claim that increasingKKdoes not change the target/reference scoring cost\.

![Refer to caption](https://arxiv.org/html/2607.16207v1/x19.png)\(a\)TPR@10% FPR\.
![Refer to caption](https://arxiv.org/html/2607.16207v1/x20.png)\(b\)TPR@1% FPR\.
![Refer to caption](https://arxiv.org/html/2607.16207v1/x21.png)\(c\)TPR@0\.1% FPR\.
![Refer to caption](https://arxiv.org/html/2607.16207v1/x22.png)

Figure 10:Detailed selected\-token budget sweep across low\-FPR operating points\. The legend is shared across all panels\. The defaultK=64K=64gives a stable operating point across domains: smaller budgets can weaken the aggregate signal, while larger budgets do not consistently improve low\-FPR detection\.Table 11:Runtime of the selected\-token budget sweep, averaged over the six LLaDA domains with clipping thresholdτ=log⁡1\.5\\tau=\\log 1\.5\. IncreasingKKdoes not materially increase wall\-clock cost because all selected positions are scored in the same joint masked\-query pipeline\.Selected\-token budgetKKTotal runtime \(s\)Runtime / sample \(s\)32392\.10\.19664385\.70\.193128365\.00\.183
### F\.2Clipping Operation Ablation

[Table˜12](https://arxiv.org/html/2607.16207#A6.T12)reports the aggregate clipping ablation corresponding to[Section˜6\.3](https://arxiv.org/html/2607.16207#S6.SS3)\. The selector is fixed to the PRISM\-C4 head, and both readouts use the same selected token positions\. The unclipped variant averages raw target/reference token log\-probability gaps, while the clipped variant caps each token contribution before aggregation\.

Table 12:Clipping ablation averaged over six MIMIR domains\.MethodROC\-AUCTPR@10%TPR@1%TPR@0\.1%No clip mean0\.81660\.54780\.23230\.1003Clipped mean0\.89870\.73580\.44530\.2185Clipping improves every metric, with the largest relative gain at the strictest operating point\. In particular, TPR@0\.1%FPR more than doubles after clipping\. This behavior is consistent with the role of clipping as a robustness correction: selected hard positions can produce extreme target/reference gaps, and an unclipped mean can be dominated by a small number of unstable token\-level contributions\. Clipping limits these isolated spikes and instead rewards sequences where the target has a stable reconstruction advantage across multiple selected positions\.

### F\.3Clipping\-threshold sweep

We provide the full clipping\-threshold sweep for the low\-FPR metrics omitted from the main text\. The thresholdcccontrols the per\-token cap in[Eq\.˜6](https://arxiv.org/html/2607.16207#S4.E6); smaller values are more aggressive and suppress token\-level outliers more strongly, while larger values approach the unclipped mean\. Across the full domain–metric grid,c=log⁡1\.5c=\\log 1\.5provides the best aggregate operating point\. Some individual domain–metric pairs can prefer a nearby threshold, butc=log⁡1\.5c=\\log 1\.5gives the most stable tradeoff once TPR at low FPR is considered\. We therefore usec=log⁡1\.5c=\\log 1\.5as the default in all main experiments\.

![Refer to caption](https://arxiv.org/html/2607.16207v1/x23.png)\(a\)TPR@10% FPR\.
![Refer to caption](https://arxiv.org/html/2607.16207v1/x24.png)\(b\)TPR@1% FPR\.
![Refer to caption](https://arxiv.org/html/2607.16207v1/x25.png)\(c\)TPR@0\.1% FPR\.
![Refer to caption](https://arxiv.org/html/2607.16207v1/x26.png)

Figure 11:Full clipping\-threshold sweep for low\-FPR metrics\. The legend is shared across all panels\. The defaultc=log⁡1\.5c=\\log 1\.5gives the best aggregate operating point across domains and metrics, even though individual domain–metric pairs may prefer nearby thresholds\.

## Appendix GImplementation and Runtime Details

### G\.1Selector Training Cost

For the selector\-corpus ablation in[Section˜6\.1](https://arxiv.org/html/2607.16207#S6.SS1), all PRISM heads are trained with the same budget of 200k samples\. This matched budget isolates the effect of the selector training corpus rather than the amount of selector supervision\. The downstream attack pipeline is otherwise identical across PRISM\-C4, PRISM\-Slim, and PRISM\-FW: each selector choosesK=64K=64positions, after which the same joint multimask query and clipped target–reference scoring rule are applied\.

The selector\-free variant removes the learned PRISM head\. Instead, it ranks valid positions by the reference model’s clean\-text true\-token logit and selects theKKpositions with the lowest values\. Thus, the selector\-free variant uses a reference\-side difficulty heuristic without any auxiliary selector training\.

PRISM\-head training is a one\-time preprocessing cost\. In our implementation, PRISM\-C4 takes 26\.9 minutes of wall time, corresponding to approximately 1\.8 GPU\-hours on 4×\\timesL40S GPUs\. PRISM\-Slim takes 23\.9 minutes, or approximately 1\.6 GPU\-hours, and PRISM\-FW takes 26\.8 minutes, or approximately 1\.8 GPU\-hours\. When amortized over the evaluation suite of six datasets with 2,000 samples each, these costs correspond to 0\.134 seconds per sample for PRISM\-C4, 0\.120 seconds per sample for PRISM\-Slim, and 0\.134 seconds per sample for PRISM\-FW\.

### G\.2Per\-Sample Runtime Accounting

All reported runtimes are normalized as seconds per sample\. Although attacks are executed in batches for efficiency, per\-sample runtime is the most direct measure of attack cost from the auditor’s perspective\. This is particularly important for DLM MIAs because attacks that repeatedly query many masked variants can become expensive even when each individual forward pass is parallelized\.

Table 13:Runtime per sample in seconds\. “Head” denotes amortized one\-time selector training\.MethodInferenceHeadTotalPRISM\-C40\.1840\.1340\.318PRISM\-Slim0\.1850\.1200\.305PRISM\-FW0\.1830\.1340\.317PRISM\-Free0\.173–0\.173SAMA1\.863–1\.863Even after amortizing selector\-head training, the learned\-selector variants remain substantially faster than SAMA\. The computational advantage comes from the single\-pass design: once positions are selected, all selected tokens are reconstructed jointly rather than through many separate masked subsets\.

## Appendix HDefense via Differentially Private Fine\-Tuning

We evaluate differentially private fine\-tuning as a principled defense against JUMP\. Whereas output\-level redaction defenses depend on hiding a particular signal from the API, DP fine\-tuning targets the underlying cause of membership leakage: the target model’s tendency to over\-fit individual fine\-tuning examples\. Because JUMP relies on subtle target–reference reconstruction\-gap differences aggregated over a small set of selected positions, even modest suppression of per\-example memorization should translate into a large reduction in attack signal\. We test this hypothesis with DP\-LoRA at\(ε,δ\)=\(8\.0,10−5\)\(\\varepsilon,\\delta\)=\(8\.0,10^\{\-5\}\), a relatively weak privacy regime by formal standards but a standard configuration for DP fine\-tuning of large language models\[Yuet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib42), Liet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib43)\]\.

### H\.1Defense Setup

We fine\-tune each LLaDA\-8B\-Base target with DP\-SGD applied only to LoRA adapter parameters\. LoRA adapters are attached to the query and value projections \(q\_proj,v\_proj\) with rankr=16r=16,α=16\\alpha=16, and dropout0\.10\.1\. This results in8\.398\.39M trainable parameters out of8\.028\.02B total, corresponding to0\.10%0\.10\\%of the full model\. The frozen backbone keeps the base reconstruction interface used by JUMP, so attack\-side scoring remains directly comparable to the undefended setting\.

Training uses4×4\\timesNVIDIA L40S GPUs in bf16 with DeepSpeed ZeRO\-3, per\-device batch size11, gradient accumulation1212\(effective batch size4848\), learning rate5×10−55\\times 10^\{\-5\}, weight decay0\.10\.1,500500warmup steps, and44epochs at maximum sequence length512512\. For privacy, we set the target privacy budget to\(ε,δ\)=\(8\.0,10−5\)\(\\varepsilon,\\delta\)=\(8\.0,10^\{\-5\}\), use flat per\-sample gradient clipping with norm1\.01\.0, and the PRV accountant\. Under this configuration, Opacus selects a noise multiplier ofσ=0\.4211\\sigma=0\.4211\. The full DP\-LoRA fine\-tuning runtime is approximately5757minutes per domain on44GPUs \(3,418\.73\{,\}418\.7seconds averaged over the six domains\)\.

### H\.2Attack Evaluation Protocol

We evaluate the defended targets using the same JUMP configuration as in the main experiments: the matched C4\-200k PRISM selector,quality\_botselection, selected\-token budgetK=64K=64, and clipped mean target/reference true\-token log\-probability gap aggregation withτ=log⁡1\.5\\tau=\\log 1\.5\. Each evaluation uses1,0001\{,\}000member and1,0001\{,\}000non\-member examples per domain, with1010bootstrap resamples \(seed4242\) for stability\. The reference model and PRISM selector are unchanged from the main experiments; only the target model is replaced by its DP\-LoRA\-fine\-tuned counterpart\.

### H\.3Utility Metric

We measure utility as the held\-out masked reconstruction loss on the1,0001\{,\}000non\-member examples from each domain, and exponentiate it to obtain a*reconstruction perplexity*\. Because LLaDA is a masked diffusion language model, this quantity is the exponentiated masked reconstruction loss under the same diffusion\-style objective the model was trained on, not autoregressive perplexity\. Reporting the metric on non\-members ensures that the utility number reflects domain adaptation rather than memorization of the fine\-tuning set\.

### H\.4Results

#### JUMP suppression\.

Table[14](https://arxiv.org/html/2607.16207#A8.T14)reports JUMP ROC\-AUC and low\-FPR TPR for the undefended target and the DP\-LoRA\-defended target across all six MIMIR domains\. DP\-LoRA atε=8\\varepsilon=8reduces mean ROC\-AUC from0\.90410\.9041to0\.50200\.5020, bringing the attack close to chance \(0\.500\.50\) on every domain\. The largest defended AUC across domains is0\.52280\.5228\(GitHub\) and the smallest is0\.46730\.4673\(Pile\-CC\)\. Low\-FPR detection is also essentially eliminated: mean TPR at1%1\\%FPR drops from0\.46360\.4636in the undefended setting \(Table 1 of the main paper\) to0\.01310\.0131under DP\-LoRA, and TPR at0\.1%0\.1\\%FPR drops to0\.00250\.0025\. These low\-FPR values are at or near the FPR levels themselves, indicating no usable high\-confidence membership signal remains\.

Table 14:JUMP performance against the DP\-LoRA\-defended LLaDA target at\(ε,δ\)=\(8\.0,10−5\)\(\\varepsilon,\\delta\)=\(8\.0,10^\{\-5\}\)\. Undefended \(non\-DP\) AUC is included for reference\. Across all six domains, DP\-LoRA drives JUMP near chance and eliminates low\-FPR detection\.Domainnon\-DP AUCDP\-LoRA AUCTPR@10%TPR@1%TPR@0\.1%ArXiv0\.93800\.51330\.11780\.01360\.0022GitHub0\.85910\.52280\.12060\.01690\.0044HackerNews0\.76640\.51980\.12760\.01450\.0028Pile\-CC0\.96220\.46730\.07770\.00960\.0030PubMed Central0\.91830\.49560\.09520\.01070\.0003Wikipedia \(en\)0\.98030\.49340\.12000\.01300\.0026Average0\.90410\.50200\.10980\.01310\.0025
#### Utility cost\.

Table[15](https://arxiv.org/html/2607.16207#A8.T15)reports held\-out non\-member reconstruction loss and reconstruction perplexity for the same targets\. Averaged across the six domains, DP\-LoRA increases non\-member reconstruction perplexity by a factor of1\.1151\.115, corresponding to roughly\+11\.5%\+11\.5\\%\. Domain\-level cost varies: Pile\-CC, GitHub, and Wikipedia \(en\) show small overheads \(\+3\.9%\+3\.9\\%to\+6\.4%\+6\.4\\%\), while HackerNews and PubMed Central show larger overheads \(\+17\.5%\+17\.5\\%and\+19\.9%\+19\.9\\%\)\. In all cases, the defended perplexity remains in the same quality regime as the undefended fine\-tuned target rather than collapsing toward base\-model perplexity, indicating that domain adaptation is preserved with bounded degradation\.

Table 15:Utility cost of DP\-LoRA on held\-out non\-member data\. Loss is the masked reconstruction loss, and reconstruction perplexity is its exponential\. The rightmost column shows the multiplicative perplexity increase \(DP\-LoRA / non\-DP\)\.Domainnon\-DP lossDP\-LoRA lossnon\-DP pplDP\-LoRA pplppl ratioArXiv1\.55511\.69984\.73565\.47291\.1557GitHub1\.01971\.05772\.77252\.87961\.0386HackerNews1\.74541\.90685\.72806\.73161\.1752Pile\-CC1\.44911\.51154\.25934\.53361\.0644PubMed Central1\.43471\.61584\.19825\.03211\.1986Wikipedia \(en\)1\.19891\.25593\.31633\.51091\.0587Average1\.40051\.50794\.16834\.69351\.1152

### H\.5Discussion

The DP\-LoRA results support a simple interpretation of why JUMP is effective and how to defend against it\. JUMP localizes membership signal at low\-reference\-confidence positions and aggregates target/reference reconstruction gaps clipped atτ=log⁡1\.5≈0\.405\\tau=\\log 1\.5\\approx 0\.405\. The per\-token signal is small in absolute terms \(Section[Section˜4\.1](https://arxiv.org/html/2607.16207#S4.SS1)reports a member/non\-member gap difference of\+0\.174\+0\.174in nats under one\-hole probing\), and the attack’s strength comes from consistently combining many such small contributions\. DP\-LoRA suppresses the per\-example memorization that produces this consistency: by adding calibrated noise during fine\-tuning, it bounds the influence of any single training example on the resulting model\. Because JUMP depends on aggregating fine\-grained per\-example reconstruction advantages, even relatively weak DP \(ε=8\\varepsilon=8\) is sufficient to bring the attack to chance\.

In contrast to output\-redaction approaches that act only at the score interface, DP\-LoRA provides a formal\(ε,δ\)\(\\varepsilon,\\delta\)\-DP guarantee on the fine\-tuning procedure\. This guarantee bounds the success of*any*membership inference attack that operates on the fine\-tuned model, including adaptive variants of JUMP, distillation\-based attacks, and future attacks that exploit other aspects of the dLLM interface\. The price is a moderate\+11\.5%\+11\.5\\%increase in held\-out reconstruction perplexity, consistent with reported overheads in DP fine\-tuning of similarly\-sized language models\[Yuet al\.,[2022](https://arxiv.org/html/2607.16207#bib.bib42)\]\. We view this trade\-off as favorable: a single DP\-LoRA fine\-tuning run, taking under one hour on four L40S GPUs per domain, neutralizes JUMP across all six MIMIR domains\.

## Appendix IBroader Impacts

This work has both positive and negative societal implications\. On the positive side,JUMPprovides a stronger and more efficient privacy\-auditing tool for identifying membership leakage in fine\-tuned diffusion language models, helping model developers evaluate and mitigate privacy risks before deployment\. On the negative side, stronger membership inference attacks are dual\-use: if misapplied, they could be used to test whether sensitive text was included in a private fine\-tuning corpus\. To reduce this risk, we frameJUMPas an auditing tool, evaluate it on benchmark datasets, and provide a DP\-LoRA fine\-tuning defense as a concrete mitigation strategy with formal\(ε,δ\)\(\\varepsilon,\\delta\)\-DP guarantees\.

## Appendix JExisting assets and licenses

We use existing publicly released models, datasets, and baseline resources\. Table[16](https://arxiv.org/html/2607.16207#A10.T16)summarizes the assets used in this work, their roles, and the corresponding license or terms\-of\-use information\. We do not redistribute the original datasets or pretrained model weights as part of this submission; users should obtain each asset from its original source and comply with the corresponding license and terms of use\.

Table 16:Existing assets used in this paper\.AssetRole in this paperCitation / sourceLicense / termsLLaDA\-8B\-BaseMain base/reference dLLM and fine\-tuning targetNieet al\.\[[2025](https://arxiv.org/html/2607.16207#bib.bib26)\]MITDream\-v0\-Base\-7BSecondary dLLM evaluation in appendixYeet al\.\[[2025](https://arxiv.org/html/2607.16207#bib.bib27)\]Apache\-2\.0MIMIRMember/non\-member evaluation benchmarkDuanet al\.\[[2024](https://arxiv.org/html/2607.16207#bib.bib15)\]MIT; gated access terms on Hugging FaceC4PRISM selector training corpusRaffelet al\.\[[2020](https://arxiv.org/html/2607.16207#bib.bib38)\]ODC\-BY; Common Crawl Terms of UseSlimPajamaSelector\-corpus ablationSobolevaet al\.\[[2023](https://arxiv.org/html/2607.16207#bib.bib40)\]Apache\-2\.0 for the original Cerebras release; subset\-specific terms if using a split or reuploadFineWebSelector\-corpus ablationPenedoet al\.\[[2024](https://arxiv.org/html/2607.16207#bib.bib39)\]ODC\-By v1\.0; Common Crawl Terms of UseSAMAPrior dLLM MIA baselineChenet al\.\[[2026](https://arxiv.org/html/2607.16207#bib.bib19)\]MIT if using the official code repository