CoDR：面向扩散语言模型的免训练置信度漂移重遮蔽方法

# Training-Free Confidence-Drift Remasking for Diffusion Language Models
Source: [https://arxiv.org/html/2610.08833](https://arxiv.org/html/2610.08833)
Yue WuAffiliation:The Hong Kong Polytechnic University, Hong Kong, ChinaEmail:[yue0301\.wu@connect\.polyu\.hk](mailto:[email protected])Affiliation:Equal contribution\.\*Corresponding authors\.Qinghe ZhengAffiliation:Southern University of Science and Technology, Shenzhen, ChinaEmail:[12532603@mail\.sustech\.edu\.cn](mailto:[email protected])Affiliation:Equal contribution\.\*Corresponding authors\.Yu ZhangAffiliation:Southern University of Science and Technology, Shenzhen, ChinaEmail:[yu\.zhang\.ust@gmail\.com](mailto:[email protected])Jian HuangAffiliation:Affiliation:The Hong Kong Polytechnic University, Hong Kong, ChinaEmail:[jianhuang@polyu\.edu\.hk](mailto:[email protected])

###### Abstract

Masked diffusion language models \(MDLMs\) decode by repeatedly committing tokens to masked positions, but these commitments are usually irreversible\. A token chosen under sparse, partial context is kept fixed, even when later context no longer supports it\. Existing samplers mainly decide when to commit a token, but rarely check whether an already committed token should still be kept, allowing early mistakes to propagate\. We trace this issue to confidence drift, where the model’s confidence in a committed token drops from its sparse commit\-time context to the denser context available later\. Based on this signal, we proposeCoDR\(Confidence\-DriftRemasking\), a training\-free and sampler\-agnostic refinement pass\. CoDR estimates drift for all committed positions in onlykkforward passes viakk\-partition probing, then remasks and regenerates only the tokens the model no longer endorses\. Across two backbones, four reasoning and coding tasks, and three base samplers, CoDR improves average accuracy across all evaluated model–sampler configurations and improves most individual task settings with modest overhead\. Controlled experiments show that the gains come from targeted confidence\-drift remasking rather than extra compute alone, and that CoDR uses far fewer forward passes than prior remasking methods\. Code is available at[https://github\.com/YueWu0301/CoDR](https://github.com/YueWu0301/CoDR)\.

## 1Introduction

Masked diffusion language models \(MDLMs\)[Austin et al\. \(2021a\)](https://arxiv.org/html/2610.08833#bib.bib9)provide a parallel alternative to autoregressive generation\([Sahoo et al\., 2024](https://arxiv.org/html/2610.08833#bib.bib1);[Nie et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib4);[Ye et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib5)\)\. Starting from a fully masked sequence, they iteratively commit tokens until generation is complete, enabling flexible any\-order decoding\([Ou et al\., 2024](https://arxiv.org/html/2610.08833#bib.bib11);[Kim et al\., 2025a](https://arxiv.org/html/2610.08833#bib.bib24)\)and potentially lower inference latency\. With model parameters fixed at inference time, recent work has therefore focused on samplers that improve commitment quality and efficiency\([Wu et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib6);[Ben\-Hamu et al\., 2026](https://arxiv.org/html/2610.08833#bib.bib7)\)\. However, these methods rarely revisit tokens after they are committed\.

![Refer to caption](https://arxiv.org/html/2610.08833v1/figs/figure1.png)Figure 1:Standard generation fixes tokens once committed, leaving early mistakes unchanged\. Drift\-guided refinement detects unstable committed tokens, remasks them, and refills them under the updated context\.This creates irreversible commitment: a token selected under incomplete context remains fixed even if later context no longer supports it\. A locally plausible token at commitment time may become incompatible after surrounding positions are filled\. The problem is amplified when several positions are committed in parallel, because co\-committed tokens do not condition on one another and may introduce local inconsistencies\([Gu et al\., 2018](https://arxiv.org/html/2610.08833#bib.bib14);[Wu et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib6);[Wu and Huang, 2026](https://arxiv.org/html/2610.08833#bib.bib26)\)\. These early commitments then condition later predictions, allowing errors to propagate through subsequent decoding\. A natural remedy is to revisit unstable tokens once richer context becomes available rather than treating every commitment as final\.

Existing samplers use confidence, entropy, or attention signals to decide which masked positions to commit\([Wu et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib6);[Ben\-Hamu et al\., 2026](https://arxiv.org/html/2610.08833#bib.bib7);[Zhou et al\., 2026](https://arxiv.org/html/2610.08833#bib.bib22)\), making the initial commitment more cautious but usually keeping committed tokens fixed\. Recent remasking methods relax this constraint: ReMDM iteratively revisits commitments\([Wang et al\., 2026a](https://arxiv.org/html/2610.08833#bib.bib8)\), while CORE probes robustness to perturbed contexts\([Zhai et al\., 2026](https://arxiv.org/html/2610.08833#bib.bib13)\)\. These approaches establish the value of reversible generation, but do not directly measure how support for a token changes as decoding progresses\. We instead ask whether the model’s confidence in the same token decreases from its early commit\-time context to a later, denser context\. Such confidence drift directly identifies commitments that lose model support during decoding\.

We proposeCoDR\(Confidence\-DriftRemasking\), a training\-free refinement pass \(Figure[1](https://arxiv.org/html/2610.08833#S1.F1)\)\. CoDR compares each token’s commit\-time confidence with its confidence under later context and treats a confidence drop as evidence that the updated context provides less support\. To avoid one forward pass per position,kk\-partition probing estimates drift for all commitments in onlykkpasses\. CoDR then remasks the selected positions and invokes the original sampler to refill them under denser bidirectional context, while leaving all other tokens fixed\. It therefore acts as a sampler\-agnostic plug\-in for decoders that expose commit\-time probabilities and support masked refill\. Across LLaDA\-8B\([Nie et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib4)\)and Dream\-7B\([Ye et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib5)\), multiple samplers, and reasoning and coding tasks, CoDR improves the accuracy–compute trade\-off in most settings\. Controlled comparisons attribute these gains to targeted confidence\-drift revision rather than random remasking or extra decoding steps alone\.

Our contributions are summarized as follows:

- •We identify confidence drift as a temporal symptom of decoding instability, where committed tokens lose model support as context grows\.
- •We propose CoDR, a training\-free plug\-in that efficiently detects and revises drifted commitments without modifying the base model or sampler\.
- •We show across models, tasks, and samplers that targeted revision improves the accuracy–compute trade\-off over unguided extra computation\.

![Refer to caption](https://arxiv.org/html/2610.08833v1/figs/block_wise_unmask_refinement.png)Figure 2:Overview of our method CoDR\. The base sampler first fills a block, then the refine pass re\-scores committed tokens under richer context, remasks positions whose confidence drifts downward, and refills only those positions\.
## 2Related Work

#### Masked Diffusion Language Models\.

Masked diffusion language modeling grew out of discrete diffusion\. D3PM\([Austin et al\., 2021a](https://arxiv.org/html/2610.08833#bib.bib9)\)introduced discrete\-state diffusion and showed that an absorbing “mask” state is particularly natural for language; subsequent work such as SEDD\([Lou et al\., 2024](https://arxiv.org/html/2610.08833#bib.bib10)\), MDLM\([Sahoo et al\., 2024](https://arxiv.org/html/2610.08833#bib.bib1)\), and RADD\([Ou et al\., 2024](https://arxiv.org/html/2610.08833#bib.bib11)\)developed increasingly effective objectives and parameterizations for discrete and masked diffusion language modeling\. Scaling these ideas produced billion\-parameter models competitive with autoregressive LLMs\. LLaDA\([Nie et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib4)\)demonstrated large\-scale pretraining from scratch, while DiffuLLaMA\([Gong et al\., 2025a](https://arxiv.org/html/2610.08833#bib.bib15)\), Dream\([Ye et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib5)\), DiffuCoder\([Gong et al\., 2025b](https://arxiv.org/html/2610.08833#bib.bib18)\), and LLaDA 2\.0\([Bie et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib21)\)explored adaptation or initialization from pretrained autoregressive models\. The paradigm has further expanded to block\-wise generation\([Arriola et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib12)\)and multimodal modeling\([You et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib16);[Yang et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib19)\)\. Recent work has also explored supervised, preference\-based, parameter\-efficient, and reinforcement\-learning\-based post\-training of diffusion language models\([Zhao et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib33);[Zhu et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib20);[Wang et al\., 2026b](https://arxiv.org/html/2610.08833#bib.bib32)\), whereas CoDR improves generation without updating model parameters\.

#### Decoding Samplers and Remasking\.

How a trained MDLM is decoded is a second, largely separate question\. Existing samplers use model\-internal signals such as entropy or margin\([Koh et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib23);[Kim et al\., 2025a](https://arxiv.org/html/2610.08833#bib.bib24)\), confidence thresholds\([Wu et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib6)\), entropy bounds\([Ben\-Hamu et al\., 2026](https://arxiv.org/html/2610.08833#bib.bib7)\), denoising history\([Wu and Huang, 2026](https://arxiv.org/html/2610.08833#bib.bib26);[Wang et al\., 2025](https://arxiv.org/html/2610.08833#bib.bib25)\), KL criteria\([Kim et al\., 2025b](https://arxiv.org/html/2610.08833#bib.bib17)\), or attention patterns\([Zhou et al\., 2026](https://arxiv.org/html/2610.08833#bib.bib22);[Kim et al\., 2026](https://arxiv.org/html/2610.08833#bib.bib27)\)to decide which positions to commit and when\. These methods improve the commitment rule, but committed tokens are typically kept fixed\. Recent remasking methods relax this constraint by allowing committed tokens to be revisited\. ReMDM\([Wang et al\., 2026a](https://arxiv.org/html/2610.08833#bib.bib8)\)introduces an iterative remasking process, while CORE\([Zhai et al\., 2026](https://arxiv.org/html/2610.08833#bib.bib13)\)selects revision targets by testing token robustness under perturbed contexts\. Our method also belongs to this reversible\-generation family, but it uses a different signal: confidence drift, the confidence drop of the same committed token from its commit\-time context to a later decoding context\. This directly targets tokens whose support decreases as more surrounding context becomes available\.

## 3Method

We introduceCoDR\(Confidence\-DriftRemasking\), a training\-free refinement method that detects drifted commitments by comparing commit\-time and later\-context confidence\. CoDR estimates this signal withkk\-partition probing, then selectively remasks and refills drifted tokens with the original base sampler \(Figure[2](https://arxiv.org/html/2610.08833#S1.F2)\)\.

### 3\.1Preliminaries: MDLM Decoding

Let𝒱\\mathcal\{V\}be a vocabulary and\[𝙼\]∉𝒱\\mathtt\{\[M\]\}\\notin\\mathcal\{V\}a special mask token\. We decode a target sequence of lengthLL, denoted asx=\(x1,…,xL\)x=\(x^\{1\},\\dots,x^\{L\}\)where eachxi∈𝒱∪\{\[𝙼\]\}x^\{i\}\\in\\mathcal\{V\}\\cup\\\{\\mathtt\{\[M\]\}\\\}, conditioned on a prompt \(omitted in notation for brevity\)\. The MDLM is parameterized by a denoising networkfθf\_\{\\theta\}that maps a partially masked sequence to a logit tensorfθ​\(x\)∈ℝL×\|𝒱\|f\_\{\\theta\}\(x\)\\in\\mathbb\{R\}^\{L\\times\|\\mathcal\{V\}\|\}in a single forward pass\. Its per\-position predictive distribution over the vocabulary is given by:

pθi\(⋅∣x\)=Softmax\(\[fθ\(x\)\]i\)∈Δ\(𝒱\)\.p\_\{\\theta\}^\{\\,i\}\(\\cdot\\mid x\)=\\mathrm\{Softmax\}\\bigl\(\[f\_\{\\theta\}\(x\)\]\_\{i\}\\bigr\)\\in\\Delta\(\\mathcal\{V\}\)\.\(1\)
Decoding proceeds over discrete stepst=1,…,Tt=1,\\dots,T\. LetCt⊆\{1,…,L\}C\_\{t\}\\subseteq\\\{1,\\dots,L\\\}denote the set of committed indices andMt=\{1,…,L\}∖CtM\_\{t\}=\\\{1,\\dots,L\\\}\\setminus C\_\{t\}the set of masked indices after steptt\. Initially,C0=∅C\_\{0\}=\\emptysetandM0=\{1,…,L\}M\_\{0\}=\\\{1,\\dots,L\\\}, with all positions inM0M\_\{0\}holding\[𝙼\]\\mathtt\{\[M\]\}\.

At steptt, a sampler selects a subset of masked positionsSt⊆Mt−1S\_\{t\}\\subseteq M\_\{t\-1\}to commit\. Each token is drawn from its per\-position marginal distribution, conditioned on the current partial sequencext−1x\_\{t\-1\}:

x^ti∼pθi\(⋅∣xt−1\),∀i∈St\.\\hat\{x\}\_\{t\}^\{i\}\\sim p\_\{\\theta\}^\{\\,i\}\(\\cdot\\mid x\_\{t\-1\}\),\\quad\\forall i\\in S\_\{t\}\.\(2\)
The sequence is then updated position\-wise as

xti=\{x^ti,i∈St\(unmask\),\[𝙼\],i∈Mt−1∖St\(stay masked\),xt−1i,i∉Mt−1\(already committed\)\.x\_\{t\}^\{i\}=\\begin\{cases\}\\hat\{x\}\_\{t\}^\{i\},&i\\in S\_\{t\}\\quad\\text\{\(unmask\)\},\\\\ \\mathtt\{\[M\]\},&i\\in M\_\{t\-1\}\\setminus S\_\{t\}\\quad\\text\{\(stay masked\)\},\\\\ x\_\{t\-1\}^\{i\},&i\\notin M\_\{t\-1\}\\quad\\text\{\(already committed\)\}\.\\end\{cases\}\(3\)
The sets are then updated asCt=Ct−1∪StC\_\{t\}=C\_\{t\-1\}\\cup S\_\{t\}andMt=Mt−1∖StM\_\{t\}=M\_\{t\-1\}\\setminus S\_\{t\}\. Committing a group in this manner treats the positions inStS\_\{t\}as conditionally independent givenxt−1x\_\{t\-1\}\.

Crucially, when positioniiis committed at steptt\(i\.e\.,i∈Sti\\in S\_\{t\}\), we record its commit\-time confidence:

ψiold=pθi​\(x^ti∣xt−1\)\.\\psi^\{\\mathrm\{old\}\}\_\{i\}=p\_\{\\theta\}^\{\\,i\}\\bigl\(\\hat\{x\}\_\{t\}^\{i\}\\mid x\_\{t\-1\}\\bigr\)\.\(4\)This scalar evaluates the marginal probability the network assigned to the chosen tokenx^ti\\hat\{x\}\_\{t\}^\{i\}under the sparse, localized contextxt−1x\_\{t\-1\}available at that exact step\. This single recorded scalar per position is the only interface our refinement procedure requires from the underlying base sampler\.

### 3\.2Confidence\-Drift Detection via k\-Partition Probing

Because each positioniiis committed at some stepttwhile many positions inMt−1M\_\{t\-1\}remain masked,xix^\{i\}is decided under a sparser context than it will eventually reside in\. Furthermore, within a co\-committed groupStS\_\{t\}, positions are sampled independently of one another\. The optimal token atiican therefore change once more context is filled in at a later timet′\>tt^\{\\prime\}\>t\. We define this discrepancy as*confidence drift*, and detect it by re\-scoring the model’s confidence in the initially chosen token under the denser, later context\.

Letx~\[i→\[𝙼\]\]\\tilde\{x\}^\{\[i\\to\\mathtt\{\[M\]\}\]\}denote the current sequence at timet′t^\{\\prime\}with positioniiartificially re\-masked\. The re\-scored confidence is the predictive probability of the same originally committed tokenxix^\{i\}under this richer context, and the drift is its confidence drop from the moment of commitment:

ψinew=pθi\(xi∣x~\[i→\[𝙼\]\]\),δi=ψiold−ψinew\.\\psi^\{\\mathrm\{new\}\}\_\{i\}=p\_\{\\theta\}^\{\\,i\}\\bigl\(x^\{i\}\\mid\\tilde\{x\}^\{\[i\\to\\mathtt\{\[M\]\}\]\}\\bigr\),\\qquad\\delta\_\{i\}=\\psi^\{\\mathrm\{old\}\}\_\{i\}\-\\psi^\{\\mathrm\{new\}\}\_\{i\}\.\(5\)Both confidences evaluate the probability of the same tokenxix^\{i\}via Equation[1](https://arxiv.org/html/2610.08833#S3.E1)\. The only difference is the conditioning context:ψiold\\psi^\{\\mathrm\{old\}\}\_\{i\}is computed under the sparse context available at commitment time, whereasψinew\\psi^\{\\mathrm\{new\}\}\_\{i\}is computed under the denser context available at the current step\. A large positiveδi\\delta\_\{i\}indicates thatxix^\{i\}was initially committed with high confidence, but is no longer strongly supported once its surrounding tokens become visible\. We treat this confidence drop as the primary signal of a drifted commitment\.

Because the logarithm is strictly increasing, the conditionδi\>0\\delta\_\{i\}\>0is equivalent tolog⁡ψiold−log⁡ψinew\>0\\log\\psi\_\{i\}^\{\\mathrm\{old\}\}\-\\log\\psi\_\{i\}^\{\\mathrm\{new\}\}\>0\. CoDR therefore revisits a commitment precisely when it loses model support under the later, denser context\. This does not imply that the token is necessarily incorrect or that confidence drift is a universally optimal revision rule; it is a temporal instability signal whose practical informativeness is supported by the trigger comparisons in Table[2](https://arxiv.org/html/2610.08833#S3.T2)and the revision diagnostics in Section[4\.4](https://arxiv.org/html/2610.08833#S4.SS4)\.

Evaluatingψinew\\psi^\{\\mathrm\{new\}\}\_\{i\}exactly requires one forward pass per committed position, resulting in\|C\|\|C\|passes per invocation\. This overhead is prohibitively expensive for generation\. Instead, we approximateψinew\\psi^\{\\mathrm\{new\}\}\_\{i\}for all committed positions simultaneously in justkkpasses\. We partition the committed setCCintokkresidue classes by stridekk, definingC\(r\)=\{i∈C:i≡r\(modk\)\}C^\{\(r\)\}=\\\{i\\in C:i\\equiv r\\\!\\\!\\pmod\{k\}\\\}forr=0,…,k−1r=0,\\dots,k\-1\. For each class, we mask the positions as a whole:

x~j,\(r\)=\{\[𝙼\]j∈C\(r\),xjotherwise\.\\tilde\{x\}^\{j,\(r\)\}=\\begin\{cases\}\\mathtt\{\[M\]\}&j\\in C^\{\(r\)\},\\\\ x^\{j\}&\\text\{otherwise\.\}\\end\{cases\}\(6\)A single forward pass onx~\(r\)\\tilde\{x\}^\{\(r\)\}yieldsψinew←pθi​\(xi∣x~\(r\)\)\\psi^\{\\mathrm\{new\}\}\_\{i\}\\leftarrow p\_\{\\theta\}^\{\\,i\}\(x^\{i\}\\mid\\tilde\{x\}^\{\(r\)\}\)for everyi∈C\(r\)i\\in C^\{\(r\)\}\. Thus,kkpasses assign aδi\\delta\_\{i\}to all committed positions\. This probe is exact whenk=\|C\|k=\|C\|\(where each class is a singleton\); for smallerkk, eachiiis re\-scored while its same\-class neighbors are also masked\. The stride\-kkspacing distributes masked positions evenly across the sequence, helping preserve regular local context around each probed position\. We usek=2k=2by default, separating even and odd indices\. We discuss the importance of confidence\-drift and k\-partition probing in Experiment[4\.3](https://arxiv.org/html/2610.08833#S4.SS3)and its ablation study in Experiment[4\.6](https://arxiv.org/html/2610.08833#S4.SS6)\.

Algorithm 1CoDR decoding with confidence\-drift remasking1:MDLM

fθf\_\{\\theta\}, sequence length

LL, block size

BB, drift threshold

τdrift\\tau\_\{\\mathrm\{drift\}\}, partition count

kk
2:

x←\[𝙼\]Lx\\leftarrow\\mathtt\{\[M\]\}^\{L\};

C←∅C\\leftarrow\\emptyset;

ℛ←∅\\mathcal\{R\}\\leftarrow\\emptyset
3:for

b=1b=1to

⌈L/B⌉\\lceil L/B\\rceildo

4:

ℬb←\{i:\(b−1\)​B<i≤min⁡\(b​B,L\)\}\\mathcal\{B\}\_\{b\}\\leftarrow\\\{i:\(b\{\-\}1\)B<i\\leq\\min\(bB,L\)\\\}
5:

\(x,C,ψold\)←Sampler⁡\(ℬb\)\(x,C,\\psi^\{\\mathrm\{old\}\}\)\\leftarrow\\mathrm\{Sampler\}\(\\mathcal\{B\}\_\{b\}\)⊳\\trianglerightdecode blockℬb\\mathcal\{B\}\_\{b\}

6:for

r=0r=0to

k−1k\-1do

7:

C\(r\)←\{i∈C:i≡r\(modk\)\}C^\{\(r\)\}\\leftarrow\\\{i\\in C:i\\equiv r\\pmod\{k\}\\\}
8:

x~←x\\tilde\{x\}\\leftarrow x;

x~i←\[𝙼\]​∀i∈C\(r\)\\tilde\{x\}^\{i\}\\leftarrow\\mathtt\{\[M\]\}\\ \\forall i\\in C^\{\(r\)\}
9:

ψinew←pθi​\(xi∣x~\)∀i∈C\(r\)\\psi^\{\\mathrm\{new\}\}\_\{i\}\\leftarrow p\_\{\\theta\}^\{\\,i\}\(x^\{i\}\\mid\\tilde\{x\}\)\\quad\\forall i\\in C^\{\(r\)\}
10:endfor

11:

ℛ←\{i∈C:ψiold−ψinew\>τdrift\}\\mathcal\{R\}\\leftarrow\\\{i\\in C:\\psi^\{\\mathrm\{old\}\}\_\{i\}\-\\psi^\{\\mathrm\{new\}\}\_\{i\}\>\\tau\_\{\\mathrm\{drift\}\}\\\}
12:if

ℛ≠∅\\mathcal\{R\}\\neq\\emptysetthen

13:

xi←\[𝙼\]∀i∈ℛx^\{i\}\\leftarrow\\mathtt\{\[M\]\}\\quad\\forall i\\in\\mathcal\{R\};

C←C∖ℛC\\leftarrow C\\setminus\\mathcal\{R\}
14:

\(x,C,ψold\)←Sampler⁡\(ℛ\)\(x,C,\\psi^\{\\mathrm\{old\}\}\)\\leftarrow\\mathrm\{Sampler\}\(\\mathcal\{R\}\)⊳\\trianglerightfill the re\-masked positions

15:endif

16:endfor

17:return

xx

Table 1:Comprehensive benchmark results over four tasks\.Acc\. \(%\)is accuracy, andNFEis the average number of forward evaluations\. Parenthesized annotations indicate absolute accuracy\-point changes brought by refinement\.Averagereports the unweighted mean Acc\. and NFE across the four tasks\.
### 3\.3Confidence\-Drift Remasking and Refill

Given the confidence\-drift scores\{δi\}i∈C\\\{\\delta\_\{i\}\\\}\_\{i\\in C\}, CoDR intervenes only on commitments whose support has decreased beyond a tolerance thresholdτdrift\\tau\_\{\\mathrm\{drift\}\}\. The remask set is defined as

ℛ=\{i∈C∣δi\>τdrift\}\.\\mathcal\{R\}=\\\{\\,i\\in C\\mid\\delta\_\{i\}\>\\tau\_\{\\mathrm\{drift\}\}\\,\\\}\.\(7\)
If no drifted position is detected, i\.e\.,ℛ=∅\\mathcal\{R\}=\\emptyset, CoDR keeps the current sequence unchanged\. Otherwise, it applies a remask–refill step\. Each position inℛ\\mathcal\{R\}is reset to\[𝙼\]\\mathtt\{\[M\]\}and removed from the committed set, after which the base sampler is invoked again to fill these re\-masked positions\. Since later tokens have already been generated, this refill step conditions on a denser bidirectional context than the original commitment\. Once the positions are regenerated, the base sampler also refreshes their commit\-time confidencesψiold\\psi^\{\\mathrm\{old\}\}\_\{i\}, so that future refinement steps compare against the latest commitment\.

Algorithm[1](https://arxiv.org/html/2610.08833#alg1)instantiates this procedure as a post\-block method with refinement after every decoded block\. At each refinement point, CoDR probes all currently committed positions, including those from previous blocks, and selectively refills the positions inℛ\\mathcal\{R\}whose confidence has drifted\. The refinement frequency can be reduced without changing the probe–remask–refill logic; we compare every\-block, periodic, final\-only, and mid\-block schedules in Table[4](https://arxiv.org/html/2610.08833#S4.T4)\.

Table 2:Trigger ablations usingfd\_refineon LLaDA\-8B \(00\-shot\) for HumanEval and GSM8K\. Thebottom\-mmbaselines remask themmcommitted tokens with the lowest current confidenceψnew\\psi^\{\\mathrm\{new\}\}, without comparing against their commit\-time confidenceψold\\psi^\{\\mathrm\{old\}\}\.HumanEvalGSM8KTriggerAcc\. \(%\)NFERem\.Edit \(%\)Acc\. \(%\)NFERem\.Edit \(%\)fdbaseline42\.6842\.6879\\phantom\{0\}790\.00\.0—76\.0076\.0074\\phantom\{0\}740\.00\.0—randomp=0\.045p=0\.04542\.6842\.68\(\+\+0\.00\)10410450\.950\.90\.210\.2176\.0076\.00\(\+\+0\.00\)99\\phantom\{0\}9951\.351\.30\.280\.28randomp=0\.20p=0\.2042\.6842\.68\(\+\+0\.00\)108108227\.6227\.60\.270\.2775\.8075\.80\(↓\\downarrow0\.20\)103103229\.7229\.70\.270\.27bottomm=6m=642\.6842\.68\(\+\+0\.00\)120\.5120\.548\.048\.04\.8376\.8076\.80\(↑\\uparrow0\.80\)120\.2120\.248\.048\.05\.43bottomm=8m=843\.2943\.29\(↑\\uparrow0\.61\)123\.8123\.864\.064\.03\.8877\.0077\.00\(↑\\uparrow1\.00\)123\.5123\.564\.064\.04\.31top1\_change43\.2943\.29\(↑\\uparrow0\.61\)99\\phantom\{0\}993\.93\.936\.0236\.0276\.8076\.80\(↑\\uparrow0\.80\)94\\phantom\{0\}943\.83\.832\.0532\.05drift \(conf\_drift\)43\.90\\bm\{43\.90\}\(↑\\uparrow1\.22\)11311351\.651\.63\.723\.7277\.40\\bm\{77\.40\}\(↑\\uparrow1\.40\)11011057\.257\.23\.203\.20Table 3:Structured stride vs\. random partition usingfd\_refineon LLaDA\-8B \(00\-shot\) across probe strideskk\. Random partition results are reported as mean±\\pmstd over 5 seeds\.Figure 3:Confidence\-drift diagnostics\. Panels \(a\)–\(b\) show median drift at each refinement boundary for tokens that are not remasked \(NR\), remasked but unchanged \(RU\), or remasked and edited \(RE\)\. Panel \(c\) reports the edit rate among positive\-drift events grouped into equal\-frequency drift quintiles\.

## 4Experiments

Our experiments are designed to answer five research questions:RQ1Does CoDR improve performance across models, tasks, and base samplers?RQ2Do the gains come from CoDR’s core mechanisms, namely drift\-guided triggering and structured partition probing?RQ3What does confidence drift capture during decoding, and how is its magnitude related to token revision and successful correction?RQ4How effectively and practically does CoDR use additional inference compute?RQ5How robust is CoDR across hyperparameters and generation lengths?

### 4\.1Experimental Setup

We evaluate two open\-weight MDLMs, LLaDA\-8B\-Instruct and Dream\-v0\-Instruct\-7B, on four zero\-shot benchmarks: HumanEval[Chen et al\. \(2021\)](https://arxiv.org/html/2610.08833#bib.bib30), MBPP[Austin et al\. \(2021b\)](https://arxiv.org/html/2610.08833#bib.bib31), MATH\-500[Hendrycks et al\. \(2021\)](https://arxiv.org/html/2610.08833#bib.bib29), and GSM8K[Cobbe et al\. \(2021\)](https://arxiv.org/html/2610.08833#bib.bib28), use Pass@1 metric\. For ablation study, we select a subset GSM8K with 500 samples\. All runs use bf16 precision, generation lengthLgen=256L\_\{\\mathrm\{gen\}\}=256, block length3232, and 0\-shot prompting\. Unless otherwise stated, the main accuracy experiments do not use a KV cache; Table[5](https://arxiv.org/html/2610.08833#S4.T5)separately evaluates latency and memory with and without KV caching for the initial Fast\-dLLM draft\. Decoding uses argmax at temperature00and is deterministic; reruns with different seeds produce identical predictions\. All experiments are performed on NVIDIA A100 GPUs\.

We evaluate Fast\-dLLM \(fd,τ=0\.9\\tau=0\.9\), EB\-Sampler \(eb,γ=0\.2\\gamma=0\.2\), andtop1as standalone generation baselines\. Unless noted, “\+CoDR” uses Algorithm[1](https://arxiv.org/html/2610.08833#alg1)withk=2k=2,τdrift=0\\tau\_\{\\mathrm\{drift\}\}=0, parity split\. In the refill stage,fd\+CoDRuses Fast\-dLLM withτ=0\.9\\tau=0\.9,eb\+CoDRuses EB\-Sampler withγ=0\.1\\gamma=0\.1, andtop1\+CoDRuses EB\-Sampler withγ=0\.01\\gamma=0\.01\(a detailed analysis in Appendix[C\.3](https://arxiv.org/html/2610.08833#A3.SS3)\)\. We additionally compare with ReMDM, following the setting used with LLaDA in the original paper, each block run3232ReMDM\-loop steps, and CORE \(detailed setting for CORE is shown in Appendix[C\.4](https://arxiv.org/html/2610.08833#A3.SS4)\)\. NFE \(number of function evaluations\) counts all decoding, probe, remasking, and refill forward passes\.

### 4\.2Main Results \(RQ1\)

Table[1](https://arxiv.org/html/2610.08833#S3.T1)shows that CoDR improves commitment\-based decoding across both models and all three base samplers\. On LLaDA\-8B, adding CoDR improves the average accuracy oftop1,fd, andebby\+0\.85\+0\.85,\+1\.06\+1\.06, and\+1\.44\+1\.44percentage points, respectively\. On Dream\-7B, the corresponding gains are\+0\.82\+0\.82,\+1\.11\+1\.11, and\+0\.79\+0\.79percentage points\. These consistent improvements indicate that CoDR is not specific to a particular decoding rule, but can be applied as a general refinement layer for different commitment\-based samplers\.

CoDR further improves the accuracy–compute trade\-off\. On LLaDA\-8B,fd\+CoDRachieves the best average accuracy of48\.93%48\.93\\%with123123NFE, outperforming ReMDM’s48\.44%48\.44\\%average accuracy while using far fewer forward evaluations than its512512NFE setting\. On Dream\-7B,top1\+CoDRobtains the best average accuracy of60\.62%60\.62\\%with324324NFE, andfd\+CoDRreaches60\.20%60\.20\\%with only135135NFE\. This suggests that CoDR uses additional computation in a targeted way by revisiting potentially unstable commitments, rather than uniformly increasing denoising iterations\.

Because decoding is deterministic, uncertainty arises from the finite test sets rather than sampling variation\. A10,00010\{,\}000\-resample paired bootstrap over LLaDA predictions finds significant four\-task average gains fortop1, Fast\-dLLM, and EB\-Sampler; all95%95\\%confidence intervals exclude zero\. CoDR also improves both samplers on two additional benchmarks, BBH[Suzgun et al\. \(2023\)](https://arxiv.org/html/2610.08833#bib.bib3)and GPQA[Rein et al\. \(2024\)](https://arxiv.org/html/2610.08833#bib.bib2)\. Full tests and results are reported in Appendix[B](https://arxiv.org/html/2610.08833#A2)\.

### 4\.3Effectiveness of Core Mechanisms \(RQ2\)

We examine whether CoDR’s gains come from \(i\) confidence drift and \(ii\) structured dense\-context probing\.

#### Baselines\.

For triggering, we compare random remasking,bottom\-mmselection using only the lowest current confidenceψnew\\psi^\{\\mathrm\{new\}\},top1\_change, and our temporalconf\_driftsignal\. The bottom\-mmsettings usem=6m=6,m=8m=8and the random remasking usep=0\.045p=0\.045,p=0\.20p=0\.20to approximately match CoDR’s remasking budget\. For probing, we compare deterministic stride and random partitions at the samekk\. We report accuracy, NFE, mean remasked positions \(Rem\.\), and the percentage of remasked tokens that change after refill \(Edit\)\.

#### \(i\) Confidence Drift\.

Table[2](https://arxiv.org/html/2610.08833#S3.T2)shows that random remasking adds compute without improving accuracy, whilebottom\-mmselection remains less accurate and more expensive than CoDR at comparable remasking budgets\. Thetop1\_changetrigger is precise but overly conservative, producing very few remasks\. In contrast,conf\_driftachieves the best accuracy on both datasets, showing that temporal comparison provides information beyond current confidence alone\.

#### \(ii\) Structured dense\-context probing\.

Table[3](https://arxiv.org/html/2610.08833#S3.T3)shows that structured stride probing preserves more regular local context than random partitions\. Atk=2k\{=\}2, random partitioning increases remasking from51\.651\.6to83\.483\.4on HumanEval and from57\.257\.2to99\.499\.4on GSM8K, while reducing accuracy from43\.90%43\.90\\%to43\.53%43\.53\\%and from77\.40%77\.40\\%to76\.40%76\.40\\%\. We therefore use deterministic stride probing by default\.

### 4\.4What Confidence Drift Captures \(RQ3\)

We analyze default CoDR with LLaDA\-8B/Fast\-dLLM on HumanEval and GSM8K \(Lgen=256L\_\{\\mathrm\{gen\}\}\{=\}256, block size3232,k=2k\{=\}2, andτdrift=0\\tau\_\{\\mathrm\{drift\}\}\{=\}0\)\. A token\-event is one committed position observed at one refinement boundary\.

#### Dynamic evolution\.

At the end of each block B1–B8, CoDR probes every position committed at that boundary and computesδi=ψiold−ψinew\\delta\_\{i\}=\\psi\_\{i\}^\{\\mathrm\{old\}\}\-\\psi\_\{i\}^\{\\mathrm\{new\}\}\. We then classify each token\-event by its outcome in that refinement round:NRwas probed but not remasked \(underτdrift=0\\tau\_\{\\mathrm\{drift\}\}=0, typicallyδi≤0\\delta\_\{i\}\\leq 0\);RUhadδi\>0\\delta\_\{i\}\>0and was remasked, but refill reproduced the same token; andREhadδi\>0\\delta\_\{i\}\>0and refill produced a different token\. Figure[3](https://arxiv.org/html/2610.08833#S3.F3)\(a–b\) reports the medianδ\\deltaof each group separately at every block\. RE maintains substantially larger drift than the other groups from B1 to B8 on both tasks, showing that drift tracks evolving contextual instability rather than a calibrated error probability\.

#### Drift magnitude and token revision\.

For each dataset, we collect all token\-events with positive drift \(δ\>0\\delta\>0\), which are selected for remasking under the defaultτdrift=0\\tau\_\{\\mathrm\{drift\}\}=0\. We sort these events byδ\\deltaand divide them into five equal\-frequency bins, from the lowest\-drift quintile Q1 to the highest\-drift quintile Q5\. Within each bin, the edit rate is the fraction of events whose refill produces a token different from the original commitment\. Figure[3](https://arxiv.org/html/2610.08833#S3.F3)\(c\) shows that this rate rises sharply with drift, reaching16\.18%16\.18\\%on HumanEval and14\.08%14\.08\\%on GSM8K in Q5\. Thus, drift magnitude identifies commitments that the model is more likely to overturn during refill\. Full numerical results are reported in Appendix[C](https://arxiv.org/html/2610.08833#A3)\.

#### Relation to successful correction\.

At the output level, Wrong→\\rightarrowCorrect examples have the largest maximum and edited\-token drift on GSM8K \(0\.734/0\.4320\.734/0\.432\); HumanEval shows the same direction \(0\.639/0\.5440\.639/0\.544\), although only two examples make this transition\. Full correctness\-transition statistics are reported in Appendix[C](https://arxiv.org/html/2610.08833#A3)\.

### 4\.5Practical Compute Efficiency \(RQ4\)

#### Compute allocation\.

Figure[4](https://arxiv.org/html/2610.08833#S4.F4)tests whether the base sampler can recover CoDR’s gains by spending a similar NFE budget\. Sweeping the Fast\-dLLM thresholdτ\\tauand EB\-Sampler boundγ\\gammareveals a clear scaling ceiling: additional base\-sampler evaluations do not improve the best HumanEval or GSM8K accuracy\. CoDR instead raises Fast\-dLLM from42\.68%42\.68\\%to43\.90%43\.90\\%on HumanEval and from76\.00%76\.00\\%to77\.80%77\.80\\%on GSM8K, with the same trend for EB\-Sampler\. Thus, its gains come from targeted revision rather than additional NFE alone\.

\(a\)HumanEval\(b\)GSM8K
Figure 4:Compute allocation on LLaDA\-8B \(00\-shot\)\. Solid curves sweep the base sampler’s compute knob, usingτ\\taufor Fast\-dLLM andγ\\gammafor EB\-Sampler\. Dashed lines mark the best swept baseline accuracy, and stars mark the best CoDR result\. CoDR achieves higher accuracy than the swept baselines on both HumanEval and GSM8K, suggesting more effective use of additional NFE\.Table 4:Refinement schedules on LLaDA\-8B/FD\.
#### Refinement schedule\.

Table[4](https://arxiv.org/html/2610.08833#S4.T4)shows that every\-block refinement is not required\. Refining every two blocks preserves HumanEval accuracy, slightly improves GSM8K, and reduces NFE by1414–15%15\\%\. Less frequent schedules trade accuracy for further savings, whereas mid\-block refinement exceeds140140NFE without a meaningful gain\. We retain every block as the simple default and recommend every two blocks when efficiency matters\.

Table 5:HumanEval latency and memory on LLaDA\-8B/FD \(Lgen=256L\_\{\\mathrm\{gen\}\}\{=\}256, batch size11\)\. Enn: refine everynnblocks; F: final only\.
#### Latency, memory, and KV caching\.

Table[5](https://arxiv.org/html/2610.08833#S4.T5)confirms that NFE reflects wall\-clock cost\. Every\-two\-block CoDR improves accuracy by1\.221\.22points while increasing latency from4\.404\.40to5\.555\.55seconds without KV caching, and from2\.772\.77to3\.903\.90seconds with KV drafting\. This schedule is faster than every\-block refinement in both settings and does not increase peak memory\. KV drafting currently accelerates only initial sampling because remasking invalidates cached states for probing and refill\.

Figure 5:ablation study for our method\.

### 4\.6Ablation Study \(RQ5\)

#### Effect of probe stridekk\.

The stridekkcontrols the number of partition groups used for probing\. Largerkkpreserves denser context in each probe but requires more forward evaluations\. Figure[5](https://arxiv.org/html/2610.08833#S4.F5)\(a\), with detailed results in Table[14](https://arxiv.org/html/2610.08833#A4.T14), shows thatk=2k=2gives the best accuracy–NFE trade\-off, while larger values add compute without clear accuracy gains\. We therefore usek=2k=2as the default\.

#### Effect of drift threshold\.

The drift thresholdτdrift\\tau\_\{\\mathrm\{drift\}\}controls how conservative the remasking trigger is\. The defaultτdrift=0\\tau\_\{\\mathrm\{drift\}\}=0remasks any token whose confidence drops under the probe context, while positive thresholds filter out small\-drift tokens\. Figure[5](https://arxiv.org/html/2610.08833#S4.F5)\(b\), with detailed results in Table[15](https://arxiv.org/html/2610.08833#A4.T15), shows thatτdrift=0\\tau\_\{\\mathrm\{drift\}\}=0achieves the best accuracy on both HumanEval and GSM8K\. Positive thresholds reduce NFE but do not improve accuracy, so we keepτdrift=0\\tau\_\{\\mathrm\{drift\}\}=0as the default\.

#### Effect of refinement passes\.

We ablate the number of remask–refill passes at each block boundary\. Figure[5](https://arxiv.org/html/2610.08833#S4.F5)\(c\), with detailed results in Table[16](https://arxiv.org/html/2610.08833#A4.T16), shows that one pass is generally sufficient\. Additional passes substantially increase NFE; they leave HumanEval accuracy unchanged and bring only a small gain on GSM8K with two passes\. This suggests that the first pass captures most useful drift signals, so we use one refinement pass by default\.

#### Effect of block length\.

The block lengthbbdetermines how often refinement is invoked\. Smaller blocks create more refinement opportunities, while larger blocks reduce intervention frequency and NFE\. Figure[5](https://arxiv.org/html/2610.08833#S4.F5)\(d\), with detailed results in Table[17](https://arxiv.org/html/2610.08833#A4.T17), shows that larger blocks degrade the base sampler, suggesting that sparse refinement boundaries allow early errors to accumulate\. CoDR improves over the corresponding baseline across most block lengths\. We useb=32b=32as the default because it gives the best GSM8K result and a strong HumanEval result at lower NFE thanb=16b=16, consistent with recent work[Wu et al\. \(2025\)](https://arxiv.org/html/2610.08833#bib.bib6)\.

#### Robustness across generation lengths\.

Figure[6](https://arxiv.org/html/2610.08833#S4.F6)compares lengths128128,256256, and512512across all four tasks\. CoDR improves average accuracy for both samplers at every length, with gains of1\.901\.90points for Fast\-dLLM and0\.900\.90for EB\-Sampler at length512512\. At length128128, gains are smaller and MATH\-500 regresses, showing robust average improvements but not uniform per\-task gains\. NFE overhead increases with sequence length\. Appendix Table[13](https://arxiv.org/html/2610.08833#A4.T13)provides full results\.

![Refer to caption](https://arxiv.org/html/2610.08833v1/generation_length_ablation.png)Figure 6:Generation\-length robustness on LLaDA\-8B\.Four\-task average accuracy of the baseline and CoDR under Fast\-dLLM \(FD\) and EB sampling across generation lengths\. The heatmap reports task\-level accuracy gains \(pp\) from adding CoDR\.

## 5Conclusion

We introduce CoDR, a training\-free method that revisits MDLM tokens whose confidence decreases as context grows\. Itskk\-partition probe checks all commitments in a few forward passes, then selectively refills unstable tokens under richer context\. Across two backbones, three samplers, and math and code tasks, CoDR improves the accuracy–compute trade\-off\. Diagnostics associate larger drift with token edits and successful corrections, while controlled comparisons distinguish targeted revision from extra decoding alone\. These results show that reversible decoding can correct early commitments without retraining or replacing the base sampler\. Future work may explore adaptive probing and cache\-aware refinement\.

## Limitations

CoDR is an inference\-time refinement method whose effectiveness depends on the underlying base sampler used for refill\. The confidence trigger identifies remasked positions whose support decreases under denser context, but the quality of revised tokens remains bounded by the same MDLM and sampler used for generation\. As a result, occasional regressions may occur when refill introduces fresh local errors, especially in arithmetic\-heavy cases\. In addition, confidence drift is designed as a targeted revision signal rather than a complete error detector\. CoDR intentionally revises only tokens whose support decreases under denser context, leaving other tokens unchanged to avoid excessive editing\. Future work could explore adaptive refinement schedules, stronger refill constraints, and broader evaluation on larger models, multilingual tasks, and KV\-cache optimized decoding\.

## Acknowledgments

The authors acknowledge support from The Hong Kong Polytechnic University \(Research Grant P0046811\), the National Natural Science Foundation of China under Grant No\. 62136005, and the Shenzhen Fundamental Research Program under Grant No\. JCYJ20250604144724032\.

## References

- M\. Arriola, A\. Gokaslan, J\. T\. Chiu, Z\. Yang, Z\. Qi, J\. Han, S\. S\. Sahoo, and V\. KuleshovBlock diffusion: interpolating between autoregressive and diffusion language models\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2503.09573)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Austinet al\.\(2021a\)J\. Austin, D\. D\. Johnson, J\. Ho, D\. Tarlow, and R\. van den BergStructured denoising diffusion models in discrete state\-spaces\.InProceedings of the 35th International Conference on Neural Information Processing Systems,NIPS ’21,Red Hook, NY, USA\.External Links:ISBN 9781713845393Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p1.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Austinet al\.\(2021b\)J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le, and C\. SuttonProgram synthesis with large language models\.External Links:2108\.07732,[Link](https://arxiv.org/abs/2108.07732)Cited by:[§4\.1](https://arxiv.org/html/2610.08833#S4.SS1.p1.1)\.
- Ben\-Hamuet al\.\(2026\)H\. Ben\-Hamu, I\. Gat, D\. Severo, N\. Nolte, and B\. KarrerAccelerated sampling from masked diffusion models via entropy bounded unmasking\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=WBcBhT1NKO)Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p1.1),[§1](https://arxiv.org/html/2610.08833#S1.p3.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px2.p1.1)\.
- Bieet al\.\(2025\)T\. Bie, M\. Cao, K\. Chen, L\. Du, M\. Gong, Z\. Gong, Y\. Gu, J\. Hu, Z\. Huang, Z\. Lan, C\. Li, C\. Li, J\. Li, Z\. Li, H\. Liu, L\. Liu, G\. Lu, X\. Lu, Y\. Ma, J\. Tan, L\. Wei, J\. Wen, Y\. Xing, X\. Zhang, J\. Zhao, D\. Zheng, J\. Zhou, J\. Zhou, Z\. Zhou, L\. Zhu, and Y\. ZhuangLLaDA2\.0: scaling up diffusion language models to 100b\.External Links:2512\.15745,[Link](https://arxiv.org/abs/2512.15745)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. de Oliveira Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, B\. Chan, S\. Gray, N\. Ryder, M\. Pavlov, A\. Power, L\. Kaiser, M\. Bavarian, C\. Winter, P\. Tillet, F\. P\. Such, D\. Cummings, M\. Plappert, F\. Chantzis, E\. Barnes, A\. Herbert\-Voss, W\. H\. Guss, A\. Nichol, A\. Paino, N\. Tezak, J\. Tang, I\. Babuschkin, S\. Balaji, S\. Jain, W\. Saunders, C\. Hesse, A\. N\. Carr, J\. Leike, J\. Achiam, V\. Misra, E\. Morikawa, A\. Radford, M\. Knight, M\. Brundage, M\. Murati, K\. Mayer, P\. Welinder, B\. McGrew, D\. Amodei, S\. McCandlish, I\. Sutskever, and W\. ZarembaEvaluating large language models trained on code\.External Links:2107\.03374,[Link](https://arxiv.org/abs/2107.03374)Cited by:[§4\.1](https://arxiv.org/html/2610.08833#S4.SS1.p1.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.External Links:2110\.14168,[Link](https://arxiv.org/abs/2110.14168)Cited by:[§4\.1](https://arxiv.org/html/2610.08833#S4.SS1.p1.1)\.
- Gonget al\.\(2025a\)S\. Gong, S\. Agarwal, Y\. Zhang, J\. Ye, L\. Zheng, M\. Li, C\. An, P\. Zhao, W\. Bi, J\. Han, H\. Peng, and L\. KongScaling diffusion language models via adaptation from autoregressive models\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=j1tSLYKwg8)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Gonget al\.\(2025b\)S\. Gong, R\. Zhang, H\. Zheng, J\. Gu, N\. Jaitly, L\. Kong, and Y\. ZhangDiffuCoder: understanding and improving masked diffusion models for code generation\.External Links:2506\.20639,[Link](https://arxiv.org/abs/2506.20639)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Guet al\.\(2018\)J\. Gu, J\. Bradbury, C\. Xiong, V\. O\. K\. Li, and R\. SocherNon\-autoregressive neural machine translation\.External Links:1711\.02281,[Link](https://arxiv.org/abs/1711.02281)Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p2.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the MATH dataset\.InThirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 2\),External Links:[Link](https://openreview.net/forum?id=7Bywt2mQsCe)Cited by:[§4\.1](https://arxiv.org/html/2610.08833#S4.SS1.p1.1)\.
- Kimet al\.\(2026\)B\. Kim, D\. Jeon, M\. Jeon, and A\. NoDependency\-aware parallel decoding via attention for diffusion llms\.External Links:2603\.12996,[Link](https://arxiv.org/abs/2603.12996)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px2.p1.1)\.
- Kimet al\.\(2025a\)J\. Kim, K\. Shah, V\. Kontonis, S\. M\. Kakade, and S\. ChenTrain for the worst, plan for the best: understanding token ordering in masked diffusions\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,External Links:[Link](https://openreview.net/forum?id=DjJmre5IkP)Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p1.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px2.p1.1)\.
- Kimet al\.\(2025b\)S\. H\. Kim, S\. Hong, H\. Jung, Y\. Park, and S\. YunKLASS: kl\-guided fast inference in masked diffusion models\.External Links:2511\.05664,[Link](https://arxiv.org/abs/2511.05664)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px2.p1.1)\.
- Kohet al\.\(2025\)H\. Koh, M\. Jhang, D\. Kim, S\. Lee, and K\. JungConditional \[mask\] discrete diffusion language model\.External Links:2411\.06438,[Link](https://arxiv.org/abs/2411.06438)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px2.p1.1)\.
- Louet al\.\(2024\)A\. Lou, C\. Meng, and S\. ErmonDiscrete diffusion modeling by estimating the ratios of the data distribution\.InProceedings of the 41st International Conference on Machine Learning,ICML’24\.Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Nieet al\.\(2025\)S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. LiLarge language diffusion models\.External Links:2502\.09992,[Link](https://arxiv.org/abs/2502.09992)Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p1.1),[§1](https://arxiv.org/html/2610.08833#S1.p4.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Ouet al\.\(2024\)J\. Ou, S\. Nie, K\. Xue, F\. Zhu, J\. Sun, Z\. Li, and C\. LiYour absorbing discrete diffusion secretly models the conditional distributions of clean data\.External Links:2406\.03736Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p1.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Reinet al\.\(2024\)D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. BowmanGPQA: a graduate\-level google\-proof q&a benchmark\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=Ti67584b98)Cited by:[§4\.2](https://arxiv.org/html/2610.08833#S4.SS2.p3.1)\.
- Sahooet al\.\(2024\)S\. S\. Sahoo, M\. Arriola, Y\. Schiff, A\. Gokaslan, E\. Marroquin, J\. T\. Chiu, A\. Rush, and V\. KuleshovSimple and effective masked diffusion language models\.InProceedings of the 38th International Conference on Neural Information Processing Systems,NIPS ’24,Red Hook, NY, USA\.External Links:ISBN 9798331314385Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p1.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Suzgunet al\.\(2023\)M\. Suzgun, N\. Scales, N\. Schärli, S\. Gehrmann, Y\. Tay, H\. W\. Chung, A\. Chowdhery, Q\. Le, E\. Chi, D\. Zhou, and J\. WeiChallenging BIG\-bench tasks and whether chain\-of\-thought can solve them\.InFindings of the Association for Computational Linguistics: ACL 2023,A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 13003–13051\.External Links:[Link](https://aclanthology.org/2023.findings-acl.824/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.824)Cited by:[§4\.2](https://arxiv.org/html/2610.08833#S4.SS2.p3.1)\.
- Wanget al\.\(2026a\)G\. Wang, Y\. Schiff, S\. S\. Sahoo, and V\. KuleshovRemasking discrete diffusion models with inference\-time scaling\.External Links:2503\.00307,[Link](https://arxiv.org/abs/2503.00307)Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p3.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2025\)K\. Wang, Z\. Jiang, H\. Feng, W\. Zhao, L\. Liu, J\. Li, Z\. Lan, and W\. LinCreditDecoding: accelerating parallel decoding in diffusion large language models with trace credits\.External Links:2510\.06133,[Link](https://arxiv.org/abs/2510.06133)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2026b\)S\. Wang, Z\. Zhuang, H\. Ruping, and Y\. ZhangNaRA: noise\-aware loRA for parameter\-efficient fine\-tuning of diffusion LLMs\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=s8UGIRF38a)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Wuet al\.\(2025\)C\. Wu, H\. Zhang, S\. Xue, Z\. Liu, S\. Diao, L\. Zhu, P\. Luo, S\. Han, and E\. XieFast\-dllm: training\-free acceleration of diffusion llm by enabling kv cache and parallel decoding\.External Links:2505\.22618,[Link](https://arxiv.org/abs/2505.22618)Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p1.1),[§1](https://arxiv.org/html/2610.08833#S1.p2.1),[§1](https://arxiv.org/html/2610.08833#S1.p3.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px2.p1.1),[§4\.6](https://arxiv.org/html/2610.08833#S4.SS6.SSS0.Px4.p1.1)\.
- Wu and Huang \(2026\)Y\. Wu and J\. HuangStability\-weighted decoding for diffusion language models\.External Links:2604\.17068,[Link](https://arxiv.org/abs/2604.17068)Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p2.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px2.p1.1)\.
- Yanget al\.\(2025\)L\. Yang, Y\. Tian, B\. Li, X\. Zhang, K\. Shen, Y\. Tong, and M\. WangMMaDA: multimodal large diffusion language models\.External Links:2505\.15809,[Link](https://arxiv.org/abs/2505.15809)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Yeet al\.\(2025\)J\. Ye, Z\. Xie, L\. Zheng, J\. Gao, Z\. Wu, X\. Jiang, Z\. Li, and L\. KongDream 7b: diffusion large language models\.arXiv preprint arXiv:2508\.15487\.Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p1.1),[§1](https://arxiv.org/html/2610.08833#S1.p4.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Youet al\.\(2025\)Z\. You, S\. Nie, X\. Zhang, J\. Hu, J\. Zhou, Z\. Lu, J\. Wen, and C\. LiLLaDA\-v: large language diffusion models with visual instruction tuning\.External Links:2505\.16933,[Link](https://arxiv.org/abs/2505.16933)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Zhaiet al\.\(2026\)K\. Zhai, S\. Mollah, Z\. Wang, and M\. ShahCORE: context\-robust remasking for diffusion language models\.External Links:2602\.04096,[Link](https://arxiv.org/abs/2602.04096)Cited by:[§C\.4](https://arxiv.org/html/2610.08833#A3.SS4.p1.1),[§1](https://arxiv.org/html/2610.08833#S1.p3.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2025\)S\. Zhao, D\. Gupta, Q\. Zheng, and A\. GroverD1: scaling reasoning in diffusion large language models via reinforcement learning\.External Links:2504\.12216,[Link](https://arxiv.org/abs/2504.12216)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.
- Zhouet al\.\(2026\)X\. Zhou, Y\. Hu, and J\. HuangDOS: dependency\-oriented sampler for masked diffusion language models\.External Links:2603\.15340,[Link](https://arxiv.org/abs/2603.15340)Cited by:[§1](https://arxiv.org/html/2610.08833#S1.p3.1),[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px2.p1.1)\.
- Zhuet al\.\(2025\)F\. Zhu, R\. Wang, S\. Nie, X\. Zhang, C\. Wu, J\. Hu, J\. Zhou, J\. Chen, Y\. Lin, J\. Wen, and C\. LiLLaDA 1\.5: variance\-reduced preference optimization for large language diffusion models\.External Links:2505\.19223,[Link](https://arxiv.org/abs/2505.19223)Cited by:[§2](https://arxiv.org/html/2610.08833#S2.SS0.SSS0.Px1.p1.1)\.

## Appendix ADetailed Experiment Setting

Unless otherwise noted, all experiments follow the model, decoding, precision, prompting, and hardware settings described in Section[4\.1](https://arxiv.org/html/2610.08833#S4.SS1)\. Sampler\-specific implementation details are provided in the later appendix sections\.

### A\.1Model and Dataset Licenses

All pre\-trained models \(LLaDA and Dream\) and datasets \(HumanEval, MBPP, GSM8K, and MATH\-500\) used in this work are publicly available under open\-source licenses, such as MIT or Apache 2\.0, and are used strictly for academic research and methodological evaluation, consistent with their intended use\.

## Appendix BStatistical Reliability and Additional Benchmarks

### B\.1Paired Bootstrap Analysis

Our argmax decoding at temperature00is deterministic: rerunning each setting with different seeds produces identical predictions and accuracies\. We therefore quantify finite\-test\-set uncertainty using paired bootstrap resampling\. For each LLaDA sampler, we resample paired baseline/CoDR predictions within each task10,00010\{,\}000times, compute the accuracy gain per task, and average the four gains equally to match Table[1](https://arxiv.org/html/2610.08833#S3.T1)\. Two\-sided results are reported in percentage points in Table[6](https://arxiv.org/html/2610.08833#A2.T6)\. All confidence intervals exclude zero, indicating a significant average gain for each sampler\.

Table 6:Paired bootstrap tests for the LLaDA four\-task average \(10,00010\{,\}000resamples\)\.
### B\.2Additional Benchmarks

We further evaluate LLaDA\-8B on BBH and GPQA\. As shown in Table[7](https://arxiv.org/html/2610.08833#A2.T7), CoDR improves accuracy over both Fast\-dLLM and EB\-Sampler on both benchmarks, extending the main results beyond the original four tasks\.

Table 7:Additional benchmark results\. Each cell reports accuracy \(%\)/NFE\.

## Appendix CConfidence Drift and Correctness Transitions

### C\.1Token\-Level Drift Diagnostics

Tables[8](https://arxiv.org/html/2610.08833#A3.T8)and[9](https://arxiv.org/html/2610.08833#A3.T9)provide the numerical results underlying Figure[3](https://arxiv.org/html/2610.08833#S3.F3): per\-block median drift by refinement outcome and edit rates across positive\-drift quintiles, respectively\.

### C\.2Output\-Level Correctness Transitions

We pair each initial Fast\-dLLM output with its final CoDR output and group examples by correctness transition\. For each example, Maxδ\\deltais the largest token\-event drift observed during refinement, and Edited\-tokenδ\\deltais the median drift over events whose refill changes the token\. Examples without edited events are excluded from the latter statistic\. Table[10](https://arxiv.org/html/2610.08833#A3.T10)reports the group median for each measure\.

On GSM8K, Wrong→\\rightarrowCorrect examples exhibit higher maximum and edited\-token drift than the other transition groups\. HumanEval follows the same direction, although the Wrong→\\rightarrowCorrect group contains only two examples\. Together with Table[9](https://arxiv.org/html/2610.08833#A3.T9), these results associate larger drift with both actual token revision and successful output correction\.

### C\.3Top1 Setting and Analysis

Both variants of our Top1 refinement procedure use top\-11decoding for the initial generation pass; they differ only in how remasked positions are*refilled*during revision\. We compare refilling with top\-11decoding against refilling with an entropy\-bounded \(EB\) sampler, holding everything else fixed \(top\-11phase\-1 generation, identical probe and remask schedule\)\. Top\-11refill commits exactly one token per forward pass, so every remasked span is regenerated one token at a time, adding one forward pass per remasked token and accumulating large number of function evaluations \(NFE\) across all blocks and revision rounds\. The EB sampler can refill multiple tokens in one forward pass and therefore requires far fewer passes\.

This change is free in terms of quality\. With the phase\-1 top\-11generation held fixed, top\-11and EB refill produce*identical*accuracy—pass@10\.42680\.4268on LLaDA\-8B HumanEval and0\.59150\.5915on Dream\-7B HumanEval—while EB refill reduces NFE from377377to303303on LLaDA and from393393to317317on Dream, a∼20%\\sim\\\!20\\%saving in both cases \(Table[11](https://arxiv.org/html/2610.08833#A3.T11)\)\. We therefore use top\-11generation with EB refill \(γ=0\.01\\gamma=0\.01\) as our default: it keeps the high\-precision, one\-token\-at\-a\-time ordering of top\-11where decision quality matters most—the main generation pass—while avoiding its redundant cost when re\-filling remasked spans\.

### C\.4CoRE Setting and Analysis

We follow the CoRE protocol[Zhai et al\. \(2026\)](https://arxiv.org/html/2610.08833#bib.bib13): generation length256256, top\-11decoding \(matching the baseline\), revision everyE=8E=8steps within the mid\-trajectory window\[0\.25,0\.75\]\[0\.25,0\.75\],k=1k=1token revised per step, and a replacement\-confidence threshold of0\.30\.3\. Unlike the original paper, which decodes the entire response as a*single*block, we adopt*block\-wise*decoding \(block=32=32\) with candidate\-set sizem=4m=4andprotect=8\\mathrm\{protect\}=8for*both*LLaDA and Dream\.

#### Block\-wise decoding outperforms single\-block on LLaDA\.

While the paper’s single\-block configuration runs without collapse on LLaDA, it is overall inferior to block\-wise decoding, particularly on multi\-step reasoning\. Table[12](https://arxiv.org/html/2610.08833#A3.T12)compares the two at matched compute \(NFE≈272\\approx 272\)\. Single\-block is marginally stronger on HumanEval, where the larger global context aids code structure, but block\-wise decoding is substantially better on MBPP, MATH\-500, and GSM8K, raising the four\-task average from0\.3940\.394to0\.4690\.469\. The gap is most pronounced on reasoning: single\-block withm=32m=32degrades MATH\-500 to0\.2480\.248and GSM8K to0\.5360\.536, whereas block\-wise decoding withm=4m=4recovers them to0\.3740\.374and0\.7230\.723, near the no\-revision baseline\. We therefore use block\-wise decoding for LLaDA as well\.

#### Single\-block decoding fails outright on Dream\.

On Dream the single\-block configuration does not merely underperform—it collapses to near\-empty generations \(with accuracy only0\.1890\.189in humaneval\)\. This is not a CoRE\-specific implementation artifact: Dream’s own validated samplers \(fastdllm,top1\) also collapse under single\-block decoding \(∼12\\sim\\\!12output tokens\)\. The failure is a property of Dream under single\-block decoding—given a single fully\-masked block, the model places high confidence on an early end\-of\-sequence token before any context has been committed, terminating generation prematurely\. Block\-wise decoding \(block=32=32\) removes this collapse\.

#### Candidate\-set size must scale with block length\.

Within block\-wise decoding, the paper’sm=32m=32is far too large: masking3232candidates inside a3232\-token block erases almost the entire committed context at revision time\. The verify pass then scores token stability under a near\-empty context, mis\-flags already\-committed tokens, and derails generation \(on Dream HumanEval, pass@10\.5732→0\.35980\.5732\\rightarrow 0\.3598; the same over\-perturbation drives LLaDA’s single\-block reasoning collapse above\)\. We therefore rescalemmby*perturbation ratio*rather than absolute count: the paper’sm=32m=32perturbs roughly25%25\\%of the∼128\\sim\\\!128tokens committed at mid\-window revision time, which on a3232\-token block \(∼16\\sim\\\!16tokens filled at that point\) corresponds tom≈4m\\approx 4\. With block=32=32andm=4m=4, both models produce normal\-length outputs and recover baseline\-level accuracy\.

Table 8:Per\-block median confidence drift by refinement outcome\. NR: not remasked; RU: remasked but unchanged; RE: remasked and edited\.Table 9:Edit rate \(%\) among positive\-drift events grouped by equal\-frequency drift quintile\.Table 10:Confidence drift by output correctness transition\.Table 11:Refill sampler comparison with top\-11generation held fixed \(HumanEval, 0\-shot\)\. Top\-11and EB refill give identical accuracy; EB refill cuts NFE by∼20%\\sim\\\!20\\%\.Table 12:LLaDA\-8B\-Instruct CoRE: single\-block \(m=32m\{=\}32\) vs\. block\-wise \(m=4m\{=\}4\), 0\-shot, NFE≈272\\approx 272\. Block\-wise decoding wins on three of four tasks and on average, with the largest gains on reasoning\.

## Appendix DComprehensive Ablation Results

This section provides the exact numerical data corresponding to the hyperparameter sensitivity analysis visualized in Section[4\.6](https://arxiv.org/html/2610.08833#S4.SS6)\. While the main text focuses on the overall Pareto\-optimal trends and performance trade\-offs, the tables below detail the discrete Accuracy and NFE metrics across all evaluated configurations of the CoDR framework\.

Table[13](https://arxiv.org/html/2610.08833#A4.T13)reports robustness across generation lengths\. Table[14](https://arxiv.org/html/2610.08833#A4.T14)reports the isolated effects of probe stridekk, drift thresholdτdrift\\tau\_\{\\mathrm\{drift\}\}\(Table[15](https://arxiv.org/html/2610.08833#A4.T15)\), and the number of refinement passesnpassn\_\{\\mathrm\{pass\}\}\(Table[16](https://arxiv.org/html/2610.08833#A4.T16)\)\. Table[17](https://arxiv.org/html/2610.08833#A4.T17)details the impact of block lengthbb, explicitly comparing the baseline generation against the refined output to illustrate the compounding effect of early errors in sparse commit boundaries\. All results are evaluated using the Fast\-dLLM base sampler \(fd,τ=0\.9\\tau\{=\}0\.9\) on LLaDA\-8B under a00\-shot setting\.

Table 13:Full numerical results for generation\-length robustness on LLaDA\-8B\. Task columns report accuracy \(%\); averages are unweighted over HumanEval \(HE\), full GSM8K, MBPP, and MATH\-500\.Table 14:Ablation results on probe stridekkusingfd\_refinewith LLaDA\-8B\. Accuracy is reported in percentage\. Bold values indicate optimal configurations\.Table 15:Ablation results on drift thresholdτdrift\\tau\_\{\\mathrm\{drift\}\}usingfd\_refinewith LLaDA\-8B\. Positive thresholds reduce NFE by making the trigger more selective\.Table 16:Ablation results on refinement passesnpassn\_\{\\mathrm\{pass\}\}usingfd\_refinewith LLaDA\-8B\. Additional refinement passes beyondn=1n=1add substantial compute for little or no gain\.Table 17:Ablation on block lengthbbusingfd\_refinewith LLaDA\-8B\. Block length controls the intervention frequency and the accuracy–NFE trade\-off\. Extreme sparse boundaries \(b=128b=128\) allow early contextual errors to compound, degrading baseline performance and limiting refinement capabilities\.
## Appendix EDetailed Case Studies

To provide a comprehensive analysis of our method, this section presents detailed generation trajectories across mathematics and code generation tasks\. We contrast the base Fast\-dLLM sampler \(τ=0\.9\\tau=0\.9\) against our CoDR formulation on LLaDA\-8B in setting\.

### E\.1Success Cases

CoDR excels at identifying and correcting early irrational commitments induced by sparse context, cleanly overwriting them once the bidirectional global context stabilizes\.

#### Code Generation Recovery\.

Figures[7](https://arxiv.org/html/2610.08833#A6.F7)and[8](https://arxiv.org/html/2610.08833#A6.F8)illustrate CoDR’s capacity to repair localized syntactic and API\-level fractures\. In Figure[7](https://arxiv.org/html/2610.08833#A6.F7)\(HumanEval/89\), the base sampler commits to a malformed character shift logic due to premature evaluation\. CoDR perfectly isolates the confidence drift precisely at this arithmetic expression and recovers the exact\+4rotation algorithm\. Similarly, Figure[8](https://arxiv.org/html/2610.08833#A6.F8)demonstrates the recovery of a missing dependency; CoDR detects the structural vulnerability of invokingmath\.ceilwithout an import and seamlessly prependsimport mathduring the refinement pass\.

#### Mathematical Reasoning Correction\.

Figures[9](https://arxiv.org/html/2610.08833#A6.F9)and[10](https://arxiv.org/html/2610.08833#A6.F10)highlight CoDR’s precision in arithmetic and algebraic contexts\. In Figure[9](https://arxiv.org/html/2610.08833#A6.F9), a pure arithmetic slip \(1764/12=1421764/12=142\) derails the entire downstream revenue projection\. CoDR flags the fractional setup, unmasks the flawed quotient, and enforces the mathematically sound answer\. More impressively, Figure[10](https://arxiv.org/html/2610.08833#A6.F10)shows CoDR untangling a distorted algebraic substitution \(introducing a sign and term error\) and re\-evaluating the true algebraic path \(P=50P=50\)\.

### E\.2Regression Cases

Despite its overall gains, CoDR is not monotonic and can occasionally degrade a correct base output\. Tracking its failure modes provides sharp boundaries regarding the limitations of training\-free inference\-time interventions\.

Figures[11](https://arxiv.org/html/2610.08833#A6.F11)and[12](https://arxiv.org/html/2610.08833#A6.F12)illustrate the two primary modes of regression: dropped context and fresh hallucinations\. In Figure[11](https://arxiv.org/html/2610.08833#A6.F11), CoDR correctly isolates a cluster of low\-confidence stylistic tokens at the block boundary but accidentally drops a vital preceding string operation \(split\(\)\) during the unmask\-refill process, reducing list evaluation to character\-level iterations\. In Figure[12](https://arxiv.org/html/2610.08833#A6.F12), the base sampler actually derives the exact combinatorial solution\. However, CoDR triggers a stylistic layout change, and during generation, the underlying MDLM introduces a new arithmetic corruption \(\(145\)→1002\\binom\{14\}\{5\}\\to 1002\)\.

#### Analysis

Our diagnostic inspection reveals that regressions are fundamentally driven by*spurious variations during the refill pass*rather than systematic failures in drift detection\. Because the targeted tokens are fed directly back into the underlying MDLM’s standard conditional sampling loop, the refinement step remains statistically bounded by the base model’s inherent local calculation failure rates\. This explains why code generation tasks exhibit virtually zero regressions \(e\.g\., 0 out of 164 on HumanEval\), while arithmetic tasks are more vulnerable to fresh hallucinations\.

## Appendix FAI Ethics Statement

The authors acknowledge the use of LLM assistants only for context refinement and some code refinement during the preparation of this manuscript, and have checked carefully for any potential ethical concerns\.

Base Sampler \(HumanEval/89\) – FAIL

defencrypt\(s\):

encrypted\_string=’’

forcharins:

ifchar\.isalpha\(\):

encrypted\_string\+=chr\(\(ord\(char\)\-ord\(’a’\)\*2\)%26\+ord\(’a’\)\)

else:

encrypted\_string\+=char

returnencrypted\_string

CoDR Refinement \(HumanEval/89\) – PASS

defencrypt\(s\):

encrypted\_string=’’

forcharins:

ifchar\.islower\(\):

encrypted\_string\+=chr\(\(ord\(char\)\-ord\(’a’\)\+4\)%26\+ord\(’a’\)\)

else:

encrypted\_string\+=char

returnencrypted\_string

Figure 7:Code Success 1: The base sampler commits to operator\-precedence garbage \(ord\(’a’\)\*2\)\. CoDR isolates the drift at this precise expression and recovers the exact rotation algorithm under dense context\.Base Sampler \(HumanEval/133\) – FAIL

defsum\_squares\(lst\):

rounded\_lst=\[math\.ceil\(x\)forxinlst\]

squared\_lst=\[x\*\*2forxinrounded\_lst\]

returnsum\(squared\_lst\)

CoDR Refinement \(HumanEval/133\) – PASS

importmath

defsum\_squares\(lst\):

rounded\_lst=\[math\.ceil\(x\)forxinlst\]

squared\_lst=\[x\*\*2forxinrounded\_lst\]

returnsum\(squared\_lst\)

Figure 8:Code Success 2: Confidence drift captures a structural failure where the base model utilizes a library function but omits the mandatory import header, leading to a runtimeNameError\. CoDR repairs the header seamlessly\.Base Sampler \(GSM8K/50\) – WRONG

Lloyd’s chickens produce 252 eggs per day\. There are 7 days in a week, so:252×7=1764​eggs/week252\\times 7=1764\\text\{ eggs/week\}\. There are 12 eggs in a dozen, so:

1764​eggs/week12​eggs/dozen=142​dozens/week\\frac\{1764\\text\{ eggs/week\}\}\{12\\text\{ eggs/dozen\}\}=142\\text\{ dozens/week\}\(BUG: Arithmetic Slip\)

Lloyd sells each dozen for $2, so:

142​dozens/week×2​dollars/dozen=284​dollars/week142\\text\{ dozens/week\}\\times 2\\text\{ dollars/dozen\}=284\\text\{ dollars/week\}Therefore, Lloyd makes284\\boxed\{284\}\.

CoDR Refinement \(GSM8K/50\) – CORRECT

Lloyd’s chickens produce 252 eggs per day\. There are 7 days in a week, so:252×7=1764​eggs/week252\\times 7=1764\\text\{ eggs/week\}\. There are 12 eggs in a dozen, so:

1764​eggs/week12​eggs/dozen=147​dozens/week\\frac\{1764\\text\{ eggs/week\}\}\{12\\text\{ eggs/dozen\}\}=147\\text\{ dozens/week\}\(FIXED: Long Division\)

Lloyd sells each dozen for $2, so:

147​dozens/week×2​dollars/dozen=294​dollars/week147\\text\{ dozens/week\}\\times 2\\text\{ dollars/dozen\}=294\\text\{ dollars/week\}Therefore, Lloyd makes294\\boxed\{294\}\.

Figure 9:Math Success 1: The base model suffers an arithmetic calculation failure \(1764/12=1421764/12=142\)\. CoDR re\-scores the fractional setup and enforces the mathematically sound quotient \(147147\), altering the downstream revenue projection\.Base Sampler \(GSM8K/931\) – WRONG

Equations: \(1\)P\+P\+J=100P\+P\+J=100, \(2\)P=J\+10P=J\+10, \(3\)P=P\+JP=P\+J\. From the third equation, substitutePPin the first:2​P\+J=1002P\+J=100\.

Now, substitutePPfrom the second equation:

2​P\+\(P−10\)=1002P\+\(P\-10\)=100\(BUG: Distorted Substitution\)

3​P−10=100⟹P=303P\-10=100\\implies P=30\. So, Peter is3030years old\.

CoDR Refinement \(GSM8K/931\) – CORRECT

Relationships: \(1\)P\+P\+J=100P\+P\+J=100, \(2\)P=J\+10P=J\+10, \(3\)P=P\+JP=P\+J\. Using the third equation, substitutePPin the first:2​P\+J=1002P\+J=100\.

Now, using the second equation, substitutePPin the above equation:

2​P\+0\+10=1002P\+0\+10=100\(FIXED: Algebraic Soundness\)

2​P=100⟹P=502P=100\\implies P=50\. So, Peter is5050years old\.

Figure 10:Math Success 2: The base model maps the constraints accurately but introduces a sign and term error during variable evaluation\. CoDR tracks the logical fracture and re\-evaluates the true algebraic path \(P=50P=50\)\.Base Sampler \(MBPP/117\) – PASS

deflong\_words\(n,words\):

words=words\.split\(\)

long\_words=\[wordforwordinwordsiflen\(word\)\>n\]

returnlong\_words

CoDR Refinement \(MBPP/117\) – FAIL

deflong\_words\(n,words\):

return\[wordforwordinwordsiflen\(word\)\>n\]

Figure 11:Code Regression: CoDR correctly isolates a cluster of low\-confidence stylistic tokens at the block boundary but accidentally drops a vital preceding token string operation \(split\(\)\) during the unmask\-refill process, reducing list evaluation to character\-level iterations\.Base Sampler \(MATH\-500 Doc 330\) – PASS

1\. Calculate the total number of ways to choose 5 members out of 14:

\(145\)=14\!5\!​\(14−5\)\!=2002\\binom\{14\}\{5\}=\\frac\{14\!\}\{5\!\(14\-5\)\!\}=2002\(CORRECT Combinatorics\)

2\. Ways from Republicans:\(85\)=56\\binom\{8\}\{5\}=56\. 3\. Ways from Democrats:\(65\)=6\\binom\{6\}\{5\}=6\.

4\. Subtract exclusions:2002−56−6=19402002\-56\-6=1940\.

CoDR Refinement \(MATH\-500 Doc 330\) – FAIL

1\. Calculate the total number of ways to choose 5 members out of 14:

\(145\)=14\!5\!​\(14−5\)\!=1002\\binom\{14\}\{5\}=\\frac\{14\!\}\{5\!\(14\-5\)\!\}=1002\(REGRESSION: Hallucinated Factorial\)

2\. Ways from Republicans:\(85\)=56\\binom\{8\}\{5\}=56\. 3\. Ways from Democrats:\(65\)=6\\binom\{6\}\{5\}=6\.

4\. Subtract exclusions:1002−56−6=9401002\-56\-6=940\.

Figure 12:Math Regression: A classic instantiation of fresh hallucination during resampling\. The base sampler derives the exact combinatorial solution\. CoDR triggers a stylistic layout change, but the model introduces a new arithmetic corruption \(\(145\)→1002\\binom\{14\}\{5\}\\to 1002\) during generation\.