When Confidence Misleads: Suffix Anchoring and Anchor-Proximity Confidence Modulation for Diffusion Language Models
Summary
Researchers propose a training-free method called Suffix-Anchored Confidence Modulation to improve confidence-based decoding in diffusion language models by addressing issues with EOT tokens and premature decoding.
View Cached Full Text
Cached at: 06/01/26, 03:20 PM
Paper page - When Confidence Misleads: Suffix Anchoring and Anchor-Proximity Confidence Modulation for Diffusion Language Models
Source: https://huggingface.co/papers/2605.28181
Abstract
Researchers investigate how confidence-based decoding in fully non-autoregressive models can be improved by addressing issues with EOT tokens and premature decoding through suffix-anchored confidence modulation.
Diffusion language modelsdecode text by iteratively denoisingmasked token sequences, making the choice of which positions to decode a central inference-time decision. Most training-free decoding strategies usemodel confidencefor position selection, assuming that high-confidence positions are ready to be decoded. In this work, we revisit this assumption by studying when confidence misleads fully non-autoregressive (fully non-AR) decoding.EOT tokenscan receive high confidence and cause incomplete generation; inserting asuffix anchorcan mitigate this issue but introduces local overconfidence near the anchor, causing anchor-adjacent tokens to be decoded too early. To address these issues, we propose Suffix-AnchoredConfidence Modulation, a simple training-free method that inserts a shortsuffix anchorto encourage response completion and modulates confidence near the anchor according to decoding progress. This preserves the response-completion benefit ofsuffix anchoring while reducing premature decoding of anchor-adjacent tokens. Across text-only reasoning,vision-language reasoning, and code-generation benchmarks, our method consistently improves confidence-based fully non-AR decoding, outperforms explicit EOT suppression, and preserves the parallel decoding advantage of fully non-AR generation.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2605\.28181
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.28181 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.28181 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.28181 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Supportive Token Revealing for Fast Diffusion Language Model Decoding
This paper proposes AXON, a training-free module that improves the quality-latency trade-off of discrete diffusion language model decoding by intelligently selecting 'anchor' tokens to reveal first, using attention, uncertainty, and confidence signals to support subsequent denoising steps. Experiments on reasoning and code-generation benchmarks show AXON reduces function evaluations while maintaining or improving accuracy.
Unsure but Certain: Uncovering the Representation-Confidence Gap in Diffusion Language Models
This paper identifies a 'representation confidence gap' in diffusion language models: internal states detect input noise accurately but reported confidence stays high and answer ranking degrades under noise. It introduces a lightweight, training-free extraction tool that leverages hidden states to improve ranking without modifying the base model.
Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models
This paper proposes Dynamic Infilling Anchors (DIA), a training-free method for diffusion large language models that dynamically estimates end-anchor positions to enforce format constraints (e.g., parseable JSON, reasoning templates) while avoiding the rigidity of fixed-span approaches. Experiments show significant zero-shot gains on GSM8K and MATH benchmarks.
CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models
论文提出 CoDR(Confidence-Drift Remasking),一种免训练、与采样器无关的后处理精炼方法,通过检测已提交 token 的置信度漂移并对其进行重遮蔽与重新生成,从而在掩码扩散语言模型的推理与编码任务上以较少的前向传播次数提升准确率。
The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Models
This paper identifies a failure mode in masked diffusion language models where confidence-based decoding leads to high-confidence errors on complex reasoning tasks, and shows that confidence-aligned training exacerbates this issue while random masking preserves reasoning performance.