When Confidence Misleads: Suffix Anchoring and Anchor-Proximity Confidence Modulation for Diffusion Language Models

Hugging Face Daily Papers Papers

Summary

Researchers propose a training-free method called Suffix-Anchored Confidence Modulation to improve confidence-based decoding in diffusion language models by addressing issues with EOT tokens and premature decoding.

Diffusion language models decode text by iteratively denoising masked token sequences, making the choice of which positions to decode a central inference-time decision. Most training-free decoding strategies use model confidence for position selection, assuming that high-confidence positions are ready to be decoded. In this work, we revisit this assumption by studying when confidence misleads fully non-autoregressive (fully non-AR) decoding. EOT tokens can receive high confidence and cause incomplete generation; inserting a suffix anchor can mitigate this issue but introduces local overconfidence near the anchor, causing anchor-adjacent tokens to be decoded too early. To address these issues, we propose Suffix-Anchored Confidence Modulation, a simple training-free method that inserts a short suffix anchor to encourage response completion and modulates confidence near the anchor according to decoding progress. This preserves the response-completion benefit of suffix anchoring while reducing premature decoding of anchor-adjacent tokens. Across text-only reasoning, vision-language reasoning, and code-generation benchmarks, our method consistently improves confidence-based fully non-AR decoding, outperforms explicit EOT suppression, and preserves the parallel decoding advantage of fully non-AR generation.
Original Article
View Cached Full Text

Cached at: 06/01/26, 03:20 PM

Paper page - When Confidence Misleads: Suffix Anchoring and Anchor-Proximity Confidence Modulation for Diffusion Language Models

Source: https://huggingface.co/papers/2605.28181

Abstract

Researchers investigate how confidence-based decoding in fully non-autoregressive models can be improved by addressing issues with EOT tokens and premature decoding through suffix-anchored confidence modulation.

Diffusion language modelsdecode text by iteratively denoisingmasked token sequences, making the choice of which positions to decode a central inference-time decision. Most training-free decoding strategies usemodel confidencefor position selection, assuming that high-confidence positions are ready to be decoded. In this work, we revisit this assumption by studying when confidence misleads fully non-autoregressive (fully non-AR) decoding.EOT tokenscan receive high confidence and cause incomplete generation; inserting asuffix anchorcan mitigate this issue but introduces local overconfidence near the anchor, causing anchor-adjacent tokens to be decoded too early. To address these issues, we propose Suffix-AnchoredConfidence Modulation, a simple training-free method that inserts a shortsuffix anchorto encourage response completion and modulates confidence near the anchor according to decoding progress. This preserves the response-completion benefit ofsuffix anchoring while reducing premature decoding of anchor-adjacent tokens. Across text-only reasoning,vision-language reasoning, and code-generation benchmarks, our method consistently improves confidence-based fully non-AR decoding, outperforms explicit EOT suppression, and preserves the parallel decoding advantage of fully non-AR generation.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2605\.28181

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.28181 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.28181 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.28181 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Supportive Token Revealing for Fast Diffusion Language Model Decoding

arXiv cs.CL

This paper proposes AXON, a training-free module that improves the quality-latency trade-off of discrete diffusion language model decoding by intelligently selecting 'anchor' tokens to reveal first, using attention, uncertainty, and confidence signals to support subsequent denoising steps. Experiments on reasoning and code-generation benchmarks show AXON reduces function evaluations while maintaining or improving accuracy.

Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models

arXiv cs.CL

This paper proposes Dynamic Infilling Anchors (DIA), a training-free method for diffusion large language models that dynamically estimates end-anchor positions to enforce format constraints (e.g., parseable JSON, reasoning templates) while avoiding the rigidity of fixed-span approaches. Experiments show significant zero-shot gains on GSM8K and MATH benchmarks.

CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models

arXiv cs.CL

论文提出 CoDR(Confidence-Drift Remasking),一种免训练、与采样器无关的后处理精炼方法,通过检测已提交 token 的置信度漂移并对其进行重遮蔽与重新生成,从而在掩码扩散语言模型的推理与编码任务上以较少的前向传播次数提升准确率。