Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models

# Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models
Source: [https://arxiv.org/html/2607.15655](https://arxiv.org/html/2607.15655)
Yingqian Cui1Wei Deng2Lantao Mei Hang Li1 Charu C\. Aggarwal3Hui Liu1Yue Xing1 1Michigan State University2Morgan Stanley3IBM T\.J\. Watson Research Center

###### Abstract

Masked diffusion language models \(DLMs\) enable parallel text generation by iteratively refining masked tokens, offering a promising alternative to autoregressive decoding\. Recent lookahead\-based decoding methods improve the accuracy–efficiency trade\-off by exploring future decoding states before committing token updates\. However, existing approaches mainly rely on shallow one\-step lookahead, which optimizes immediate information gain but can be suboptimal for longer\-horizon decoding trajectories\. Meanwhile, we find that a naive extension for deeper lookahead is also ineffective, as fixed\-depth rollout introduces additional computation and cannot adapt to heterogeneous intermediate decoding states\. Thus, in this work, we propose AdaLook, an adaptive lookahead framework for DLM decoding\. AdaLook dynamically determines whether to continue rollout based on candidate\-score variance and further enables branch expansion when intermediate rollout states require additional exploration\. This design avoids unnecessary deep rollout while allowing the decoder to re\-trigger lookahead from informative intermediate states\. Experiments on various benchmarks and models demonstrate that AdaLook achieves a better accuracy–decoding steps trade\-off than existing one\-step lookahead decoding methods\.

Adaptive Multi\-Step Lookahead Decoding for Diffusion Language Models

Yingqian Cui1Wei Deng2Lantao Mei Hang Li1Charu C\. Aggarwal3Hui Liu1Yue Xing11Michigan State University2Morgan Stanley3IBM T\.J\. Watson Research Center

## 1Introduction

Masked diffusion language models \(DLMs\) have recently emerged as an alternative to traditional autoregressive \(AR\) models for text generation\. Unlike AR models, which generate tokens sequentially in a token\-by\-token manner, DLMs operate by iteratively refining a sequence of masked tokens, enabling parallel decoding across multiple positionsSahooet al\.\([2024](https://arxiv.org/html/2607.15655#bib.bib1)\); Nieet al\.\([2026](https://arxiv.org/html/2607.15655#bib.bib2)\); Yeet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib25)\)\. This property offers a fundamental advantage over AR decoding: By relaxing the strict left\-to\-right dependency, DLMs can potentially improve generation efficiency and enable more flexible inference strategies\.

![Refer to caption](https://arxiv.org/html/2607.15655v1/intro_new.png)Figure 1:Accuracy vs Decoding Steps \(evaluated on MATH500 with LLaDA\-8B\-Instruct\)While the diffusion framework supports parallel token updates in principle, its denoising\-based training objective does not prescribe a specific decoding order or token selection strategy, leaving a large design space for inference\-time decoding that can substantially affect generation quality and efficiency\. To better exploit the parallel nature of DLMs, a growing line of work focuses on designing efficient decoding strategies\. Existing methods have evolved from fixed\-step parallel decodingNieet al\.\([2026](https://arxiv.org/html/2607.15655#bib.bib2)\)to more adaptive confidence\-aware decoding frameworksYuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib4)\); Wuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib17)\); Weiet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib16)\)\.

While most methods rely on greedy token selection based on current confidence estimates,Fuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib15)\)propose a lookahead\-based strategic exploration mechanism that evaluates candidate token updates by their downstream decoding benefit\. Specifically, this method performs a lookahead search over future decoding states when the model exhibits low confidence over the remaining masked positions\. It aims to identify informative tokens whose commitment leads to a chain of subsequent high\-confidence predictions, thereby enabling more tokens to be decoded per step and reducing the total number of decoding steps required\.

Since the method is designed from an immediate information\-gain perspective, it primarily evaluates whether a tentative token commitment can make more masked positions confident in the next decoding state\. This naturally leads to a one\-step lookahead design\. However, from a longer\-horizon perspective, such local exploration can be suboptimal: the token that maximizes immediate confidence gain may not lead to the best future trajectory, causing the search to be trapped in a locally favorable but globally suboptimal commitment\.

To bridge this gap, we focus on developing a multi\-step lookahead framework that evaluates candidate token updates over longer decoding horizons rather than based on immediate one\-step evidence alone\. A straightforward approach is to perform deeper rollouts over future decoding trajectories\. However, this extension introduces additional technical challenges\. According to our initial experiments shown in Figure[1](https://arxiv.org/html/2607.15655#S1.F1), naively increasing lookahead depth does not improve the trade\-off between generation quality and decoding steps\.

This limitation mainly arises from two aspects\.\(L1\)Deeper lookahead introduces additional computational overhead, while the optimal rollout depth can vary significantly across different examples within the same dataset and decoding stages within the same example\. As a result, a uniformly fixed rollout depth may introduce additional computation without yielding proportional performance gains, ultimately degrading the overall quality\-efficiency trade\-off\.\(L2\)Naive multi\-step rollout follows a fixed forward trajectory for each hypothesis, with no mechanism to assess whether intermediate rollout states require further branching\. This prevents the decoder from dynamically branching into alternative decoding paths when intermediate rollout states remain uncertain\.

To address the limitations, we proposeAdaLook\(AdaptiveLookahead Decoding\), a multi\-step lookahead framework that dynamically determines the rollout depth based on the decoding state while enabling branch expansion during lookahead\. To address\(L1\), before each additional rollout step, AdaLook determines whether further rollout is necessary based on the variance of candidate scores computed from frontier confidence\. Only when the variance exceeds a predefined threshold does the decoder continue expanding future decoding trajectories\. To address\(L2\), after each rollout step, each branch is re\-evaluated to determine whether additional expansion is still required\. The decoder then jointly considers the expansion status of all candidate branches: branches that no longer require further expansion are prioritized\. If all branches require further expansion, the highest\-scoring branch is selected and its decoding state is passed back as the starting point for the next round of lookahead exploration\. This adaptive process avoids unnecessary deep rollout while enabling flexible re\-triggering of lookahead at intermediate stages, allowing the decoder to discover and commit to more informative decoding trajectories\.

As shown in Figure[1](https://arxiv.org/html/2607.15655#S1.F1), AdaLook achieves a better accuracy–decoding steps trade\-off than the existing 1\-step lookahead decoding method\. Extensive experiments further demonstrate that this advantage consistently generalizes across different datasets and model backbones\.

## 2Related Works

Diffusion Language Models\.Diffusion models have achieved remarkable success in continuous domains such as image and audio generationHoet al\.\([2020](https://arxiv.org/html/2607.15655#bib.bib13)\); Songet al\.\([2020](https://arxiv.org/html/2607.15655#bib.bib14)\); Konget al\.\([2020](https://arxiv.org/html/2607.15655#bib.bib7)\), which has motivated efforts to extend the diffusion paradigm to discrete text\. Early work on discrete diffusion, notably D3PMAustinet al\.\([2021](https://arxiv.org/html/2607.15655#bib.bib12)\), established a general framework for defining noising processes over categorical variables, including absorbing\-state masking corruption\. Subsequent works leverage this masking\-based formulation to develop masked diffusion language models \(MDLMs\) that generate text via iterative unmaskingSahooet al\.\([2024](https://arxiv.org/html/2607.15655#bib.bib1)\); Shiet al\.\([2024](https://arxiv.org/html/2607.15655#bib.bib11)\); Ouet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib8)\)\. Building on these foundations, recent large\-scale MDLMs have reached performance competitive with autoregressive counterparts: LLaDANieet al\.\([2026](https://arxiv.org/html/2607.15655#bib.bib2)\)trains an 8B model from scratch with bidirectional attention, DreamYeet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib25)\)develops a diffusion\-based LLM initializing from a pretrained autoregressive model, and LLaDA 2\.0Bieet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib10)\)further scales to 100B parameters with a Mixture\-of\-Experts architecture\.

Decoding Strategies Methods for MDLMs\.To fully leverage the parallel nature of MDLMs, a growing line of work focuses on designing efficient decoding strategies\. Initial attempts focus on fixed\-step parallel decodingNieet al\.\([2026](https://arxiv.org/html/2607.15655#bib.bib2)\); Yeet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib25)\), while subsequent works introduce more adaptive confidence\-aware decoding strategies, including fixed\-threshold and dynamic threshold decodingYuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib4)\); Wuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib17)\)\. Other studies further improve decoding efficiency through multi\-stage decoding strategiesWeiet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib16)\), confidence calibrationHuanget al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib24)\), entropy\-bounded unmaskingBen\-Hamuet al\.\([2026](https://arxiv.org/html/2607.15655#bib.bib23)\), and KV Caching techniquesWuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib17)\); Liuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib22)\)\. More recently, lookahead\-based decoding methods introduce inference\-time search by exploring future decoding trajectories before committing updatesLeeet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib3)\); Fuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib15)\)\. However, as discussed in Section[1](https://arxiv.org/html/2607.15655#S1), existing methods mainly rely on shallow one\-step lookahead, which limits their ability to perform long\-horizon planning during decoding\. Notably, our work mainly builds on the efficiency\-oriented lookahead framework ofFuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib15)\), which targets a better accuracy–decoding steps trade\-off\. In contrast,Leeet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib3)\)uses lookahead primarily to enhance generation performance, with less emphasis on the decoding efficiency\.

## 3Preliminary

Our multi\-step lookahead framework is built upon the Explore\-then\-Exploit \(ETE\) decoding strategy introduced byFuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib15)\)\. In this section, we introduce the two components of ETE:Fast Block Diffusion SamplingandConfidence\-based Lookahead Mechanism\.

### 3\.1Fast Block Diffusion Sampling

Block diffusion decoding generates the sequence progressively in a block\-by\-block manner, where a sequence of lengthnnis partitioned intoLLdecoding blocks with block sizenbn\_\{b\}\. Within each block, the decoder performs iterative refinement before proceeding to the next blockNieet al\.\([2026](https://arxiv.org/html/2607.15655#bib.bib2)\)\.

Fast Block Diffusion Sampling improves decoding efficiency by assigning a fixed budget ofNNdecoding steps to each block and moving forward to the next block once the budget is exhausted, rather than waiting for complete convergence within the current block\. Since earlier blocks may still contain masked tokens when the current block begins decoding, the decoder commits high\-confidence tokens across all unlocked blocks simultaneously at each step, thereby increasing the number of tokens unmasked per decoding step compared to decoding within a single block alone\.

### 3\.2Confidence\-Based Lookahead Mechanism

The core idea of the confidence\-based lookahead mechanism is to selectively perform lookahead exploration when the model’s confidence over remaining masked positions is low\. It uses a 1\-step forward rollout to identify which token updates are most likely to unlock subsequent high\-confidence predictions\. The confidence here refers to the model’s predicted probability for its most likely token at each masked position\. The exploration procedure can be divided into three components: exploration triggering, candidate construction, and hypothesis selection\.

1\. Exploration triggering\.The decoder activates lookahead exploration when the average confidence over the current decoding frontierℱ\\mathcal\{F\}falls below a thresholdγ\\gamma, while the number of remaining masked positions exceeds a minimumNeN\_\{e\}\. The frontierℱ\\mathcal\{F\}is defined as the set of masked positions up to the midpoint of the current block\.

2\. Candidate construction\.According to the candidate selection strategy ofFuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib15)\), once exploration is triggered, the method identifies a set of informative candidate positions from the masked positions:

ℋ=Topki∈ℳt​\(−\|cti​\(𝐱t\)−cinfo\|\+β⋅\(i−\(bt−1\)​nb\)\),\\mathcal\{H\}=\\mathrm\{Topk\}\_\{i\\in\\mathcal\{M\}\_\{t\}\}\\left\(\-\\left\|c\_\{t\}^\{i\}\(\\mathbf\{x\}\_\{t\}\)\-c^\{\\mathrm\{info\}\}\\right\|\+\\beta\\cdot\(i\-\(b\_\{t\}\-1\)n\_\{b\}\)\\right\),where𝐱t\\mathbf\{x\}\_\{t\}denotes the partially decoded sequence at decoding steptt, with masked positionsℳt\\mathcal\{M\}\_\{t\}\.cti​\(𝐱t\)c\_\{t\}^\{i\}\(\\mathbf\{x\}\_\{t\}\)denotes the confidence at positionii,btb\_\{t\}is the index of the current decoding block at steptt,nbn\_\{b\}is the block size, andβ\\betais a hyper\-parameter balancing the two terms\. The first term selects positions whose confidence is close to the target exploration levelcinfoc^\{\\mathrm\{info\}\}, which captures tokens that are uncertain but potentially informative\. Following the empirical findings ofFuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib15)\), we setcinfo=0\.2c^\{\\mathrm\{info\}\}=0\.2, as tokens around this confidence level are shown to be more informative for triggering more downstream high\-confidence predictions\. The second term usesi−\(bt−1\)​nbi\-\(b\_\{t\}\-1\)n\_\{b\}, the relative position of tokeniiwithin the current block, to assign slightly larger scores to later positions whenβ\>0\\beta\>0, thereby encouraging the decoding frontier to move forward\.

3\. Hypothesis selection\.For each candidate positionj∈ℋj\\in\\mathcal\{H\}, the decoder commits positionjjand performs a one\-step forward pass to obtain the resulting statext\+1;jx\_\{t\+1;j\}\. Each hypothesis is scored as:

s​\(j\)=α⋅log⁡ctj​\(𝐱t\)\\displaystyle s\(j\)=\\alpha\\cdot\\log c\_\{t\}^\{j\}\(\\mathbf\{x\}\_\{t\}\)\+log∑i∈S^t\+1​\(j\)ct\+1i\(𝐱t\+1;j\)\)𝟏\(cti\(𝐱t\+1;j\)≥C\)\.\\displaystyle\\quad\+\\log\\sum\_\{i\\in\\hat\{S\}\_\{t\+1\}\(j\)\}c\_\{t\+1\}^\{i\}\(\\mathbf\{x\}\_\{t\+1;j\}\)\)\\mathbf\{1\}\\\!\\left\(c\_\{t\}^\{i\}\(\\mathbf\{x\}\_\{t\+1;j\}\)\\geq C\\right\)\.S^t\+1​\(j\)\\hat\{S\}\_\{t\+1\}\(j\)denotes the predicted token positions after rollout,CCis the confidence threshold, andα\\alphais a regularization parameter balancing the two terms\. Intuitively, the scoring function jointly considers both the confidence of the explored token itself and the amount of downstream high\-confidence predictions induced after rollout\. The candidate with the highest score is selected as the committed decoding trajectory for subsequent inference\.

## 4Adaptive Multi\-Step Lookahead

As in Section[1](https://arxiv.org/html/2607.15655#S1), one\-step lookahead remains suboptimal due to its limited rollout horizon, while naively extending rollout depth fails to improve the quality\-efficiency trade\-off\. To more flexibly determine rollout depth and better handle intermediate rollout states that require further exploration, we propose AdaLook, an adaptive multi\-step lookahead framework in this section\.

### 4\.1General Framework

The general workflow of the method is shown in Figure[2](https://arxiv.org/html/2607.15655#S4.F2)\. It consists of two key components:Adaptive Rollout ContinuationandDynamic Branch Expansion, which respectively address the\(L1\)and\(L2\)discussed in Section[1](https://arxiv.org/html/2607.15655#S1)\. For simplicity, the figure illustrates a special case with the maximum lookahead depth set to 2\. The detailed algorithm is shown in Algorithm[1](https://arxiv.org/html/2607.15655#alg1)in Appendix[A](https://arxiv.org/html/2607.15655#A1)\.

![Refer to caption](https://arxiv.org/html/2607.15655v1/framework.png)Figure 2:Overview of the adaptive multi\-step lookahead mechanism \(when the maximum lookahead depth is 2\)\.
### 4\.2Adaptive Rollout Continuation

To extend one\-step lookahead to a multi\-step setting, we maintain a set ofkkcandidate decoding trajectories and evaluate each trajectory after every rollout step\. For thejj\-th candidate trajectory, we compute a cumulative rollout score:

s~\(R\)​\(j\)=α​log⁡ctj​\(𝐱t\)\+log⁡\(∑r=1R∑i∈𝒯jrct\+ri​\(𝐱t\+r;j\)\)\\displaystyle\\tilde\{s\}^\{\(R\)\}\(j\)=\\alpha\\log c\_\{t\}^\{j\}\(\\mathbf\{x\}\_\{t\}\)\+\\log\\left\(\\sum\_\{r=1\}^\{R\}\\sum\_\{i\\in\\mathcal\{T\}\_\{j\}^\{r\}\}c\_\{t\+r\}^\{i\}\(\\mathbf\{x\}\_\{t\+r;j\}\)\\right\)\(1\)whereRRdenotes the current rollout depth,𝒯jr\\mathcal\{T\}\_\{j\}^\{r\}is the set of positions that become high\-confidence and are selected for commitment at therr\-th rollout step under candidatejj\. The score is a direct extension of the original ETE score: when multi\-step lookahead is used, we accumulate the downstream confidence gains across all rollout steps\.

Before each additional rollout step, we compute the variance of the current candidate trajectory scores, denoted asVar​\(\{s~\(R\)​\(j\)\}j=1k\)\\mathrm\{Var\}\(\\\{\\tilde\{s\}^\{\(R\)\}\(j\)\\\}\_\{j=1\}^\{k\}\)\. If the variance falls below a thresholdτ\\tau, indicating that the candidates remain not sufficiently discriminative, the decoder proceeds with another rollout step\. Once the variance exceedsτ\\tau, or the rollout depth reaches the maximumTmaxT\_\{\\text\{max\}\}, the decoder commitsj∗=argmaxj​\(s~\(R\)​\(j\)\)j^\{\*\}=\\text\{argmax\}\_\{j\}\(\\tilde\{s\}^\{\(R\)\}\(j\)\)and returns the selected committed state to the main decoding loop for subsequent decoding\.

### 4\.3Dynamic Branch Expansion

During multi\-step rollout, individual branches may evolve into qualitatively different decoding states: some may reach stable high\-confidence regions where continuing the current rollout is sufficient, while others may encounter new low\-confidence regions that require re\-triggering lookahead exploration\. To handle this heterogeneity, after each extra forward step, every rollout branch independently determines whether further lookahead exploration should be triggered again according to the original ETE triggering condition\. Based on the expansion status of all candidate branches, the decoder handles three cases:

Case 1: No branch requires further expansion\.All hypotheses have reached stable decoding states\. In this case, the decoder continues the adaptive rollout continuation procedure until the candidate trajectory scores become sufficiently distinguishable or the rollout depth reaches the maximum rollout stepTmaxT\_\{\\max\}\.

Case 2: All branches require further expansion\.When all trajectories remain uncertain, the decoder first selects the highest\-scoring candidate according to the updated rollout score:

j∗=arg⁡maxj⁡\(s~​\(j\)\+log⁡c¯ℱj\),j^\{\*\}=\\arg\\max\_\{j\}\\left\(\\tilde\{s\}\(j\)\+\\log\\bar\{c\}^\{j\}\_\{\\mathcal\{F\}\}\\right\),wheres~​\(j\)\\tilde\{s\}\(j\)accumulates the confidence of tokens already committed along hypothesisjjbefore the latest rollout forward pass\.c¯ℱj\\bar\{c\}^\{j\}\_\{\\mathcal\{F\}\}denotes the average confidence over the decoding frontier after the latest rollout forward pass\. Intuitively, the additional frontier\-confidence term encourages selecting hypotheses whose latest rollout state is more promising for future decoding\.

The current lookahead round then ends and the selected statext\+1\+r;j∗x\_\{t\+1\+r;j^\{\*\}\}is returned to the main decoding loop as the initialization for the next round of lookahead exploration\. This allows the decoder to dynamically re\-trigger lookahead from intermediate rollout states, enabling flexible branching rather than being restricted to a fixed linear rollout trajectory\. Furthermore, since evaluating the trigger condition already requires a forward pass to obtaincjc\_\{j\}for all hypotheses, the confidence predictions obtained for the selected state𝐱t\+1\+r;j∗\\mathbf\{x\}\_\{t\+1\+r;j^\{\*\}\}are cached and reused as the initial forward pass of the next lookahead round, avoiding one redundant model forward\.

Case 3: Mixed expansion status\.When only part of the candidate trajectories require further expansion, the decoder prunes branches that still trigger lookahead and retains only stable branches for subsequent rollout iterations\. The adaptive rollout continuation loop resumes over the remaining stable candidates\. This process continues until a single branch remains, the score variance exceedsτ\\tau, or the rollout budgetTmaxT\_\{\\text\{max\}\}is exhausted, at which point the best remaining candidate is committed\. This design prioritizes trajectories that have already transitioned into stable decoding regimes, while avoiding excessive exploration on repeatedly uncertain branches that may lead to diminishing returns in decoding efficiency\.

#### Hyperparameter Selection

Our algorithm introduces several hyperparameters that control candidate selection, rollout triggering, and decoding progression\. Since exhaustively tuning all hyperparameters on each dataset would be computationally expensive, we discuss our hyperparameter selection strategy in this subsection\.

Specifically, we group the hyperparameters into three main categories:

1\. Fixed hyperparameters\.We fixk=4k=4,α=0\.1\\alpha=0\.1,β=0\.01\\beta=0\.01, ,Tmax=2T\_\{\\text\{max\}\}=2andcinfo=0\.2c\_\{\\mathrm\{info\}\}=0\.2across all experiments\. These values are either adopted from prior workFuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib15)\)or chosen based on calibration results\. We observe no significant dataset\-specific variation for these hyperparameters\.

2\. Calibration\-guided hyperparameters\.For hyperparameters that are not fixed, we use a small calibration set to evaluate different combinations and identify general selection rules\. Since parameters such asCCandNNinduce different accuracy\-efficiency trade\-offs, we analyze the configurations on the empirical Pareto frontier to identify the best parameter patterns\.

In particular, we observe a clear interaction between the confidence thresholdCCand the per\-block unlocking budgetNN\. Both parameters affect the decoding pace: smallerNNunlocks subsequent blocks earlier, while smallerCCcommits more tokens per step\. Both choices can reduce the number of decoding steps, but may also make the decoding process more aggressive\. Among Pareto\-frontier configurations, the preferred threshold tends to increase slowly with the decoding budget\. This empirical trend can be roughly summarized by the following range\-valued guideline:

C∗​\(N\)∈\[0\.5\+0\.05​log2⁡N,0\.5\+0\.10​log2⁡N\],C^\{\*\}\(N\)\\in\\left\[0\.5\+0\.05\\log\_\{2\}N,\\;0\.5\+0\.10\\log\_\{2\}N\\right\],where the selected value is clipped to the calibration range\[0\.5,0\.9\]\[0\.5,0\.9\]\.

To explain this relationship, whenNNis small, the decoding frontier advances more quickly because subsequent blocks are unlocked after fewer decoding rounds\. In this case, a lower confidence threshold is often preferred so that the decoder can commit enough reliable tokens before the frontier moves forward, providing useful context for later blocks\. In contrast, whenNNis large, the decoder spends more refinement rounds on the current block before unlocking later blocks\. We provide justification about the log\-scale relationship in Appendix[B](https://arxiv.org/html/2607.15655#A2)\.

We also find thatNeN\_\{e\}, the minimum number of remaining frontier tokens required to trigger lookahead, has little impact on performance within a reasonable range\. To reduce the number of tunable hyperparameters, we setNe=NN\_\{e\}=Nby default\.

Notably, these trends are generally consistent across different datasets, suggesting that the calibration rules are not strongly dataset\-specific\.

3\. Empirical hyperparameters without stable trends\.The main hyperparameter that does not show a stable pattern on the calibration set isγ\\gamma, which controls when lookahead is triggered\. We find thatγ\\gammagenerally performs reasonably within the range\[0\.1,0\.4\]\[0\.1,0\.4\], but no single value consistently dominates across datasets or decoding budgets\. We therefore consider both a universal setting and an optimized setting in our experiments; the details of these settings are discussed in Section[5](https://arxiv.org/html/2607.15655#S5)\.

## 5Empirical Evaluation

### 5\.1Experimental Setups

Models, Benchmarks and BaselinesWe conduct experiments mainly usingLLaDA\-8B\-InstructNieet al\.\([2026](https://arxiv.org/html/2607.15655#bib.bib2)\)\. We evaluate AdaLook on four widely used benchmarks, includingMMLUHendryckset al\.\([2020](https://arxiv.org/html/2607.15655#bib.bib19)\),GSM8KCobbeet al\.\([2021](https://arxiv.org/html/2607.15655#bib.bib18)\),MATH500Hendryckset al\.\([2021](https://arxiv.org/html/2607.15655#bib.bib21)\), andBBHSuzgunet al\.\([2023](https://arxiv.org/html/2607.15655#bib.bib5)\), covering both general knowledge and mathematical reasoning tasks\. For GSM8K, we use the full test set with 1,319 examples, and for MATH500, we use all 500 examples\. For BBH and MMLU, we randomly sample 500 examples from the full test set for evaluation\. For all benchmarks, we use a generation length of 512 and a block size of 64\. All experiments are performed using greedy decoding and run on NVIDIA H200 GPUs\. We compare AdaLook with ETEFuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib15)\)and the confidence\-aware parallel decoding strategy from Fast\-dLLMWuet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib17)\), which adaptively adjusts the decoding progress according to both the number of decoded tokens and their confidence scores\.

Hyperparameter Configurations\.Both ETE and AdaLook involve several hyperparameters\. As discussed in Section[4\.3](https://arxiv.org/html/2607.15655#S4.SS3.SSS0.Px1), the hyperparameters that need to be varied across configurations for both methods are the per\-block unlocking budgetNN, the confidence thresholdCC, and the lookahead triggering thresholdγ\\gamma\. Importantly, AdaLook does not introduce additional tunable hyperparameters beyond those required by ETE; the extra hyperparameters specific to our adaptive multi\-step mechanism are determined once on a small calibration set and fixed across all experiments\.

ForNNandCC, we follow the calibration guideline in Section[4\.3](https://arxiv.org/html/2607.15655#S4.SS3.SSS0.Px1)\. Specifically, we chooseNNfrom powers of two in the range\[2,64\)\[2,64\), and pair eachNNwith confidence thresholdsCCsuggested by the corresponding range\-valued rule, forming a set of accuracy–efficiency trade\-off configurations\. Forγ\\gamma, since no stable calibration pattern is observed, we follow the discussion in Section[4\.3](https://arxiv.org/html/2607.15655#S4.SS3.SSS0.Px1)and consider values in\[0\.1,0\.4\]\[0\.1,0\.4\]\. For both ETE and AdaLook, we report two settings\. Theoptimizedsetting scans all combinations ofγ∈\[0\.1,0\.4\]\\gamma\\in\[0\.1,0\.4\]and the calibratedN/CN/Cpairs, and reports the best Pareto\-frontier results\. Thestandardsetting fixesγ=0\.2\\gamma=0\.2, a default value that appears frequently among optimized configurations, and uses the sameN/CN/Ccalibration rule\.

![Refer to caption](https://arxiv.org/html/2607.15655v1/main.png)Figure 3:Accuracy vs decoding steps across different benchmarks
### 5\.2Main Results

We report the accuracy–efficiency trade\-off obtained under differentN/CN/Cconfigurations in Figure[3](https://arxiv.org/html/2607.15655#S5.F3)\. Each point corresponds to one decoding configuration, and the curves compare how accuracy changes as the decoding budget varies across different datasets\. Here, decoding steps refer to the average number of model forward passes used during generation across evaluation examples\. Based on Figure[3](https://arxiv.org/html/2607.15655#S5.F3), we have the following observations\.

First, AdaLook consistently achieves a better accuracy–efficiency trade\-off than the baselines across datasets\. When comparing \(AdaLook \(Standard\),ETE \(Standard\), Fast\-dLLM\), and \(AdaLook \(Optimized\),ETE \(Optimized\), and Fast\-dLLM\), we generally observe the orderingAdaLook\>\>ETE\>\>Fast\-dLLM in terms of the trade\-off between accuracy and decoding steps\. For example, at around 45 decoding steps,AdaLook \(Optimized\)achieves approximately 4\.5% higher accuracy thanETE \(Optimized\)\. This suggests that adaptive multi\-step lookahead improves upon one\-step lookahead by exploring more informative future decoding trajectories, while still avoiding unnecessary rollout computation\. In contrast, Fast\-dLLM only achieves a comparable trade\-off on the relatively easier GSM8K benchmark when sufficient decoding steps are allowed\. This suggests that confidence\-aware decoding without more strategic exploration mechanisms is insufficient for harder reasoning tasks where low\-confidence regions are more frequently encountered\.

Second, the improvement varies across datasets\. The gain is relatively modest on GSM8K, but becomes more pronounced on more challenging benchmarks such as MATH and BBH\. This indicates that harder examples may benefit more from additional lookahead steps, as they often require longer\-range exploration to identify reliable token commitments\. To explain this, more difficult tasks may involve higher uncertainty in decoding, where locally confident token decisions may still lead to suboptimal reasoning paths, making shallow exploration insufficient\. In contrast, simpler examples often exhibit more stable and less ambiguous decoding trajectories, where local confidence is already a strong indicator of correct token commitments, leaving less room for deeper lookahead to further improve the trade\-off\.

Third, AdaLook also tends to achieve a higher maximum reachable accuracy, particularly on more challenging benchmarks such as MATH, MMLU, and BBH\. For example, on MATH, the best performance achieved by AdaLook is 43\.6%, compared to 42\.6% for ETE and 42\.2% for Fast\-dLLM\. This suggests that adaptive multi\-step lookahead not only improves decoding efficiency but also helps the decoder discover higher\-quality reasoning trajectories that may be difficult to reach with shallow exploration alone\.

To further evaluate the generality of AdaLook across different model backbones, we also conduct experiments on Dream\-v0\-Instruct\-7BYeet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib25)\); the corresponding results are provided in Appendix[C](https://arxiv.org/html/2607.15655#A3)\.

### 5\.3Analysis on Latency

In Section 5\.2, we use the average number of forward steps per example to evaluate decoding efficiency\. This metric provides a hardware\-independent estimate of inference cost\. However, our adaptive multi\-step lookahead may require more batched forward passes during rollout, since multiple candidate trajectories are evaluated in parallel within a single lookahead step\. As a result, this additional batched computation may increase the overall generation latency, even under a similar number of effective decoding steps\.

To better understand this effect, we further analyze the generation latency under different beam sizeskk, which corresponds to the batch size used for parallel candidate evaluation\. Table[1](https://arxiv.org/html/2607.15655#S5.T1)reports the average time per decoding step on H200 and B200 GPUs, measured in milliseconds\. Notably, this value is not computed by timing an isolated model forward pass\. Instead, we measure the end\-to\-end generation time, including model forward passes, score computation, candidate selection, branch evaluation, and other decoding overhead, and divide it by the average number of forward steps\. Therefore, the reported value should be interpreted as the average time corresponding to one decoding step reported in Figure[3](https://arxiv.org/html/2607.15655#S5.F3)for different decoding methods\.

According to the results in Table[1](https://arxiv.org/html/2607.15655#S5.T1), the latency per decoding step increases for both ETE and AdaLook as the beam size grows, reflecting the additional cost of evaluating more candidate trajectories in parallel during lookahead decoding\. Due to the additional rollout and branch expansion operations, AdaLook introduces slightly higher latency than ETE\. However, this gap becomes much smaller on stronger GPUs such as the B200\. Under the beam sizek=4k=4used in our main experiments, AdaLook increases the average per\-step latency by only around7%7\\%compared with ETE on B200 GPUs\. This suggests that, with a reasonable beam size and modern GPU hardware, the additional latency introduced by our adaptive multi\-step lookahead is nearly negligible in practice\.

Table 1:Average latency on MATH with different GPUs
### 5\.4Discussion on Code Generation Tasks

![Refer to caption](https://arxiv.org/html/2607.15655v1/code2.png)Figure 4:Performance of different methods in HumanEvalIn this section, we analyze the behavior of lookahead\-based decoding on code generation tasks, which exhibit a qualitatively different response to lookahead exploration compared to mathematical reasoning and general knowledge benchmarks\. Specifically, we find that confidence\-based lookahead signals are less effective on coding benchmarks such as HumanEvalChenet al\.\([2021](https://arxiv.org/html/2607.15655#bib.bib20)\): even the one\-step lookahead strategy in ETE provides only marginal gains over Fast Block Diffusion Sampling\. As shown in Figure[4](https://arxiv.org/html/2607.15655#S5.F4), the trade\-off curves of Fast Block Diffusion Sampling and Fast Block Diffusion Sampling \+ Lookahead remain highly similar, with only minor differences across decoding budgets\. Moreover, our adaptive multi\-step lookahead method also lies roughly on the same frontier, suggesting that both one\-step and multi\-step lookahead provide limited benefits for code generation tasks\.

We hypothesize that this limitation stems from the structural nature of code generation\. Unlike mathematical reasoning tasks, where informative token commitments can effectively resolve local ambiguity and lead to cascades of high\-confidence future predictions, code generation often involves long\-range structural and semantic dependencies that span much of the generated sequence\. As a result, the quality of a token update is difficult to reliably assess within a limited rollout horizon, and the confidence gain observed during short lookahead trajectories may provide a weak signal for identifying truly beneficial token commitments\. Consequently, although our method extends one\-step lookahead to adaptive multi\-step exploration, the rollout horizon may still remain insufficient to capture the long\-range dependencies required for code generation, limiting the benefit of strategic lookahead\. We leave the development of lookahead mechanisms better suited for code generation to future work\.

## 6Conclusion

In this work, we present AdaLook, an adaptive multi\-step lookahead framework for efficient masked diffusion language model decoding\. Unlike prior one\-step lookahead methods that focus on immediate information gain, AdaLook explores longer decoding trajectories while dynamically controlling rollout depth and branch expansion\. By using candidate\-score variance to decide whether to continue rollout and re\-evaluating branches at intermediate states, the proposed method avoids unnecessary computation while enabling flexible exploration\. Experiments on multiple benchmarks and model backbones show that AdaLook improves the accuracy–decoding steps trade\-off over existing decoding baselines\.

## Limitations

While our method consistently improves the accuracy–efficiency trade\-off in terms of the number of decoding steps, the adaptive multi\-step lookahead introduces slightly higher computational overhead per decoding step due to the batched forward passes required for hypothesis evaluation, even on high\-end GPUs\. We believe that this overhead will become increasingly negligible as hardware capabilities continue to advance and batched inference becomes more efficient\.

## References

- Structured denoising diffusion models in discrete state\-spaces\.Advances in neural information processing systems34,pp\. 17981–17993\.Cited by:[§2](https://arxiv.org/html/2607.15655#S2.p1.1)\.
- H\. Ben\-Hamu, I\. Gat, D\. Severo, N\. S\. Nolte, and B\. Karrer \(2026\)Accelerated sampling from masked diffusion models via entropy bounded unmasking\.Advances in Neural Information Processing Systems38,pp\. 55981–56007\.Cited by:[§2](https://arxiv.org/html/2607.15655#S2.p2.1)\.
- T\. Bie, M\. Cao, K\. Chen, L\. Du, M\. Gong, Z\. Gong, Y\. Gu, J\. Hu, Z\. Huang, Z\. Lan,et al\.\(2025\)Llada2\. 0: scaling up diffusion language models to 100b\.arXiv preprint arXiv:2512\.15745\.Cited by:[§2](https://arxiv.org/html/2607.15655#S2.p1.1)\.
- M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. D\. O\. Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman,et al\.\(2021\)Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§5\.4](https://arxiv.org/html/2607.15655#S5.SS4.p1.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§5\.1](https://arxiv.org/html/2607.15655#S5.SS1.p1.1)\.
- H\. Fu, B\. Huang, V\. Adams, C\. Wang, V\. Srinivasan, and J\. Jiao \(2025\)From bits to rounds: parallel decoding with exploration for diffusion language models\.arXiv preprint arXiv:2511\.21103\.Cited by:[§1](https://arxiv.org/html/2607.15655#S1.p3.1),[§2](https://arxiv.org/html/2607.15655#S2.p2.1),[§3\.2](https://arxiv.org/html/2607.15655#S3.SS2.p3.1),[§3\.2](https://arxiv.org/html/2607.15655#S3.SS2.p4.14),[§3](https://arxiv.org/html/2607.15655#S3.p1.1),[§4\.3](https://arxiv.org/html/2607.15655#S4.SS3.SSS0.Px1.p3.5),[§5\.1](https://arxiv.org/html/2607.15655#S5.SS1.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2020\)Measuring massive multitask language understanding\.arXiv preprint arXiv:2009\.03300\.Cited by:[§5\.1](https://arxiv.org/html/2607.15655#S5.SS1.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021\)Measuring mathematical problem solving with the math dataset\.arXiv preprint arXiv:2103\.03874\.Cited by:[§5\.1](https://arxiv.org/html/2607.15655#S5.SS1.p1.1)\.
- J\. Ho, A\. Jain, and P\. Abbeel \(2020\)Denoising diffusion probabilistic models\.Advances in neural information processing systems33,pp\. 6840–6851\.Cited by:[§2](https://arxiv.org/html/2607.15655#S2.p1.1)\.
- P\. Huang, S\. Liu, Z\. Liu, Y\. Yan, S\. Wang, Z\. Chen, and T\. Xiao \(2025\)Pc\-sampler: position\-aware calibration of decoding bias in masked diffusion models\.arXiv preprint arXiv:2508\.13021\.Cited by:[§2](https://arxiv.org/html/2607.15655#S2.p2.1)\.
- Z\. Kong, W\. Ping, J\. Huang, K\. Zhao, and B\. Catanzaro \(2020\)Diffwave: a versatile diffusion model for audio synthesis\.arXiv preprint arXiv:2009\.09761\.Cited by:[§2](https://arxiv.org/html/2607.15655#S2.p1.1)\.
- S\. Lee, S\. Kim, J\. Park, and D\. Park \(2025\)Lookahead unmasking elicits accurate decoding in diffusion language models\.arXiv preprint arXiv:2511\.05563\.Cited by:[§2](https://arxiv.org/html/2607.15655#S2.p2.1)\.
- Z\. Liu, Y\. Yang, Y\. Zhang, J\. Chen, C\. Zou, Q\. Wei, S\. Wang, and L\. Zhang \(2025\)Dllm\-cache: accelerating diffusion large language models with adaptive caching\.arXiv preprint arXiv:2506\.06295\.Cited by:[§2](https://arxiv.org/html/2607.15655#S2.p2.1)\.
- S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. Li \(2026\)Large language diffusion models\.Advances in Neural Information Processing Systems38,pp\. 50608–50646\.Cited by:[§1](https://arxiv.org/html/2607.15655#S1.p1.1),[§1](https://arxiv.org/html/2607.15655#S1.p2.1),[§2](https://arxiv.org/html/2607.15655#S2.p1.1),[§2](https://arxiv.org/html/2607.15655#S2.p2.1),[§3\.1](https://arxiv.org/html/2607.15655#S3.SS1.p1.3),[§5\.1](https://arxiv.org/html/2607.15655#S5.SS1.p1.1)\.
- J\. Ou, S\. Nie, K\. Xue, F\. Zhu, J\. Sun, Z\. Li, and C\. Li \(2025\)Your absorbing discrete diffusion secretly models the conditional distributions of clean data\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 64972–65009\.Cited by:[§2](https://arxiv.org/html/2607.15655#S2.p1.1)\.
- S\. S\. Sahoo, M\. Arriola, Y\. Schiff, A\. Gokaslan, E\. Marroquin, J\. T\. Chiu, A\. Rush, and V\. Kuleshov \(2024\)Simple and effective masked diffusion language models\.Advances in Neural Information Processing Systems37,pp\. 130136–130184\.Cited by:[§1](https://arxiv.org/html/2607.15655#S1.p1.1),[§2](https://arxiv.org/html/2607.15655#S2.p1.1)\.
- J\. Shi, K\. Han, Z\. Wang, A\. Doucet, and M\. Titsias \(2024\)Simplified and generalized masked diffusion for discrete data\.Advances in neural information processing systems37,pp\. 103131–103167\.Cited by:[§2](https://arxiv.org/html/2607.15655#S2.p1.1)\.
- Y\. Song, J\. Sohl\-Dickstein, D\. P\. Kingma, A\. Kumar, S\. Ermon, and B\. Poole \(2020\)Score\-based generative modeling through stochastic differential equations\.arXiv preprint arXiv:2011\.13456\.Cited by:[§2](https://arxiv.org/html/2607.15655#S2.p1.1)\.
- M\. Suzgun, N\. Scales, N\. Schärli, S\. Gehrmann, Y\. Tay, H\. W\. Chung, A\. Chowdhery, Q\. Le, E\. Chi, D\. Zhou,et al\.\(2023\)Challenging big\-bench tasks and whether chain\-of\-thought can solve them\.InFindings of the Association for Computational Linguistics: ACL 2023,pp\. 13003–13051\.Cited by:[§5\.1](https://arxiv.org/html/2607.15655#S5.SS1.p1.1)\.
- Q\. Wei, Y\. Zhang, Z\. Liu, D\. Liu, and L\. Zhang \(2025\)Accelerating diffusion large language models with slowfast: the three golden principles\.arXiv e\-prints,pp\. arXiv–2506\.Cited by:[§1](https://arxiv.org/html/2607.15655#S1.p2.1),[§2](https://arxiv.org/html/2607.15655#S2.p2.1)\.
- C\. Wu, H\. Zhang, S\. Xue, Z\. Liu, S\. Diao, L\. Zhu, P\. Luo, S\. Han, and E\. Xie \(2025\)Fast\-dllm: training\-free acceleration of diffusion llm by enabling kv cache and parallel decoding\.arXiv preprint arXiv:2505\.22618\.Cited by:[§1](https://arxiv.org/html/2607.15655#S1.p2.1),[§2](https://arxiv.org/html/2607.15655#S2.p2.1),[§5\.1](https://arxiv.org/html/2607.15655#S5.SS1.p1.1)\.
- J\. Ye, Z\. Xie, L\. Zheng, J\. Gao, Z\. Wu, X\. Jiang, Z\. Li, and L\. Kong \(2025\)Dream 7b: diffusion large language models\.arXiv preprint arXiv:2508\.15487\.Cited by:[Appendix C](https://arxiv.org/html/2607.15655#A3.p1.1),[§1](https://arxiv.org/html/2607.15655#S1.p1.1),[§2](https://arxiv.org/html/2607.15655#S2.p1.1),[§2](https://arxiv.org/html/2607.15655#S2.p2.1),[§5\.2](https://arxiv.org/html/2607.15655#S5.SS2.p5.1)\.
- R\. Yu, X\. Ma, and X\. Wang \(2025\)Dimple: discrete diffusion multimodal large language model with parallel decoding\.arXiv preprint arXiv:2505\.16990\.Cited by:[§1](https://arxiv.org/html/2607.15655#S1.p2.1),[§2](https://arxiv.org/html/2607.15655#S2.p2.1)\.

## Appendix AAlgorithm

We present the detailed algorithm of AdaLook in Algorithm[1](https://arxiv.org/html/2607.15655#alg1)\.

Algorithm 1Adaptive Multi\-Step Lookahead Algorithm1:Model

pθp\_\{\\theta\}, current state

𝐱t\\mathbf\{x\}\_\{t\}, masked token set

ℳ\\mathcal\{M\}, beam width

kk, medium\-confidence target

cinfoc^\{\\text\{info\}\}, position bias

β\\beta, downstream threshold

CC, sample\-quality weight

α\\alpha, variance threshold

τ\\tau, lookahead budget

MM
2:// Step 1: Selectkkexploration candidates

3:Score each masked position

i∈ℳi\\in\\mathcal\{M\}:

ϕi=−\|ci−cinfo\|\+β​\(i−\(b−1\)​nb\)\\quad\\phi\_\{i\}=\-\|c\_\{i\}\-c^\{\\text\{info\}\}\|\+\\beta\\,\(i\-\(b\-1\)\\,n\_\{b\}\)
4:

ℋ←Top​\-​k​\(\{ϕi\}i∈ℱ\)\\mathcal\{H\}\\leftarrow\\mathrm\{Top\\text\{\-\}\}k\\bigl\(\\\{\\phi\_\{i\}\\\}\_\{i\\in\\mathcal\{F\}\}\\bigr\)
5:// Step 2: Buildkkhypotheses and run one\-step lookahead

6:foreach candidate

j∈ℋj\\in\\mathcal\{H\}do

7:

𝐱j←𝐱t\\mathbf\{x\}^\{j\}\\leftarrow\\mathbf\{x\}\_\{t\}with position

jjcommitted

8:endfor

9:

\{\(x^j,cj\)\}j∈ℋ←BatchedForward​\(pθ,\{𝐱j\}j∈ℋ\)\\\{\(\\hat\{x\}^\{j\},c^\{j\}\)\\\}\_\{j\\in\\mathcal\{H\}\}\\leftarrow\\textsc\{BatchedForward\}\(p\_\{\\theta\},\\,\\\{\\mathbf\{x\}^\{j\}\\\}\_\{j\\in\\mathcal\{H\}\}\)
10:foreach

j∈ℋj\\in\\mathcal\{H\}do

11:

sj←Eq\.\(1\)s\_\{j\}\\leftarrow\\text\{Eq\.\(1\)\}⊳\\trianglerightScorejjby unlocked confidence mass

12:endfor

13:// Step 3: Variance gate — decide whether to run extra forward chain

14:if

Var⁡\(\{sj\}j∈ℋ\)<τ\\operatorname\{Var\}\(\\\{s\_\{j\}\\\}\_\{j\\in\\mathcal\{H\}\}\)<\\tauthen

15:for

r=1,…,M−1r=1,\\ldots,M\-1do

16:

\{xj\}←Commit​\(\{\(x^j,cj\)\}\)\\\{x^\{j\}\\\}\\leftarrow\\textsc\{Commit\}\(\\\{\(\\hat\{x\}^\{j\},c^\{j\}\)\\\}\)for all

j∈ℋj\\in\\mathcal\{H\}
17:

\{\(x^j,cj\)\}←BatchedForward​\(pθ,\{𝐱j\}j∈ℋ\)\\\{\(\\hat\{x\}^\{j\},c^\{j\}\)\\\}\\leftarrow\\textsc\{BatchedForward\}\(p\_\{\\theta\},\\,\\\{\\mathbf\{x\}^\{j\}\\\}\_\{j\\in\\mathcal\{H\}\}\)
18:Check retrigger:

rj←CheckTrigger​\(\{\(x^j,cj\)\}\)\\mathrm\{r\}\_\{j\}\\leftarrow\\textsc\{CheckTrigger\}\(\\\{\(\\hat\{x\}^\{j\},c^\{j\}\)\\\}\)for all

jj
19:ifall

rj=True\\mathrm\{r\}\_\{j\}=\\text\{True\}then⊳\\trianglerightCase 2: all trigger

20:

j∗←arg⁡maxj⁡\(sj\+log⁡c¯ℱj\)j^\{\*\}\\leftarrow\\arg\\max\_\{j\}\\,\(s\_\{j\}\+\\log\\bar\{c\}^\{j\}\_\{\\mathcal\{F\}\}\)
21:return

\(x^j∗,cj∗\)\(\\hat\{x\}^\{j^\{\*\}\},c^\{j^\{\*\}\}\)as initial state for next decoding round

22:elseifno

rj=True\\mathrm\{r\}\_\{j\}=\\text\{True\}then⊳\\trianglerightCase 1: no trigger

23:

sj←Eq\.\(1\)s\_\{j\}\\leftarrow\\text\{Eq\.\(1\)\}for all

j∈ℋj\\in\\mathcal\{H\}
24:if

Var⁡\(sj\)≥τ\\operatorname\{Var\}\(s\_\{j\}\)\\geq\\tauthen

25:

j∗←arg⁡maxj⁡sjj^\{\*\}\\leftarrow\\arg\\max\_\{j\}\\,\{s\}\_\{j\};go toCommit

26:endif

27:else⊳\\trianglerightCase 3: Mixed — shrink to stable hypotheses

28:

ℋ←\{j∈ℋ:rj=False\}\\mathcal\{H\}\\leftarrow\\\{j\\in\\mathcal\{H\}:\\mathrm\{r\}\_\{j\}=\\texttt\{False\}\\\}
29:

sj←Eq\.\(1\)s\_\{j\}\\leftarrow\\text\{Eq\.\(1\)\}for all

j∈ℋj\\in\\mathcal\{H\}
30:if

\|ℋ\|=1\|\\mathcal\{H\}\|=1then

31:

j∗←j^\{\*\}\\leftarrowthe unique element in

ℋ\\mathcal\{H\};go toCommit

32:endif

33:if

Var⁡\(sj\)≥τ\\operatorname\{Var\}\(s\_\{j\}\)\\geq\\tauthen

34:

j∗←arg⁡maxj⁡sjj^\{\*\}\\leftarrow\\arg\\max\_\{j\}\\,\{s\}\_\{j\};go toCommit

35:endif

36:endif

j∗←arg⁡maxj⁡sjj^\{\*\}\\leftarrow\\arg\\max\_\{j\}\\,\{s\}\_\{j\};go toCommit

37:endfor

38:else

j∗←arg⁡maxj⁡sjj^\{\*\}\\leftarrow\\arg\\max\_\{j\}\\,\{s\}\_\{j\};go toCommit

39:endif

40:// Final Step: Commit

41:

𝐱j∗←Commit​\(x^j∗,cj\)\\mathbf\{x\}^\{j^\{\*\}\}\\leftarrow\\textsc\{Commit\}\(\\hat\{x\}^\{j^\{\*\}\},c^\{j\}\)
42:return

𝐱j∗\\mathbf\{x\}^\{j^\{\*\}\}

## Appendix BJustification for the log\-scale calibration rule

![Refer to caption](https://arxiv.org/html/2607.15655v1/log.png)Figure 5:Relationship betweenlog⁡\(p​\(c\)\)\\log\(p\(c\)\)andCC\(evaluated with MATH500\)\.We provide an additional justification for why the preferred confidence thresholdCCincreases approximately logarithmically with the per\-block unlocking budgetNN\.

For a fixed confidence thresholdCC, the expected fraction of tokens that can be committed in one refinement step is

p​\(C\)=Pr⁡\(c≥C\)\.p\(C\)=\\Pr\(c\\geq C\)\.AfterNNrounds the expected fraction of tokens committed within the block can be approximated as1−\(1−p​\(C\)\)N1\-\(1\-p\(C\)\)^\{N\}\. If we want to maintain a comparable block progression rate with different values ofNN, this probability should remain roughly at a constant level, then we have

\(1−p​\(C\)\)N≈const\.\(1\-p\(C\)\)^\{N\}\\approx\\text\{const\}\.Taking log for both sides:

N​log⁡\(1−p​\(C\)\)=−const\.N\\log\(1\-p\(C\)\)=\-\\text\{const\}\.Whenppis small, we havelog⁡\(1−p\)≈−p\\log\(1\-p\)\\approx\-p, then we have

p​\(C\)≈constNp\(C\)\\approx\\frac\{\\text\{const\}\}\{N\}\(2\)
Thus, asNNincreases, the required per\-step commitment probabilityp​\(C\)p\(C\)decreases, meaning that a larger confidence thresholdCCcan be used\.

To connect this relationship to the logarithmic schedule, we empirically examine the tail probability of masked\-token confidence\. As shown in Figure[5](https://arxiv.org/html/2607.15655#A2.F5), over the practical calibration range, the log\-tail probabilitylog⁡Pr⁡\(c≥C\)\\log\\Pr\(c\\geq C\)decreases approximately linearly withCCwhenC∈\[0\.2,0\.9\]C\\in\[0\.2,0\.9\]\. This suggests that the high\-confidence tail can be approximated by an exponential form:

p​\(C\)≈exp⁡\(−λ​C\),p\(C\)\\approx\\exp\(\-\\lambda C\),for someλ\>0\\lambda\>0\. Combining this empirical approximation with Eq\.\([2](https://arxiv.org/html/2607.15655#A2.E2)\) yield:

exp⁡\(−λ​C\)≈constN\.\\exp\(\-\\lambda C\)\\approx\\frac\{\\text\{const\}\}\{N\}\.Therefore,

C≈1λ​log⁡N\+const\.C\\approx\\frac\{1\}\{\\lambda\}\\log N\+\\mathrm\{const\}\.This provides a theoretical justification for using a log\-scale guideline to selectCCas the unlocking budgetNNvaries, while the exact threshold is chosen empirically on the calibration set\.

## Appendix CAdditional Empirical Results for DREAM\-v0\-Instruct\-7B

![Refer to caption](https://arxiv.org/html/2607.15655v1/dream.png)Figure 6:Accuracy vs decoding steps across different benchmarks with DREAM\-v0\-Instruct\-7BIn this subsection, we provide additional evaluation of AdaLook usingDream\-v0\-Instruct\-7BYeet al\.\([2025](https://arxiv.org/html/2607.15655#bib.bib25)\)\. The results on GSM8K and MMLU are presented in Figure[6](https://arxiv.org/html/2607.15655#A3.F6)\. As shown in the figure, AdaLook consistently achieves a better accuracy–efficiency trade\-off than the two baselines, demonstrating that the benefits of adaptive multi\-step lookahead generalize across different DLM backbones\.

## Appendix DDisclosure of AI Assistant Usage

AI assistants were used solely for language polishing and proofreading during manuscript preparation\. Specifically, they were used to improve grammar, spelling, clarity, and overall readability of the text\. All technical content, experiments, analyses, and conclusions were developed and verified by the authors\.

## Appendix EArtifacts, Licensing, and Intended Use

All datasets and pretrained models used in this work are publicly available and are utilized in accordance with their respective licenses\. Our use is strictly limited to research and evaluation purposes, consistent with the intended scope of each artifact\. No third\-party artifacts are redistributed or modified in terms of their access conditions\.