Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation

# Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation
Source: [https://arxiv.org/html/2605.30753](https://arxiv.org/html/2605.30753)
###### Abstract

Diffusion\-based large language models \(dLLMs\) support parallel text generation via iterative denoising, yet inference remains latency\-heavy because many steps are spent on redundant refinement and repeated remasking of tokens whose final values are already determined\. Prior acceleration methods mainly depend on step\-local confidence heuristics or fixed schedules, which are sensitive to prompt and task variation and ignore strong positional effects within a sequence\. We cast diffusion decoding as a dynamic control problem and show that token\-wise denoising trajectories provide the key signal for reliable control\. We propose a trace\-aware decoding framework with two components\. First,Temporal\-Spatial Parallel Decoding \(TSPD\)uses a lightweight temporal\-spatial controller that consumes per\-token trajectory features, including confidence, entropy, and momentum, together with token position, to decide when a token has converged and can be safely fixed\. Second, we introduceConfidence Extrapolation \(CE\), a training\-free state\-space module that forecasts future logit trends with uncertainty to support proactive decisions, including safe look\-ahead and targeted stabilization when trajectories are oscillatory or underconfident\. Together, TSPD and CE reduce unnecessary denoising iterations while preserving output quality, and they compose cleanly with system optimizations such as KV caching\.

Machine Learning, ICML

## 1Introduction

Diffusion\-based large language models \(dLLMs\)\(Nieet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib26); Yeet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib17); Khannaet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib25); Zhaoet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib15)\)offer an alternative to autoregressive decoding\(Schuurmanset al\.,[2024](https://arxiv.org/html/2605.30753#bib.bib11); Radfordet al\.,[2018](https://arxiv.org/html/2605.30753#bib.bib21)\)by updating many token positions in parallel through iterative denoising\. This framework supports flexible generation orders, infilling, and a direct quality speed trade off through the number of denoising steps\(Hoet al\.,[2020](https://arxiv.org/html/2605.30753#bib.bib20); Songet al\.,[2020](https://arxiv.org/html/2605.30753#bib.bib19); Nichol and Dhariwal,[2021](https://arxiv.org/html/2605.30753#bib.bib18)\)\. Recent masked diffusion Language models\(Sahooet al\.,[2024](https://arxiv.org/html/2605.30753#bib.bib101); Heet al\.,[2023](https://arxiv.org/html/2605.30753#bib.bib23); Nieet al\.,[2024](https://arxiv.org/html/2605.30753#bib.bib22)\)and LLaDA style models\(Nieet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib26)\)show that diffusion can be competitive at scale\. Despite this parallelism, inference latency remains high in practice\. Each denoising step typically requires a full sequence forward pass under bidirectional attention\(Seoet al\.,[2016](https://arxiv.org/html/2605.30753#bib.bib14); Wibisono and Wang,[2023](https://arxiv.org/html/2605.30753#bib.bib13); Fenget al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib12)\), and many steps are spent refining tokens whose final values are already determined\. Repeated remasking further amplifies this cost when uncertainty persists at a small subset of positions\.

Prior work reduces this overhead by reusing computation across steps, adjusting unmasking schedules, or committing more tokens per step\(Wuet al\.,[2025b](https://arxiv.org/html/2605.30753#bib.bib104),[a](https://arxiv.org/html/2605.30753#bib.bib110); Konget al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib111); Wanget al\.,[2025c](https://arxiv.org/html/2605.30753#bib.bib10); Israelet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib9)\)\. These methods improve throughput, but most commit and stopping decisions still depend on step local heuristics, fixed schedules, or global thresholds\(Liet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib118); Chenet al\.,[2025b](https://arxiv.org/html/2605.30753#bib.bib119),[a](https://arxiv.org/html/2605.30753#bib.bib120)\)\. Such signals are brittle because diffusion confidence is not uniformly calibrated across prompts, domains, and token positions\. Empirically, diffusion decoding exhibits structured temporal and spatial behaviors, including delayed stabilization, underconfident but correct tokens, and oscillatory confidence\(Huanget al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib121); Wanget al\.,[2025b](https://arxiv.org/html/2605.30753#bib.bib122)\)\. A single step snapshot or heuristic rule cannot separate these regimes\. As a result, local rules often pay for redundant denoising after convergence, or they commit prematurely and lose quality\.

In this work, we treat diffusion decoding as a*sequential control problem*rather than a sequence of independent threshold tests\. A key observation is that denoising produces token wise*trajectories*across steps, such as confidence and entropy traces, whose temporal patterns reveal both correctness and stability\. This shifts the question from “is the token confident at steptt” to “does the trajectory indicate convergence, and how will it change if we continue denoising\.” Answering this requires a temporal\-spatial\-aware controller that can interpret patterns such as steady growth, oscillation, and delayed stabilization, as well as a mechanism that can exploit predictable trends to avoid unnecessary steps\.

We proposeTemporal\-Spatial Parallel Decoding \(TSPD\), a temporal\-spatial\-aware framework that improves both efficiency and accuracy\. TSPD introduces a lightweight*temporal\-spatial controller*that predicts whether a position is ready to be fixed using sequential trace features \(confidence, entropy, and momentum over recent steps\) together with relative spatial position within a block\. A shared sequential perception model captures dynamics that step local classifiers and hand tuned thresholds often miss, improving robustness across prompts and positional effects\. To reduce waiting time when confidence is rising but not yet decisive, we further introduce a training\-free and plug\-inConfidence Extrapolation mechanism \(CE\)\. CE models confidence evolution as a simple state space process and forecasts near future confidence together with uncertainty\. A risk\-aware horizon rule limits look ahead based on left context progress and forecast uncertainty, so extrapolation is applied only when supported by reliable history\.

We evaluate our framework \(TSPD and CE\) on LLaDA\-8B\-Instruct\(Nieet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib26)\)across four benchmarks spanning arithmetic reasoning and code generation, including GSM8K\(Cobbeet al\.,[2021](https://arxiv.org/html/2605.30753#bib.bib114)\), MATH\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.30753#bib.bib115)\), HumanEval\(Chen,[2021](https://arxiv.org/html/2605.30753#bib.bib116)\), and MBPP\(Austinet al\.,[2021b](https://arxiv.org/html/2605.30753#bib.bib117)\)\. Across multiple dLLM backbones and generation settings, our method delivers a consistently stronger speed quality tradeoff than state of the art baselines\. On GSM8K with 256 token generation, it achieves5\.0×5\.0\\timesspeedup without cache, and compounds to11\.2×11\.2\\timeswhen combined with KV caching, showing that our controller level gains are complementary to system optimizations\. The advantage increases with longer generation: at 1024 tokens, our approach attains 64\.1 Tokens Per Second \(TPS\) and up to58\.3×58\.3\\timesspeedup, while keeping accuracy essentially unchanged and in some cases slightly improved\. In summary, our contributions are threefold:

- •We proposeTSPD, a temporal spatial aware decoding controller that uses temporal confidence dynamics and positional context to make robust token fixing decisions, mitigating the brittleness of step local heuristics\.
- •We introduce a training free and plug and playconfidence extrapolationmodule that forecasts near future confidence under a state space model with quantified uncertainty, and applies risk aware look ahead to complement diverse decoding controllers\.
- •We provide extensive evaluations and ablations on a wide range of dLLMs, task domains, and generation settings, demonstrating substantial speedups with near lossless accuracy and full compatibility with KV cache\.

## 2Related Work

### 2\.1Diffusion\-Based Large Language Models

Diffusion models\(Sohl\-Dicksteinet al\.,[2015](https://arxiv.org/html/2605.30753#bib.bib96); Song and Ermon,[2019](https://arxiv.org/html/2605.30753#bib.bib2)\)have been extended from continuous data to discrete sequences, enabling non\-autoregressive generation via iterative denoising\. Early discrete diffusion work established Markov formulations in categorical spaces, including multinomial diffusion\(Hoogeboomet al\.,[2021](https://arxiv.org/html/2605.30753#bib.bib97)\)and D3PM\(Austinet al\.,[2021a](https://arxiv.org/html/2605.30753#bib.bib98)\)\. Continuous\-time views\(Campbellet al\.,[2022](https://arxiv.org/html/2605.30753#bib.bib99)\)further unify discrete diffusion and support flexible sampling schedules\. SEDD\(Louet al\.,[2023](https://arxiv.org/html/2605.30753#bib.bib100)\)improves learning by modeling ratios of marginal probabilities with a score\-entropy objective, narrowing the gap to autoregressive language models \(LMs\)\.

Building on these foundations, masked dLLMs have become a practical paradigm for text generation\. MDLM\(Sahooet al\.,[2024](https://arxiv.org/html/2605.30753#bib.bib101)\)shows that a simplified masked\-diffusion recipe can substantially improve perplexity and sampling quality\. At scale, diffusion LLMs are developed either by adapting strong autoregressive backbones \(e\.g\., DiffuLLaMA\)\(Gonget al\.,[2024](https://arxiv.org/html/2605.30753#bib.bib102)\)or by training from scratch \(e\.g\., LLaDA\)\(Nieet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib26)\), achieving competitive instruction following and reasoning\. Dream 7B\(Yeet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib17)\)further advances open diffusion LLMs and highlights diffusion\-specific capabilities such as arbitrary\-order generation and infilling with tunable quality–speed trade\-offs\.

![Refer to caption](https://arxiv.org/html/2605.30753v1/x1.png)Figure 1:Same local confidence with different trace can lead to different decoding results\.
![Refer to caption](https://arxiv.org/html/2605.30753v1/x2.png)Figure 2:Tokens at more rightward spatial positions tend to stabilize later\.

### 2\.2Decoding Acceleration for Diffusion LLMs

Despite parallel updates, diffusion decoding can be latency\-heavy because each denoising step typically requires a full\-sequence forward pass under bidirectional attention\. Prior work improves efficiency via three main directions: \(i\) computation reuse and caching, \(ii\) better parallel commitment or sampling schedules, and \(iii\) plug\-in accelerators\.

On computation reuse, dLLM\-Cache\(Liuet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib103)\)caches static prompts and selectively updates stable tokens; Fast\-dLLM\(Wuet al\.,[2025b](https://arxiv.org/html/2605.30753#bib.bib104)\)introduces block\-wise approximate KV caching with confidence\-aware parallel decoding; DInfer\(Maet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib109)\)modularizes the inference pipeline and reports∼\\sim10×\\timesspeedup over Fast\-dLLM without quality loss; Fast\-dLLM v2\(Wuet al\.,[2025a](https://arxiv.org/html/2605.30753#bib.bib110)\)further supports block\-diffusion adaptation of pretrained autoregressive backbones and hierarchical KV caching, achieving up to2\.5×2\.5\\timesspeedup over standard autoregressive decoding\. On algorithmic decoding, SlowFast Sampling\(Weiet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib106)\)alternates conservative exploration and accelerated decoding; Local Leap\(Konget al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib111)\)exploits local determinism around high\-confidence anchors; Learn2PD\(Baoet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib107)\)learns a parallel decoding policy with strong speedups on LLaDA; and adaptive parallel decoding\(Israelet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib9)\)dynamically controls parallelism and uses a small autoregressive verifier for a controllable speed–quality trade\-off\. Training\-free plug\-ins include FlashDLM\(Huet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib105)\)\(FreeCache \+ guided diffusion\), ES\-dLLM\(Anonymous,[2026](https://arxiv.org/html/2605.30753#bib.bib112)\)\(early skipping via convergence signals\), and foreseeing movement decoding\(Moet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib113)\)\.

Most existing methods still make decisions from per\-step heuristics or fixed schedules and respond to uncertainty by passively running more denoising steps\. In contrast, we emphasize that diffusion decoding exposes informative*token\-wise trajectories*\(confidence/probability traces\) that enable more input\-adaptive control\. Accordingly, we develop \(i\) a temporal–spatial correctness sensor that fuses history features with positional cues to infer stabilization, and \(ii\) an*active forecasting*module that extrapolates near\-future confidences with calibrated uncertainty \(e\.g\., via state\-space/Kalman\-style models\(Kalman,[1960](https://arxiv.org/html/2605.30753#bib.bib108)\)\) for risk\-limited look\-ahead decisions\. This trajectory\-aware perspective complements caching and sampler optimizations and targets finer\-grained speed–quality control for diffusion LLM inference\.

![Refer to caption](https://arxiv.org/html/2605.30753v1/x3.png)Figure 3:Missed acceleration opportunity of passive waiting compared to lookahead\.
![Refer to caption](https://arxiv.org/html/2605.30753v1/x4.png)Figure 4:Distribution of consistent steps ratio that can be extrapolated\.

![Refer to caption](https://arxiv.org/html/2605.30753v1/x5.png)Figure 5:Comparison between standard dLLM parallel decoding \(left\) and our framework \(right\)\. Standard decoding applies step\-local heuristics after each full\-sequence denoising pass, often revisiting already\-correct tokens\. Our framework inserts a risk\-aware confidence extrapolator \(CE\) and a temporal\-spatial decoding controller \(TSPD\): CE provides uncertainty\-aware look\-ahead confidence, and TSPD uses token\-wise confidence traces and position context to fix stable tokens\. This reduces redundant steps while preserving quality\.

## 3Methods

### 3\.1Preliminary

#### 3\.1\.1Parallel Decoding of Diffusion LLMs

Diffusion LLMs \(dLLMs\) generate a length\-LLsequencex0∈\{0,1,…,V−1\}Lx\_\{0\}\\in\\\{0,1,\\ldots,V\-1\\\}^\{L\}by iteratively denoising a masked version of the sequence\. Letα∈\[0,1\]\\alpha\\in\[0,1\]denote the masking ratio \(noise level\) to avoid overloadingttas “time\.” The forward process independently replaces each token with the mask tokenmmwith probabilityα\\alpha:

qα​\(xα∣x\)=∏i=1L\[\(1−α\)​𝟏​\[xiα=xi\]\+α​1​\[xiα=m\]\]\.q\_\{\\alpha\}\(x^\{\\alpha\}\\\!\\mid x\)=\\prod\_\{i=1\}^\{L\}\\big\[\(1\-\\alpha\)\\mathbf\{1\}\[x\_\{i\}^\{\\alpha\}=x\_\{i\}\]\+\\alpha\\,\\mathbf\{1\}\[x\_\{i\}^\{\\alpha\}=m\]\\big\]\.\(1\)wherexαx\_\{\\alpha\}is the corrupted sequence at masking ratioα\\alpha\. The reverse process denoises inKKdiscrete steps by repeatedly predicting a token distribution for masked positions\. Given a promptc=\(c1,…,cM\)c=\(c\_\{1\},\\ldots,c\_\{M\}\)and current masked responsey\(k\)y^\{\(k\)\}, a mask predictorpθp\_\{\\theta\}outputs per\-position distributions and we form a provisional sequence by greedy decoding:

y^i\(k\)=argmaxv∈\[V\]pθ\(v\|c,y\(k\)\),i∈\[L\]\.\\hat\{y\}^\{\(k\)\}\_\{i\}=\\arg\\max\_\{v\\in\[V\]\}p\_\{\\theta\}\\\!\\left\(v\\,\\middle\|\\,c,y^\{\(k\)\}\\right\),\\quad i\\in\[L\]\.\(2\)and remask thenkn\_\{k\}positions with the smallestcic\_\{i\}, wherenkn\_\{k\}\(equivalently the masking ratio\) follows a predefined schedule that decreases over steps\. This parallel denoising–remasking loop yields the final response afterKKiterations\.

#### 3\.1\.2Early Stopping for Parallel Decoding

Diffusion\-based LLMs \(dLLMs\) generate text by iteratively denoising a masked sequence\. Although additional steps can refine quality, many tokens reach their final values well before the last denoising step, so continuing refinement wastes compute and increases latency\. Early stopping accelerates inference by committing tokens once further updates are unlikely to change the final outcome\.

Existing early\-stopping methods are dominated by*heuristic or step\-local*controllers\. Heuristic rules \(e\.g\., Fast\-dLLM\(Wuet al\.,[2025b](https://arxiv.org/html/2605.30753#bib.bib104)\), Prophet\(Liet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib118)\), Credit Decoding\(Wanget al\.,[2025a](https://arxiv.org/html/2605.30753#bib.bib3)\)apply global thresholds on current\-step statistics such as top\-1 confidence or entropy, while learned controllers \(e\.g\., Learn2PD\(Baoet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib107)\)\) train a lightweight predictor but still make thresholded accept/reject decisions at inference\. We show that this step\-local, uniformly\-applied design is unreliable due to the temporal and spatial structure of diffusion trajectories, and that passive waiting misses forecastable acceleration opportunities\.

Step\-local and Spatial\-uniform Control Is Unreliable\.The same current\-step confidence can correspond to very different future outcomes, and the ambiguity varies across token positions\. We validate this on a held\-out C4\(Raffelet al\.,[2020](https://arxiv.org/html/2605.30753#bib.bib39)\)subset by extracting token\-states from intermediate denoising steps\. For positioniiat stepkk, letpi\(k\)∈ΔVp\_\{i\}^\{\(k\)\}\\in\\Delta^\{V\}be the predictive distribution; we group token\-states with nearly indistinguishable step\-local distributions \(matched by entropy and tight Jensen–Shannon divergence\) but different future trajectories\. We define the flip probabilityPflip​\(k\)=Pr⁡\[y^0,i\(K\)≠y^0,i\(k\)\]P\_\{\\text\{flip\}\}\(k\)=\\Pr\[\\hat\{y\}\_\{0,i\}^\{\(K\)\}\\neq\\hat\{y\}\_\{0,i\}^\{\(k\)\}\]and observe that, within the same confidence bin, unstable traces have multiple\-×\\timeshigher flip rates than stable ones \(Fig\.[2](https://arxiv.org/html/2605.30753#S2.F2)\)\. Moreover, stabilization is position\-dependent: tokens at more rightward positions stabilize later \(Fig\.[2](https://arxiv.org/html/2605.30753#S2.F2)\), so no single global threshold \(heuristic or learned\) can be well\-calibrated for all positions simultaneously\.

Heuristic Controllers Are Fragile\.Most prior methods ultimately threshold a step\-local score with a fixedτ\\tau\. This includes heuristic rules and learned filters such as Learn2PD\(Baoet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib107)\), which replace hand\-crafted scores with a learned scorefθ​\(⋅\)f\_\{\\theta\}\(\\cdot\)but still decide “accept” iffθ​\(⋅\)\>τf\_\{\\theta\}\(\\cdot\)\>\\tau\. When the score–stability relationship shifts across steps or positions, a staticτ\\taubecomes either overly conservative \(hurting speed\) or overly aggressive \(hurting quality\), making tuning sensitive and workload\-dependent\.

Passive Waiting Misses forecastable Acceleration\.When uncertain, existing controllers typically wait for additional denoising steps until a threshold is crossed\. However, once a token enters a trend\-consistent regime, its future confidence is often predictable from a short prefix\. Fig\.[4](https://arxiv.org/html/2605.30753#S2.F4)shows a case where the winning token follows a stable upward trend long before reaching a standard confidence threshold, suggesting that a look\-ahead mechanism can commit earlier\. Fig\.[4](https://arxiv.org/html/2605.30753#S2.F4)further indicates that many tokens exhibit trend\-consistent portions, leaving substantial headroom for saving steps\.

These findings motivate two principles: \(i\) early stopping should be*temporal\- and position\-aware*, leveraging token\-wise trajectories and spatial heterogeneity rather than step\-local uniform thresholds; and \(ii\) it should be*proactive*, using predictable dynamics with calibrated uncertainty to anticipate stabilization instead of passively waiting\. As shown in Fig\.[5](https://arxiv.org/html/2605.30753#S2.F5), we instantiate these principles with \(i\) TSPD, a trajectory\-based controller that models token history and relative position, and \(ii\) a training\-free confidence extrapolation module that forecasts near\-future confidences with uncertainty to enable risk\-limited look\-ahead decisions\.

### 3\.2Temporal\-Spatial Parallel Decoding \(TSPD\)

In practice, token\-wise uncertainty evolves with diverse trajectories \(e\.g\., monotone convergence, oscillation, or delayed stabilization\), and these patterns are not reliably inferred from a single\-step snapshot\. We therefore develop*Temporal\-Spatial Parallel Decoding \(TSPD\)*, a lightweight learnable controller that tracks per\-token temporal traces and spatial position within each decoding block, and outputs a binary*fix\-or\-continue*decision for every token at every diffusion step\.

Temporal\-spatial Decoding Signals\.Letx\(t\)=\(x1\(t\),…,xL\(t\)\)x^\{\(t\)\}=\(x^\{\(t\)\}\_\{1\},\\ldots,x^\{\(t\)\}\_\{L\}\)denote the intermediate sequence at diffusion steptt\(withttdecreasing fromTTto0\), and letpθ\(⋅∣c,x\(t\)\)p\_\{\\theta\}\(\\cdot\\mid c,x^\{\(t\)\}\)be the dLLM token distribution\. For each positionii, we take the top\-1 candidatex^i\(t\)=arg⁡maxv⁡pi\(t\)​\(v\)\\hat\{x\}^\{\(t\)\}\_\{i\}=\\arg\\max\_\{v\}p^\{\(t\)\}\_\{i\}\(v\)and its confidencepi\(t\)=maxv⁡pi\(t\)​\(v\)p^\{\(t\)\}\_\{i\}=\\max\_\{v\}p^\{\(t\)\}\_\{i\}\(v\), and measure uncertainty by the predictive entropyHi\(t\)=−∑vpi\(t\)​\(v\)​log⁡pi\(t\)​\(v\)H^\{\(t\)\}\_\{i\}=\-\\sum\_\{v\}p^\{\(t\)\}\_\{i\}\(v\)\\log p^\{\(t\)\}\_\{i\}\(v\)\. To reduce step\-wise noise, we track an exponential moving average confidencep¯i\(t\)=α​pi\(t\)\+\(1−α\)​p¯i\(t\+1\)\\bar\{p\}^\{\(t\)\}\_\{i\}=\\alpha p^\{\(t\)\}\_\{i\}\+\(1\-\\alpha\)\\bar\{p\}^\{\(t\+1\)\}\_\{i\}and use its one\-step difference as a momentum featureΔ​p¯i\(t\)=p¯i\(t\)−p¯i\(t\+1\)\\Delta\\bar\{p\}^\{\(t\)\}\_\{i\}=\\bar\{p\}^\{\(t\)\}\_\{i\}\-\\bar\{p\}^\{\(t\+1\)\}\_\{i\}\. We also include the token’s relative position within the block,ϕ​\(i\)=i/\(L−1\)\\phi\(i\)=i/\(L\-1\)\. Thus, at each stepttTSPD constructs the per\-token feature vector as:

ri\(t\)≜\[pi\(t\),Hi\(t\),Δ​p¯i\(t\),ϕ​\(i\)\],r^\{\(t\)\}\_\{i\}\\triangleq\\big\[p^\{\(t\)\}\_\{i\},\\;H^\{\(t\)\}\_\{i\},\\;\\Delta\\bar\{p\}^\{\(t\)\}\_\{i\},\\;\\phi\(i\)\\big\],\(3\)which captures instantaneous evidence \(pi\(t\)p^\{\(t\)\}\_\{i\},Hi\(t\)H^\{\(t\)\}\_\{i\}\), temporal trend \(Δ​p¯i\(t\)\\Delta\\bar\{p\}^\{\(t\)\}\_\{i\}\), and spatial context \(ϕ​\(i\)\\phi\(i\)\)\.

Sequential binary controller\.TSPD employs a shared recurrent controller to maintain a hidden state for each token position, enabling memory over the denoising trajectory\. Specifically, for each tokeniiwe update

hi\(t\)=fψ​\(hi\(t\+1\),ri\(t\)\),h^\{\(t\)\}\_\{i\}=f\_\{\\psi\}\\\!\\left\(h^\{\(t\+1\)\}\_\{i\},\\;r^\{\(t\)\}\_\{i\}\\right\),\(4\)wherefψ​\(⋅\)f\_\{\\psi\}\(\\cdot\)is a lightweight recurrent update \(e\.g\., a GRU\(Choet al\.,[2014](https://arxiv.org/html/2605.30753#bib.bib6)\)/LSTM\(Hochreiter and Schmidhuber,[1997](https://arxiv.org/html/2605.30753#bib.bib7)\)cell\) with parametersψ\\psi\. A linear head produces a gate logit

zi\(t\)=W​hi\(t\)\+b,z^\{\(t\)\}\_\{i\}=Wh^\{\(t\)\}\_\{i\}\+b,\(5\)which is converted to a soft fixing propensityπi\(t\)=σ​\(zi\(t\)\)\\pi^\{\(t\)\}\_\{i\}=\\sigma\(z^\{\(t\)\}\_\{i\}\)\. Unlike threshold\-tuned score predictors, TSPD executes a*direct binary action*by hard gating:

ai\(t\)∈\{0,1\},ai\(t\)=𝕀​\[πi\(t\)≥0\.5\],a^\{\(t\)\}\_\{i\}\\in\\\{0,1\\\},\\qquad a^\{\(t\)\}\_\{i\}=\\mathbb\{I\}\\big\[\\pi^\{\(t\)\}\_\{i\}\\geq 0\.5\\big\],\(6\)whereai\(t\)=1a^\{\(t\)\}\_\{i\}=1indicates that tokeniiis*fixed*\(committed and excluded from further denoising\) andai\(t\)=0a^\{\(t\)\}\_\{i\}=0keeps it active\.

Training by Oracle Imitation with Discrete Decisions\.We supervise TSPD using an oracle fixing policy that marks a token as fixable once its candidate token matches the final target at that step\. Formally, letyi\(t\)∈\{0,1\}y^\{\(t\)\}\_\{i\}\\in\\\{0,1\\\}denote the oracle*fix*label for tokeniiat steptt\. We train the controller via binary cross\-entropy on logits:

ℒgate=1L​T​∑t=0T∑i=1LBCEWithLogits​\(zi\(t\),yi\(t\)\)\.\\mathcal\{L\}\_\{\\text\{gate\}\}=\\frac\{1\}\{LT\}\\sum\_\{t=0\}^\{T\}\\sum\_\{i=1\}^\{L\}\\mathrm\{BCEWithLogits\}\\\!\\left\(z^\{\(t\)\}\_\{i\},\\;y^\{\(t\)\}\_\{i\}\\right\)\.\(7\)To align training with inference\-time hard gating, we use a straight\-through estimator \(STE\): the forward pass applies the hard decisionai\(t\)a^\{\(t\)\}\_\{i\}to simulate fixing, while gradients are backpropagated through the continuousπi\(t\)\\pi^\{\(t\)\}\_\{i\}\(equivalently, throughzi\(t\)z^\{\(t\)\}\_\{i\}in \([7](https://arxiv.org/html/2605.30753#S3.E7)\)\)\. By jointly leveraging temporal traces and spatial position, TSPD provides robust parallel decoding control with negligible overhead\.

Algorithm 1Inference Workflow0:dLLM

ℳ\\mathcal\{M\}with

KKsteps; prompt

xx; generation length

LL; controllerTSPD; extrapolatorCE\.

1:

X←concat​\(x,\[MASK\]L\)X\\leftarrow\\mathrm\{concat\}\(x,\\,\[\\texttt\{MASK\}\]^\{L\}\);

F←𝟎F\\leftarrow\\mathbf\{0\}\{

Fj=1F\_\{j\}\{=\}1means token

jjis fixed\}

2:Initialize per\-token trace buffer

ℋ\\mathcal\{H\}
3:for

t=K,…,1t=K,\\ldots,1do

4:

p​\(⋅\)←ℳ​\(X\)p\(\\cdot\)\\leftarrow\\mathcal\{M\}\(X\)\{one dLLM forward pass\}

5:Update trace history

ℋ←update​\(ℋ,p\)\\mathcal\{H\}\\leftarrow\\text\{update\}\(\\mathcal\{H\},p\)
6:

r^←CE​\(ℋ,F\)\\hat\{r\}\\leftarrow\\textsc\{CE\}\(\\mathcal\{H\},F\)
7:

A←TSPD​\(r^,pos,F\)A\\leftarrow\\textsc\{TSPD\}\(\\hat\{r\},\\text\{pos\},F\)\{binary fix decisions\}

8:

X←commit​\(X,p,A\)X\\leftarrow\\text\{commit\}\(X,p,A\);

F←F∨AF\\leftarrow F\\lor A
9:if

∑F=L\\sum F=Lthen

10:break

11:endif

12:

X←remask​\(X,p,F\)X\\leftarrow\\text\{remask\}\(X,p,F\)
13:endfor

14:return

resp​\(X\)\\text\{resp\}\(X\)

### 3\.3Confidence Extrapolation

Most existing decoding accelerations are passive: when confidence is low, the model simply runs more denoising steps until the distribution sharpens\. We instead propose*Confidence Extrapolation*\(CE\), a training\-free, plug\-in module that*forecasts near\-future confidence*from the observed confidence trace and feeds a calibrated, risk\-aware confidence signal to any downstream decoding controller \(e\.g\., TSPD, Learn2PD, or threshold rules\)\. CE does not modify the base diffusion model parameters and can be combined with existing controllers without changing their decision logic\.

State\-space Extrapolator on Confidence\.For each positionjjin a decoding block, letct​\(j\)∈\[0,1\]c\_\{t\}\(j\)\\in\[0,1\]denote the confidence at diffusion steptt\(e\.g\., top\-1 probability or a margin\-based confidence\)\. To obtain a stable time series, we apply a monotone transformyt​\(j\)=ϕ​\(ct​\(j\)\)y\_\{t\}\(j\)=\\phi\(c\_\{t\}\(j\)\)\(e\.g\., log\-odds\)\. We modelyt​\(j\)y\_\{t\}\(j\)as a noisy observation of a latent confidence level:

yt​\(j\)=ℓt​\(j\)\+nt,nt∼𝒩​\(0,Rt​\(j\)\)\.y\_\{t\}\(j\)=\\ell\_\{t\}\(j\)\+n\_\{t\},\\quad n\_\{t\}\\sim\\mathcal\{N\}\(0,R\_\{t\}\(j\)\)\.\(8\)CE maintains a level–velocity latent state

xt​\(j\)=\[ℓt​\(j\),ℓ˙t​\(j\)\]⊤,xt\+1​\(j\)=A​xt​\(j\)\+wt,x\_\{t\}\(j\)=\\big\[\\ell\_\{t\}\(j\),\\,\\dot\{\\ell\}\_\{t\}\(j\)\\big\]^\{\\top\},\\quad x\_\{t\+1\}\(j\)=Ax\_\{t\}\(j\)\+w\_\{t\},\(9\)
whereQQdetermines how rapidly the underlying trend is allowed to change, andAAis a fixed two\-by\-two transition matrix with ones on the diagonal and the upper\-right entry, and zeros elsewhere\. Given the filtered estimate\(x^t,Pt\)\(\\hat\{x\}\_\{t\},P\_\{t\}\), thehh\-step forecast is

x^t\+h\|t=Ah​x^t,Pt\+h\|t=Ah​Pt​\(Ah\)⊤\+Qh,\\hat\{x\}\_\{t\+h\|t\}=A^\{h\}\\hat\{x\}\_\{t\},\\qquad P\_\{t\+h\|t\}=A^\{h\}P\_\{t\}\(A^\{h\}\)^\{\\top\}\+Q\_\{h\},\(10\)whereQh≜∑i=0h−1Ai​Q​\(Ai\)⊤Q\_\{h\}\\triangleq\\sum\_\{i=0\}^\{h\-1\}A^\{i\}Q\(A^\{i\}\)^\{\\top\}is the accumulated process noise overhhsteps\. And the forecast variance of the confidence level isσt\+h2​\(j\)=\(Pt\+h\|t​\(j\)\)1,1\\sigma^\{2\}\_\{t\+h\}\(j\)=\\big\(P\_\{t\+h\|t\}\(j\)\\big\)\_\{1,1\}\. We map forecasts back to confidence byc^t\+h​\(j\)=ϕ−1​\(ℓ^t\+h​\(j\)\)\\hat\{c\}\_\{t\+h\}\(j\)=\\phi^\{\-1\}\(\\hat\{\\ell\}\_\{t\+h\}\(j\)\)\.

Risk\-aware Horizon Selection\.CE selects the*furthest*look\-ahead horizon that is both \(i\) supported by a reliable history window and \(ii\) safe under extrapolation uncertainty\. Although denoising within a block is parallel, later positions become reliable only after sufficient left context is fixed\. We use\[x\]01≜min⁡\{1,max⁡\{0,x\}\}\[x\]\_\{0\}^\{1\}\\triangleq\\min\\\{1,\\max\\\{0,x\\\}\\\}to clampxxinto\[0,1\]\[0,1\]\. Letleft\_coveraget​\(j\)∈\[0,1\]\\textit\{left\\\_coverage\}\_\{t\}\(j\)\\in\[0,1\]be the fraction of fixed positions left ofjjat steptt, and define an extrapolation potential

rt​\(j\)=\[left\_cvgt​\(j\)−τ1−τ\]01,hpot=H​rt​\(j\)\.r\_\{t\}\(j\)=\\left\[\\frac\{\\textit\{left\\\_cvg\}\_\{t\}\(j\)\-\\tau\}\{1\-\\tau\}\\right\]\_\{0\}^\{1\},\\quad h\_\{\\text\{pot\}\}=Hr\_\{t\}\(j\)\.\(11\)whereτ\\tauis a coverage threshold andHHis the maximum look\-ahead\. For a candidate horizonh≤hpoth\\leq h\_\{\\mathrm\{pot\}\}, we form a conservative lower bound in the transformed space:

LBt\+h≜ℓ^t\+h−z​σt\+h,c^t\+hLB=ϕ−1​\(LBt\+h\)\.\\mathrm\{LB\}\_\{t\+h\}\\triangleq\\hat\{\\ell\}\_\{t\+h\}\-z\\,\\sigma\_\{t\+h\},\\qquad\\hat\{c\}^\{\\mathrm\{LB\}\}\_\{t\+h\}=\\phi^\{\-1\}\\\!\\big\(\\mathrm\{LB\}\_\{t\+h\}\\big\)\.\(12\)wherezzcontrols risk tolerance\. For brevity, we writeℓ^t\+h≡ℓ^t\+h∣t\\hat\{\\ell\}\_\{t\+h\}\\equiv\\hat\{\\ell\}\_\{t\+h\\mid t\}and omit the position indexjjwhen clear\. We choose

h\(t,j\)=max⁡\{h≤hpot​\(t,j\):c^t\+hLB​\(j\)≥cmin\}\.h^\{\(t,j\)\}=\\max\\\{\\,h\\leq h\_\{\\mathrm\{pot\}\}\(t,j\):\\ \\hat\{c\}^\{\\mathrm\{LB\}\}\_\{t\+h\}\(j\)\\geq c\_\{\\min\}\\,\\\}\.\(13\)and fall back toh\(t,j\)=0h^\{\(t,j\)\}=0if no horizon passes\. The controller then consumesc~t​\(j\)=c^t\+h\(t,j\)LB​\(j\)\\tilde\{c\}\_\{t\(j\)\}=\\hat\{c\}^\{\\mathrm\{LB\}\}\_\{t\+h^\{\(t,j\)\}\}\(j\)in place of the rawct​\(j\)c\_\{t\(j\)\}\.

Integration with Decoding Controllers\.CE exposes a minimal interface that outputsc~t​\(j\)\\tilde\{c\}\_\{t\(j\)\}\(and optionallyh\(t,j\)h^\{\(t,j\)\}\) per step and position, making it directly compatible with threshold\-based or learned controllers\. As shown in Alg\.[1](https://arxiv.org/html/2605.30753#alg1), we simply replace the controller’s per\-step confidence feature withc~t​\(j\)\\tilde\{c\}\_\{t\}\(j\), enabling earlier commitment when the forecast indicates a reliably increasing confidence trajectory\.

Table 1:Benchmark results on the LLaDA\-8B\-Instruct suite with a generation length of 256\. Performance is measured using TPS, speedup, and accuracy \(†\\daggerdenotes without KV cache\)\.TaskMethodsInference EfficiencyAcc\.TPS↑\\uparrowSpeed↑\\uparrowGSM8K\(5\-shot\)Vanilla6\.91\.0×\\times79\.3F\-dLLM53\.87\.8×\\times78\.5CD54\.57\.9×\\times78\.7Prophet†11\.01\.6×\\times79\.4Learn2PD†29\.04\.2×\\times79\.1Ours†34\.25\.0×\\times79\.4Ours77\.311\.2×\\times78\.8MATH\(4\-shot\)Vanilla9\.21\.0×\\times33\.5F\-dLLM51\.55\.6×\\times33\.2CD50\.65\.5×\\times33\.1Prophet†15\.71\.7×\\times32\.7Learn2PD†29\.43\.2×\\times32\.3Ours†36\.84\.0×\\times33\.0Ours76\.48\.3×\\times33\.2HumanEval\(0\-shot\)Vanilla6\.51\.0×\\times41\.5F\-dLLM24\.13\.7×\\times43\.3CD24\.73\.8×\\times43\.7Prophet†7\.81\.2×\\times41\.5Learn2PD†23\.43\.6×\\times40\.3Ours†27\.34\.2×\\times42\.5Ours53\.38\.2×\\times43\.0MBPP\(3\-shot\)Vanilla5\.81\.0×\\times29\.4F\-dLLM41\.87\.2×\\times28\.2CD40\.67\.0×\\times29\.3Prophet†8\.11\.4×\\times29\.2L2PD†29\.65\.1×\\times29\.2Ours†32\.55\.6×\\times29\.5Ours68\.411\.8×\\times29\.2Table 2:Benchmark results on the Dream\-7B and LLaDA\-MoE suite with a generation length of 256\. Performance is measured using TPS \(tokens/sec\), speedup, and accuracy \(†\\daggerdenotes without KV cache\)\.TaskMethodsTPS↑\\uparrowSpeed↑\\uparrowAcc\.Dream\-7BVanilla9\.11\.0×\\times75\.0F\-dLLM48\.25\.3×\\times74\.8CD42\.84\.7×\\times74\.5Prophet†15\.11\.7×\\times74\.9Learn2PD†19\.12\.1×\\times73\.4Ours†28\.23\.1×\\times75\.3Ours69\.27\.6×\\times75\.0LLaDA\-MoEVanilla4\.21\.0×\\times75\.2F\-dLLM9\.72\.3×\\times75\.3CD10\.52\.5×\\times75\.2Prophet†6\.71\.6×\\times75\.0Learn2PD†13\.03\.1×\\times75\.3Ours†13\.83\.3×\\times75\.4Ours21\.25\.1×\\times75\.2

## 4Experiments

### 4\.1Experimental Settings

Models and Datasets\.We mainly implement our method on the representative dLLM: LLaDA\-8B\-Instruct\(Nieet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib26)\)to measure the acceleration of the inference process across various benchmarks\. To ensure the broad applicability of the method, we conduct experiments on four datasets covering three different types of problems, which are GSM8K\(Cobbeet al\.,[2021](https://arxiv.org/html/2605.30753#bib.bib114)\), MATH\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.30753#bib.bib115)\), HumanEval\(Chen,[2021](https://arxiv.org/html/2605.30753#bib.bib116)\), and MBPP\(Austinet al\.,[2021b](https://arxiv.org/html/2605.30753#bib.bib117)\)\. To demonstrate the generalizability of our method across diverse dLLM architectures, we conduct additional comparisons with existing methods on Dream\-7B\(Yeet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib17)\)and LLaDA\-MoE\(Zhuet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib8)\), further confirming the effectiveness of our approach\. In this paper, we primarily compare our method with state\-of\-the\-art dLLM acceleration approaches, including Fast\-dLLM\(Wuet al\.,[2025b](https://arxiv.org/html/2605.30753#bib.bib104)\), Credit Decoding \(CD\)\(Wanget al\.,[2025a](https://arxiv.org/html/2605.30753#bib.bib3)\), Prophet\(Liet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib118)\), and Learn2PD\(Baoet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib107)\)\. Among these methods, Fast\-dLLM and Credit Decoding leverage Key\-Value \(KV\) caching, whereas Learn2PD and Prophet do not utilize KV cache mechanisms\.

TSPD Training\.To train the TSPD model that can be applied to a wide range of tasks, we select 40 samples from each of the 66 types of questions in the FLAN dataset \(out of the used evaluation benchmarks\), resulting in a total of 2,640 samples for training\. In this experiment, we adopt the simplest two\-layer LSTM as our filter model\. Since the dLLMs remain frozen and onlyfθf\_\{\\theta\}is trained, the number of trainable parameters is extremely limited\. For example, for an LLaDA with a block size of 32, the total number of trainable parameters is only 2,112\. We train thefθf\_\{\\theta\}for 5,000 epochs until the model converged\. The learning rate is set to 0\.001, and the AdamW optimizer is used to optimizefθf\_\{\\theta\}\.

Our training process consists of two stages\. In the first stage, samples are collected by following an Extremely Greedy Parallel\(Baoet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib107)\)policy, recording the confidence scores and token selections at each step during parallel decoding\. This data is then used in the second stage to train the TSPD modelfθf\_\{\\theta\}\. The data collection in the first stage is conducted on 2 NVIDIA RTX A100 GPUs and takes approximately two hours\. The subsequent training of the TSPD model in the second stage is deployed on the same GPU and required only 6 minutes\. The details of training are provided in Appendix[A\.6](https://arxiv.org/html/2605.30753#A1.SS6)\.

Evaluation\.We evaluate the inference acceleration and generation quality of TSPD and CE methods using quantitative metrics\. The inference speed is quantified with Tokens Per Second \(TPS\), indicating the average number of tokens generated per second\. The generation quality is measured using task\-specific metrics, such as accuracy for GSM8K, reflecting the model’s performance under acceleration\. In addition, we set the generation length to 256 and 1024, and the block size to 32\.

Table 3:Benchmark results on the LLaDA\-8B\-Instruct suite with a generation length of 512 and 1024\.†\\daggerdenotes without KV cache\.Gen Len\.MethodsTPS↑\\uparrowSpeed↑\\uparrowAcc\.512Vanilla3\.21\.0×\\times77\.5Fast\-dLLM35\.311\.0×\\times77\.2Learn2PD†39\.412\.3×\\times77\.8Ours†45\.414\.2×\\times77\.9Ours79\.024\.7×\\times77\.61024Vanilla1\.11\.0×\\times77\.0Fast\-dLLM21\.619\.6×\\times74\.7Learn2PD†24\.922\.6×\\times78\.7Ours†27\.324\.8×\\times79\.3Ours64\.158\.3×\\times78\.5
### 4\.2Main Results

Overall Performance\.Table[1](https://arxiv.org/html/2605.30753#S3.T1)reports results on the LLaDA\-8B\-Instruct suite with generation length 256\. Across GSM8K, MATH, HumanEval, and MBPP, our method consistently achieves the highest throughput and speedup while preserving accuracy close to Vanilla\. With KV cache, our method reaches77\.3TPS /11\.2×\\timeson GSM8K,76\.4/8\.3×\\timeson MATH,53\.3/8\.2×\\timeson HumanEval, and68\.4/11\.8×\\timeson MBPP\. Without cache, our method still outperforms Prophet and Learn2PD with same settings on all tasks \(e\.g\., 34\.2 vs\. 29\.0 TPS on GSM8K, 36\.8 vs\. 29\.4 on MATH\), demonstrating that our gains do not rely on additional caching\.

Performance over Different dLLMs\.Table[2](https://arxiv.org/html/2605.30753#S3.T2)shows that the improvements transfer across backbones\. On Dream\-7B, our method achieves69\.2TPS /7\.6×\\timeswith accuracy matching Vanilla \(75\.0%\), and our method \(without KV cache\) remains competitive \(28\.2 TPS / 3\.1×\\times\)\. On LLaDA\-MoE, our method improves throughput to21\.2TPS /5\.1×\\timeswith essentially unchanged accuracy \(75\.2%\), indicating that CE and the controller are not model\-specific and generalize to MoE architectures\.

Table 4:A comparison of our method with and without KV Cache\. The results show a significant performance improvement when augmented with both Dual and Prefix Caches, underscoring that our method is orthogonal to and fully compatible with existing KV caching strategies\.MethodsTPS↑\\uparrowSpeed↑\\uparrowAcc\.TSPD & CE34\.25\.0×\\times79\.4\+ Dual Cache77\.311\.2×\\times78\.8\+ Prefix Cache37\.35\.4×\\times79\.3Table 5:Ablation of the effectiveness of TSPD and CE modules using LLaDA\-8B\-Instruct model on GSM8K benchmark at generation length of 256 tokens\.†\\daggerdenotes without KV cache\.MethodsTPS↑\\uparrowSpeed↑\\uparrowAcc\.Ours†34\.25\.0×\\times79\.4w\.o\. TSPD20\.02\.9×\\times79\.1w\.o\. CE30\.34\.4×\\times79\.5Table 6:A comparison of the acceleration performance using filter models of varying complexity\. The results indicate that a two\-layer LSTM model achieves the optimal balance by providing significant speedup and maintaining high accuracy\.\# LayersTPS↑\\uparrowSpeed↑\\uparrowAcc\.1\-layer MLP29\.74\.3×\\times78\.72\-layer MLP30\.44\.4×\\times78\.84\-layer MLP26\.83\.9×\\times79\.01\-layer LSTM29\.04\.2×\\times79\.22\-layer LSTM30\.34\.4×\\times79\.54\-layer LSTM28\.34\.1×\\times79\.5Performance over Different Generation Lengths\.Table[3](https://arxiv.org/html/2605.30753#S4.T3)evaluates longer generations \(512/1024\)\. Speedups increase with length: at 512 tokens, our method achieves79\.0TPS /24\.7×\\times; at 1024 tokens, it reaches64\.1TPS /58\.3×\\times, while maintaining accuracy comparable to Vanilla\. In contrast, Fast\-dLLM shows a larger accuracy drop at 1024 \(from 77\.0% to 74\.7%\)\. Overall, our method scales favorably with generation length, yielding larger end\-to\-end acceleration without sacrificing generation quality\.

### 4\.3Compatibility with Key\-Value Cache

We evaluate compatibility with KV caching\(Popeet al\.,[2023](https://arxiv.org/html/2605.30753#bib.bib5); Liuet al\.,[2024](https://arxiv.org/html/2605.30753#bib.bib4)\)by integrating Dual Cache\(Wuet al\.,[2025b](https://arxiv.org/html/2605.30753#bib.bib104)\)and Prefix Cache\(Wuet al\.,[2025b](https://arxiv.org/html/2605.30753#bib.bib104)\)on GSM8K \(256 tokens\)\. As shown in Table[4](https://arxiv.org/html/2605.30753#S4.T4), the baseline TSPD \+ CE achieves 34\.2 TPS \(5\.0×\\times\) with 79\.4% accuracy\. Adding Dual Cache boosts throughput to 77\.3 TPS \(11\.2×\\times\) with a modest accuracy drop \(78\.8%\), while Prefix Cache improves to 37\.3 TPS \(5\.4×\\times\) with 79\.3% accuracy\. Overall, our method is orthogonal to standard KV caching and can directly benefit from these techniques for further speedups\.

Table 7:A ablation of the effectiveness of the features used for TSPD using LLaDA\-8B\-Instruct model on GSM8K benchmark at generation length of 256 tokens\.MethodsTPS↑\\uparrowSpeed↑\\uparrowAcc\.TSPD30\.34\.4×\\times79\.5w\.o\. confidence19\.32\.8×\\times78\.8w\.o\. entropy28\.34\.1×\\times79\.0w\.o\. momentum29\.74\.3×\\times79\.2w\.o\. position31\.14\.5×\\times78\.5
### 4\.4Sensitivity Analysis

Parameter Budget on TSPD\.We vary the TSPD parameter budget within the same 2\-layer LSTM architecture by scaling the hidden size\. As shown in Table[8](https://arxiv.org/html/2605.30753#S4.T8), the current2Ksetting provides the best speed–accuracy tradeoff\. Reducing the budget to1Kslightly improves TPS \(30\.9→30\.330\.9\\rightarrow 30\.3\) but lowers accuracy \(79\.2→79\.579\.2\\rightarrow 79\.5\)\. Increasing the budget to4Kor8Kbrings at most marginal accuracy gain, while throughput drops more noticeably\. This suggests that the current controller is already near the sweet spot: smaller models underfit the temporal traces, whereas larger ones add complexity with little benefit\.

Table 8:Impact of parameter budget on TSPD performance under the same 2\-layer LSTM architecture\.Param budgetHidden sizeTPS↑\\uparrowAcc\.↑\\uparrow1K830\.979\.22K1230\.379\.54K1729\.879\.58K2526\.879\.7Training Sensitivity to Data Collection Policy\.We compare EGP\-collected, standard parallel\-decoding, and mixed training traces under the same test\-time setting\. As shown in Table[9](https://arxiv.org/html/2605.30753#S4.T9), EGP gives the best tradeoff \(30\.3 TPS,79\.5 Acc\), while standard traces are much slower \(7\.5 TPS\) with similar accuracy; the mixed policy is in between \(16\.1 TPS,79\.5 Acc\)\. This suggests that EGP does not cause harmful distribution shift, but instead provides more informative borderline states for learning when to safely fix tokens\.

Table 9:Training sensitivity of data collection policy\.EGPStandard PDMixedTPS↑\\uparrow30\.37\.516\.1Acc↑\\uparrow79\.579\.679\.5Sensitivity to Training Data Scale\.

We evaluate TSPD with different numbers of FLAN training prompts while keeping the controller architecture and decoding settings fixed\. As shown in Table[10](https://arxiv.org/html/2605.30753#S4.T10), TSPD is already close to convergence with1320samples: compared with the default2640\-sample setting, the accuracy gap is only0\.1, and further increasing the data to3960samples brings negligible gains\. This is expected because TSPD is a lightweight controller trained on compact trajectory features rather than full token logits\.

Table 10:Sensitivity to TSPD training data scale\.Training SamplesTPS↑\\uparrowAcc\.↑\\uparrowVanilla6\.979\.333027\.178\.866028\.779\.1132030\.079\.4264030\.379\.5396030\.279\.5Sensitivity to Training Data Distribution\.

We evaluate whether TSPD is sensitive to the difficulty distribution of training traces, measured by the average*stabilization\-step ratio*, \(i\.e\., the fraction of denoising steps where a token already matches its final prediction\)\. Lower ratios indicate harder traces with later stabilization, while higher ratios indicate easier traces with earlier stabilization\. As shown in Table[11](https://arxiv.org/html/2605.30753#S4.T11), TSPD is stable across different distributions, with variations within1\.1 TPSand0\.5 Acc\. Harder traces make the controller slightly more conservative and improve accuracy, while easier traces make it more aggressive and improve TPS\. Our default mixed distribution \(∼0\.45\\sim 0\.45\) gives the best overall tradeoff\.

Table 11:Sensitivity to training data distribution\.Stabilization RatioTPS↑\\uparrowAcc\.↑\\uparrowVanilla6\.979\.3∼0\.25\\sim 0\.2529\.679\.7∼0\.45\\sim 0\.4530\.379\.5∼0\.50\\sim 0\.5030\.179\.5∼0\.75\\sim 0\.7530\.779\.2Sensitivity to CE Hyper\-parameters\.We evaluate the two key CE hyper\-parameters, the coverage thresholdτ\\tauand the maximum look\-ahead horizonHH, while keeping all other settings fixed\. The masking ratioα\\alphabelongs to the base dLLM denoising schedule rather than our method, and is therefore not tuned here\. As shown in Table[12](https://arxiv.org/html/2605.30753#S4.T12), CE shows a stable speed–accuracy tradeoff: smallerτ\\tauor largerHHmakes CE more aggressive and improves TPS, while largerτ\\tauor smallerHHbetter preserves accuracy\. We useτ=0\.6\\tau=0\.6andH=20H=20by default\.

Table 12:Sensitivity to CE hyper\-parameters\.\(a\)Coverage thresholdτ\\tau\.τ\\tauTPS↑\\uparrowAcc\.↑\\uparrowVanilla6\.979\.30\.479\.278\.20\.577\.078\.70\.677\.378\.80\.775\.278\.80\.874\.978\.9
\(b\)Max horizonHH\.HHTPS↑\\uparrowAcc\.↑\\uparrowVanilla6\.979\.31070\.179\.01572\.478\.82077\.378\.82578\.078\.23078\.277\.9

### 4\.5Ablation Study

Effectiveness of TSPD and CE\.As shown in Tab\.[5](https://arxiv.org/html/2605.30753#S4.T5), on GSM8K with generation length 256, both TSPD and CE improve throughput with negligible accuracy change\. Vanilla decoding runs at 6\.9 TPS with 79\.3% accuracy\. Removing TSPD drops throughput to 20\.0 TPS \(2\.9×\\times\), indicating that trace\-aware token fixing provides the main speedup\. Removing CE yields 30\.3 TPS \(4\.4×\\times\), showing that confidence extrapolation brings additional gains beyond TSPD alone\. Combining both achieves the best throughput, 34\.2 TPS \(5\.0×\\times\), while maintaining comparable accuracy \(79\.4%\)\.

Complexity and Overhead Analysis\.Table[6](https://arxiv.org/html/2605.30753#S4.T6)shows that moderate\-capacity filters perform best\. While a two\-layer MLP achieves the highest efficiency among MLP variants \(30\.4 TPS, 4\.4× speedup\), a two\-layer LSTM strikes a more favorable balance, delivering substantial speedup while maintaining high accuracy \(30\.3 TPS, 4\.4× speedup, and 79\.5% accuracy\)\. Since deeper variants yield diminishing returns, we adopt the two\-layer LSTM as the default\. TSPD and CE operate only on lightweight confidence features \(per\-position scalars and short traces\), so their runtime overhead is negligible compared to a diffusion model forward\. Profiling on GSM8K \(256 tokens\) shows that the per\-step latency of TSPD and CE is only a small fraction of one base\-model forward by 0\.30% and 0\.13%, respecively\. Thus, the observed speedups primarily come from reducing redundant denoising steps rather than shifting cost to the controller\.

Ablation of the Features Used for TSPD\.Table[7](https://arxiv.org/html/2605.30753#S4.T7)ablates TSPD features, removing*confidence*causes the largest drop \(30\.3→\\rightarrow19\.3 TPS; 4\.4×⁣→\\times\\rightarrow2\.8×\\times\), confirming it as the key signal\. Entropy and momentum provide smaller but consistent gains \(w\.o\. entropy: 4\.1×\\times; w\.o\. momentum: 4\.3×\\times\)\. Removing*position*slightly improves speed \(31\.1 TPS, 4\.5×\\times\) but harms quality \(78\.5% accuracy\), indicating position mainly prevents premature fixing for later tokens\. Overall, confidence is essential; entropy/momentum are complementary; position improves accuracy\.

Confidence Extrapolation with Different Controllers\.Table[13](https://arxiv.org/html/2605.30753#S4.T13)shows CE consistently improves efficiency across controllers\. For TSPD, CE increases throughput from 30\.3 to 34\.2 TPS and speedup from 4\.4×\\timesto 5\.0×\\timeswith similar quality; Learn2PD and heuristics used in Fast\-dLLM exhibit similar gains\. This indicates CE is controller\-agnostic and complements existing stopping policies by supplying forward\-looking confidence\.

Table 13:A comparison of our confidence extrapolator method with different decoding controllers using LLaDA\-8B\-Instruct model on GSM8K benchmark at generation length of 256 tokens\.MethodsTPS↑\\uparrowSpeed↑\\uparrowAcc\.TSPD30\.34\.4×\\times79\.5TSPD & CE34\.25\.0×\\times79\.4Learn2PD29\.04\.2×\\times79\.1Learn2PD & CE32\.44\.7×\\times79\.3Fast\-dLLM52\.87\.8×\\times78\.5Fast\-dLLM & CE64\.29\.3×\\times78\.8CD54\.57\.9×\\times78\.7CD & CE58\.08\.4×\\times78\.7

## 5Conclusion

In this work, we address dLLM inference inefficiency and unreliability caused by repetitive denoising and step\-local stopping criteria\. We propose Temporal\-Spatial Parallel Decoding \(TSPD\), a lightweight trace\-aware controller that uses token\-wise trajectories and token position to decide when tokens have converged and can be fixed\. We further introduce Confidence Extrapolation \(CE\), a training\-free state\-space plug\-in that forecasts future confidences with uncertainty to enable proactive acceleration\. Across four benchmarks on LLaDA\-8B\-Instruct \(GSM8K, MATH, HumanEval, MBPP\), TSPD delivers consistent and distinguished throughput gains at different generation settings with negligible accuracy change\. Our approach also composes with KV caching for additional speedups, indicating strong compatibility with system optimizations\.

## Impact Statement

This work enhances inference efficiency of diffusion\-based large language models \(dLLMs\) via lightweight, training\-free techniques \(TSPD and Confidence Extrapolation\), reducing latency and computation while preserving output quality\. This facilitates practical deployment in resource\-limited settings like edge devices, real\-time applications, and large\-scale systems\.

Positive impacts include lower energy consumption for sustainable AI and greater accessibility to high\-quality generation in education, scientific reasoning, coding, and productivity tools, democratizing advanced generative AI\.

Faster inference may heighten misuse risks \(e\.g\., rapid misinformation or harmful content\)\. However, our method introduces no new vulnerabilities and is fully compatible with existing safeguards \(alignment, filtering, watermarking\)\. Responsible deployment practices should adequately mitigate concerns\. Overall, the efficiency gains are predominantly beneficial for machine learning and real\-world applications\.

## References

- Anonymous \(2026\)ES\-dLLM: efficient inference for diffusion large language models by early\-skipping\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=O2WvMkJbws)Cited by:[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p2.3)\.
- J\. Austin, D\. D\. Johnson, J\. Ho, D\. Tarlow, and R\. Van Den Berg \(2021a\)Structured denoising diffusion models in discrete state\-spaces\.Advances in Neural Information Processing Systems34,pp\. 17981–17993\.External Links:2107\.03006Cited by:[§2\.1](https://arxiv.org/html/2605.30753#S2.SS1.p1.1)\.
- J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le,et al\.\(2021b\)Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p5.3),[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p1.1)\.
- W\. Bao, Z\. Chen, D\. Xu, and Y\. Shang \(2025\)Learning to parallel: accelerating diffusion large language models via learnable parallel decoding\.arXiv preprint arXiv:2509\.25188\.Cited by:[§A\.5](https://arxiv.org/html/2605.30753#A1.SS5.p1.11),[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p2.3),[§3\.1\.2](https://arxiv.org/html/2605.30753#S3.SS1.SSS2.p2.1),[§3\.1\.2](https://arxiv.org/html/2605.30753#S3.SS1.SSS2.p4.4),[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p3.1)\.
- A\. Campbell, J\. Benton, V\. De Bortoli, T\. Rainforth, G\. Deligiannidis, and A\. Doucet \(2022\)A continuous time framework for discrete denoising models\.Advances in Neural Information Processing Systems35,pp\. 28266–28279\.Cited by:[§2\.1](https://arxiv.org/html/2605.30753#S2.SS1.p1.1)\.
- K\. Chen, Z\. Liu, X\. Tao, H\. Liu, X\. Fu, S\. Zhang, D\. Tu, L\. Kong, R\. Liu, and H\. Li \(2025a\)Beyond confidence: adaptive and coherent decoding for diffusion language models\.arXiv preprint arXiv:2512\.02044\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p2.1)\.
- M\. Chen \(2021\)Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p5.3),[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p1.1)\.
- Z\. Chen, G\. Fang, X\. Ma, R\. Yu, and X\. Wang \(2025b\)Dparallel: learnable parallel decoding for dllms\.arXiv preprint arXiv:2509\.26488\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p2.1)\.
- K\. Cho, B\. Van Merriënboer, C\. Gulcehre, D\. Bahdanau, F\. Bougares, H\. Schwenk, and Y\. Bengio \(2014\)Learning phrase representations using rnn encoder\-decoder for statistical machine translation\.arXiv preprint arXiv:1406\.1078\.Cited by:[§3\.2](https://arxiv.org/html/2605.30753#S3.SS2.p3.3)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p5.3),[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p1.1)\.
- Z\. Feng, J\. Ma, E\. Chersoni, X\. Zhao, and X\. Bao \(2025\)Learning to look at the other side: a semantic probing study of word embeddings in llms with enabled bidirectional attention\.InProceedings of the Annual Meeting of the Association for Computational Linguistics: ACL 2025,Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- S\. Gong, S\. Agarwal, Y\. Zhang, J\. Ye, L\. Zheng, M\. Li, C\. An, P\. Zhao, W\. Bi, J\. Han,et al\.\(2024\)Scaling diffusion language models via adaptation from autoregressive models\.arXiv preprint arXiv:2410\.17891\.Cited by:[§2\.1](https://arxiv.org/html/2605.30753#S2.SS1.p2.1)\.
- Z\. He, T\. Sun, Q\. Tang, K\. Wang, X\. Huang, and X\. Qiu \(2023\)Diffusionbert: improving generative masked language models with diffusion models\.InProceedings of the 61st annual meeting of the association for computational linguistics \(volume 1: Long papers\),pp\. 4521–4534\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021\)Measuring mathematical problem solving with the math dataset\.arXiv preprint arXiv:2103\.03874\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p5.3),[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p1.1)\.
- J\. Ho, A\. Jain, and P\. Abbeel \(2020\)Denoising diffusion probabilistic models\.Advances in neural information processing systems33,pp\. 6840–6851\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- S\. Hochreiter and J\. Schmidhuber \(1997\)Long short\-term memory\.Neural computation9\(8\),pp\. 1735–1780\.Cited by:[§3\.2](https://arxiv.org/html/2605.30753#S3.SS2.p3.3)\.
- E\. Hoogeboom, D\. Nielsen, P\. Jaini, P\. Forré, and M\. Welling \(2021\)Argmax flows and multinomial diffusion: learning categorical distributions\.Advances in neural information processing systems34,pp\. 12454–12465\.Cited by:[§2\.1](https://arxiv.org/html/2605.30753#S2.SS1.p1.1)\.
- Z\. Hu, J\. Meng, Y\. Akhauri, M\. S\. Abdelfattah, J\. Seo, Z\. Zhang, and U\. Gupta \(2025\)Accelerating diffusion language model inference via efficient kv caching and guided diffusion\.arXiv preprint arXiv:2505\.21467\.Cited by:[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p2.3)\.
- P\. Huang, S\. Liu, Z\. Liu, Y\. Yan, S\. Wang, Z\. Chen, and T\. Xiao \(2025\)Pc\-sampler: position\-aware calibration of decoding bias in masked diffusion models\.arXiv preprint arXiv:2508\.13021\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p2.1)\.
- D\. Israel, G\. V\. d\. Broeck, and A\. Grover \(2025\)Accelerating diffusion llms via adaptive parallel decoding\.arXiv preprint arXiv:2506\.00413\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p2.3)\.
- R\. E\. Kalman \(1960\)A new approach to linear filtering and prediction problems\.Journal of Basic Engineering82\(1\),pp\. 35–45\.Cited by:[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p3.1)\.
- S\. Khanna, S\. Kharbanda, S\. Li, H\. Varma, E\. Wang, S\. Birnbaum, Z\. Luo, Y\. Miraoui, A\. Palrecha, S\. Ermon,et al\.\(2025\)Mercury: ultra\-fast language models based on diffusion\.arXiv preprint arXiv:2506\.172981\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- F\. Kong, J\. Zhang, Y\. Liu, Z\. Wu, Y\. Tian, G\. Zhou,et al\.\(2025\)Accelerating diffusion llm inference via local determinism propagation\.arXiv preprint arXiv:2510\.07081\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p2.3)\.
- P\. Li, Y\. Zhou, D\. Muhtar, L\. Yin, S\. Yan, L\. Shen, Y\. Liang, S\. Vosoughi, and S\. Liu \(2025\)Diffusion language models know the answer before decoding\.arXiv preprint arXiv:2508\.19982\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p2.1),[§3\.1\.2](https://arxiv.org/html/2605.30753#S3.SS1.SSS2.p2.1),[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p1.1)\.
- A\. Liu, J\. Liu, Z\. Pan, Y\. He, G\. Haffari, and B\. Zhuang \(2024\)Minicache: kv cache compression in depth dimension for large language models\.Advances in Neural Information Processing Systems37,pp\. 139997–140031\.Cited by:[§4\.3](https://arxiv.org/html/2605.30753#S4.SS3.p1.3)\.
- Z\. Liu, Y\. Yang, Y\. Zhang, J\. Chen, C\. Zou, Q\. Wei, S\. Wang, and L\. Zhang \(2025\)Dllm\-cache: accelerating diffusion large language models with adaptive caching\.arXiv preprint arXiv:2506\.06295\.Cited by:[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p2.3)\.
- A\. Lou, C\. Meng, and S\. Ermon \(2023\)Discrete diffusion modeling by estimating the ratios of the data distribution\.arXiv preprint arXiv:2310\.16834\.Cited by:[§2\.1](https://arxiv.org/html/2605.30753#S2.SS1.p1.1)\.
- Y\. Ma, L\. Du, L\. Wei, K\. Chen, Q\. Xu, K\. Wang, G\. Feng, G\. Lu, L\. Liu, X\. Qi,et al\.\(2025\)Dinfer: an efficient inference framework for diffusion language models\.arXiv preprint arXiv:2510\.08666\.Cited by:[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p2.3)\.
- Y\. Mo, Q\. Chen, M\. Li, Z\. Wei, and Y\. Wang \(2025\)Decoding large language diffusion models with foreseeing movement\.arXiv preprint arXiv:2512\.04135\.Cited by:[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p2.3)\.
- A\. Q\. Nichol and P\. Dhariwal \(2021\)Improved denoising diffusion probabilistic models\.InInternational conference on machine learning,pp\. 8162–8171\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- S\. Nie, F\. Zhu, C\. Du, T\. Pang, Q\. Liu, G\. Zeng, M\. Lin, and C\. Li \(2024\)Scaling up masked diffusion models on text\.arXiv preprint arXiv:2410\.18514\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. Li \(2025\)Large language diffusion models\.arXiv preprint arXiv:2502\.09992\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1),[§1](https://arxiv.org/html/2605.30753#S1.p5.3),[§2\.1](https://arxiv.org/html/2605.30753#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p1.1)\.
- R\. Pope, S\. Douglas, A\. Chowdhery, J\. Devlin, J\. Bradbury, J\. Heek, K\. Xiao, S\. Agrawal, and J\. Dean \(2023\)Efficiently scaling transformer inference\.Proceedings of machine learning and systems5,pp\. 606–624\.Cited by:[§4\.3](https://arxiv.org/html/2605.30753#S4.SS3.p1.3)\.
- A\. Radford, K\. Narasimhan, T\. Salimans, I\. Sutskever,et al\.\(2018\)Improving language understanding by generative pre\-training\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. Liu \(2020\)Exploring the limits of transfer learning with a unified text\-to\-text transformer\.Journal of machine learning research21\(140\),pp\. 1–67\.Cited by:[§3\.1\.2](https://arxiv.org/html/2605.30753#S3.SS1.SSS2.p3.5)\.
- S\. Sahoo, M\. Arriola, Y\. Schiff, A\. Gokaslan, E\. Marroquin, J\. Chiu, A\. Rush, and V\. Kuleshov \(2024\)Simple and effective masked diffusion language models\.Advances in Neural Information Processing Systems37,pp\. 130136–130184\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.30753#S2.SS1.p2.1)\.
- D\. Schuurmans, H\. Dai, and F\. Zanini \(2024\)Autoregressive large language models are computationally universal\.arXiv preprint arXiv:2410\.03170\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- M\. Seo, A\. Kembhavi, A\. Farhadi, and H\. Hajishirzi \(2016\)Bidirectional attention flow for machine comprehension\.arXiv preprint arXiv:1611\.01603\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- J\. Sohl\-Dickstein, E\. Weiss, N\. Maheswaranathan, and S\. Ganguli \(2015\)Deep unsupervised learning using nonequilibrium thermodynamics\.InProceedings of the 32nd International Conference on Machine Learning \(ICML\),Cited by:[§2\.1](https://arxiv.org/html/2605.30753#S2.SS1.p1.1)\.
- J\. Song, C\. Meng, and S\. Ermon \(2020\)Denoising diffusion implicit models\.arXiv preprint arXiv:2010\.02502\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- Y\. Song and S\. Ermon \(2019\)Generative modeling by estimating gradients of the data distribution\.Advances in neural information processing systems32\.Cited by:[§2\.1](https://arxiv.org/html/2605.30753#S2.SS1.p1.1)\.
- K\. Wang, Z\. Jiang, H\. Feng, W\. Zhao, L\. Liu, J\. Li, Z\. Lan, and W\. Lin \(2025a\)Creditdecoding: accelerating parallel decoding in diffusion large language models with trace credits\.arXiv preprint arXiv:2510\.06133\.Cited by:[§3\.1\.2](https://arxiv.org/html/2605.30753#S3.SS1.SSS2.p2.1),[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p1.1)\.
- W\. Wang, B\. Fang, C\. Jing, Y\. Shen, Y\. Shen, Q\. Wang, H\. Ouyang, H\. Chen, and C\. Shen \(2025b\)Time is a feature: exploiting temporal dynamics in diffusion language models\.arXiv preprint arXiv:2508\.09138\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p2.1)\.
- X\. Wang, C\. Xu, Y\. Jin, J\. Jin, H\. Zhang, and Z\. Deng \(2025c\)Diffusion llms can do faster\-than\-ar inference via discrete diffusion forcing\.arXiv preprint arXiv:2508\.09192\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p2.1)\.
- Q\. Wei, Y\. Zhang, Z\. Liu, D\. Liu, and L\. Zhang \(2025\)Accelerating diffusion large language models with slowfast sampling: the three golden principles\.arXiv preprint arXiv:2506\.10848\.Cited by:[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p2.3)\.
- K\. C\. Wibisono and Y\. Wang \(2023\)Bidirectional attention as a mixture of continuous word experts\.InUncertainty in Artificial Intelligence,pp\. 2271–2281\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- C\. Wu, H\. Zhang, S\. Xue, S\. Diao, Y\. Fu, Z\. Liu, P\. Molchanov, P\. Luo, S\. Han, and E\. Xie \(2025a\)Fast\-dllm v2: efficient block\-diffusion llm\.arXiv preprint arXiv:2509\.26328\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p2.3)\.
- C\. Wu, H\. Zhang, S\. Xue, Z\. Liu, S\. Diao, L\. Zhu, P\. Luo, S\. Han, and E\. Xie \(2025b\)Fast\-dllm: training\-free acceleration of diffusion llm by enabling kv cache and parallel decoding\.arXiv preprint arXiv:2505\.22618\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.30753#S2.SS2.p2.3),[§3\.1\.2](https://arxiv.org/html/2605.30753#S3.SS1.SSS2.p2.1),[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p1.1),[§4\.3](https://arxiv.org/html/2605.30753#S4.SS3.p1.3)\.
- J\. Ye, Z\. Xie, L\. Zheng, J\. Gao, Z\. Wu, X\. Jiang, Z\. Li, and L\. Kong \(2025\)Dream 7b: diffusion large language models\.arXiv preprint arXiv:2508\.15487\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.30753#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p1.1)\.
- S\. Zhao, D\. Gupta, Q\. Zheng, and A\. Grover \(2025\)D1: scaling reasoning in diffusion large language models via reinforcement learning\.arXiv preprint arXiv:2504\.12216\.Cited by:[§1](https://arxiv.org/html/2605.30753#S1.p1.1)\.
- F\. Zhu, Z\. You, Y\. Xing, Z\. Huang, L\. Liu, Y\. Zhuang, G\. Lu, K\. Wang, X\. Wang, L\. Wei,et al\.\(2025\)Llada\-moe: a sparse moe diffusion language model\.arXiv preprint arXiv:2509\.24389\.Cited by:[§4\.1](https://arxiv.org/html/2605.30753#S4.SS1.p1.1)\.

## Appendix AImplementation and Training Details

### A\.1Software and Hardware

We implement all methods in PyTorch \(v2\.1\) with CUDA \(v12\.1\) and run inference in FP16\. Experiments are executed on NVIDIA A100 80GB GPUs\. Unless noted otherwise, we use batch size11and greedy decoding to avoid confounding factors from sampling\. Throughput is reported astokens/sec\(TPS\) computed byTPS=B⋅LgenTwall,\\mathrm\{TPS\}=\\frac\{B\\cdot L\_\{\\text\{gen\}\}\}\{T\_\{\\text\{wall\}\}\},whereBBis the batch size andTwallT\_\{\\text\{wall\}\}is the end\-to\-end wall\-clock time from the first denoising step to termination, including remasking, controller overhead, and optional KV caching\. For timing stability, we warm up 20 runs and report the mean over 200 samples \(or the full evaluation set if smaller\)\.

### A\.2Base dLLM Decoding Configuration

We follow the default inference configuration of LLaDA\-8B\-Instruct\. The response span has lengthLgen=256L\_\{\\text\{gen\}\}\{=\}256and is initialized as all\[MASK\]tokens\. Denoising proceeds for at mostK=256K\{=\}256steps\. At steptt, the model outputs per\-position logitsℓi\(t\)∈ℝV\\ell\_\{i\}^\{\(t\)\}\\in\\mathbb\{R\}^\{V\}and the token distributionpi\(t\)​\(v\)=softmax​\(ℓi\(t\)\)v\.p\_\{i\}^\{\(t\)\}\(v\)=\\mathrm\{softmax\}\(\\ell\_\{i\}^\{\(t\)\}\)\_\{v\}\.We compute the top\-1 predictiony^i\(t\)=arg⁡maxv⁡pi\(t\)​\(v\)\\hat\{y\}\_\{i\}^\{\(t\)\}=\\arg\\max\_\{v\}p\_\{i\}^\{\(t\)\}\(v\)and confidenceci\(t\)=maxv⁡pi\(t\)​\(v\),c\_\{i\}^\{\(t\)\}=\\max\_\{v\}p\_\{i\}^\{\(t\)\}\(v\),and use the standard low\-confidence remasking rule\. Let𝒜\(t\)\\mathcal\{A\}^\{\(t\)\}be the set of*active*\(unfixed\) positions at steptt\. We remask thentn\_\{t\}lowest\-confidence positions in𝒜\(t\)\\mathcal\{A\}^\{\(t\)\}, wherentn\_\{t\}follows the default schedule used by LLaDA\. Concretely, we setnt=⌈ρ​\(t\)⋅\|𝒜\(t\)\|⌉,ρ​\(t\)=ρmax⋅\(tK\)γ,n\_\{t\}=\\left\\lceil\\rho\(t\)\\cdot\|\\mathcal\{A\}^\{\(t\)\}\|\\right\\rceil,\\quad\\rho\(t\)=\\rho\_\{\\max\}\\cdot\\Big\(\\frac\{t\}\{K\}\\Big\)^\{\\gamma\},withρmax=0\.5\\rho\_\{\\max\}=0\.5andγ=1\\gamma=1as a representative setting, and we keep fixed tokens excluded from remasking\. Decoding terminates when all response tokens are fixed or whent=1t=1\.

### A\.3TSPD Controller: Architecture and Features

TSPD is a per\-token sequential classifier shared across positions\. We use a 2\-layer LSTM with hidden sizedh=16d\_\{h\}\{=\}16and input dimensiondin=6d\_\{\\text\{in\}\}\{=\}6, followed by a linear head that outputs a Bernoulli logit\. For each token positioniiat steptt, we compute a compact trace feature vectorri\(t\)=\[ci\(t\),Hi\(t\),c¯i\(t\),Δ​c¯i\(t\),ϕ​\(i\),ui\(t\)\],r\_\{i\}^\{\(t\)\}=\[\\,c\_\{i\}^\{\(t\)\},\\ H\_\{i\}^\{\(t\)\},\\ \\bar\{c\}\_\{i\}^\{\(t\)\},\\ \\Delta\\bar\{c\}\_\{i\}^\{\(t\)\},\\ \\phi\(i\),\\ u\_\{i\}^\{\(t\)\}\\,\],where entropy isHi\(t\)=−∑vpi\(t\)​\(v\)​log⁡pi\(t\)​\(v\),H\_\{i\}^\{\(t\)\}=\-\\sum\_\{v\}p\_\{i\}^\{\(t\)\}\(v\)\\log p\_\{i\}^\{\(t\)\}\(v\),EWMA confidence isc¯i\(t\)=α​ci\(t\)\+\(1−α\)​c¯i\(t\+1\)​with​α=0\.25,\\bar\{c\}\_\{i\}^\{\(t\)\}=\\alpha c\_\{i\}^\{\(t\)\}\+\(1\-\\alpha\)\\bar\{c\}\_\{i\}^\{\(t\+1\)\}\\;\\;\\text\{with\}\\;\\;\\alpha=0\.25,and momentum isΔ​c¯i\(t\)=c¯i\(t\)−c¯i\(t\+1\)\\Delta\\bar\{c\}\_\{i\}^\{\(t\)\}=\\bar\{c\}\_\{i\}^\{\(t\)\}\-\\bar\{c\}\_\{i\}^\{\(t\+1\)\}\.ϕ​\(i\)∈\[0,1\]\\phi\(i\)\\in\[0,1\]denotes the relative position in the response span\.ui\(t\)u\_\{i\}^\{\(t\)\}is an optional uncertainty feature provided by CE \(Section[A\.7](https://arxiv.org/html/2605.30753#A1.SS7)\); when CE is disabled we setui\(t\)=0u\_\{i\}^\{\(t\)\}=0\. TSPD updates a hidden statehi\(t\)h\_\{i\}^\{\(t\)\}from\(hi\(t\+1\),ri\(t\)\)\(h\_\{i\}^\{\(t\+1\)\},r\_\{i\}^\{\(t\)\}\)and outputs a fixing probabilityπi\(t\)=σ​\(W​hi\(t\)\+b\)\.\\pi\_\{i\}^\{\(t\)\}=\\sigma\(Wh\_\{i\}^\{\(t\)\}\+b\)\.We fix tokeniiat stepttifπi\(t\)≥0\.5\\pi\_\{i\}^\{\(t\)\}\\geq 0\.5, commity^i\(t\)\\hat\{y\}\_\{i\}^\{\(t\)\}to the sequence, and mark it as fixed for the remainder of decoding\.

### A\.4Parameter Count and Overhead

The controller is intentionally lightweight\. Withdh=16d\_\{h\}\{=\}16anddin=6d\_\{\\text\{in\}\}\{=\}6, the 2\-layer LSTM has approximately∑ℓ=124​\(dℓ​\(dℓ−1\+dℓ\)\+dℓ\)≈2,000\\sum\_\{\\ell=1\}^\{2\}4\(d\_\{\\ell\}\(d\_\{\\ell\-1\}\+d\_\{\\ell\}\)\+d\_\{\\ell\}\)\\approx 2\{,\}000parameters \(plus a small linear head\), matching the order\-of\-magnitude reported in Section[4\.1](https://arxiv.org/html/2605.30753#S4.SS1)\. At inference, TSPD evaluates one LSTM step per active token per denoising step\. In practice this overhead is negligible relative to a full forward pass of the 8B dLLM; the controller runs on CPU or GPU and uses only vector operations on low\-dimensional states\.

### A\.5Training Data Collection \(Stage 1\)

We collect supervision traces using an Extremely Greedy Parallel policy\(Baoet al\.,[2025](https://arxiv.org/html/2605.30753#bib.bib107)\)\. For each prompt, we run the base dLLM to completion forKKsteps and store, for each positioniiand steptt: \(i\) trace featuresri\(t\)r\_\{i\}^\{\(t\)\}, \(ii\) the current top\-1 tokeny^i\(t\)\\hat\{y\}\_\{i\}^\{\(t\)\}, and \(iii\) the final tokeny^i\(1\)\\hat\{y\}\_\{i\}^\{\(1\)\}\. We then assign a binary labelyi\(t\)=𝕀​\[y^i\(t\)=y^i\(1\)\],y\_\{i\}^\{\(t\)\}=\\mathbb\{I\}\\big\[\\hat\{y\}\_\{i\}^\{\(t\)\}=\\hat\{y\}\_\{i\}^\{\(1\)\}\\big\],which indicates whether fixing at stepttpreserves the final outcome\. To reduce storage, we store only the feature tensors and labels \(not full vocabulary logits\)\. For the FLAN subset used in Section[4\.1](https://arxiv.org/html/2605.30753#S4.SS1)\(2,640 prompts\) withLgen=256L\_\{\\text\{gen\}\}\{=\}256andK=256K\{=\}256, this yields on the order of2,640×256×2562\{,\}640\\times 256\\times 256token\-step instances before filtering; we subsample active positions and steps to control dataset size\.

### A\.6Training Objective and Optimization \(Stage 2\)

We train TSPD with weighted binary cross\-entropy to address label imbalance:ℒ=−w1​y​log⁡π−w0​\(1−y\)​log⁡\(1−π\),\\mathcal\{L\}=\-w\_\{1\}y\\log\\pi\-w\_\{0\}\(1\-y\)\\log\(1\-\\pi\),withw1/w0w\_\{1\}/w\_\{0\}set by inverse class frequency on the training split\. We use AdamW with learning rate10−310^\{\-3\}, weight decay10−210^\{\-2\}\. We train for 5,000 epochs with early stopping on a held\-out 10% validation split\. To improve generalization across tasks, we randomly drop trace channels during training \(dropoutp=0\.1p=0\.1onri\(t\)r\_\{i\}^\{\(t\)\}\) and apply small Gaussian noise to confidence\-related features \(σ=0\.01\\sigma=0\.01\)\. The dLLM remains frozen; only TSPD parameters are updated\.

### A\.7Confidence Extrapolation \(CE\)

CE is a training\-free per\-token forecasting module that models the evolution of a scalar confidence margin\. Letδi\(t\)=ℓi,1\(t\)−ℓi,2\(t\)\\delta\_\{i\}^\{\(t\)\}=\\ell\_\{i,1\}^\{\(t\)\}\-\\ell\_\{i,2\}^\{\(t\)\}be the margin between top\-1 and top\-2 confidences at positionii\. We use a 2D constant\-velocity state modelxi\(t\)=\[δi\(t\)δ˙i\(t\)\],xi\(t−1\)=A​xi\(t\)\+ϵ,A=\[1101\],x\_\{i\}^\{\(t\)\}=\\begin\{bmatrix\}\\delta\_\{i\}^\{\(t\)\}\\\\ \\dot\{\\delta\}\_\{i\}^\{\(t\)\}\\end\{bmatrix\},\\quad x\_\{i\}^\{\(t\-1\)\}=Ax\_\{i\}^\{\(t\)\}\+\\epsilon,\\quad A=\\begin\{bmatrix\}1&1\\\\ 0&1\\end\{bmatrix\},with Gaussian process noiseϵ∼𝒩​\(0,Q\)\\epsilon\\sim\\mathcal\{N\}\(0,Q\)and observationoi\(t\)=δi\(t\)\+ηo\_\{i\}^\{\(t\)\}=\\delta\_\{i\}^\{\(t\)\}\+\\etawithη∼𝒩​\(0,R\)\\eta\\sim\\mathcal\{N\}\(0,R\)\. We run a Kalman filter to estimate\(x^i\(t\),Pi\(t\)\)\(\\hat\{x\}\_\{i\}^\{\(t\)\},P\_\{i\}^\{\(t\)\}\)and performhh\-step prediction to obtain a conservative margin forecastδ^i,h\(t\)−z​Var​\(δi,h\(t\)\),\\hat\{\\delta\}\_\{i,h\}^\{\(t\)\}\-z\\sqrt\{\\mathrm\{Var\}\(\\delta\_\{i,h\}^\{\(t\)\}\)\},wherezzcontrols risk sensitivity\. This value is mapped to a conservative confidence proxyc~i\(t\)\\tilde\{c\}\_\{i\}^\{\(t\)\}via a monotone transform \(e\.g\., sigmoid calibration\) and used as an additional trace signal and/or to choose a safe look\-ahead horizonhh\. CE adds only scalar operations per token per step and is negligible compared to the base model forward pass\.

### A\.8Inference Integration

At inference steptt, we run one forward pass ofℳ\\mathcal\{M\}, compute trace features for active tokens, optionally run CE to obtain forecast\-based signals, and apply TSPD to decide which tokens to fix\. Fixed tokens are excluded from remasking and remain constant in subsequent steps\. We then apply the standard remasking schedule to the remaining active tokens and proceed to stept−1t\-1\. We terminate when all response tokens are fixed or the step budget is exhausted\.

![Refer to caption](https://arxiv.org/html/2605.30753v1/x6.png)Figure 6:Learning curve of the TSPD\. The learning curves illustrate the progression of training and validation loss across 5,000 epochs\.
### A\.9Baselines and KV Caching

For Fast\-dLLM and Credit Decoding, we enable their KV caching implementations and report their default settings\. For Learn2PD and Prophet, we follow their published inference procedures without KV caching\. When evaluating compatibility, we apply the same KV caching strategy to our method by caching the dLLM intermediate states used by the underlying forward passes, while keeping the controller and CE unchanged\. This isolates the gains from TSPD and CE and verifies orthogonality to system\-level reuse\.

## Appendix BTraining Curve of TSPD

As shown in Figure[6](https://arxiv.org/html/2605.30753#A1.F6), both the training loss \(blue\) and validation loss \(orange\) drop rapidly in the early epochs, decreasing from roughly0\.70\.7to around0\.280\.28within the initial phase\. After this sharp descent, the curves enter a slow refinement regime and decrease gradually over the remainder of the 5,000\-epoch run, forming a near\-flat tail\. Throughout training, the validation loss stays slightly above the training loss, indicating a small and stable generalization gap rather than severe overfitting\. The mild jitter in both curves reflects stochastic optimization noise, while the long, slowly decreasing tail suggests diminishing returns and that the model has effectively converged by the end of training\.