Survival-Guided Length Control for Efficient Diffusion Language Models

# Survival-Guided Length Control for Efficient Diffusion Language Models
Source: [https://arxiv.org/html/2608.26374](https://arxiv.org/html/2608.26374)
Abbas Ghaddar11footnotemark:1Yufei CuiAffiliation:Huawei Noah’s Ark Lab, Montreal Research Center, CanadaEmail:[\{ivan\.kobyzev,abbas\.ghaddar,yufei\.cui\}@huawei\.com](mailto:)

###### Abstract

Diffusion language models \(DLMs\) generate text by iteratively denoising masked sequences, but standard decoding either fixes the sequence length or relies on ad hoc stopping rules, often leading to unnecessary denoising steps\. We recast length selection as a discrete\-time survival problem over the end\-of\-sequence token and propose a plug\-in, training\-free length predictor that can be added to any existing DLM\. Across reasoning and code\-generation benchmarks, survival\-guided length decoding speeds up inference by up to7×7\\timeswhile preserving task accuracy\. We further find that predicted lengths vary widely even within the same dataset, making model performance sensitive to the chosen length\.

## 1Introduction

Masked Diffusion Language Models \(DLMs\)[Austin et al\. \(2021a\)](https://arxiv.org/html/2608.26374#bib.bib6);[Shi et al\. \(2024\)](https://arxiv.org/html/2608.26374#bib.bib7)generate text by iteratively denoising a masked canvas\. At each diffusion step, masked positions are updated using the model’s predictive distribution, gradually transforming an all\-\[MASK\]suffix into the text\. This iterative refinement enables flexible any\-order decoding and parallelism, but it also raises a core question for decoding cost: when to stop denoising and which tokens to commit at each step?

Standard decoding for masked DLMs often follows an any\-order autoregressive \(AOAR\) pattern[Ou et al\. \(2024\)](https://arxiv.org/html/2608.26374#bib.bib8): choose a large task\-agnostic maximum lengthLmaxL\_\{\\max\}, run a fixed denoising schedule, and stop only when all masks are removed or an iteration budget is reached\. While simple, this approach is frequently wasteful: many prompts require far fewer tokens thanLmaxL\_\{\\max\}, yet the decoder still spends computation refining positions that correspond to an unnecessarily long canvas\.

As a result, a major source of overhead in masked DLM inference is not the denoising rule itself, but the mismatch between a conservative global length budget and the instance\-specific length needed\. In this work, we focus on length selection: for a given prompt, how many new tokens should we generate? We show that length selection admits a natural interpretation in terms of survival analysis[Andersen et al\. \(1993\)](https://arxiv.org/html/2608.26374#bib.bib9)\.

Concretely, we treat generation length as a discrete\-time survival variable over the end\-of\-sequence token\[EOS\]\. Using the DLM’s per\-position\[EOS\]probabilities from a single forward pass on a long masked canvas, we obtain a plug\-in estimate of the discrete\-time hazard and recover a closed\-form estimate of the expected length via standard survival identities\. This yields atraining\-free, model\-agnostic length predictorthat can be plugged into existing masked DLMs without modifying model parameters or changing the underlying denoising schedule\. Empirically, survival\-guided length prediction reduces unnecessary refinement of positions beyond the true end of the sequence and speeds up inference without sacrificing performance relative to baselines that decode with a sufficiently largeLmaxL\_\{\\max\}\.

Extensive experiments with two large\-scale DLMs, LLaDA[Nie et al\. \(2025\)](https://arxiv.org/html/2608.26374#bib.bib4)and Dream[Ye et al\. \(2025\)](https://arxiv.org/html/2608.26374#bib.bib5), consistently demonstrate inference speedups of up to7×7\\timesacross reasoning and code\-generation benchmarks, with no loss in task performance\. Additionally, our analysis suggests that length prediction is indeed sample\-dependent, with lengths varying significantly across samples of the same distribution\.

## 2Method

Survival analysis\([Andersen et al\., 1993](https://arxiv.org/html/2608.26374#bib.bib9)\)studies the distribution of random event times, classically in settings such as time\-to\-failure or time\-to\-death\. A central object is the hazard function, which models the probability that an event occurs at timettgiven that it has not occurred before\. This viewpoint is naturally discrete and sequential: at each step we either survive to the next step or terminate\. Applying this framework to DLMs, the unknown sequence length becomes a discrete event time for the end\-of\-sequence token, and at each candidate position we ask if the sequence has not ended yet, how likely is it to end here?

This allows us to reuse standard survival identities to obtain a plug\-in estimate of the expected length from a single pass of the DLM\. We consider masked diffusion language models[Austin et al\. \(2021a\)](https://arxiv.org/html/2608.26374#bib.bib6);[Shi et al\. \(2024\)](https://arxiv.org/html/2608.26374#bib.bib7)that generate text by iteratively denoising a masked canvas\. Let

𝐱\(0\)=\[x1:P,\[MASK\],…,\[MASK\]\]\\mathbf\{x\}^\{\(0\)\}=\[x\_\{1:P\},\\texttt\{\[MASK\]\},\\dots,\\texttt\{\[MASK\]\}\]\(1\)be the initial sequence consisting of a prompt of lengthPPfollowed byTTmasked positions, and let𝐱\(s\)\\mathbf\{x\}^\{\(s\)\}denote the sequence afterssdenoising steps\. At each stepss, the DLM produces logits and probabilities:

𝐳\(s\)=fθ​\(𝐱\(s\),ts\),pi,v\(s\)=softmaxv​zi,v\(s\),\\mathbf\{z\}^\{\(s\)\}=f\_\{\\theta\}\\bigl\(\\mathbf\{x\}^\{\(s\)\},t\_\{s\}\\bigr\),\\quad p^\{\(s\)\}\_\{i,v\}=\\text\{softmax\}\_\{v\}z^\{\(s\)\}\_\{i,v\},for positioniiand vocabulary tokenvv, wheretst\_\{s\}is the \(discrete\) diffusion time\.

We treat the unknown sequence length as a discrete\-time survival problem over positions in the generation span\. Consider the masked input to the DLM as in Eq\.[1](https://arxiv.org/html/2608.26374#S2.E1), where the prompt occupies the positions1,…,P1,\\dots,P, and the model is tasked to generate up toTTnew tokens in positionsP\+1,…,P\+TP\+1,\\dots,P\+T\. However, the ideal generation length is unknown\. In standard practiceTTis set to a large, task\-agnostic upper boundLmaxL\_\{\\max\}, so that many prompts terminate long beforeLmaxL\_\{\\max\}but the decoder still runs the full denoising schedule on allTTslots\. This mismatch between a fixed global budget and the ideal instance\-specific length is a major source of wasted compute\.

To predict the ideal generation lengthLL, let us run one pass of the diffusion model on the initial canvas and denote the resulting logits as𝐳\(0\)\\mathbf\{z\}^\{\(0\)\}\. The quality of the text decoded after one step of DLM is usually extremely low, but these logits are still informative and can be used for our task of length prediction if we interpret its per\-position\[EOS\]probabilities as a length\-survival signal\.

Letttbe the position of the logits𝐳\(0\)\\mathbf\{z\}^\{\(0\)\}in the sequence, and define

pt\\displaystyle p\_\{t\}:=ℙ⁡\(\[EOS\]at position​t\|prompt, masks\)\\displaystyle:=\\mathbb\{P\}\\big\(\\texttt\{\[EOS\]\}\\text\{ at position \}t\\,\\big\|\\,\\text\{prompt, masks\}\\big\)\(2\)=softmax⁡\(𝐳t\(0\)\)​\[\[EOS\]\],\\displaystyle=\\mathrm\{softmax\}\(\\mathbf\{z\}^\{\(0\)\}\_\{t\}\)\[\\texttt\{\[EOS\]\}\],fort=P\+1,…,P\+T\.t=P\{\+\}1,\\dots,P\{\+\}T\.We model the sequence termination with a discrete\-time survival process\. Define the hazard at relative positionk=1,…,Tk=1,\\dots,Tas the probability that the sequence ends now, given that it has not ended before:

hk:=ℙ⁡\(L=k∣L≥k,prompt\)\.h\_\{k\}:=\\mathbb\{P\}\(L=k\\mid L\\geq k,\\text\{prompt\}\)\.Then assuming that the diffusion model is well\-trained and fits the data distribution, we can estimate the hazard ashk≈pP\+kh\_\{k\}\\approx p\_\{P\+k\}\.

We make a mean\-field\-style approximation[Blei et al\. \(2017\)](https://arxiv.org/html/2608.26374#bib.bib10), and treat the\[EOS\]events across positions as conditionally independent given the prompt\. Under this assumption, the survival up to stepkkis

S⁡\(k\)=∏i=1k\(1−hi\),S⁡\(0\)=1,S\(k\)\\;=\\;\\prod\_\{i=1\}^\{k\}\\bigl\(1\-h\_\{i\}\\bigr\),\\qquad S\(0\)=1,\(3\)and the probability of termination exactly atkkis

πk=ℙ⁡\(L=k\)≈hk​S​\(k−1\)\.\\pi\_\{k\}\\;=\\;\\mathbb\{P\}\(L=k\)\\;\\approx\\;h\_\{k\}\\,S\(k\-1\)\.\(4\)
Given the plug\-in length distribution\{πk\}k=1T\\\{\\pi\_\{k\}\\\}\_\{k=1\}^\{T\}, we can compute the truncated expected length:

𝔼⁡\[L\]=∑k=1Tk​πk=∑k=1TS⁡\(k−1\),\\displaystyle\\mathbb\{E\}\[L\]\\;=\\;\\sum\_\{k=1\}^\{T\}k\\,\\pi\_\{k\}\\;=\\;\\sum\_\{k=1\}^\{T\}S\(k\-1\),\(5\)where the second equality is the standard identity that the expectation equals the sum of survivals \(see proof in Appendix[A](https://arxiv.org/html/2608.26374#A1)\)\.

At test time we choose a single plug\-in length estimateL^​\(x\)=𝔼​\[L\]\\hat\{L\}\(x\)=\\mathbb\{E\}\[L\]and pass it to the DLM as the maximum number of new tokens\. Crucially, this requires no extra training or parameters\. The length predictor is entirely derived from the base model’s\[EOS\]logits\. See Algorithm[1](https://arxiv.org/html/2608.26374#alg1)in Appendix[B](https://arxiv.org/html/2608.26374#A2)\.

## 3Experiments

### 3\.1Experimental Setting

We experiment with two powerful masked diffusion language models \(DLMs\) that are trained in notably different ways: LLaDA\-8B\-Base[Nie et al\. \(2025\)](https://arxiv.org/html/2608.26374#bib.bib4)and Dream\-v0\-Base\-7B[Ye et al\. \(2025\)](https://arxiv.org/html/2608.26374#bib.bib5)\. Both are implemented in PyTorch[Paszke et al\. \(2019\)](https://arxiv.org/html/2608.26374#bib.bib2)on top of the Transformers library[Wolf et al\. \(2020\)](https://arxiv.org/html/2608.26374#bib.bib3)\. Hereafter, we refer to these two models as LLaDA and Dream, respectively\.

We consider a set of reasoning and code\-generation benchmarks used in the original evaluations of both models, including BBH[Suzgun et al\. \(2023\)](https://arxiv.org/html/2608.26374#bib.bib18), GSM8K[Cobbe et al\. \(2021\)](https://arxiv.org/html/2608.26374#bib.bib19), MATH[Hendrycks et al\. \(2021\)](https://arxiv.org/html/2608.26374#bib.bib20), HumanEval[Chen et al\. \(2021\)](https://arxiv.org/html/2608.26374#bib.bib21), and MBPP[Austin et al\. \(2021b\)](https://arxiv.org/html/2608.26374#bib.bib22)\. We follow the standard LM Evaluation Harness[Gao et al\. \(2024\)](https://arxiv.org/html/2608.26374#bib.bib1)setup, reporting 3\-shot accuracy on BBH, strict match 5\-shot accuracy on GSM8K, 4\-shot accuracy on MATH, pass@1 on HumanEval \(0\-shot\)111For Dream on HumanEval we apply the post\-processing following the authors’ public implementation at https://github\.com/DreamLM/Dream\., and pass@1 on MBPP \(3\-shot\)\. We use each model’s custom evaluation code and per\-benchmark hyperparameters\.222Authors of both LLaDA and Dream use a suffix\-masked sequence length ofLmax=1024L\_\{\\max\}\\\!=\\\!1024for all benchmarks\.All experiments are conducted on a single modern compute accelerator, and we report results with a batch size of 1\.

### 3\.2Main Results

Table 1:Decoding speed \(seconds per sample\) for baseline \(w/o\) vs\. length prediction \(w/\)\. Parentheses report speedup relative to the baseline\.Table[1](https://arxiv.org/html/2608.26374#S3.T1)shows the decoding speed \(in terms of seconds per example\) for LLaDA and Dream, when using standard AOAR decoding with a fixed maximum length \(w/o\) against one that is equipped with our survival\-guided length predictor \(w/\)\. We observe that, for both models, using the predicted length yields substantial decoding\-speed improvements across all benchmarks, with speedups ranging from3\.2×3\.2\\timesto6\.6×6\.6\\times\. In addition, we find that on most benchmarks \(except HumanEval\), the speedup is in a similar range for both models despite their structural differences\.

Table[2](https://arxiv.org/html/2608.26374#S3.T2)shows the per\-task performances, as well as standard deviation, of models with and without our length predictor\. As one can see, the efficiency gains reported in[Table 1](https://arxiv.org/html/2608.26374#S3.T1)do not come at the expense of task performance\. Across all benchmarks, the differences betweenw/oandw/remain within the reported standard deviations, indicating no statistically meaningful change\.

Table 2:Task performance and standard deviation for baseline \(w/o\) vs\. length prediction \(w/\)\.This indicates that our survival\-guided length predictor reliably trims redundant tail steps while leaving the quality of the generated sequence essentially unchanged\. A notable aspect of these results is that the procedure is completely model\-agnostic and training\-free\. It consists of a single forward pass on a long masked canvas, interpretation of per\-position\[EOS\]probabilities as a discrete\-time hazard, and computing a closed\-form expected length from the survival curve\. Despite architectural and training differences, this plug\-in estimator consistently delivers33–7×7\\timesspeedups with no measurable loss in accuracy, supporting the view that survival\-guided length control is a robust primitive for efficient decoding in diffusion language models\.

### 3\.3Fixed Mean\-Length Ablation

Table 3:We report the per\-task mean predicted length \(L¯\\bar\{L\}\) along with its standard deviation, as well as the performance change \(Δ\\Delta\) and its standard deviation when usingL¯\\bar\{L\}as a fixed horizon, compared to our per\-sample survival\-guided length selection\.To test whether our gains come simply from choosing a shorter global budget, rather than from instance\-wise adaptation, we run an ablation where each model uses a*single*fixed generation length equal to the dataset\-wide mean predicted lengthL¯\\bar\{L\}from our method\. Table[3](https://arxiv.org/html/2608.26374#S3.T3)reports the mean predicted length \(L¯\\bar\{L\}\) and its standard deviation for each model and benchmark\. It also shows the performance change \(Δ\\Delta\) when replacing our per\-sample length prediction with a fixed horizon set toL¯\\bar\{L\}, rather than using the maximum lengthLmaxL\_\{\\max\}\. We observe that the fixed\-length policy based on the mean predicted length generally underperforms survival\-guided decoding across most settings, with a few exceptions \(e\.g\., LLaDA on MBPP\)\. We attribute this to substantial per\-sample variation in required decoding length within each dataset, which exists even in carefully curated standard benchmarks\. This observation suggests that per\-sample length prediction is important for maintaining performance, as both models appear sensitive to the chosen decoding horizon\.333More results and ablations are presented in Appendix[C](https://arxiv.org/html/2608.26374#A3)\.

## 4Related Work

#### Diffusion language models\.

Discrete denoising diffusion models for text were introduced in work on structured diffusion in discrete state spaces\([Austin et al\., 2021a](https://arxiv.org/html/2608.26374#bib.bib6)\)and later specialized into diffusion\-style language models \(DLMs\) such as LLaDA and Dream, which iteratively unmask tokens on a fixed canvas\([Nie et al\., 2025](https://arxiv.org/html/2608.26374#bib.bib4);[Ye et al\., 2025](https://arxiv.org/html/2608.26374#bib.bib5)\)\. These models trade the strictly left\-to\-right factorization of autoregressive LMs for parallel refinement and flexible token orders, but typical decoders still assume a conservative, fixed maximum lengthTTand a fixed number of refinement steps\. Recent work has focused mainly on improving generation quality and scaling to larger models, with less attention to length prediction and training\-free decoding policies for efficiency\.

#### Length control for diffusion LMs\.

Closest to our length predictor is DAEDAL\([Li et al\., 2026](https://arxiv.org/html/2608.26374#bib.bib11)\), which also targets the fixed\-length limitation of DLMs\. DAEDAL starts from a short canvas and repeatedly reruns the model with expanded lengths, using the average\[EOS\]confidence in a trailing window and a hand\-tuned threshold to decide when the current length is sufficient\. Thus length is recovered by an iterative search over candidate horizons and requires several extra forward passes beyond the main decoding trajectory\. In contrast, our method runs the DLM once on a generously long canvas, interprets per\-position\[EOS\]probabilities as a discrete\-time hazard, and uses a standard survival identity to obtain a closed\-form expected length, with no length loop or per\-task thresholds\.

A different line of work, exemplified by Prophet\([Li et al\., 2025](https://arxiv.org/html/2608.26374#bib.bib12)\), studies “early answer convergence” in DLMs: the model often internally identifies the correct answer well before the last refinement step\. Prophet uses the gap between the top\-2 candidates to decide when to commit all remaining tokens in one shot\. This addresses when to stop refinement for a fixed\-length canvas, not how long the sequence itself should be, and it is based on confidence gaps over content tokens rather than\[EOS\]hazards\. Our length estimator is complementary: it predicts the output length before decoding, while leaving the subsequent refinement schedule \(including Prophet\-style early stops\) unchanged\.

#### Sampling strategies and unmasking schedules\.

There is a line of work to accelerate the decoding of DLMs with efficient unmasking schedulers\. Standard sampling for masked diffusion models follows the MaskGIT paradigm\([Chang et al\., 2022](https://arxiv.org/html/2608.26374#bib.bib13)\), where at each step the model updates a top\-kkor top\-ppsubset of masked positions by confidence\. Adaptive schedulers such as EB\-Sampler\([Ben\-Hamu et al\., 2025](https://arxiv.org/html/2608.26374#bib.bib14)\)and SlowFast Sampling\([Wei et al\., 2026](https://arxiv.org/html/2608.26374#bib.bib15)\)dynamically vary which tokens to refine based on entropy, convergence, or span structure, while other works focus on architectural and caching improvements\([Liu et al\., 2025](https://arxiv.org/html/2608.26374#bib.bib16);[Wu et al\., 2025](https://arxiv.org/html/2608.26374#bib.bib17)\)\. All of these methods operate on a fixed\-length canvas and are orthogonal to our method; in principle, they could be combined with our length predictor to further improve the decoding efficiency\.

## 5Conclusion and Future Work

We propose a training\-free, survival\-guided length predictor for diffusion language models that estimates the expected output length from a single forward pass\. Across LLaDA and Dream, this plug\-in length control delivers33–7×7\\timesdecoding speedups on reasoning and code benchmarks without statistically meaningful changes in accuracy\. Beyond length control, the same survival perspective suggests a broader family of decoding rules for diffusion LMs, for example by viewing token\-level commitment and refinement decisions through a survival or risk\-allocation lens\. Developing such token\-level survival models and combining them with our length predictor in a unified, learnable framework is an interesting direction for future work on efficient and reliable diffusion decoding\.

## Limitations

Like most efficient decoding methods for Diffusion Language Models, our approach is limited to a particular setting and several extensions remain open\. First, we evaluate survival\-guided length control on two large\-scale masked DLMs \(LLaDA\-8B\-Base and Dream\-7B\-Base\) and a standard set of reasoning and code\-generation benchmarks \(BBH, GSM8K, MATH, HumanEval, MBPP\)\. This follows common evaluation practice in recent Diffusion LM works, but it does not cover settings such as extremely long\-context generation, or multi\-turn dialogue, so efficiency in those regimes remains to be verified\. Second, our objective is strictly computational efficiency at inference time under a fixed model and decoding rule\. We do not modify training, nor attempt to optimize for other properties of the outputs \(e\.g\., calibration, robustness, or bias\)\. Understanding how survival\-guided decoding interacts with these broader aspects of model behavior is an interesting direction for future work\.

## Acknowledgments

We thank the anonymous reviewers for their insightful comments\.

## References

- Andersenet al\.\(1993\)P\. K\. Andersen, Ø\. Borgan, R\. D\. Gill, and N\. KeidingStatistical models based on counting processes\.Springer Series in Statistics,Springer,New York\.Cited by:[§1](https://arxiv.org/html/2608.26374#S1.p3.1),[§2](https://arxiv.org/html/2608.26374#S2.p1.1)\.
- Austinet al\.\(2021a\)J\. Austin, D\. D\. Johnson, J\. Ho, D\. Tarlow, and R\. van den BergStructured denoising diffusion models in discrete state\-spaces\.InAdvances in Neural Information Processing Systems,A\. Beygelzimer, Y\. Dauphin, P\. Liang, and J\. W\. Vaughan \(Eds\.\),Cited by:[§1](https://arxiv.org/html/2608.26374#S1.p1.1),[§2](https://arxiv.org/html/2608.26374#S2.p2.1),[§4](https://arxiv.org/html/2608.26374#S4.SS0.SSS0.Px1.p1.1)\.
- Austinet al\.\(2021b\)J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. J\. Cai, M\. Terry, Q\. V\. Le, and C\. SuttonProgram synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§3\.1](https://arxiv.org/html/2608.26374#S3.SS1.p2.1)\.
- Ben\-Hamuet al\.\(2025\)H\. Ben\-Hamu, I\. Gat, D\. Severo, N\. Nolte, and B\. KarrerAccelerated sampling from masked diffusion models via entropy bounded unmasking\.ArXivabs/2505\.24857\.External Links:[Link](https://api.semanticscholar.org/CorpusID:279070422)Cited by:[§4](https://arxiv.org/html/2608.26374#S4.SS0.SSS0.Px3.p1.1)\.
- Bleiet al\.\(2017\)D\. M\. Blei, A\. Kucukelbir, and J\. D\. McAuliffeVariational inference: a review for statisticians\.Journal of the American statistical Association112\(518\),pp\. 859–877\.Cited by:[§2](https://arxiv.org/html/2608.26374#S2.p7.1)\.
- Changet al\.\(2022\)H\. Chang, H\. Zhang, L\. Jiang, C\. Liu, and W\. T\. FreemanMaskGIT: masked generative image transformer\.2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 11305–11315\.External Links:[Link](https://api.semanticscholar.org/CorpusID:246680316)Cited by:[§4](https://arxiv.org/html/2608.26374#S4.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. Pondé, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, B\. Chan, S\. Gray, N\. Ryder, M\. Pavlov, A\. Power, L\. Kaiser, M\. Bavarian, C\. Winter, P\. Tillet, F\. P\. Such, D\. W\. Cummings, M\. Plappert, F\. Chantzis, E\. Barnes, A\. Herbert\-Voss, W\. H\. Guss, A\. Nichol, I\. Babuschkin, S\. Balaji, S\. Jain, A\. Carr, J\. Leike, J\. Achiam, V\. Misra, E\. Morikawa, A\. Radford, M\. M\. Knight, M\. Brundage, M\. Murati, K\. Mayer, P\. Welinder, B\. McGrew, D\. Amodei, S\. McCandlish, I\. Sutskever, and W\. ZarembaEvaluating large language models trained on code\.ArXivabs/2107\.03374\.Cited by:[§3\.1](https://arxiv.org/html/2608.26374#S3.SS1.p2.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problemsTraining verifiers to solve math word problems\.ArXivabs/2110\.14168\.Cited by:[§3\.1](https://arxiv.org/html/2608.26374#S3.SS1.p2.1)\.
- Gaoet al\.\(2024\)L\. Gao, J\. Tow, B\. Abbasi, S\. Biderman, S\. Black, A\. DiPofi, C\. Foster, L\. Golding, J\. Hsu, A\. Le Noac’h, H\. Li, K\. McDonell, N\. Muennighoff, C\. Ociepa, J\. Phang, L\. Reynolds, H\. Schoelkopf, A\. Skowron, L\. Sutawika, E\. Tang, A\. Thite, B\. Wang, K\. Wang, and A\. ZouThe language model evaluation harness\.Zenodo\.External Links:[Document](https://dx.doi.org/10.5281/zenodo.12608602),[Link](https://zenodo.org/records/12608602)Cited by:[§3\.1](https://arxiv.org/html/2608.26374#S3.SS1.p2.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the math dataset\.InThirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 2\),Cited by:[§3\.1](https://arxiv.org/html/2608.26374#S3.SS1.p2.1)\.
- Liet al\.\(2026\)J\. Li, X\. Dong, Y\. Zang, Y\. Cao, J\. Wang, and D\. LinBeyond fixed: training\-free variable\-length denoising for diffusion large language models\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Ic2A2gCseC)Cited by:[§4](https://arxiv.org/html/2608.26374#S4.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2025\)P\. Li, Y\. Zhou, D\. Muhtar, L\. Yin, S\. Yan, L\. Shen, Y\. Liang, S\. Vosoughi, and S\. LiuDiffusion language models know the answer before decoding\.ArXivabs/2508\.19982\.External Links:[Link](https://api.semanticscholar.org/CorpusID:280919257)Cited by:[§4](https://arxiv.org/html/2608.26374#S4.SS0.SSS0.Px2.p2.1)\.
- Liuet al\.\(2025\)Z\. Liu, Y\. Yang, Y\. Zhang, J\. Chen, C\. Zou, Q\. Wei, S\. Wang, and L\. ZhangDLLM\-cache: accelerating diffusion large language models with adaptive caching\.ArXivabs/2506\.06295\.External Links:[Link](https://api.semanticscholar.org/CorpusID:279250715)Cited by:[§4](https://arxiv.org/html/2608.26374#S4.SS0.SSS0.Px3.p1.1)\.
- Nieet al\.\(2025\)S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. LiLarge language diffusion models\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2608.26374#S1.p5.1),[§3\.1](https://arxiv.org/html/2608.26374#S3.SS1.p1.1),[§4](https://arxiv.org/html/2608.26374#S4.SS0.SSS0.Px1.p1.1)\.
- Ouet al\.\(2024\)J\. Ou, S\. Nie, K\. Xue, F\. Zhu, J\. Sun, Z\. Li, and C\. LiYour absorbing discrete diffusion secretly models the conditional distributions of clean data\.ArXivabs/2406\.03736\.Cited by:[§1](https://arxiv.org/html/2608.26374#S1.p2.1)\.
- Paszkeet al\.\(2019\)A\. Paszke, S\. Gross, F\. Massa, A\. Lerer, J\. Bradbury, G\. Chanan, T\. Killeen, Z\. Lin, N\. Gimelshein, L\. Antiga, A\. Desmaison, A\. Kopf, E\. Yang, Z\. DeVito, M\. Raison, A\. Tejani, S\. Chilamkurthy, B\. Steiner, L\. Fang, J\. Bai, and S\. ChintalaPyTorch: an imperative style, high\-performance deep learning library\.InAdvances in Neural Information Processing Systems,H\. Wallach, H\. Larochelle, A\. Beygelzimer, F\. d'Alché\-Buc, E\. Fox, and R\. Garnett \(Eds\.\),Vol\.32,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf)Cited by:[§3\.1](https://arxiv.org/html/2608.26374#S3.SS1.p1.1)\.
- Shiet al\.\(2024\)J\. Shi, K\. Han, Z\. Wang, A\. Doucet, and M\. TitsiasSimplified and generalized masked diffusion for discrete data\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2608.26374#S1.p1.1),[§2](https://arxiv.org/html/2608.26374#S2.p2.1)\.
- Suzgunet al\.\(2023\)M\. Suzgun, N\. Scales, N\. Schärli, S\. Gehrmann, Y\. Tay, H\. W\. Chung, A\. Chowdhery, Q\. Le, E\. Chi, D\. Zhou,et al\.Challenging big\-bench tasks and whether chain\-of\-thought can solve them\.InFindings of the Association for Computational Linguistics: ACL 2023,pp\. 13003–13051\.Cited by:[§3\.1](https://arxiv.org/html/2608.26374#S3.SS1.p2.1)\.
- Weiet al\.\(2026\)Q\. Wei, Y\. Zhang, Z\. Liu, P\. Zeng, Y\. Wang, B\. Qi, D\. Liu, and L\. ZhangAccelerating diffusion large language models with slowfast sampling: the three golden principles\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 4340–4354\.Cited by:[§4](https://arxiv.org/html/2608.26374#S4.SS0.SSS0.Px3.p1.1)\.
- Wolfet al\.\(2020\)T\. Wolf, L\. Debut, V\. Sanh, J\. Chaumond, C\. Delangue, A\. Moi, P\. Cistac, T\. Rault, R\. Louf, M\. Funtowicz, J\. Davison, S\. Shleifer, P\. von Platen, C\. Ma, Y\. Jernite, J\. Plu, C\. Xu, T\. L\. Scao, S\. Gugger, M\. Drame, Q\. Lhoest, and A\. M\. RushTransformers: state\-of\-the\-art natural language processing\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,pp\. 38–45\.Cited by:[§3\.1](https://arxiv.org/html/2608.26374#S3.SS1.p1.1)\.
- Wuet al\.\(2025\)C\. Wu, H\. Zhang, S\. Xue, Z\. Liu, S\. Diao, L\. Zhu, P\. Luo, S\. Han, and E\. XieFast\-dllm: training\-free acceleration of diffusion llm by enabling kv cache and parallel decoding\.ArXivabs/2505\.22618\.External Links:[Link](https://api.semanticscholar.org/CorpusID:278959508)Cited by:[§4](https://arxiv.org/html/2608.26374#S4.SS0.SSS0.Px3.p1.1)\.
- Yeet al\.\(2025\)J\. Ye, Z\. Xie, L\. Zheng, J\. Gao, Z\. Wu, X\. Jiang, Z\. Li, and L\. KongDream 7b: diffusion large language models\.ArXivabs/2508\.15487\.Cited by:[§1](https://arxiv.org/html/2608.26374#S1.p5.1),[§3\.1](https://arxiv.org/html/2608.26374#S3.SS1.p1.1),[§4](https://arxiv.org/html/2608.26374#S4.SS0.SSS0.Px1.p1.1)\.

## Appendix AExpectation as a sum of survivals

LetLLbe an integer\-valued random variable taking values in\{1,…,T\}\\\{1,\\dots,T\\\}with

πk\\displaystyle\\pi\_\{k\}=ℙ\(L=k\),\\displaystyle=\\mathbb\{P\}\(L=k\),\\qquadS⁡\(k\)\\displaystyle S\(k\)=ℙ\(L\>k\),k=0,…,T\.\\displaystyle=\\mathbb\{P\}\(L\>k\),\\quad k=0,\\dots,T\.
Recall that in our constructionS⁡\(k\)=∏i=1k\(1−hi\)S\(k\)=\\prod\_\{i=1\}^\{k\}\(1\-h\_\{i\}\)and henceS⁡\(k−1\)=ℙ⁡\(L≥k\)S\(k\-1\)=\\mathbb\{P\}\(L\\geq k\)\.

###### Lemma 1\.

𝔼⁡\[L\]=∑k=1Tk​πk=∑k=1TS⁡\(k−1\)\.\\mathbb\{E\}\[L\]\\;=\\;\\sum\_\{k=1\}^\{T\}k\\,\\pi\_\{k\}\\;=\\;\\sum\_\{k=1\}^\{T\}S\(k\-1\)\.

###### Proof\.

By definition of mathematical expectation,

𝔼⁡\[L\]=∑k=1Tk​πk\.\\mathbb\{E\}\[L\]=\\sum\_\{k=1\}^\{T\}k\\,\\pi\_\{k\}\.Then we can write an integerkkas a sum ofkkones:

k=∑j=1k1,k=\\sum\_\{j=1\}^\{k\}1,and substitute into the expectation:

𝔼⁡\[L\]=∑k=1T\(∑j=1k1\)​πk\.\\mathbb\{E\}\[L\]=\\sum\_\{k=1\}^\{T\}\\left\(\\sum\_\{j=1\}^\{k\}1\\right\)\\pi\_\{k\}\.After swapping the order of summation we get:

𝔼⁡\[L\]=∑j=1T∑k=jTπk\.\\mathbb\{E\}\[L\]=\\sum\_\{j=1\}^\{T\}\\sum\_\{k=j\}^\{T\}\\pi\_\{k\}\.The inner sum is the tail probability

∑k=jTπk=ℙ⁡\(L≥j\)=ℙ⁡\(L\>j−1\)=S⁡\(j−1\)\.\\sum\_\{k=j\}^\{T\}\\pi\_\{k\}=\\mathbb\{P\}\(L\\geq j\)=\\mathbb\{P\}\(L\>j\-1\)=S\(j\-1\)\.Therefore

𝔼⁡\[L\]=∑j=1TS⁡\(j−1\),\\mathbb\{E\}\[L\]=\\sum\_\{j=1\}^\{T\}S\(j\-1\),which is exactly the claim of the Lemma\. ∎

## Appendix BAlgorithm for length prediction plug\-in

Algorithm 1Survival\-guided length prediction for diffusion LMs0:Prompt tokens

𝐱1:P\\mathbf\{x\}\_\{1:P\}, maximal horizon

TT, EOS token id

0:Predicted number of new tokens

L^∈\{1,…,T\}\\hat\{L\}\\in\\\{1,\\dots,T\\\}
1:Input canvas\.Form the initial canvas

𝐱\(0\)=\[𝐱1:P,\[MASK\],…,\[MASK\]\]\\mathbf\{x\}^\{\(0\)\}=\[\\mathbf\{x\}\_\{1:P\},\\texttt\{\[MASK\]\},\\dots,\\texttt\{\[MASK\]\}\]of total length

P\+TP\+T\(prompt \+

TTmasked positions\)\.

2:Single forward pass\.Run the diffusion LM once on

x\(0\)x^\{\(0\)\}at the initial diffusion time to obtain logits

𝐳t\(0\),t=1,…,P\+T\.\\mathbf\{z\}^\{\(0\)\}\_\{t\},\\qquad t=1,\\dots,P\+T\.
3:EOS probabilities\.For each candidate generation position

t=P\+1,…,P\+Tt=P\{\+\}1,\\dots,P\{\+\}T, compute

pt←softmax⁡\(𝐳t\(0\)\)​\[\[EOS\]\]\.p\_\{t\}\\leftarrow\\mathrm\{softmax\}\(\\mathbf\{z\}^\{\(0\)\}\_\{t\}\)\[\\texttt\{\[EOS\]\}\]\.
4:Plug\-in hazards\.For relative index

k=1,…,Tk=1,\\dots,T, set

hk←pP\+kh\_\{k\}\\leftarrow p\_\{P\+k\}as the plug\-in estimate of the hazard

P⁡\(L=k∣L≥k,prompt\)P\(L=k\\mid L\\geq k,\\text\{prompt\}\)\.

5:Survival curve\.Initialize

S⁡\(0\)←1S\(0\)\\leftarrow 1\. For

k=1,…,Tk=1,\\dots,T:

S⁡\(k\)←S⁡\(k−1\)​\(1−hk\)\.S\(k\)\\leftarrow S\(k\-1\)\\,\\bigl\(1\-h\_\{k\}\\bigr\)\.
6:Truncated expectation\.Compute

L^cont←∑k=1TS⁡\(k−1\)\.\\hat\{L\}\_\{\\text\{cont\}\}\\;\\leftarrow\\;\\sum\_\{k=1\}^\{T\}S\(k\-1\)\.
7:Post\-processing\.Clip and round to an integer horizon:

L^cont←min⁡\(max⁡\(L^cont,1\),T\),L^←⌈L^cont⌉\.\\hat\{L\}\_\{\\text\{cont\}\}\\leftarrow\\min\\bigl\(\\max\(\\hat\{L\}\_\{\\text\{cont\}\},1\),T\\bigr\),\\qquad\\hat\{L\}\\leftarrow\\big\\lceil\\hat\{L\}\_\{\\text\{cont\}\}\\big\\rceil\.
8:return

L^\\hat\{L\}

## Appendix CAdditional Results

### C\.1Comparison with DAEDAL

Conceptually, DAEDAL also addresses length, but it relies on an iterative length\-expansion procedure with a tuned stopping threshold, and its stopping criterion depends on how reliably\[EOS\]is produced under the model’s decoding regime\. In our experiments on DLMs\-Base,\[EOS\]is often not emitted as an argmax early enough to support such thresholding reliably, whereas our estimator uses soft\[EOS\]probabilities and therefore applies uniformly to both base and instruction\-tuned variants\. This is a conceptual advantage of our approach over DAEDAL, which is only applicable to instruction\-tuned models\. Nevertheless, we evaluate against DAEDAL on instruction\-tuned variants of LLaDA and Dream\. Tables[4](https://arxiv.org/html/2608.26374#A3.T4)and[5](https://arxiv.org/html/2608.26374#A3.T5)report decoding time and task accuracy, respectively\.

Table 4:Decoding speed \(seconds per sample\) on instruction\-tuned models for DAEDAL and our length prediction method\. Parentheses report speedup relative to the baseline decoder without \(w/o\) length prediction\.Table 5:Task performance on instruction\-tuned models of baseline without length prediction compared with both DAEDAL and our survival\-guided length selection method\.First, both methods preserve task performance relative to the corresponding baseline decoder\. Second, our method achieves larger end\-to\-end speedups\. This difference is expected from the structure of the methods: DAEDAL performs an iterative search over candidate lengths and depends on a thresholded stopping rule, whereas our estimator uses a single forward pass on a conservative canvas and converts the resulting per\-position\[EOS\]probabilities into a closed\-form expected length\.

### C\.2Predictor Step Ablation

To further probe the stability of the length signal, we recompute the expected length from later denoising stepss∈\{2,5,7,10\}s\\in\\\{2,5,7,10\\\}and compare it to the estimate from the initial forward pass using LLaDa\-Base model\.

Table 6:Absolute change in predicted lengthδ​L^\(s\)=L^\(s\)−L^\(0\)\\delta\\hat\{L\}^\{\(s\)\}=\\hat\{L\}^\{\(s\)\}\-\\hat\{L\}^\{\(0\)\}when the estimator is recomputed from later denoising steps for LLaDA\.Table[6](https://arxiv.org/html/2608.26374#A3.T6)shows that the implied termination signal does not drift substantially during early decoding\. These results support the use of a single\-pass estimator and suggest that the initial\[EOS\]probabilities already contain most of the information needed for practical length selection\.