ELF-REG：将连续扩散语言模型扩展至推理任务

# Scaling Continuous DiffusionLanguage Models to Reasoning Tasks
Source: [https://arxiv.org/html/2609.29102](https://arxiv.org/html/2609.29102)
## ELF\-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks

Zeyu Michael Li††thanks:Correspondence to Zeyu Michael Li atzeyu \[dot\] li030 \[at\] duke\.edu\.William Xingxu ChenAffiliation:Duke UniversityBingshuo QianAffiliation:Duke UniversityJiayin LiuAffiliation:Tsinghua UniversityXiang ChengAffiliation:Duke University

###### Abstract

Fully continuous diffusion language models \(dLMs\) denoise continuous representations without intermediate discretization, then decode all response tokens in parallel at the final step\. Their performance on challenging reasoning tasks remains less established than that of autoregressive \(AR\) LLMs and masked dLMs\. We scale Embedded Language Flows \(ELF\) to mathematical reasoning and code generation on GSM8K, MATH\-500, HumanEval, and MBPP\. We introduce ELF\-REG, which improves learning with representation alignment and entanglement \(REPA\+REG\), where a frozen AR teacher supervises intermediate denoiser features and supplies a global representation that is jointly denoised with the response\. ELF\-REG\-L achieves 55\.96% pass@1 on GSM8K at 64 network function evaluations \(NFE\), and 13\.39% on MATH\-500 and 22\.56% on HumanEval at 128 NFE\. It outperforms the evaluated comparable\-scale dLMs in pass@1 on GSM8K and code, and improves MATH\-500 pass@1 from 10\.55% for the ELF\-L baseline to 13\.39% with ELF\-REG\-L\. Without few\-step training, the same task\-specific checkpoints support strong low\-NFE performance through early\-stop, which decodes an intermediate clean prediction without completing the denoising trajectory\. At 16 NFE, ELF\-REG\-L reaches 41\.21% HumanEval pass@10, outperforming recent continuous dLMs of comparable scale\.

\(a\) Representation alignment and entanglement ![Refer to caption](https://arxiv.org/html/2609.29102v1/method_fig.png)

Figure 1:ELF\-REG improves generation with fewer task\-training tokens\.\(a\) A frozen AR teacher supervises intermediate features through REPA and a global REG token denoised jointly with the response\. \(b,c\) ELF\-REG\-L and ELF\-REG\-B with early\-stop \(ρ=8\\rho=8\) show superior performance across NFE, compared with PlaidQ\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\], FMLM\+\[[Agarwal et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib1)\], S\-FLM\[[Deschenaux & Gulcehre, 2026](https://arxiv.org/html/2609.29102#bib.bib14)\], and DBTM\[[Tang & Wang, 2026](https://arxiv.org/html/2609.29102#bib.bib46)\]\. Code uses pass@10 on MBPP\-378; GSM8K uses mean pass@1\. \(d\) Non\-padding training token budgets\. Appendix[B](https://arxiv.org/html/2609.29102#A2)gives more details regarding benchmark mappings, model sizes, NFE accounting, budget estimates, and evaluation differences\.## 1Introduction

Diffusion language models \(dLMs\) offer an efficient approach to parallel text generation by iteratively refining a corrupted sequence, in contrast to autoregressive \(AR\) LLMs that decode tokens in sequence\. Continuous dLMs carry out this refinement in a representation space, bringing techniques from continuous diffusion to language and improving parallelism by updating all token positions simultaneously during denoising\. Much of the development of continuous dLMs has focused on language modeling and conditional text generation\. Plaid\[[Gulrajani & Hashimoto, 2023](https://arxiv.org/html/2609.29102#bib.bib20)\]studies likelihood\-based language modeling, while LangFlow, LDLM, and Embedded Language Flows \(ELF\) evaluate unconditional generation using perplexity and entropy\[[Chen et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib12);[Meshchaninov et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib36);[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\]\. Conditional applications include machine translation with CDCD\[[Dieleman et al\., 2022](https://arxiv.org/html/2609.29102#bib.bib15)\], simplification and paraphrasing with DiffuSeq\[[Gong et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib19)\], and translation and summarization with ELF\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\]\. Together, these studies establish continuous denoising as a viable approach to producing and transforming text\.

Masked dLMs such as LLaDA and Dream already demonstrate high\-performing reasoning and code generation\[[Nie et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib38);[Ye et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib52)\], while recent continuous dLM works’ performance on GSM8K and code generation \(S\-FLM, FMLM\+, MLFM, PlaidQ\[[Deschenaux & Gulcehre, 2026](https://arxiv.org/html/2609.29102#bib.bib14);[Agarwal et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib1);[Azangulov et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib7);[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\]\) demonstrates progress in mathematical reasoning with continuous dLMs, giving reason to pursue reasoning capabilities of continuous dLMs\. These advances show promise for continuous dLMs on reasoning tasks and motivate improving their accuracy at low compute scales, including with limited denoising budget\.

In this work, we extend ELF’s\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\]fully continuous dLM to mathematical reasoning on GSM8K and MATH\-500 and to code generation on HumanEval and MBPP\. We use representation alignment and entanglement \(REPA\+REG\) to improve performance: a frozen AR teacher supervises intermediate features through REPA\[[Yu et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib54)\]and supplies a global representation jointly denoised with the response through REG\[[Wu et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib50)\]\. We call this methodELF\-REGand evaluate two sizes, ELF\-REG\-B and ELF\-REG\-L\. At 64 NFE, our ELF\-REG\-B reaches 38\.11% GSM8K pass@1, exceeding FMLM\+\[[Agarwal et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib1)\]at the same NFE and MDLM\[[Sahoo et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib43)\]and Duo\[[Sahoo et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib44)\]at 1024 NFE with comparable parameter count\. ELF\-REG\-L reaches 55\.96% at 64 NFE \(Table[2](https://arxiv.org/html/2609.29102#S4.T2)\)\. On code, ELF\-REG\-L achieves the highest pass@1 among the comparable\-scale dLMs on all four benchmarks at 128 NFE \(Table[1](https://arxiv.org/html/2609.29102#S4.T1)\)\. On MBPP\-500 and MBPP\-378, ELF\-REG\-L at 32 NFE already surpasses PlaidQ\+CFG’s\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\]results at 257 NFE \(Table[14](https://arxiv.org/html/2609.29102#A4.T14)\)\. ELF\-REG\-L also improves MATH\-500 accuracy over ELF\-L baseline, demonstrating its performance on more complex tasks\.

Strong performance also extends to low NFE without dedicated few\-step training\. Building on prior early stopping works for continuous dLMs\[[Vaina et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib48);[Gao et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib17);[Du & Ma, 2026](https://arxiv.org/html/2609.29102#bib.bib16)\], we use early\-stop to decode an intermediate clean prediction and analyse its benefits\. On GSM8K, continuing denoising beyond this exit substantially increases agreement with the final response but adds little aggregate accuracy \(Figure[2](https://arxiv.org/html/2609.29102#S3.F2)\)\. Thus, useful task performance can emerge before responses stabilize\. We use a flow\-map formulation to analyse early\-stop endpoint error and apply the decoder\-margin bound of[Du & Ma \[2026, Theorem 1\]](https://arxiv.org/html/2609.29102#bib.bib16)to study decoded\-output agreement\. Our contributions are:111Code:[https://anonymous\.4open\.science/r/scaling\_dLM\-9B14](https://anonymous.4open.science/r/scaling_dLM-9B14)\.

- •We advance fully continuous dLMs on mathematical reasoning and code, outperforming other dLMs of comparable scale in our comparisons of GSM8K and code pass@1, with further gains over ELF\-L baseline on MATH\-500\.
- •Without dedicated few\-step training, ELF\-REG\-L reaches 46\.39% pass@10 on MBPP\-378 using early\-stop at 8 NFE, exceeding PlaidQ\-D16’s 40\.49% at 17 NFE \(Table[15](https://arxiv.org/html/2609.29102#A4.T15)\)\.
- •On GSM8K, we show that accuracy approaches its final level while responses continue changing\. We analyse endpoint error and decoded\-output agreement using decoder margins and connections between early\-stop and flow maps\.

## 2Methodology

### 2\.1ELF preliminaries

We largely follow ELF\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\]for its architecture, training, and sampling\. We provide a brief recap here\. ELF\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\]generates continuous response representations from Gaussian noise while holding the prompt fixed\. A frozen Qwen3\-0\.6B\-Base encoder supplies clean text statesxx\. Training corrupts response states with noise scaleσ\\sigmaas

zt=t​x\+\(1−t\)​σ​ε,ε∼𝒩⁡\(0,I\),t∈\[0,1\]\.z\_\{t\}=tx\+\(1\-t\)\\sigma\\varepsilon,\\qquad\\varepsilon\\sim\\mathcal\{N\}\(0,I\),\\quad t\\in\[0,1\]\.\(1\)A bidirectional transformer predicts clean statesx^θ\\hat\{x\}\_\{\\theta\}, giving velocityvθ=\(x^θ−zt\)/\(1−t\)v\_\{\\theta\}=\(\\hat\{x\}\_\{\\theta\}\-z\_\{t\}\)/\(1\-t\)fort<1t<1\. We retain ELF’s self\-conditioning guidance, using each clean prediction to condition the next sampling update\. The same transformer learns to decode representations through token cross\-entropy; these decoding examples and velocity regression defineℒELF\\mathcal\{L\}\_\{\\mathrm\{ELF\}\}\. One final decoder evaluation produces all response tokens in parallel\.

### 2\.2Representation alignment and entanglement

A separate frozen Qwen3\-1\.7B\-Base AR teacher supplies REPA\+REG targets \(Figure[1](https://arxiv.org/html/2609.29102#S0.F1)\(a\)\)\. REPA\[[Yu et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib54)\]aligns student statesui\(ℓ\)u\_\{i\}^\{\(\\ell\)\}after blockℓ\\ellwith clean teacher stateshih\_\{i\}at the same token position through a learned projectorpϕp\_\{\\phi\}into the teacher dimension, minimizing1−cos⁡\(pϕ​\(ui\(ℓ\)\),hi\)1\-\\cos\(p\_\{\\phi\}\(u\_\{i\}^\{\(\\ell\)\}\),h\_\{i\}\)\. The student retains bidirectional attention\. Inspired by REG\[[Wu et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib50)\], we standardize the teacher’s last content\-token state to form a global targetrr, corrupted asrt=t​r\+\(1−t\)​σ​εrr\_\{t\}=tr\+\(1\-t\)\\sigma\\varepsilon\_\{r\}whereεr∼𝒩⁡\(0,I\)\\varepsilon\_\{r\}\\sim\\mathcal\{N\}\(0,I\)\. An additional token exchanges information with text through self\-attention and predicts the clean REG state, supervised by velocity regression with the same self\-conditioning guidance as text\. REPA also supervises this token\. We optimize

ℒ=ℒELF\+λREPA​ℒREPA\+λREG​ℒREG\.\\mathcal\{L\}=\\mathcal\{L\}\_\{\\mathrm\{ELF\}\}\+\\lambda\_\{\\mathrm\{REPA\}\}\\mathcal\{L\}\_\{\\mathrm\{REPA\}\}\+\\lambda\_\{\\mathrm\{REG\}\}\\mathcal\{L\}\_\{\\mathrm\{REG\}\}\.\(2\)Text and REG start from noise at inference; REG remains available to the decoder, which emits only text\. The teacher and projector are training\-only\. Appendix[A](https://arxiv.org/html/2609.29102#A1)specifies guidance targets, loss normalization, and numerical implementation; Table[6](https://arxiv.org/html/2609.29102#A2.T6)gives target layers and defaults\.

### 2\.3Prefix early\-stop generation

For an actual budget ofNNnetwork function evaluations \(NFE\), sample a logit\-normal grid of denoising timesteps withM−1M\-1updates for master budgetM=ρ​NM=\\rho N\. Execute its firstN−1N\-1denoiser updates and decode the latest predicted\-clean text and REG states at decoder time11\. We useρ=8\\rho=8\. Full\-span generation instead traverses the complete grid for budgetNNand decodes the final updated states\. Early\-stop is a plug\-and\-play inference technique and does not require few\-step training\. NFE includes one final decoder evaluation; note thatρ\\rhorelates evaluation budgets in NFE, not fractions of continuous time\. Algorithm[1](https://arxiv.org/html/2609.29102#algorithm1)gives the generation procedure following ELF\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24), Algorithm 2\]with our prefix early\-stop modification\.

## 3Analysis

We relate early\-stop decoding to the remaining flow map, then examine endpoint error, decoder stability, and task accuracy along the sampling trajectory\. In the result displays, pass@1 is answer accuracy averaged over questions and generation seeds\. See Appendix[B\.6](https://arxiv.org/html/2609.29102#A2.SS6)for evaluation settings\.

### 3\.1Early\-stop decoding and the remaining flow map

We begin with a smooth continuous flow, with the prompt and guidance scale fixed\. Letyty\_\{t\}denote its generated state: response latents together with REG when present\. The prompt is a fixed input, outside this state\. The flow mapFs,tF\_\{s,t\}transports a state from timessto timettunder the velocity fieldvv\[[Boffi et al\., 2025a](https://arxiv.org/html/2609.29102#bib.bib8);[Boffi et al\., 2025b](https://arxiv.org/html/2609.29102#bib.bib9);[Lee et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib30)\]\. Starting from Gaussian noisey0=σ​εy\_\{0\}=\\sigma\\varepsilon, withε∼𝒩⁡\(0,I\)\\varepsilon\\sim\\mathcal\{N\}\(0,I\), we compare the endpointy1y\_\{1\}att=1t=1with the clean predictiony^t\\hat\{y\}\_\{t\}made at exit timet<1t<1:

yt=F0,t​\(y0\),y1=Ft,1​\(yt\),y^t=Dt​\(yt\)\.y\_\{t\}=F\_\{0,t\}\(y\_\{0\}\),\\qquad y\_\{1\}=F\_\{t,1\}\(y\_\{t\}\),\\qquad\\hat\{y\}\_\{t\}=D\_\{t\}\(y\_\{t\}\)\.\(3\)HereDtD\_\{t\}is the denoiser’s clean predictor, including its text and REG outputs\. The predictiony^t\\hat\{y\}\_\{t\}is made at timettand targets the clean endpoint\. Let𝒟\\mathcal\{D\}denote the deterministic parallel token decoder at decoder time11\. Continuing the flow tot=1t=1and decoding the endpoint returns𝒟⁡\(y1\)\\mathcal\{D\}\(y\_\{1\}\), and early\-stop returns𝒟⁡\(y^t\)\\mathcal\{D\}\(\\hat\{y\}\_\{t\}\)\. Thus early\-stop replaces the remaining flow map with a clean prediction before applying the same decoder\. Flow\-map models learn finite\-time transport explicitly\[[Lee et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib30)\]; here we examine the error of using the existing predictor for that transport\. The continuous formulation usesDt​\(y\)=y\+\(1−t\)​v​\(y,t\)D\_\{t\}\(y\)=y\+\(1\-t\)v\(y,t\)\. Appendix[C\.3](https://arxiv.org/html/2609.29102#A3.SS3)treats the self\-conditioned sampler\. We denote the Euclidean norm over the full generated state with∥⋅∥\\\|\\cdot\\\|, including text positions after EOS and REG when present\.

###### Proposition 1\(Endpoint error\)\.

Supposevvis continuously differentiable and its flow exists uniquely through time11\. For a stateyyat timet<1t<1, writeys=Ft,s​\(y\)y\_\{s\}=F\_\{t,s\}\(y\)and define the average remaining velocity asv¯t:1\(y\)=\(1−t\)−1∫t1v\(ys,s\)ds\\bar\{v\}\_\{t:1\}\(y\)=\(1\-t\)^\{\-1\}\\int\_\{t\}^\{1\}v\(y\_\{s\},s\)\\,ds\. The flow map satisfiesFt,1\(y\)=y\+∫t1v\(Ft,s\(y\),s\)ds=y\+\(1−t\)v¯t:1\(y\)F\_\{t,1\}\(y\)=y\+\\int\_\{t\}^\{1\}v\(F\_\{t,s\}\(y\),s\)\\,ds=y\+\(1\-t\)\\bar\{v\}\_\{t:1\}\(y\)\. Hence

et\(y\):=Dt\(y\)−Ft,1\(y\)=\(1−t\)\(v\(y,t\)−v¯t:1\(y\)\)\.e\_\{t\}\(y\):=D\_\{t\}\(y\)\-F\_\{t,1\}\(y\)=\(1\-t\)\\bigl\(v\(y,t\)\-\\bar\{v\}\_\{t:1\}\(y\)\\bigr\)\.\(4\)Letas=dd​s​v​\(ys,s\)=∂sv⁡\(ys,s\)\+Jy​v​\(ys,s\)​v​\(ys,s\)a\_\{s\}=\\frac\{d\}\{ds\}v\(y\_\{s\},s\)=\\partial\_\{s\}v\(y\_\{s\},s\)\+J\_\{y\}v\(y\_\{s\},s\)v\(y\_\{s\},s\)be the acceleration along this trajectory, whereJy​vJ\_\{y\}vis the Jacobian ofvvwith respect to the state\. If‖as‖≤A\\\|a\_\{s\}\\\|\\leq Afor everys∈\[t,1\]s\\in\[t,1\], then‖et​\(y\)‖≤A​\(1−t\)2/2\\\|e\_\{t\}\(y\)\\\|\\leq A\(1\-t\)^\{2\}/2\.

The flow map integrates velocity along the evolving trajectory\. The clean predictor instead takes one Euler step across the remaining interval using the current velocity\. This step reaches the endpoint att=1t=1exactly when the current velocity equals the average remaining velocity\. The acceleration bound quantifies the error when the velocity changes along the trajectory\. Near the endpoint, the remaining interval is short\. At an early exit, the bound is informative when the velocity changes little over the longer remaining interval\. Flow\-matching training alone does not ensure this condition\.

Decoding can preserve a response even when the predicted endpoint differs from the endpoint att=1t=1\. At each response position, the decoder chooses the token with the largest logit\. Letmmbe the smallest gap between that winning logit and any competing logit aty1y\_\{1\}, over valid text positions and \(if present\) the first EOS token\. A larger marginmmallows more change in the logits before a token changes\. Proposition[2](https://arxiv.org/html/2609.29102#Thmproposition2)follows from the decoder\-margin argument of[Du & Ma \[2026, Theorem 1\]](https://arxiv.org/html/2609.29102#bib.bib16)\.

###### Proposition 2\(Decoded\-output agreement\)\.

Supposem\>0m\>0\. LetL≥0L\\geq 0be a finite pairwise\-logit Lipschitz constant: each winning\-token versus competing\-token logit difference changes by at mostL​‖a−b‖L\\\|a\-b\\\|between any two statesa,ba,bon the segment joiningy^t\\hat\{y\}\_\{t\}andy1y\_\{1\}\. IfL​‖et​\(yt\)‖<mL\\\|e\_\{t\}\(y\_\{t\}\)\\\|<m, then

𝒟⁡\(y^t\)=𝒟⁡\(y1\)\.\\mathcal\{D\}\(\\hat\{y\}\_\{t\}\)=\\mathcal\{D\}\(y\_\{1\}\)\.\(5\)

###### Corollary 1\(Distributional agreement\)\.

Fort<1t<1letPtP\_\{t\}andP1P\_\{1\}be the distributions ofy^t\\hat\{y\}\_\{t\}andy1y\_\{1\}induced by initial noiseε∼𝒩⁡\(0,I\)\\varepsilon\\sim\\mathcal\{N\}\(0,I\), with finite second moments\. Coupling the endpoints through the same initial noise bounds their 2\-Wasserstein distance:

W2​\(Pt,P1\)≤𝔼​‖et​\(yt\)‖2\.W\_\{2\}\(P\_\{t\},P\_\{1\}\)\\leq\\sqrt\{\\mathbb\{E\}\\\|e\_\{t\}\(y\_\{t\}\)\\\|^\{2\}\}\.\(6\)LetQt,Q1Q\_\{t\},Q\_\{1\}be the distributions of𝒟⁡\(y^t\),𝒟⁡\(y1\)\\mathcal\{D\}\(\\hat\{y\}\_\{t\}\),\\mathcal\{D\}\(y\_\{1\}\)\. If the margin and Lipschitz assumptions of Proposition[2](https://arxiv.org/html/2609.29102#Thmproposition2)hold almost surely, then their total variation satisfiesTV\(Qt,Q1\)≤Pr\[L∥et\(yt\)∥≥m\]\\operatorname\{TV\}\(Q\_\{t\},Q\_\{1\}\)\\leq\\Pr\[L\\\|e\_\{t\}\(y\_\{t\}\)\\\|\\geq m\]\.

Corollary[1](https://arxiv.org/html/2609.29102#Thmcorollary1)bounds latent distributional discrepancy in Wasserstein distance and decoded distributional discrepancy in total variation\. The decoder condition permits latent differences that preserve the winning tokens\. Appendix[C](https://arxiv.org/html/2609.29102#A3)gives the proofs and a bound on the decoder failure probability\.

Proposition[2](https://arxiv.org/html/2609.29102#Thmproposition2)gives a sufficient condition for preserving the complete endpoint response, whether that response is correct or incorrect\. Different responses can nevertheless yield the same numerical answer\. We therefore compare extracted answers and changes in correctness on matched question–seed pairs, alongside complete\-response agreement\.

### 3\.2Endpoint error and decoder stability along the sampling trajectory

We compare early\-stop predictions with the endpoints of their own sampling trajectories\. Figure[2](https://arxiv.org/html/2609.29102#S3.F2)\(b–e\) follows the ELF\-B baseline and ELF\-REG\-B on GSM8K through and beyond the early\-stop exit\. Text endpoint error is the Euclidean distance between the predicted\-clean text and the final updated text state, over the full response canvas including positions after EOS\. This measures the text component of the discrete endpoint discrepancy in equation[14](https://arxiv.org/html/2609.29102#A3.E14)\. Intermediate observations decode predicted\-clean states; the endpoint decodes the final updated state\. A “stable token” agrees with the endpoint at every subsequent observation\. Endpoint response agreement instead requires an identical token sequence, EOS status, and length at the current observation\.

Figure 2:ELF\-REG\-B has lower discrete acceleration and lower early\-stop endpoint error on GSM8K\. \(a\) Median local joint acceleration with uniform time sampling over 512 Euler updates\. \(b–e\) Mean text endpoint error, stable tokens, endpoint response agreement, and pass@1, using logit\-normal time samples as specified in Appendix[A\.4](https://arxiv.org/html/2609.29102#A1.SS4)\. Stable tokens are those that agree with the endpoint at every later observation; response agreement requires the current sequence, EOS status, and length to match\. Dashed lines in \(b–e\) mark NFE 64; continuing beyond this exit increases agreement substantially but adds little accuracy\. Results average over four seeds: the median across questions in \(a\) and the mean across questions in \(b–e\)\. At NFE 512 in \(b–e\), the updated endpoint gives zero error and full agreement by construction\.Withρ=8\\rho=8, the NFE\-64 exit occurs early in denoising timettdue to the non\-uniform denoising schedule: the prediction time averagest=0\.0825t=0\.0825across four seeds\. Across these schedules, the nominal SNRt2/\[σ2​\(1−t\)2\]t^\{2\}/\[\\sigma^\{2\}\(1\-t\)^\{2\}\]averages2\.03×10−32\.03\\times 10^\{\-3\}forσ=2\\sigma=2\(Table[11](https://arxiv.org/html/2609.29102#A3.T11)\)\. Thus the exit corresponds to a noise\-dominated training corruption level\. Appendix[C\.4](https://arxiv.org/html/2609.29102#A3.SS4.SSS0.Px3)gives additional details\.

At NFE 64, ELF\-REG\-B has 16\.1% lower mean text endpoint error than the baseline \(Figure[2](https://arxiv.org/html/2609.29102#S3.F2)b\)\. Its endpoint response agreement is 8\.61%, compared with 1\.46% for the baseline \(Figure[2](https://arxiv.org/html/2609.29102#S3.F2)d\)\. ELF\-REG\-B also reaches token stability earlier \(Figure[2](https://arxiv.org/html/2609.29102#S3.F2)c\)\. Beyond the early exit at NFE 64, its mean text error decreases as response agreement rises, whereas the baseline’s mean text error instead rises between NFE 64\-257, even as its stable\-token fraction increases\.

To examine the acceleration term in Proposition[1](https://arxiv.org/html/2609.29102#Thmproposition1), we measure consecutive velocity differences divided by time\-difference with uniform time sampling over 512 Euler updates\. Figure[2](https://arxiv.org/html/2609.29102#S3.F2)\(a\) shows lower local joint acceleration for ELF\-REG\-B through the early and middle trajectory\. Att=5/64t=5/64, its median maximum remaining joint acceleration is 84\.8% lower than ELF\-B baseline, accompanied by 7\.5% lower mean joint endpoint error \(Table[12](https://arxiv.org/html/2609.29102#A3.T12)\)\. Smaller remaining acceleration thus accompanies smaller endpoint error, consistent with Proposition[1](https://arxiv.org/html/2609.29102#Thmproposition1)\. Note that these measurements are finite differences along the discrete sampler; Proposition[1](https://arxiv.org/html/2609.29102#Thmproposition1)concerns a smooth continuous flow\. See Appendix[C\.4](https://arxiv.org/html/2609.29102#A3.SS4.SSS0.Px4)for details regarding the discrete acceleration measurements\.

At NFE 64, numerical answers match those at NFE 512 on 80\.27% of question–seed pairs for ELF\-B baseline and 87\.11% for ELF\-REG\-B \(Table[10](https://arxiv.org/html/2609.29102#A3.T10)\)\. Correctness gains and losses during further denoising nearly cancel for ELF\-B baseline, while ELF\-REG\-B gains 0\.64 percentage points\. Most numerical answers therefore match the endpoint even though complete responses usually differ\. Proposition[2](https://arxiv.org/html/2609.29102#Thmproposition2)gives a sufficient condition for preserving the complete response\. Table[10](https://arxiv.org/html/2609.29102#A3.T10)numerical answer agreement results show that early\-stop often preserves the numerical answer without preserving every token, so complete\-response agreement is a stronger requirement than preserving the task answer\.

## 4Experiments

We compare ELF\-REG with other dLMs on math reasoning and code generation across NFE budgets, and measure its gains over ELF baseline\. Additional MMLU results appear in Appendix[D\.1](https://arxiv.org/html/2609.29102#A4.SS1)\. We train separate ELF baseline and ELF\-REG variants for GSM8K, code, MATH, and MMLU; the MATH variants are initialized from their corresponding GSM8K weights\. Our headline math\-reasoning and code results use zero\-shot early\-stop generation withρ=8\\rho=8, at NFE 64 for GSM8K\[[Cobbe et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib13)\]and NFE 128 for MATH\-500\[[Hendrycks et al\., 2021b](https://arxiv.org/html/2609.29102#bib.bib23);[Lightman et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib33)\], MBPP\-378 & MBPP\-500\[[Austin et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib5);[Liu et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib34)\], and HumanEval\(\+\)\[[Chen et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib11);[Liu et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib34)\]\. We report mean pass@1 over sixteen samples per question\. Appendix[B\.2](https://arxiv.org/html/2609.29102#A2.SS2)gives training data, budgets, and initialization; Table[6](https://arxiv.org/html/2609.29102#A2.T6)and Appendix[B\.3](https://arxiv.org/html/2609.29102#A2.SS3)specify hyperparameters and evaluation protocols\.

We list the performance of various AR LLMs and large dLMs in Tables[1](https://arxiv.org/html/2609.29102#S4.T1)\-[2](https://arxiv.org/html/2609.29102#S4.T2)\. We study task\-specific training of small continuous dLMs for math reasoning and code generation\. The AR LLMs/large dLMs also receive math and code training within much larger training\-data budgets and are not comparable with our method\[[Yang et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib51);[Allal et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib3);[Nie et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib38);[Ye et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib52)\]\. Tables[2](https://arxiv.org/html/2609.29102#S4.T2)and[1](https://arxiv.org/html/2609.29102#S4.T1)separate these references from comparable\-scale dLMs\. Other baselines use different settings for prompting, response budgets, and code post\-processing; Appendix[B\.5](https://arxiv.org/html/2609.29102#A2.SS5)details these settings\.

Table 1:Zero\-shot code generation, pass@1 \(%\)\. Our evaluations report means, with standard deviation across 16 seeds in parentheses\. Slashes separate base/instruct names and scores; Qwen3\-0\.6B results use non\-thinking mode\. Bold marks the best comparable\-scale dLM mean per benchmark\. Both MBPP columns use base tests; HumanEval\+ uses extended tests\. Parameter counts follow Table[2](https://arxiv.org/html/2609.29102#S4.T2)\. Budgets specify NFE or sampling steps; n\.r\. means unreported\. Sources and protocols appear in Appendix[B](https://arxiv.org/html/2609.29102#A2)\.Table 2:Mathematical reasoning, pass@1 \(%\)\. Our evaluations report mean with standard deviation across 16 seeds in parentheses\. Slashes separate base/instruct models, scores, and differing shot counts\. Qwen3\-0\.6B MATH\-500 accuracy uses thinking mode\. Bold marks the best comparable\-scale dLM performance per benchmark\. We show parameter count as non\-decoder\(\+decoder\) for ELF, including REPA\+REG components\. Budgets specify NFE or sampling steps; shots count demonstrations, and n\.r\. means unreported\. Sources and protocols appear in Appendix[B](https://arxiv.org/html/2609.29102#A2)\.ModelParam\.NFE / stepsShotspass@1GSM8K: large dLMs/AR LLMsLLaDA\-8B\-Base\[[Nie et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib38)\]8\.0B1,024 steps470\.30Dream\-v0\-Base\-7B\[[Ye et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib52)\]7\.6B256 steps877\.20TESS 2 v0\.1 \(GSM8K fine\-tuned\)\[[Tae et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib45)\]7B1,000 steps868\.9Qwen3\-0\.6B\-Base/Qwen3\-0\.6B\[[Yang et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib51)\]0\.6B–4 / 059\.59 / 79\.20Llama\-3\.2\-1B/Llama\-3\.2\-1B\-Instruct\[[Meta, 2024](https://arxiv.org/html/2609.29102#bib.bib37)\]1\.2B–5 / 87\.60 / 44\.40SmolLM2\-1\.7B/SmolLM2\-1\.7B\-Instruct\[[Allal et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib3)\]1\.7B–531\.10 / 48\.80Qwen3\-0\.6B\-Base \(our evaluation\)\[[Yang et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib51)\]0\.6B–039\.53\(0\.73\)GSM8K: comparable\-scale dLMsS\-FLM\[[Deschenaux & Gulcehre, 2026](https://arxiv.org/html/2609.29102#bib.bib14)\]163M512 NFE018\.18FMLM\+ \(Init\)\[[Agarwal et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib1)\]0\.168B64 NFE033\.60MLFM\[[Azangulov et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib7)\]1\.35B256 stepsn\.r\.31\.24MDLM\[[Sahoo et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib43)\]168M1,024 NFE033\.90Duo\[[Sahoo et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib44)\]168M1,024 NFE036\.02ELF\-B baseline90\(\+156\)M64 NFE037\.71\(0\.95\)ELF\-REG\-B \(ours\)104\(\+156\)M64 NFE038\.11\(1\.07\)ELF\-L baseline637\(\+157\)M64 NFE051\.85\(0\.88\)ELF\-REG\-L \(ours\)652\(\+157\)M64 NFE055\.96\(0\.94\)MATH\-500: large dLMs/AR LLMsQwen3\-0\.6B\-Base/Qwen3\-0\.6B\[[Yang et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib51)\]0\.6B–4 / n\.r\.29\.80 / 77\.60Llama\-3\.2\-1B/Llama\-3\.2\-1B\-Instruct\[[Meta, 2024](https://arxiv.org/html/2609.29102#bib.bib37)\]1\.2B–4 / 01\.60 / 24\.80SmolLM2\-1\.7B/SmolLM2\-1\.7B\-Instruct\[[Allal et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib3)\]1\.7B–4 / 011\.60 / 19\.20Qwen3\-0\.6B\-Base \(our evaluation\)\[[Yang et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib51)\]0\.6B–026\.30\(1\.76\)MATH\-500: comparable\-scale dLMsELF\-L baseline637\(\+157\)M128 NFE010\.55\(0\.90\)ELF\-REG\-L \(ours\)652\(\+157\)M128 NFE013\.39\(1\.03\)Figure 3:GSM8K pass@1 with early\-stop \(NFE 64\), averaged over 4 seeds\. Arrows show reductions in epochs to first attainment of matched accuracy\.##### Mathematical reasoning\.

On GSM8K and at comparable scales, our ELF\-REG\-B at NFE 16 \(34\.42%\) already exceeds the NFE 64 FMLM\+ \(Init\) result \(33\.60%\) \(Table[13](https://arxiv.org/html/2609.29102#A4.T13)\)\. At NFE 64, ELF\-REG\-B reaches 38\.11%, exceeding S\-FLM at NFE 512 and MDLM and Duo at NFE 1024\. On GSM8K, ELF\-REG\-L reaches 55\.96% pass@1 at NFE 64\. Figure[1](https://arxiv.org/html/2609.29102#S0.F1)shows the accuracy\-vs\-NFE comparisons\. Relative to ELF\-L baseline, ELF\-REG\-L raises GSM8K pass@1 from 51\.85% to 55\.96% and MATH\-500 pass@1 from 10\.55% to 13\.39%\. ELF\-REG\-L reaches the 51\.48% GSM8K accuracy threshold in 8 epochs versus 14 for ELF\-L baseline, requiring approximately 43% fewer epochs \(Figure[3](https://arxiv.org/html/2609.29102#S4.F3)\)\.

##### Code generation\.

ELF\-REG\-L at NFE 128 exceeds the comparable\-scale PlaidQ\+CFG\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\]at NFE 257 on all four benchmarks in Table[1](https://arxiv.org/html/2609.29102#S4.T1)\. Compared to PlaidQ\+CFG\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\], our ELF\-REG\-L pass@1 reaches 28\.92% vs 24\.58% on MBPP\-378 and 22\.56% vs 22\.04% on HumanEval\. ELF\-REG\-L also outperforms Open\-dCoder and oDLM\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41), Table 1\]at matched NFE 128 on all four benchmarks\. ELF\-REG\-L improves over ELF\-L baseline on each benchmark\. Appendix[D\.1](https://arxiv.org/html/2609.29102#A4.SS1)reports the complete comparisons and Appendix[B\.3](https://arxiv.org/html/2609.29102#A2.SS3)specifies prompting/execution protocols\.

Table 3:Mathematical reasoning, pass@kk\(%\), at NFE 64\. ELF\-REG\-L improves ELF\-L baseline across the sample budgets on both benchmarks\. Results are shown across 16 seeds\. Bold marks the best value per benchmark and sample budget\.
##### Multiple\-sample performance\.

At NFE 128, ELF\-REG\-L reaches 55\.05% pass@10 on MBPP\-378 and 48\.87% on HumanEval, compared with 44\.79% and 39\.87% for PlaidQ\+CFG at NFE 257 \(Table[16](https://arxiv.org/html/2609.29102#A4.T16)\)\. At NFE 16 and without few\-step training, ELF\-REG\-L already achieves higher pass@10 than PlaidQ\+CFG at NFE 257 on both MBPP\-378 and HumanEval \(Table[15](https://arxiv.org/html/2609.29102#A4.T15)\)\. On math reasoning, ELF\-REG\-L improves over ELF\-L baseline at reported pass@kkbudgets at NFE 64, including MATH\-500 pass@16 increasing from 44\.0% \(ELF\-L baseline\) to 52\.2% \(ELF\-REG\-L\) \(Table[3](https://arxiv.org/html/2609.29102#S4.T3)\)\.

### 4\.1Ablations and additional analyses

##### Training components\.

Table[4](https://arxiv.org/html/2609.29102#S4.T4)compares ELF\-B variants at epoch 6\. REPA improves over the baseline, and REPA\+REG improves further\. Stripped REPA\+REG uses the REPA\+REG weights but does not generate the REG token, retaining most of the gains of REPA\+REG\. REPA with one learnable token \(REPA\+OLT\) does not reproduce the REG benefit\. The comparison therefore identifies a training benefit that persists after REG is removed at inference\.

Table 4:REPA\+REG training benefits largely survive removal of REG at inference\. At epoch 6 on GSM8K, REPA\-only improves over Baseline, and REPA\+REG improves further\. Stripped REPA\+REG retains most of this additional gain using the same trained weights as REPA\+REG, except the REG component which is removed\. REPA\+OLT adds one learnable token \(OLT\) to REPA and does not reproduce the REG benefit\. All settings use ELF\-B\. The REPA\+REG setting corresponds to ELF\-REG\-B \(ours\)\.Table 5:GSM8K pass@1 \(%\) at matched NFE, ELF\-REG\-B \(ours\) at epoch 12, averaged over sixteen generation seeds\.
##### Prefix early\-stop\.

At matched NFE, early\-stop improves ELF\-REG\-B accuracy over full\-span generation across budgets \(Table[5](https://arxiv.org/html/2609.29102#S4.T5)\)\. Early\-stop uses the beginning of a longer time grid and decodes predicted\-clean states; full\-span generation traverses its shorter time grid and decodes the updated latent\. Appendix[D\.5\.1](https://arxiv.org/html/2609.29102#A4.SS5.SSS1)gives a sweep of early\-stop ratioρ\\rho\.

##### Sensitivity to coarse\-sampling errors\.

We inject errors from coarse \(fewer\-NFE\) trajectories and continue denoising\. Errors induced by coarser fewer\-NFE sampling are more damaging than random errors of matched magnitude\. Appendix[D\.5\.1](https://arxiv.org/html/2609.29102#A4.SS5.SSS1.Px1)and Figure[8](https://arxiv.org/html/2609.29102#A4.F8)describe the details\.

##### Additional representation analyses\.

The default Qwen3\-0\.6B\-Base encoder supports more effective learning and gold\-answer recovery under corruption than t5\-small \(Appendix[D\.3\.1](https://arxiv.org/html/2609.29102#A4.SS3.SSS1)\)\. Position\-shifted AR teacher targets strengthen forward perturbation influence at the supervised block; the final\-block ordering depends on corruption time \(Appendix[D\.3\.2](https://arxiv.org/html/2609.29102#A4.SS3.SSS2)\)\.

## 5Related Works

### 5\.1Diffusion language models

##### Discrete dLMs\.

Discrete diffusion language models \(dLMs\) reverse token corruption or masking to generate text\[[Austin et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib6);[Lou et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib35);[Sahoo et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib43)\]\. LLaDA and Dream scale masked denoising to reasoning and code, using bidirectional context to predict tokens\[[Nie et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib38);[Ye et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib52)\]\. Block Diffusion combines autoregressive \(AR\) generation across blocks with parallel denoising within each block\[[Arriola et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib4)\]\.

##### Continuous dLMs\.

Early continuous dLMs denoise token embeddings for controllable generation and sequence\-to\-sequence tasks\[[Li et al\., 2022](https://arxiv.org/html/2609.29102#bib.bib32);[Gong et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib19)\]\. CDCD and Plaid develop continuous formulations for categorical prediction and likelihood\-based language modeling\[[Dieleman et al\., 2022](https://arxiv.org/html/2609.29102#bib.bib15);[Gulrajani & Hashimoto, 2023](https://arxiv.org/html/2609.29102#bib.bib20)\]\. Recent methods improve both the denoising process and its representation space\. LangFlow\[[Chen et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib12)\]connects embedding diffusion to flow matching and learns the noise schedule\. LDLM\[[Meshchaninov et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib36)\]jointly trains a latent encoder and decoder with the diffusion model\. ELF\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\]denoises contextual encoder representations and shares the denoising network with the final token decoder, focusing on unconditional language generation, translation, and summarization\. S\-FLM\[[Deschenaux & Gulcehre, 2026](https://arxiv.org/html/2609.29102#bib.bib14)\]extends continuous generation to mathematical reasoning using hyperspherical flows\. Continuous diffusion also appears within AR models: L2D\[[Cetin et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib10)\]diffuses each next\-token embedding, while LaDiR\[[Kang et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib28)\]denoises latent thought blocks and generates the final answer autoregressively\.

##### Few\-step generation\.

FMLM\[[Lee et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib30)\]distills continuous flows into learned flow maps for one/few\-step generation\. FMLM\+\[[Agarwal et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib1)\]adds conditioning on clean tokens and commits confident predictions through posterior refinement\. MLFM\[[Azangulov et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib7)\]learns continuous flows over masked positions and progressively commits tokens during sampling\. PlaidQ\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\]applies fully continuous diffusion to code with an AR\-initialized denoiser, and trains separate few/one\-step generators through few\-step training methods\. Early stopping provides a sampling\-time alternative\[[Li et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib31)\]: Just on Time\[[Kohut et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib29)\]finalizes individual tokens in masked dLMs based on confidence, whereas continuous dLMs can decode the entire sequence early based on decoder confidence\[[Du & Ma, 2026](https://arxiv.org/html/2609.29102#bib.bib16)\]\.

##### Early\-stop\.

Prior work establishes early stopping for diffusion text generation through fixed and adaptive exits\[[Vaina et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib48);[Gao et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib17);[Du & Ma, 2026](https://arxiv.org/html/2609.29102#bib.bib16);[Li et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib31)\], and for image generation through Truncated Jump Sampling\[[Peng & Gao, 2026](https://arxiv.org/html/2609.29102#bib.bib42)\]\. Prophet observes early answer emergence in masked dLMs and uses answer confidence to commit all remaining tokens\[[Li et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib31)\]\. Building on these works, we demonstratelow\-NFE mathematical reasoning and code generationwith early\-stop and ELF\-REG, without dedicated few\-step training\. Our GSM8K analysis shows thatELF\-REG\-B improves intermediate predictions, reducing endpoint error and bringing earlier token stability\. We further interpret early\-stop as approximating the remaining flow map, and analyse early\-stop endpoint error and decoded\-output agreement\.

### 5\.2Representation alignment

##### Representation alignment\.

REPA\[[Yu et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib54)\]improves diffusion training by aligning noisy intermediate features to clean representations from a frozen teacher\. HASTE\[[Wang et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib49)\]stops the alignment loss during training, while SRA\[[Jiang et al\., 2026a](https://arxiv.org/html/2609.29102#bib.bib25)\]obtains targets from the diffusion model itself\. Language applications differ in where they apply this supervision\. Portuguese masked\-dLM REPA\[[Junior et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib27)\]aligns denoiser features with clean text representations from pretrained language encoders\. REPR\-ALIGN\[[Peng et al\., 2026a](https://arxiv.org/html/2609.29102#bib.bib40)\]aligns a masked bidirectional denoiser with the frozen AR model used to initialize it\. TextLDM\[[Jiang et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib26)\]instead aligns the VAE encoder with a frozen language model before training continuous latent diffusion\.

##### Jointly denoised representations\.

REG\[[Wu et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib50)\]jointly denoises image latents and a global representation supervised by a pretrained encoder\. Related language models combine continuous representations with discrete tokens\[[Zheng et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib56);[Zhou et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib57)\]\.

##### Our contribution\.

ELF\-REG extends ELF’s fully continuous dLM design to complex reasoning\. REPA aligns noisy intermediate states of the bidirectional denoiser with clean AR teacher states, while REG adds a jointly denoised global representation\. We combine representation alignment with inference\-time early\-stop to push the efficiency\-performance frontierwithout dedicated few\-step training, and analyse early\-stop properties of the trained dLM\. Appendix[E](https://arxiv.org/html/2609.29102#A5)extends the discussion of related works\.

## 6Conclusion and Discussion

We scale ELF’s\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\]fully continuous dLM to mathematical reasoning and code generation using representation alignment and entanglement\. We demonstrate superior performance compared to similar\-scale dLMs on math \(GSM8K\) and code generation \(HumanEval, MBPP\)\.

Building on prior early stopping work\[[Vaina et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib48);[Gao et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib17);[Du & Ma, 2026](https://arxiv.org/html/2609.29102#bib.bib16)\], we apply and analyze early\-stop for strong low\-NFE generation without dedicated few\-step training\. Early\-stop decodes an intermediate clean prediction, allowing the same trained models to generate responses with a limited denoising budget\. ELF\-REG\-L exceeds PlaidQ\-D16 on MBPP\-378 pass@10 with fewer NFE, despite PlaidQ\-D16’s dedicated few\-step training\. Future work will develop methods for improving and scaling continuous dLMs into generalist models, aiming to match AR LLM performance at lower cost\.

#### Acknowledgments

We acknowledge the Duke Compute Cluster \(DCC\), maintained by Duke Research Computing, for providing computational resources used in this work\. We also acknowledge the computational resources provided by NCShare, which is supported by National Science Foundation \(NSF\) grants OAC\-2201525, OAC\-2201105, and OAC\-2430141\.

## References

- Agarwal et al\. \[2026\]Manan Agarwal, Sheel Shah, Chanhyuk Lee, Jaehoon Yoo, Jerry Huang, Seunghoon Hong, Aditi Raghunathan, Jinwoo Kim, and Nicholas M\. Boffi\.Posterior refinement: Fast language generation via any\-order flow maps, 2026\.URL[https://arxiv\.org/abs/2606\.24773](https://arxiv.org/abs/2606.24773)\.
- Ahmad et al\. \[2025\]Wasi Uddin Ahmad, Aleksander Ficek, Mehrzad Samadi, Jocelyn Huang, Vahid Noroozi, Somshubra Majumdar, and Boris Ginsburg\.Opencodeinstruct: A large\-scale instruction tuning dataset for code llms, 2025\.URL[https://arxiv\.org/abs/2504\.04030](https://arxiv.org/abs/2504.04030)\.
- Allal et al\. \[2025\]Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Kydlíček, Agustín Piqueres Lajarín, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan\-Son Nguyen, Clémentine Fourrier, Ben Burtenshaw, Hugo Larcher, Haojun Zhao, Cyril Zakka, Mathieu Morlon, Colin Raffel, Leandro von Werra, and Thomas Wolf\.Smollm2: When smol goes big – data\-centric training of a small language model, 2025\.URL[https://arxiv\.org/abs/2502\.02737](https://arxiv.org/abs/2502.02737)\.
- Arriola et al\. \[2025\]Marianne Arriola, Aaron Gokaslan, Justin T\. Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and Volodymyr Kuleshov\.Block diffusion: Interpolating between autoregressive and diffusion language models, 2025\.URL[https://arxiv\.org/abs/2503\.09573](https://arxiv.org/abs/2503.09573)\.
- Austin et al\. \[2021\]Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton\.Program synthesis with large language models, 2021\.URL[https://arxiv\.org/abs/2108\.07732](https://arxiv.org/abs/2108.07732)\.
- Austin et al\. \[2023\]Jacob Austin, Daniel D\. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg\.Structured denoising diffusion models in discrete state\-spaces, 2023\.URL[https://arxiv\.org/abs/2107\.03006](https://arxiv.org/abs/2107.03006)\.
- Azangulov et al\. \[2026\]Iskander Azangulov, Kianoosh Ashouritaklimi, Leo Zhang, Simon Vary, and Patrick Rebeschini\.Masked language flow models, 2026\.URL[https://arxiv\.org/abs/2606\.27617](https://arxiv.org/abs/2606.27617)\.
- Boffi et al\. \[2025a\]Nicholas M\. Boffi, Michael S\. Albergo, and Eric Vanden\-Eijnden\.Flow map matching with stochastic interpolants: A mathematical framework for consistency models, 2025a\.URL[https://arxiv\.org/abs/2406\.07507](https://arxiv.org/abs/2406.07507)\.
- Boffi et al\. \[2025b\]Nicholas M\. Boffi, Michael S\. Albergo, and Eric Vanden\-Eijnden\.How to build a consistency model: Learning flow maps via self\-distillation, 2025b\.URL[https://arxiv\.org/abs/2505\.18825](https://arxiv.org/abs/2505.18825)\.
- Cetin et al\. \[2025\]Edoardo Cetin, Tianyu Zhao, and Yujin Tang\.Large language models to diffusion finetuning, 2025\.URL[https://arxiv\.org/abs/2501\.15781](https://arxiv.org/abs/2501.15781)\.
- Chen et al\. \[2021\]Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert\-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N\. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba\.Evaluating large language models trained on code, 2021\.URL[https://arxiv\.org/abs/2107\.03374](https://arxiv.org/abs/2107.03374)\.
- Chen et al\. \[2026\]Yuxin Chen, Chumeng Liang, Hangke Sui, Ruihan Guo, Chaoran Cheng, Jiaxuan You, and Ge Liu\.Langflow: Continuous diffusion rivals discrete in language modeling, 2026\.URL[https://arxiv\.org/abs/2604\.11748](https://arxiv.org/abs/2604.11748)\.
- Cobbe et al\. \[2021\]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman\.Training verifiers to solve math word problems, 2021\.URL[https://arxiv\.org/abs/2110\.14168](https://arxiv.org/abs/2110.14168)\.
- Deschenaux & Gulcehre \[2026\]Justin Deschenaux and Caglar Gulcehre\.Language modeling with hyperspherical flows, 2026\.URL[https://arxiv\.org/abs/2605\.11125](https://arxiv.org/abs/2605.11125)\.
- Dieleman et al\. \[2022\]Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H\. Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, Curtis Hawthorne, Rémi Leblond, Will Grathwohl, and Jonas Adler\.Continuous diffusion for categorical data, 2022\.URL[https://arxiv\.org/abs/2211\.15089](https://arxiv.org/abs/2211.15089)\.
- Du & Ma \[2026\]Zhicheng Du and Lan Ma\.Continuous language diffusion as a decoder\-interface problem, 2026\.URL[https://arxiv\.org/abs/2606\.08810](https://arxiv.org/abs/2606.08810)\.
- Gao et al\. \[2024\]Zhujin Gao, Junliang Guo, Xu Tan, Yongxin Zhu, Fang Zhang, Jiang Bian, and Linli Xu\.Empowering diffusion models on the embedding space for text generation\.In Kevin Duh, Helena Gomez, and Steven Bethard \(eds\.\),*Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\)*, pp\. 4664–4683, Mexico City, Mexico, June 2024\. Association for Computational Linguistics\.doi:10\.18653/v1/2024\.naacl\-long\.261\.URL[https://aclanthology\.org/2024\.naacl\-long\.261/](https://aclanthology.org/2024.naacl-long.261/)\.
- Gat et al\. \[2024\]Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky T\. Q\. Chen, Gabriel Synnaeve, Yossi Adi, and Yaron Lipman\.Discrete flow matching, 2024\.URL[https://arxiv\.org/abs/2407\.15595](https://arxiv.org/abs/2407.15595)\.
- Gong et al\. \[2023\]Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong\.Diffuseq: Sequence to sequence text generation with diffusion models, 2023\.URL[https://arxiv\.org/abs/2210\.08933](https://arxiv.org/abs/2210.08933)\.
- Gulrajani & Hashimoto \[2023\]Ishaan Gulrajani and Tatsunori B\. Hashimoto\.Likelihood\-based diffusion language models, 2023\.URL[https://arxiv\.org/abs/2305\.18619](https://arxiv.org/abs/2305.18619)\.
- Havasi et al\. \[2025\]Marton Havasi, Brian Karrer, Itai Gat, and Ricky T\. Q\. Chen\.Edit flows: Flow matching with edit operations, 2025\.URL[https://arxiv\.org/abs/2506\.09018](https://arxiv.org/abs/2506.09018)\.
- Hendrycks et al\. \[2021a\]Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt\.Measuring massive multitask language understanding, 2021a\.URL[https://arxiv\.org/abs/2009\.03300](https://arxiv.org/abs/2009.03300)\.
- Hendrycks et al\. \[2021b\]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt\.Measuring mathematical problem solving with the math dataset, 2021b\.URL[https://arxiv\.org/abs/2103\.03874](https://arxiv.org/abs/2103.03874)\.
- Hu et al\. \[2026\]Keya Hu, Linlu Qiu, Yiyang Lu, Hanhong Zhao, Tianhong Li, Yoon Kim, Jacob Andreas, and Kaiming He\.Elf: Embedded language flows, 2026\.URL[https://arxiv\.org/abs/2605\.10938](https://arxiv.org/abs/2605.10938)\.
- Jiang et al\. \[2026a\]Dengyang Jiang, Mengmeng Wang, Liuzhuozheng Li, Lei Zhang, Haoyu Wang, Wei Wei, Guang Dai, Yanning Zhang, and Jingdong Wang\.No other representation component is needed: Diffusion transformers can provide representation guidance by themselves, 2026a\.URL[https://arxiv\.org/abs/2505\.02831](https://arxiv.org/abs/2505.02831)\.
- Jiang et al\. \[2026b\]Jiaxiu Jiang, Jingjing Ren, Wenbo Li, Bo Wang, Haoze Sun, Yijun Yang, Jianhui Liu, Yanbing Zhang, Shenghe Zheng, Yuan Zhang, Haoyang Huang, Nan Duan, and Wangmeng Zuo\.Textldm: Language modeling with continuous latent diffusion, 2026b\.URL[https://arxiv\.org/abs/2605\.07748](https://arxiv.org/abs/2605.07748)\.
- Junior et al\. \[2026\]Adalberto Ferreira Barbosa Junior, Lucas Lima Neves, and Adriano César Santana\.Accelerating Portuguese masked diffusion models through representation alignment\.In Marlo Souza, Iria de Dios\-Flores, Diana Santos, Larissa Freitas, Jackson Wilke da Cruz Souza, and Eugénio Ribeiro \(eds\.\),*Proceedings of the 17th International Conference on Computational Processing of Portuguese \(PROPOR 2026\) \- Vol\. 1*, pp\. 968–973, Salvador, Brazil, April 2026\. Association for Computational Linguistics\.ISBN 979\-8\-89176\-387\-6\.URL[https://aclanthology\.org/2026\.propor\-1\.97/](https://aclanthology.org/2026.propor-1.97/)\.
- Kang et al\. \[2026\]Haoqiang Kang, Yizhe Zhang, Nikki Lijing Kuang, Nicklas Majamaki, Navdeep Jaitly, Yi\-An Ma, and Lianhui Qin\.Ladir: Latent diffusion enhances llms for text reasoning, 2026\.URL[https://arxiv\.org/abs/2510\.04573](https://arxiv.org/abs/2510.04573)\.
- Kohut et al\. \[2026\]Zakhar Kohut, Severyn Shykula, Mykola Vysotskyi, Serhii Dmytryshyn, Dmytro Khamula, Michal Zakrzewski, Damian Rynczak, Jacek Małecki, Taras Rumezhak, and Volodymyr Karpiv\.Just on time: Token\-level early stopping for diffusion language models, 2026\.URL[https://arxiv\.org/abs/2602\.11133](https://arxiv.org/abs/2602.11133)\.
- Lee et al\. \[2026\]Chanhyuk Lee, Jaehoon Yoo, Manan Agarwal, Sheel Shah, Jerry Huang, Aditi Raghunathan, Seunghoon Hong, Nicholas M\. Boffi, and Jinwoo Kim\.Flow map language models: One\-step language modeling via continuous denoising, 2026\.URL[https://arxiv\.org/abs/2602\.16813](https://arxiv.org/abs/2602.16813)\.
- Li et al\. \[2026\]Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Li Shen, Soroush Vosoughi, and Shiwei Liu\.Diffusion language models know the answer before decoding, 2026\.URL[https://arxiv\.org/abs/2508\.19982](https://arxiv.org/abs/2508.19982)\.
- Li et al\. \[2022\]Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B\. Hashimoto\.Diffusion\-lm improves controllable text generation, 2022\.URL[https://arxiv\.org/abs/2205\.14217](https://arxiv.org/abs/2205.14217)\.
- Lightman et al\. \[2023\]Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe\.Let’s verify step by step, 2023\.URL[https://arxiv\.org/abs/2305\.20050](https://arxiv.org/abs/2305.20050)\.
- Liu et al\. \[2023\]Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang\.Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation, 2023\.URL[https://arxiv\.org/abs/2305\.01210](https://arxiv.org/abs/2305.01210)\.
- Lou et al\. \[2024\]Aaron Lou, Chenlin Meng, and Stefano Ermon\.Discrete diffusion modeling by estimating the ratios of the data distribution, 2024\.URL[https://arxiv\.org/abs/2310\.16834](https://arxiv.org/abs/2310.16834)\.
- Meshchaninov et al\. \[2026\]Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov, Nikita Gushchin, Ilya Koziev, Alexander Korotin, and Dmitry Vetrov\.How to train your latent diffusion language model jointly with the latent space, 2026\.URL[https://arxiv\.org/abs/2605\.07933](https://arxiv.org/abs/2605.07933)\.
- Meta \[2024\]Meta\.Llama\-3\.2\-1b\-instruct model card, 2024\.URL[https://huggingface\.co/meta\-llama/Llama\-3\.2\-1B\-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)\.
- Nie et al\. \[2025\]Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji\-Rong Wen, and Chongxuan Li\.Large language diffusion models, 2025\.URL[https://arxiv\.org/abs/2502\.09992](https://arxiv.org/abs/2502.09992)\.
- Peng et al\. \[2025\]Fred Zhangzhi Peng, Shuibai Zhang, Alex Tong, et al\.Open\-dllm: Open diffusion large language models\.[https://github\.com/pengzhangzhi/Open\-dLLM](https://github.com/pengzhangzhi/Open-dLLM), 2025\.
- Peng et al\. \[2026a\]Fred Zhangzhi Peng, Alexis Fox, Anru R\. Zhang, and Alexander Tong\.Don’t retrain, align: Adapting autoregressive lms to diffusion lms via representation alignment, 2026a\.URL[https://arxiv\.org/abs/2605\.06885](https://arxiv.org/abs/2605.06885)\.
- Peng et al\. \[2026b\]Fred Zhangzhi Peng, Kaiwen Zheng, and Anru R\. Zhang\.Distilled continuous diffusion language models can write code in few steps—or one, 2026b\.URL[https://arxiv\.org/abs/2609\.04531](https://arxiv.org/abs/2609.04531)\.
- Peng & Gao \[2026\]Xin Peng and Ang Gao\.x\-prediction is all you need:training\-free accelerated generation via endpoint decodability, 2026\.URL[https://arxiv\.org/abs/2607\.06114](https://arxiv.org/abs/2607.06114)\.
- Sahoo et al\. \[2024\]Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, and Volodymyr Kuleshov\.Simple and effective masked diffusion language models, 2024\.URL[https://arxiv\.org/abs/2406\.07524](https://arxiv.org/abs/2406.07524)\.
- Sahoo et al\. \[2025\]Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, and Volodymyr Kuleshov\.The diffusion duality, 2025\.URL[https://arxiv\.org/abs/2506\.10892](https://arxiv.org/abs/2506.10892)\.
- Tae et al\. \[2025\]Jaesung Tae, Hamish Ivison, Sachin Kumar, and Arman Cohan\.Tess 2: A large\-scale generalist diffusion language model, 2025\.URL[https://arxiv\.org/abs/2502\.13917](https://arxiv.org/abs/2502.13917)\.
- Tang & Wang \[2026\]Sophia Tang and Shiyi Wang\.Discrete beckmann transport models for one\-step language modeling and reasoning, 2026\.URL[https://arxiv\.org/abs/2609\.15903](https://arxiv.org/abs/2609.15903)\.
- Toshniwal et al\. \[2024\]Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman\.Openmathinstruct\-2: Accelerating ai for math with massive open\-source instruction data, 2024\.URL[https://arxiv\.org/abs/2410\.01560](https://arxiv.org/abs/2410.01560)\.
- Vaina et al\. \[2024\]Sofia Maria Lo Cicero Vaina, Nikita Balagansky, and Daniil Gavrilov\.Diffusion language models generation can be halted early, 2024\.URL[https://arxiv\.org/abs/2305\.10818](https://arxiv.org/abs/2305.10818)\.
- Wang et al\. \[2025\]Ziqiao Wang, Wangbo Zhao, Yuhao Zhou, Zekai Li, Zhiyuan Liang, Mingjia Shi, Xuanlei Zhao, Pengfei Zhou, Kaipeng Zhang, Zhangyang Wang, Kai Wang, and Yang You\.Repa works until it doesn’t: Early\-stopped, holistic alignment supercharges diffusion training, 2025\.URL[https://arxiv\.org/abs/2505\.16792](https://arxiv.org/abs/2505.16792)\.
- Wu et al\. \[2025\]Ge Wu, Shen Zhang, Ruijing Shi, Shanghua Gao, Zhenyuan Chen, Lei Wang, Zhaowei Chen, Hongcheng Gao, Yao Tang, Jian Yang, Ming\-Ming Cheng, and Xiang Li\.Representation entanglement for generation: Training diffusion transformers is much easier than you think, 2025\.URL[https://arxiv\.org/abs/2507\.01467](https://arxiv.org/abs/2507.01467)\.
- Yang et al\. \[2025\]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu\.Qwen3 technical report, 2025\.URL[https://arxiv\.org/abs/2505\.09388](https://arxiv.org/abs/2505.09388)\.
- Ye et al\. \[2025\]Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong\.Dream 7b: Diffusion large language models, 2025\.URL[https://arxiv\.org/abs/2508\.15487](https://arxiv.org/abs/2508.15487)\.
- Yu et al\. \[2024\]Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T\. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu\.Metamath: Bootstrap your own mathematical questions for large language models, 2024\.URL[https://arxiv\.org/abs/2309\.12284](https://arxiv.org/abs/2309.12284)\.
- Yu et al\. \[2025\]Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie\.Representation alignment for generation: Training diffusion transformers is easier than you think, 2025\.URL[https://arxiv\.org/abs/2410\.06940](https://arxiv.org/abs/2410.06940)\.
- Zhao et al\. \[2026\]Changsheng Zhao, Ernie Chang, Zechun Liu, Chia\-Jung Chang, Wei Wen, Chen Lai, Sheng Cao, Yuandong Tian, Raghuraman Krishnamoorthi, Yangyang Shi, and Vikas Chandra\.Mobilellm\-r1: Exploring the limits of sub\-billion language model reasoners with open training recipes, 2026\.URL[https://arxiv\.org/abs/2509\.24945](https://arxiv.org/abs/2509.24945)\.
- Zheng et al\. \[2025\]Huangjie Zheng, Shansan Gong, Ruixiang Zhang, Tianrong Chen, Jiatao Gu, Mingyuan Zhou, Navdeep Jaitly, and Yizhe Zhang\.Continuously augmented discrete diffusion model for categorical generative modeling, 2025\.URL[https://arxiv\.org/abs/2510\.01329](https://arxiv.org/abs/2510.01329)\.
- Zhou et al\. \[2026\]Cai Zhou, Chenxiao Yang, Yi Hu, Chenyu Wang, Chubin Zhang, Muhan Zhang, Lester Mackey, Tommi Jaakkola, Stephen Bates, and Dinghuai Zhang\.Coevolutionary continuous discrete diffusion: Make your diffusion language model a latent reasoner, 2026\.URL[https://arxiv\.org/abs/2510\.03206](https://arxiv.org/abs/2510.03206)\.
- Zuo et al\. \[2025\]Jingwei Zuo, Maksim Velikanov, Ilyas Chahed, Younes Belkada, Dhia Eddine Rhayem, Guillaume Kunsch, Hakim Hacid, Hamza Yous, Brahim Farhat, Ibrahim Khadraoui, Mugariya Farooq, Giulia Campesan, Ruxandra Cojocaru, Yasser Djilali, Shi Hu, Iheb Chaabane, Puneesh Khanna, Mohamed El Amine Seddik, Ngoc Dung Huynh, Phuc Le Khac, Leen AlQadi, Billel Mokeddem, Mohamed Chami, Abdalgader Abubaker, Mikhail Lubinets, Kacper Piskorski, and Slim Frikha\.Falcon\-h1: A family of hybrid\-head language models redefining efficiency and performance, 2025\.URL[https://arxiv\.org/abs/2507\.22448](https://arxiv.org/abs/2507.22448)\.

###### Contents

1. [1Introduction](https://arxiv.org/html/2609.29102#S1)
2. [2Methodology](https://arxiv.org/html/2609.29102#S2)1. [2\.1ELF preliminaries](https://arxiv.org/html/2609.29102#S2.SS1) 2. [2\.2Representation alignment and entanglement](https://arxiv.org/html/2609.29102#S2.SS2) 3. [2\.3Prefix early\-stop generation](https://arxiv.org/html/2609.29102#S2.SS3)
3. [3Analysis](https://arxiv.org/html/2609.29102#S3)1. [3\.1Early\-stop decoding and the remaining flow map](https://arxiv.org/html/2609.29102#S3.SS1) 2. [3\.2Endpoint error and decoder stability along the sampling trajectory](https://arxiv.org/html/2609.29102#S3.SS2)
4. [4Experiments](https://arxiv.org/html/2609.29102#S4)1. [4\.1Ablations and additional analyses](https://arxiv.org/html/2609.29102#S4.SS1)
5. [5Related Works](https://arxiv.org/html/2609.29102#S5)1. [5\.1Diffusion language models](https://arxiv.org/html/2609.29102#S5.SS1) 2. [5\.2Representation alignment](https://arxiv.org/html/2609.29102#S5.SS2)
6. [6Conclusion and Discussion](https://arxiv.org/html/2609.29102#S6)
7. [References](https://arxiv.org/html/2609.29102#bib)
8. [AMethod details](https://arxiv.org/html/2609.29102#A1)1. [A\.1Clean encoder and teacher representations](https://arxiv.org/html/2609.29102#A1.SS1) 2. [A\.2Alignment and loss normalization](https://arxiv.org/html/2609.29102#A1.SS2) 3. [A\.3Training\-time guidance and velocity targets](https://arxiv.org/html/2609.29102#A1.SS3) 4. [A\.4Sampling behavior](https://arxiv.org/html/2609.29102#A1.SS4)
9. [BExperimental settings and comparison protocols](https://arxiv.org/html/2609.29102#A2)1. [B\.1Default hyperparameters](https://arxiv.org/html/2609.29102#A2.SS1) 2. [B\.2Training data and initialization](https://arxiv.org/html/2609.29102#A2.SS2) 3. [B\.3Benchmarks, metrics, and evaluation protocols](https://arxiv.org/html/2609.29102#A2.SS3) 4. [B\.4NFE and parameter accounting](https://arxiv.org/html/2609.29102#A2.SS4) 5. [B\.5External comparison sources and training budgets](https://arxiv.org/html/2609.29102#A2.SS5) 6. [B\.6Analysis protocols](https://arxiv.org/html/2609.29102#A2.SS6)
10. [CEarly\-stop proofs and the discrete sampler](https://arxiv.org/html/2609.29102#A3)1. [C\.1Endpoint error](https://arxiv.org/html/2609.29102#A3.SS1) 2. [C\.2Decoding and distributional agreement](https://arxiv.org/html/2609.29102#A3.SS2) 3. [C\.3Self\-conditioning, REG, and the denominator clamp](https://arxiv.org/html/2609.29102#A3.SS3) 4. [C\.4Endpoint\-error and discrete\-acceleration measurements](https://arxiv.org/html/2609.29102#A3.SS4)
11. [DAdditional results and analyses](https://arxiv.org/html/2609.29102#A4)1. [D\.1Additional headline results](https://arxiv.org/html/2609.29102#A4.SS1) 2. [D\.2Numerical values underlying Figure 1](https://arxiv.org/html/2609.29102#A4.SS2) 3. [D\.3Additional representation analyses](https://arxiv.org/html/2609.29102#A4.SS3) 4. [D\.4Training ablations](https://arxiv.org/html/2609.29102#A4.SS4) 5. [D\.5Sampling analyses](https://arxiv.org/html/2609.29102#A4.SS5)
12. [EComparison with related work](https://arxiv.org/html/2609.29102#A5)1. [E\.1Early\-stop generation](https://arxiv.org/html/2609.29102#A5.SS1)
13. [FLimitations](https://arxiv.org/html/2609.29102#A6)
14. [GQualitative Examples](https://arxiv.org/html/2609.29102#A7)

## Appendix AMethod details

### A\.1Clean encoder and teacher representations

Table[6](https://arxiv.org/html/2609.29102#A2.T6)gives the architecture, training, and sampling defaults for the headline configurations\. REG uses the teacher state at the last non\-special content token\. The clean Qwen3 text states and global targets are normalized across features\. For a feature vectoraa, this gives\(a−a¯\)/sa2\+10−6\(a\-\\bar\{a\}\)/\\sqrt\{s\_\{a\}^\{2\}\+10^\{\-6\}\}, wherea¯\\bar\{a\}is its mean andsa2s\_\{a\}^\{2\}its variance\. Text REPA targets retain their raw unnormalized feature values\. The REPA target at the REG position uses the pooled representation from the REPA teacher layer\. The clean REG denoising target uses the REG teacher layer\.

### A\.2Alignment and loss normalization

The causal teacher sees both clean prompt and response\. On denoising examples, the bidirectional dLM student sees clean prompt and noisy response states\. REPA uses a position\-wise MLP projector\. Text alignment excludes special tokens and padding\. REG contributes an additional valid position to the REPA mean\. Alignment is also applied to decoder examples\. Position\-shifted REPA targets change only text targets and their valid\-position mask; the REG target stays fixed\.

##### Loss normalization\.

ELF training selects decoder examples with some probability\. The token cross\-entropy and text velocity losses are summed over their respective examples and divided by the total number of supervised response positions\. With EOS padding which we use for all our experiments, this includes post\-response EOS positions\. REPA takes separate masked means for prompt and response positions before computing a weighted sum; by default, the prompt and response REPA losses have weights 0 and 1, respectively\. REG averages squared velocity error over features, sums it over denoising examples, and divides by the expected number of denoising examples in a batch\. Decoder examples receive corrupted REG inputs but no REG velocity loss\.

### A\.3Training\-time guidance and velocity targets

We follow ELF\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\]in applying self\-conditioning independently to denoising examples, with indicatorb∈\{0,1\}b\\in\\\{0,1\\\}\. We sample a guidance scalew\>0w\>0, favoring smaller values and following ELF’s sampling strategy\. A first auxiliary prediction uses zero response self\-conditioning\. Its predicted clean states condition a second auxiliary prediction, at the same noisy state, time, and guidance scale, with the prompt held fixed\. The zero\-conditioned and conditioned text velocities arevno​\-​scv\_\{\\mathrm\{no\\text\{\-\}sc\}\}andvscv\_\{\\mathrm\{sc\}\}in equation[7](https://arxiv.org/html/2609.29102#A1.E7), respectively; the corresponding REG velocities arevno​\-​scrv^\{r\}\_\{\\mathrm\{no\\text\{\-\}sc\}\}andvscrv^\{r\}\_\{\\mathrm\{sc\}\}, whererrdenotes the REG component\. The trainable prediction conditions on the first auxiliary prediction’s detached clean states whenb=1b=1and zero response self\-conditioning whenb=0b=0\.

##### Implemented velocity targets\.

Following ELF\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\], conversions from clean predictions to velocities usect=max⁡\(1−t,tϵ\)c\_\{t\}=\\max\(1\-t,t\_\{\\epsilon\}\)\. In particular,vθ=\(x^θ−zt\)/ctv\_\{\\theta\}=\(\\hat\{x\}\_\{\\theta\}\-z\_\{t\}\)/c\_\{t\}andvθr=\(r^θ−rt\)/ctv^\{r\}\_\{\\theta\}=\(\\hat\{r\}\_\{\\theta\}\-r\_\{t\}\)/c\_\{t\}\. Heresg\\operatorname\{sg\}stops gradients through the regression target\. The detached regression targets are

vtarget\\displaystyle v\_\{\\mathrm\{target\}\}=sg⁡\[x−ztct\+b⁡\(1−1w\)​\(vsc−vno​\-​sc\)\],\\displaystyle=\\operatorname\{sg\}\\\!\\left\[\\frac\{x\-z\_\{t\}\}\{c\_\{t\}\}\+b\\left\(1\-\\frac\{1\}\{w\}\\right\)\(v\_\{\\mathrm\{sc\}\}\-v\_\{\\mathrm\{no\\text\{\-\}sc\}\}\)\\right\],\(7\)vtargetr\\displaystyle v^\{r\}\_\{\\mathrm\{target\}\}=sg⁡\[r−rtct\+b⁡\(1−1w\)​\(vscr−vno​\-​scr\)\]\.\\displaystyle=\\operatorname\{sg\}\\\!\\left\[\\frac\{r\-r\_\{t\}\}\{c\_\{t\}\}\+b\\left\(1\-\\frac\{1\}\{w\}\\right\)\(v^\{r\}\_\{\\mathrm\{sc\}\}\-v^\{r\}\_\{\\mathrm\{no\\text\{\-\}sc\}\}\)\\right\]\.\(8\)Without self\-conditioning \(b=0b=0\), the targets reduce to the base velocities\(x−zt\)/ct\(x\-z\_\{t\}\)/c\_\{t\}and\(r−rt\)/ct\(r\-r\_\{t\}\)/c\_\{t\}\. The guided targets train the network to incorporate guidance in a single denoiser evaluation at inference\. The clean targets in equation[9](https://arxiv.org/html/2609.29102#A1.E9)use these same clamped denominators:xt⋆=sg⁡\(zt\+ct​vtarget\)x\_\{t\}^\{\\star\}=\\operatorname\{sg\}\(z\_\{t\}\+c\_\{t\}v\_\{\\mathrm\{target\}\}\)andrt⋆=sg⁡\(rt\+ct​vtargetr\)r\_\{t\}^\{\\star\}=\\operatorname\{sg\}\(r\_\{t\}\+c\_\{t\}v^\{r\}\_\{\\mathrm\{target\}\}\)\. The identity‖vθ−vtarget‖2=ct−2​‖x^θ−xt⋆‖2\\\|v\_\{\\theta\}\-v\_\{\\mathrm\{target\}\}\\\|^\{2\}=c\_\{t\}^\{\-2\}\\\|\\hat\{x\}\_\{\\theta\}\-x\_\{t\}^\{\\star\}\\\|^\{2\}converts clean\-prediction error to the implemented velocity error; there is no additional time weight on the velocity loss\. The velocity losses apply only to denoising examples\. Decoder examples receive time conditioningt=1t=1, zero self\-conditioning, and independently corrupted inputs, with per\-token logit\-normal corruption scale\. The REG decoder input uses a separately sampled corruption scale from the same distribution as the decoder examples\.

##### Expanded training objective\.

Let the time\-weighting coefficient beω⁡\(t\)=ct−2\\omega\(t\)=c\_\{t\}^\{\-2\}\. Leta∼Bernoulli⁡\(pdec\)a\\sim\\operatorname\{Bernoulli\}\(p\_\{\\mathrm\{dec\}\}\)indicate a decoder example, and writeℓCE\\ell^\{\\mathrm\{CE\}\}for token cross\-entropy\. The text and REG feature dimensions areddanddrd\_\{r\}, respectively, andr^θ\\hat\{r\}\_\{\\theta\}is the REG head’s clean prediction\. Written out, the loss is

ℒ=𝔼⁡\[a​ℓCE\+\(1−a\)​ω⁡\(t\)d​‖x^θ−xt⋆‖22\]⏟ℒELF\+λREPA​𝔼⁡\[1−cos⁡\(pϕ​\(u\(ℓ\)\),h\)\]⏟ℒREPA\+λREG​𝔼⁡\[ω⁡\(t\)dr​‖r^θ−rt⋆‖22\|a=0\]⏟ℒREG\.\\displaystyle\\mathcal\{L\}=\\underbrace\{\\mathbb\{E\}\\\!\\left\[a\\ell^\{\\mathrm\{CE\}\}\+\(1\-a\)\\frac\{\\omega\(t\)\}\{d\}\\\|\\hat\{x\}\_\{\\theta\}\-x\_\{t\}^\{\\star\}\\\|\_\{2\}^\{2\}\\right\]\}\_\{\\mathcal\{L\}\_\{\\mathrm\{ELF\}\}\}\+\\lambda\_\{\\mathrm\{REPA\}\}\\underbrace\{\\mathbb\{E\}\\\!\\left\[1\-\\cos\\\!\\left\(p\_\{\\phi\}\(u^\{\(\\ell\)\}\),h\\right\)\\right\]\}\_\{\\mathcal\{L\}\_\{\\mathrm\{REPA\}\}\}\+\\lambda\_\{\\mathrm\{REG\}\}\\underbrace\{\\mathbb\{E\}\\\!\\left\[\\frac\{\\omega\(t\)\}\{d\_\{r\}\}\\\|\\hat\{r\}\_\{\\theta\}\-r\_\{t\}^\{\\star\}\\\|\_\{2\}^\{2\}\\,\\middle\|\\,a=0\\right\]\}\_\{\\mathcal\{L\}\_\{\\mathrm\{REG\}\}\}\.

\(9\)ℒELF\\mathcal\{L\}\_\{\\mathrm\{ELF\}\}averages over supervised response positions, using time\-weighted clean\-prediction error for denoising examples \(a=0a=0\) and token cross\-entropy for decoder examples \(a=1a=1\)\.ℒREPA\\mathcal\{L\}\_\{\\mathrm\{REPA\}\}aligns intermediate states with the teacher representationhhat valid response content positions and the REG position, including decoder examples\.ℒREG\\mathcal\{L\}\_\{\\mathrm\{REG\}\}trains the model to reconstruct the guided global target and averages over denoising examples only\. Both squared errors use the same time weightω⁡\(t\)\\omega\(t\), withd−1d^\{\-1\}anddr−1d\_\{r\}^\{\-1\}averaging over features\. The coefficientsλREPA\\lambda\_\{\\mathrm\{REPA\}\}andλREG\\lambda\_\{\\mathrm\{REG\}\}weight the auxiliary losses\.

### A\.4Sampling behavior

The Euler ODE sampler samples denoising time from a logit\-normal distribution and appends endpoints00and11\. Its time distribution and noise scale match denoising training \(Table[6](https://arxiv.org/html/2609.29102#A2.T6)\)\. Decoding uses time11and zero self\-conditioning\. Ordinary REPA\+REG retains REG; stripped REPA\+REG loads the shared weights into the baseline architecture without the REG token position and associated weights\. Please refer to Algorithm[1](https://arxiv.org/html/2609.29102#algorithm1)for the sampling algorithm with plug\-and\-play early\-stop, adapted from ELF\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\]\.

NFE counts denoiser evaluations plus one final decoder evaluation\. Full\-span denoising, vocabulary projections, and cached AR decoding have different FLOP and latency costs; Appendix[B\.4](https://arxiv.org/html/2609.29102#A2.SS4)specifies the accounting conventions\.

## Appendix BExperimental settings and comparison protocols

### B\.1Default hyperparameters

Table[6](https://arxiv.org/html/2609.29102#A2.T6)contains the default settings for the headline experiments\. Appendix[B\.6](https://arxiv.org/html/2609.29102#A2.SS6)specifies the analysis and ablation protocols\.

Table 6:Default hyperparameters for the headline ELF configurations\. GSM8K pairs give ELF\-B / ELF\-L when the sizes differ; other slashes separate the quantities named in the row\. Parameters are non\-decoder \(\+decoder\), in millions\. REPA\+REG denotes ELF\-REG\-B \(ours\) or ELF\-REG\-L \(ours\), according to model size; the baseline omits both objectives\. Training epochs identify the evaluated checkpoints\. MATH\-500 epochs count task training after GSM8K initialization \(Appendix[B\.2](https://arxiv.org/html/2609.29102#A2.SS2)\)\. Shared entries span the task columns\.SettingGSM8KCodeMATH\-500MMLUArchitectureModel sizeELF\-B / ELF\-LELF\-LELF\-LELF\-BTransformer blocks12 / 32323212Hidden width768 / 1,2801,2801,280768Attention heads12 / 16161612SwiGLU hidden width2,048 / 3,4133,4133,4132,048Vocabulary size151,669Parameters \(M\), baseline90 \(\+156\) / 637 \(\+157\)637 \(\+157\)637 \(\+157\)90 \(\+156\)Parameters \(M\), REPA\+REG104 \(\+156\) / 652 \(\+157\)652 \(\+157\)652 \(\+157\)104 \(\+156\)Text latent / decoder width1024Text projection bottleneck128Time / guidance / mode tokens4 / 4 / 4Attention / projection dropout0\.0OptimizationOptimizerMuon \+ Nesterov\-AdamMuon momentum0\.95Adam\(β1,β2\);ϵ\(\\beta\_\{1\},\\beta\_\{2\}\);\\epsilon\(0\.9,0\.999\);10−8\(0\.9,0\.999\);10^\{\-8\}Learning rate0\.002Global batch size512Weight decay0\.0Gradient clipping norm1\.0LR warmup \(epochs\)0\.50\.50\.252\.5LR scheduleLinear warmup, then constantEMA warmup \(updates\)1000100001000Training epochsbaseline: 18 / 15REPA\+REG: 12 / 1212830Training objectivesEncoderQwen3\-0\.6B\-BaseEncoder layer20Denoising logit\-time mean / std\.\-1\.5 / 0\.8Denoising noise scaleσ\\sigma2\.0Decoder\-example probability0\.2Decoder logit\-corruption mean / std\.0\.8 / 0\.8Decoder noise scale1\.0Self\-conditioning probability0\.5Training guidance rangeww\[0\.5, 5\.0\]REPA\+REGTeacher \(REPA and REG\)Qwen3\-1\.7B\-BaseTeacher layers, REPA / REG20 / 24Teacher target width2048REPA supervised block4 / 8884Projector layers / hidden width3 / 2,048REPA weightλREPA\\lambda\_\{\\mathrm\{REPA\}\}0\.5REG weightλREG\\lambda\_\{\\mathrm\{REG\}\}0\.1REPA prompt / response weights0\.0 / 1\.0Teacher target shift0REPA training cutoffNoneEpoch 8NoneNoneSamplingSamplerEuler ODETime gridSorted logit\-normalCFG / SCCFG1 / 31 / 21 / 21 / 2Evaluated EMA0\.99990\.99990\.99990\.999Prompt / sequence limit256 / 1,024512 / 1,024256 / 1,024800 / 1,056Denominator clamptϵt\_\{\\epsilon\}0\.05Table 6:Default hyperparameters for the headline ELF configurations \(continued\)\.
### B\.2Training data and initialization

##### Initialization\.

We determine checkpoints using fixed training budgets\. Table[6](https://arxiv.org/html/2609.29102#A2.T6)gives the training epochs of the evaluated checkpoints, and Table[7](https://arxiv.org/html/2609.29102#A2.T7)gives the training\-token budgets and GPU\-hours\. We initialize the ELF model from scratch for GSM8K, code, and MMLU\. MATH models are finetuned for eight epochs from the corresponding GSM8K weights at epoch 7\.5, using the EMA 0\.999 checkpoint as initialization\.

Table 7:Training epochs, non\-padding task\-training tokens, and GPU\-hours on H200 GPUs for the headline checkpoints\. Init\. counts inherited task training; Task counts training on the current corpus; Total sums them\. Token counts are epoch\-based estimates in billions\. MATH\-500 GPU\-hours are given including / excluding GSM8K initialization\.Table 8:Training GPU\-hours on H200 GPUs to reach the GSM8K accuracy thresholds in Figure[3](https://arxiv.org/html/2609.29102#S4.F3)\. Reduction is relative to the baseline of the same architecture\. REPA\+REG denotes ELF\-REG\-B \(ours\) or ELF\-REG\-L \(ours\), according to architecture\.At the GSM8K accuracy thresholds in Figure[3](https://arxiv.org/html/2609.29102#S4.F3), ELF\-REG reduces training GPU\-hours relative to the baseline by 14\.1% for ELF\-B and 31\.4% for ELF\-L \(Table[8](https://arxiv.org/html/2609.29102#A2.T8)\)\.

##### GSM8K training data\.

We combine the GSM8K\-derived portions of OpenMathInstruct\-2\[[Toshniwal et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib47)\]and MetaMathQA\[[Yu et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib53)\]\. We remove examples with missing or unsupported numerical answers, examples exceeding the token limits, and groups of questions that match the GSM8K test set\[[Cobbe et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib13)\]\. After grouping related questions, we sample 10% of the source questions from each corpus for validation and hold out their entire groups\. Validation uses a fixed subset of 1024 examples from distinct groups\. The training dataset contains 2,404,028 examples\.

##### MATH training data\.

We use MATH\-derived question–solution pairs from the five\-million\-example release of OpenMathInstruct\-2\[[Toshniwal et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib47)\]\. We remove examples with malformed answers, groups with conflicting answers, examples exceeding the token limits, and groups matching MATH\-500\[[Hendrycks et al\., 2021b](https://arxiv.org/html/2609.29102#bib.bib23);[Lightman et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib33)\]\. We hold out 10% of the related\-question groups for validation and use a fixed subset of 1024 examples from distinct groups\. The training dataset contains 3,572,583 examples\.

##### Code training data\.

We use Python question–solution pairs from OpenCodeInstruct\[[Ahmad et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib2)\]\. We filter by the dataset’s quality scores, require solutions to parse as Python, and enforce the token limits\. We remove training examples matching the HumanEval and MBPP task sets distributed with EvalPlus\[[Chen et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib11);[Austin et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib5);[Liu et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib34)\]\. Matching uses normalized prompt equality or word\-8\-gram Jaccard similarity of at least 0\.5, as well as exact matches between solution abstract syntax trees\. We deduplicate normalized task descriptions\. The code dataset contains 2,740,479 examples\.

##### MMLU training data\.

We use the MMLU auxiliary training dataset\[[Hendrycks et al\., 2021a](https://arxiv.org/html/2609.29102#bib.bib22)\]with rationales generated by Qwen3\-30B\-A3B\-Instruct\-2507\. We retain examples whose generated answer matches the gold answer and whose rationale satisfies the format and length checks\. The training dataset contains 96,112 examples\.

##### Training\-token accounting\.

We count prompt and response tokens in the processed training datasets, including EOS tokens and excluding padding, and multiply the per\-epoch count by the training epochs\. Table[7](https://arxiv.org/html/2609.29102#A2.T7)reports these epoch\-based estimates\. For MATH, eight fine\-tuning epochs contribute 13\.625B tokens; the initial 7\.5 GSM8K training epochs contribute 4\.185B tokens, giving 17\.810B tokens in total\. These task\-training totals are separate from the pretraining of the frozen Qwen3 encoder and the training\-time Qwen3 teacher\.

### B\.3Benchmarks, metrics, and evaluation protocols

##### Code benchmarks\.

MBPP\-378 contains the 378 MBPP tasks retained by EvalPlus, evaluated with base tests; MBPP\-500 contains the original 500\-task MBPP test partition\[[Austin et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib5);[Liu et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib34)\]\. HumanEval contains 164 tasks, and HumanEval\+ evaluates the same tasks with the additional EvalPlus tests\[[Chen et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib11);[Liu et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib34)\]\. PlaidQ Appendix B\.7\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\]names the corresponding MBPP\-378 and MBPP\-500 benchmarks “MBPP\+” and “MBPP”, respectively\. Figure[1](https://arxiv.org/html/2609.29102#S0.F1)\(b\) uses MBPP\-378 pass@10\.

##### Metrics\.

We report mean pass@1 and standard deviation across generation seeds\. Each question has sixteen generated samples\. For pass@kk, ifccsamples are correct, the question\-level estimate is1−\(16−ck\)/\(16k\)1\-\\binom\{16\-c\}\{k\}/\\binom\{16\}\{k\}; we average these subset estimates across questions\[[Chen et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib11)\]\. HumanEval and HumanEval\+ use the same generated samples; HumanEval\+ requires passing both the base and extended tests\.

##### ELF evaluation\.

All headline ELF results use sixteen generation seeds and the sampling settings in Table[6](https://arxiv.org/html/2609.29102#A2.T6)\. GSM8K\[[Cobbe et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib13)\]includes all 1319 test questions, MATH\-500\[[Hendrycks et al\., 2021b](https://arxiv.org/html/2609.29102#bib.bib23);[Lightman et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib33)\]all 500 questions, and MMLU\[[Hendrycks et al\., 2021a](https://arxiv.org/html/2609.29102#bib.bib22)\]all 14,042 test questions\. GSM8K evaluation extracts a numerical answer from generated reasoning\. MATH\-500 uses symbolic equivalence scoring withmath\-verify0\.9\.0\. MMLU scores the final answer letter extracted from the generated rationale\. The headline mathematical\-reasoning and code results useρ=8\\rho=8early\-stop\. We tune self\-conditioning CFG \(SCCFG\) on validation data for GSM8K and MATH, and separately on the MBPP and HumanEval benchmarks for code \(we choose SCCFG 2 for both MBPP and HumanEval\)\.

##### Code prompts and response budgets\.

ELF requests a complete Python solution using the following template:

> ``` Write a complete Python solution. Return only code. {task} Python solution: ```

For HumanEval\(\+\) and MBPP\-378, the task is the prompt supplied by EvalPlus\. For MBPP\-500, it is the problem statement followed by “Your code should pass these tests:” and the three public assertions; trailing assertions are removed only when necessary to fit the prompt limit\. The prompt occupies at most 512 tokens of a 1024\-token sequence\. The response can occupy all remaining positions, giving a maximum of 1024 minus the actual prompt length\. We retain the response up to its first EOS token, or to the end of the sequence if EOS is absent\.

##### Code extraction and execution\.

We use the EvalPlus framework\[[Liu et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib34)\]for code execution and its sanitizer to extract Python from each completion\. If the expected function name is absent and the extracted code contains exactly one top\-level function, we add an alias from the expected name to that function, then sanitize for the expected entry point\. HumanEval\(\+\) uses HumanEval\+ v0\.1\.10, and MBPP\-378 uses the base tests from MBPP\+ v0\.2\.0\. MBPP\-500 uses its original three assertions and setup code, with a 20\-second program timeout\. EvalPlus uses a minimum test timeout of 20 seconds and a reference\-runtime multiplier of 10\. Each program has a 6 GiB memory limit; execution failures and timeouts count as incorrect\.

##### Our Qwen3 evaluations\.

The Qwen3\-0\.6B\-Base evaluations use zero\-shot evaluation, BF16, no chat template, sixteen samples per question, temperature 0\.8, and top\-pp0\.95\. We use the same template for both ELF and Qwen3 evaluation: GSM8K\[[Cobbe et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib13)\]prompts end with “Answer:”, and MATH\-500\[[Hendrycks et al\., 2021b](https://arxiv.org/html/2609.29102#bib.bib23);[Lightman et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib33)\]prompts end with “Step\-by\-Step Answer:” after the question\. Both benchmarks use the same answer scoring as our ELF evaluations\. For HumanEval and HumanEval\+\[[Chen et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib11);[Liu et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib34)\], we concatenate the benchmark prompt and generated continuation before execution since the HumanEval prompts contain the function header\. MBPP\-378\[[Austin et al\., 2021](https://arxiv.org/html/2609.29102#bib.bib5);[Liu et al\., 2023](https://arxiv.org/html/2609.29102#bib.bib34)\]requests a complete Python solution and includes the task description and first example assertion\. Code extraction follows the procedure above; execution uses HumanEval\+ v0\.1\.10 and MBPP\+ v0\.2\.0 as above\. Table[9](https://arxiv.org/html/2609.29102#A2.T9)records response caps and the observed fraction of generations that reach the caps\.

Table 9:Response\-token caps and the percentage of Qwen3\-0\.6B\-Base\[[Yang et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib51)\]generations that reach them in our evaluations\. Each question has sixteen samples\. HumanEval and HumanEval\+ share the same generations\.

### B\.4NFE and parameter accounting

NFE includes a final decoder call when one is required\. Where exact NFE is unresolved, tables report sampling steps which may not be equivalent to NFE\. The conventions are:

- •Our results useN−1N\-1denoiser evaluations and one final decoder evaluation, forNNNFE total\. Self\-conditioning guidance is learned during training and does not add an inference evaluation at CFG 1; note, we use CFG 1 throughout the paper\.
- •PlaidQ usesK\+1K\+1evaluations forKKdenoising steps and a final decode\. Applying CFG to PlaidQ adds one additional model evaluation at every denoising step, giving2​K\+12K\+1\(PlaidQ Appendices B\.4–B\.5\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\]\)\. We note that PlaidQ distills a distinct model for each low\-NFE evaluation budget \(where distillation applies\)\.
- •FMLM\+\[[Agarwal et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib1)\], DBTM\[[Tang & Wang, 2026](https://arxiv.org/html/2609.29102#bib.bib46)\], and S\-FLM\[[Deschenaux & Gulcehre, 2026](https://arxiv.org/html/2609.29102#bib.bib14)\]all report NFE in their headline results\.
- •MLFM uses a configured budget of 256 sampling steps\[[Azangulov et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib7)\]\. We report steps because the exact NFE for the reported result is unresolved\.
- •LLaDA and Dream describe sampling steps/diffusion timesteps\[[Nie et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib38);[Ye et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib52)\], which we report without converting them to NFE\.
- •Edit Flow uses 10000 sampling steps for pass@1 and 5000 for pass@10; Mask DFM uses 1000 for each\[[Havasi et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib21)\]\. We retain these step budgets without converting them to NFE\.

These counts cover generative denoiser and decoder evaluations; the fixed prompt\-encoder pass is outside our NFE count\. They do not equate FLOPs or latency across architectures\.

##### Parameter counts\.

ELF’s decoder comprises a projection from transformer width to width 1024, followed by the vocabulary projection and their biases\. ELF\-REG parameter counts include the REPA projector and both REG projections\. MLFM’s total includes its frozen backbone and trainable adapters\[[Azangulov et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib7)\]; Open\-dCoder’s count includes shared parameters once\. Our frozen Qwen3\-0\.6B\-Base encoder supplies prompt representations at inference\. The Qwen3\-1\.7B\-Base teacher supplies training targets and is not used during inference\. The additional parameter counts from the frozen Qwen3 encoder/teacher are separate from the ELF parameter counts\.

### B\.5External comparison sources and training budgets

Figure[1](https://arxiv.org/html/2609.29102#S0.F1)compares continuous and continuous/masked hybrid dLMs using results from PlaidQ, FMLM\+, S\-FLM, and DBTM\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41);[Agarwal et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib1);[Deschenaux & Gulcehre, 2026](https://arxiv.org/html/2609.29102#bib.bib14);[Tang & Wang, 2026](https://arxiv.org/html/2609.29102#bib.bib46)\]\. Training corpora, tokenizers, and evaluation protocols differ across methods\.

##### Comparison curves\.

We extract results from FMLM\+ \(Init\) and DBTM using the initialization and refinement\-in\-loop rows, respectively, from Table 8 of each paper\[[Agarwal et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib1);[Tang & Wang, 2026](https://arxiv.org/html/2609.29102#bib.bib46)\]\. For S\-FLM, we use its best result at each NFE across the architecture, temperature, and velocity\-decoding sweeps in Figures 1, 3, and 6–9\[[Deschenaux & Gulcehre, 2026](https://arxiv.org/html/2609.29102#bib.bib14)\]; the resulting curve combines different configurations\. PlaidQ\+CFG uses Figure 5\(b\), and distilled PlaidQ uses Section 3\.3 and Figure 6\(b\)\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\]\. The Duo\[[Sahoo et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib44)\]model’s GSM8K results that we cite is trained on TinyGSM and evaluated on GSM8K by the S\-FLM authors\[[Deschenaux & Gulcehre, 2026](https://arxiv.org/html/2609.29102#bib.bib14)\]\. The MDLM and Duo results in Table[2](https://arxiv.org/html/2609.29102#S4.T2)are extracted from S\-FLM Figure 3\(b\) atT=0\.1T=0\.1and 1024 NFE\[[Deschenaux & Gulcehre, 2026](https://arxiv.org/html/2609.29102#bib.bib14)\]\. Values extracted from figures are approximate\.

##### Baseline training budgets\.

PlaidQ Appendix B\.1 reports 327\.90B non\-padding tokens for base training\. Its bar shows this base\-training budget only; the additional distillation budget is unreported and excluded from our comparisons\. For the TinyGSM\-trained methods \(note: S\-FLM, DBTM, and FMLM\+ all report training for 250k steps\), we estimate non\-padding tokens as optimization steps times global batch size times the reported mean length of 194 tokens under the SmolLM tokenizer \(S\-FLM Figure 5\)\. This rounded mean is measured before filtering sequences longer than 512 tokens, so these budgets are approximate\. S\-FLM uses250,000×512×194≈24\.8250\{,\}000\\times 512\\times 194\\approx 24\.8B tokens\. FMLM\+ \(Init\) requires an MDLM run and a subsequent FMLM\+ run, each with this budget, giving approximately24\.8\+24\.8=49\.724\.8\+24\.8=49\.7B \(FMLM\+ Appendix B\.3 and Table 11\)\. Repeated forwards for self\-distillation are not counted as additional training data\. DBTM\[[Tang & Wang, 2026](https://arxiv.org/html/2609.29102#bib.bib46)\]reports batch size 256 for TinyGSM training, giving250,000×256×194≈12\.4250\{,\}000\\times 256\\times 194\\approx 12\.4B \(note: DBTM reports conflicting batch sizes in different places for TinyGSM training; we take 256 as a potential underestimate for DBTM’s training budget\)\.

##### Scope of training budgets\.

The bars in Figure[1](https://arxiv.org/html/2609.29102#S0.F1)\(d\) count task\-training tokens, including required task\-specific initialization\. Foundation pretraining such as the pretraining cost of Qwen3 weights used by PlaidQ and our frozen encoder/teacher is separate and not included in the training budget\. Token counts do not measure training FLOPs\.

##### AR mathematical\-reasoning sources\.

Slashes in Table[2](https://arxiv.org/html/2609.29102#S4.T2)consistently mean base/instruct\. Qwen3 Table 8\[[Yang et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib51)\]supplies the four\-shot GSM8K base score; MobileLLM\-R1 Table 9\[[Zhao et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib55)\]supplies the zero\-shot instruct score without explicitly identifying thinking mode\. SmolLM2 Table 4\[[Allal et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib3)\]supplies the five\-shot GSM8K base scores for Llama\-3\.2\-1B and SmolLM2\-1\.7B\. The Llama\-3\.2 model card\[[Meta, 2024](https://arxiv.org/html/2609.29102#bib.bib37)\]supplies its eight\-shot instruct score, and SmolLM2 Table 5 supplies the five\-shot SmolLM2 instruct score\. For MATH\-500, the base scores and the Llama/SmolLM2 instruct scores come from MobileLLM\-R1 Tables 8–9, using four and zero demonstrations, respectively\. Qwen3 Table 19 supplies its thinking\-mode MATH\-500 instruct score; the report does not specify its shot count, which we mark n\.r\.

##### AR code sources\.

SmolLM2 Tables 4–5 and Sections 4\.7 and 5\.4\[[Allal et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib3)\]provide the base/instruct HumanEval comparisons for Llama\-3\.2\-1B and SmolLM2\-1\.7B\. MobileLLM\-R1 Table 8\[[Zhao et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib55)\]supplies the Qwen3\-0\.6B\-Base HumanEval score\. The Qwen3\-0\.6B HumanEval/HumanEval\+ pair comes from the same Falcon\-H1 evaluation\[[Zuo et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib58)\], which explicitly disables thinking\.

##### TESS 2\.

The GSM8K result is approximately 68\.9% at 1000 diffusion steps, extracted from TESS 2 Figure 3\(b\)\[[Tae et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib45)\]\. This 7B\-scale model uses Mistral\-v0\.1 initialization, diffusion adaptation, instruction tuning, and GSM8K\-specific fine\-tuning; TESS 2 Appendix D specifies eight chain\-of\-thought demonstrations\. We note that TESS 2 is not comparable to our method due to its scale\.

##### dLM code evaluations\.

The LLaDA\-8B\-Base, Dream\-v0\-Base\-7B, and comparable\-scale rows follow PlaidQ Table 1\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\]\. Open\-dCoder\[[Peng et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib39)\]and oDLM\[[Peng et al\., 2026a](https://arxiv.org/html/2609.29102#bib.bib40)\]use 128 NFE for the documented P2 configuration with 128 sampling steps and a 128\-token response budget\. NFE for the LLaDA and Dream code scores are not reported\.

##### Code comparison settings\.

PlaidQ Appendix B\.7\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\]specifies a 128\-token completion budget, with total sequence length capped at 2048 tokens\. Our ELF evaluations allow the response to fill the remaining positions of a 1024\-token sequence, as described in Appendix[B\.3](https://arxiv.org/html/2609.29102#A2.SS3)\. PlaidQ uses the HumanEval prompt verbatim and prepends it to the completion before code extraction; ELF instead requests a complete solution\. For MBPP\-500, PlaidQ includes the problem statement and three public assertions, followed by an opening Python code fence\. Its MBPP\-378 prompt uses the same format with the task set’s base assertions\.

PlaidQ truncates completions at benchmark\-specific stop strings, removes code fences, and normalizes whitespace\. It then retains the longest contiguous span among the first 100 lines that parses as Python and executes the resulting program with a 15\-second timeout\. Our code extraction uses EvalPlus sanitization with the single\-function alias rule and execution limits described above\. We use PlaidQ’s reported values\.

### B\.6Analysis protocols

##### GSM8K generation\.

The component comparison in Table[4](https://arxiv.org/html/2609.29102#S4.T4)uses ELF\-B epoch 6, early\-stop NFE 64, ratioρ=8\\rho=8, and 16 seeds\. Learning curves in Figure[3](https://arxiv.org/html/2609.29102#S4.F3)evaluate every integer epoch: baseline epochs 1–18 for ELF\-B and 1–15 for ELF\-L, and epochs 1–12 for both ELF\-REG\-B and ELF\-REG\-L\. Each point uses early\-stop NFE 64, ratio 8, and 4 generation seeds\. Both analyses cover all 1319 test questions with the GSM8K sampling settings in Table[6](https://arxiv.org/html/2609.29102#A2.T6), EOS padding, no model attention mask, and no output truncation\. The stripped and unstripped REPA\+REG variants share trained weights\.

##### Training progress \(GSM8K\)\.

One epoch contains 4696 optimizer steps in the GSM8K learning curves, so epocheecorresponds to4696​e4696esteps\. The speed\-up comparison in Figure[3](https://arxiv.org/html/2609.29102#S4.F3)uses the first evaluated checkpoint that reaches a given accuracy to ensure fairness\. Generation seeds share trained weights and do not represent independent training runs\. Table[7](https://arxiv.org/html/2609.29102#A2.T7)reports the corresponding non\-padding token budgets, and Figure[1](https://arxiv.org/html/2609.29102#S0.F1)\(d\) compares task\-training budgets across methods\.

##### Continuing trajectories and error injection\.

Decoder stability \(Figure[2](https://arxiv.org/html/2609.29102#S3.F2)\(b–e\)\) and numerical\-answer agreement \(Table[10](https://arxiv.org/html/2609.29102#A3.T10)\) use ELF\-B baseline epoch 18 and ELF\-REG\-B epoch 12, four generation seeds, SCCFG 3, and a master\-NFE\-512 trajectory\. We observe updates 1–64, then 72–504 every eight updates, and update 511\. Intermediate observations decode predicted\-clean states; the endpoint decodes the updated state\. The headline early\-exit is update 63, NFE 64\. NFE accounting excludes 119 additional diagnostic decoder calls\. Stability is averaged over questions within each seed and then across seeds, and we do not measure changes between observations\. Coarse\-error injection \(Figure[8](https://arxiv.org/html/2609.29102#A4.F8)\) uses the same checkpoints and four seeds with a 64\-NFE reference and a nested 16\-NFE coarse grid\. Full\-span answer retention \(Table[28](https://arxiv.org/html/2609.29102#A4.T28)\) uses these checkpoints and sixteen generation seeds\. Other generation settings follow the GSM8K protocol above\.

##### Encoder comparisons\.

The GSM8K encoder learning curves \(Figure[4](https://arxiv.org/html/2609.29102#A4.F4), top\) use t5\-small epochs 1–14 and Qwen3 epochs 1–18, with early\-stop NFE 64, ratio 8, EMA 0\.9999, and SCCFG 3\. The MMLU curves \(Figure[4](https://arxiv.org/html/2609.29102#A4.F4), top\) use epochs 10/20/30, NFE 64, EMA 0\.999, and SCCFG 2 on 14,042 test questions; whole\-sequence \(which includes both prompt and response\) and prompt token limits are 1,056 and 800 tokens\. Both datasets use 4 generation seeds\. Recovery \(Figure[4](https://arxiv.org/html/2609.29102#A4.F4), bottom\) uses GSM8K epoch 6 and MMLU epoch 30, with each dataset’s own EMA and guidance settings\. Clean reconstruction uses one decoder call\. Corrupted recovery uses one denoiser call and one decoder call for each of 4 seeds\.

##### Perturbation, settling, and training ablations\.

The single\-latent perturbation \(Figures[5](https://arxiv.org/html/2609.29102#A4.F5)and[6](https://arxiv.org/html/2609.29102#A4.F6)\) and token\-settling analyses \(Figure[7](https://arxiv.org/html/2609.29102#A4.F7)\) use all three ELF\-B conditions at epoch 6, EMA 0\.9999, 1 generation seed, and 1024 validation examples\. Token\-settling uses 64 denoiser updates, CFG 1, and SCCFG 3, with diagnostic decoding every four updates\. The secondary training ablations \(Tables[25](https://arxiv.org/html/2609.29102#A4.T25)and[26](https://arxiv.org/html/2609.29102#A4.T26)\) use 1 generation seed, CFG 1, SCCFG 3, full\-span 64 updates plus one decode \(65 NFE\), and output truncation\. They report validation accuracy on 1024 examples from the validation set and test accuracy on all 1319 GSM8K questions\. ELF\-B uses epoch 6 and EMA 0\.9999; ELF\-L uses epoch 2\.5 and EMA 0\.999\.

## Appendix CEarly\-stop proofs and the discrete sampler

### C\.1Endpoint error

###### Proof of Proposition[1](https://arxiv.org/html/2609.29102#Thmproposition1)\.

Integrating the velocity givesFt,1​\(y\)=y\+∫t1v⁡\(ys,s\)​𝑑sF\_\{t,1\}\(y\)=y\+\\int\_\{t\}^\{1\}v\(y\_\{s\},s\)\\,ds\. Subtracting it fromDt​\(y\)=y\+\(1−t\)​v​\(y,t\)D\_\{t\}\(y\)=y\+\(1\-t\)v\(y,t\)proves equation[4](https://arxiv.org/html/2609.29102#S3.E4)\. Along the trajectory,dd​s​v​\(ys,s\)=as\\frac\{d\}\{ds\}v\(y\_\{s\},s\)=a\_\{s\}\. Therefore,

et​\(y\)\\displaystyle e\_\{t\}\(y\)=−∫t1\(v\(ys,s\)−v\(y,t\)\)ds=−∫t1∫tsaududs\\displaystyle=\-\\int\_\{t\}^\{1\}\\bigl\(v\(y\_\{s\},s\)\-v\(y,t\)\\bigr\)\\,ds=\-\\int\_\{t\}^\{1\}\\int\_\{t\}^\{s\}a\_\{u\}\\,du\\,ds\(10\)=−∫t1\(1−u\)audu\.\\displaystyle=\-\\int\_\{t\}^\{1\}\(1\-u\)a\_\{u\}\\,du\.\(11\)The triangle inequality yields‖et​\(y\)‖≤∫t1\(1−u\)​A​𝑑u=A​\(1−t\)2/2\\\|e\_\{t\}\(y\)\\\|\\leq\\int\_\{t\}^\{1\}\(1\-u\)A\\,du=A\(1\-t\)^\{2\}/2\. ∎

### C\.2Decoding and distributional agreement

The 2\-Wasserstein distanceW2W\_\{2\}is the minimum root mean squared Euclidean distance over all couplings of the latent distributions\. Total variation,TV\\operatorname\{TV\}, is the largest difference in probability assigned to the same event\. Small root mean squared endpoint error therefore places the latent distributions close in Wasserstein distance\. The decoder condition separately requires the effect on logit differences to remain below the winning\-token margin; latents being exactly the same is unnecessary\. These bounds concern agreement with the outputs of full\-span generation\. Whether changed responses improve task accuracy depends on their correctness\.

###### Proof of Proposition[2](https://arxiv.org/html/2609.29102#Thmproposition2)\.

We apply the pairwise\-logit margin argument of[Du & Ma \[2026, Theorem 1\]](https://arxiv.org/html/2609.29102#bib.bib16)at each response position\. Letb=y1b=y\_\{1\},a=y^ta=\\hat\{y\}\_\{t\}, and letℓi,k​\(y\)\\ell\_\{i,k\}\(y\)denote the logit of tokenkkat positionii\. Ifkik\_\{i\}wins atbb, definegi,k​\(y\)=ℓi,ki​\(y\)−ℓi,k​\(y\)g\_\{i,k\}\(y\)=\\ell\_\{i,k\_\{i\}\}\(y\)\-\\ell\_\{i,k\}\(y\)for each competitork≠kik\\neq k\_\{i\}\. By assumption,gi,k​\(b\)≥mg\_\{i,k\}\(b\)\\geq mand\|gi,k​\(a\)−gi,k​\(b\)\|≤L​‖a−b‖\|g\_\{i,k\}\(a\)\-g\_\{i,k\}\(b\)\|\\leq L\\\|a\-b\\\|\. Consequently,gi,k​\(a\)≥m−L​‖et​\(yt\)‖\>0g\_\{i,k\}\(a\)\\geq m\-L\\\|e\_\{t\}\(y\_\{t\}\)\\\|\>0\. Every endpoint winner through the first EOS is unchanged\. The EOS position and the decoded response therefore agree\. If the endpoint has no EOS, the argument applies to the full response canvas\. The decoder input is the full latent state, including positions after EOS that can influence earlier logits through attention\. ∎

###### Proof of Corollary[1](https://arxiv.org/html/2609.29102#Thmcorollary1)\.

The joint law of\(y^t,y1\)\(\\hat\{y\}\_\{t\},y\_\{1\}\)is a coupling ofPtP\_\{t\}andP1P\_\{1\}\. The definition of Wasserstein distance gives

W22​\(Pt,P1\)≤𝔼​‖y^t−y1‖2=𝔼​‖et​\(yt\)‖2\.W\_\{2\}^\{2\}\(P\_\{t\},P\_\{1\}\)\\leq\\mathbb\{E\}\\\|\\hat\{y\}\_\{t\}\-y\_\{1\}\\\|^\{2\}=\\mathbb\{E\}\\\|e\_\{t\}\(y\_\{t\}\)\\\|^\{2\}\.Applying the deterministic decoder gives a coupling ofQtQ\_\{t\}andQ1Q\_\{1\}\. The coupling inequality and Proposition[2](https://arxiv.org/html/2609.29102#Thmproposition2)imply

TV\(Qt,Q1\)≤Pr\[𝒟\(y^t\)≠𝒟\(y1\)\]≤Pr\[L∥et\(yt\)∥≥m\]\.\\operatorname\{TV\}\(Q\_\{t\},Q\_\{1\}\)\\leq\\Pr\[\\mathcal\{D\}\(\\hat\{y\}\_\{t\}\)\\neq\\mathcal\{D\}\(y\_\{1\}\)\]\\leq\\Pr\[L\\\|e\_\{t\}\(y\_\{t\}\)\\\|\\geq m\]\.The margin and local Lipschitz constant may depend on the sampled trajectory\. They must satisfy the proposition almost surely\. For any accuracy function taking values in\[0,1\]\[0,1\], the difference in expected accuracy is at mostTV⁡\(Qt,Q1\)\\operatorname\{TV\}\(Q\_\{t\},Q\_\{1\}\)\. ∎

##### Bounding the decoder failure probability\.

Under the assumptions of Corollary[1](https://arxiv.org/html/2609.29102#Thmcorollary1), fix the prompt, guidance scale, and continuous exit timet<1t<1\. All probabilities and expectations below are over initial noise\. For any fixedη\>0\\eta\>0,

Pr\[L∥et\(yt\)∥≥m\]≤Pr\[m≤Lη\]\+𝔼​‖et​\(yt\)‖2η2\.\\Pr\[L\\\|e\_\{t\}\(y\_\{t\}\)\\\|\\geq m\]\\leq\\Pr\[m\\leq L\\eta\]\+\\frac\{\\mathbb\{E\}\\\|e\_\{t\}\(y\_\{t\}\)\\\|^\{2\}\}\{\\eta^\{2\}\}\.To see this, if‖et​\(yt\)‖<η\\\|e\_\{t\}\(y\_\{t\}\)\\\|<\\etaandm\>L​ηm\>L\\eta, thenL​‖et​\(yt\)‖≤L​η<mL\\\|e\_\{t\}\(y\_\{t\}\)\\\|\\leq L\\eta<mbecauseL≥0L\\geq 0\. Thus

\{L∥et\(yt\)∥≥m\}⊆\{m≤Lη\}∪\{∥et\(yt\)∥≥η\}\.\\\{L\\\|e\_\{t\}\(y\_\{t\}\)\\\|\\geq m\\\}\\subseteq\\\{m\\leq L\\eta\\\}\\cup\\\{\\\|e\_\{t\}\(y\_\{t\}\)\\\|\\geq\\eta\\\}\.The union bound and Markov’s inequality applied to‖et​\(yt\)‖2\\\|e\_\{t\}\(y\_\{t\}\)\\\|^\{2\}give

Pr\[L∥et\(yt\)∥≥m\]\\displaystyle\\Pr\[L\\\|e\_\{t\}\(y\_\{t\}\)\\\|\\geq m\]≤Pr\[m≤Lη\]\+Pr\[∥et\(yt\)∥≥η\]\\displaystyle\\leq\\Pr\[m\\leq L\\eta\]\+\\Pr\[\\\|e\_\{t\}\(y\_\{t\}\)\\\|\\geq\\eta\]≤Pr\[m≤Lη\]\+𝔼​‖et​\(yt\)‖2η2\.\\displaystyle\\leq\\Pr\[m\\leq L\\eta\]\+\\frac\{\\mathbb\{E\}\\\|e\_\{t\}\(y\_\{t\}\)\\\|^\{2\}\}\{\\eta^\{2\}\}\.The second moment is finite because‖et​\(yt\)‖2≤2​‖y^t‖2\+2​‖y1‖2\\\|e\_\{t\}\(y\_\{t\}\)\\\|^\{2\}\\leq 2\\\|\\hat\{y\}\_\{t\}\\\|^\{2\}\+2\\\|y\_\{1\}\\\|^\{2\}and both endpoint distributions have finite second moments\. The marginmmand sensitivityLLmay depend on the trajectory; no independence is required\. The first term measures the probability that the margin is small relative to decoder sensitivity at discrepancy scaleη\\eta\. The second controls the probability of an endpoint error at leastη\\eta\. Corollary[1](https://arxiv.org/html/2609.29102#Thmcorollary1)therefore gives a small TV bound when both terms are small\.

### C\.3Self\-conditioning, REG, and the denominator clamp

Algorithm 1ELF inference with prefix early\-stop\.Adapted from Algorithm 2 of ELF\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\]\. We apply prefix early\-stop to ELF’s generation procedure by executing only the prefix of a longer sampling time grid and decoding the latest predicted\-clean states\. Full\-span generation denoises the full time grid and decodes the updated endpointz\. Both full\-span and early\-stop useN−1N\-1denoiser evaluations and one decoder evaluation\.master\_nfe=rho\*N

ts=\[0\]\+sort\(sample\_logit\_normal\(master\_nfe\-2\)\)\+\[1\]

z=sigma\*randn\(shape\)

x\_pred\_prev=zeros\_like\(z\)

foriinrange\(N\-1\):

t=ts\[i\]

dt=ts\[i\+1\]\-t

x\_pred=net\(z,t,self\_cond=x\_pred\_prev,mode="denoise"\)

v=\(x\_pred\-z\)/max\(1\-t,t\_eps\)

z=z\+dt\*v

x\_pred\_prev=x\_pred

ifrho\>1:

z=x\_pred

h=net\(z,t=1,self\_cond=0,mode="decode"\)

token\_logits=unembed\(h\)

tokens=argmax\(token\_logits,axis=\-1\)

returntokens

The implemented sampler keeps more state than the response latents\. Letyjy\_\{j\}collect the current text and REG latent states at grid timetjt\_\{j\}, and letqjq\_\{j\}collect their clean predictions from stepj−1j\-1, so thatqj=dj−1q\_\{j\}=d\_\{j\-1\}forj≥1j\\geq 1, withq0=0q\_\{0\}=0\. The baseline state contains only text\. Herejjindexes updates, whereas the analysis elsewhere uses continuous time\. With the prompt fixed, letdj=Dθ​\(yj,qj,tj\)d\_\{j\}=D\_\{\\theta\}\(y\_\{j\},q\_\{j\},t\_\{j\}\)be the clean prediction including the self\-conditioning guidance\. Following Algorithm[1](https://arxiv.org/html/2609.29102#algorithm1), forcj=max⁡\(1−tj,tϵ\)c\_\{j\}=\\max\(1\-t\_\{j\},t\_\{\\epsilon\}\)andΔj=tj\+1−tj\\Delta\_\{j\}=t\_\{j\+1\}\-t\_\{j\}, Euler sampling is

vj=dj−yjcj,yj\+1=yj\+Δj​vj,qj\+1=dj\.v\_\{j\}=\\frac\{d\_\{j\}\-y\_\{j\}\}\{c\_\{j\}\},\\qquad y\_\{j\+1\}=y\_\{j\}\+\\Delta\_\{j\}v\_\{j\},\\qquad q\_\{j\+1\}=d\_\{j\}\.\(12\)Thus\(yj,qj\)\(y\_\{j\},q\_\{j\}\)determines the remaining discrete trajectory\. A flow map onyjy\_\{j\}alone would discard self\-conditioning information\.

LetMMdenote the master NFE budget whereMMdetermines the denoising time grid, as in Section[2](https://arxiv.org/html/2609.29102#S2)\. The grid hasM−1M\-1denoiser updates and ends attM−1=1t\_\{M\-1\}=1; its final decoder evaluation completes the NFE budget\. Afterkkupdates, early\-stop decodesdk−1d\_\{k\-1\}, which was predicted attk−1t\_\{k\-1\}before the last update\. Full\-span generation decodesyM−1y\_\{M\-1\}\. Writingj=k−1j=k\-1and summing the updates in equation[12](https://arxiv.org/html/2609.29102#A3.E12)gives

dj−yM−1\\displaystyle d\_\{j\}\-y\_\{M\-1\}=cj​vj−∑ℓ=jM−2Δℓ​vℓ\\displaystyle=c\_\{j\}v\_\{j\}\-\\sum\_\{\\ell=j\}^\{M\-2\}\\Delta\_\{\\ell\}v\_\{\\ell\}\(13\)=\(cj−\(1−tj\)\)​vj\+∑ℓ=jM−2Δℓ​\(vj−vℓ\)\.\\displaystyle=\\bigl\(c\_\{j\}\-\(1\-t\_\{j\}\)\\bigr\)v\_\{j\}\+\\sum\_\{\\ell=j\}^\{M\-2\}\\Delta\_\{\\ell\}\(v\_\{j\}\-v\_\{\\ell\}\)\.\(14\)The first term is the denominator\-clamp correction\. The second measures changes in velocity along the actual self\-conditioned trajectory\. The decoder\-agreement proposition applies directly to the resulting endpoint discrepancy because both decoder calls use time 1 and zero self\-conditioning inputs\.

##### Numerical prefix error\.

In the smooth, unclamped model, lety~t\\tilde\{y\}\_\{t\}be a numerical approximation toyty\_\{t\}\. IfDtD\_\{t\}isKDK\_\{D\}\-Lipschitz between these states, then

‖Dt​\(y~t\)−Ft,1​\(yt\)‖≤KD​‖y~t−yt‖\+‖et​\(yt\)‖\.\\\|D\_\{t\}\(\\tilde\{y\}\_\{t\}\)\-F\_\{t,1\}\(y\_\{t\}\)\\\|\\leq K\_\{D\}\\\|\\tilde\{y\}\_\{t\}\-y\_\{t\}\\\|\+\\\|e\_\{t\}\(y\_\{t\}\)\\\|\.This separates error accumulated before the exit from replacement of the remaining flow\. Our trajectory measurement instead compares states on the same numerical trajectory, as in equation[14](https://arxiv.org/html/2609.29102#A3.E14)\.

### C\.4Endpoint\-error and discrete\-acceleration measurements

##### Endpoint\-error measurement\.

We use the checkpoints and master\-NFE\-512 grid specified in Appendix[B\.6](https://arxiv.org/html/2609.29102#A2.SS6)\. A first deterministic pass obtains the endpoint latent state\. A second pass from the same initial text and REG noise records each clean prediction and its discrepancy from that endpoint\. Text error includes the entire generated canvas because the decoder attends beyond the first EOS\. Token stability is computed backwards from the endpoint: a final\-response token position is stable only if its token matches at the current and every later saved observation\. Missing positions count as mismatches\. The stable\-token fraction is the proportion of final\-response positions, including the first EOS and excluding padding, whose tokens are stable\. We average this fraction over questions and seeds\. Figure[2](https://arxiv.org/html/2609.29102#S3.F2)\(b\) reports the mean text Euclidean error, averaging questions within each seed and then equally over four seeds\. Decoded\-response agreement includes EOS status and length\. The last predicted\-clean observation we make is at NFE 505; the NFE\-512 observation uses the updated endpoint itself\. The extra reference pass and diagnostic decoder calls are analysis\-related costs, separate from the reported exit NFE\.

##### Numerical\-answer agreement and correctness transitions\.

Table[10](https://arxiv.org/html/2609.29102#A3.T10)compares NFE 64 with NFE 512 on the same question–seed pairs\. We extract answers by taking the first number after the last answer marker, or the last number when no marker is present\. Missing extracted answers count as disagreement and as incorrect\. The difference between improvement and regression rates is the change in accuracy\.

Table 10:Numerical\-answer agreement and correctness transitions from early\-stop NFE 64 to the endpoint at NFE 512 on GSM8K\. Each early\-stop prediction is paired with the endpoint of its own master\-NFE\-512 trajectory\. ELF\-B baseline uses epoch 18 and ELF\-REG\-B epoch 12\. Each column covers 5,276 question–seed pairs: all 1,319 test questions and four generation seeds\. Most numerical answers agree despite low complete\-response agreement; correctness gains and losses nearly cancel for ELF\-B baseline\.
##### Prediction time and training\-interpolant SNR\.

For the NFE\-64 early\-stop withρ=8\\rho=8, the master time grid has 511 denoiser updates, 510 interior time steps drawn from a logit\-normal distribution, and endpoints 0 and 1\. After 63 updates, early\-stop decodes the latest clean prediction, produced at denoising grid index 62 before the last update; the updated state is at index 63\. Each generation seed supplies one schedule shared by both models\. Table[11](https://arxiv.org/html/2609.29102#A3.T11)summarizes the four distinct prediction times\.

For the training interpolant in equation[1](https://arxiv.org/html/2609.29102#S2.E1), the signal is the clean encoder statexx, scaled bytt, and the noise is\(1−t\)​σ​ε\(1\-t\)\\sigma\\varepsilon\. With approximately unit\-variance clean encoder states, nominal power SNR ist2/\[σ2​\(1−t\)2\]t^\{2\}/\[\\sigma^\{2\}\(1\-t\)^\{2\}\], withσ=2\\sigma=2\. The clean encoder standardizes each state across features using variance\. We compute SNR separately at each saved prediction time, then report the mean and range over schedules\. This calculation describes the training corruption level at the exit time; it does not decompose the generated ODE state into signal and noise because the generated trajectory need not follow the training noise\-data interpolation\.

Table 11:Prediction time and nominal training\-interpolant power SNR at the NFE\-64 exit \(ρ=8\\rho=8\) in Figure[2](https://arxiv.org/html/2609.29102#S3.F2)\. Means and ranges are over four saved logit\-normal schedules, shared by ELF\-B baseline and ELF\-REG\-B \(ours\)\. SNR is computed separately for each schedule before averaging\.
##### Discrete acceleration measurement\.

We reuse the checkpoints and GSM8K settings of the endpoint\-error measurement, replacing logit\-normal time sampling withdeterministic uniform time gridover 512 Euler updates\. From the sampling velocities in equation[12](https://arxiv.org/html/2609.29102#A3.E12), we compute

tj=j512,a^j=vj\+1−vj1/512,A^j=maxj≤r<512⁡‖a^r‖,0≤j<512\.t\_\{j\}=\\frac\{j\}\{512\},\\qquad\\hat\{a\}\_\{j\}=\\frac\{v\_\{j\+1\}\-v\_\{j\}\}\{1/512\},\\qquad\\hat\{A\}\_\{j\}=\\max\_\{j\\leq r<512\}\\\|\\hat\{a\}\_\{r\}\\\|,\\qquad 0\\leq j<512\.\(15\)The discrete accelerationa^j\\hat\{a\}\_\{j\}measures the change in sampling velocity per unit time between consecutive grid points\. The remaining maximumA^j\\hat\{A\}\_\{j\}is the largest acceleration norm from stepjjthrough step511511\. The joint norm includes the full response canvas, including positions after EOS, and REG when present, and excludes the prompt\. One additional denoiser evaluation att=1t=1retains the final self\-conditioning and suppliesv512v\_\{512\}for the purposes of measuring the acceleration without changing the endpoint\. Velocities use the denominator clampcj=max⁡\(1−tj,0\.05\)c\_\{j\}=\\max\(1\-t\_\{j\},0\.05\)\. The two\-pass procedure above measures endpoint errors on these same uniform time\-grid trajectories\. Due to the different time\-grid \(uniform vs logit\-normal\), the prediction att=5/64t=5/64uses 41 denoiser evaluations and one decode, giving NFE 42; the reference pass and diagnostic evaluations are separate analysis costs\.

Table[12](https://arxiv.org/html/2609.29102#A3.T12)compares the remaining maxima and endpoint errors\. The maximumA^j\\hat\{A\}\_\{j\}measures finite velocity differences along the discrete self\-conditioned trajectory\. Proposition[1](https://arxiv.org/html/2609.29102#Thmproposition1)instead assumes a bound on acceleration at every time along a smooth continuous flow\. Since we compute a discrete approximation of the acceleration, substitutingA^j\\hat\{A\}\_\{j\}forAAdoes not give a certified continuous endpoint\-error bound\. Appendix[C\.3](https://arxiv.org/html/2609.29102#A3.SS3)accounts for self\-conditioning and the denominator clamp\.

Table 12:Remaining joint acceleration and joint endpoint error att=5/64t=5/64on GSM8K, using 512 Euler updates with uniform time sampling\. Each early\-stop prediction is compared with its own trajectory endpoint\. Acceleration is the median over all 1,319 questions within each generation seed; endpoint error is the mean Euclidean norm\. Both statistics then average four generation seeds\. ELF\-B baseline uses epoch 18 and ELF\-REG\-B epoch 12\.

## Appendix DAdditional results and analyses

### D\.1Additional headline results

##### GSM8K across NFE\.

Table[13](https://arxiv.org/html/2609.29102#A4.T13)compares the two ELF sizes with FMLM\+ \(Init\) and DBTM at matched NFE\. All ELF rows use early\-stop\. Our numerical observations underlying Figure[1](https://arxiv.org/html/2609.29102#S0.F1)\(c\) appear in Table[23](https://arxiv.org/html/2609.29102#A4.T23)\.

Table 13:GSM8K mean pass@1 \(%\) across NFE\. ELF\-REG\-L achieves the highest accuracy; at comparable parameter counts, ELF\-REG\-B also outperforms FMLM\+ \(Init\) and DBTM at every NFE\. FMLM\+ \(Init\) and DBTM values are from their respective Table 8\[[Agarwal et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib1);[Tang & Wang, 2026](https://arxiv.org/html/2609.29102#bib.bib46)\]; ELF values are our evaluations\. Bold marks the best value in each NFE column within each group\.
##### Code across inference and sample budgets\.

Tables[14](https://arxiv.org/html/2609.29102#A4.T14)and[15](https://arxiv.org/html/2609.29102#A4.T15)report pass@1 and pass@10 at each method’s sampling budget\. Each PlaidQ\-D row uses the student trained for that particular denoising budget\. Table[16](https://arxiv.org/html/2609.29102#A4.T16)gives the ELF\-L sample\-budget sweep at NFE 128 and the available PlaidQ comparisons\. ELF\-REG\-L reaches 55\.05% pass@10 on MBPP\-378 and 46\.69% on HumanEval\+, compared with 52\.90% and 42\.84% for the ELF baseline\. On MBPP\-500, the baseline reaches 40\.50% and ELF\-REG\-L reaches 39\.92%\.

Table 14:Code pass@1 \(%\) across inference budgets\. ELF rows are our early\-stop evaluations; external values are from PlaidQ Table 1 and Section 3\.3\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\]\. Budget entries specify NFE or sampling steps; dashes denote unavailable measurements\. Bold marks the maximum in each benchmark column over all methods and budgets\.Table 15:Code pass@10 \(%\) across inference budgets, using the same benchmark mapping and NFE / steps convention as Table[14](https://arxiv.org/html/2609.29102#A4.T14)\. The guided PlaidQ Table 1 row uses NFE 257\. Bold marks the maximum in each benchmark column over all methods and budgets\.Table 16:Pass@kk\(%\) on MBPP\-500, MBPP\-378, HumanEval, and HumanEval\+\. ELF\-L baseline and ELF\-REG\-L use NFE 128; we report sampling NFE for PlaidQ and PlaidQ\+CFG\. Bold marks the best result for each benchmark and sample budget across all methods\.
##### MATH\-500\.

Table[17](https://arxiv.org/html/2609.29102#A4.T17)reports mean pass@1 across NFE, and Table[18](https://arxiv.org/html/2609.29102#A4.T18)reports pass@kkat NFE 64\. ELF\-REG\-L reaches 13\.31% mean pass@1 at NFE 64 and 13\.39% at NFE 128\. Its pass@10 at NFE 64 is 44\.04%, compared with 36\.73% for the baseline\.

Table 17:MATH\-500 mean pass@1 \(%\) across NFE, ELF\-L baseline and ELF\-REG\-L with early\-stop generation\. Bold marks the best value in each NFE column\.Table 18:MATH\-500 pass@kk\(%\) at NFE 64, ELF\-L baseline and ELF\-REG\-L with early\-stop generation\. Bold marks the best value at each sample budget\.
##### MMLU\.

ELF\-REG\-B improves ELF\-B mean pass@1 at every NFE budget \(Table[19](https://arxiv.org/html/2609.29102#A4.T19)\), reaching 46\.71% at NFE 64 compared with 45\.34% for the baseline\.

Table 19:MMLU mean pass@1 \(%\), ELF\-B baseline and ELF\-REG\-B with full\-span generation\. Bold marks the best value in each NFE column\.
##### Early stopping in FMLM\+\.

Table[20](https://arxiv.org/html/2609.29102#A4.T20)compares the FMLM\+ \(Init\) results with our reproduction of full\-span \(which is the default setup of FMLM\+\) and early\-stop evaluations of the same checkpoint\. Both local rows average the same four generation seeds over all 1319 GSM8K questions, with Python execution scoring\. Full\-span generation allocates its token\-commit schedule to full NFE budget; early\-stop generation follows a master schedule of NFE 64 and reads its complete prediction at each exit\. Early stopping lowers accuracy at NFE 4–32 for FMLM\+\. Its posterior\-refinement sampler commits confident token predictions and re\-noises uncommitted positions\. The accuracy loss likely reflects truncation of the 64\-NFE token\-commit schedule before completion\. At NFE 64, both settings share the full\-span endpoint\.

Table 20:FMLM\+ \(Init\) GSM8K mean pass@1 \(%\) across NFE\. Reference values are from Table 8 in\[[Agarwal et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib1)\]; our two rows use 4 generation seeds\.

### D\.2Numerical values underlying Figure[1](https://arxiv.org/html/2609.29102#S0.F1)

Tables[23](https://arxiv.org/html/2609.29102#A4.T23)and[23](https://arxiv.org/html/2609.29102#A4.T23)report our numerical results underlying Figure[1](https://arxiv.org/html/2609.29102#S0.F1)\(b–c\)\. Table[23](https://arxiv.org/html/2609.29102#A4.T23)gives the training\-token comparison in panel \(d\), including the external methods\.

Table 21:Figure[1](https://arxiv.org/html/2609.29102#S0.F1)\(b\): MBPP\-378 pass@10 \(%\)\.
Table 22:Figure[1](https://arxiv.org/html/2609.29102#S0.F1)\(c\): GSM8K mean pass@1 \(%\)\.

Table 23:Figure[1](https://arxiv.org/html/2609.29102#S0.F1)\(d\): task\-training tokens\. Token counts for methods that use TinyGSM are estimated using the reported mean sequence length; PlaidQ reports its base\-training tokens\.
### D\.3Additional representation analyses

#### D\.3\.1Clean encoder comparisons

##### Clean encoder configurations\.

Figure[4](https://arxiv.org/html/2609.29102#A4.F4)compares ELF\-B baseline using Qwen3\-0\.6B\-Base or t5\-small as the text encoder\. The configurations differ in data preprocessing as well as the clean text encoder due to tokenizer differences\.

Figure 4:As a text encoder, Qwen3\-0\.6B\-Base supports stronger learning and stronger gold\-answer recovery than t5\-small under comparable training and evaluation configurations\. Top: baseline learning curves on GSM8K and MMLU; the MMLU curves use full\-span generation at NFE 64\. Bottom: gold\-answer recovery after corrupting representations of supplied gold answers\. Both configurations achieve reliable gold\-answer recovery without corruption, while Qwen3\-0\.6B\-Base retains higher gold\-answer recovery when inputs are corrupted, especially on MMLU\. Generation and gold\-answer recovery with corrupted inputs average four seeds; gold\-answer recovery without corruption is evaluated once\.
##### Gold\-answer recovery\.

We test on epoch 6 and 30 checkpoints for GSM8K and MMLU, respectively, for gold\-answer recovery\. GSM8K supplies full gold worked solutions for both text encoder configurations\. MMLU supplies only the gold final\-answer letter at epoch 30\. Gold\-answer recovery without corruption decodes encoder states directly with one decoder\-mode forward pass\. For gold\-answer recovery with corrupted inputs, we choosettper example to match the requested noise/signal RMS ratio in equation[1](https://arxiv.org/html/2609.29102#S2.E1), then use one denoiser evaluation followed by decoding\. The ratio isRMS⁡\(\(1−t\)​σ​ε\)/RMS⁡\(t​x\)\\operatorname\{RMS\}\(\(1\-t\)\\sigma\\varepsilon\)/\\operatorname\{RMS\}\(tx\)over valid response positions, including the first EOS\. For a requested noise/signal RMS ratioqq, letX=RMS⁡\(x\)X=\\operatorname\{RMS\}\(x\)andE=RMS⁡\(ε\)E=\\operatorname\{RMS\}\(\\varepsilon\)over these positions; we uset=σ​E/\(q​X\+σ​E\)t=\\sigma E/\(qX\+\\sigma E\)\. Recovery scores the final numerical answer on GSM8K and the final\-answer letter on MMLU, averaging questions within each seed and then across seeds\. This evaluates each encoder together with its trained denoiser and decoder\.

#### D\.3\.2Single\-latent perturbation across corruption times

Zero REPA target position shift \(which is the default setup\) aligns student positioniito teacher positionii; REPA target position shift\-1 aligns student positioniito teacher positioni−1i\-1\. The student remains bidirectional\. Figure[5](https://arxiv.org/html/2609.29102#A4.F5)shows stronger forward influence for shift\-1 at the supervised block att=0\.7t=0\.7; this advantage disappears at the final block\.

Figure 5:REPA target position shift\-1 strengthens forward influence at the supervised block on GSM8K\. We replace the clean latent at a token representing a single digit with the latent obtained after substituting a different digit\. We measure hidden\-state change between the original and perturbed passes as1−1\-cosine similarity, then subtract the mean change over positions before the edited token from the mean change over positions after it\. Positive values indicate stronger forward influence\. Att=0\.7t=0\.7, shift\-1 has a larger effect than shift\-0 at the REPA\-supervised block \(dashed line\), but this advantage disappears at the final block\. ELF\-B checkpoints are from epoch 6\. REPA\+REG shift\-0 and shift\-1 are ELF\-REG\-B settings\.We select a token representing a single digit near the midpoint of a GSM8K gold rationale and replace that digit with a different digit that also occupies one token\. We encode the modified response and copy only the clean latent at the replacement token’s position into the original response; all other clean latents stay fixed\. The original and perturbed denoiser passes share the noise, prompt, REG state, and self\-conditioning inputs\. At the hidden states following each block, we measure hidden\-state change as1−1\-cosine similarity\. The earlier and later windows exclude the edited position and have equal length, given by the smaller distance from the edit to either end of the valid response\. Letδedit\\delta\_\{\\mathrm\{edit\}\}be the change at the edited position, and letδ−\\delta\_\{\-\}andδ\+\\delta\_\{\+\}be the mean changes in these earlier and later windows\. Positiveδ\+−δ−\\delta\_\{\+\}\-\\delta\_\{\-\}indicates stronger forward influence\. Each plotted value averages this difference over the same 1024 validation examples, separately for each block, corruption time, and condition\.

Figure[6](https://arxiv.org/html/2609.29102#A4.F6)extends thet=0\.7t=0\.7comparison tot=0\.05t=0\.05and0\.20\.2\. Shift\-1 has a larger later\-minus\-earlier change than shift\-0 at block 4 for all three times\. At block 12, shift\-1 has a slightly larger difference att=0\.05t=0\.05, but a smaller difference att=0\.2t=0\.2and0\.70\.7\. The final\-block pattern att=0\.7t=0\.7therefore depends on the corruption time\.

Figure 6:On GSM8K, the direction of perturbation influence depends on the block and corruption time\. Thet=0\.7t=0\.7panel shows the same measurements as Figure[5](https://arxiv.org/html/2609.29102#A4.F5)\. Each panel shows later\-minus\-earlier hidden\-state change, measured as1−1\-cosine similarity in equally sized windows around one edited latent\. Positive values indicate stronger forward influence\. Shift\-1 exceeds shift\-0 at the REPA\-supervised block \(dashed line\) at all three timettvalues, while their final\-block ordering varies with time\. We choose epoch 6 ELF\-B checkpoints\. REPA\+REG shift\-0 and shift\-1 are ELF\-REG\-B settings\.Table[24](https://arxiv.org/html/2609.29102#A4.T24)separates perturbation magnitude from direction at blocks 4 and 12\. For each example, normalized direction isν=\(δ\+−δ−\)/\(δ\+\+δ−\)\\nu=\(\\delta\_\{\+\}\-\\delta\_\{\-\}\)/\(\\delta\_\{\+\}\+\\delta\_\{\-\}\)when the denominator is positive, and zero otherwise\. We average this ratio across examples\. Table[24](https://arxiv.org/html/2609.29102#A4.T24)also reports the edited\-position change, both window means, and the difference between window means\. All statistics use the same single\-latent interventions as Figure[6](https://arxiv.org/html/2609.29102#A4.F6)\.

Table 24:Perturbation magnitude and direction on GSM8K at the supervised and final blocks\. Values average paired single\-latent interventions for the three ELF\-B settings at epoch 6\. REPA\+REG shift\-0 and shift\-1 are ELF\-REG\-B \(ours\) settings\. Shift\-1 has a larger later\-minus\-earlier change than shift\-0 at block 4 at each corruption time\. At block 12, this ordering reverses att=0\.2t=0\.2and0\.70\.7, while the difference att=0\.05t=0\.05is small\.δedit\\delta\_\{\\mathrm\{edit\}\}measures change at the edit,δ−\\delta\_\{\-\}andδ\+\\delta\_\{\+\}are means in matched earlier and later windows, andν\\nuis the mean per\-example normalized difference\. Hidden\-state change is1−1\-cosine similarity\.
#### D\.3\.3Position\-dependent token settling

We decode predicted\-clean states every four updates along a trajectory with 64 denoiser updates, and decode the final updated state at update 64\. A position’s settling time is the last observed update in the trajectory where the token changes \(in other words, the token does not change at subsequent observed updates\), with update 4 assigned if it never changes\. Figure[7](https://arxiv.org/html/2609.29102#A4.F7)averages this count within ten response\-position bins, excluding prompt and padding\. REPA\+REG shift\-0 settles earlier in every bin\. Settling does not progress monotonically from earlier to later positions\.

Figure 7:On GSM8K, REPA\+REG shift\-0 settles earlier in every response\-position bin\. Settling is the last observed update at which a token changes; lower values indicate earlier settling\. Positions are grouped within the final response for three ELF\-B conditions at epoch 6\. The pattern is not a monotonic progression from earlier to later tokens\. REPA\+REG shift\-0 and shift\-1 are ELF\-REG\-B settings\.

### D\.4Training ablations

##### Training settings\.

Tables[25](https://arxiv.org/html/2609.29102#A4.T25)and[26](https://arxiv.org/html/2609.29102#A4.T26)compare the performance of different training settings within each model size, at epoch 6 for ELF\-B and epoch 2\.5 for ELF\-L\. Appendix[B\.6](https://arxiv.org/html/2609.29102#A2.SS6)specifies their evaluation protocol\. We use the validation pass@1 accuracy in Tables[25](https://arxiv.org/html/2609.29102#A4.T25)and[26](https://arxiv.org/html/2609.29102#A4.T26)to determine the settings for our headline results\.

Table 25:GSM8K training ablations for ELF\-B at epoch 6\. Reference configurations appear in the top rows\. Each remaining row follows the default REPA\+REG setup \(alignment depth 4, last\-token REG target, response\-only text REPA, andλREG=0\.1\\lambda\_\{\\mathrm\{REG\}\}=0\.1\), except for the stated difference in setting\. The time interval restriction for REPA loss applies to text positions, excluding REG, on denoising examples; decoder\-example alignment is unchanged\. Values are single\-seed pass@1 \(%\) at 65 NFE\. Bold marks the best validation and test values across all rows\.Table 26:GSM8K alignment\-depth ablations for ELF\-L at epoch 2\.5\. Reference configurations appear in the top rows\. All settings follow the default ELF\-L setup except for the stated difference in alignment depth; the default is block 8\. Values are single\-seed pass@1 \(%\) at 65 NFE\. Other defaults and evaluation settings follow Appendix[B\.6](https://arxiv.org/html/2609.29102#A2.SS6)\. Bold marks the best validation and test values across all rows\.

### D\.5Sampling analyses

#### D\.5\.1Effect of the early\-stop ratio

The early\-stop sweep evaluatesρ∈\{1,2,4,8,16\}\\rho\\in\\\{1,2,4,8,16\\\}using the same ELF\-REG\-B checkpoint and all 16 generation seeds\. Forρ\>1\\rho\>1, generation decodes the predicted\-clean states afterN−1N\-1updates of the master grid\. Theρ=1\\rho=1control is full\-span generation and decodes the updated terminal states\. Table[27](https://arxiv.org/html/2609.29102#A4.T27)reports mean pass@1\. We chooseρ=8\\rho=8to balance stronger low\-NFE performance with gains at high NFE\. Figure[1](https://arxiv.org/html/2609.29102#S0.F1)and our headline results on math reasoning and code useρ=8\\rho=8at every budget\.

Table 27:GSM8K accuracy \(%\) for the early\-stop ratio sweep, ELF\-REG\-B \(ours\) at epoch 12\.##### Sensitivity to coarse\-sampling errors\.

We compare a full\-span 64\-NFE reference denoising trajectory with a nested 16\-NFE coarse trajectory\. At updates 16, 32, and 48, we inject the difference between the 64\-NFE and 16\-NFE states \(specifically, the noisy text and text self\-conditioning\) into the 64\-NFE reference, then finish generation after the error injection\. REG is not affected and evolves normally\. Figure[8](https://arxiv.org/html/2609.29102#A4.F8)injects each model’s own errors, errors from the other model, and random errors\. Cross\-model and random errors are normalized at each response position to the 64\-NFE recipient’s own combined text\-state error\. ELF\-REG\-B retains more shared solutions under its own error\. The ordering depends on the error source, and random errors of matched size are less damaging\.

The 64\-NFE reference time grid has 63 denoiser updates\. The coarse 16\-NFE time grid retains reference indices0,4,8,…,48,53,58,630,4,8,\\ldots,48,53,58,63, giving 15 updates and the same endpoint time\. At each injection time, the error is the coarse state minus the reference state\. We concatenate the noisy\-text error and previous\-clean\-prediction error at each response position\. Cross\-model errors are rescaled to the recipient’s own error norm at that position\. Random controls draw Gaussian noise in this concatenated space and rescale it to the same norm\. We split the vector back into the noisy\-text error and previous\-clean\-prediction error\. We scale both components of error by a multiplier from\{0,0\.5,1,2\}\\\{0,0\.5,1,2\\\}and add them to the corresponding components of the reference state\. We then continue generation using the remaining reference time grid\. Prompt positions stay fixed, and the reference REG state and its self\-conditioning are not modified by injection\.

Retention is the percentage of question–seed pairs whose numerical answer remains correct among the question–seed pairs that both unperturbed 64\-NFE reference runs solve\. We pool these shared successes over 4 generation seeds\. Thus the comparison holds the evaluated solutions fixed across error sources and 64\-NFE recipients\. Appendix[B\.6](https://arxiv.org/html/2609.29102#A2.SS6)gives checkpoint and sampling settings\.

Figure 8:Coarse\-sampling errors are more damaging than random errors of matched size\. Retention measures correctness on the 1,219 question–seed pairs both models solve before injection \(this is the number of questions solved across 4 seeds\)\. Rows identify the error source and columns the injection update; each curve identifies the receiving model\. The multiplier scales the injected error\. ELF\-REG\-B retains more answers under its own coarse error, while the recipient ordering changes with the error source\.
##### Full\-span answer retention\.

We select the question–seed pairs that both models solve at full\-span NFE 64, then measure correctness on those pairs at lower full\-span budgets\. Table[28](https://arxiv.org/html/2609.29102#A4.T28)reports this conditional retention\. These are separate full\-span trajectories at each budget, whereas Figure[2](https://arxiv.org/html/2609.29102#S3.F2)follows one continuing trajectory\.

Table 28:ELF\-REG\-B retains more shared solutions at every reduced full\-span budget\. Retention \(%\) is measured on the 4,693 question–seed pairs both ELF\-B baseline and ELF\-REG\-B solve at NFE 64\. Each lower budget uses its own full\-span trajectory; retention at NFE 64 is 100% by construction\.

## Appendix EComparison with related work

Table 29:Selected methods for language generation and representation supervision\. Panel A distinguishes the generated state, generation structure, and training or sampling approach\. Panel B distinguishes intermediate\-feature alignment from jointly denoised representations\. AR denotes autoregressive generation\.A\. Language generationMethodGeneration stateResponse generationTraining or sampling approachLangFlow\[[Chen et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib12)\]Token embeddingsParallel sequence denoisingFlow matching with a learned noise schedule\.LDLM\[[Meshchaninov et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib36)\]Learned contextual latentsParallel denoising, then token decodingJoint training of the latent encoder, denoiser, and decoder\.ELF\[[Hu et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib24)\]Contextual encoder representationsParallel denoising; final token decodingFlow matching with a shared denoiser and decoder\.S\-FLM\[[Deschenaux & Gulcehre, 2026](https://arxiv.org/html/2609.29102#bib.bib14)\]Hyperspherical token embeddingsParallel sequence denoisingRiemannian flow matching on learned embeddings\.FMLM\[[Lee et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib30)\]Noisy one\-hot token vectorsParallel sequence generationDistillation of a continuous flow into a flow map\.FMLM\+\[[Agarwal et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib1)\]Noisy one\-hot vectors and committed tokensAny\-order token commitmentFlow\-map self\-distillation; posterior refinement during sampling\.MLFM\[[Azangulov et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib7)\]Token embeddings and committed tokensContinuous updates with token commitmentMask\-conditioned flow training; confidence\-based token commitment\.PlaidQ\[[Peng et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib41)\]Token embeddingsParallel denoising; final token decodingAR initialization; distribution\-matching distillation for few\-step generation, paired trajectories for one\-step generation\.L2D\[[Cetin et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib10)\]Next\-token embeddingAR token sequenceA trainable diffusion path uses a frozen AR backbone\.LaDiR\[[Kang et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib28)\]Latent thought blocksBlockwise latent denoising; AR final answerLatent reasoning followed by answer generation conditioned on the latent thoughts\.CADD\[[Zheng et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib56)\]Discrete tokens and token embeddingsJoint continuous and discrete updatesContinuous token embeddings guide masked\-token prediction\.CCDD\[[Zhou et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib57)\]Discrete tokens and contextual latentsJoint continuous and discrete updatesJoint denoising of tokens and encoder representations\.ELF\-REG \(ours\)Encoder representations and global REG stateParallel full\-response denoising; final token decodingREPA\+REG training and prefix early\-stop sampling\.B\. Representation supervisionMethodSupervised representationSupervision sourceRole during generationREPA\[[Yu et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib54)\]Intermediate image\-denoiser featuresFrozen visual encoderAlignment is used during training; inference denoises image latents\.REG\[[Wu et al\., 2025](https://arxiv.org/html/2609.29102#bib.bib50)\]Global representation alongside image latents\. REPA is also applied\.Pretrained visual encoder’s class tokenThe global representation is jointly denoised with image latents\.Portuguese masked\-dLM REPA\[[Junior et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib27)\]Intermediate masked\-denoiser featuresClean text representations from pretrained encodersAlignment is used during training; inference uses masked token denoising\.REPR\-ALIGN\[[Peng et al\., 2026a](https://arxiv.org/html/2609.29102#bib.bib40)\]Intermediate masked\-denoiser featuresFrozen AR model used for initializationAlignment is used during training; inference uses masked token denoising\.TextLDM\[[Jiang et al\., 2026b](https://arxiv.org/html/2609.29102#bib.bib26)\]VAE encoder featuresFrozen AR language modelContinuous latent diffusion uses the trained VAE decoder for token generation\.CCDD\[[Zhou et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib57)\]Per\-token contextual latentsFixed pretrained text encoderContextual latents are jointly denoised with discrete tokens\.ELF\-REG \(ours\)Intermediate continuous\-denoiser features and global REG stateClean AR teacher statesThe global REG state is jointly denoised with the continuous response\. REPA supervises intermediate features during training\.Table 29:Selected methods for language generation and representation supervision \(continued\)\.### E\.1Early\-stop generation

Early\-stop methods differ in the evidence used to stop denoising and the predictions they finalize\. For continuous text diffusion,[Vaina et al\. \[2024\]](https://arxiv.org/html/2609.29102#bib.bib48)compare fixed exits with adaptive criteria based on prediction confidence or changes between steps\. Difformer\[[Gao et al\., 2024](https://arxiv.org/html/2609.29102#bib.bib17)\]motivates early stopping by showing that intermediate clean predictions can yield better text than later predictions\. In masked dLMs, confidence can determine either individual token commitments or termination of the remaining generation\. Just on Time\[[Kohut et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib29)\]finalizes confident tokens individually; Prophet\[[Li et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib31)\]uses confidence over an answer region to commit all remaining tokens and observes that answer tokens can emerge before the surrounding response settles\.

For continuous generation, stopping also requires choosing the state returned by the sampler\. Truncated Jump Sampling \(TJS\)\[[Peng & Gao, 2026](https://arxiv.org/html/2609.29102#bib.bib42)\]truncates image\-generation trajectories and returns a clean prediction, sharing the principle used by our prefix early\-stop sampler\. Its mean\-squared error decomposition concerns prediction of the clean data sample under the corruption distribution\. Our endpoint discrepancy compares the prediction with the endpoint of the same generated flow\. The curvature\-independent decomposition and our acceleration\-based bound therefore concern different errors\. FMLM\[[Lee et al\., 2026](https://arxiv.org/html/2609.29102#bib.bib30)\]already interprets the clean predictor as one Euler step over the remaining interval and learns flow maps for finite\-time transport\. We use the existing clean predictor for that approximation\. Token decoding introduces a further distinction: latent discrepancies need not change the decoded tokens\. The decoder\-margin argument of[Du & Ma \[2026, Theorem 1\]](https://arxiv.org/html/2609.29102#bib.bib16), applied in Proposition[2](https://arxiv.org/html/2609.29102#Thmproposition2), gives a sufficient condition for preserving the complete endpoint response, including an incorrect response\.

We use REPA\+REG to improve intermediate clean predictions in fully continuous diffusion language models and demonstrate strong low\-NFE mathematical reasoning and code generation through prefix early\-stop, without dedicated few\-step training\. On GSM8K, ELF\-REG\-B reduces endpoint error and brings earlier token stability\. Our analysis characterizes the discrepancy between an intermediate clean prediction and the endpoint of the same generated flow, bounds this discrepancy through remaining acceleration, and applies the decoder\-margin argument to complete\-response agreement \(Section[3\.1](https://arxiv.org/html/2609.29102#S3.SS1)\)\. We separately derive the endpoint discrepancy for the implemented self\-conditioned sampler \(Appendix[C\.3](https://arxiv.org/html/2609.29102#A3.SS3)\)\.

## Appendix FLimitations

We train separate models for GSM8K, MATH, code, and MMLU\. The code model is shared across the MBPP/HumanEval code benchmarks, and the MATH models are initialized from the corresponding GSM8K weights\. Joint training of a single generalist model across these tasks remains future work\. Our code training and evaluation also focus on Python, so the study does not cover other programming languages\. Extending the training data and benchmarks to additional programming languages is a promising further direction\. ELF is a continuous dLM trained on a fixed\-length response canvas; adapting it to different canvas lengths without performance reductions requires finetuning and we leave this as future work\. Due to compute constraints, we do not repeat training with multiple random seeds\.

## Appendix GQualitative Examples

We show one correct and one incorrect response from ELF\-REG\-L on each benchmark, using the checkpoints and settings of Tables[2](https://arxiv.org/html/2609.29102#S4.T2)and[1](https://arxiv.org/html/2609.29102#S4.T1): 64 NFE for GSM8K and 128 NFE for MATH\-500, HumanEval, and MBPP\-378, withρ=8\\rho=8early\-stop\. Prompts and generated responses are reproduced verbatim, with reference answers for incorrect responses\. Appendix[B\.3](https://arxiv.org/html/2609.29102#A2.SS3)gives the evaluation protocols\.

GSM8KCorrectPrompt[⬇](data:text/plain;base64,UXVlc3Rpb246IElmIEFubiBpcyA5IHllYXJzIG9sZCBhbmQgaGVyIGJyb3RoZXIgaXMgdHdpY2UgaGVyIGFnZSwgaG93IG9sZCB3aWxsIGhlciBicm90aGVyIGJlIGluIDMgeWVhcnM/CkFuc3dlcjo=)Question:IfAnnis9yearsoldandherbrotheristwiceherage,howoldwillherbrotherbein3years?Answer:Generated response[⬇](data:text/plain;base64,IElmIEFubiBpcyA5IHllYXJzIG9sZCwgdGhlbiBoZXIgYnJvdGhlciBpcyB0d2ljZSBoZXIgYWdlLCB3aGljaCBpcyA5IHggMiA9IDE4IHllYXJzIG9sZC4KSW4gMyB5ZWFycywgaGVyIGJyb3RoZXIgd2lsbCBiZSAxOCArIDMgPSAyMSB5ZWFycyBvbGQuCiMjIyMgMjE=)IfAnnis9yearsold,thenherbrotheristwiceherage,whichis9x2=18yearsold\.In3years,herbrotherwillbe18\+3=21yearsold\.\#\#\#\#21

GSM8KIncorrectPrompt[⬇](data:text/plain;base64,UXVlc3Rpb246IExvcmkgd2FudHMgdG8gYnV5IGEgJDMyMC4wMCBwYWlyIG9mIHNob2VzIGFuZCBhIG1hdGNoaW5nIGJlbHQgdGhhdCBpcyAkMzIuMDAuICBIZXIgcGFydC10aW1lIGpvYiBwYXlzIGhlciAkOC4wMCBhbiBob3VyLiAgSG93IG1hbnkgaG91cnMgd2lsbCBzaGUgaGF2ZSB0byB3b3JrIGJlZm9yZSBzaGUgY2FuIG1ha2UgaGVyIHB1cmNoYXNlPwpBbnN3ZXI6)Question:Loriwantstobuya$320\.00pairofshoesandamatchingbeltthatis$32\.00\.Herpart\-timejobpaysher$8\.00anhour\.Howmanyhourswillshehavetoworkbeforeshecanmakeherpurchase?Answer:Generated response[⬇](data:text/plain;base64,IExvcmkncyB0b3RhbCBleHBlbnNlcyBmb3IgdGhlIHNob2VzIGFuZCB0aGUgYmVsdCBhcmUgJDMyMCArIDMyID0gMzUyJC4KU2luY2Ugc2hlIGNhbiBtYWtlICQ4IHBlciBob3VyLCBzaGUgbmVlZHMgdG8gd29yayAkMzUyIC8gOCA9IDQzJCBob3Vycy4KVGh1cyBMb3JpIHdpbGwgaGF2ZSB0byB3b3JrIFxib3hlZHs0M30gaG91cnMgdG8gbWFrZSBoZXIgcHVyY2hhc2UuCiMjIyMgNDM=)Lori'stotalexpensesfortheshoesandthebeltare$320\+32=352$\.Sinceshecanmake$8perhour,sheneedstowork$352/8=43$hours\.ThusLoriwillhavetowork\\boxed\{43\}hourstomakeherpurchase\.\#\#\#\#43Reference answer[⬇](data:text/plain;base64,NDQ=)44

MATH\-500CorrectPrompt[⬇](data:text/plain;base64,UXVlc3Rpb246IElmICQzeCArIDIoMSArIHgpID0gMTckLCB3aGF0IGlzIHRoZSB2YWx1ZSBvZiAkNnggKyA1JD8KU3RlcC1ieS1TdGVwIEFuc3dlcjo=)Question:If$3x\+2\(1\+x\)=17$,whatisthevalueof$6x\+5$?Step\-by\-StepAnswer:Generated response[⬇](data:text/plain;base64,IEZpcnN0LCBzb2x2ZSB0aGUgZXF1YXRpb24gZm9yICR4JDoKXFsgM3ggKyAyICsgMnggPSAxNyBcUmlnaHRhcnJvdyA1eCArIDIgPSAxNyBcUmlnaHRhcnJvdyA1eCA9IDE1IFxSaWdodGFycm93IHggPSAzIFxdCgpOb3cgc3Vic3RpdHV0ZSB0aGUgdmFsdWUgb2YgaW50byB0aGUgZXF1YXRpb246ClxbIDZ4ICsgNSA9IDYoMykgKyA1ID0gMTggKyA1ID0gXGJveGVkezIzfSBcXQpGaW5hbCBhbnN3ZXI6ICRcYm94ZWR7MjN9JA==)First,solvetheequationfor$x$:\\\[3x\+2\+2x=17\\Rightarrow5x\+2=17\\Rightarrow5x=15\\Rightarrowx=3\\\]Nowsubstitutethevalueofintotheequation:\\\[6x\+5=6\(3\)\+5=18\+5=\\boxed\{23\}\\\]Finalanswer:$\\boxed\{23\}$

MATH\-500IncorrectPrompt[⬇](data:text/plain;base64,UXVlc3Rpb246IElmICRhXHN0YXIgYiA9IDlhKzJiLWFiKzUkLCB3aGF0IGlzIHRoZSB2YWx1ZSBvZiAkNVxzdGFyMSQ/ClN0ZXAtYnktU3RlcCBBbnN3ZXI6)Question:If$a\\starb=9a\+2b\-ab\+5$,whatisthevalueof$5\\star1$?Step\-by\-StepAnswer:Generated response[⬇](data:text/plain;base64,IFRvIGZpbmQgdGhlIHZhbHVlIG9mICQ1XHN0YXIxJCwgd2UgbmVlZCB0byBzdWJzdGl0dXRlICRhPTUkIGFuZCAkYj0xJCBpbnRvIHRoZSBnaXZlbiBlcXVhdGlvbi4KClxbIGFcc3RhciBiID0gOWErMmItYWIrNSBcXQoKXFsgNVxzdGFyMSA9IDkoNSkrMigxKS0oNSkoMSkrNSBcUmlnaHRhcnJvdyA1XHN0YXIxID0gNDUrMi01KzUgXFJpZ2h0YXJyb3cgNVxzdGFyMSA9IDQ1IFxdCgpTbyB0aGUgYW5zd2VyIGlzICRcYm94ZWR7NDV9JC4KRmluYWwgYW5zd2VyOiAkXGJveGVkezQ1fSQ=)Tofindthevalueof$5\\star1$,weneedtosubstitute$a=5$and$b=1$intothegivenequation\.\\\[a\\starb=9a\+2b\-ab\+5\\\]\\\[5\\star1=9\(5\)\+2\(1\)\-\(5\)\(1\)\+5\\Rightarrow5\\star1=45\+2\-5\+5\\Rightarrow5\\star1=45\\\]Sotheansweris$\\boxed\{45\}$\.Finalanswer:$\\boxed\{45\}$Reference answer[⬇](data:text/plain;base64,NDc=)47

HumanEvalCorrectPrompt[⬇](data:text/plain;base64,V3JpdGUgYSBjb21wbGV0ZSBQeXRob24gc29sdXRpb24uIFJldHVybiBvbmx5IGNvZGUuCgpkZWYgZ2V0X3Bvc2l0aXZlKGw6IGxpc3QpOgogICAgIiIiUmV0dXJuIG9ubHkgcG9zaXRpdmUgbnVtYmVycyBpbiB0aGUgbGlzdC4KICAgID4+PiBnZXRfcG9zaXRpdmUoWy0xLCAyLCAtNCwgNSwgNl0pCiAgICBbMiwgNSwgNl0KICAgID4+PiBnZXRfcG9zaXRpdmUoWzUsIDMsIC01LCAyLCAtMywgMywgOSwgMCwgMTIzLCAxLCAtMTBdKQogICAgWzUsIDMsIDIsIDMsIDksIDEyMywgMV0KICAgICIiIgoKUHl0aG9uIHNvbHV0aW9uOgo=)WriteacompletePythonsolution\.Returnonlycode\.defget\_positive\(l:list\):"""Returnonlypositivenumbersinthelist\.\>\>\>get\_positive\(\[\-1,2,\-4,5,6\]\)\[2,5,6\]\>\>\>get\_positive\(\[5,3,\-5,2,\-3,3,9,0,123,1,\-10\]\)\[5,3,2,3,9,123,1\]"""Pythonsolution:Generated response[⬇](data:text/plain;base64,ZGVmIGdldF9wb3NpdGl2ZShsOiBsaXN0KSAtPiBsaXN0OgogICAgIiIiUmV0dXJuIG9ubHkgcG9zaXRpdmUgbnVtYmVycyBpbiB0aGUgbGlzdCBsLiIiIgogICAgcmV0dXJuIFtudW0gZm9yIG51bSBpbiBsIGlmIG51bSA+IDBd)defget\_positive\(l:list\)\-\>list:"""Returnonlypositivenumbersinthelistl\."""return\[numfornuminlifnum\>0\]

HumanEvalIncorrectPrompt[⬇](data:text/plain;base64,V3JpdGUgYSBjb21wbGV0ZSBQeXRob24gc29sdXRpb24uIFJldHVybiBvbmx5IGNvZGUuCgpkZWYgdHJpYW5nbGVfYXJlYShhLCBoKToKICAgICIiIkdpdmVuIGxlbmd0aCBvZiBhIHNpZGUgYW5kIGhpZ2ggcmV0dXJuIGFyZWEgZm9yIGEgdHJpYW5nbGUuCiAgICA+Pj4gdHJpYW5nbGVfYXJlYSg1LCAzKQogICAgNy41CiAgICAiIiIKClB5dGhvbiBzb2x1dGlvbjoK)WriteacompletePythonsolution\.Returnonlycode\.deftriangle\_area\(a,h\):"""Givenlengthofasideandhighreturnareaforatriangle\.\>\>\>triangle\_area\(5,3\)7\.5"""Pythonsolution:Generated response[⬇](data:text/plain;base64,ZGVmIHRyaWFuZ2xlX2FyZWEoYSwgaCk6CiAgICAiIiIKICAgIENhbGN1bGF0ZSB0aGUgYXJlYSBvZiBhIHRyaWFuZ2xlIGdpdmVuIHNpZGUgYW5kIGhpZ2guCgogICAgUGFyYW1ldGVyczoKICAgIGEgKGZsb2F0KTogTGVuZ3RoIG9mIHRoZSBzaWRlLgogICAgaCAoZmxvYXQpOiBIaWdoIG9mIHRoZSBzaWRlLgoKICAgIFJldHVybnM6CiAgICBmbG9hdDogQXJlYSBvZiB0aGUgdHJpYW5nbGUuCiAgICAiIiIKICAgIHJldHVybiBhICogaCAqKiAy)deftriangle\_area\(a,h\):"""Calculatetheareaofatrianglegivensideandhigh\.Parameters:a\(float\):Lengthoftheside\.h\(float\):Highoftheside\.Returns:float:Areaofthetriangle\."""returna\*h\*\*2Reference answer[⬇](data:text/plain;base64,ZGVmIHRyaWFuZ2xlX2FyZWEoYSwgaCk6CiAgICAiIiJHaXZlbiBsZW5ndGggb2YgYSBzaWRlIGFuZCBoaWdoIHJldHVybiBhcmVhIGZvciBhIHRyaWFuZ2xlLgogICAgPj4+IHRyaWFuZ2xlX2FyZWEoNSwgMykKICAgIDcuNQogICAgIiIiCgoKICAgIHJldHVybiBhICogaCAvIDI=)deftriangle\_area\(a,h\):"""Givenlengthofasideandhighreturnareaforatriangle\.\>\>\>triangle\_area\(5,3\)7\.5"""returna\*h/2

MBPP\-378CorrectPrompt[⬇](data:text/plain;base64,V3JpdGUgYSBjb21wbGV0ZSBQeXRob24gc29sdXRpb24uIFJldHVybiBvbmx5IGNvZGUuCgoiIiIKV3JpdGUgYSBmdW5jdGlvbiB0byBmaW5kIHRoZSB2b2x1bWUgb2YgYSBjdWJlIGdpdmVuIGl0cyBzaWRlIGxlbmd0aC4KYXNzZXJ0IHZvbHVtZV9jdWJlKDMpPT0yNwoiIiIKClB5dGhvbiBzb2x1dGlvbjoK)WriteacompletePythonsolution\.Returnonlycode\."""Writeafunctiontofindthevolumeofacubegivenitssidelength\.assertvolume\_cube\(3\)==27"""Pythonsolution:Generated response[⬇](data:text/plain;base64,ZGVmIHZvbHVtZV9jdWJlKHNpZGVfbGVuZ3RoKToKICAgICIiIgogICAgQ2FsY3VsYXRlIHRoZSB2b2x1bWUgb2YgYSBjdWJlIGdpdmVuIGl0cyBzaWRlIGxlbmd0aC4KCiAgICBQYXJhbWV0ZXJzOgogICAgc2lkZV9sZW5ndGggKGZsb2F0KTogVGhlIHNpZGUgbGVuZ3RoIG9mIHRoZSBjdWJlLgoKICAgIFJldHVybnM6CiAgICBmbG9hdDogVGhlIHZvbHVtZSBvZiB0aGUgY3ViZS4KICAgICIiIgogICAgcmV0dXJuIHNpZGVfbGVuZ3RoICoqIDM=)defvolume\_cube\(side\_length\):"""Calculatethevolumeofacubegivenitssidelength\.Parameters:side\_length\(float\):Thesidelengthofthecube\.Returns:float:Thevolumeofthecube\."""returnside\_length\*\*3

MBPP\-378IncorrectPrompt[⬇](data:text/plain;base64,V3JpdGUgYSBjb21wbGV0ZSBQeXRob24gc29sdXRpb24uIFJldHVybiBvbmx5IGNvZGUuCgoiIiIKV3JpdGUgYSBmdW5jdGlvbiB0byBmaW5kIHRoZSBudGggb2N0YWdvbmFsIG51bWJlci4KYXNzZXJ0IGlzX29jdGFnb25hbCg1KSA9PSA2NQoiIiIKClB5dGhvbiBzb2x1dGlvbjoK)WriteacompletePythonsolution\.Returnonlycode\."""Writeafunctiontofindthenthoctagonalnumber\.assertis\_octagonal\(5\)==65"""Pythonsolution:Generated response[⬇](data:text/plain;base64,ZGVmIGlzX29jdGFnb25hbChuKToKICAgICIiIgogICAgUmV0dXJucyB0aGUgbnRoIG9jdGFnb25hbCBudW1iZXIuCgogICAgUGFyYW1ldGVyczoKICAgIG4gKGludCk6IFRoZSBwb3NpdGlvbiBpbiB0aGUgb2N0YWdvbmFsIHNlcXVlbmNlLgoKICAgIFJldHVybnM6CiAgICBpbnQ6IFRoZSBudGggb2N0YWdvbmFsIG51bWJlci4KICAgICIiIgogICAgcmV0dXJuIG4gKiAobiArIDEpIC8vIDI=)defis\_octagonal\(n\):"""Returnsthenthoctagonalnumber\.Parameters:n\(int\):Thepositionintheoctagonalsequence\.Returns:int:Thenthoctagonalnumber\."""returnn\*\(n\+1\)//2Reference answer[⬇](data:text/plain;base64,ZGVmIGlzX29jdGFnb25hbChuKToKCXJldHVybiAzICogbiAqIG4gLSAyICogbg==)defis\_octagonal\(n\):return3\*n\*n\-2\*n