Teaching Diffusion to Speculate Left-to-Right

# Teaching Diffusion to Speculate Left-to-Right
Source: [https://arxiv.org/html/2606.11552](https://arxiv.org/html/2606.11552)
###### Abstract

Large language models \(LLMs\) achieve remarkable performance across a wide range of tasks, but their autoregressive decoding process incurs substantial inference costs due to inherently sequential token generation\. Speculative decoding addresses this bottleneck by employing a lightweight draft model to propose multiple future tokens that are subsequently verified in parallel by a larger target model\. Recent work has demonstrated that diffusion language models are well suited for this setting, as they can generate entire blocks of draft tokens in parallel and thereby alleviate the sequential constraints of autoregressive drafting\. A subtlety of this regime is that block\-diffusion drafters generate tokensbidirectionallywithin a block, whereas verification is performed by an autoregressive target model that evaluates tokens in a strictlyleft\-to\-right manner, leaving a gap between the symmetric training\-time objective and the asymmetric verification\-time reward\. In this work, we offer an empirical analysis of three training\-time interventions that narrow this gap: token positional weighting, a first\-error focal loss that targets the position that breaks the accepted prefix within each block, and a chain loss term that substitutes a differentiable surrogate for the expected accepted length\. The three interventions act along orthogonal axes \(position, block\-conditional first error, joint prefix\) and compose additively; they are likewise orthogonal to test\-time alignment mechanisms such as multi\-draft self\-selection, with which they can in principle be combined\. Across four target models and six reasoning, code, and dialogue benchmarks, the three interventions raise accepted draft length by2121–76%76\\%per benchmark over a position\-uniform baseline, without adding additional forward passes and without changing the inference pipeline or the rejection\-sampling exactness contract\.

![Refer to caption](https://arxiv.org/html/2606.11552v1/x1.png)Figure 1:The training–verification mismatch in diffusion\-based speculative drafters and the three training\-time interventions we analyse\.\(a\) Standard training\.The drafter is trained with fully bidirectional attention over theKK\-token block, so every position is conditioned symmetrically on every other\. At inference time, however, the target verifies the block strictly left\-to\-right: a single early rejection \(here, position22\) truncates the entire suffix\. The drafter’saccdmatches the ground\-truthabcdat positions33and44, but both are discarded along with the rejected position\-22token\.\(b\) Aligned training\.We analyze three complementary training\-time interventions that bring the drafter’s objective into closer correspondence with the causal acceptance contract: position\-wise loss decay \(Weighting\), an auxiliary cross\-entropy term targeting the first mispredicted position of each block \(Focal\), and a differentiable surrogate for the joint prefix acceptance probability \(Chaining\)\. Together they raise the expected accepted prefix length and recover the per\-token correctness that bidirectional training already provides\.## 1Introduction

Large language models \(LLMs\) have become a central computational primitive across applications ranging from conversational assistants and code generation\(Chenet al\.[2021](https://arxiv.org/html/2606.11552#bib.bib1); Rozièreet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib2)\)to retrieval\-augmented question answering, multi\-step agent systems, and long\-horizon reasoning\(OpenAI[2024](https://arxiv.org/html/2606.11552#bib.bib3); DeepSeek\-AIet al\.[2025](https://arxiv.org/html/2606.11552#bib.bib4)\)\. As these systems mature, the dominant cost in their lifecycle has shifted from training to inference: a model is trained once but served continuously, and per\-query cost is amplified by output length, by recursive agentic invocations, and by the long chains of thought emitted by recent reasoning models\. Reducing the latency and per\-token cost of LLM inference is therefore one of the most consequential systems problems in contemporary machine learning\.

Single\-stream LLM decoding is bottlenecked by memory bandwidth rather than arithmetic throughput: each autoregressive step streams the entire parameter set from high\-bandwidth memory \(HBM\) to produce a single token, an imbalance that widens with every hardware generation as compute scales faster than HBM\(Popeet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib5)\)\. The standard responses—quantization\(Dettmerset al\.[2022](https://arxiv.org/html/2606.11552#bib.bib6); Frantaret al\.[2023](https://arxiv.org/html/2606.11552#bib.bib7); Linet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib8)\), sparsity\(Frantar and Alistarh[2023](https://arxiv.org/html/2606.11552#bib.bib10)\), distillation\(Hintonet al\.[2015](https://arxiv.org/html/2606.11552#bib.bib12)\), and system\-level techniques such as FlashAttention\(Daoet al\.[2022](https://arxiv.org/html/2606.11552#bib.bib14)\), PagedAttention, and continuous batching\(Kwonet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib16); Yuet al\.[2022](https://arxiv.org/html/2606.11552#bib.bib17)\)—all reduce the cost of a single forward pass\. Speculative decoding\(Leviathanet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib20); Chenet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib21)\)instead reduces the*number*of serialized forward passes required: a cheap drafter proposesKKcandidate tokens, the target verifies them in a single parallel pass, and a rejection\-sampling step accepts the longest prefix consistent with the target distribution\. The procedure is exact—samples are statistically identical to those of the target model—which eliminates the quality–versus–speed trade\-off that complicates the other techniques and makes the central question purely one of drafter design: how to construct a draft distribution that is simultaneously cheap to evaluate and well aligned with the target\.

The EAGLE family\(Liet al\.[2024b](https://arxiv.org/html/2606.11552#bib.bib27),[a](https://arxiv.org/html/2606.11552#bib.bib28),[2025b](https://arxiv.org/html/2606.11552#bib.bib29)\)pursues this goal by drafting in*feature*space rather than token space, coupling a small auxiliary head to the target’s own hidden states\. Successive iterations have introduced dynamic tree expansion and multi\-layer feature aggregation, and EAGLE\-3 has become the de facto baseline for production speculative decoding\. The EAGLE drafter nevertheless remains*autoregressive*within each speculation step, requiring token predictions to be generated sequentially and thereby limiting its maximum achievable speedup\.

In this work we focus on*block\-diffusion drafters*—of which DFlash\(Chenet al\.[2026](https://arxiv.org/html/2606.11552#bib.bib48)\)is a prominent recent example—a class that replaces autoregressive feature prediction with a*diffusion\-style*parallel block decoder\. Conditioned on a configurable set of intermediate target hidden states, such a drafter emits an entireKK\-token block in a single non\-autoregressive forward pass; the block is verified by the target under the standard rejection\-sampling contract, preserving exactness\. WithK=16K=16, this raises the per\-step ceiling by more than a factor of three over EAGLE\-3 while retaining competitive per\-token acceptance through multi\-layer conditioning\. Pairing a block\-diffusion drafter with an autoregressive target also surfaces a representational asymmetry that is less pronounced in the autoregressive\-drafter regime: diffusion models are trained to denoise blocks under fully bidirectional attention, so each position conditions on context from both directions, whereas the target verifies strictly left\-to\-right and accepts only the longest causally consistent prefix\. The drafter must therefore allocate its predictive capacity asymmetrically—early positions are disproportionately load\-bearing, since a single early divergence truncates all subsequent draft tokens regardless of their quality—an objective in tension with the symmetric denoising loss inherited from the diffusion formulation\. The present work studies this tension empirically by analysing three training\-time interventions that reshape the position\-wise loss profile along orthogonal axes; the analysis is complementary to test\-time alignment mechanisms developed for the same drafter class, such as ddTree\(Ringel and Romano[2026](https://arxiv.org/html/2606.11552#bib.bib50)\)\.

## 2Related Work

#### Efficient LLM inference\.

A broad portfolio of techniques mitigates the memory\-bandwidth cost of single\-stream decoding: low\-bit quantization\(Dettmerset al\.[2022](https://arxiv.org/html/2606.11552#bib.bib6); Frantaret al\.[2023](https://arxiv.org/html/2606.11552#bib.bib7); Linet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib8); Xiaoet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib9)\), sparsity\(Frantar and Alistarh[2023](https://arxiv.org/html/2606.11552#bib.bib10); Sunet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib11)\), distillation\(Hintonet al\.[2015](https://arxiv.org/html/2606.11552#bib.bib12); Sanhet al\.[2019](https://arxiv.org/html/2606.11552#bib.bib13)\), FlashAttention\(Daoet al\.[2022](https://arxiv.org/html/2606.11552#bib.bib14); Dao[2024](https://arxiv.org/html/2606.11552#bib.bib15)\), and system\-level techniques such as PagedAttention, continuous batching, and chunked prefill\(Kwonet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib16); Yuet al\.[2022](https://arxiv.org/html/2606.11552#bib.bib17); Agrawalet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib18)\)\.

#### Speculative decoding\.

Block\-parallel decoding was first explored as a deterministic acceleration technique\(Sternet al\.[2018](https://arxiv.org/html/2606.11552#bib.bib19)\); the modern, distribution\-preserving formulation was introduced concurrently byLeviathanet al\.\([2023](https://arxiv.org/html/2606.11552#bib.bib20)\)andChenet al\.\([2023](https://arxiv.org/html/2606.11552#bib.bib21)\)\. SpecInfer\(Miaoet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib22)\)generalised linear\-chain drafts to trees verified in parallel under tree\-structured attention, substantially raising expected accepted lengths\. Subsequent work has expanded the design space along largely orthogonal axes: Medusa\(Caiet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib23)\)attaches parallel prediction heads directly to the target, Lookahead Decoding\(Fuet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib24)\)sidesteps drafter training via Jacobi\-style fixed\-point iteration on n\-gram trajectories, self\-speculative decoding\(Zhanget al\.[2024a](https://arxiv.org/html/2606.11552#bib.bib25)\)reuses a subset of target layers as the drafter, and online speculative decoding\(Liuet al\.[2024b](https://arxiv.org/html/2606.11552#bib.bib26)\)adapts the drafter continuously to the deployed workload—each accepting the same exactness contract under a different cheap\-candidate mechanism\.

#### Feature\-level drafters\.

A particularly successful line draws candidates from the target’s own intermediate representations rather than from an independently trained small LM\. EAGLE\(Liet al\.[2024b](https://arxiv.org/html/2606.11552#bib.bib27)\)introduced feature\-level autoregression; EAGLE\-2\(Liet al\.[2024a](https://arxiv.org/html/2606.11552#bib.bib28)\)added context\-dependent dynamic tree expansion; and EAGLE\-3\(Liet al\.[2025b](https://arxiv.org/html/2606.11552#bib.bib29)\)augmented the drafter with multi\-layer feature aggregation, establishing the de facto baseline for production speculative decoding\. Related designs include Hydra\(Ankneret al\.[2024](https://arxiv.org/html/2606.11552#bib.bib30)\), GliDe with CaPE\(Duet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib31)\), Kangaroo\(Liuet al\.[2024a](https://arxiv.org/html/2606.11552#bib.bib32)\), and HASS\(Zhanget al\.[2024b](https://arxiv.org/html/2606.11552#bib.bib33)\)\. All remain*autoregressive*within a speculation step, so their empirical horizons are bounded by tree depth rather than by the cost of a single drafter pass\.

#### Diffusion and non\-autoregressive language modelling\.

Parallel\-block generation has a long history outside speculative decoding: non\-autoregressive translation\(Guet al\.[2018](https://arxiv.org/html/2606.11552#bib.bib34)\)first demonstrated single\-pass sequence generation, and iterative refinement\(Leeet al\.[2018](https://arxiv.org/html/2606.11552#bib.bib35)\)together with masked\-LM decoders such as Mask\-Predict\(Ghazvininejadet al\.[2019](https://arxiv.org/html/2606.11552#bib.bib36)\)and SUNDAE\(Savinovet al\.[2022](https://arxiv.org/html/2606.11552#bib.bib37)\)recovered much of the resulting quality gap via repeated denoising\. Discrete diffusion LMs\(Austinet al\.[2021a](https://arxiv.org/html/2606.11552#bib.bib38); Liet al\.[2022](https://arxiv.org/html/2606.11552#bib.bib39); Gonget al\.[2023](https://arxiv.org/html/2606.11552#bib.bib40); Gulrajani and Hashimoto[2023](https://arxiv.org/html/2606.11552#bib.bib41); Louet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib42); Nieet al\.[2025](https://arxiv.org/html/2606.11552#bib.bib43)\)scaled this paradigm to LLM size via bidirectional denoising of masked blocks, and block diffusion\(Arriolaet al\.[2025](https://arxiv.org/html/2606.11552#bib.bib44)\)interpolates between this regime and autoregression by combining within\-block bidirectional denoising with between\-block causal conditioning\. More recent work targets diffusion LMs as efficient generators in their own right: EfficientDLM\(Fuet al\.[2026b](https://arxiv.org/html/2606.11552#bib.bib66)\)uses position\-dependent token masking; TiDAR\(Liuet al\.[2025](https://arxiv.org/html/2606.11552#bib.bib65)\)casts a single model as both drafter and verifier; and Nemotron\-Labs\-Diffusion\(Fuet al\.[2026a](https://arxiv.org/html/2606.11552#bib.bib67)\)introduces a tri\-modal architecture unifying autoregressive, diffusion, and self\-speculative decoding\. The natural synthesis pursued in this work—block\-diffusion models as speculative drafters, of which DFlash\(Chenet al\.[2026](https://arxiv.org/html/2606.11552#bib.bib48)\)is a prominent recent example—inherits the parallel\-block speedup of these models while introducing, as we show in Section[4](https://arxiv.org/html/2606.11552#S4), a training objective in tension with the causal acceptance contract\.

#### Aligning drafters with the acceptance contract\.

A rapidly growing literature targets the same gap we do\.Zhouet al\.\([2024](https://arxiv.org/html/2606.11552#bib.bib56)\)study reverse\-KL and total\-variation alternatives to the standard forward\-KL distillation loss for autoregressive drafters, and the recent LK losses ofSamarinet al\.\([2026](https://arxiv.org/html/2606.11552#bib.bib49)\)optimize a TV\-based per\-token acceptance objective\. These methods change the per\-token*loss family*but weight allKKdraft positions uniformly and remain agnostic to the joint\-prefix structure of the acceptance contract; they are also developed against autoregressive drafters \(Medusa\-style heads, EAGLE variants, MTP modules\) rather than bidirectional block decoders\. SpecDiff\-2 ofSandleret al\.\([2025](https://arxiv.org/html/2606.11552#bib.bib51)\), targets the same block\-diffusion\-drafter / autoregressive\-verifier mismatch via*streak\-distillation*—a fine\-tuning objective maximising a differentiable surrogate for the expected accepted streak under verifier\-sampled teacher trajectories—paired with a test\-time*self\-selection acceptance*mechanism\. The chain reward \(Section[5\.5](https://arxiv.org/html/2606.11552#S5.SS5)\) shares streak\-distillation’s animating idea but evaluates the surrogate along the teacher\-forced ground\-truth tokens already materialized byℒCE\\mathcal\{L\}\_\{\\mathrm\{CE\}\}, adding only a cumulative sum and exponentiation per block—no verifier rollout \(Section[6\.5](https://arxiv.org/html/2606.11552#S6.SS5)shows that we can use both\)\. A contemporaneous D\-PACE\(Wuet al\.[2026](https://arxiv.org/html/2606.11552#bib.bib52)\)also targets this gap with a prefix\-product accepted\-length surrogate similar to the chain reward we investigate, but applies its gradient as a detached per\-position weight on the standard cross\-entropy rather than as an additive reward term\. The present analysis is otherwise scoped entirely to training\-time interventions and leaves the inference pipeline at the standard rejection\-sampling contract; test\-time mechanisms such as the tree\-based selection of DDTree\(Ringel and Romano[2026](https://arxiv.org/html/2606.11552#bib.bib50)\)are orthogonal and can be combined with the interventions we explore \( Sections[6\.4](https://arxiv.org/html/2606.11552#S6.SS4)–[6\.5](https://arxiv.org/html/2606.11552#S6.SS5)\)\.

## 3Preliminaries

### 3\.1Notation

We consider an autoregressive target modelpϕ​\(xt∣x<t\)p\_\{\\phi\}\(x\_\{t\}\\mid x\_\{<t\}\)over a vocabulary𝒱\\mathcal\{V\}with parametersϕ\\phi, generating sequencesx1:Tx\_\{1:T\}according topϕ​\(x1:T\)=∏t=1Tpϕ​\(xt∣x<t\)p\_\{\\phi\}\(x\_\{1:T\}\)=\\prod\_\{t=1\}^\{T\}p\_\{\\phi\}\(x\_\{t\}\\mid x\_\{<t\}\)\. We writeh1:t\(ℓ\)h^\{\(\\ell\)\}\_\{1:t\}for the hidden states produced by theℓ\\ell\-th transformer block ofpϕp\_\{\\phi\}when conditioned onx<tx\_\{<t\}\. A*drafter*is a parametric distributionqψ​\(xt∣x<t\)q\_\{\\psi\}\(x\_\{t\}\\mid x\_\{<t\}\)designed to approximatepϕp\_\{\\phi\}at lower per\-token cost\.

### 3\.2Speculative Decoding

Given a prefixx<tx\_\{<t\}and a draft horizonK∈ℕK\\in\\mathbb\{N\}, one iteration of speculative decoding\(Leviathanet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib20); Chenet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib21)\)proceeds in three phases\. The drafter first samplesKKcandidate tokensx~t,…,x~t\+K−1\\tilde\{x\}\_\{t\},\\ldots,\\tilde\{x\}\_\{t\+K\-1\}fromqψq\_\{\\psi\}\. The target is then invoked once on the extended sequence and produces, in a single parallel pass, the conditionalspϕ\(⋅∣x<t\+k\)p\_\{\\phi\}\(\\cdot\\mid x\_\{<t\+k\}\)for allk∈\{0,…,K\}k\\in\\\{0,\\ldots,K\\\}, wherex<t\+k≡\(x<t,x~t:t\+k−1\)x\_\{<t\+k\}\\equiv\(x\_\{<t\},\\tilde\{x\}\_\{t:t\+k\-1\}\)\. The candidates are finally traversed left\-to\-right;x~t\+k\\tilde\{x\}\_\{t\+k\}is accepted with probability

ak=min⁡\(1,pϕ​\(x~t\+k∣x<t\+k\)qψ​\(x~t\+k∣x<t\+k\)\),a\_\{k\}\\;=\\;\\min\\\!\\left\(1,\\;\\frac\{p\_\{\\phi\}\(\\tilde\{x\}\_\{t\+k\}\\mid x\_\{<t\+k\}\)\}\{q\_\{\\psi\}\(\\tilde\{x\}\_\{t\+k\}\\mid x\_\{<t\+k\}\)\}\\right\),\(1\)and on first rejection a replacement is drawn from the residual distribution

xt\+k∼norm\(max\(0,pϕ\(⋅∣x<t\+k\)−qψ\(⋅∣x<t\+k\)\)\),x\_\{t\+k\}\\;\\sim\\;\\mathrm\{norm\}\\\!\\big\(\\max\(0,\\;p\_\{\\phi\}\(\\cdot\\mid x\_\{<t\+k\}\)\-q\_\{\\psi\}\(\\cdot\\mid x\_\{<t\+k\}\)\)\\big\),\(2\)terminating the iteration\. If allKKcandidates are accepted, one bonus token is sampled frompϕ\(⋅∣x<t\+K\)p\_\{\\phi\}\(\\cdot\\mid x\_\{<t\+K\}\)at no extra cost\. The tokens emitted by this procedure are distributed identically to those of standard autoregressive sampling frompϕp\_\{\\phi\}\(Leviathanet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib20)\)\.

![Refer to caption](https://arxiv.org/html/2606.11552v1/x2.png)Figure 2:Per\-position breakdown of the drafter’s correct predictions on HumanEval, for a position\-uniform DFlash baseline trained against Llama\-3\-8B\-Instruct with block sizeK=16K=16\. Bars are stacked: the green segment is the fraction of draft tokens accepted under the rejection\-sampling contract \([1](https://arxiv.org/html/2606.11552#S3.E1)\), and the red segment is the fraction that actually match the target, but is nevertheless discarded as a consequence of an upstream rejection within the same block\. Per\-position correctness \(total bar height\) decays only mildly with offsetkk, whereas per\-position acceptance \(green\) decays at the geometric rate implied by \([3](https://arxiv.org/html/2606.11552#S3.E3)\), so the discarded fraction grows monotonically and dominates beyond a short prefix\.Writingαk=𝔼x~​\[ak\]\\alpha\_\{k\}=\\mathbb\{E\}\_\{\\tilde\{x\}\}\[a\_\{k\}\]for the marginal acceptance probability at positionkkand assuming i\.i\.d\. rateα\\alphaacross positions, the expected number of tokens emitted per iteration is

τ​\(α,K\)=1−αK\+11−α,\\tau\(\\alpha,K\)\\;=\\;\\frac\{1\-\\alpha^\{K\+1\}\}\{1\-\\alpha\},\(3\)including the bonus token\. Withc∈\[0,1\]c\\in\[0,1\]the wall\-clock cost of one drafter pass relative to one target pass, the expected speedup over standard decoding is

𝒮​\(α,K,c\)=1−αK\+1\(1−α\)​\(1\+K​c\)\.\\mathcal\{S\}\(\\alpha,K,c\)\\;=\\;\\frac\{1\-\\alpha^\{K\+1\}\}\{\(1\-\\alpha\)\\,\(1\+Kc\)\}\.\(4\)𝒮\\mathcal\{S\}grows with bothKKandα\\alpha, but each increment toKKalso incurs additive drafter cost and, typically, a decrease inα\\alphaat deeper positions\(Liet al\.[2024a](https://arxiv.org/html/2606.11552#bib.bib28)\)\. Drafter design reduces to maximising the realised profile\{αk\}\\\{\\alpha\_\{k\}\\\}subject to a constraint oncc\.

### 3\.3Block\-Diffusion Drafters

Discrete diffusion language models\(Austinet al\.[2021a](https://arxiv.org/html/2606.11552#bib.bib38); Louet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib42); Nieet al\.[2025](https://arxiv.org/html/2606.11552#bib.bib43)\)replace the autoregressive factorization with a denoising formulation: a forward chain progressively masks a clean sequence, and a reverse model trained under bidirectional self\-attention emits tokens by iteratively denoising a fully masked block\. Block diffusion\(Arriolaet al\.[2025](https://arxiv.org/html/2606.11552#bib.bib44)\)interpolates between this regime and ordinary autoregression by partitioning a sequence into contiguous blocks𝐛1,…,𝐛N\\mathbf\{b\}\_\{1\},\\ldots,\\mathbf\{b\}\_\{N\}of sizeKKand factoring

pθ​\(x1:T\)=∏n=1Npθ​\(𝐛n∣𝐛<n\),p\_\{\\theta\}\(x\_\{1:T\}\)\\;=\\;\\prod\_\{n=1\}^\{N\}p\_\{\\theta\}\(\\mathbf\{b\}\_\{n\}\\mid\\mathbf\{b\}\_\{<n\}\),\(5\)where each block\-conditional is parameterised by a discrete diffusion model with bidirectional attention restricted to positions inside𝐛n\\mathbf\{b\}\_\{n\}\.

A*block\-diffusion drafter*instantiates a block\-diffusion model as the speculative drafterqψq\_\{\\psi\}in the framework of \([1](https://arxiv.org/html/2606.11552#S3.E1)\)–\([4](https://arxiv.org/html/2606.11552#S3.E4)\); DFlash\(Chenet al\.[2026](https://arxiv.org/html/2606.11552#bib.bib48)\)is a prominent recent realization of this design and the concrete instance we adopt throughout\. Following the feature\-level drafting principle of EAGLE\(Liet al\.[2024b](https://arxiv.org/html/2606.11552#bib.bib27),[2025b](https://arxiv.org/html/2606.11552#bib.bib29)\), the drafter is conditioned not on discrete tokens but on hidden states extracted from a fixed subset of target layersℒ=\{ℓ1,…,ℓL\}\\mathcal\{L\}=\\\{\\ell\_\{1\},\\ldots,\\ell\_\{L\}\\\}\. Concretely, given a verified prefixx<tx\_\{<t\}already processed bypϕp\_\{\\phi\}, the drafter consumes the multi\-layer feature tensorH<t=\[h<t\(ℓ1\)​‖⋯‖​h<t\(ℓL\)\]H\_\{<t\}=\[h^\{\(\\ell\_\{1\}\)\}\_\{<t\}\\,\\\|\\,\\cdots\\,\\\|\\,h^\{\(\\ell\_\{L\}\)\}\_\{<t\}\]and emits the entireKK\-token candidate block in a single non\-autoregressive forward pass through a small denoising transformerfψf\_\{\\psi\}with full self\-attention over theKKdraft positions:

\(x~t,…,x~t\+K−1\)∼qψ\(⋅∣H<t\)=Cat\(fψ\(H<t\)\)\.\(\\tilde\{x\}\_\{t\},\\ldots,\\tilde\{x\}\_\{t\+K\-1\}\)\\;\\sim\\;q\_\{\\psi\}\(\\cdot\\mid H\_\{<t\}\)\\;=\\;\\mathrm\{Cat\}\(f\_\{\\psi\}\(H\_\{<t\}\)\)\.\(6\)The parallel block is then fed into the target verification step unchanged\. In our experiments we setK=16K=16\.

Table 1:Draft\-token utilisation of a position\-uniform DFlash drafter trained against Llama\-3\-8B\-Instruct with block sizeK=16K=16, evaluated across five benchmarks\. Each block contributes 16 candidate slots: 15 drafted positions plus the target’s bonus correction sampled at the rejection point \(always retained, always matches the target’s greedy output\)\.Acceptis the fraction of the 16 slots whose token is kept by the rejection\-sampling contract \([1](https://arxiv.org/html/2606.11552#S3.E1)\), i\.e\. accepted drafts plus the bonus;Correctis the fraction whose draft \(or bonus\) matches the target’s greedy output;Wasteis the fraction of those correct tokens that the verification contract is nevertheless forced to discard due to an upstream rejection within the same block, i\.e\.Wasted/\(Accepted\+Bonus\+Wasted\)\\mathrm\{Wasted\}/\(\\mathrm\{Accepted\}\+\\mathrm\{Bonus\}\+\\mathrm\{Wasted\}\)\.#### Training objective\.

The drafter is trained by teacher\-forced position\-wise cross\-entropy: for a target block𝐛n=\(xt,…,xt\+K−1\)\\mathbf\{b\}\_\{n\}=\(x\_\{t\},\\ldots,x\_\{t\+K\-1\}\)with corresponding featuresH<tH\_\{<t\},

ℒCE​\(ψ\)=−𝔼​∑k=1K−1log⁡qψ​\(xt\+k∣H<t\),\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\(\\psi\)\\;=\\;\-\\,\\mathbb\{E\}\\\!\\sum\_\{k=1\}^\{K\-1\}\\log q\_\{\\psi\}\(x\_\{t\+k\}\\mid H\_\{<t\}\),\(7\)where each summand is computed under bidirectional self\-attention over theKKdraft positions\. The weighting in \([7](https://arxiv.org/html/2606.11552#S3.E7)\) is*uniform*across positions, whereas the speedup𝒮​\(α,K,c\)\\mathcal\{S\}\(\\alpha,K,c\)depends on the*compounding*of left\-to\-right acceptance probabilities through \([3](https://arxiv.org/html/2606.11552#S3.E3)\)\. This mismatch between the symmetric training\-time objective and the asymmetric verification\-time reward is the central technical problem the remainder of this work addresses\.

IDLRγ\\gammaαf\\alpha\_\{\\mathrm\{f\}\}αc\\alpha\_\{\\mathrm\{c\}\}MT\-Benchτ\\tauGSM8Kτ\\tauHumanEvalτ\\tauAIMEτ\\tauMBPPτ\\tauLiveCodeτ\\tauAvg\.τ\\taua1e\-4None001\.3771\.3201\.3891\.1691\.3271\.3101\.315b1e\-3None001\.9312\.2792\.8032\.7052\.2982\.2412\.376c1e\-2None001\.6701\.9052\.3192\.0311\.9071\.8611\.949d1e\-37002\.0502\.2892\.9493\.0172\.2392\.2322\.463e1e\-310002\.1052\.3823\.1263\.1522\.4122\.3172\.583f1e\-320002\.0172\.2182\.9212\.8122\.3182\.2192\.418g1e\-3100\.302\.2222\.7533\.5213\.4652\.7202\.6682\.892h1e\-3100\.502\.0562\.6503\.5193\.6782\.6372\.5512\.849i1e\-310102\.2042\.6503\.4013\.6482\.6062\.4872\.833k1e\-3100\.352\.2222\.7033\.7243\.8332\.6642\.6912\.973j1e\-3100\.3102\.1852\.7623\.9574\.2782\.9042\.7693\.143l1e\-3100\.3202\.2142\.9403\.9444\.2012\.7532\.8253\.146m1e\-3100\.3302\.3912\.9704\.2234\.7343\.0322\.8433\.365n1e\-3100\.3402\.3413\.0744\.3364\.7573\.0882\.9223\.420o1e\-3100\.3502\.3493\.0444\.1924\.6073\.0232\.9343\.358Table 2:Average acceptance length \(τ\\tau\) across evaluation benchmarks for different training configurations\.γ\\gammais the loss\-decay constant;αf\\alpha\_\{\\mathrm\{f\}\}is the first\-error focal coefficient; andαc\\alpha\_\{\\mathrm\{c\}\}is the chain\-loss coefficient\. The draft horizon is fixed throughout\. Higherτ\\tauis better\. Bold entries indicate the best\-performing configuration within each ablation group\.

## 4The Training–Verification Gap

The DFlash training objective \([7](https://arxiv.org/html/2606.11552#S3.E7)\) and the throughput functional \([3](https://arxiv.org/html/2606.11552#S3.E3)\) measure different things\. The lossℒCE\\mathcal\{L\}\_\{\\mathrm\{CE\}\}is a position\-wise sum of log\-likelihoods under bidirectional self\-attention, weighting allKKslots equally; the gradient at positionkkdepends on neither the value nor the acceptance of any earlier draft token\. The expected accepted length, by contrast, depends on the draft as a whole: positionkkcontributes toτ​\(α,K\)\\tau\(\\alpha,K\)only when every preceding position is accepted under \([1](https://arxiv.org/html/2606.11552#S3.E1)\), and that joint event decays geometrically inkkwhenever per\-position acceptance is below one\. The training loss therefore spends capacity as if theKKpositions contributed independently to throughput, while the verifier credits only the longest causally consistent prefix\. An early\-position error truncates the rest of the block; an error deep in the block typically has no effect at all\.

Figure[2](https://arxiv.org/html/2606.11552#S3.F2)quantifies the resulting waste at the individual\-token level\. For a standard block\-diffusion drafter \(DFlash trained against Llama\-3\-8B\-Instruct and evaluated on HumanEval\), draft tokens that match the target’s greedy output are split by block position into those that verification accepts \(green\) and those discarded because of an upstream rejection in the same block \(red\)\. Per\-position correctness decays only mildly withkk, consistent with the symmetric training signal; per\-position acceptance decays at the geometric rate implied by \([3](https://arxiv.org/html/2606.11552#S3.E3)\), so the red region grows steadily withkkand dominates beyond a short prefix\.

Table[1](https://arxiv.org/html/2606.11552#S3.T1)confirms that the effect persists across workloads: across GSM8K, MT\-Bench, HumanEval, AIME, and MBPP, the drafter’s proposals match the target’s greedy output in22\.1%22\.1\\%of block slots on average, but only11\.6%11\.6\\%of slots survive the rejection\-sampling contract, so an average of46\.9%46\.9\\%of the drafter’s correct predictions are discarded as a consequence of upstream rejections, with per\-benchmark waste rates between42\.9%42\.9\\%and52\.6%52\.6\\%\.

Together, Figure[2](https://arxiv.org/html/2606.11552#S3.F2)and Table[1](https://arxiv.org/html/2606.11552#S3.T1)pinpoint how capacity is misallocated\. The loss \([7](https://arxiv.org/html/2606.11552#S3.E7)\) keeps rewarding likelihood gains at deep positions whose expected contribution to \([3](https://arxiv.org/html/2606.11552#S3.E3)\) is geometrically suppressed, while underweighting gains at shallow positions whose acceptance gates the entire suffix\. The rest of this work studies modifications to the training signal that bring the per\-position weight ofℒCE\\mathcal\{L\}\_\{\\mathrm\{CE\}\}closer to each position’s actual contribution to the verifier’s accepted length\.

## 5Training Techniques to Bridge the Gap

The three interventions studied in this section reshape the drafter’s training loss along orthogonal axes\. Loss decay \(Section[5\.3](https://arxiv.org/html/2606.11552#S5.SS3)\) reweights the per\-position contribution toℒCE\\mathcal\{L\}\_\{\\mathrm\{CE\}\}along the*position*axis as a hand\-specified, ex\-ante function ofkk\. The first\-error focal loss \(Section[5\.4](https://arxiv.org/html/2606.11552#S5.SS4)\) is sparser: it adds an auxiliary cross\-entropy term restricted to the single position per block that the drafter’s argmax decoder currently mispredicts—the*chain breaker*whose rejection truncates the rest of the block—and contributes nothing on blocks the drafter already decodes correctly\. The chain reward \(Section[5\.5](https://arxiv.org/html/2606.11552#S5.SS5)\) is the densest of the three along the*joint\-prefix*axis: it augments the loss with a differentiable surrogate for the expected accepted length, so that the gradient at every position is reweighted online by the drafter’s current estimate of the prefix\-acceptance probability\. The three interventions are mutually composable, and the experiments below layer them in this order\.

The peak learning rate is ablated first to factor optimizer dynamics out of the subsequent loss\-shape comparisons\. Table[2](https://arxiv.org/html/2606.11552#S3.T2)reports the average acceptance lengthτ\\tauon the standard suite for every configuration and is referenced throughout this section; Table[3](https://arxiv.org/html/2606.11552#S5.T3)extends the same configurations across Llama\-3\.2\-3B, Llama\-3\-8B, Qwen\-3\-4B, and Qwen\-3\-8B as a cross\-target generalisation check\.

### 5\.1Experimental Setup

Target\.All ablations in Table[2](https://arxiv.org/html/2606.11552#S3.T2)use Meta\-Llama\-3\-8B\-Instruct as the targetpϕp\_\{\\phi\}:3232transformer blocks, hidden size40964096,3232attention heads with88KV heads \(grouped\-query attention\), intermediate size14,33614\{,\}336, vocabulary size128,256128\{,\}256, RoPE base5×1055\\times 10^\{5\}with the Llama\-3 long\-context scaling\.

Drafter\.The drafterqψq\_\{\\psi\}is a four\-layer DFlash denoiser of the architecture introduced in Section[3](https://arxiv.org/html/2606.11552#S3): four full\-attention transformer blocks, hidden size and head configuration matched to the target \(d=4096d=4096,32:832\{:\}8GQA,dhead=128d\_\{\\mathrm\{head\}\}=128,dff=14,336d\_\{\\mathrm\{ff\}\}=14\{,\}336\), bfloat16, block sizeK=16K=16\. Multi\-layer feature conditioningℒ\\mathcal\{L\}is set to the four evenly\-spaced target layers\{0,10,20,30\}\\\{0,10,20,30\\\}\.

Dataset\.Training data is ShareGPT\(Aealaet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib57)\): a compilation of multi\-turn user–assistant conversations rendered into Llama\-3 chat\-template format and truncated at a sequence length of4,0964\{,\}096tokens\. Within each sequence,512512block anchors are sampled per step to construct\(H<t,𝐛n\)\(H\_\{<t\},\\mathbf\{b\}\_\{n\}\)training pairs as defined in \([6](https://arxiv.org/html/2606.11552#S3.E6)\)\. The corpus is not target\-distilled: prompts and assistant continuations are taken as\-is\.

Optimization\.Drafters are trained for33epochs with AdamW\(Loshchilov and Hutter[2019](https://arxiv.org/html/2606.11552#bib.bib68)\)under a cosine schedule, a warm\-up fraction of1\.5%1\.5\\%, and a per\-device micro batch of11with gradient accumulation of44, yielding an effective batch size of3232across the88NVIDIA H100 80 GB GPUs\(NVIDIA Corporation[2022](https://arxiv.org/html/2606.11552#bib.bib63)\)of a single node \(11\-way tensor parallel, pure data parallel\)\. The peak learning rate is the value reported in each row of Table[2](https://arxiv.org/html/2606.11552#S3.T2); unless mentioned, peak LR is fixed at the value selected in Section[5\.2](https://arxiv.org/html/2606.11552#S5.SS2)\. Training is done using the SpecForge framework\(Liet al\.[2025a](https://arxiv.org/html/2606.11552#bib.bib64)\)\.

Evaluation\.Acceptance lengthτ\\tauis averaged over six benchmarks spanning open\-ended dialogue \(MT\-Bench;Zhenget al\.[2023](https://arxiv.org/html/2606.11552#bib.bib58)\), mathematical reasoning \(GSM8K,Cobbeet al\.[2021](https://arxiv.org/html/2606.11552#bib.bib59); AIME, drawn from the AI\-MO validation set of9090problems from AIME 2022–2024,Project Numina \(AI\-MO\)[2024](https://arxiv.org/html/2606.11552#bib.bib62)\), and code generation \(HumanEval,Chenet al\.[2021](https://arxiv.org/html/2606.11552#bib.bib1); MBPP,Austinet al\.[2021b](https://arxiv.org/html/2606.11552#bib.bib60); LiveCodeBench,Jainet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib61)\), reported per\-benchmark and as the meanτ¯\\bar\{\\tau\}in the rightmost column of Table[2](https://arxiv.org/html/2606.11552#S3.T2)\.

### 5\.2Impact of Learning Rate

The peak learning rate of the AdamW optimizer is ablated first with all other hyperparameters held fixed at the configuration of Section[5\.1](https://arxiv.org/html/2606.11552#S5.SS1)\. Rows a–c of Table[2](https://arxiv.org/html/2606.11552#S3.T2)sweep the peak learning rate over\{10−4,10−3,10−2\}\\\{10^\{\-4\},10^\{\-3\},10^\{\-2\}\\\}at the position\-uniform baseline \(γ=None\\gamma=\\texttt\{None\},αc=αf=0\\alpha\_\{\\mathrm\{c\}\}=\\alpha\_\{\\mathrm\{f\}\}=0\)\. The optimum sits at10−310^\{\-3\}\(row b\), which yieldsτ¯=2\.376\\bar\{\\tau\}=2\.376against1\.3151\.315at10−410^\{\-4\}\(row a\) and1\.9491\.949at10−210^\{\-2\}\(row c\)—i\.e\. both an order of magnitude lower and an order of magnitude higher cost the drafter45%45\\%and18%18\\%of average acceptance length, respectively\. All subsequent experiments fix the peak learning rate at10−310^\{\-3\}\.

### 5\.3Position\-Wise Reweighting: Loss Decay

![Refer to caption](https://arxiv.org/html/2606.11552v1/x3.png)Figure 3:Per\-position loss weightwk​\(γ\)w\_\{k\}\(\\gamma\)of \([8](https://arxiv.org/html/2606.11552#S5.E8)\) as a function of the within\-block token offsetk∈\{0,…,K−1\}k\\in\\\{0,\\ldots,K\-1\\\}, plotted for several decay constantsγ\\gammaat block sizeK=16K=16\.The simplest reconciliation between the position\-uniform weight implicit in \([7](https://arxiv.org/html/2606.11552#S3.E7)\) and the geometrically decaying inference value of each position, established in Section[4](https://arxiv.org/html/2606.11552#S4), is to promote the per\-position weight to an explicit hyperparameter\. This parameterisation was introduced in the original DFlash work\(Chenet al\.[2026](https://arxiv.org/html/2606.11552#bib.bib48)\); we adopt it here and study it more systematically as one of three composable interventions\. The per\-position weight at offsetk∈\{1,…,K−1\}k\\in\\\{1,\\ldots,K\-1\\\}\(the anchork=0k=0is excluded from the loss, as it is in \([7](https://arxiv.org/html/2606.11552#S3.E7)\)\) is set to

wk​\(γ\)=exp⁡\(−k−1γ\),w\_\{k\}\(\\gamma\)\\;=\\;\\exp\\\!\\left\(\-\\,\\frac\{k\-1\}\{\\gamma\}\\right\),\(8\)parameterised by a decay constantγ\>0\\gamma\>0\. The reweighted objective then reads

ℒCE\(γ\)​\(ψ\)=−𝔼​∑k=1K−1wk​\(γ\)​log⁡qψ​\(xt\+k∣H<t\)\.\\mathcal\{L\}\_\{\\mathrm\{CE\}\}^\{\(\\gamma\)\}\(\\psi\)\\;=\\;\-\\,\\mathbb\{E\}\\\!\\sum\_\{k=1\}^\{K\-1\}w\_\{k\}\(\\gamma\)\\,\\log q\_\{\\psi\}\(x\_\{t\+k\}\\mid H\_\{<t\}\)\.\(9\)The limitγ→∞\\gamma\\to\\inftyrecovers the position\-uniform objective of \([7](https://arxiv.org/html/2606.11552#S3.E7)\); the limitγ→0\+\\gamma\\to 0^\{\+\}collapses to a single\-position objective atk=1k=1\. Intermediate values induce an exponential profile that, by construction, allocates more gradient signal to the early positions whose acceptance gates the entire suffix and less to the deep positions whose contribution to \([3](https://arxiv.org/html/2606.11552#S3.E3)\) is geometrically suppressed\.

This is the cheapest intervention available: it adds no forward pass through either the drafter or the target, no architectural change, and a single scalar hyperparameter\. Its principal limitation is that the weighting profile \([8](https://arxiv.org/html/2606.11552#S5.E8)\) is fixed and decoupled from the drafter’s per\-position acceptance behavior at any point in training; every block is downweighted identically, irrespective of its difficulty or of which positions the current drafter is already capable of resolving\. Sections[5\.4](https://arxiv.org/html/2606.11552#S5.SS4)and[5\.5](https://arxiv.org/html/2606.11552#S5.SS5)address this limitation in complementary ways: the first\-error focal loss conditions on the chain\-breaker position selected by the current drafter, and the chain reward couples the per\-position weight to the drafter’s own prefix\-acceptance probability\.

Rows d–f of Table[2](https://arxiv.org/html/2606.11552#S3.T2)sweepγ∈\{7,10,20\}\\gamma\\in\\\{7,10,20\\\}at the tuned learning rate of Section[5\.2](https://arxiv.org/html/2606.11552#S5.SS2); the position\-uniform baseline \(row b\) recordsτ¯=2\.376\\bar\{\\tau\}=2\.376as the reference point\. Performance peaks at the intermediate setting:γ=7\\gamma=7yieldsτ¯=2\.463\\bar\{\\tau\}=2\.463\(\+3\.7%\+3\.7\\%\) andγ=20\\gamma=20yieldsτ¯=2\.418\\bar\{\\tau\}=2\.418\(\+1\.8%\+1\.8\\%\), whileγ=10\\gamma=10\(row e\) reachesτ¯=2\.583\\bar\{\\tau\}=2\.583, a\+8\.7%\+8\.7\\%gain over the position\-uniform baseline\. This matches the reasoning of Section[4](https://arxiv.org/html/2606.11552#S4): a sharp profile \(γ=7\\gamma=7\) cuts off the deep\-position signal too aggressively, a slack profile \(γ=20\\gamma=20\) is close to the uniform baseline, and the intermediate setting best tracks the geometrically decaying inference value of each position implied by \([3](https://arxiv.org/html/2606.11552#S3.E3)\)\. We fixγ=10\\gamma=10for subsequent ablations\.

Table 3:Acceptance length \(τ\\tau\) across benchmarks for different target\-model families and training configurations\.γ\\gammais the loss\-decay constant \(Nonedenotes the position\-uniform objective\),αf\\alpha\_\{\\mathrm\{f\}\}is the first\-error focal coefficient, andαc\\alpha\_\{\\mathrm\{c\}\}is the chain\-loss coefficient\. Higherτ\\tauis better\.
### 5\.4Targeting the Chain Breaker: Focal Loss

The decay weighting of Section[5\.3](https://arxiv.org/html/2606.11552#S5.SS3)reshapes the gradient with a position\-only schedule applied uniformly to every block\. The analysis of Section[4](https://arxiv.org/html/2606.11552#S4)suggests a more targeted intervention\. Under the rejection\-sampling contract \([1](https://arxiv.org/html/2606.11552#S3.E1)\), a block ofKKdraft tokens contributes to the accepted length up to and including the first position at which the draft is rejected; the identity of that single position—the*chain breaker*—is sufficient to determine the block’s acceptance outcome under the argmax simplification\. From the perspective of the throughput functional \([3](https://arxiv.org/html/2606.11552#S3.E3)\), fixing the chain breaker of a block whose prefix is otherwise correct extends its accepted length by exactly one, whereas additional gradient on positions deeper than the breaker yields no improvement until the breaker itself is fixed\.

The first\-error focal loss instantiates this intuition as an auxiliary cross\-entropy term restricted to the chain\-breaker position of each block\. Concretely, for each blocknnwith draft logitsℓn,k\\ell\_\{n,k\}and ground\-truth tokensxn,k∗x^\{\*\}\_\{n,k\}, denote the argmax decoder predictions byx^n,k=arg⁡maxv⁡ℓn,kv\\hat\{x\}\_\{n,k\}=\\arg\\max\_\{v\}\\ell\_\{n,k\}^\{v\}and the set of positions where the argmax disagrees with the gold byEn=\{k∈\{1,…,K−1\}:x^n,k≠xn,k∗\}E\_\{n\}=\\\{k\\in\\\{1,\\ldots,K\-1\\\}:\\hat\{x\}\_\{n,k\}\\neq x^\{\*\}\_\{n,k\}\\\}\. Blocks withEn=∅E\_\{n\}=\\emptysetcontribute nothing to the auxiliary term\. For blocks with at least one disagreement, letkn∗=min⁡Enk\_\{n\}^\{\*\}=\\min E\_\{n\}be the chain\-breaker offset and define

ℒfe​\(ψ\)=∑n:En≠∅wkn∗​\(γ\)​\(−log⁡qψ​\(xn,kn∗∗∣H<tn\)\)∑n:En≠∅wkn∗​\(γ\)\+ε,\\mathcal\{L\}\_\{\\mathrm\{fe\}\}\(\\psi\)\\;=\\;\\frac\{\\sum\_\{n:\\,E\_\{n\}\\neq\\emptyset\}w\_\{k\_\{n\}^\{\*\}\}\(\\gamma\)\\,\\bigl\(\-\\log q\_\{\\psi\}\(x^\{\*\}\_\{n,k\_\{n\}^\{\*\}\}\\mid H\_\{<t\_\{n\}\}\)\\bigr\)\}\{\\sum\_\{n:\\,E\_\{n\}\\neq\\emptyset\}w\_\{k\_\{n\}^\{\*\}\}\(\\gamma\)\\;\+\\;\\varepsilon\},\(10\)the mean per\-position cross entropy at the chain breaker, weighted by the same position decaywk​\(γ\)w\_\{k\}\(\\gamma\)used in \([9](https://arxiv.org/html/2606.11552#S5.E9)\)\. Withαf≥0\\alpha\_\{\\mathrm\{f\}\}\\geq 0the focal coefficient, the decay\-plus\-focal objective reads

ℒ\(γ,αf\)​\(ψ\)=ℒCE\(γ\)​\(ψ\)\+αf​ℒfe​\(ψ\)\.\\mathcal\{L\}^\{\(\\gamma,\\alpha\_\{\\mathrm\{f\}\}\)\}\(\\psi\)\\;=\\;\\mathcal\{L\}\_\{\\mathrm\{CE\}\}^\{\(\\gamma\)\}\(\\psi\)\\;\+\\;\\alpha\_\{\\mathrm\{f\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{fe\}\}\(\\psi\)\.\(11\)
The construction is qualitatively distinct from the decay weighting\. Loss decay applies at every position of every block, redistributing gradient by a smooth function ofkk\. The first\-error focal term, by contrast, is*block\-conditional*\(it activates only when the block is breakable\) and*position\-sparse*\(it activates at exactly one offset per such block, selected adaptively by the drafter’s current decoder\)\. Two further consequences follow\. First, blocks that the drafter already decodes correctly receive no extra signal, which prevents the auxiliary term from over\-fitting easy patterns\. Second, as the drafter improves, the chain\-breaker distribution shifts deeper into the block, so the auxiliary term automatically retargets toward later positions without an explicit schedule\.

Rows g–i of Table[2](https://arxiv.org/html/2606.11552#S3.T2)sweepαf∈\{0\.3,0\.5,1\.0\}\\alpha\_\{\\mathrm\{f\}\}\\in\\\{0\.3,\\,0\.5,\\,1\.0\\\}on top of the tunedγ=10\\gamma=10of Section[5\.3](https://arxiv.org/html/2606.11552#S5.SS3)\(row e,τ¯=2\.583\\bar\{\\tau\}=2\.583\)\. All three settings improve over the decay\-only baseline, and the objective is monotone\-decreasing inαf\\alpha\_\{\\mathrm\{f\}\}over this range:αf=0\.3\\alpha\_\{\\mathrm\{f\}\}=0\.3\(row g\) attainsτ¯=2\.892\\bar\{\\tau\}=2\.892, a\+12\.0%\+12\.0\\%gain over the focal\-free decay baseline and a cumulative\+21\.7%\+21\.7\\%over the position\-uniform row b;αf=0\.5\\alpha\_\{\\mathrm\{f\}\}=0\.5\(row h\) andαf=1\.0\\alpha\_\{\\mathrm\{f\}\}=1\.0\(row i\) regress slightly toτ¯=2\.849\\bar\{\\tau\}=2\.849andτ¯=2\.833\\bar\{\\tau\}=2\.833respectively\. The improvement is largest on the reasoning\- and code\-heavy benchmarks—HumanEval \(2\.803→3\.5212\.803\\to 3\.521,\+25\.6%\+25\.6\\%\) and AIME \(2\.705→3\.4652\.705\\to 3\.465,\+28\.1%\+28\.1\\%\) relative to row b\. We adoptαf=0\.3\\alpha\_\{\\mathrm\{f\}\}=0\.3as the reference setting for the chain sweep that follows\.

### 5\.5Joint\-Prefix Reweighting: Chain Reward

The decay weighting of Section[5\.3](https://arxiv.org/html/2606.11552#S5.SS3)is a position\-only schedule fixed ex ante; the first\-error focal loss of Section[5\.4](https://arxiv.org/html/2606.11552#S5.SS4)is sparse and block\-conditional\. A third axis of intervention, complementary to both, is to substitute a differentiable surrogate for the expected accepted length itself, so that the gradient at every position is reweighted online by the prefix\-acceptance probability under the current drafter\. Letpk=qψ​\(xt\+k∣H<t\)p\_\{k\}=q\_\{\\psi\}\(x\_\{t\+k\}\\mid H\_\{<t\}\)denote the drafter’s marginal probability on the ground\-truth token at offsetkk, and writeρk=∏j=1kpj\\rho\_\{k\}=\\prod\_\{j=1\}^\{k\}p\_\{j\}for the corresponding prefix probability\.

Treatingρk\\rho\_\{k\}as a proxy for the probability that positions1,…,k1,\\ldots,kof the draft are jointly accepted under \([1](https://arxiv.org/html/2606.11552#S3.E1)\), the expected accepted length is approximated by

Rchain​\(ψ\)=1K−1​∑k=1K−1exp⁡\(∑j=1klog⁡pj\),R\_\{\\mathrm\{chain\}\}\(\\psi\)\\;=\\;\\frac\{1\}\{K\-1\}\\sum\_\{k=1\}^\{K\-1\}\\exp\\\!\\left\(\\sum\_\{j=1\}^\{k\}\\log p\_\{j\}\\right\),\(12\)normalised to\[0,1\]\[0,1\]\. The full training objective combining the three interventions of Sections[5\.3](https://arxiv.org/html/2606.11552#S5.SS3)–[5\.5](https://arxiv.org/html/2606.11552#S5.SS5)is then

ℒ​\(ψ\)=ℒCE\(γ\)​\(ψ\)\+αf​ℒfe​\(ψ\)−αc​Rchain​\(ψ\),\\mathcal\{L\}\(\\psi\)\\;=\\;\\mathcal\{L\}\_\{\\mathrm\{CE\}\}^\{\(\\gamma\)\}\(\\psi\)\\;\+\\;\\alpha\_\{\\mathrm\{f\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{fe\}\}\(\\psi\)\\;\-\\;\\alpha\_\{\\mathrm\{c\}\}\\,R\_\{\\mathrm\{chain\}\}\(\\psi\),\(13\)withαc≥0\\alpha\_\{\\mathrm\{c\}\}\\geq 0controlling the trade\-off between marginal correctness and joint chain probability\. Differentiating \([12](https://arxiv.org/html/2606.11552#S5.E12)\) with respect tolog⁡pk\\log p\_\{k\}yields\(K−1\)−1​∑j≥kρj\(K\-1\)^\{\-1\}\\sum\_\{j\\geq k\}\\rho\_\{j\}: each position is reinforced in proportion to the cumulative prefix probability of its own and every deeper position, so that early positions receive a strictly larger gradient than deep ones, with the relative weighting determined by the drafter’s current calibration rather than by a hand\-specified schedule\.

The implementation reuses the per\-token log\-probabilities already computed forℒCE\\mathcal\{L\}\_\{\\mathrm\{CE\}\}, since−CE​\(logits,x∗\)=log⁡p\-\\mathrm\{CE\}\(\\mathrm\{logits\},x^\{\*\}\)=\\log p; the chain reward therefore adds only a cumulative sum and exponentiation per block and does not require a second pass through the language modelling head\.

Rows k–o of Table[2](https://arxiv.org/html/2606.11552#S3.T2)sweepαc∈\{5,10,20,30,40,50\}\\alpha\_\{\\mathrm\{c\}\}\\in\\\{5,10,20,30,40,50\\\}on top of the\(γ,αf\)=\(10,0\.3\)\(\\gamma,\\alpha\_\{\\mathrm\{f\}\}\)=\(10,0\.3\)stack of Section[5\.4](https://arxiv.org/html/2606.11552#S5.SS4)\(row g,τ¯=2\.892\\bar\{\\tau\}=2\.892\)\. Even modest coefficients are markedly accretive:αc=5\\alpha\_\{\\mathrm\{c\}\}=5\(row k\) raisesτ¯\\bar\{\\tau\}to2\.9732\.973;αc=10\\alpha\_\{\\mathrm\{c\}\}=10\(row j\) to3\.1433\.143;αc=20\\alpha\_\{\\mathrm\{c\}\}=20\(row l\) to3\.1463\.146;αc=30\\alpha\_\{\\mathrm\{c\}\}=30\(row m\) to3\.3653\.365; andαc=40\\alpha\_\{\\mathrm\{c\}\}=40\(row n\) peaks atτ¯=3\.420\\bar\{\\tau\}=3\.420, a\+18\.3%\+18\.3\\%gain over the chain\-free row g and a cumulative\+43\.9%\+43\.9\\%over the position\-uniform baseline of row b\. The improvement is largest on the reasoning\- and code\-heavy benchmarks—HumanEval \(2\.803→4\.3362\.803\\to 4\.336,\+54\.7%\+54\.7\\%\) and AIME \(2\.705→4\.7572\.705\\to 4\.757,\+75\.9%\+75\.9\\%\) relative to row b—where long, well\-determined draft chains are most plentiful and most rewarded by the joint\-prefix reweighting\. The objective turns over past the peak: atαc=50\\alpha\_\{\\mathrm\{c\}\}=50\(row o\)τ¯\\bar\{\\tau\}regresses to3\.3583\.358, as the chain term begins to dominate the per\-position cross entropy\. We adoptαc=40\\alpha\_\{\\mathrm\{c\}\}=40as the reference setting for the cross\-target generalisation experiments that follow\.

Table 4:Each training intervention applied in isolation to the position\-uniform baseline at the tuned learning rate of Section[5\.2](https://arxiv.org/html/2606.11552#S5.SS2)\. Row a reproduces row b of Table[2](https://arxiv.org/html/2606.11552#S3.T2); row b reproduces row e; rows c and d are single\-technique runs with the other two coefficients held at zero\. Higherτ\\tauis better\.
### 5\.6Experiments on Other Models

Table[3](https://arxiv.org/html/2606.11552#S5.T3)replicates the four reference configurations of Section[5\.1](https://arxiv.org/html/2606.11552#S5.SS1)—position\-uniform baseline \(γ=None\\gamma=\\texttt\{None\},αf=αc=0\\alpha\_\{\\mathrm\{f\}\}=\\alpha\_\{\\mathrm\{c\}\}=0\),\+γ\+\\gamma,\+αf\+\\alpha\_\{\\mathrm\{f\}\},\+αc\+\\alpha\_\{\\mathrm\{c\}\}—across four instruction\-tuned target models: Llama\-3\.2\-3B\-Instruct\(Meta AI[2024](https://arxiv.org/html/2606.11552#bib.bib46)\), Meta\-Llama\-3\-8B\-Instruct\(Grattafioriet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib45)\), and the instruction\-tuned Qwen3\-4B and Qwen3\-8B releases\(Yanget al\.[2025](https://arxiv.org/html/2606.11552#bib.bib47)\)—to confirm that the ranking is target\-agnostic\. All four are the instruction\-tuned variants of their respective families; no separate post\-training is performed on the target\. The compounding pattern of Table[2](https://arxiv.org/html/2606.11552#S3.T2)reproduces on every target: each successive intervention strictly improves average acceptance length, and the fully stacked\(γ,αf,αc\)=\(10,0\.3,40\)\(\\gamma,\\alpha\_\{\\mathrm\{f\}\},\\alpha\_\{\\mathrm\{c\}\}\)=\(10,0\.3,40\)configuration delivers the largestτ¯\\bar\{\\tau\}for all four targets, with cumulative gains over the position\-uniform baseline of\+27\.5%\+27\.5\\%\(Llama\-3\.2\-3B\),\+43\.9%\+43\.9\\%\(Llama\-3\-8B\),\+20\.9%\+20\.9\\%\(Qwen\-3\-4B\), and\+24\.7%\+24\.7\\%\(Qwen\-3\-8B\)\. The absolute gains are largest on the reasoning\- and code\-heavy benchmarks \(HumanEval, AIME\) across every target, mirroring the per\-benchmark profile of Section[5\.5](https://arxiv.org/html/2606.11552#S5.SS5)and indicating that the joint\-prefix reweighting transfers target\-independently to the workloads on which long, well\-determined draft chains are most plentiful\.

## 6Ablations

### 6\.1Each Technique in Isolation

The sweeps of Section[5](https://arxiv.org/html/2606.11552#S5)layer the three interventions cumulatively: loss decay is tuned alone, the focal term is added on top of the chosenγ\\gamma, and the chain reward is added on top of the decay–plus–focal stack\. This construction is the one that yields the strongest configuration of Table[2](https://arxiv.org/html/2606.11552#S3.T2), but it leaves open the question of how much each intervention contributes*in isolation*—i\.e\., applied directly to the position\-uniform baseline of \([7](https://arxiv.org/html/2606.11552#S3.E7)\) with the other two coefficients held at zero\. We isolate each technique by holding the learning rate fixed at the value selected in Section[5\.2](https://arxiv.org/html/2606.11552#S5.SS2)and training a separate drafter for each of the four single\-technique configurations\.

Table 5:Average acceptance length \(τ\\tau\) for the four reference training configurations on the target\-aligned Nemotron\-V2 \+ CodeAlpaca split\. Higher is better; the best row is bolded\. All runs use the same drafter architecture, optimiser settings, and evaluation suite as Table[2](https://arxiv.org/html/2606.11552#S3.T2)\.Three observations follow from Table[4](https://arxiv.org/html/2606.11552#S5.T4)\. First, every intervention is individually accretive over the position\-uniform baseline, and the relative ranking \(chain\>\>focal\>\>decay\) is consistent with the analysis of Section[4](https://arxiv.org/html/2606.11552#S4): the more directly an intervention couples its gradient to the verification rule, the larger its isolated effect\. Loss decay reweights by a fixed ex\-ante schedule that only approximately tracks the geometric decay of acceptance value; the focal term conditions on the drafter’s own argmax breaker; the chain reward integrates the joint prefix probability across all positions\. Second, the chain reward alone already recovers a large fraction of the fully\-stacked gain \(row n of Table[2](https://arxiv.org/html/2606.11552#S3.T2),τ¯=3\.420\\bar\{\\tau\}=3\.420\), suggesting that the joint\-prefix axis is the dominant source of the improvement\. Third, the gap between the chain\-only configuration and the fully\-stacked one nevertheless remains substantial, suggesting that the three interventions do not encode the same training signal\.

### 6\.2Impact of Training Data

All experiments of Section[5](https://arxiv.org/html/2606.11552#S5)use the ShareGPT dataset\(Aealaet al\.[2023](https://arxiv.org/html/2606.11552#bib.bib57)\)as the drafter training corpus, taken as\-authored rather than re\-distilled from the target\. To characterize how data composition interacts with the loss\-shape interventions of Section[5](https://arxiv.org/html/2606.11552#S5), we re\-run the four reference configurations on a target\-aligned corpus\. To provide a diverse mixture of instruction\-following and code prompts, following\(Chenet al\.[2026](https://arxiv.org/html/2606.11552#bib.bib48)\), we collect approximately800800K samples from the NVIDIA Nemotron Post\-Training Dataset V2\(Nathawani and NVIDIA[2025](https://arxiv.org/html/2606.11552#bib.bib54)\)and CodeAlpaca\(Chaudhary[2023](https://arxiv.org/html/2606.11552#bib.bib55)\)\. Rather than training against the as\-authored assistant responses, we regenerate each completion under the target modelpϕp\_\{\\phi\}\(Meta\-Llama\-3\-8B\-Instruct,Grattafioriet al\.[2024](https://arxiv.org/html/2606.11552#bib.bib45)\) served via sglang at temperature0\.70\.7,max\_tokens=2048=2048, and bfloat16, yielding a target\-aligned training split of the same prompt set\. The four reference configurations of Section[5](https://arxiv.org/html/2606.11552#S5)are then retrained on this split for six epochs at the tuned learning rate and effective batch size of Section[5\.1](https://arxiv.org/html/2606.11552#S5.SS1); metrics are reported at the75,00075\{,\}000\-step checkpoint\.

Two observations follow from Table[5](https://arxiv.org/html/2606.11552#S6.T5)\. First, the relative ordering and the cumulative compounding pattern of Section[5](https://arxiv.org/html/2606.11552#S5)both transfer to the target\-aligned data: each successive intervention adds toτ¯\\bar\{\\tau\}\(decay\+8\.1%\+8\.1\\%, focal a further\+9\.5%\+9\.5\\%, chain a further\+7\.6%\+7\.6\\%, for a cumulative\+27\.4%\+27\.4\\%over the position\-uniform baseline\), and the fully stacked configuration remains strictly best\. Second, the change of training corpus is itself a large independent lever: the position\-uniform baseline rises fromτ¯=2\.376\\bar\{\\tau\}=2\.376on ShareGPT \(row b of Table[2](https://arxiv.org/html/2606.11552#S3.T2)\) toτ¯=3\.914\\bar\{\\tau\}=3\.914on the target\-aligned split, a\+64\.7%\+64\.7\\%absolute lift before any of the loss\-shape interventions are applied; composed with them, the fully stacked configuration reachesτ¯=4\.985\\bar\{\\tau\}=4\.985, a\+109\.8%\+109\.8\\%improvement over the ShareGPT position\-uniform baseline\. The training\-time interventions of Section[5](https://arxiv.org/html/2606.11552#S5)therefore stack multiplicatively with target\-aligned training data rather than substituting for it\.

Table 6:Acceptance length \(τ\\tau\) as a function of the speculation horizonKKfor the position\-uniform baseline \(Base\) and the fully\-aligned\(γ,αf,αc\)=\(10,0\.3,40\)\(\\gamma,\\alpha\_\{\\mathrm\{f\}\},\\alpha\_\{\\mathrm\{c\}\}\)=\(10,0\.3,40\)drafter \(Aligned\)\. Drafters are trained atK=16K=16and evaluated at the indicated horizon\. Higher is better; the best row per column is bolded\.
### 6\.3Impact of Block Size for Evaluation

The draft horizonKKcontrols both the per\-iteration speedup ceiling \([4](https://arxiv.org/html/2606.11552#S3.E4)\) and the geometric decay of the per\-position acceptance profile\{αk\}\\\{\\alpha\_\{k\}\\\}\. To characterise this trade\-off, we evaluate the position\-uniform baseline \(row b of Table[2](https://arxiv.org/html/2606.11552#S3.T2)\) and the fully\-aligned configuration \(row n\) at speculation horizonsK∈\{2,4,8,16,32\}K\\in\\\{2,4,8,16,32\\\}, holding training at theK=16K=16block size of Section[5\.1](https://arxiv.org/html/2606.11552#S5.SS1)fixed\. Table[6](https://arxiv.org/html/2606.11552#S6.T6)reports the average acceptance length on AIME, HumanEval, and MBPP\.

Two patterns emerge\. First,τ¯\\bar\{\\tau\}peaks at the training block size of1616for both drafters and falls off once the inference horizon exceeds it, withK=32K=32regressing on every benchmark\. The drafter was trained to predict1616\-token blocks under bidirectional attention; pushing the inference horizon past that size asks the model to predict positions it never saw during training, and the resulting acceptance loss outweighs the extra speedup ceiling\. Second, the aligned drafter’s gain over the baseline grows steadily withKKbelow the training horizon: atK=2K=2the aligned drafter improves AIMEτ¯\\bar\{\\tau\}by\+8\.8%\+8\.8\\%, atK=4K=4by\+31\.5%\+31\.5\\%, atK=8K=8by\+62\.4%\+62\.4\\%, and atK=16K=16by\+72\.0%\+72\.0\\%\(with comparable trajectories on HumanEval and MBPP\)\. The position\-uniform baseline stops improving well beforeK=16K=16because the deep\-position acceptance rate is already near zero; the aligned drafter, whose training signal explicitly targets the early\-position acceptance gates and the joint prefix probability, keeps gaining speedup all the way up to the training block size\.

Table 7:Average throughput \(tokens per second, TPS\) and average accepted\-token length \(τ\\tau\) under progressive training enhancements, comparing standard speculative decoding \(no ddTree, contiguous block acceptance with block size 16\) and ddTree verification\. Results are averaged across six benchmarks \(GSM8K, HumanEval, AIME25, MBPP, MT\-Bench, and LiveCodeBench\)\. Higher is better\.
### 6\.4Application to Inference\-time Strategies

The training\-time interventions of Section[5](https://arxiv.org/html/2606.11552#S5)leave the inference pipeline at the standard rejection\-sampling contract: each round verifies a single drafted trajectory ofKKtokens\. A complementary test\-time mechanism is to exploit the fact that a single block\-diffusion drafter pass already produces a per\-position distributionqi\(⋅∣H<t\)q\_\{i\}\(\\cdot\\mid H\_\{<t\}\)at every offset, rather than only the argmax sample\. DDTree\(Ringel and Romano[2026](https://arxiv.org/html/2606.11552#bib.bib50)\)converts these marginals into a draft tree of at mostBBnodes whose prefix probabilities under the factorised distributionQ=∏iqiQ=\\prod\_\{i\}q\_\{i\}are recovered by a best\-first heap walk, and verifies the resulting tree in one target\-model forward pass with an ancestor\-only attention mask\. DDTree is wholly orthogonal to the training\-time interventions studied here: it changes neither the drafter objective nor the rejection\-sampling exactness contract, only the set of continuations the verifier scores per round\.

Table[7](https://arxiv.org/html/2606.11552#S6.T7)stacks DDTree verification on top of the four reference configurations of Section[5](https://arxiv.org/html/2606.11552#S5)and reports both throughput and acceptance length averaged across the six benchmarks\. The two axes compose cleanly\. Holding the drafter at the position\-uniform baseline, DDTree alone raisesτ¯\\bar\{\\tau\}from2\.3762\.376to3\.7913\.791\(\+59\.6%\+59\.6\\%\) and average TPS from126\.75126\.75to188\.68188\.68\(\+48\.9%\+48\.9\\%\)\. Holding inference at standard contiguous\-block verification, the layered training\-time stack raisesτ¯\\bar\{\\tau\}from2\.3762\.376to3\.4203\.420\(\+44\.0%\+44\.0\\%\) and TPS from126\.75126\.75to225\.12225\.12\(\+77\.6%\+77\.6\\%\)\. Applied jointly, the fully stacked configuration with DDTree verification reachesτ¯=4\.609\\bar\{\\tau\}=4\.609and294\.68294\.68TPS, a\+94\.0%\+94\.0\\%and\+132\.5%\+132\.5\\%lift over the position\-uniform, no\-DDTree baseline\. Both directions of the cross\-product are monotone: every training\-time increment retains its gain under DDTree, and DDTree’s lift is preserved at every point along the training\-time stack\. The two surfaces therefore add rather than substitute, consistent with the observation that DDTree reshapes the verifier’s search over fixed drafter marginals while the training\-time interventions reshape the marginals themselves\.

### 6\.5Compatibility with SpecDiff\-2

Table 8:Average accepted\-token length \(τ\\tau\) on LiveCodeBench, AIME25, and HumanEval for SpecDiff2\-trained draft models under progressive training enhancements\. Results are reported using the final checkpoint of each training run\. Higher is better\.The contemporaneous SpecDiff\-2 ofSandleret al\.\([2025](https://arxiv.org/html/2606.11552#bib.bib51)\)targets the same training–verification mismatch we analyse through a different mechanism\. Its train\-time component,*streak\-distillation*, replaces the position\-uniform cross\-entropy with a pathwise surrogate for the expected accepted streak length: at each prefixss, a continuationx1:γx\_\{1:\\gamma\}is sampled from the frozen verifierpϕp\_\{\\phi\}, and the drafter’s per\-position marginals are scored as∑m=1γ∏j=1mqj​\(xj∣s\)\\sum\_\{m=1\}^\{\\gamma\}\\prod\_\{j=1\}^\{m\}q\_\{j\}\(x\_\{j\}\\mid s\), which is then maximised through the drafter parameters\. The outer expectation is taken over teacher prefixes drawn frompϕp\_\{\\phi\}, so each gradient step requires a fresh sample of lengthγ\\gammafrom the verifier in addition to the standard drafter pass\. The dominant incremental cost of streak\-distillation, then, is not the surrogate itself but the verifier rollout that supplies the teacher continuation: training compute scales linearly with the rollout length and, in the published implementation, requires a verifier forward pass per training example above the drafter forward pass already needed for the base loss\.

Our three training\-time interventions are compatible with streak\-distillation along this axis\. The chain reward of Section[5\.5](https://arxiv.org/html/2606.11552#S5.SS5)is also a differentiable surrogate for𝔼​\[accepted length\]\\mathbb\{E\}\[\\text\{accepted length\}\], but it is evaluated along the teacher\-forced ground\-truth tokens already materialised byℒCE\\mathcal\{L\}\_\{\\mathrm\{CE\}\}, so adding it costs only a cumulative sum and exponentiation per block\. The decay weighting of Section[5\.3](https://arxiv.org/html/2606.11552#S5.SS3)and the focal term of Section[5\.4](https://arxiv.org/html/2606.11552#S5.SS4)reuse the same per\-token cross\-entropy tensor for the same reason\. Layering the three interventions on top of streak\-distillation therefore inherits the latter’s rollout cost but adds nothing further, while reshaping the objective along the position and block\-conditional first\-error axes that streak\-distillation does not address\. Table[8](https://arxiv.org/html/2606.11552#S6.T8)reports the result of this layering on LiveCodeBench, AIME25, and HumanEval, using SpecDiff\-2\-trained drafters as the base in every row\. Streak\-distillation alone \(the position\-uniform row of the table\) records an averageτ¯=2\.339\\bar\{\\tau\}=2\.339on this three\-benchmark slice; addingγ=10\\gamma=10on top raisesτ¯\\bar\{\\tau\}to2\.7912\.791, a\+19\.3%\+19\.3\\%gain over the streak\-distilled baseline; theαf=0\.3\\alpha\_\{\\mathrm\{f\}\}=0\.3focal term adds a further\+14\.5%\+14\.5\\%forτ¯=3\.196\\bar\{\\tau\}=3\.196\(\+36\.6%\+36\.6\\%cumulative\); and theαc=40\\alpha\_\{\\mathrm\{c\}\}=40chain reward adds another\+27\.4%\+27\.4\\%for a fully stackedτ¯=4\.071\\bar\{\\tau\}=4\.071, a\+74\.0%\+74\.0\\%improvement over the streak\-distilled baseline\.

## 7Conclusion

We studied the training of block\-diffusion drafters for speculative decoding through the lens of the gap between the position\-uniform, bidirectional objective inherited from diffusion language modelling and the strictly causal, prefix\-truncating acceptance contract that governs verification\. A consequence of this gap, visible in the position\-uniform DFlash baseline, is that a large majority of the tokens the target ratifies as correct are discarded by the rejection\-sampling rule simply because they sit downstream of an earlier within\-block rejection—capacity that the standard cross\-entropy loss has no mechanism to reclaim\.

We analysed three complementary training\-time interventions that narrow this gap—position\-wise loss decay, a first\-error focal loss targeting the chain\-breaking position of each block, and a chain reward that substitutes a differentiable surrogate for the expected accepted length\. The three reshape the loss along orthogonal axes \(position, block\-conditional first error, joint prefix\), compose additively, preserve the exactness contract of speculative decoding, and add negligible compute over the base objective\. They are individually accretive and jointly compounding across the model families and benchmark categories we evaluated, and remain orthogonal to test\-time alignment mechanisms developed for the same drafter class; combining them with such mechanisms is a natural direction for follow\-up work\.

## References

- Aeala, gozfarb, and anon8231489123 \(2023\)ShareGPT\_Vicuna\_unfiltered: a cleaned dump of multi\-turn user–ChatGPT conversations\.Note:https://huggingface\.co/datasets/Aeala/ShareGPT\_Vicuna\_unfilteredCited by:[§5\.1](https://arxiv.org/html/2606.11552#S5.SS1.p3.3),[§6\.2](https://arxiv.org/html/2606.11552#S6.SS2.p1.5)\.
- A\. Agrawal, A\. Panwar, J\. Mohan, N\. Kwatra, B\. S\. Gulavani, and R\. Ramjee \(2023\)SARATHI: efficient LLM inference by piggybacking decodes with chunked prefills\.arXiv preprint arXiv:2308\.16369\.Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Ankner, R\. Parthasarathy, A\. Nrusimha, C\. Rinard, J\. Ragan\-Kelley, and W\. Brandon \(2024\)Hydra: sequentially\-dependent draft heads for Medusa decoding\.InConference on Language Modeling \(COLM\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px3.p1.1)\.
- M\. Arriola, A\. Gokaslan, J\. T\. Chiu, Z\. Yang, Z\. Qi, J\. Han, S\. S\. Sahoo, and V\. Kuleshov \(2025\)Block diffusion: interpolating between autoregressive and diffusion language models\.Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1),[§3\.3](https://arxiv.org/html/2606.11552#S3.SS3.p1.2)\.
- J\. Austin, D\. D\. Johnson, J\. Ho, D\. Tarlow, and R\. van den Berg \(2021a\)Structured denoising diffusion models in discrete state\-spaces\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1),[§3\.3](https://arxiv.org/html/2606.11552#S3.SS3.p1.2)\.
- J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le, and C\. Sutton \(2021b\)Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§5\.1](https://arxiv.org/html/2606.11552#S5.SS1.p5.3)\.
- T\. Cai, Y\. Li, Z\. Geng, H\. Peng, J\. D\. Lee, D\. Chen, and T\. Dao \(2024\)Medusa: simple LLM inference acceleration framework with multiple decoding heads\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Chaudhary \(2023\)Code Alpaca: an instruction\-following LLaMA model for code generation\.Note:https://github\.com/sahil280114/codealpacaCited by:[§6\.2](https://arxiv.org/html/2606.11552#S6.SS2.p1.5)\.
- C\. Chen, S\. Borgeaud, G\. Irving, J\. Lespiau, L\. Sifre, and J\. Jumper \(2023\)Accelerating large language model decoding with speculative sampling\.arXiv preprint arXiv:2302\.01318\.Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p2.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2606.11552#S3.SS2.p1.9)\.
- J\. Chen, Y\. Liang, and Z\. Liu \(2026\)DFlash: block diffusion for flash speculative decoding\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p4.2),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1),[§3\.3](https://arxiv.org/html/2606.11552#S3.SS3.p2.8),[§5\.3](https://arxiv.org/html/2606.11552#S5.SS3.p1.2),[§6\.2](https://arxiv.org/html/2606.11552#S6.SS2.p1.5)\.
- M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. d\. O\. Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman,et al\.\(2021\)Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p1.1),[§5\.1](https://arxiv.org/html/2606.11552#S5.SS1.p5.3)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman \(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§5\.1](https://arxiv.org/html/2606.11552#S5.SS1.p5.3)\.
- T\. Dao, D\. Y\. Fu, S\. Ermon, A\. Rudra, and C\. Ré \(2022\)FlashAttention: fast and memory\-efficient exact attention with IO\-awareness\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p2.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- T\. Dao \(2024\)FlashAttention\-2: faster attention with better parallelism and work partitioning\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- DeepSeek\-AI, D\. Guo, D\. Yang, H\. Zhang, J\. Song, R\. Zhang, R\. Xu, Q\. Zhu, S\. Ma, P\. Wang,et al\.\(2025\)DeepSeek\-R1: incentivizing reasoning capability in LLMs via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p1.1)\.
- T\. Dettmers, M\. Lewis, Y\. Belkada, and L\. Zettlemoyer \(2022\)LLM\.int8\(\): 8\-bit matrix multiplication for transformers at scale\.Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p2.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Du, J\. Jiang, X\. Yuanchen, J\. Wu, S\. Yu, Y\. Li, S\. Li, K\. Xu, L\. Nie, Z\. Tu, and Y\. You \(2024\)GliDe with a CaPE: a low\-hassle method to accelerate speculative decoding\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px3.p1.1)\.
- E\. Frantar and D\. Alistarh \(2023\)SparseGPT: massive language models can be accurately pruned in one\-shot\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p2.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- E\. Frantar, S\. Ashkboos, T\. Hoefler, and D\. Alistarh \(2023\)GPTQ: accurate post\-training quantization for generative pre\-trained transformers\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p2.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Fu, P\. Bailis, I\. Stoica, and H\. Zhang \(2024\)Break the sequential dependency of LLM inference using lookahead decoding\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Fu, L\. Whalen, A\. Garg, C\. Wu, M\. Khadkevich, N\. Oswald, E\. Xie, D\. Egert, S\. T\. Sreenivas, S\. Diao, C\. Yu, Y\. Yu, W\. Chen, S\. Norouzi, J\. Liu, S\. Lan, L\. Zhu, J\. Wang, J\. Jiang, M\. Mardani, M\. Maghoumi, S\. Han, A\. Jukić, N\. Tajbakhsh, J\. Kautz, and P\. Molchanov \(2026a\)Nemotron\-labs\-diffusion: a tri\-mode language model unifying autoregressive, diffusion, and self\-speculation decoding\.Technical reportNVIDIA\.Note:Technical reportCited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1)\.
- Y\. Fu, L\. Whalen, Z\. Ye, X\. Dong, S\. Diao, J\. Liu, C\. Wu, H\. Zhang, E\. Xie, S\. Han, M\. Khadkevich, J\. Kautz, Y\. C\. Lin, and P\. Molchanov \(2026b\)Efficient\-dlm: from autoregressive to diffusion language models, and beyond in speed\.External Links:2512\.14067,[Link](https://arxiv.org/abs/2512.14067)Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1)\.
- M\. Ghazvininejad, O\. Levy, Y\. Liu, and L\. Zettlemoyer \(2019\)Mask\-Predict: parallel decoding of conditional masked language models\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1)\.
- S\. Gong, M\. Li, J\. Feng, Z\. Wu, and L\. Kong \(2023\)DiffuSeq: sequence to sequence text generation with diffusion models\.Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§5\.6](https://arxiv.org/html/2606.11552#S5.SS6.p1.11),[§6\.2](https://arxiv.org/html/2606.11552#S6.SS2.p1.5)\.
- J\. Gu, J\. Bradbury, C\. Xiong, V\. O\. K\. Li, and R\. Socher \(2018\)Non\-autoregressive neural machine translation\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1)\.
- I\. Gulrajani and T\. B\. Hashimoto \(2023\)Likelihood\-based diffusion language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1)\.
- G\. Hinton, O\. Vinyals, and J\. Dean \(2015\)Distilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p2.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- N\. Jain, K\. Han, A\. Gu, W\. Li, F\. Yan, T\. Zhang, S\. Wang, A\. Solar\-Lezama, K\. Sen, and I\. Stoica \(2024\)LiveCodeBench: holistic and contamination free evaluation of large language models for code\.arXiv preprint arXiv:2403\.07974\.Cited by:[§5\.1](https://arxiv.org/html/2606.11552#S5.SS1.p5.3)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with PagedAttention\.InProceedings of the 29th Symposium on Operating Systems Principles \(SOSP\),Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p2.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Lee, E\. Mansimov, and K\. Cho \(2018\)Deterministic non\-autoregressive neural sequence modeling by iterative refinement\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1)\.
- Y\. Leviathan, M\. Kalman, and Y\. Matias \(2023\)Fast inference from transformers via speculative decoding\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p2.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2606.11552#S3.SS2.p1.12),[§3\.2](https://arxiv.org/html/2606.11552#S3.SS2.p1.9)\.
- S\. Li, Y\. Zhu, C\. Wang, F\. Yin, S\. Shi, Y\. Wang, Y\. Zhang, Y\. Huang, H\. Zheng, and Y\. Zhang \(2025a\)SpecForge: train speculative decoding models effortlessly and port them smoothly to SGLang serving\.Note:https://github\.com/sgl\-project/SpecForgeCited by:[§5\.1](https://arxiv.org/html/2606.11552#S5.SS1.p4.7)\.
- X\. L\. Li, J\. Thickstun, I\. Gulrajani, P\. Liang, and T\. B\. Hashimoto \(2022\)Diffusion\-LM improves controllable text generation\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1)\.
- Y\. Li, F\. Wei, C\. Zhang, and H\. Zhang \(2024a\)EAGLE\-2: faster inference of language models with dynamic draft trees\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p3.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2606.11552#S3.SS2.p2.11)\.
- Y\. Li, F\. Wei, C\. Zhang, and H\. Zhang \(2024b\)EAGLE: speculative sampling requires rethinking feature uncertainty\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p3.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px3.p1.1),[§3\.3](https://arxiv.org/html/2606.11552#S3.SS3.p2.8)\.
- Y\. Li, F\. Wei, C\. Zhang, and H\. Zhang \(2025b\)EAGLE\-3: scaling up inference acceleration of large language models via training\-time test\.arXiv preprint arXiv:2503\.01840\.Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p3.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px3.p1.1),[§3\.3](https://arxiv.org/html/2606.11552#S3.SS3.p2.8)\.
- J\. Lin, J\. Tang, H\. Tang, S\. Yang, W\. Chen, W\. Wang, G\. Xiao, X\. Dang, C\. Gan, and S\. Han \(2024\)AWQ: activation\-aware weight quantization for LLM compression and acceleration\.InProceedings of Machine Learning and Systems \(MLSys\),Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p2.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- F\. Liu, Y\. Tang, Z\. Liu, Y\. Ni, K\. Han, and Y\. Wang \(2024a\)Kangaroo: lossless self\-speculative decoding via double early exiting\.Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px3.p1.1)\.
- J\. Liu, X\. Dong, Z\. Ye, R\. Mehta, Y\. Fu, V\. Singh, J\. Kautz, C\. Zhang, and P\. Molchanov \(2025\)TiDAR: think in diffusion, talk in autoregression\.External Links:2511\.08923,[Link](https://arxiv.org/abs/2511.08923)Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1)\.
- X\. Liu, L\. Hu, P\. Bailis, I\. Stoica, Z\. Deng, A\. Cheung, and H\. Zhang \(2024b\)Online speculative decoding\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px2.p1.1)\.
- I\. Loshchilov and F\. Hutter \(2019\)Decoupled weight decay regularization\.External Links:1711\.05101,[Link](https://arxiv.org/abs/1711.05101)Cited by:[§5\.1](https://arxiv.org/html/2606.11552#S5.SS1.p4.7)\.
- A\. Lou, C\. Meng, and S\. Ermon \(2024\)Discrete diffusion modeling by estimating the ratios of the data distribution\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1),[§3\.3](https://arxiv.org/html/2606.11552#S3.SS3.p1.2)\.
- Meta AI \(2024\)Llama 3\.2: revolutionizing edge AI and vision with open, customizable models\.Note:https://ai\.meta\.com/blog/llama\-3\-2\-connect\-2024\-vision\-edge\-mobile\-devices/Cited by:[§5\.6](https://arxiv.org/html/2606.11552#S5.SS6.p1.11)\.
- X\. Miao, G\. Oliaro, Z\. Zhang, X\. Cheng, Z\. Wang, Z\. Zhang, R\. Y\. Y\. Wong, A\. Zhu, L\. Yang, X\. Shi, C\. Shi, Z\. Chen, D\. Arfeen, R\. Abhyankar, and Z\. Jia \(2024\)SpecInfer: accelerating large language model serving with tree\-based speculative inference and verification\.InProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems \(ASPLOS\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px2.p1.1)\.
- D\. Nathawani and NVIDIA \(2025\)Nemotron post\-training dataset V2\.Note:https://huggingface\.co/datasets/nvidia/Nemotron\-Post\-Training\-Dataset\-v2Cited by:[§6\.2](https://arxiv.org/html/2606.11552#S6.SS2.p1.5)\.
- S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. Li \(2025\)Large language diffusion models\.arXiv preprint arXiv:2502\.09992\.Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1),[§3\.3](https://arxiv.org/html/2606.11552#S3.SS3.p1.2)\.
- NVIDIA Corporation \(2022\)NVIDIA H100 tensor core GPU architecture\.Note:https://resources\.nvidia\.com/en\-us\-tensor\-core/gtc22\-whitepaper\-hopperCited by:[§5\.1](https://arxiv.org/html/2606.11552#S5.SS1.p4.7)\.
- OpenAI \(2024\)OpenAI o1 system card\.arXiv preprint arXiv:2412\.16720\.Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p1.1)\.
- R\. Pope, S\. Douglas, A\. Chowdhery, J\. Devlin, J\. Bradbury, J\. Heek, K\. Xiao, S\. Agrawal, and J\. Dean \(2023\)Efficiently scaling transformer inference\.Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p2.1)\.
- Project Numina \(AI\-MO\) \(2024\)AIMO validation AIME: 90 american invitational mathematics examination problems from AIME 2022–2024\.Note:https://huggingface\.co/datasets/AI\-MO/aimo\-validation\-aimeCited by:[§5\.1](https://arxiv.org/html/2606.11552#S5.SS1.p5.3)\.
- L\. Ringel and Y\. Romano \(2026\)Accelerating speculative decoding with block diffusion draft trees\.arXiv preprint arXiv:2604\.12989\.Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p4.2),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px5.p1.2),[§6\.4](https://arxiv.org/html/2606.11552#S6.SS4.p1.4)\.
- B\. Rozière, J\. Gehring, F\. Gloeckle, S\. Sootla, I\. Gat, X\. E\. Tan, Y\. Adi, J\. Liu, T\. Remez, J\. Rapin,et al\.\(2023\)Code Llama: open foundation models for code\.arXiv preprint arXiv:2308\.12950\.Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p1.1)\.
- A\. Samarin, S\. Krutikov, A\. Shevtsov, S\. Skvortsov, F\. Fisin, and A\. Golubev \(2026\)LK losses: direct acceptance rate optimization for speculative decoding\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px5.p1.2)\.
- J\. Sandler, J\. K\. Christopher, T\. Hartvigsen, and F\. Fioretto \(2025\)SpecDiff\-2: scaling diffusion drafter alignment for faster speculative decoding\.arXiv preprint arXiv:2511\.00606\.Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px5.p1.2),[§6\.5](https://arxiv.org/html/2606.11552#S6.SS5.p1.6)\.
- V\. Sanh, L\. Debut, J\. Chaumond, and T\. Wolf \(2019\)DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter\.arXiv preprint arXiv:1910\.01108\.Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- N\. Savinov, J\. Chung, M\. Binkowski, E\. Elsen, and A\. van den Oord \(2022\)Step\-unrolled denoising autoencoders for text generation\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px4.p1.1)\.
- M\. Stern, N\. Shazeer, and J\. Uszkoreit \(2018\)Blockwise parallel decoding for deep autoregressive models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px2.p1.1)\.
- M\. Sun, Z\. Liu, A\. Bair, and J\. Z\. Kolter \(2024\)A simple and effective pruning approach for large language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- T\. Wu, Y\. Yao, Z\. Qi, H\. Zheng, Z\. Wang, H\. Ma, L\. Liao, H\. Lakkaraju, J\. Li, and Y\. Du \(2026\)D\-PACE: dynamic position\-aware cross\-entropy for parallel speculative drafting\.arXiv preprint arXiv:2605\.18810\.Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px5.p1.2)\.
- G\. Xiao, J\. Lin, M\. Seznec, H\. Wu, J\. Demouth, and S\. Han \(2023\)SmoothQuant: accurate and efficient post\-training quantization for large language models\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§5\.6](https://arxiv.org/html/2606.11552#S5.SS6.p1.11)\.
- G\. Yu, J\. S\. Jeong, G\. Kim, S\. Kim, and B\. Chun \(2022\)Orca: a distributed serving system for transformer\-based generative models\.In16th USENIX Symposium on Operating Systems Design and Implementation \(OSDI\),Cited by:[§1](https://arxiv.org/html/2606.11552#S1.p2.1),[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Zhang, J\. Wang, H\. Li, L\. Shou, K\. Chen, G\. Chen, and S\. Mehrotra \(2024a\)Draft & verify: lossless large language model acceleration via self\-speculative decoding\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px2.p1.1)\.
- L\. Zhang, X\. Wang, Y\. Huang, and R\. Xu \(2024b\)Learning harmonized representations for speculative sampling\.arXiv preprint arXiv:2408\.15766\.Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px3.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-judge with MT\-Bench and Chatbot Arena\.InAdvances in Neural Information Processing Systems \(NeurIPS\), Datasets and Benchmarks Track,Cited by:[§5\.1](https://arxiv.org/html/2606.11552#S5.SS1.p5.3)\.
- Y\. Zhou, K\. Lyu, A\. S\. Rawat, A\. K\. Menon, A\. Rostamizadeh, S\. Kumar, J\. Kagénäck, and R\. Agarwal \(2024\)DistillSpec: improving speculative decoding via knowledge distillation\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2606.11552#S2.SS0.SSS0.Px5.p1.2)\.