DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

# DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting
Source: [https://arxiv.org/html/2607.07409](https://arxiv.org/html/2607.07409)
Hong\-Kai Zheng, Piji Li College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China The Key Laboratory of Brain\-Machine Intelligence Technology, Ministry of Education, Nanjing, China \{dt\_ttt,pjli\}@nuaa\.edu\.cn

###### Abstract

Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel\. Block\-parallel drafters such as DFlash further improve drafting efficiency by predicting an entire block in one pass, but their position\-wise predictions lack explicit intra\-block causal conditioning\. Recent methods such as Domino and DSpark attempt to introduce such causality into block\-parallel drafting, but they require training the draft model from scratch, which limits their flexibility and increases training cost\. We proposeDeLS\-Spec, a decoupled long\-short context speculative decoding method\. DeLS\-Spec treats the fixed DFlash model as a long\-context expert and introduces a lightweight local head as a short\-context expert\. The local head can be trained independently with a standard next\-token prediction objective, without joint training with the target model or the DFlash backbone, leading to extremely low training cost\. At inference time, DeLS\-Spec combines long\-context and short\-context logits, and the local head is not tied to a specific DFlash checkpoint, making the method more modular and flexible\. Experiments on Qwen3 models show that DeLS\-Spec consistently improves speedup and average acceptance length over DFlash across math, code, and dialogue benchmarks\. Code is available at[GitHub](https://github.com/dt-3t/DeLS-Spec)\.

## 1Introduction

Large language models are typically decoded autoregressively\(Achiamet al\.,[2023](https://arxiv.org/html/2607.07409#bib.bib34); Liuet al\.,[2024](https://arxiv.org/html/2607.07409#bib.bib35); Yanget al\.,[2025](https://arxiv.org/html/2607.07409#bib.bib36)\), where each generated token requires a new forward pass of the target model\. This sequential process becomes a major latency bottleneck, especially for long\-form generation and interactive applications\. Speculative decoding mitigates this problem by using a lightweight draft model to propose multiple future tokens, which are then verified in parallel by the target model while preserving the target distribution\(Leviathanet al\.,[2023](https://arxiv.org/html/2607.07409#bib.bib2); Chenet al\.,[2023](https://arxiv.org/html/2607.07409#bib.bib3)\)\.

The efficiency of speculative decoding depends heavily on the drafter\(Caiet al\.,[2024](https://arxiv.org/html/2607.07409#bib.bib6); Ankneret al\.,[2024](https://arxiv.org/html/2607.07409#bib.bib7); Liet al\.,[2024b](https://arxiv.org/html/2607.07409#bib.bib8);[a](https://arxiv.org/html/2607.07409#bib.bib9);[2026b](https://arxiv.org/html/2607.07409#bib.bib10)\)\. Traditional autoregressive drafters maintain strong local consistency but still generate draft tokens sequentially\(Liet al\.,[2024b](https://arxiv.org/html/2607.07409#bib.bib8); Chenget al\.,[2024](https://arxiv.org/html/2607.07409#bib.bib11)\)\. Recent block\-parallel drafters, such as DFlash, improve drafting efficiency by predicting an entire block of tokens in one pass\(Anet al\.,[2025](https://arxiv.org/html/2607.07409#bib.bib23); Liuet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib24); Chenet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib25)\)\. However, this parallelism also introduces a limitation: tokens inside the same draft block are predicted largely independently, without explicitly conditioning on previously drafted tokens in that block\(Chenet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib25); Huanget al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib28); Rheinboldtet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib30)\)\. As a result, DFlash provides strong long\-context predictions from the prefix, but lacks explicit short\-context causal modeling within the draft block\.

Several recent methods, including Domino and DSpark, attempt to address this issue by introducing intra\-block causality into block\-parallel drafting\(Huanget al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib28); Chenget al\.,[2026b](https://arxiv.org/html/2607.07409#bib.bib1); Huet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib29); Rheinboldtet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib30)\)\. These methods show that modeling causal dependencies inside the draft block can improve acceptance\. However, they usually require training a new draft model from scratch or jointly training the draft backbone with additional causal components\. This makes them costly to apply when a DFlash\-style drafter has already been trained, and limits their flexibility across different draft checkpoints\.

In this work, we proposeDeLS\-Spec, a decoupled long\-short context speculative decoding method\. Instead of modifying or retraining the DFlash backbone\(Chenet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib25)\), DeLS\-Spec keeps DFlash fixed as a long\-context expert and introduces a lightweight local head as a short\-context expert\. The DFlash expert captures semantic and task\-level information from the full prefix, while the local head captures causal dependencies from the already drafted tokens inside the current block\.

A key property of DeLS\-Spec is its decoupled training\. The local head is trained independently with a standard next\-token prediction objective on plain text data\. It does not require target\-model hidden states, DFlash hidden states, or joint optimization with the speculative decoding pipeline\. This makes training extremely cheap and allows the local head to be attached to existing DFlash\-style checkpoints after they have been trained\. At inference time, DeLS\-Spec combines the long\-context logits from DFlash and the short\-context logits from the local head\(Hinton,[2002](https://arxiv.org/html/2607.07409#bib.bib33)\)\. Since the local head is not trained together with a specific DFlash checkpoint, it is not one\-to\-one tied to the DFlash weights during inference\. This makes the method more modular and flexible: a trained local head can be reused or transferred across compatible DFlash checkpoints, rather than requiring a new draft model to be trained for each setting\.

We evaluate DeLS\-Spec on Qwen3 models\(Yanget al\.,[2025](https://arxiv.org/html/2607.07409#bib.bib36)\)across math, code, and dialogue benchmarks\. Experiments show that DeLS\-Spec consistently improves both decoding speedup and average acceptance length over DFlash\. These results demonstrate that decoupling long\-context block\-parallel drafting from short\-context causal correction is an efficient and practical way to enhance existing speculative decoding systems\.

Our contributions are summarized as follows:

- •We propose DeLS\-Spec, a decoupled long\-short context speculative decoding method that improves DFlash\-style block\-parallel drafting without retraining the DFlash backbone\.
- •We introduce a lightweight local head that provides intra\-block causal information and can be trained independently with a standard next\-token prediction objective\.
- •DeLS\-Spec consistently enhances DFlash on Qwen3 models across math, code, and dialogue benchmarks, while offering modular flexibility across different checkpoints\.

## 2Related Work

#### Speculative decoding and classical drafters\.

Speculative decoding accelerates autoregressive inference by using a lightweight drafter to propose multiple tokens and a target model to verify them in parallel while preserving the target distribution\(Leviathanet al\.,[2023](https://arxiv.org/html/2607.07409#bib.bib2); Chenet al\.,[2023](https://arxiv.org/html/2607.07409#bib.bib3)\)\. Early blockwise decoding and tree\-based verification reduce sequential decoding rounds or expand candidate coverage\(Sternet al\.,[2018](https://arxiv.org/html/2607.07409#bib.bib4); Miaoet al\.,[2024](https://arxiv.org/html/2607.07409#bib.bib5)\)\. Later drafters improve proposal quality through target\-attached heads, sequentially dependent heads, feature\-level drafting, dynamic draft trees, recurrent drafting, and distillation\(Caiet al\.,[2024](https://arxiv.org/html/2607.07409#bib.bib6); Ankneret al\.,[2024](https://arxiv.org/html/2607.07409#bib.bib7); Liet al\.,[2024b](https://arxiv.org/html/2607.07409#bib.bib8);[a](https://arxiv.org/html/2607.07409#bib.bib9);[2026b](https://arxiv.org/html/2607.07409#bib.bib10); Chenget al\.,[2024](https://arxiv.org/html/2607.07409#bib.bib11); Zhouet al\.,[2024](https://arxiv.org/html/2607.07409#bib.bib12)\)\. However, many drafters still require sequential drafting, feature updates, or tree construction, making drafting cost grow with the speculative budget\.

#### Parallel and diffusion\-based drafting\.

Parallel generation reduces drafting latency by predicting multiple positions simultaneously, but also weakens token dependency, as observed in non\-autoregressive and diffusion\-style language generation\(Guet al\.,[2017](https://arxiv.org/html/2607.07409#bib.bib13); Nieet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib14); Arriolaet al\.,[2025](https://arxiv.org/html/2607.07409#bib.bib15); Chenget al\.,[2026a](https://arxiv.org/html/2607.07409#bib.bib17); Liuet al\.,[2025](https://arxiv.org/html/2607.07409#bib.bib18)\)\. Recent speculative drafters therefore use diffusion or parallel prediction to generate draft blocks efficiently\(Christopheret al\.,[2025](https://arxiv.org/html/2607.07409#bib.bib20); Liet al\.,[2026a](https://arxiv.org/html/2607.07409#bib.bib21); Sandleret al\.,[2025](https://arxiv.org/html/2607.07409#bib.bib22); Anet al\.,[2025](https://arxiv.org/html/2607.07409#bib.bib23); Liuet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib24)\)\. DFlash is most relevant to our work: it uses a lightweight block\-diffusion drafter with target hidden\-state injection to obtain strong long\-context conditioning and generate a whole block in one pass\(Chenet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib25)\)\. Follow\-up methods build draft trees or improve training objectives for such parallel drafters\(Ringel and Romano,[2026](https://arxiv.org/html/2607.07409#bib.bib26); Wuet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib27)\)\. Nevertheless, DFlash\-style block\-parallel prediction mainly models position\-wise long\-context distributions and lacks explicit conditioning on the locally drafted prefix, which can hurt later\-token acceptance\.

#### Intra\-block causality and parallel\-drafter correction\.

Several methods address the missing local causality in parallel drafting\. Hydra introduces dependency among draft heads\(Ankneret al\.,[2024](https://arxiv.org/html/2607.07409#bib.bib7)\); Domino corrects DFlash\-style logits with a GRU causal encoder and low\-rank residual head\(Huanget al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib28)\); DSpark adds semi\-autoregressive Markov/RNN heads and confidence scheduling\(Chenget al\.,[2026b](https://arxiv.org/html/2607.07409#bib.bib1)\); JetSpec uses tree\-causal attention for branch\-wise causal conditioning\(Huet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib29)\); and TreeFlash approximates local autoregressive distributions on top of DFlash\(Rheinboldtet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib30)\)\. D\-PACE further shows that accepted length is tied to position\-dependent prefix acceptance\(Wuet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib27)\)\. Although structured non\-autoregressive decoding and CTC\-based drafting can introduce dependencies or alignments\(Sunet al\.,[2019](https://arxiv.org/html/2607.07409#bib.bib31); Wenet al\.,[2024](https://arxiv.org/html/2607.07409#bib.bib32)\), they are less compatible with the exact per\-position probabilities required by standard lossless verification\. Unlike these unified correction pipelines, DeLS\-Spec keeps the trained DFlash drafter fixed as a long\-context expert, independently trains a lightweight short\-context expert for intra\-block causality, and fuses their logits with unigram\-prior correction to avoid double\-counting frequency bias\.

## 3Preliminaries

### 3\.1Parallel Draft Model: DFlash

Lety=x<ky=x\_\{<k\}denote the observed long context, and letxk,xk\+1,…,xk\+sx\_\{k\},x\_\{k\+1\},\\ldots,x\_\{k\+s\}denote the block of future tokens to be drafted\. Parallel draft models such as DFlash\(Chenet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib25)\)perform block\-parallel drafting by predicting multiple future tokens in a single forward pass\. Specifically, for each position inside the draft block, the model predicts

pL​\(xi∣y\),i∈\{k,…,k\+s\}\.p\_\{L\}\(x\_\{i\}\\mid y\),\\qquad i\\in\\\{k,\\ldots,k\+s\\\}\.\(1\)Although DFlash is described as a diffusion language model, its practical drafting procedure consists of one forward pass followed by one sampling step, which can be viewed as a diffusion language model with a single denoising step\. This design significantly reduces draft latency compared with autoregressive draft models\. However, because tokens within the block are predicted in parallel, each position is unaware of the tokens generated at other positions\. As a result, the causal dependency inside the draft block is weak, which limits the accepted length during speculative decoding\.

### 3\.2Introducing Intra\-Block Causality

Recent approaches such as Domino\(Huanget al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib28)\)and DSpark\(Chenget al\.,[2026b](https://arxiv.org/html/2607.07409#bib.bib1)\)address this issue by introducing intra\-block causal modeling on top of the parallel DFlash backbone\. They attach a lightweight causal correction head after the parallel backbone and directly optimize

ℒcc=−∑i=k\+1k\+slog⁡p​\(xi∣y,zi\),zi=xk:i−1\.\\mathcal\{L\}\_\{\\mathrm\{cc\}\}=\-\\sum\_\{i=k\+1\}^\{k\+s\}\\log p\(x\_\{i\}\\mid y,z\_\{i\}\),\\qquad z\_\{i\}=x\_\{k:i\-1\}\.\(2\)whereziz\_\{i\}denotes the local prefix within the draft block\. Although this causal correction improves local consistency, it is typically trained as part of the complete draft pipeline\. In particular, it depends on the target model and requires joint training or fine\-tuning of the parallel DFlash backbone and the causal correction head\. Therefore, migrating such causal dependency mechanisms to an already trained DFlash model can be expensive, especially when the model size increases\.

## 4Method

![Refer to caption](https://arxiv.org/html/2607.07409v1/x1.png)Figure 1:Overview of DeLS\-Spec\. DFlash produces long\-context logits in parallel, while a lightweight local head sequentially estimates short\-context logits from the draft prefix\. DeLS\-Spec fuses them with a unigram prior to obtain final draft logits, which are sampled and then verified by the target model under standard speculative decoding\.### 4\.1Decoupled Long\-Short Contexts

We proposeDeLS\-Spec, a decoupled long\-short context modeling method for speculative drafting\. The key observation is that an already trained DFlash model provides a long\-context conditional distributionpL​\(xi∣y\)p\_\{L\}\(x\_\{i\}\\mid y\), which captures semantic and task\-level constraints from the full prefixyy\. What is missing is the short\-range causal dependency induced by the local block prefixziz\_\{i\}\. Therefore, a natural starting point is a product\-of\-experts\(Hinton,[2002](https://arxiv.org/html/2607.07409#bib.bib33)\)formulation:

p​\(xi∣y,zi\)≈p​\(xi∣y\)​p​\(xi∣zi\)\.p\(x\_\{i\}\\mid y,z\_\{i\}\)\\approx p\(x\_\{i\}\\mid y\)\\,p\(x\_\{i\}\\mid z\_\{i\}\)\.\(3\)In this view, the long\-context expert provides global semantic constraints, while the short\-context expert provides local coherence constraints within the draft block\. However, this factorization is not an exact equality\. More precisely, we have

p​\(xi∣y,zi\)\\displaystyle p\(x\_\{i\}\\mid y,z\_\{i\}\)=p​\(xi∣y\)​p​\(xi∣zi\)p​\(xi\)⋅p​\(y\)​p​\(zi\)p​\(y,zi\)⋅p​\(y,zi∣xi\)p​\(y∣xi\)​p​\(zi∣xi\)\.\\displaystyle=\\frac\{p\(x\_\{i\}\\mid y\)\\,p\(x\_\{i\}\\mid z\_\{i\}\)\}\{p\(x\_\{i\}\)\}\\cdot\\frac\{p\(y\)\\,p\(z\_\{i\}\)\}\{p\(y,z\_\{i\}\)\}\\cdot\\frac\{p\(y,z\_\{i\}\\mid x\_\{i\}\)\}\{p\(y\\mid x\_\{i\}\)\\,p\(z\_\{i\}\\mid x\_\{i\}\)\}\.\(4\)The proof of this factorization is provided in Appendix[B](https://arxiv.org/html/2607.07409#A2)\. The second factor,

p​\(y\)​p​\(zi\)p​\(y,zi\),\\frac\{p\(y\)\\,p\(z\_\{i\}\)\}\{p\(y,z\_\{i\}\)\},\(5\)does not depend on the candidate tokenxix\_\{i\}and can therefore be absorbed into the normalization constant when applying softmax or ranking candidate tokens\. Thus, the token\-dependent part can be written as

p​\(xi∣y,zi\)∝p​\(xi∣y\)​p​\(xi∣zi\)p​\(xi\)⋅p​\(y,zi∣xi\)p​\(y∣xi\)​p​\(zi∣xi\)\.p\(x\_\{i\}\\mid y,z\_\{i\}\)\\propto\\frac\{p\(x\_\{i\}\\mid y\)\\,p\(x\_\{i\}\\mid z\_\{i\}\)\}\{p\(x\_\{i\}\)\}\\cdot\\frac\{p\(y,z\_\{i\}\\mid x\_\{i\}\)\}\{p\(y\\mid x\_\{i\}\)\\,p\(z\_\{i\}\\mid x\_\{i\}\)\}\.\(6\)
This decomposition yields two important implications\. First, directly multiplyingp​\(xi∣y\)p\(x\_\{i\}\\mid y\)andp​\(xi∣zi\)p\(x\_\{i\}\\mid z\_\{i\}\)double\-counts the unigram prior of the tokenxix\_\{i\}, since both conditional distributions already contain the intrinsic frequency bias ofxix\_\{i\}\. Without correction, the product would over\-emphasize high\-frequency tokens\. Therefore, the unigram priorp​\(xi\)p\(x\_\{i\}\)should be divided out\.

Second, the residual term

R​\(xi;y,zi\)=p​\(y,zi∣xi\)p​\(y∣xi\)​p​\(zi∣xi\)R\(x\_\{i\};y,z\_\{i\}\)=\\frac\{p\(y,z\_\{i\}\\mid x\_\{i\}\)\}\{p\(y\\mid x\_\{i\}\)\\,p\(z\_\{i\}\\mid x\_\{i\}\)\}\(7\)captures the interaction between the long contextyyand the local prefixziz\_\{i\}that cannot be modeled by two independent conditional experts\. Fully modeling this residual may improve the approximation quality, but doing so requires joint access to the target model, the parallel draft backbone, and the causal correction module\. This corresponds to the design philosophy of methods such as Domino and DSpark, which attempt to learn such residual interactions through end\-to\-end training\.

In contrast, DeLS\-Spec deliberately ignores this residual term in exchange for modularity and training efficiency\. We approximate

log⁡p​\(xi∣y,zi\)≈log⁡pL​\(xi∣y\)\+log⁡pS​\(xi∣zi\)−log⁡pP​\(xi\),\\log p\(x\_\{i\}\\mid y,z\_\{i\}\)\\approx\\log p\_\{L\}\(x\_\{i\}\\mid y\)\+\\log p\_\{S\}\(x\_\{i\}\\mid z\_\{i\}\)\-\\log p\_\{P\}\(x\_\{i\}\),\(8\)wherepLp\_\{L\}is the long\-context distribution provided by the trained DFlash model,pSp\_\{S\}is an independently trained short\-context model, andpPp\_\{P\}is the unigram prior estimated from the training corpus\.

### 4\.2Independent Short\-Context Training

A key advantage of DeLS\-Spec is that the short\-context model can be trained completely independently\. It does not require hidden states from the target model, nor does it require access to the DFlash draft model\. The only requirement is that it shares the same tokenizer as the target model\. In our implementation,pS​\(xi∣zi\)p\_\{S\}\(x\_\{i\}\\mid z\_\{i\}\)is parameterized by a lightweight local RNN head and trained on plain text corpora with the standard next\-token prediction objective:

ℒS=−∑i=k\+1k\+slog⁡pS​\(xi∣zi\),zi=xk:i−1\.\\mathcal\{L\}\_\{S\}=\-\\sum\_\{i=k\+1\}^\{k\+s\}\\log p\_\{S\}\(x\_\{i\}\\mid z\_\{i\}\),\\qquad z\_\{i\}=x\_\{k:i\-1\}\.\(9\)This training objective encourages the short\-context model to capture local causal dependencies inside a draft block, independent of the long\-context drafting model\.

### 4\.3Logit Fusion at Inference

During inference, DeLS\-Spec is directly attached to an already trained DFlash draft model\. LetℓL​\(xi∣y\)\\ell\_\{L\}\(x\_\{i\}\\mid y\)denote the long logits produced by DFlash,ℓS​\(xi∣zi\)\\ell\_\{S\}\(x\_\{i\}\\mid z\_\{i\}\)denote the short logits produced by the local RNN head, andℓP​\(xi\)\\ell\_\{P\}\(x\_\{i\}\)denote the log unigram prior estimated from the training corpus\. We fuse these logits as

ℓ​\(xi\)=ℓL​\(xi∣y\)\+α​ℓS​\(xi∣zi\)−β​ℓP​\(xi\),\\ell\(x\_\{i\}\)=\\ell\_\{L\}\(x\_\{i\}\\mid y\)\+\\alpha\\,\\ell\_\{S\}\(x\_\{i\}\\mid z\_\{i\}\)\-\\beta\\,\\ell\_\{P\}\(x\_\{i\}\),\(10\)whereα\\alphaandβ\\betaare calibration coefficients\. The theoretical decomposition corresponds toα=β=1\\alpha=\\beta=1\. In practice, however, the long logits, short logits, and unigram prior are obtained from different sources and may have different scales\. Therefore,α\\alphaandβ\\betaare introduced to calibrate their relative contributions\. The unigram prior termℓP​\(xi\)\\ell\_\{P\}\(x\_\{i\}\)is computed once from corpus statistics and introduces no additional inference\-time model computation, its estimation details are provided in Appendix[A](https://arxiv.org/html/2607.07409#A1)\. The complete inference procedure is summarized in Figure[1](https://arxiv.org/html/2607.07409#S4.F1)and Algorithm[1](https://arxiv.org/html/2607.07409#alg1)\.

Algorithm 1DeLS\-Spec Inference1:Target model

MTM\_\{T\}, DFlash draft model

MLM\_\{L\}, local head

MSM\_\{S\}, unigram\-prior logits

ℓP\\ell\_\{P\}, fusion weights

α,β\\alpha,\\beta, draft span

ss, initial prefix

yy
2:Generated continuation appended to

yy
3:whilenot finisheddo

4:Get DFlash logits:

\{ℓL\(j\)\}j=0s←ML​\(y\)\\\{\\ell\_\{L\}^\{\(j\)\}\\\}\_\{j=0\}^\{s\}\\leftarrow M\_\{L\}\(y\)\.

5:Sample the first draft token:

x0∼Decode⁡\(softmax⁡\(ℓL\(0\)\)\)x\_\{0\}\\sim\\operatorname\{Decode\}\(\\operatorname\{softmax\}\(\\ell\_\{L\}^\{\(0\)\}\)\)\.

6:for

j=1j=1to

ssdo

7:Get short\-context logits:

ℓS\(j\)←MS​\(x0:j−1\)\\ell\_\{S\}^\{\(j\)\}\\leftarrow M\_\{S\}\(x\_\{0:j\-1\}\)\.

8:Fuse logits:

ℓ\(j\)=ℓL\(j\)\+α​ℓS\(j\)−β​ℓP\\ell^\{\(j\)\}=\\ell\_\{L\}^\{\(j\)\}\+\\alpha\\,\\ell\_\{S\}^\{\(j\)\}\-\\beta\\,\\ell\_\{P\}\.

9:Sample draft token:

xj∼Decode⁡\(softmax⁡\(ℓ\(j\)\)\)x\_\{j\}\\sim\\operatorname\{Decode\}\(\\operatorname\{softmax\}\(\\ell^\{\(j\)\}\)\)\.

10:endfor

11:Verify

\(x0,…,xs\)\(x\_\{0\},\\ldots,x\_\{s\}\)with one target\-model forward pass\.

12:Append accepted tokens and the bonus token, if any, to

yy\.

13:endwhile

14:return

yy

## 5Experiments

### 5\.1Experimental Setup

#### Models and Benchmarks\.

We evaluate DeLS\-Spec on Qwen3\-4B and Qwen3\-8B\(Yanget al\.,[2025](https://arxiv.org/html/2607.07409#bib.bib36)\)\. For the DFlash backbone\(Chenet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib25)\), we use the officially released checkpointsz\-lab/Qwen3\-4B\-DFlash\-b16andz\-lab/Qwen3\-8B\-DFlash\-b16, both with a draft block size of 16\. We evaluate on math reasoning, code generation, and open\-ended dialogue benchmarks\. The math benchmarks include GSM8K\(Cobbeet al\.,[2021](https://arxiv.org/html/2607.07409#bib.bib38)\), MATH\-500\(Hendryckset al\.,[2021](https://arxiv.org/html/2607.07409#bib.bib39)\), and AIME25\(Zhang and Math\-AI,[2025](https://arxiv.org/html/2607.07409#bib.bib37)\); the code benchmarks include HumanEval\(Chenet al\.,[2021](https://arxiv.org/html/2607.07409#bib.bib40)\), MBPP\(Austinet al\.,[2021](https://arxiv.org/html/2607.07409#bib.bib41)\), and LiveCodeBench\(Jainet al\.,[2025](https://arxiv.org/html/2607.07409#bib.bib42)\); and the dialogue benchmarks include MT\-Bench\(Zhenget al\.,[2023](https://arxiv.org/html/2607.07409#bib.bib43)\)and Alpaca\(Taoriet al\.,[2023](https://arxiv.org/html/2607.07409#bib.bib44)\)\. We report end\-to\-end decoding speedup over autoregressive decoding and the average acceptance lengthτ\\tau\.

#### Training Data\.

We use the DFlash authors’Qwen3\-4B\-Instruct\-100Kdata\(Chenet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib25)\)for the 4B setting and the Domino authors’Qwen3\-8B\-ShareGPTdata\(Huanget al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib28)\)for the 8B setting\. These corpora are used both to train the local head and to estimate the unigram prior\. Unlike methods that jointly fine\-tune the draft pipeline, DeLS\-Spec only trains the local head with the standard next\-token prediction objective on the training corpus\. The DFlash backbone used during evaluation is kept fixed and directly loaded from the releasedz\-labcheckpoints\.

#### Baselines\.

We compare DeLS\-Spec with autoregressive decoding and representative speculative decoding baselines, including EAGLE\-3, DART, and DFlash\(Liet al\.,[2026b](https://arxiv.org/html/2607.07409#bib.bib10); Liuet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib24); Chenet al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib25)\)\. DFlash is the block\-parallel backbone on which DeLS\-Spec is built\. We also include additional comparisons with Domino\-style fine\-tuning and other DFlash settings where applicable\.

#### Implementation\.

Unless otherwise specified, all results are evaluated with the Hugging Face Transformers backend\(Wolfet al\.,[2019](https://arxiv.org/html/2607.07409#bib.bib45)\)\. All training and evaluation runs are conducted on a single NVIDIA L20 GPU\. The local head is implemented as a GRU\-based RNN by default\. For DeLS\-Spec inference, we setα=β=0\.3\\alpha=\\beta=0\.3unless otherwise stated\. Following Domino, we optimize the inference implementation with CUDA Graphs and fused Triton kernels\(Tilletet al\.,[2019](https://arxiv.org/html/2607.07409#bib.bib46)\)to reduce the overhead of the local\-head decoding loop\. Detailed local\-head training hyperparameters are provided in Table[6](https://arxiv.org/html/2607.07409#A1.T6)\.

### 5\.2Main Results

Table[1](https://arxiv.org/html/2607.07409#S5.T1)presents the main results on Qwen3\-4B and Qwen3\-8B across math, code, and chat benchmarks\. DFlash already outperforms tree\-based baselines such as EAGLE\-3 and DART by a large margin\. Built upon this strong baseline, DeLS\-Spec further improves both decoding speedup and average acceptance lengthτ\\tauin almost all settings\.

The gains are particularly clear on math and code tasks, where local token dependencies are stronger\. For Qwen3\-4B at temperature0, DeLS\-Spec improves the speedup of DFlash from4\.74×4\.74\\timesto5\.02×5\.02\\timeson MBPP, from6\.09×6\.09\\timesto6\.35×6\.35\\timeson MATH\-500, and from5\.69×5\.69\\timesto5\.95×5\.95\\timeson AIME25\. Meanwhile, the acceptance length increases by up to0\.440\.44on AIME25 and0\.400\.40on MATH\-500\. Under stochastic decoding, DeLS\-Spec also brings notable improvements, e\.g\., on Qwen3\-4B at temperature11, HumanEval speedup increases from4\.61×4\.61\\timesto4\.85×4\.85\\times, withτ\\tauimproved from5\.845\.84to6\.216\.21\. Similar trends hold for Qwen3\-8B, where DeLS\-Spec improvesτ\\tauby0\.330\.33on HumanEval and MBPP at temperature0, and by0\.290\.29on MT\-Bench at temperature11\.

These results show that DeLS\-Spec complements DFlash by improving the local consistency of draft tokens\. The local head enables more drafted tokens to be accepted during verification, especially on math and code benchmarks, while preserving the original DFlash parallel drafting process\.

Table 1:Decoding speedup over vanilla autoregressive decoding and average acceptance length \(τ\\tau\) on Qwen3 models with a maximum of 2048 generated tokens\. Parenthesized values indicate the draft tree size for EAGLE\-3 and DART, and the draft block size for DFlash and DeLS\-Spec\. The average is computed over all listed benchmarks\.
### 5\.3General Adaptability of the DeLS\-Spec Local Head

We further test the general adaptability of the local head on two DFlash block\-7 checkpoints from DSpark\(Chenget al\.,[2026b](https://arxiv.org/html/2607.07409#bib.bib1)\):deepseek\-ai/dflash\_qwen3\_4b\_block7anddeepseek\-ai/dflash\_qwen3\_8b\_block7\. This setting examines whether the local head can transfer beyond our default DFlash configuration\. Since the local head is trained with block size1616, it can be directly used with DFlash models whose block size is no larger than1616\.

As shown in Table[2](https://arxiv.org/html/2607.07409#S5.T2), DeLS\-Spec consistently improves these DSpark checkpoints\. On the 4B checkpoint, DeLS\-Spec brings speedup gains on MATH\-500, AIME25, and HumanEval, improving DFlash by0\.26×0\.26\\timeson each benchmark\. It also increases the acceptance length by0\.330\.33on HumanEval and0\.310\.31on MATH\-500\. On the 8B checkpoint, DeLS\-Spec improvesτ\\tauby0\.320\.32on GSM8K and MATH\-500, and by0\.310\.31on LCB, while also increasing speedup across all benchmarks\.

These consistent improvements on independently released block\-7 checkpoints demonstrate that the DeLS\-Spec local head is not tied to a specific DFlash implementation\. Instead, it can serve as a plug\-in module that improves local draft\-token modeling for different DFlash\-style draft models\.

Table 2:Results of directly applying DeLS\-Spec to DSpark’s DFlash block\-7 checkpoints\. DeLS\-Spec achieves consistent performance improvements across all benchmarks\.
### 5\.4Effect ofα\\alphaandβ\\beta

We further study the effect of the two combination hyperparameters,α\\alphaandβ\\beta, by performing a grid search on Qwen3\-4B\. Figure[2](https://arxiv.org/html/2607.07409#S5.F2)summarizes the results\. The left panel shows thatα\\alphaandβ\\betashould be increased jointly to achieve the best performance, rather than tuning either one alone\. The heatmap also exhibits a clear diagonal trend, suggesting that in practiceα\\alphaandβ\\betacan simply be set to the same value, with values below0\.50\.5yielding the best performance\. In particular, the best region lies aroundα≈0\.3\\alpha\\approx 0\.3andβ≈0\.3\\beta\\approx 0\.3, indicating that a balanced contribution from the local head and the unigram\-prior correction is important\.

To better highlight the importance of subtracting the unigram prior, the middle panel normalizes each row by subtracting the corresponding value atβ=0\\beta=0\. This row\-wise comparison makes the effect ofβ\\betamore explicit: onceα\\alphais properly set, using a positiveβ\\betaconsistently improves the acceptance length over theβ=0\\beta=0case\. The right panel further summarizes this trend by plotting, for eachα\\alpha, the improvement of the bestβ\\betaoverβ=0\\beta=0\. We observe consistent gains across almost allα\\alphavalues, which confirms that unigram\-prior subtraction is a key component of the proposed formulation\.

![Refer to caption](https://arxiv.org/html/2607.07409v1/x2.png)

![Refer to caption](https://arxiv.org/html/2607.07409v1/x3.png)

![Refer to caption](https://arxiv.org/html/2607.07409v1/x4.png)

Figure 2:α,β\\alpha,\\;\\betascan on Qwen3\-4B\. Left: average acceptance length under differentα\\alphaandβ\\betavalues\. Middle: row\-wise improvement after subtracting the correspondingβ=0\\beta=0value\. Right: improvement of the bestβ\\betafor eachα\\alphaoverβ=0\\beta=0\.
### 5\.5Impact of the Omitted Residual Term

We next quantify how much performance is lost by omitting the residual termR​\(xi;y,zi\)R\(x\_\{i\};y,z\_\{i\}\)in DeLS\-Spec\. Table[3](https://arxiv.org/html/2607.07409#S5.T3)compares several variants in terms of average acceptance lengthτ\\tau\.DeLS\-Spec \(Markov\)follows the Markov head used in DSpark\(Chenget al\.,[2026b](https://arxiv.org/html/2607.07409#bib.bib1)\), where the logits of each draft token are corrected only based on the immediately preceding token\. In our framework, this corresponds to generating the short\-context logits using only the previous token\.Domino\-FTfreezes the DFlash backbone and trains only the Domino head\(Huanget al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib28)\), using the same training data and number of epochs as our DeLS\-Spec local head\.

Although Domino\-FT achieves the highest averageτ\\tau, the gap between DeLS\-Spec and Domino\-FT is small\. On average, DFlash obtainsτ=6\.04\\tau=6\.04, DeLS\-Spec \(RNN\) improves it to6\.356\.35, while Domino\-FT reaches6\.456\.45\. Thus, omitting the residual term only leads to a0\.100\.10drop in average acceptance length compared with Domino\-FT, while DeLS\-Spec still captures75\.69%75\.69\\%of Domino\-FT’s improvement over DFlash\. Theτ\\taugain ratio is computed asτDeLS\-Spec−τDFlashτDomino\-FT−τDFlash\.\\frac\{\\tau\_\{\\text\{DeLS\-Spec\}\}\-\\tau\_\{\\text\{DFlash\}\}\}\{\\tau\_\{\\text\{Domino\-FT\}\}\-\\tau\_\{\\text\{DFlash\}\}\}\.

The gains are also consistent across benchmarks\. DeLS\-Spec \(RNN\) recovers more than80%80\\%of Domino\-FT’s improvement on AIME25 and HumanEval, and even surpasses Domino\-FT on LCB with aτ\\taugain ratio of105\.56%105\.56\\%\. In addition, DeLS\-Spec \(RNN\) consistently outperforms DeLS\-Spec \(Markov\), improving the averageτ\\taufrom6\.286\.28to6\.356\.35\. This shows that modeling richer short\-range dependencies with an RNN local head is more effective than a one\-step Markov correction, while still avoiding the full residual modeling cost of Domino\-FT\.

Table 3:Average acceptance length comparison for estimating the impact of the omitted residual term\. The gain ratio reports how much of the Domino\-FT improvement over DFlash is recovered by DeLS\-Spec without explicitly learning the residual term\.
### 5\.6Do Learnableα\\alphaandβ\\betaHelp?

We further investigate whether learning the fusion weightsα\\alphaandβ\\betacan bring additional gains over fixed values\. As shown in Table[4](https://arxiv.org/html/2607.07409#S5.T4),α=β=0\\alpha=\\beta=0corresponds to the original DFlash baseline, whileα=β=0\.3\\alpha=\\beta=0\.3is our default setting\. For the learnable variant, we initializeα\\alphaandβ\\betato0\.30\.3and optimize them for 10K steps on the training data\. During this process, the target model, DFlash, and the local head are all loaded but frozen, and onlyα\\alphaandβ\\betaare updated with the cross\-entropy loss\.

The learnable variant consistently improves over the DFlash baseline, increasing the average acceptance length from6\.336\.33to6\.556\.55\. This confirms that assigning positive weights to the local correction and unigram\-prior subtraction is beneficial\. However, it is still worse than our simple fixed settingα=β=0\.3\\alpha=\\beta=0\.3, which achieves the best averageτ\\tauof6\.656\.65and outperforms the learnable variant on every benchmark\. For example, the fixed setting improvesτ\\tauover the learnable variant by0\.200\.20on MATH\-500,0\.130\.13on GSM8K and MBPP, and0\.060\.06on LCB\.

Appendix[C](https://arxiv.org/html/2607.07409#A3)reports the learned values ofα\\alphaandβ\\betain Table[7](https://arxiv.org/html/2607.07409#A3.T7)\. They are consistently positive and increase with draft\-token position, which supports the usefulness of both terms\. Nevertheless, directly optimizing these weights with cross\-entropy does not translate to better acceptance length on downstream benchmarks\. This suggests that the fixedα=β=0\.3\\alpha=\\beta=0\.3setting provides a better trade\-off between effectiveness, robustness, and simplicity, avoiding the need to load all components and perform extra tuning\.

Table 4:Comparison between fixed and learnable fusion weights on Qwen3\-4B\. The fixed settingα=β=0\.3\\alpha=\\beta=0\.3provides the best average acceptance length, while learnable weights offer limited additional benefit and can slightly underperform the tuned fixed fusion\.
### 5\.7Training Cost Comparison

We compare the training cost of Domino\-FT and DeLS\-Spec on a single NVIDIA L20 GPU with 48GB memory\. As shown in Table[5](https://arxiv.org/html/2607.07409#S5.T5), all methods are trained with batch size11, accumulation steps44, and one epoch\. For Domino\-FT and DeLS\-Spec \(RNN\), we sample 256 anchors from each example to construct draft blocks\. In contrast, DeLS\-Spec \(Markov\) does not rely on anchors or block construction, and is trained with dense token\-level supervision directly on the corpus\.

Table 5:Training cost comparison on a single NVIDIA L20 GPU \(48G\)\. DeLS\-Spec local heads require much less training time and memory than Domino\-FT, with the Markov variant being the cheapest due to dense supervision and its minimal parameterization\.DeLS\-Spec is substantially more efficient than Domino\-FT\. On Qwen3\-4B, Domino\-FT requires 13\.4 hours and 42\.6GB VRAM, while DeLS\-Spec \(RNN\) only takes 1\.1 hours and 9\.0GB VRAM, reducing training time by over12×12\\timesand memory usage by about4\.7×4\.7\\times\. The Markov variant is even cheaper, requiring only 0\.4 hours and 6\.5GB VRAM\. The gap becomes more pronounced on Qwen3\-8B: Domino\-FT runs out of memory on a 48GB L20 GPU, whereas DeLS\-Spec \(RNN\) still fits comfortably with 10\.1GB VRAM and finishes training in 1\.1 hours\. DeLS\-Spec \(Markov\) further reduces the cost to 0\.5 hours and 5\.9GB VRAM\.

These results highlight the practical advantage of DeLS\-Spec\. The key reason is that the DeLS\-Spec local head can be trained independently of both the target model and DFlash\. In contrast, Domino\-FT requires the target model to produce hidden states and DFlash to provide base logits, while the Domino head only learns the residual correction logits\. As a result, Domino\-FT must keep multiple large components involved during training, leading to much higher memory usage and training time\. DeLS\-Spec avoids this dependency by training a lightweight local head separately, making it significantly easier to apply under limited hardware resources\.

## 6Conclusion

We presentedDeLS\-Spec, a decoupled long\-short context method for block\-parallel speculative decoding\. DeLS\-Spec keeps DFlash fixed as a long\-context expert and adds a lightweight local head as a short\-context expert\. By combining their logits at inference time, it introduces intra\-block causal information without retraining the draft model from scratch\. The local head can be independently trained with a standard next\-token prediction objective, making training cheap and modular\. Since it is not one\-to\-one tied to a specific DFlash checkpoint, it can also be flexibly attached to compatible DFlash\-style drafters\. Experiments on Qwen3 models show that DeLS\-Spec consistently improves speedup and average acceptance length over DFlash, demonstrating an efficient way to enhance existing block\-parallel speculative decoding systems\.

## 7Limitations and Future Work

While DeLS\-Spec demonstrates significant efficiency and performance gains, our approach has several limitations that present opportunities for future research:

Omission of Residual Interactions\.To achieve extreme training efficiency and modularity, DeLS\-Spec deliberately omits the residual interaction term between the long context and the local short context\. As a result, while it recovers the majority of performance improvements, its average acceptance length is slightly bounded and falls marginally short of end\-to\-end jointly trained methods like Domino, which explicitly learn this full residual interaction\.

Generalization to Other Parallel Drafters\.Our current empirical evaluation is primarily built upon the DFlash architecture\. However, the theoretical framework of DeLS\-Spec is fundamentally agnostic to the underlying draft model\. Since the local head operates independently, future work will explore applying and validating DeLS\-Spec on a broader range of non\-autoregressive and parallel draft models beyond DFlash\.

Adaptive Logit Fusion Strategies\.Currently, DeLS\-Spec employs a straightforward logit fusion mechanism with fixed hyperparameters \(α\\alphaandβ\\beta\)\. We believe the fusion process can be further optimized\. Future research could explore dynamic fusion strategies, such as adaptively adjusting the local head and unigram\-prior weights based on the token\-level entropy or confidence scores of the long\-context logits, allowing for a more context\-aware balance between global semantics and local causality\.

Mitigating Exposure Mismatch in Local Heads\.As observed in our analysis, directly learning the fusion weights \(α\\alphaandβ\\beta\) via cross\-entropy yields suboptimal inference performance compared to using fixed values\. We attribute this to an exposure mismatch \(or teacher\-forcing bias\) during training, where the local head is conditioned on ground\-truth prefixes rather than its own drafted tokens\. Future work could investigate techniques such as scheduled sampling or alignment\-based tuning to bridge this train\-test gap, potentially unlocking the full capability of learnable fusion parameters\.

## References

- J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.\(2023\)Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p1.1)\.
- Z\. An, H\. Bai, Z\. Liu, D\. Li, and E\. Barsoum \(2025\)Pard: accelerating llm inference with low\-cost parallel draft model adaptation\.arXiv preprint arXiv:2504\.18583\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p2.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Ankner, R\. Parthasarathy, A\. Nrusimha, C\. Rinard, J\. Ragan\-Kelley, and W\. Brandon \(2024\)Hydra: sequentially\-dependent draft heads for medusa decoding\.arXiv preprint arXiv:2402\.05109\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p2.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px3.p1.1)\.
- M\. Arriola, A\. Gokaslan, J\. Chiu, Z\. Yang, Z\. Qi, J\. Han, S\. Sahoo, and V\. Kuleshov \(2025\)Block diffusion: interpolating between autoregressive and diffusion language models\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 50726–50753\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le,et al\.\(2021\)Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px1.p1.1)\.
- S\. Bengio, O\. Vinyals, N\. Jaitly, and N\. Shazeer \(2015\)Scheduled sampling for sequence prediction with recurrent neural networks\.Advances in neural information processing systems28\.Cited by:[Appendix C](https://arxiv.org/html/2607.07409#A3.p2.4)\.
- T\. Cai, Y\. Li, Z\. Geng, H\. Peng, J\. D\. Lee, D\. Chen, and T\. Dao \(2024\)Medusa: simple llm inference acceleration framework with multiple decoding heads\.arXiv preprint arXiv:2401\.10774\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p2.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Chen, S\. Borgeaud, G\. Irving, J\. Lespiau, L\. Sifre, and J\. Jumper \(2023\)Accelerating large language model decoding with speculative sampling\.arXiv preprint arXiv:2302\.01318\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p1.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Chen, Y\. Liang, and Z\. Liu \(2026\)DFlash: block diffusion for flash speculative decoding\.arXiv preprint arXiv:2602\.06036\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p2.1),[§1](https://arxiv.org/html/2607.07409#S1.p4.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2607.07409#S3.SS1.p1.2),[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px3.p1.1)\.
- M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. D\. O\. Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman,et al\.\(2021\)Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px1.p1.1)\.
- S\. Cheng, Y\. Bian, D\. Liu, Y\. Jiang, Y\. Liu, L\. Zhang, Q\. Yao, Z\. Tian, W\. Wang, Q\. Guo,et al\.\(2026a\)Sdar: a synergistic diffusion\-autoregression paradigm for scalable sequence generation\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 22058–22075\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1)\.
- X\. Cheng, X\. Yu, C\. Shao, J\. Li, Y\. Xiong, Y\. Qian, J\. Zhu, S\. Ma, X\. Zhang, J\. Ye, Q\. Chen, C\. Deng, J\. Yu, D\. Dai, Z\. Zhang, Y\. Wei, Y\. Tan, W\. Yang, R\. Xu, Y\. Wu, Z\. Xu, X\. Wang, M\. Chen, R\. Tian, X\. Bi, Z\. Hao, S\. Chen, H\. Cao, W\. Zhang, A\. Xu, H\. Zhang, D\. Zhao, and W\. Liang \(2026b\)DSpark: confidence\-scheduled speculative decoding with semi\-autoregressive generation\.External Links:2607\.05147,[Link](https://arxiv.org/abs/2607.05147)Cited by:[Appendix A](https://arxiv.org/html/2607.07409#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2607.07409#S1.p3.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2607.07409#S3.SS2.p1.2),[§5\.3](https://arxiv.org/html/2607.07409#S5.SS3.p1.2),[§5\.5](https://arxiv.org/html/2607.07409#S5.SS5.p1.2)\.
- Y\. Cheng, A\. Zhang, X\. Zhang, C\. Wang, and Y\. Wang \(2024\)Recurrent drafter for fast speculative decoding in large language models\.arXiv preprint arXiv:2403\.09919\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p2.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px1.p1.1)\.
- J\. K\. Christopher, B\. R\. Bartoldson, T\. Ben\-Nun, M\. Cardei, B\. Kailkhura, and F\. Fioretto \(2025\)Speculative diffusion decoding: accelerating language generation through diffusion\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 12042–12059\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px1.p1.1)\.
- J\. Gu, J\. Bradbury, C\. Xiong, V\. O\. Li, and R\. Socher \(2017\)Non\-autoregressive neural machine translation\.arXiv preprint arXiv:1711\.02281\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021\)Measuring mathematical problem solving with the math dataset\.arXiv preprint arXiv:2103\.03874\.Cited by:[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px1.p1.1)\.
- G\. E\. Hinton \(2002\)Training products of experts by minimizing contrastive divergence\.Neural computation14\(8\),pp\. 1771–1800\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p5.1),[§4\.1](https://arxiv.org/html/2607.07409#S4.SS1.p1.3)\.
- L\. Hu, Z\. Feng, Y\. Wu, H\. Yuan, Y\. Zhao, Y\. Qian, B\. Wang, P\. Zhao, D\. Jiang, Y\. Zhu, T\. Rosing, and H\. Zhang \(2026\)JetSpec: breaking the scaling ceiling of speculative decoding with parallel tree drafting\.External Links:2606\.18394,[Link](https://arxiv.org/abs/2606.18394)Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p3.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px3.p1.1)\.
- J\. Huang, Y\. Zhang, Q\. Zhang, H\. Lin, H\. Xu, and L\. Zhang \(2026\)Domino: decoupling causal modeling from autoregressive drafting in speculative decoding\.arXiv preprint arXiv:2605\.29707\.Cited by:[Appendix A](https://arxiv.org/html/2607.07409#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2607.07409#S1.p2.1),[§1](https://arxiv.org/html/2607.07409#S1.p3.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2607.07409#S3.SS2.p1.2),[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px2.p1.1),[§5\.5](https://arxiv.org/html/2607.07409#S5.SS5.p1.2)\.
- N\. Jain, K\. Han, A\. Gu, W\. Li, F\. Yan, T\. Zhang, S\. Wang, A\. Solar\-Lezama, K\. Sen, and I\. Stoica \(2025\)LiveCodeBench: holistic and contamination free evaluation of large language models for code\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=chfJJYC3iL)Cited by:[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px1.p1.1)\.
- Y\. Leviathan, M\. Kalman, and Y\. Matias \(2023\)Fast inference from transformers via speculative decoding\.InInternational Conference on Machine Learning,pp\. 19274–19286\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p1.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px1.p1.1)\.
- G\. Li, Z\. Fu, M\. Fang, Q\. Zhao, M\. Tang, C\. Yuan, and J\. Wang \(2026a\)Diffuspec: unlocking diffusion language models for speculative decoding\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 20896–20910\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Li, F\. Wei, C\. Zhang, and H\. Zhang \(2024a\)Eagle\-2: faster inference of language models with dynamic draft trees\.InProceedings of the 2024 conference on empirical methods in natural language processing,pp\. 7421–7432\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p2.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Li, F\. Wei, C\. Zhang, and H\. Zhang \(2024b\)Eagle: speculative sampling requires rethinking feature uncertainty\.arXiv preprint arXiv:2401\.15077\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p2.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Li, F\. Wei, C\. Zhang, and H\. Zhang \(2026b\)Eagle\-3: scaling up inference acceleration of large language models via training\-time test\.Advances in Neural Information Processing Systems38,pp\. 136737–136756\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p2.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px3.p1.1)\.
- A\. Liu, B\. Feng, B\. Xue, B\. Wang, B\. Wu, C\. Lu, C\. Zhao, C\. Deng, C\. Zhang, C\. Ruan,et al\.\(2024\)Deepseek\-v3 technical report\.arXiv preprint arXiv:2412\.19437\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p1.1)\.
- F\. Liu, X\. Li, K\. Zhao, Y\. Gao, Z\. Zhou, Z\. Zhang, Z\. Wang, W\. Dou, S\. Zhong, and C\. Tian \(2026\)DART: diffusion\-inspired speculative decoding for fast llm inference\.arXiv preprint arXiv:2601\.19278\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p2.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px3.p1.1)\.
- J\. Liu, X\. Dong, Z\. Ye, R\. Mehta, Y\. Fu, V\. Singh, J\. Kautz, C\. Zhang, and P\. Molchanov \(2025\)Tidar: think in diffusion, talk in autoregression\.arXiv preprint arXiv:2511\.08923\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1)\.
- X\. Miao, G\. Oliaro, Z\. Zhang, X\. Cheng, Z\. Wang, Z\. Zhang, R\. Y\. Y\. Wong, A\. Zhu, L\. Yang, X\. Shi,et al\.\(2024\)Specinfer: accelerating large language model serving with tree\-based speculative inference and verification\.InProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3,pp\. 932–949\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. Li \(2026\)Large language diffusion models\.Advances in Neural Information Processing Systems38,pp\. 50608–50646\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1)\.
- P\. Rheinboldt, F\. Berdoz, and R\. Wattenhofer \(2026\)TreeFlash: parallel ar\-approximation for faster speculative decoding\.arXiv preprint arXiv:2606\.03819\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p2.1),[§1](https://arxiv.org/html/2607.07409#S1.p3.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px3.p1.1)\.
- L\. Ringel and Y\. Romano \(2026\)Accelerating speculative decoding with block diffusion draft trees\.arXiv preprint arXiv:2604\.12989\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Sandler, J\. K\. Christopher, T\. Hartvigsen, and F\. Fioretto \(2025\)Specdiff\-2: scaling diffusion drafter alignment for faster speculative decoding\.arXiv preprint arXiv:2511\.00606\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1)\.
- M\. Stern, N\. Shazeer, and J\. Uszkoreit \(2018\)Blockwise parallel decoding for deep autoregressive models\.Advances in Neural Information Processing Systems31\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Sun, Z\. Li, H\. Wang, D\. He, Z\. Lin, and Z\. Deng \(2019\)Fast structured decoding for sequence models\.Advances in Neural Information Processing Systems32\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px3.p1.1)\.
- R\. Taori, I\. Gulrajani, T\. Zhang, Y\. Dubois, X\. Li, C\. Guestrin, P\. Liang, and T\. B\. Hashimoto \(2023\)Alpaca: a strong, replicable instruction\-following model\.Stanford Center for Research on Foundation Models\. https://crfm\. stanford\. edu/2023/03/13/alpaca\. html3\(6\),pp\. 7\.Cited by:[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px1.p1.1)\.
- P\. Tillet, H\. Kung, and D\. Cox \(2019\)Triton: an intermediate language and compiler for tiled neural network computations\.InProceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages,pp\. 10–19\.Cited by:[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px4.p1.1)\.
- Z\. Wen, S\. Gui, and Y\. Feng \(2024\)Speculative decoding with ctc\-based draft model for llm inference acceleration\.Advances in Neural Information Processing Systems37,pp\. 92082–92100\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px3.p1.1)\.
- T\. Wolf, L\. Debut, V\. Sanh, J\. Chaumond, C\. Delangue, A\. Moi, P\. Cistac, T\. Rault, R\. Louf, M\. Funtowicz,et al\.\(2019\)Huggingface’s transformers: state\-of\-the\-art natural language processing\.arXiv preprint arXiv:1910\.03771\.Cited by:[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px4.p1.1)\.
- T\. Wu, Y\. Yao, Z\. Qi, H\. Zheng, Z\. Wang, H\. Ma, L\. Liao, H\. Lakkaraju, J\. Li, and Y\. Du \(2026\)D\-pace: dynamic position\-aware cross\-entropy for parallel speculative drafting\.arXiv preprint arXiv:2605\.18810\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§1](https://arxiv.org/html/2607.07409#S1.p1.1),[§1](https://arxiv.org/html/2607.07409#S1.p6.1),[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px1.p1.1)\.
- Y\. Zhang and T\. Math\-AI \(2025\)American invitational mathematics examination \(aime\) 2025\.Cited by:[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px1.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing,et al\.\(2023\)Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.Advances in neural information processing systems36,pp\. 46595–46623\.Cited by:[§5\.1](https://arxiv.org/html/2607.07409#S5.SS1.SSS0.Px1.p1.1)\.
- Y\. Zhou, K\. Lyu, A\. S\. Rawat, A\. K\. Menon, A\. Rostamizadeh, S\. Kumar, J\. Kagy, and R\. Agarwal \(2024\)Distillspec: improving speculative decoding via knowledge distillation\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 32011–32050\.Cited by:[§2](https://arxiv.org/html/2607.07409#S2.SS0.SSS0.Px1.p1.1)\.

## Appendix AImplementation Details

#### Local Head Implementation\.

We instantiatepS​\(xi∣zi\)p\_\{S\}\(x\_\{i\}\\mid z\_\{i\}\)with a lightweight local head over the local draft prefix\. Since the local head is evaluated autoregressively during inference, we use a low\-rank bottleneck to reduce the sequential computation cost, following the design choice in Domino and DSpark\(Huanget al\.,[2026](https://arxiv.org/html/2607.07409#bib.bib28); Chenget al\.,[2026b](https://arxiv.org/html/2607.07409#bib.bib1)\)\. For the RNN variant, token ids are mapped by the frozen target embedding table and passed through a one\-layer bias\-free GRU; the final hidden state is projected to a low\-rank state with SiLU activation and then mapped to vocabulary logits:

ri=SiLU​\(Wrank​GRU​\(Etgt​\(zi\)\)\),ℓSRNN​\(xi∣zi\)=Wvocab​ri\.r\_\{i\}=\\mathrm\{SiLU\}\\\!\\left\(W\_\{\\mathrm\{rank\}\}\\mathrm\{GRU\}\(E\_\{\\mathrm\{tgt\}\}\(z\_\{i\}\)\)\\right\),\\qquad\\ell\_\{S\}^\{\\mathrm\{RNN\}\}\(x\_\{i\}\\mid z\_\{i\}\)=W\_\{\\mathrm\{vocab\}\}r\_\{i\}\.For the Markov variant, we directly look up a low\-rank state from the most recent token and multiply it with a low\-rank vocabulary head:

ri=EM​\(xi−1\),ℓSMarkov​\(xi∣zi\)=ri​WM⊤,r\_\{i\}=E\_\{M\}\(x\_\{i\-1\}\),\\qquad\\ell\_\{S\}^\{\\mathrm\{Markov\}\}\(x\_\{i\}\\mid z\_\{i\}\)=r\_\{i\}W\_\{M\}^\{\\top\},whereEM,WM∈ℝ\|𝒱\|×drE\_\{M\},W\_\{M\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\\times d\_\{r\}\}\. Both variants output full\-vocabulary logits and are trained with the same next\-token prediction objective as the short\-context model\. The hyperparameters for the local head are summarized in Table[6](https://arxiv.org/html/2607.07409#A1.T6)\.

#### Unigram Prior Estimation\.

We construct a fixed unigram log\-prior over target tokens using the same tokenizer and chat\-template preprocessing as training\. For each training sequence, we count only tokens whose loss mask equals one, i\.e\., assistant\-response tokens that contribute to the supervised objective\. Letc​\(v\)c\(v\)denote the resulting count for vocabulary itemvv\. We form a smoothed prior

p~0​\(v\)=c​\(v\)\+λ∑v′\(c​\(v′\)\+λ\),\\tilde\{p\}\_\{0\}\(v\)=\\frac\{c\(v\)\+\\lambda\}\{\\sum\_\{v^\{\\prime\}\}\(c\(v^\{\\prime\}\)\+\\lambda\)\},where we use additive smoothing withλ=1\.0\\lambda=1\.0by default\. We then useb0​\(v\)=log⁡p~0​\(v\)b\_\{0\}\(v\)=\\log\\tilde\{p\}\_\{0\}\(v\)as a fixed vocabulary\-level bias\.

Table 6:Training hyperparameters for the local head\.

## Appendix BProof of the Product\-of\-Experts Factorization

In Section[4\.1](https://arxiv.org/html/2607.07409#S4.SS1), we introduce the exact factorization of the conditional probabilityp​\(xi∣y,zi\)p\(x\_\{i\}\\mid y,z\_\{i\}\)\. Here, we provide the step\-by\-step derivation for this decomposition\.

Starting from Bayes’ theorem, the conditional probability of the candidate tokenxix\_\{i\}given the long contextyyand the short contextziz\_\{i\}can be written as:

p​\(xi∣y,zi\)=p​\(y,zi∣xi\)​p​\(xi\)p​\(y,zi\)\.p\(x\_\{i\}\\mid y,z\_\{i\}\)=\\frac\{p\(y,z\_\{i\}\\mid x\_\{i\}\)\\,p\(x\_\{i\}\)\}\{p\(y,z\_\{i\}\)\}\.\(11\)
We can multiply both the numerator and the denominator by the product of the marginalsp​\(y\)​p​\(zi\)p\(y\)\\,p\(z\_\{i\}\)and the conditional probabilitiesp​\(y∣xi\)​p​\(zi∣xi\)p\(y\\mid x\_\{i\}\)\\,p\(z\_\{i\}\\mid x\_\{i\}\):

p​\(xi∣y,zi\)=p​\(y,zi∣xi\)​p​\(xi\)p​\(y,zi\)⋅p​\(y∣xi\)​p​\(zi∣xi\)p​\(y∣xi\)​p​\(zi∣xi\)⋅p​\(y\)​p​\(zi\)p​\(y\)​p​\(zi\)\.p\(x\_\{i\}\\mid y,z\_\{i\}\)=\\frac\{p\(y,z\_\{i\}\\mid x\_\{i\}\)\\,p\(x\_\{i\}\)\}\{p\(y,z\_\{i\}\)\}\\cdot\\frac\{p\(y\\mid x\_\{i\}\)\\,p\(z\_\{i\}\\mid x\_\{i\}\)\}\{p\(y\\mid x\_\{i\}\)\\,p\(z\_\{i\}\\mid x\_\{i\}\)\}\\cdot\\frac\{p\(y\)\\,p\(z\_\{i\}\)\}\{p\(y\)\\,p\(z\_\{i\}\)\}\.\(12\)
By rearranging the terms, we group them into three distinct factors:

p​\(xi∣y,zi\)=\[p​\(y∣xi\)​p​\(zi∣xi\)​p​\(xi\)p​\(y\)​p​\(zi\)\]⋅p​\(y\)​p​\(zi\)p​\(y,zi\)⋅p​\(y,zi∣xi\)p​\(y∣xi\)​p​\(zi∣xi\)\.p\(x\_\{i\}\\mid y,z\_\{i\}\)=\\left\[\\frac\{p\(y\\mid x\_\{i\}\)\\,p\(z\_\{i\}\\mid x\_\{i\}\)\\,p\(x\_\{i\}\)\}\{p\(y\)\\,p\(z\_\{i\}\)\}\\right\]\\cdot\\frac\{p\(y\)\\,p\(z\_\{i\}\)\}\{p\(y,z\_\{i\}\)\}\\cdot\\frac\{p\(y,z\_\{i\}\\mid x\_\{i\}\)\}\{p\(y\\mid x\_\{i\}\)\\,p\(z\_\{i\}\\mid x\_\{i\}\)\}\.\(13\)
Recall that by applying Bayes’ theorem to the individual contexts, we can express the conditional probabilities ofxix\_\{i\}as:

p​\(xi∣y\)=p​\(y∣xi\)​p​\(xi\)p​\(y\)andp​\(xi∣zi\)=p​\(zi∣xi\)​p​\(xi\)p​\(zi\)\.p\(x\_\{i\}\\mid y\)=\\frac\{p\(y\\mid x\_\{i\}\)\\,p\(x\_\{i\}\)\}\{p\(y\)\}\\quad\\text\{and\}\\quad p\(x\_\{i\}\\mid z\_\{i\}\)=\\frac\{p\(z\_\{i\}\\mid x\_\{i\}\)\\,p\(x\_\{i\}\)\}\{p\(z\_\{i\}\)\}\.\(14\)
Multiplying these two equations together and dividing by the unigram priorp​\(xi\)p\(x\_\{i\}\)yields the exact expression inside the square brackets of Equation[13](https://arxiv.org/html/2607.07409#A2.E13):

p​\(xi∣y\)​p​\(xi∣zi\)p​\(xi\)=p​\(y∣xi\)​p​\(zi∣xi\)​p​\(xi\)p​\(y\)​p​\(zi\)\.\\frac\{p\(x\_\{i\}\\mid y\)\\,p\(x\_\{i\}\\mid z\_\{i\}\)\}\{p\(x\_\{i\}\)\}=\\frac\{p\(y\\mid x\_\{i\}\)\\,p\(z\_\{i\}\\mid x\_\{i\}\)\\,p\(x\_\{i\}\)\}\{p\(y\)\\,p\(z\_\{i\}\)\}\.\(15\)
Substituting this equivalence back into Equation[13](https://arxiv.org/html/2607.07409#A2.E13)gives the final exact equality:

p​\(xi∣y,zi\)\\displaystyle p\(x\_\{i\}\\mid y,z\_\{i\}\)=p​\(xi∣y\)​p​\(xi∣zi\)p​\(xi\)⋅p​\(y\)​p​\(zi\)p​\(y,zi\)⋅p​\(y,zi∣xi\)p​\(y∣xi\)​p​\(zi∣xi\)\.\\displaystyle=\\frac\{p\(x\_\{i\}\\mid y\)\\,p\(x\_\{i\}\\mid z\_\{i\}\)\}\{p\(x\_\{i\}\)\}\\cdot\\frac\{p\(y\)\\,p\(z\_\{i\}\)\}\{p\(y,z\_\{i\}\)\}\\cdot\\frac\{p\(y,z\_\{i\}\\mid x\_\{i\}\)\}\{p\(y\\mid x\_\{i\}\)\\,p\(z\_\{i\}\\mid x\_\{i\}\)\}\.\(16\)This completes the proof\.

## Appendix CAnalysis of Learnableα\\alphaandβ\\beta

Table[7](https://arxiv.org/html/2607.07409#A3.T7)reports the learned values ofα\\alphaandβ\\betaat different draft\-token positions\. We observe that both weights become relatively large at later positions\. For example,α\\alphaincreases from0\.5470\.547at position22to around0\.7620\.762after position1212, whileβ\\betaalso increases from0\.5120\.512to around0\.6130\.613\. This trend suggests that the learned fusion weights assign increasingly higher importance to the local head and the unigram\-prior correction for later draft tokens\.

A possible explanation is the mismatch introduced by teacher forcing during training\(Bengioet al\.,[2015](https://arxiv.org/html/2607.07409#bib.bib47)\)\. When optimizingα\\alphaandβ\\betaon the training data, later positions are conditioned on longer ground\-truth prefixes\. Under this setting, the local head can exploit more accurate short\-context information and may appear more reliable than DFlash for predicting later draft tokens\. As a result, the optimization tends to assign largerα\\alphaandβ\\betavalues to later positions\.

However, this advantage may not fully transfer to inference\. During speculative decoding, later draft tokens are conditioned on previously drafted tokens rather than ground\-truth tokens, which introduces exposure bias\. Therefore, overly large learned weights at later positions may overestimate the reliability of the local correction and lead to suboptimal acceptance length\. This explains why the learnable variant in Table[4](https://arxiv.org/html/2607.07409#S5.T4)improves over the DFlash baseline but still underperforms the fixed settingα=β=0\.3\\alpha=\\beta=0\.3\. The fixed setting is less sensitive to the teacher\-forcing bias and provides a more robust choice for inference\.

Table 7:Learnableα\\alphaandβ\\betavalues by position\.
## Appendix DComparison of Acceptance Rate Results

Table[8](https://arxiv.org/html/2607.07409#A4.T8)provides a position\-wise view of the acceptance behavior on Qwen3\-4B at temperature0\. Unlike the average acceptance lengthτ\\taureported in Table[1](https://arxiv.org/html/2607.07409#S5.T1), the quantityr​a​t​ei=P​\(acceptance​length≥i\)rate\_\{i\}=P\(\\mathrm\{acceptance\\ length\}\\geq i\)measures the probability that verification accepts at least the firstiidraft tokens\. It therefore describes the tail of the accepted draft length distribution: improvements at larger positions indicate that the method more often preserves longer consecutive draft spans, which is the direct source of higher average acceptance length and decoding speedup\.

The results show that DeLS\-Spec improves the acceptance curve over DFlash across almost all nontrivial positions\. The first position is accepted by both methods by definition, and the second position is nearly unchanged \(81\.45%81\.45\\%versus81\.63%81\.63\\%\)\. From position33onward, however, DeLS\-Spec consistently achieves higher acceptance rates\. The gains are largest in the middle of the draft block, reaching about2\.62\.6percentage points around positions55–66, and remain positive even at the end of the block, with a1\.401\.40percentage\-point improvement at position1616\. This pattern supports the main hypothesis of DeLS\-Spec: the local head and unigram\-prior correction do not merely improve isolated early predictions, but make the draft sequence locally more coherent so that verification can accept longer prefixes\.

Table 8:Acceptance\-rate comparison by position on Qwen3\-4B at temperature 0, wherer​a​t​ei=P​\(acceptance​length≥i\)rate\_\{i\}=P\(\\mathrm\{acceptance\\ length\}\\geq i\)\.