当并行起草器遇上并行推测解码

# When Parallel Drafter Meets Parallel Speculative Decoding
Source: [https://arxiv.org/html/2609.27396](https://arxiv.org/html/2609.27396)
Fuliang LiuAffiliation:State Key Laboratory of Novel Software Technology, Nanjing UniversityAffiliation:Alibaba GroupKun QianAffiliation:Alibaba GroupZhibin Wang††thanks:Corresponding author: Zhibin Wang \(wzbwangzhibin@gmail\.com\.\)Affiliation:State Key Laboratory of Novel Software Technology, Nanjing UniversityWanchun DouAffiliation:State Key Laboratory of Novel Software Technology, Nanjing UniversityWenyuan YuAffiliation:Alibaba GroupChen TianAffiliation:State Key Laboratory of Novel Software Technology, Nanjing University

###### Abstract

DSpark\-style parallel drafters have made speculative decoding highly effective, yet their draft phase remains serialized on the critical path of every round\. Parallel speculative decoding \(PSD\) overlaps drafting with verification, yet existing methods must guess the accepted prefix and bonus token in advance: a wrong guess reverts the whole batch to serial drafting\. We present DPara, a PSD framework that reuses effective parallel drafters yet guarantees backbone–verification overlap in every round, thereby eliminating this probabilistic fallback altogether\. While the target verifies, DPara’s diffusion backbone precomputes draft representations for*every*acceptance boundary with the bonus left unspecified; a lightweight autoregressive head then combines the revealed verification outcome with the matching precomputed representation to emit the next round’s draft tokens almost instantly—fully parallelizing the dominant backbone forward with verification and leaving only the negligible head cost serial\. Experiments on Qwen3\-8B and Qwen3\-14B across seven math, coding, and chat benchmarks show that DPara achieves average speedups of3\.21×3\.21\\timesand3\.52×3\.52\\timesover autoregressive decoding, surpassing the strongest serial and parallel speculative decoding baselines alike\.

Figure 1:Performance of DPara and other baselines on Qwen3\-14B\.Left: throughput–latency frontier, averaging over the four benchmarks shown on the right; each curve traces batch sizesb=1,2,4,8,16b\{=\}1,2,4,8,16, with batch labels on DPara\.Right: end\-to\-end speedup over AR atb=1b\{=\}1; DPara reaches an arithmetic mean of3\.62×3\.62\\timesacross these four benchmarks, exceeding DSpark on each\.## 1Introduction

Speculative decoding \(SD\) accelerates large language model inference effectively while preserving the target distribution\([Leviathan et al\., 2023](https://arxiv.org/html/2609.27396#bib.bib5);[Chen et al\., 2023](https://arxiv.org/html/2609.27396#bib.bib6)\)\. Its end\-to\-end efficiency is governed by two quantities:*average acceptance length*and*drafting latency*on the critical path\.

*Diffusion\-inspired parallel drafters*, like DART\([Liu et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib1)\)and DFlash\([Chen et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib2)\), predict logits for multiple future positions in a single forward pass and permit greater model capacity under the same latency budget\. Parallel logit prediction alone, however, does not capture dependencies among the sampled draft tokens\. Subsequent designs such as DSpark\([Cheng et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib4)\)and Domino\([Huang et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib3)\)address this limitation by coupling the parallel diffusion backbone with a lightweight autoregressive head, which restores sequence consistency\. Yet even under these advanced drafters, the conventional sequential draft–verify schedule keeps drafting on the critical path, where it accounts for24%24\\%–32%32\\%of end\-to\-end latency \(Appendix[A](https://arxiv.org/html/2609.27396#A1)\)\.

Figure 2:SSD drafts and verifies a batch of requests together\. When any request’s predicted bonus token mismatches \(red\), the whole batch loses its precomputed continuation and reverts to serial drafting\. As the batch grows, the chance of at least one mismatch rises, so larger batches fall back more often and forfeit the parallel draft–verify gain\.*Parallel speculative decoding*\(PSD\) attacks this serialization directly by overlapping draft generation with target verification\([Liu et al\., 2025](https://arxiv.org/html/2609.27396#bib.bib7);[McDanel, 2024](https://arxiv.org/html/2609.27396#bib.bib9);[Shen et al\., 2025](https://arxiv.org/html/2609.27396#bib.bib10);[Kumar et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib8)\)\. On behalf of these methods, SSD\([Kumar et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib8)\)extends speculation to verification itself: while the target verifies the current step, the drafter predicts the accepted prefix and*bonus token*and generates subsequent draft tokens in advance\. A matching outcome makes the next step’s draft tokens immediately available, eliminating drafting latency from the critical path\. A mismatch, however, reverts SSD to serial speculative decoding \(Figure[2](https://arxiv.org/html/2609.27396#S1.F2)\), and because the probability of at least one mismatch grows with batch size, the entire batch is forced back to serial execution ever more frequently—at batch size1616, up to96%96\\%of steps fall back \(Appendix[B](https://arxiv.org/html/2609.27396#A2)\)\. Outcome\-dependent overlap therefore exchanges a best\-case gain for a regression that intensifies with serving load\. This tension raises a natural question:

*Can wecompletely eliminate probable serial fallbackfrom parallel speculative decoding?*

We answer this question affirmatively by severing the probabilistic dependence on bonus\-token prediction: the key is to separate the computation that must await the verification outcome from the computation that can proceed independently of it\. The DSpark\-style architecture affords precisely this separation—the expensive diffusion backbone computes draft representations while verification is still in flight, and the lightweight autoregressive head incorporates the actual bonus token afterward\. Deferring bonus\-token conditioning to the head yieldsbackbone–verification overlap in every round, without any prediction\-induced fallback\.

We present DPara, a PSD framework that makes this overlap independent of the verification outcome\. DPara prepares diffusion backbone prediction outputs for*every*possible accepted prefix boundary and leaves the bonus token unspecified until verification completes\. Because this guarantee holds for each request individually, enlarging the batch cannot reintroduce the prediction\-induced serial fallback\. Covering all acceptance boundaries in a single pass, however, requires a backbone that predicts from multiple feature\-less anchors—a regime the standard DFlash model does not satisfy\. We close this gap by finetuning: M\-DFlash initializes from the published DSpark checkpoint and adapts*only the diffusion backbone*to the multiple\-anchor, feature\-less regime, introducing no new trainable parameters and keeping the AR head frozen as published\. Provisioned to complete within the verification window, the diffusion backbone—the dominant component of drafting latency—is fully hidden, leaving only the lightweight autoregressive head on the critical path\. DPara thereby preserves the effectiveness of parallel drafters while excising backbone execution entirely from the serial draft–verify schedule\.

Figure[1](https://arxiv.org/html/2609.27396#S0.F1)previews the empirical gains\. Across math, coding, and chat benchmarks on Qwen3\-8B and Qwen3\-14B, DPara exceeds other baselines on every model–dataset pair, averaging3\.21×3\.21\\timesand3\.52×3\.52\\timesspeedup over autoregressive decoding \(§[4\.2](https://arxiv.org/html/2609.27396#S4.SS2)\)\. Against DSpark, the strongest serial drafter, this corresponds to average improvements of11\.1%11\.1\\%and5\.1%5\.1\\%; against the parallel baselines PEARL and SSD on Qwen3\-14B, the average speedup rises by31%31\\%and64%64\\%\. The advantage does not erode under batching: DPara stays ahead of DSpark at every batch size from11to1616and retains over2×2\\timesAR throughput at batch1616, while outcome\-dependent methods such as SSD suffer a markedly steeper throughput drop\-off as batches grow \(Figure[1](https://arxiv.org/html/2609.27396#S0.F1)\)\. The gains likewise survive a leaner drafting budget: DPara\-A10 retains98%98\\%of DPara’s speedup while cutting the drafting\-device cost by∼90%\{\\sim\}90\\%, whereas DPara\-single retains90%90\\%–95%95\\%at small batches \(§[4\.3](https://arxiv.org/html/2609.27396#S4.SS3)\)\. Ablations further confirm that finetuning only the diffusion backbone raises the average acceptance length by30\.7%30\.7\\%\.

This paper makes the following contributions:

- •Thefirstparallel speculative decoding framework powered by DSpark\-style parallel drafters, establishingstate\-of\-the\-artinference performance across both serial and parallel baselines\.
- •DPara, a PSD framework that covers all acceptance boundarieswithout any serial backbone fallbackand hides the backbone latency entirely within the verification window, enabled by M\-DFlash, a multiple\-anchor backbone finetuned from a published DSpark checkpoint\.
- •An extensive evaluation on Qwen3\-8B and Qwen3\-14B: DPara exceeds the best serial and parallel baselines on every model–dataset pair \(up to4\.53×4\.53\\times, averaging3\.21×3\.21\\times/3\.52×3\.52\\timesover AR\) and sustains its advantage from batch11to1616; further explorations of heterogeneous drafting and single\-GPU colocation retain most of the speedup under a reduced drafting budget\.

## 2Background and Related Work

Figure 3:Scheduling of drafting and verification across speculative decoding paradigms\.\(a\) Serial SD: autoregressive \(EAGLE3\) or parallel drafters \(DFlash/DSpark\) run draft–verify in sequence, leaving drafting on the critical path\.\(b\) SSD: overlaps drafting with verification, but a mispredicted bonus token forces serial fallback\.\(c\) DPara \(ours\): the backbone overlaps verification for every acceptance outcome while drafting stays parallel\.### 2\.1Speculative Decoding

Speculative decoding\([Leviathan et al\., 2023](https://arxiv.org/html/2609.27396#bib.bib5);[Chen et al\., 2023](https://arxiv.org/html/2609.27396#bib.bib6)\)alleviates memory\-bound decoding by amortizing the weight reads of a large target modelℳt\\mathcal\{M\}\_\{t\}across multiple tokens\. Given a prefixCC, a lightweight drafterℳd\\mathcal\{M\}\_\{d\}proposes a blocky~1:K∼q\(⋅∣C\)\\tilde\{y\}\_\{1:K\}\\sim q\(\\cdot\\mid C\), whichℳt\\mathcal\{M\}\_\{t\}scores in one forward pass\. Distribution\-preserving rejection sampling retains an accepted prefix and appends a*bonus token*bb: a correction sampled at the first rejection, or an additional target sample after full acceptance\.

### 2\.2Parallel Drafter

To overcome the sequential drafting overhead of autoregressive drafters such as EAGLE3\([Li et al\., 2025](https://arxiv.org/html/2609.27396#bib.bib11)\), DART\([Liu et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib1)\)and DFlash\([Chen et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib2)\)draw inspiration from diffusion language models \(dLLMs\) and predict a block of future positions in one forward pass, substantially reducing draft\-generation cost\. Independent prediction at different positions, however, can produce incoherent draft sequences and declining acceptance length \(τ\\tau\)\. DART applies*N\-gram*corrections to its parallel logits, whereas Domino\([Huang et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib3)\)and DSpark\([Cheng et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib4)\)learn a lightweight autoregressive head that conditions each position on the preceding sampled tokens, which preserves the efficient parallel backbone while restoring local causal dependence, leading to longerτ\\tau\. Nevertheless, the drafter and verifier remain sequentially coupled across decoding steps:the drafter must finish before verification can begin, leaving drafting on the critical path and making it a nontrivial component of end\-to\-end latency\.

### 2\.3Parallel speculative decoding

Figure[3](https://arxiv.org/html/2609.27396#S2.F3)contrasts the execution schedules of serial SD, SSD, and DPara\. PSD overlaps next\-step drafting with current\-step verification to hide the drafting interval that otherwise delays the next verification\. The drafter must therefore generate tentative continuations before the accepted prefix and*bonus token*are known\. PEARL\([Liu et al\., 2025](https://arxiv.org/html/2609.27396#bib.bib7)\)and AMUSD\([McDanel, 2024](https://arxiv.org/html/2609.27396#bib.bib9)\)generate continuations during verification under an all\-accepted assumption, which provides only limited opportunities for parallelism\. SpecBranch\([Shen et al\., 2025](https://arxiv.org/html/2609.27396#bib.bib10)\)and SSD\([Kumar et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib8)\)predict likely rejection positions and bonus tokens to reduce rollback\. In all these methods, an unanticipated verification outcome can invalidate the precomputed continuation and fall back to*serial speculation*,so none of them guarantees draft–verify parallelism without fallback in every round\.

## 3DPara Design

In this section, we will go through the design of DPara to guarantee complete backbone–verification overlap in every PSD step\.

Figure 4:One decoding step of DPara\. WithAAas the bonus token from last step, the target verifiesA​B​C​DABCDwhile the parallel backbone concurrently precomputesddfuture logits for every acceptance boundary\. AcceptingB​CBCand rejectingDDyields bonus tokenD∗D^\{\*\}\. The lightweight AR head combinesD∗D^\{\*\}with the logits followingCCto generate next step’s draft tokensE​F​GEFGquickly\.### 3\.1Backbone requirement of DPara

DPara places a requirement on the*diffusion backbone*that the standard DFlash model does not satisfy\. Under the conventional serial schedule, drafting runs*after*verification: the target has already finished verification, so a fresh target feature exists at every position before bonus token—*the bonus itself carries no feature of its own*\. DFlash\([Chen et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib2)\)anchors on this*single*token, appended to the fully featured prefix, to predict the logits of the nextddpositions in one backbone forward\. DPara instead launches the backbone*while*the target is still verifying the current block \(Figure[4](https://arxiv.org/html/2609.27396#S3.F4)\)\. Write the draft tokens from last step asA,B,C,DA,B,C,D, whereAAis the previous bonus andB,C,DB,C,Dare the drafts being scored, and letr∈\{0,…,d\}r\\in\\\{0,\\dots,d\\\}be the number of accepted drafts andbbthe new bonus\. At launch time the backbone knows neitherrrnorbb; all it holds are the tokens carried over from the previous round, of which the accepted prefixA,t1​…,trA,t\_\{1\}\\dots,t\_\{r\}formsk=r\+1\(≥1\)k=r\+1\\ \(\\geq 1\)anchor tokens that have*not*yet passed through the target and therefore carry*no*target features\. This requires:

The DPara’s backbone must predict the nextddpositions frommultiple, feature\-less anchors, and must do so for every possiblerrat once, since the true acceptance length is not revealed until verification finishes\.

This requirement, however, does not change*what*the backbone computes; it*generalizes*it\. DFlash already predicts future logits from*a single anchor*given the fully featured prefix; DPara only asks it to extend this*from a single anchor to multiple anchors*\. Because multiple\-anchor prediction is a strict extension of DFlash’s single\-anchor semantics rather than a different function, we can obtain the backbone DPara needs by finetuning DFlash \(Figure[5](https://arxiv.org/html/2609.27396#S3.F5)\)\. We call this multiple\-anchor diffusion backboneM\-DFlash\(Multiple\-anchor\-token DFlash\)\.

### 3\.2Finetuning DFlash to M\-DFlash

M\-DFlash introduces*no new trainable network parameters*\. We initialize it from the official DSpark checkpoint\([Cheng et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib4)\)—a DFlash\-family drafter that couples the block\-diffusion backbone with a lightweight Markov autoregressive \(AR\) head—and adapt*only the backbone part*to the multiple\-anchor, feature\-less regime above\. The AR head is precisely the component that lets DPara defer bonus conditioning to the post\-verification stage, so we keep it frozen as published\.

Algorithm 1One steady\-state round of DPara \(draft lengthdd\)\.1:feature cache

CC; feature\-less draft block

t0:dt\_\{0:d\}\(

t0t\_\{0\}= previous bonus\)

2:accepted tokens, bonus

bb; updated

CCand next block

3:// launched together, run concurrently

4:Precompute\(drafter\): M\-DFlash on spine

t0:dt\_\{0:d\}, cache

CC
5:

→\\rightarrowbase logits

ℓ\(r\)1:d\\ell^\{\(r\)\}\_\{1:d\}for all

r∈\{0:d\}r\\in\\\{0\{:\}d\\\}
6:Verify\(target\):

ℳt\\mathcal\{M\}\_\{t\}forwards

t0:dt\_\{0:d\}
7:

→\\rightarrowaccepted length

rr, bonus

bb, features of

t0:rt\_\{0:r\}
8:// barrier:\(r,b\)\(r,b\)now known

9:select precomputed branch

rr⊳\\trianglerightenumerated⇒\\Rightarrowno fallback

10:

y0←by\_\{0\}\\leftarrow b
11:for

i=1,…,di=1,\\dots,ddo⊳\\trianglerightAR head: lightweight cost

12:

zi←ℓi\(r\)\+W​E​\(yi−1\)z\_\{i\}\\leftarrow\\ell^\{\(r\)\}\_\{i\}\+WE\(y\_\{i\-1\}\);

yi∼softmax⁡\(zi\)y\_\{i\}\\sim\\mathrm\{softmax\}\(z\_\{i\}\)
13:endfor

14:

C←\[C,C0:rn​e​w\]C\\leftarrow\[\\,C,C^\{new\}\_\{0:r\}\\,\]⊳\\trianglerightdrop rejectedtr\+1:dt\_\{r\+1:d\}

15:

t0:d←\(b,y1:d\)t\_\{0:d\}\\leftarrow\(b,\\,y\_\{1:d\}\)⊳\\trianglerightbb= next feature\-less anchor

16:returnaccepted

t1:rt\_\{1:r\}, bonus

bb⊳\\trianglerightstop at EOS / length budget

Trainable and frozen parameters\.We train the backbone blocks \(attention, MLP, and their normalizations\), the projection that fuses the concatenated multi\-layer target features, and the input and output normalizations\. We freeze the entire targetℳt\\mathcal\{M\}\_\{t\}, which only supplies online features and supervision, together with the drafter’s token embedding, its full\-vocabulary LM projection, and both the AR head and the confidence head\. Only the backbone faces a changed input regime—*multiple feature\-less anchors instead of a single one*—so it is the only part that must adapt\.

Figure 5:Input regimes of DFlash and M\-DFlash\.Top: DFlash anchors on a single forwarded token appended to a fully featured prefix\.Bottom: M\-DFlash predicts the nextddlogits from multiple \(1∼d\+11\{\\sim\}d\{\+\}1\) feature\-less anchor tokens for every acceptance boundary in one forward pass—the regime required to launch the backbone during verification\.Shared causal spine with isolated bidirectional branches\.M\-DFlash covers alld\+1d\+1acceptance boundaries in one backbone forward using the attention pattern in Figure[6](https://arxiv.org/html/2609.27396#S3.F6)\. In the illustrated example \(d=3d=3\), the previous bonusAAand draft tokensB,C,DB,C,Dform a shared, feature\-less causal spine\. Each anchor attends to the cached target features, itself, and earlier anchors, but never to later anchors or mask tokens\. The four groups of three masks correspond to the possible accepted prefixesAA,A​BAB,A​B​CABC, andA​B​C​DABCD\. Within each group, the masks attend bidirectionally to one another, read the cached target features, and access only the corresponding anchor prefix; all other branches remain invisible\. For example, the branch followingCCreadsA,B,CA,B,Cbut notDD, so its predictions remain valid if verification acceptsB,CB,Cand rejectsDD\. The triangular light blue region and diagonal medium blue blocks therefore encode two complementary constraints: causal sharing along the spine and bidirectional attention within isolated branches\. Unlike an ordinary triangular causal mask, this tree\-structured mask computes the shared anchors once while producingddfuture logits for every possible acceptance boundary\.

Loss\.For a selected boundary, letℓi\\ell\_\{i\}be the base logits at mask positioniiand let the frozen AR head add a correction from the previously sampled token,

zi=ℓi\+WE\(yi−1\),y0=b,i=1,…,d,z\_\{i\}=\\ell\_\{i\}\+WE\(y\_\{i\-1\}\),\\qquad y\_\{0\}=b,\\quad i=1,\\dots,d,whereEEis the head’s independent low\-rank token embedding andWWprojects it to the vocabulary; the head reads only the preceding token\. Writingqi=softmax⁡\(zi\)q\_\{i\}=\\mathrm\{softmax\}\(z\_\{i\}\)for the student andpip\_\{i\}for the target distribution at the same position, we minimize a position\-weighted mixture of cross\-entropy and full\-vocabulary probability distance\([Cheng et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib4)\),

ℒ=∑imi​wi​\[0\.1​\(−log⁡qi​\(yi\)\)\+0\.9​∑v∈𝒱\|qi​\(v\)−pi​\(v\)\|\]∑imi​wi,wi=e−\(i−1\)/γ,\\mathcal\{L\}=\\frac\{\\sum\_\{i\}m\_\{i\}\\,w\_\{i\}\\left\[\\,0\.1\\,\\big\(\-\\log q\_\{i\}\(y\_\{i\}\)\\big\)\+0\.9\\sum\_\{v\\in\{\\mathcal\{V\}\}\}\\lvert q\_\{i\}\(v\)\-p\_\{i\}\(v\)\\rvert\\right\]\}\{\\sum\_\{i\}m\_\{i\}\\,w\_\{i\}\},\\qquad w\_\{i\}=e^\{\-\(i\-1\)/\\gamma\},with decayγ=4\\gamma=4andmim\_\{i\}a validity mask that keeps only positions whose supervision falls inside the assistant span\. The distribution\-matching term aligns the drafter’s proposal with the target’s acceptance behavior over the whole vocabulary; the confidence loss is disabled, so the fixed budgetddis never dynamically truncated\.

### 3\.3DPara Algorithm

Figure 6:M\-DFlash attention mask ford=3d=3\. Colored cells indicate visible connections and white indicate masked connections\. All queries access cached target features \(gray\)\. The previous bonusAA\(dark blue\) and draftsB,C,DB,C,D\(light blue\) form a shared causal spine\. Four bidirectional three\-mask branches \(medium blue\) read the anchor prefixesAA,A​BAB,A​B​CABC, andA​B​C​DABCD, respectively, without attending to one another\.Algorithm[1](https://arxiv.org/html/2609.27396#alg1)details one steady\-state round\. DPara maintains a target\-feature cacheCCover the accepted history and a feature\-less draft blockt0:dt\_\{0:d\}carried over from the previous round, wheret0t\_\{0\}is the previous bonus andt1:dt\_\{1:d\}are the drafts now being scored\. The M\-DFlash backbone runs on the shared spinet0:dt\_\{0:d\}concurrently with the target verifying that same token block, and emits base logits for*all*d\+1d\+1acceptance boundaries\. This is exactly where DPara departs from SSD\([Kumar et al\., 2026](https://arxiv.org/html/2609.27396#bib.bib8)\), which predicts the accepted prefix and bonus token to pre\-generate the next draft and reverts to serial speculation whenever that prediction misses\. DPara instead predicts future logits, and it never has to*guess the verification outcome*: it commits to no particular accepted prefix or particular bonus token, so whatever\(r,b\)\(r,b\)the verifier returns, a finished branch is already waiting\.

Only the head stays on the critical path\.When verification returns\(r,b\)\(r,b\), DPara selects branchrrand runs*only*the lightweight AR head to turn its corresponding predicted logits into the next drafty1:dy\_\{1:d\}; since the backbone already overlapped with verification, this low\-rank correction is the sole additional cost exposed between two rounds\. The verified features oft0:rt\_\{0:r\}are appended toCCand the rejected suffix dropped, while the bonusbb—sampled at the last accepted position and therefore carrying no target feature*yet*—becomes the feature\-lesst0t\_\{0\}of the next round, to be forwarded until next verification\. Approximating a round as

Tstep≈max⁡\(Tverify,Tprecompute\)\+THead,T\_\{\\text\{step\}\}\\approx\\max\\\!\\big\(T\_\{\\text\{verify\}\},T\_\{\\text\{precompute\}\}\\big\)\+T\_\{\\text\{Head\}\},the diffusion backbone term is hidden whenever it fits inside the verification window\.

## 4Experiments

### 4\.1Experiments Setup

##### Target models\.

We train M\-DFlash drafters for two models of the Qwen3 family \(Qwen3\-8B and Qwen3\-14B\([Yang et al\., 2025](https://arxiv.org/html/2609.27396#bib.bib12)\)\); the detailed training recipe is provided in Appendix[E](https://arxiv.org/html/2609.27396#A5)\.

##### Hardware\.

All training is conducted on a server equipped with NVIDIA GB200 GPUs \(4 GPUs, 268 GB\)\. The main results are obtained on an NVIDIA H800 node \(8 GPUs, 80 GB each\)\. We also conduct additional inference experiments on NVIDIA GB200 GPUs\. For PSD methods, including PEARL, SSD, and DPara, we use one additional GPU dedicated to drafting\. We further explore an NVIDIA A10 \(1×241\\times 24GB\) as a resource\-constrained drafting device \(§[4\.3](https://arxiv.org/html/2609.27396#S4.SS3)\)\.

##### Benchmarks\.

We evaluate DPara on seven datasets across three workload categories:Mathfor mathematical reasoning, including GSM8K\([Cobbe et al\., 2021](https://arxiv.org/html/2609.27396#bib.bib15)\)and MATH\-500\([Lightman et al\., 2023](https://arxiv.org/html/2609.27396#bib.bib18)\);Chatfor conversation and instruction following, including MT\-Bench\([Zheng et al\., 2023](https://arxiv.org/html/2609.27396#bib.bib13)\)and Alpaca\([Taori et al\., 2023](https://arxiv.org/html/2609.27396#bib.bib16)\); andCodingfor code generation, including HumanEval\([Chen et al\., 2024](https://arxiv.org/html/2609.27396#bib.bib14)\), MBPP\([Austin et al\., 2021](https://arxiv.org/html/2609.27396#bib.bib19)\), and CodeAlpaca\([Lhoest et al\., 2021](https://arxiv.org/html/2609.27396#bib.bib17)\)\.

### 4\.2Main Result

Table 1:DPara against parallel \(PSD\) and serial \(SD\) speculative decoding baselines on Qwen3\-8B and Qwen3\-14B\. Within each model block, the upper group of rows lists PSD baselines, the middle group serial SD baselines, and the last row DPara\. We report decoding speedup over the AR baseline and the average acceptance lengthτ\\tau, with batch size11, greedy decoding, up to512512new tokens, draft length77\. SSD\-E3 takes corresponding EAGLE3 weight of target as draft model\. SSD and PEARL in Qwen3\-14B take Qwen3\-0\.6B as draft model\.##### Gains over both serial and parallel speculation\.

DPara achieves average speedups over autoregressive decoding of3\.21×3\.21\\timeson Qwen3\-8B and3\.52×3\.52\\timeson Qwen3\-14B across seven benchmarks at batch size11\(Table[1](https://arxiv.org/html/2609.27396#S4.T1)\)\. It attains the highest reported speedup on every model–dataset pair, with a tie with DSpark on Qwen3\-14B Alpaca\. The advantage spans math, coding, and chat, and extends beyond methods that serialize drafting and verification: on Qwen3\-14B, DPara improves the average speedup over PEARL and SSD by31%31\\%and64%64\\%, respectively\. These results show that effective backbone–verification overlap translates into lower end\-to\-end decoding latency\. Across the evaluated models and workloads, DPara achieves state\-of\-the\-art speedups across both serial and parallel speculative decoding methods\.

##### Balancing draft acceptance and critical\-path cost\.

DPara is designed around a joint optimization of acceptance length and drafting cost for end\-to\-end latency: it preserves comparatively high acceptance through a high\-capacity diffusion backbone while hiding the backbone forward pass entirely within verification, leaving only the lightweight AR head on the drafting critical path\. This balance yields the best overall end\-to\-end decoding latency among the evaluated methods\. Specifically, DPara retains average acceptance lengths of4\.434\.43and4\.414\.41on Qwen3\-8B and Qwen3\-14B, respectively, compared with5\.365\.36for DSpark, the strongest serial SD baseline \(Table[1](https://arxiv.org/html/2609.27396#S4.T1)\)\. This acceptance gap reflects the information constraints imposed by PSD: when the backbone starts, neither the accepted length nor the new bonus is known, and fresh target features for the draft tokens carried over from the previous round are unavailable\. Unlike serial drafting, which conditions on a resolved acceptance boundary and a fully featured prefix, DPara must precompute all boundaries from feature\-less anchors and defer bonus conditioning to the AR head\. The resulting information deficit constrains the acceptance achievable by the parallel backbone relative to serial drafting\. Nevertheless, the end\-to\-end results show that removing backbone execution from the critical path more than compensates for this acceptance gap: DPara improves average speedup over DSpark by11\.1%11\.1\\%on Qwen3\-8B and5\.1%5\.1\\%on Qwen3\-14B\. The net benefit is lower latency per generated token, achieved by retaining strong draft acceptance without paying for a serial backbone forward pass\.

Table 2:Batched inference on Qwen3\-8B/14B \(Q3\-8B/14B\)\. Each method reports throughput \(tok/s\) and speedup over AR\.ModelTaskMethodBatch size124816Q3\-8BGSM8KAR1412775299371520DSpark4919011507242636273\.48×\\times3\.25×\\times2\.85×\\times2\.59×\\times2\.39×\\timesDPara58810521772273638694\.17×\\times3\.80×\\times3\.35×\\times2\.92×\\times2\.55×\\timesHuman\-EvalAR1412695329571684DSpark4458151464228333823\.15×\\times3\.02×\\times2\.75×\\times2\.39×\\times2\.01×\\timesDPara4959141565256736543\.50×\\times3\.39×\\times2\.94×\\times2\.68×\\times2\.17×\\timesQ3\-14BGSM8KAR871713305971020DSpark3556471142181328474\.07×\\times3\.79×\\times3\.47×\\times3\.03×\\times2\.79×\\timesDPara3957211269199930104\.53×\\times4\.22×\\times3\.85×\\times3\.35×\\times2\.95×\\timesHuman\-EvalAR871703336221080DSpark3195981082168825233\.66×\\times3\.52×\\times3\.25×\\times2\.71×\\times2\.34×\\timesDPara3356341140182427243\.84×\\times3\.74×\\times3\.42×\\times2\.93×\\times2\.52×\\times

##### The advantage persists under batching\.

DPara remains faster than DSpark at every measured batch size from11to1616on every evaluated benchmark, for both target models \(Table[2](https://arxiv.org/html/2609.27396#S4.T2)shows GSM8K and HumanEval\)\. The single\-request advantage therefore survives batched execution, rather than disappearing as more requests share each forward pass\. At batch size1616, DPara retains2\.55×2\.55\\timesand2\.17×2\.17\\timesspeedups over AR on Qwen3\-8B, and2\.95×2\.95\\timesand2\.52×2\.52\\timeson Qwen3\-14B, for GSM8K and HumanEval, respectively\. Its throughput exceeds DSpark’s by approximately6%6\\%–8%8\\%across these four settings\. Nevertheless, batching reduces the relative benefit of speculation: absolute throughput increases, but speedup over the correspondingly batched AR baseline declines for both methods\. Outcome\-independent overlap thus does not imply batch\-invariant speedup; the empirical result is a sustained advantage over DSpark and more than2×2\\timesAR throughput throughout the measured range\.

### 4\.3Further explorations of resource\-constrained drafting

Dedicating a GPU to drafting lets PSD reduce latency by overlapping backbone execution with verification, but incurs an additional hardware cost\. We therefore explore two complementary ways to cut the drafting budget while retaining DPara’s latency advantage: drafting on a much weaker GPU paired with the H800 target \(*DPara\-A10*, §[4\.3\.1](https://arxiv.org/html/2609.27396#S4.SS3.SSS1)\), and colocating drafting and verification on one GPU \(*DPara\-single*, §[4\.3\.2](https://arxiv.org/html/2609.27396#S4.SS3.SSS2)\)\. Both are compared with the default two\-GPU DPara; implementation details are deferred to Appendix[D](https://arxiv.org/html/2609.27396#A4)\.

Figure 7:Left: anatomy of one speculation round on Qwen3\-14B \(GSM8K, batch11\); bars are measured critical paths and hatched sub\-bars show drafting work overlapped with verification\. DSpark serializes its draft stage \(4\.44\.4ms,25%25\\%of the step\); DPara hides the backbone and exposes only1\.31\.3ms \(10%10\\%\); DPara\-A10 also keeps the A10\-side backbone and hidden\-state push inside the verification window, adding only the target\-side LM head \(0\.30\.3ms\) to the critical path\.Right: Qwen3\-8B speedup over AR at batch size11with the M\-DFlash backbone running on an NVIDIA A10 as the drafting GPU, compared with DSpark and DPara\.#### 4\.3\.1Heterogeneous drafting

##### Engineering adaptation to the heterogeneous placement\.

DPara\-A10 pairs the H800 target with an NVIDIA A10 for drafting \(Appendix[D](https://arxiv.org/html/2609.27396#A4)\), so the drafting pipeline must be partitioned across two unequal GPUs\. A naive port keeps the whole pipeline—backbone, LM head, and AR\-head sampling—on the A10\. Its AR\-head sampling then sits on the critical path, where the latency negligible on an H800 is magnified on the weaker GPU, motivating a move to the target\. But a target\-side AR head must be fed full\-vocabulary logits, an expensive cross\-interconnect transfer\. We therefore also move the LM head to the target, so the A10 ships only the backbone hidden states—roughly1/301/30the size of the logits—which cross the link quickly\. The residual cost is one LM\-head forward on the target’s critical path: we trade a little compute for a large reduction in network transfer\.

##### Impact on end\-to\-end speed\.

This partition preserves DPara’s overlap structure on the weaker GPU \(Figure[7](https://arxiv.org/html/2609.27396#S4.F7), left\): within a single verification window the A10 completes both the drafting backbone forward and the asynchronous transfer of the hidden states of all predicted branches, leaving only the target\-side LM head on the critical path\. DPara\-A10 thus retains about98%98\\%of DPara’s speedup on all four benchmarks and stays well ahead of DSpark \(Figure[7](https://arxiv.org/html/2609.27396#S4.F7), right\); detailed per\-step and per\-benchmark measurements are in Appendix[D](https://arxiv.org/html/2609.27396#A4)\.

#### 4\.3\.2Single\-GPU colocation

DPara\-single executes verification and backbone precomputation on two CUDA streams of one GPU, captured in a single CUDA graph \(Appendix[D](https://arxiv.org/html/2609.27396#A4)\)\. On Qwen3\-8B with GB200, it reaches2\.62×2\.62\\timesspeedup over AR at batch size11, compared with2\.34×2\.34\\timesfor DSpark and2\.77×2\.77\\timesfor DPara, and retains90%90\\%–95%95\\%of DPara’s speedup across batch sizes11–44while halving the GPU count \(Figure[8](https://arxiv.org/html/2609.27396#S4.F8), left\)\. The advantage over DSpark disappears at batch sizes88and1616: decoding there turns compute\-bound, and the colocated backbone competes with the target for execution resources and bandwidth \(Appendix[D](https://arxiv.org/html/2609.27396#A4)\)\. Colocation is therefore a resource\-efficient option for small batches, whereas dedicated drafting resources remain preferable as batch size increases\.

Figure 8:Left: Qwen3\-8B speedup over AR under DPara\-single and DPara with DSpark at different batch sizes; each bar averages the seven benchmarks\.Right: acceptance lengthτ\\taugains after finetuning on Qwen3\-8B’s original DSpark checkpoint; both checkpoints run within DPara with the same frozen AR head, and red labels show absolute gains\.

### 4\.4Ablation Study

##### Effectiveness of backbone finetuning\.

We test whether finetuning only the backbone adapts the drafter to the multiple\-anchor, feature\-less regime required by DPara \(§[3\.2](https://arxiv.org/html/2609.27396#S3.SS2)\)\. We run DPara on Qwen3\-8B with the original DSpark checkpoint and the finetuned M\-DFlash checkpoint, keeping the AR head and decoding configuration unchanged\. Figure[8](https://arxiv.org/html/2609.27396#S4.F8)\(right\) shows that finetuning increasesτ\\tauon all four benchmarks: from4\.074\.07to5\.825\.82on GSM8K,4\.064\.06to5\.415\.41on MATH\-500,3\.813\.81to4\.784\.78on HumanEval, and2\.552\.55to2\.922\.92on Alpaca\. These correspond to relative gains of43\.1%43\.1\\%,33\.3%33\.3\\%,25\.5%25\.5\\%, and14\.4%14\.4\\%, respectively; the meanτ\\tauacross these four datasets rises from3\.623\.62to4\.734\.73\(30\.7%30\.7\\%\)\. The consistency of these gains across math, coding, and chat confirms both the necessity of adapting the backbone to multiple feature\-less anchors and the reusability of the published AR head without any finetuning\.

## 5Conclusion

We presented DPara, the first parallel speculative decoding framework powered by DSpark\-style parallel drafters\. By replacing outcome\-dependent speculation with outcome\-independent precomputation, DPara eliminates prediction\-induced serial backbone fallback entirely while hiding the dominant drafting cost inside the verification window\. This decoupling of backbone execution from bonus\-conditioned token generation reconciles high\-capacity drafting with a minimal critical path, and it translates into state\-of\-the\-art end\-to\-end decoding latency across math, coding, and chat workloads\.

## References

- Austinet al\.\(2021\)J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le,et al\.Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§4\.1](https://arxiv.org/html/2609.27396#S4.SS1.SSS0.Px3.p1.1)\.
- Chenet al\.\(2023\)C\. Chen, S\. Borgeaud, G\. Irving, J\. Lespiau, L\. Sifre, and J\. JumperAccelerating large language model decoding with speculative sampling\.arXiv preprint arXiv:2302\.01318\.Cited by:[§1](https://arxiv.org/html/2609.27396#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.27396#S2.SS1.p1.1)\.
- Chenet al\.\(2026\)J\. Chen, Y\. Liang, and Z\. LiuDFlash: Block Diffusion for Flash Speculative Decoding\.arXiv preprint arXiv:2602\.06036\.Cited by:[§1](https://arxiv.org/html/2609.27396#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.27396#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2609.27396#S3.SS1.p1.1)\.
- Chenet al\.\(2024\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. de Oliveira Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, B\. Chan, S\. Gray, N\. Ryder, M\. Pavlov, A\. Power, L\. Kaiser, M\. Bavarian, C\. Winter, P\. Tillet, F\. P\. Such, D\. Cummings, M\. Plappert, F\. Chantzis, E\. Barnes, A\. Herbert\-Voss, W\. H\. Guss, A\. Nichol, A\. Paino, N\. Tezak, J\. Tang, I\. Babuschkin, S\. Balaji, S\. Jain, W\. Saunders, C\. Hesse, A\. N\. Carr, J\. Leike, J\. Achiam, V\. Misra, E\. Morikawa, A\. Radford, M\. Knight, M\. Brundage, M\. Murati, K\. Mayer, P\. Welinder, B\. McGrew, D\. Amodei, S\. McCandlish, I\. Sutskever, and W\. ZarembaEvaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§4\.1](https://arxiv.org/html/2609.27396#S4.SS1.SSS0.Px3.p1.1)\.
- Chenget al\.\(2026\)X\. Cheng, X\. Yu, C\. Shao, J\. Li, Y\. Xiong, Y\. Qian, J\. Zhu, S\. Ma, X\. Zhang, J\. Ye, Q\. Chen, C\. Deng, J\. Yu, D\. Dai, Z\. Zhang, Y\. Wei, Y\. Tan, W\. Yang, R\. Xu, Y\. Wu, Z\. Xu, X\. Wang, M\. Chen, R\. Tian, X\. Bi, Z\. Hao, S\. Chen, H\. Cao, W\. Zhang, A\. Xu, H\. Zhang, D\. Zhao, and W\. LiangDSpark: confidence\-scheduled speculative decoding with semi\-autoregressive generation\.External Links:2607\.05147,[Link](https://arxiv.org/abs/2607.05147)Cited by:[§1](https://arxiv.org/html/2609.27396#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.27396#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2609.27396#S3.SS2.p1.1),[§3\.2](https://arxiv.org/html/2609.27396#S3.SS2.p4.2)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§4\.1](https://arxiv.org/html/2609.27396#S4.SS1.SSS0.Px3.p1.1)\.
- Huanget al\.\(2026\)J\. Huang, Y\. Zhang, Q\. Zhang, H\. Lin, H\. Xu, and L\. ZhangDomino: decoupling causal modeling from autoregressive drafting in speculative decoding\.arXiv preprint arXiv:2605\.29707\.External Links:2605\.29707,[Document](https://dx.doi.org/10.48550/arXiv.2605.29707),[Link](https://arxiv.org/abs/2605.29707)Cited by:[§1](https://arxiv.org/html/2609.27396#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.27396#S2.SS2.p1.1)\.
- Kumaret al\.\(2026\)T\. Kumar, T\. Dao, and A\. MaySpeculative speculative decoding\.External Links:2603\.03251,[Link](https://arxiv.org/abs/2603.03251)Cited by:[§1](https://arxiv.org/html/2609.27396#S1.p3.1),[§2\.3](https://arxiv.org/html/2609.27396#S2.SS3.p1.1),[§3\.3](https://arxiv.org/html/2609.27396#S3.SS3.p1.1)\.
- Leviathanet al\.\(2023\)Y\. Leviathan, M\. Kalman, and Y\. MatiasFast inference from transformers via speculative decoding\.InInternational Conference on Machine Learning,pp\. 19274–19286\.Cited by:[§1](https://arxiv.org/html/2609.27396#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.27396#S2.SS1.p1.1)\.
- Lhoestet al\.\(2021\)Q\. Lhoest, A\. V\. Del Moral, Y\. Jernite, A\. Thakur, P\. Von Platen, S\. Patil, J\. Chaumond, M\. Drame, J\. Plu, L\. Tunstall,et al\.Datasets: a community library for natural language processing\.InProceedings of the 2021 conference on empirical methods in natural language processing: system demonstrations,pp\. 175–184\.Cited by:[§4\.1](https://arxiv.org/html/2609.27396#S4.SS1.SSS0.Px3.p1.1)\.
- Liet al\.\(2025\)Y\. Li, F\. Wei, C\. Zhang, and H\. ZhangEAGLE\-3: scaling up inference acceleration of large language models via training\-time test\.InAnnual Conference on Neural Information Processing Systems,Cited by:[§2\.2](https://arxiv.org/html/2609.27396#S2.SS2.p1.1)\.
- Lightmanet al\.\(2023\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.InThe Twelfth International Conference on Learning Representations,Cited by:[§4\.1](https://arxiv.org/html/2609.27396#S4.SS1.SSS0.Px3.p1.1)\.
- Liuet al\.\(2026\)F\. Liu, X\. Li, K\. Zhao, Y\. Gao, Z\. Zhou, Z\. Zhang, Z\. Wang, W\. Dou, S\. Zhong, and C\. TianDART: diffusion\-inspired speculative decoding for fast llm inference\.arXiv preprint arXiv:2601\.19278\.Cited by:[§1](https://arxiv.org/html/2609.27396#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.27396#S2.SS2.p1.1)\.
- Liuet al\.\(2025\)T\. Liu, Y\. Li, Q\. Lv, K\. Liu, J\. Zhu, W\. Hu, and X\. SunPEARL: parallel speculative decoding with adaptive draft length\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2609.27396#S1.p3.1),[§2\.3](https://arxiv.org/html/2609.27396#S2.SS3.p1.1)\.
- McDanel \(2024\)B\. McDanelAMUSD: asynchronous multi\-device speculative decoding for llm acceleration\.arXiv\.Cited by:[§1](https://arxiv.org/html/2609.27396#S1.p3.1),[§2\.3](https://arxiv.org/html/2609.27396#S2.SS3.p1.1)\.
- Shenet al\.\(2025\)Y\. Shen, J\. Shen, Q\. Kong, T\. Liu, Y\. Lu, and C\. WangSpeculative decoding via hybrid drafting and rollback\-aware branch parallelism\.arXiv\.Cited by:[§1](https://arxiv.org/html/2609.27396#S1.p3.1),[§2\.3](https://arxiv.org/html/2609.27396#S2.SS3.p1.1)\.
- Taoriet al\.\(2023\)R\. Taori, I\. Gulrajani, T\. Zhang, Y\. Dubois, X\. Li, C\. Guestrin, P\. Liang, and T\. B\. HashimotoStanford alpaca: an instruction\-following llama model\.GitHub\.Note:[https://github\.com/tatsu\-lab/stanford\_alpaca](https://github.com/tatsu-lab/stanford_alpaca)Cited by:[§4\.1](https://arxiv.org/html/2609.27396#S4.SS1.SSS0.Px3.p1.1)\.
- Xuet al\.\(2024\)T\. Xu, E\. Helenowski, K\. A\. Sankararaman, D\. Jin, K\. Peng, E\. Han, S\. Nie, C\. Zhu, H\. Zhang, W\. Zhou,et al\.The perfect blend: redefining rlhf with mixture of judges\.arXiv preprint arXiv:2409\.20370\.Cited by:[Appendix E](https://arxiv.org/html/2609.27396#A5.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. QiuQwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§4\.1](https://arxiv.org/html/2609.27396#S4.SS1.SSS0.Px1.p1.1)\.
- Zhenget al\.\(2023\)L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing,et al\.Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.Advances in Neural Information Processing Systems36,pp\. 46595–46623\.Cited by:[§4\.1](https://arxiv.org/html/2609.27396#S4.SS1.SSS0.Px3.p1.1)\.

## Appendix ALatency Breakdown

##### Profiling setup\.

We profile per\-stage latency on the H800 node used for our main results \(§[4\.1](https://arxiv.org/html/2609.27396#S4.SS1)\), with Qwen3\-14B as the target model on GSM8K:8080requests, greedy decoding, draft length77, up to512512new tokens, thinking mode disabled\. We instrument the verifier call and each drafting component with per\-phase timers and report the mean interval per speculation round over the3\.63\.6K–4\.34\.3K rounds of each run\. For DPara, which overlaps drafting with verification, the overlapped intervals are reported against the verification window they hide within and are never summed into the step latency; small residuals between independently timed intervals are folded into the context and scheduling segments\. The same pattern holds on the other datasets and on Qwen3\-8B\.

Figure 9:Per\-step latency across batch sizes on Qwen3\-14B \(GSM8K, same profiling setup as Figure[7](https://arxiv.org/html/2609.27396#S4.F7)\)\. DPara is the lower at every batch size; from batch11to1616its step grows30%30\\%, versus32%32\\%for DSpark\.
##### Anatomy of one speculation round\.

Figure[7](https://arxiv.org/html/2609.27396#S4.F7)\(left\) decomposes a single round at batch size11\. Under the conventional draft–verify schedule, DSpark executes its entire draft stage on the critical path before verification can start: the diffusion backbone costs2\.72\.7ms, the AR head0\.30\.3ms, and context bookkeeping1\.41\.4ms, exposing4\.44\.4ms—25%25\\%of its17\.717\.7ms step\. Across both target models and all seven benchmarks, this serial draft stage accounts for24%24\\%–32%32\\%of the per\-step latency of DFlash/DSpark\-style parallel drafters, which is exactly the overhead that PSD must hide to pay off\. DFlash behaves the same way \(4\.14\.1ms,24%24\\%of a17\.317\.3ms step\)\. DPara precomputes the same diffusion backbone against the verification window and exposes only the bonus\-conditioned AR head plus acceptance bookkeeping:1\.31\.3ms \(10%10\\%\) on top of a12\.512\.5ms verify, for a13\.813\.8ms step—just1\.21×1\.21\\timesthe AR per\-token forward and the lowest per\-step latency among the compared methods\. The colocated single\-GPU deployment \(§[4\.3](https://arxiv.org/html/2609.27396#S4.SS3)\) keeps the same structure and still leads DSpark at15\.115\.1ms per step\.

##### Per\-step latency across batch sizes\.

Figure[9](https://arxiv.org/html/2609.27396#A1.F9)tracks the same quantities from batch size11to1616\. DPara is the fastest at every batch size, and its step grows only mildly, from13\.813\.8to18\.018\.0ms \(\+30%\+30\\%\), as verification lengthens\. DSpark follows a similar slope \(17\.7→23\.317\.7\\rightarrow 23\.3ms,\+32%\+32\\%\) because its serial draft stage also lengthens with the batch, but it never closes the gap: its step stays28%28\\%–30%30\\%longer than DPara’s at every batch size\. Read together with acceptance length, the per\-step gap is decisive: at batch size11, DSpark’s higherτ\\tau\(6\.56\.5vs\.5\.75\.7\) does not offset its28%28\\%longer step \(2\.712\.71vs\.2\.412\.41ms per token\)\.

## Appendix BSSD Fallback Rate under Batching

Figure 10:Fraction of speculation steps in which SSD falls back to serial re\-drafting, measured on Qwen3\-14B \(seven benchmarks, batch11–1616, same configuration as Appendix[A](https://arxiv.org/html/2609.27396#A1)\)\. A step falls back when the revealed bonus token of any request in the batch misses SSD’s speculation cache, forcing the drafter to re\-draft the whole batch serially\. The rate grows with batch size on every workload and reaches57%57\\%–96%96\\%at batch1616\. DPara has no such events at any batch size\.##### Measuring fallback\.

SSD pre\-drafts against a predicted bonus token while the target verifies \(§[2](https://arxiv.org/html/2609.27396#S2)\)\. When the revealed bonus token of any request in the batch misses the speculation cache, the precomputed continuation is discarded and the drafter re\-drafts the whole batch serially, just in time for the next step\. Our profiler instruments these serial re\-drafting calls on the drafting GPU; we report the fraction of speculation steps that trigger at least one such call, measured on Qwen3\-14B with the configuration of Appendix[A](https://arxiv.org/html/2609.27396#A1)\.

##### Fallback rate grows with batch size\.

Figure[10](https://arxiv.org/html/2609.27396#A2.F10)shows the measured rates\. At batch size11,8%8\\%–32%32\\%of steps already fall back, depending on how predictable the workload’s bonus token is; by batch size1616the rate reaches57%57\\%–96%96\\%\. The mechanism is the amplification of a per\-request bonus\-mismatch probabilityppinto a per\-step fallback probability1−\(1−p\)B1\-\(1\-p\)^\{B\}for a batch ofBBrequests, since a single mismatch voids the whole batch’s precomputed drafts\. Chat workloads suffer most: their per\-request mismatch rate is highest at batch11\(29%29\\%–32%32\\%\), and by batch1616nearly every step is serial \(91%91\\%–96%96\\%\), consistent with SSD’s weakest speedups on Alpaca and MT\-Bench in Table[1](https://arxiv.org/html/2609.27396#S4.T1)and its steep throughput drop under batching in §[4\.2](https://arxiv.org/html/2609.27396#S4.SS2)\. Math workloads, with the most predictable bonus tokens \(8%8\\%at batch11\), still fall back in over half of all steps at batch1616\. DPara records no such events at any batch size: by covering every acceptance boundary instead of betting on a predicted bonus token, it removes the prediction whose failure triggers the fallback, so the rate is structurally zero\.

## Appendix CResults under Sampling

We repeat the batch\-size\-11comparison of Table[1](https://arxiv.org/html/2609.27396#S4.T1)under sampling: temperature1\.01\.0for both the target model and the drafter, top\-pp1\.01\.0, draft length77, the same8080requests per benchmark, and up to512512new tokens\. Acceptance follows the standard speculative\-sampling rejection rule, under which a draft token can be rejected even when it coincides with the target’s most likely continuation\.

Table 3:DPara against serial speculative decoding \(SD\) baselines on Qwen3\-8B and Qwen3\-14B with sampling decoding\. We report decoding speedup over the AR baseline and the average acceptance lengthτ\\tau, with batch size11, temperature1\.01\.0for both the target and the drafter, up to512512new tokens, draft length77\.##### Sampling lowers acceptance for every method\.

Each method’sτ\\taudrops relative to greedy decoding \(Table[1](https://arxiv.org/html/2609.27396#S4.T1)\): for DPara, from4\.434\.43to4\.114\.11on Qwen3\-8B and from4\.414\.41to4\.094\.09on Qwen3\-14B; DSpark falls from5\.365\.36to5\.055\.05and5\.045\.04, respectively\. Average speedups fall correspondingly: DPara goes from3\.21×3\.21\\timesto2\.72×2\.72\\timeson Qwen3\-8B and from3\.52×3\.52\\timesto3\.13×3\.13\\timeson Qwen3\-14B, with similar relative declines for the serial baselines\.

##### DPara retains the best average speedup\.

The ranking of methods is preserved under sampling \(Table[3](https://arxiv.org/html/2609.27396#A3.T3)\)\. DPara keeps the highest average speedup on both models—2\.72×2\.72\\timeson Qwen3\-8B and3\.13×3\.13\\timeson Qwen3\-14B, versus2\.54×2\.54\\timesand3\.05×3\.05\\timesfor DSpark—and is the fastest on every dataset of both models, including the chat workload where DSpark tied it under greedy decoding \(Table[1](https://arxiv.org/html/2609.27396#S4.T1)\)\. DPara’s advantage is orthogonal to the acceptance rule: its backbone stays hidden inside verification whether tokens are accepted greedily or by sampling, so the per\-step latency gap over serial drafting \(Appendix[A](https://arxiv.org/html/2609.27396#A1)\) carries over unchanged and only the acceptance length moves\.

## Appendix DDeployment Details for Resource\-Constrained Drafting

This appendix expands on the two deployments of §[4\.3](https://arxiv.org/html/2609.27396#S4.SS3)\.

##### Heterogeneous role split\.

The naive port of DPara’s two\-GPU role split places the entire drafting side, including the M\-DFlash LM head and the AR head, on the drafting GPU\. On an A10 this hurts twice: the AR head, which sits on the critical path, adds substantial latency on the weak GPU, and shipping full\-vocabulary logits crosses the slow interconnect\. DPara\-A10 therefore narrows the drafting GPU to the diffusion backbone alone: verification, branch selection, the LM head, and AR\-head sampling of the next draft tokens all run on the H800 target\. The two machines are connected over Ethernet \(NCCL data plane on TCP sockets, RTT∼0\.8\{\\sim\}0\.8ms, measured streaming bandwidth3\.93\.9–4\.04\.0GB/s\)\. The A10 pushes only the final hidden states of alld\+1d\{\+\}1branches acrossddpositions \(573573KB for Qwen3\-14B, about1/301/30of the corresponding logits,≈0\.14\{\\approx\}0\.14ms to transfer\), and the H800 runs the LM head itself \(≈0\.3\{\\approx\}0\.3ms of extra target\-side compute\) before the AR head\. Trading cheap target\-side compute for network transfer keeps every network delay off the critical path\.

##### Asynchronous cross\-machine transfer\.

Decoding proceeds in verification windows\. While the H800 verifies, the A10 runs the backbone for the next round and asynchronously pushes the branch hidden states; under full overlap they have all arrived by the time verification returns, so the H800 can immediately run the LM head and AR head\. It then packs the accepted length and the target features at the accepted positions into a single asynchronous message back to the A10, which advances its drafting context and starts the next backbone pass\. Both sides replay captured CUDA graphs to eliminate scheduling overhead\.

##### Per\-step anatomy and end\-to\-end cost\.

Figure[7](https://arxiv.org/html/2609.27396#S4.F7)\(left\) dissects one speculation round on Qwen3\-14B \(GSM8K, batch11\)\. DSpark serializes its4\.44\.4ms draft stage—25%25\\%of the step—before verification, whereas DPara hides the backbone entirely and exposes only1\.31\.3ms \(10%10\\%\)\. On the A10, the backbone pass takes7\.37\.3ms and the asynchronous hidden\-state push1\.51\.5ms; both complete within the12\.512\.5ms verification window, and the target\-side LM head adds only0\.30\.3ms to the critical path, for a14\.314\.3ms step versus13\.813\.8ms for DPara\. End to end, DPara\-A10 retains about98%98\\%of DPara’s speedup on all four benchmarks; averaged over all seven benchmarks, it reaches3\.14×3\.14\\timeson Qwen3\-8B and3\.43×3\.43\\timeson Qwen3\-14B, versus3\.21×3\.21\\timesand3\.52×3\.52\\timesfor DPara\.

##### Single\-GPU execution\.

We colocate the target and drafter in one process on the same GPU and execute verification and backbone precomputation on two CUDA streams\. The branch\-selection and bonus\-conditioned AR\-head logic is unchanged\. The head and both forward passes are captured in a single CUDA graph, and acceptance checks and request scheduling remain outside the graph\. Incremental projection of cached history features avoids repeated work, while per\-request branch state and fixed buffers support safe graph reuse\. Unlike the two\-GPU deployment, however, the two streams share compute and memory bandwidth: logical concurrency does not guarantee that backbone latency is fully hidden\.

##### Why colocation loses ground at larger batches\.

The single\-GPU advantage over DSpark disappears at batch sizes88and1616, and its gap to DPara widens\. This trend is consistent with a shift from memory\-bound toward compute\-bound decoding: larger batches amortize weight reads across more requests\. At small, memory\-bound batches, spare compute capacity allows useful overlap between drafting and verification\. As compute demand grows, the colocated backbone competes with the target for the same execution resources and bandwidth, increasing the effective per\-step cost; separate GPUs avoid this direct competition\.

## Appendix ETraining Recipe

All M\-DFlash backbones are finetuned for one epoch with AdamW under a linear\-warmup cosine schedule \(learning rate 3e\-5,4%4\\%warmup\)\. The effective global batch is128128\(micro\-batch11per GPU with gradient accumulation\); each text is truncated to40964096tokens and sampled with up to256256prediction windows that balance all legal anchor countsk∈\{1,…,d\+1\}k\\in\\\{1,\\dots,d\+1\\\}\. The loss follows §[3\.2](https://arxiv.org/html/2609.27396#S3.SS2), and target features and supervision are produced online by the corresponding frozen target model\. We construct a shared training set from responses generated by Qwen3\-8B on prompts from Open\-PerfectBlend\([Xu et al\., 2024](https://arxiv.org/html/2609.27396#bib.bib20)\), and use it to finetune the M\-DFlash drafters for Qwen3\-8B and Qwen3\-14B\. We initialize from the corresponding official DSpark checkpoints and keep the AR heads frozen\.

## Appendix FLLM Usage

We employed LLM\-based tools solely during manuscript preparation to refine language and improve readability\. The research questions, technical approach, experimental methodology, interpretation of results, and conclusions were developed independently by the authors\. The authors reviewed the complete manuscript and remain fully responsible for its content and accuracy\.