Decoupled Contrastive Decoding via Expert-Aligned Drafting

# Decoupled Contrastive Decoding via Expert-Aligned Drafting
Source: [https://arxiv.org/html/2608.12913](https://arxiv.org/html/2608.12913)
Zhixuan LiuThanks:Corresponding author: lzx993124494@sjtu\.edu\.cnAffiliation:Shanghai Jiao Tong UniversityAffiliation:Shanghai Artificial Intelligence LaboratoryZhichen DongAffiliation:Shanghai Jiao Tong UniversityAffiliation:Shanghai Artificial Intelligence LaboratoryChao YangAffiliation:Shanghai Artificial Intelligence Laboratory

###### Abstract

Contrastive Decoding \(CD\) improves generation quality, but its amateur\-model pass makes decoding expensive\. Accelerating CD with speculative decoding raises a proposal\-alignment question: should the contrastive signal shape the drafter, or should it remain only in verification? We study this question in the lightweight feature\-level drafter regime\. Two controlled diagnostics, matched Cross\-α\\alphatraining and an Approximate Dual\-Drafter decomposition, give the same diagnosis: contrastive\-aware drafting does not consistently improve over expert\-aligned drafting because the contrastive correction is usually weaker than drafter error, and reconstruction can amplify that error\. We introduceDecoupled Contrastive Decoding \(DCD\), which drafts with an expert\-aligned lightweight proposer and applies the amateur only in unchanged CD verification\. Standard speculative verification preserves the vanilla\-CD output distribution\. Across the main 8B settings, EAGLE3\-based DCD achieves average greedy speedups of 1\.65 to 1\.95×\\timesover vanilla CD and reduces MMLU proposal\-path latency by about 5 to 12×\\timesrelative to amateur\-coupled proposal paths\.

## 1Introduction

Large Language Models \(LLMs\) still hallucinate and make reasoning errors\([Huang et al\. 2025](https://arxiv.org/html/2608.12913#bib.bib11);[Plaat et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib23)\)\. Contrastive Decoding \(CD\)\([Li et al\. 2023](https://arxiv.org/html/2608.12913#bib.bib15);[O’Brien and Lewis 2023](https://arxiv.org/html/2608.12913#bib.bib22)\)improves generation by correcting a strongexpertwith a weakeramateur\. It also underlies lightweight tuning schemes such as Emulator Fine\-Tuning\([Mitchell et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib21)\)and Proxy Tuning\([Liu et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib19)\), but each token requires both models\.

Speculative decoding can reduce this cost, but CD adds a proposal\-alignment choice absent from single\-model decoding: the serial proposal path can use the amateur signal, imitate the full contrastive distribution, or leave contrastive scoring to verification\. We study three proposal routes: amateur\-coupled proposals, contrastive\-aware lightweight proposals, and expert\-aligned proposals with contrastive verification\. Figure[1](https://arxiv.org/html/2608.12913#S1.F1)illustrates the system\-level paths in the main comparison: vanilla CD, amateur\-coupled SCD, and DCD with expert\-aligned proposals plus contrastive verification\.

![Refer to caption](https://arxiv.org/html/2608.12913v1/overview_new.png)Figure 1:DCD keeps the CD target in verification while avoiding an amateur\-coupled proposal route\. The expertℳp\\mathcal\{M\}\_\{p\}and amateurℳq\\mathcal\{M\}\_\{q\}still defineπC​D\\pi\_\{CD\}; the lightweight proposerEEchanges only the serial draft path\.We first diagnose proposal\-side contrastive alignment under matched lightweight\-drafter conditions\. In the Cross\-α\\alphadiagnostic, a dual\-input EAGLE variant receives concatenated expert and amateur hidden states; within that variant, varying train\-α\\alphachanges only the target distribution, while the architecture, data, and training recipe remain fixed\. An Approximate Dual\-Drafter separately approximates expert and amateur distributions and recombines them at inference\. Both diagnostics show the same failure mode: the contrastive signal is usually weaker than the drafter error it must overcome, and reconstruction can amplify that error\. Across 24 configurations, 81\.1% of positions have a contrastive signal below1\.01\.0, while 48\.7% have expert\-side proposalDK​L≥2\.0D\_\{KL\}\\geq 2\.0; positiveΔ\\DeltaTop\-1 values concentrate in high\-signal, low\-error cells\. Neither contrastive\-aware variant yields consistent accepted\-length gains over expert\-aligned drafting \(Section[3\.1](https://arxiv.org/html/2608.12913#S3.SS1)\)\.

DCD validates the expert\-aligned route at system level\. It reuses a lightweight expert\-aligned drafter for proposals and retains the amateur only in verification, so EAGLE\-style drafting can accelerate CD without retraining for each amateur\. SCD and CoS keep the amateur in the serial proposal loop\.

Instantiated with EAGLE3\([Li et al\. 2024a](https://arxiv.org/html/2608.12913#bib.bib16);[Li et al\. 2024b](https://arxiv.org/html/2608.12913#bib.bib17);[Li et al\. 2025](https://arxiv.org/html/2608.12913#bib.bib18)\), DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}achieves average greedy speedups of 1\.65 to 1\.95×\\timesover vanilla CD and reduces MMLU per\-step proposal latency by about 5 to 12×\\timesrelative to amateur\-coupled proposal paths\. DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}remains close to2×2\\timesin a greedy\-only 70B extension, and a DCD variant with a matched N\-gram proposer also gives average speedups above vanilla CD\. Because DCD changes only the proposal path, its lossless guarantee follows from standard speculative verification\.

Our contributions are threefold: \(1\) we formulate proposal alignment as the central design choice in speculative contrastive decoding, separating amateur\-coupled, contrastive\-aware, and expert\-aligned proposal routes; \(2\) we provide controlled diagnostics showing that contrastive\-aware lightweight drafting does not reliably improve accepted length over expert\-aligned drafting, because the contrastive signal is often smaller than proposal error and reconstruction can amplify that error; and \(3\) we instantiate this diagnosis as Decoupled Contrastive Decoding \(DCD\), a lossless CD accelerator that keeps contrastive scoring in verification and achieves deployment\-level speedups with both EAGLE3 and matched N\-gram proposers\.

## 2Methodology

This section formalizes proposal alignment and the DCD verification recipe\.

### 2\.1Preliminaries

#### Contrastive Decoding\.

Given historyhh, expertℳp\\mathcal\{M\}\_\{p\}, and amateurℳq\\mathcal\{M\}\_\{q\}, Contrastive Decoding\([Li et al\. 2023](https://arxiv.org/html/2608.12913#bib.bib15);[O’Brien and Lewis 2023](https://arxiv.org/html/2608.12913#bib.bib22)\)favors tokens high under the expert and low under the amateur:

log⁡πC​D​\(x∣h\)=\\displaystyle\\log\\pi\_\{CD\}\(x\\mid h\)=\{\}\(1\+α\)​log⁡πp​\(x∣h\)\\displaystyle\(1\+\\alpha\)\\log\\pi\_\{p\}\(x\\mid h\)−α​log⁡πq​\(x∣h\)−log⁡ZC​D​\(h\),\\displaystyle\}\{\\displaystyle\-\\alpha\\log\\pi\_\{q\}\(x\\mid h\)\-\\log Z\_\{CD\}\(h\),whereα≥0\\alpha\\geq 0controls contrastive strength\. Equivalently,

πC​D​\(x∣h\)=1ZC​D​\(h\)​πp​\(x∣h\)1\+α​πq​\(x∣h\)−α,\\pi\_\{CD\}\(x\\mid h\)=\\frac\{1\}\{Z\_\{CD\}\(h\)\}\\pi\_\{p\}\(x\\mid h\)^\{1\+\\alpha\}\\pi\_\{q\}\(x\\mid h\)^\{\-\\alpha\},\(1\)whereZC​D​\(h\)Z\_\{CD\}\(h\)normalizes over the vocabulary\.

Rewriting Eq\.[1](https://arxiv.org/html/2608.12913#S2.E1)asπC​D​\(x∣h\)∝πp​\(x∣h\)​\(πp​\(x∣h\)πq​\(x∣h\)\)α\\pi\_\{CD\}\(x\\mid h\)\\propto\\pi\_\{p\}\(x\\mid h\)\\left\(\\frac\{\\pi\_\{p\}\(x\\mid h\)\}\{\\pi\_\{q\}\(x\\mid h\)\}\\right\)^\{\\alpha\}separates the dominant expert term from the contrastive factor that a contrastive\-aware lightweight drafter must also model\.

The original CD plausibility constraint\([Li et al\. 2023](https://arxiv.org/html/2608.12913#bib.bib15)\)is also compatible with DCD: masking Eq\.[1](https://arxiv.org/html/2608.12913#S2.E1)to𝒫β​\(h\)=\{x:πp​\(x∣h\)≥β​maxx′​πp​\(x′∣h\)\}\\mathcal\{P\}\_\{\\beta\}\(h\)=\\\{x:\\pi\_\{p\}\(x\\mid h\)\\geq\\beta\\max\_\{x^\{\\prime\}\}\\pi\_\{p\}\(x^\{\\prime\}\\mid h\)\\\}and renormalizing still yields a valid speculative\-verification target\. Proposals outside the mask simply receive zero target probability, so losslessness is unchanged after renormalization\. Our experiments use the untruncated target to isolate proposal alignment without adding a threshold hyperparameter\.

#### Speculative Decoding\.

Speculative Decoding \(SD\)\([Leviathan et al\. 2023](https://arxiv.org/html/2608.12913#bib.bib14);[Chen et al\. 2023](https://arxiv.org/html/2608.12913#bib.bib2)\)draftsγ\\gammacandidate tokens and verifies them in parallel with the target model\. Its speed depends on draft\-target alignment; we useDK​L\(πdraft∥πtarget\)D\_\{KL\}\(\\pi\_\{\\text\{draft\}\}\\\|\\pi\_\{\\text\{target\}\}\)as a diagnostic proxy and report acceptance directly as mean accepted lengthLL\.

### 2\.2Diagnosing Proposal\-Side Alternatives in the Lightweight\-Drafter Regime

We compare three routes under lightweight drafting\. They differ only in where the amateur signal enters the speculative decoding pipeline: directly in proposal generation, indirectly through a contrastive\-aware lightweight proposer, or only during verification\.

#### Route 1: Direct amateur drafting\.

This route usesπq\\pi\_\{q\}for proposals even though the unnormalized CD score assigns coefficient−α\-\\alphatolog⁡πq​\(x∣h\)\\log\\pi\_\{q\}\(x\\mid h\)\. It therefore has both a distributional mismatch and a systems cost: serial proposal cost remains tied to the amateur, roughlyγ⋅tq\\gamma\\cdot t\_\{q\}before method\-specific bonus or delayed\-drafting terms\.

#### Route 2: Contrastive\-aware lightweight drafting\.

A lightweight drafter must approximateπp\\pi\_\{p\}and the extra contrastive factor\(πpπq\)α\\left\(\\frac\{\\pi\_\{p\}\}\{\\pi\_\{q\}\}\\right\)^\{\\alpha\}\. Training on the contrastive target increases modeling burden without removing baseline proposal error, and inference\-time reconstruction adds another error source\. Section[3\.1](https://arxiv.org/html/2608.12913#S3.SS1)shows that this added signal is usually weaker than the proposal error it must overcome\.

#### Route 3: Expert\-aligned proposals with contrastive verification\.

Decoupled Contrastive Decoding \(DCD\)proposesy~1:γ\\tilde\{y\}\_\{1:\\gamma\}without usingℳq\\mathcal\{M\}\_\{q\}in the serial proposal path\. The expert and amateur are coupled only during verification, where proposals are evaluated against the unchangedπC​D\\pi\_\{CD\}\. Algorithm[1](https://arxiv.org/html/2608.12913#algorithm1)summarizes one DCD round\. Appendix[B\.3](https://arxiv.org/html/2608.12913#A2.SS3)proves that replacing the proposer does not change the output distribution as long as speculative verification targets the normalized CD distributionπC​D\\pi\_\{CD\}; the proposer only changes acceptance rate and efficiency\. We instantiate the proposerEEwithEAGLE3\([Li et al\. 2025](https://arxiv.org/html/2608.12913#bib.bib18)\)for the main deployment experiments because it gives the strongest average operating point among our tested proposers\. This EAGLE3 instantiation is expert\-aligned; the same DCD verification rule can also wrap other amateur\-independent proposers, such as the N\-gram proposer\.

Data:

𝐲=\[⋯,y0\]\\mathbf\{y\}=\[\\cdots,y\_\{0\}\],

𝐡d​r​a​f​t\(0\)\\mathbf\{h\}\_\{draft\}^\{\(0\)\},

ℳp\\mathcal\{M\}\_\{p\},

ℳq\\mathcal\{M\}\_\{q\},

EE,

γ\\gamma,

α\\alpha
Result:updated prefix after one DCD round

y~0←y0\\tilde\{y\}\_\{0\}\\leftarrow y\_\{0\}1

//Draft generation

for*i←1i\\leftarrow 1toγ\\gamma*do2

πe\(i\),𝐡d​r​a​f​t\(i\)←E⁡\(𝐡d​r​a​f​t\(i−1\),y~i−1\)\\pi\_\{e\}^\{\(i\)\},\\mathbf\{h\}\_\{draft\}^\{\(i\)\}\\leftarrow E\(\\mathbf\{h\}\_\{draft\}^\{\(i\-1\)\},\\tilde\{y\}\_\{i\-1\}\)3

y~i∼πe\(i\)\\tilde\{y\}\_\{i\}\\sim\\pi\_\{e\}^\{\(i\)\}4

𝐲~←\[y~1,…,y~γ\]\\tilde\{\\mathbf\{y\}\}\\leftarrow\[\\tilde\{y\}\_\{1\},\\ldots,\\tilde\{y\}\_\{\\gamma\}\]5

//Parallel model evaluation

\[πp\(1\),…,πp\(γ\+1\)\]←ℳp​\(𝐲⊕𝐲~\)\[\\pi\_\{p\}^\{\(1\)\},\\ldots,\\pi\_\{p\}^\{\(\\gamma\+1\)\}\]\\leftarrow\\mathcal\{M\}\_\{p\}\(\\mathbf\{y\}\\oplus\\tilde\{\\mathbf\{y\}\}\)6

\[πq\(1\),…,πq\(γ\+1\)\]←ℳq​\(𝐲⊕𝐲~\)\[\\pi\_\{q\}^\{\(1\)\},\\ldots,\\pi\_\{q\}^\{\(\\gamma\+1\)\}\]\\leftarrow\\mathcal\{M\}\_\{q\}\(\\mathbf\{y\}\\oplus\\tilde\{\\mathbf\{y\}\}\)7

//Speculative accept or fallback

for*i←1i\\leftarrow 1toγ\\gamma*do8

compute

πC​D\(i\)\\pi\_\{CD\}^\{\(i\)\}from Eq\. \([1](https://arxiv.org/html/2608.12913#S2.E1)\) using

πp\(i\)\\pi\_\{p\}^\{\(i\)\}and

πq\(i\)\\pi\_\{q\}^\{\(i\)\}, and draw

ri∼𝒰⁡\(0,1\)r\_\{i\}\\sim\\mathcal\{U\}\(0,1\)9

k←min⁡\(\{i∣ri\>πC​D\(i\)​\(y~i\)πe\(i\)​\(y~i\)\}∪\{γ\+1\}\)k\\leftarrow\\min\\left\(\\left\\\{i\\mid r\_\{i\}\>\\frac\{\\pi\_\{CD\}^\{\(i\)\}\(\\tilde\{y\}\_\{i\}\)\}\{\\pi\_\{e\}^\{\(i\)\}\(\\tilde\{y\}\_\{i\}\)\}\\right\\\}\\cup\\\{\\gamma\+1\\\}\\right\)10

if*k≤γk\\leq\\gamma*then11

accept

y~1:k−1\\tilde\{y\}\_\{1:k\-1\}12

Pr​\(x\)←Normalize⁡\(max⁡\(0,πC​D\(k\)​\(x\)−πe\(k\)​\(x\)\)\)P\_\{r\}\(x\)\\leftarrow\\mathrm\{Normalize\}\(\\max\(0,\\pi\_\{CD\}^\{\(k\)\}\(x\)\-\\pi\_\{e\}^\{\(k\)\}\(x\)\)\)13

sample

yk∼Pry\_\{k\}\\sim P\_\{r\}and return

𝐲⊕y~1:k−1⊕yk\\mathbf\{y\}\\oplus\\tilde\{y\}\_\{1:k\-1\}\\oplus y\_\{k\}14

else15

compute

πC​D\(γ\+1\)\\pi\_\{CD\}^\{\(\\gamma\+1\)\}from Eq\. \([1](https://arxiv.org/html/2608.12913#S2.E1)\) using

πp\(γ\+1\)\\pi\_\{p\}^\{\(\\gamma\+1\)\}and

πq\(γ\+1\)\\pi\_\{q\}^\{\(\\gamma\+1\)\}16

sample

yγ\+1∼πC​D\(γ\+1\)y\_\{\\gamma\+1\}\\sim\\pi\_\{CD\}^\{\(\\gamma\+1\)\}and return

𝐲⊕y~1:γ⊕yγ\+1\\mathbf\{y\}\\oplus\\tilde\{y\}\_\{1:\\gamma\}\\oplus y\_\{\\gamma\+1\}17

Algorithm 1Decoupled Contrastive Decoding \(One Round\)DCD separates distributional correctness from proposal efficiency\. Because verification still targetsπC​D\\pi\_\{CD\}, changing the proposer does not change the output distribution of vanilla CD under standard speculative sampling\. The change falls on the serial proposal path: DCD replaces the amateur cost of roughlyγ⋅tq\\gamma\\cdot t\_\{q\}with the cost of the chosen proposer, namelyγ⋅te\\gamma\\cdot t\_\{e\}for EAGLE3 withte≪tqt\_\{e\}\\ll t\_\{q\}, or near\-zero measured proposal latency for the N\-gram variant\.

### 2\.3Alpha\-Robustness Intuition

Effective Draft Alignment means that the proposer places more mass than the amateur on tokens where the expert has a larger log\-likelihood advantage\. Under this condition, the amateur proposal’s draft\-target KL grows faster withα\\alphathan an expert\-aligned drafter’s KL\. This slope comparison suggests that expert\-aligned DCD can lose accepted length more slowly even when SCD or CoS start with a higher acceptance baseline\. Appendix[B](https://arxiv.org/html/2608.12913#A2)proves the KL\-slope inequality and connects it to the observed robustness trend\.

## 3Experiments

The experiments follow the design question from Section[2\.2](https://arxiv.org/html/2608.12913#S2.SS2)\. We first ask whether lightweight drafters benefit from absorbing the contrastive signal, then evaluate the expert\-aligned DCD route at deployment scale and explain its speed source\.

#### Shared setting\.

Unless noted otherwise, we use 200 samples per dataset \(or the full dataset if smaller\), greedy decoding \(T=0T\{=\}0\), draft lengthγ=5\\gamma\{=\}5, and chain\-style, greedy drafting for EAGLE3 proposers\. Main deployment tables report three\-run averages; diagnostics and ablation studies use single\-run traces\. Runtime flags are listed in Appendix[H](https://arxiv.org/html/2608.12913#A8)\.

\(a\) Contrastive signal

distribution

\[0,0\.25\)\[0,0\.25\)\[0\.25,0\.5\)\[0\.25,0\.5\)\[0\.5,1\)\[0\.5,1\)\[1,2\)\[1,2\)\[2,4\)\[2,4\)\[4,8\)\[4,8\)≥8\\geq 800202040406060Signal bucketShare \(%\)

\(b\) Effect by signal and proposal error

<0\.05<0\.050\.050\.05–0\.10\.10\.10\.1–0\.20\.20\.20\.2–0\.50\.50\.50\.5–1111–22≥2\\geq 2<0\.25<0\.250\.250\.25–0\.50\.50\.50\.5–1111–2222–4444–88≥8\\geq 8KL\(πe∥πp\)\(\\pi\_\{e\}\\,\\\|\\,\\pi\_\{p\}\)Signal\-0\.2\-0\.100\.10\.2Δ\\DeltaTop\-1

\(c\) KL\(πe∥πp\)\(\\pi\_\{e\}\\,\\\|\\,\\pi\_\{p\}\)

distribution

<0\.05<0\.050\.050\.05–0\.10\.10\.10\.1–0\.20\.20\.20\.2–0\.50\.50\.50\.5–1111–22≥2\\geq 20010102020303040405050KL\(πe∥πp\)\(\\pi\_\{e\}\\,\\\|\\,\\pi\_\{p\}\)bucketShare \(%\)

Figure 2:Contrastive\-aware lightweight drafting has only a small favorable region to exploit\. Most positions have weak signal \(81\.1% below 1\.0\), gains concentrate in high\-signal/low\-error cells, and 48\.7% have KL\(πe∥πp\)≥2\.0\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)\\geq 2\.0\. Black dots mark cells with fewer than 100 positions\. Signal is\|log⁡\(πp​\(x⋆∣h\)/πq​\(x⋆∣h\)\)\|\|\\log\(\\pi\_\{p\}\(x^\{\\star\}\\mid h\)/\\pi\_\{q\}\(x^\{\\star\}\\mid h\)\)\|atx⋆=arg⁡maxx​πC​D​\(x∣h\)x^\{\\star\}=\\arg\\max\_\{x\}\\pi\_\{CD\}\(x\\mid h\);Δ\\DeltaTop\-1 compares contrastive\-aligned and expert\-aligned Top\-1 agreement withx⋆x^\{\\star\}\.\(a\) Effect by signal and proposal error

<0\.01<0\.010\.010\.01–0\.050\.050\.050\.05–0\.10\.10\.10\.1–0\.20\.20\.20\.2–0\.50\.50\.50\.5–1111–22≥2\\geq 2<0\.25<0\.250\.250\.25–0\.50\.50\.50\.5–1111–2222–4444–88≥8\\geq 8KL\(πr∥πp\)\(\\pi\_\{r\}\\,\\\|\\,\\pi\_\{p\}\)Signal\-0\.4\-0\.200\.20\.4Δ\\DeltaTop\-1

\(b\) Proposer\-side KL marginal

<0\.01<0\.010\.010\.01–0\.050\.050\.050\.05–0\.10\.10\.10\.1–0\.20\.20\.20\.2–0\.50\.50\.50\.5–1111–22≥2\\geq 20010102020303040405050KL\(πr∥πp\)\(\\pi\_\{r\}\\,\\\|\\,\\pi\_\{p\}\)bucketShare \(%\)

Figure 3:Even a stronger same\-family proposer leaves gains confined to high\-signal, low\-KL regions, with negative overall effect\.Δ\\DeltaTop\-1 is the CD\-aware reconstruction’s Top\-1 agreement gain over the unmodified 3B proposer\.
### 3\.1Why Not Contrastive\-Aware Drafting?

This subsection examines Route 2 from Section[2\.2](https://arxiv.org/html/2608.12913#S2.SS2): moving the contrastive signal into lightweight proposal generation before verification\. We first characterize where such alignment could help, then test whether lightweight drafters can exploit that region under matched controls, and finally examine whether the negative result persists with a substantially stronger proposer\.

#### Signal–Proposal\-Error Diagnostic\.

Before testing contrastive\-aware drafting, we first ask whether the contrastive signal occupies a sufficiently favorable region relative to baseline proposal error\. At each position, letx⋆=arg⁡maxx​πC​D​\(x∣h\)x^\{\\star\}=\\arg\\max\_\{x\}\\pi\_\{CD\}\(x\\mid h\)\. We measure the contrastive signal magnitude as\|log⁡πp​\(x⋆∣h\)πq​\(x⋆∣h\)\|\\left\|\\log\\frac\{\\pi\_\{p\}\(x^\{\\star\}\\mid h\)\}\{\\pi\_\{q\}\(x^\{\\star\}\\mid h\)\}\\right\|, baseline expert\-side proposal error asDK​L\(πe∥πp\)D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)for the expert\-aligned drafterπe\\pi\_\{e\}, andΔ\\DeltaTop\-1 as the difference between contrastive\-aligned and expert\-aligned Top\-1 agreement withx⋆x^\{\\star\}\. Figure[2](https://arxiv.org/html/2608.12913#S3.F2)aggregates these quantities, count\-weighted, over 24 configurations:Llama\-3,Llama\-EFT, andQwen3, each evaluated on GSM8K and MMLU atα∈\{0\.1,0\.3,0\.5,1\.0\}\\alpha\\in\\\{0\.1,0\.3,0\.5,1\.0\\\}\.

Across all pooled positions in this count\-weighted diagnostic, 81\.1% have signal below1\.01\.0and 48\.7% haveDK​L\(πe∥πp\)≥2\.0D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)\\geq 2\.0; positiveΔ\\DeltaTop\-1 values concentrate in high\-signal, low\-error cells\. Thus, the tested regime leaves only a relatively small favorable region for contrastive\-aware proposal alignment, while substantial baseline proposal error is widespread\. We useΔ\\DeltaTop\-1 only as a position\-level diagnostic; whether these local gains translate into accepted\-length improvements is tested directly in the matched experiments below\.

#### Matched Lightweight Drafting Experiments\.

We next test whether a lightweight drafter can exploit this signal in a controlled setting\. Both variants use the matchedLlama\-3/Qwen3pairs on GSM8K/MMLU, with the dual\-input EAGLE architecture, ShareGPT training data, and training recipe fixed\. The direct variant learns the contrastive target itself; the decomposed variant separately learns expert\- and amateur\-aligned distributions and recombines them at inference time\.

#### Direct Contrastive\-Aligned Drafter\.

The most direct realization trains a dual\-input EAGLE drafter to model the contrastive residual in Eq\.[1](https://arxiv.org/html/2608.12913#S2.E1):

πC​D​\(x∣h\)πp​\(x∣h\)\\displaystyle\\frac\{\\pi\_\{CD\}\(x\\mid h\)\}\{\\pi\_\{p\}\(x\\mid h\)\}∝\(πp​\(x∣h\)πq​\(x∣h\)\)α=eα​s​\(x∣h\),\\displaystyle\\propto\\left\(\\frac\{\\pi\_\{p\}\(x\\mid h\)\}\{\\pi\_\{q\}\(x\\mid h\)\}\\right\)^\{\\alpha\}=e^\{\\alpha s\(x\\mid h\)\},s⁡\(x∣h\)\\displaystyle s\(x\\mid h\)=log⁡πp​\(x∣h\)πq​\(x∣h\)\.\\displaystyle=\\log\\frac\{\\pi\_\{p\}\(x\\mid h\)\}\{\\pi\_\{q\}\(x\\mid h\)\}\.We vary both train\-α\\alphaand infer\-α\\alphaover\{0\.0,0\.1,0\.3,0\.5\}\\\{0\.0,0\.1,0\.3,0\.5\\\}while holding the remaining setup fixed; train\-α=0\\alpha=0recovers the expert\-aligned training target\. Table[1](https://arxiv.org/html/2608.12913#S3.T1)shows no consistent gain from non\-zero train\-α\\alpha: only Llama\-3/GSM8K has a small gain averaged over the four infer\-α\\alphavalues \(\+0\.015\+0\.015inL¯\\overline\{L\}, or0\.77%0\.77\\%\), whereas train\-α=0\\alpha=0is best in the other three settings\. Thus, in this matched control, directly learning the contrastive residual does not reliably translate into accepted\-length gains\. The full matrix is in Appendix[A\.2](https://arxiv.org/html/2608.12913#A1.SS2)\.

Model/DataBesttrain\-α\\alphaAvgΔ​L¯\\Delta\\overline\{L\}vsα=0\\alpha\{=\}0Avg % gainvsα=0\\alpha\{=\}0α=0\\alpha\{=\}0winsLlama\-3/GSM8K0\.1\+0\.015\+0\.77%0/4Llama\-3/MMLU0\.0\+0\.000\+0\.00%3/4Qwen3/GSM8K0\.0\+0\.000\+0\.00%3/4Qwen3/MMLU0\.0\+0\.000\+0\.00%4/4Table 1:Cross\-α\\alphacontrol: contrastive\-target training does not consistently improveL¯\\overline\{L\}\. Best train\-α\\alphamaximizes averageL¯\\overline\{L\}; only Llama\-3/GSM8K shows a small non\-zero gain\. Full matrix in Appendix[A\.2](https://arxiv.org/html/2608.12913#A1.SS2)\.
#### Decomposed Dual\-Drafter\.

Directly modelingπC​D\\pi\_\{CD\}may be unnecessarily difficult, so we also test a more favorable decomposition that separates expert\- and amateur\-side approximation before recombining the two analytically at inference time,

π^C​D​\(x∣h\)∝πe​\(x∣h\)1\+α​πf​\(x∣h\)−α,\\hat\{\\pi\}\_\{CD\}\(x\\mid h\)\\propto\\pi\_\{e\}\(x\\mid h\)^\{1\+\\alpha\}\\pi\_\{f\}\(x\\mid h\)^\{\-\\alpha\},whereπe\\pi\_\{e\}andπf\\pi\_\{f\}are the expert\- and amateur\-aligned draft distributions\. This separates expert\- and amateur\-side approximation, but it amplifies expert\-side log\-approximation error by1\+α1\+\\alpha; Appendix[A\.1](https://arxiv.org/html/2608.12913#A1.SS1)provides the reconstruction derivation\.

Even under this more favorable formulation, Approximate Dual\-Drafter underperforms expert\-aligned drafting at every reportedα\\alphain mean accepted lengthLL, draft\-to\-targetDK​L\(⋅∥πC​D\)D\_\{KL\}\(\\cdot\\\|\\pi\_\{CD\}\), and Top\-1 Match, with gaps widening asα\\alphaincreases \(Appendix Table[6](https://arxiv.org/html/2608.12913#A1.T6)\)\. At the position level,π^C​D\\hat\{\\pi\}\_\{CD\}is closer toπC​D\\pi\_\{CD\}thanπe\\pi\_\{e\}at only 31\.9–39\.7% of positions\.

#### Stress Test with a Stronger Full\-LM Proposer\.

The preceding negative results could reflect limitations of the lightweight proposer rather than the contrastive objective itself\. We therefore repeat the position\-level diagnostic withLlama\-3\.2\-3B\-Instructas a stronger Llama\-family proposer, while fixing theLlama\-3\.1\-8B\-Instruct/Llama\-3\.2\-1B\-Instructexpert–amateur pair\. On shared response histories, we apply contrastive correction using true amateur logits\. This is an optimistic offline check on GSM8K and MMLU atα∈\{0\.1,0\.5\}\\alpha\\in\\\{0\.1,0\.5\\\}, rather than an online rollout evaluation\. Figure[3](https://arxiv.org/html/2608.12913#S3.F3)shows that the stronger proposer enlarges the favorable high\-signal, low\-error pocket, but the setting\-level meanΔ\\DeltaTop\-1 remains negative in all four settings \(see also Appendix[A\.3](https://arxiv.org/html/2608.12913#A1.SS3)\)\. The stronger proposer therefore enlarges the favorable region in this check without reversing the aggregate disadvantage\.

#### Takeaway\.

Neither direct nor decomposed contrastive\-aware lightweight drafting consistently outperforms expert\-aligned drafting in the tested regime\. The stronger full\-LM stress test enlarges the favorable region but does not reverse the position\-level disadvantage\. We therefore keep proposals expert\-aligned and amateur\-independent in the system experiments, applying contrastive scoring only during verification\.

### 3\.2Deployment\-Level Setup

#### Datasets and model pairs\.

Following[Fu et al\. 2025](https://arxiv.org/html/2608.12913#bib.bib7), we evaluate HumanEval\([Chen et al\. 2021](https://arxiv.org/html/2608.12913#bib.bib3)\), GSM8K\([Cobbe et al\. 2021](https://arxiv.org/html/2608.12913#bib.bib5)\), MMLU\([Hendrycks et al\. 2021](https://arxiv.org/html/2608.12913#bib.bib10)\), and CNN/DM\([See et al\. 2017](https://arxiv.org/html/2608.12913#bib.bib24)\)under three 8B\-class configurations:Llama\-3,Qwen3, andLlama\-EFT\. Their expert and amateur pairs areLlama\-3\.1\-8B\-Instruct/Llama\-3\.2\-1B\-Instruct,Qwen3\-8B/Qwen3\-1\.7B, andLlama\-3\.1\-8B\-Instruct/Llama\-3\.1\-8B\. Baselines are vanilla CD, SCD, and CoS\([Fu et al\. 2025](https://arxiv.org/html/2608.12913#bib.bib7)\)\. DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}uses community\-released EAGLE3 drafters and requires no new drafter training\. DCDngram\{\}\_\{\\textsc\{ngram\}\}is reported as an alternative DCD instantiation with a simple prefix\-matching N\-gram proposer\. Appendix[G](https://arxiv.org/html/2608.12913#A7)lists the checkpoint identifiers\.

#### Fairness protocol\.

All deployment methods use the same SGLang stack, single NVIDIA H200 GPU, prompts, maximum generation lengths, and expert and amateur pair within each operating point\. The main speed tables use 200 samples per dataset with maximum generation length 256 and report three\-run averages; the task\-level accuracy check uses 1000 samples with maximum generation length 1024\. Appendix Table[13](https://arxiv.org/html/2608.12913#A5.T13)gives the draft\-length tuning check\.

#### Operating points and metrics\.

The main greedy andT=1T=1comparisons useα∈\{0\.1,0\.5\}\\alpha\\in\\\{0\.1,0\.5\\\}; the latency decomposition usesα=0\.1\\alpha=0\.1; and the ablations sweepα\\alphaandγ\\gamma\. We report tokens/sec and speedup over vanilla CD\. When diagnosing speed sources, we also report accepted length and latency components\. Appendix[H](https://arxiv.org/html/2608.12913#A8)gives runtime details\.

### 3\.3System Validation of the Diagnostic: DCD

MethodHEGSMMMLUCNN/DMAvg\.Llama\-3α=0\.1\\alpha=0\.1SCD1\.52×\\times1\.53×\\times1\.26×\\times1\.40×\\times1\.43×\\timesCoS1\.71×\\times1\.77×\\times1\.29×\\times1\.37×\\times1\.54×\\timesDCDngram\{\}\_\{\\textsc\{ngram\}\}1\.48×\\times1\.43×\\times1\.29×\\times2\.07×\\times1\.56×\\timesDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}2\.07×\\times1\.84×\\times1\.60×\\times1\.80×\\times1\.83×\\timesα=0\.5\\alpha=0\.5SCD1\.59×\\times1\.43×\\times1\.21×\\times1\.12×\\times1\.34×\\timesCoS1\.67×\\times1\.56×\\times1\.20×\\times1\.13×\\times1\.40×\\timesDCDngram\{\}\_\{\\textsc\{ngram\}\}1\.39×\\times1\.34×\\times1\.55×\\times2\.10×\\times1\.59×\\timesDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}1\.89×\\times1\.66×\\times1\.44×\\times1\.59×\\times1\.65×\\timesQwen3α=0\.1\\alpha=0\.1SCD1\.38×\\times1\.35×\\times0\.99×\\times1\.05×\\times1\.19×\\timesCoS1\.45×\\times1\.33×\\times1\.17×\\times1\.14×\\times1\.27×\\timesDCDngram\{\}\_\{\\textsc\{ngram\}\}1\.45×\\times1\.35×\\times1\.17×\\times1\.14×\\times1\.28×\\timesDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}2\.30×\\times2\.01×\\times1\.66×\\times1\.85×\\times1\.95×\\timesα=0\.5\\alpha=0\.5SCD1\.23×\\times1\.24×\\times0\.88×\\times0\.82×\\times1\.05×\\timesCoS1\.25×\\times1\.31×\\times0\.97×\\times0\.79×\\times1\.08×\\timesDCDngram\{\}\_\{\\textsc\{ngram\}\}1\.32×\\times1\.28×\\times1\.17×\\times1\.00×\\times1\.19×\\timesDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}1\.98×\\times1\.85×\\times1\.60×\\times1\.39×\\times1\.71×\\timesLlama\-EFTα=0\.1\\alpha=0\.1SCD1\.20×\\times1\.16×\\times1\.06×\\times1\.03×\\times1\.12×\\timesCoS1\.29×\\times1\.22×\\times1\.01×\\times1\.06×\\times1\.15×\\timesDCDngram\{\}\_\{\\textsc\{ngram\}\}1\.43×\\times1\.31×\\times1\.32×\\times2\.06×\\times1\.52×\\timesDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}2\.03×\\times1\.74×\\times1\.55×\\times1\.74×\\times1\.77×\\timesα=0\.5\\alpha=0\.5SCD1\.19×\\times1\.14×\\times0\.96×\\times0\.92×\\times1\.05×\\timesCoS1\.31×\\times1\.15×\\times0\.98×\\times0\.97×\\times1\.10×\\timesDCDngram\{\}\_\{\\textsc\{ngram\}\}1\.55×\\times1\.35×\\times1\.31×\\times2\.11×\\times1\.58×\\timesDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}2\.06×\\times1\.62×\\times1\.61×\\times1\.61×\\times1\.72×\\timesTable 2:DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}is strongest on average in every model family; DCDngram\{\}\_\{\\textsc\{ngram\}\}also stays above vanilla CD on average\. Avg\. averages datasets; highlights mark DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}\.#### Main comparison\.

Table[2](https://arxiv.org/html/2608.12913#S3.T2)shows DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}has the highest average speedup for every family and bothα\\alphavalues\. At the harderQwen3,α=0\.5\\alpha=0\.5setting, SCD and CoS drop below 1\.0×\\timeson MMLU/CNN\-DM, while DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}reaches 1\.60×\\times/1\.39×\\times\. DCDngram\{\}\_\{\\textsc\{ngram\}\}is included as a proposer\-agnostic sanity check: it is weaker than EAGLE3 but remains above vanilla CD on average\. Appendix[E\.1](https://arxiv.org/html/2608.12913#A5.SS1)gives theT=1T\{=\}1breakdown, Appendix[F\.1](https://arxiv.org/html/2608.12913#A6.SS1)reports multi\-request serving, and Appendix[F\.2](https://arxiv.org/html/2608.12913#A6.SS2)gives the KV\-cache result\.

This pattern validates the proposal\-path diagnostic at system level\. All methods use the same contrastive target and the same expert and amateur models, but DCD alone moves the expensive amateur out of the serial proposal path\. The next two subsections separate the two remaining concerns: Figure[4](https://arxiv.org/html/2608.12913#S3.F4)checks that the verified CD target is preserved in task accuracy, and the latency decomposition in Section[3\.5](https://arxiv.org/html/2608.12913#S3.SS5)shows why the cheaper proposal path can dominate even when SCD or CoS accept more draft tokens\.

SettingARExpert SDDCDngram\{\}\_\{\\textsc\{ngram\}\}DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}GSM8K1\.27×\\times2\.55×\\times1\.43×\\times1\.75×\\timesMMLU1\.31×\\times1\.88×\\times1\.34×\\times1\.45×\\timesTable 3:Atα=0\\alpha\{=\}0, DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}remains faster than vanilla CD and AR on Llama\-3\. Expert SD uses the same EAGLE3 drafter\.
#### α=0\\alpha=0boundary\.

Atα=0\\alpha=0, CD reduces to the expert distribution\. Table[3](https://arxiv.org/html/2608.12913#S3.T3)positions DCD against autoregressive decoding and expert\-only speculative decoding with the same EAGLE3 drafter\. DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}remains above vanilla CD and autoregressive decoding at 1\.75×\\times/1\.45×\\timeson GSM8K/MMLU, while the expert\-only row marks the expected upper bound when contrastive verification is removed\.

#### 70B\-class validation\.

In the greedy\-only 70B extension, Table[4](https://arxiv.org/html/2608.12913#S3.T4)uses the sameα∈\{0\.1,0\.5\}\\alpha\\in\\\{0\.1,0\.5\\\}operating points as the main comparison\. DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}remains fastest on average, reaching 2\.02×\\timesatα=0\.1\\alpha=0\.1and 1\.96×\\timesatα=0\.5\\alpha=0\.5\. Appendix[F\.4](https://arxiv.org/html/2608.12913#A6.SS4)gives the per\-dataset results\.

Average speedup over vanilla CDMethodα=0\.1\\alpha=0\.1α=0\.5\\alpha=0\.5SCD1\.79×\\times1\.69×\\timesCoS1\.89×\\times1\.75×\\timesDCDngram\{\}\_\{\\textsc\{ngram\}\}1\.46×\\times1\.44×\\timesDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}2\.02×\\times1\.96×\\timesTable 4:DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}stays near2×2\\timesaverage speedup in the greedy 70B extension\. Values average four datasets; full results are in Appendix[F\.4](https://arxiv.org/html/2608.12913#A6.SS4)\.

### 3\.4Task\-Level Consistency Check

DCD changes only proposals, so the efficiency gains in Table[2](https://arxiv.org/html/2608.12913#S3.T2)should not be interpreted as a new decoding objective\. Verification still targetsπC​D\\pi\_\{CD\}; Appendix[B\.3](https://arxiv.org/html/2608.12913#A2.SS3)proves losslessness\. Figure[4](https://arxiv.org/html/2608.12913#S3.F4)shows the GSM8K greedy sweep forLlama\-3withγ=5\\gamma\{=\}5and a matched DCDngram\{\}\_\{\\textsc\{ngram\}\}comparison; Appendix[E\.2](https://arxiv.org/html/2608.12913#A5.SS2)gives the full GSM8K/MMLU sweep under bothT=0T\{=\}0andT=1T\{=\}1\.

000\.20\.20\.40\.40\.60\.60\.80\.811757578788181848487879090Contrastive Strength \(α\\alpha\)Accuracy \(%\)GSM8K,T=0T\{=\}0

DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}DCDngram\{\}\_\{\\textsc\{ngram\}\}SCDCoSVanilla CDAR

Figure 4:DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}tracks vanilla CD accuracy acrossα\\alphaon Llama\-3 GSM8K \(γ=5\\gamma\{=\}5\)\. Error bars are 95% confidence intervals; the dashed line is AR\.#### Consistency\.

Figure[4](https://arxiv.org/html/2608.12913#S3.F4)shows that DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}tracks vanilla CD acrossα\\alphaon GSM8K under greedy decoding, with error bars largely overlapping across the operating range\. It indicates that DCD preserves the contrastive target while changing only the proposal mechanism\.

### 3\.5Source of the DCD Speedup

Method𝑳\\boldsymbol\{L\}𝒕𝒆\\boldsymbol\{t\_\{e\}\}𝒕𝒑\\boldsymbol\{t\_\{p\}\}𝒕𝒒\\boldsymbol\{t\_\{q\}\}SpeedTheo\.Llama\-3\(8B\-Ins / 1B\-Ins\)CD0\.00\-6\.953\.831\.00x1\.00xSCD2\.45\-6\.843\.471\.34x1\.48xCoS2\.45\-7\.083\.591\.34x1\.48xDCDngram\{\}\_\{\\textsc\{ngram\}\}0\.480\.007\.044\.061\.36x1\.43xDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}1\.440\.637\.984\.381\.50x1\.70xQwen3\(8B / 1\.7B\)CD0\.00\-8\.476\.521\.00x1\.00xSCD2\.49\-9\.066\.531\.14x1\.20xCoS2\.49\-9\.156\.631\.17x1\.23xDCDngram\{\}\_\{\\textsc\{ngram\}\}0\.410\.009\.077\.231\.22x1\.30xDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}1\.480\.6110\.157\.931\.61x1\.76xLlama\-EFT\(8B\-Ins / 8B\)CD0\.00\-7\.197\.251\.00x1\.00xSCD2\.64\-6\.776\.471\.20x1\.28xCoS2\.64\-7\.677\.381\.11x1\.17xDCDngram\{\}\_\{\\textsc\{ngram\}\}0\.490\.006\.987\.121\.45x1\.52xDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}1\.460\.607\.427\.501\.79x1\.98xTable 5:DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}beats SCD/CoS by lowering MMLU serial proposal latency despite smallerLL\. Results useT=0T\{=\}0,α=0\.1\\alpha\{=\}0\.1,γ=5\\gamma\{=\}5;tet\_\{e\},tpt\_\{p\}, andtqt\_\{q\}are per\-step ms\.Table[5](https://arxiv.org/html/2608.12913#S3.T5)decomposes MMLU speedup usingS≈\(L\+1\)/TiterS\\approx\(L\+1\)/T\_\{\\text\{iter\}\}and reports accepted length plus drafter, expert, and amateur latencies\. The “Theo\.” column evaluates this coarse serial model with the measured components; measured speed remains the deployment metric\.

Serial proposal cost explains the speedup\. Across the three MMLU settings, SCD/CoS spend 3\.5 to 7\.4ms per proposal step, while DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}uses about 0\.6ms\. OnLlama\-3, SCD reachesL=2\.45L=2\.45and DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}reachesL=1\.44L=1\.44, yet DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}is faster because cheaper proposals offset the shorter accepted length\. Appendix[D](https://arxiv.org/html/2608.12913#A4)gives the multiplicative breakdown into acceptance and serial\-cost factors\.

000\.10\.10\.20\.20\.30\.30\.40\.40\.50\.56060707080809090100100Contrastive Strength \(α\\alpha\)Llama\-3MMLU000\.10\.10\.20\.20\.30\.30\.40\.40\.50\.5Contrastive Strength \(α\\alpha\)Qwen3MMLU000\.10\.10\.20\.20\.30\.30\.40\.40\.50\.5Contrastive Strength \(α\\alpha\)Llama\-3GSM8K000\.10\.10\.20\.20\.30\.30\.40\.40\.50\.5Contrastive Strength \(α\\alpha\)Qwen3GSM8KDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}SCDCoS

Figure 5:DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}loses accepted length more mildly as contrastive strength increases\. Relative mean accepted length is normalized toα=0\\alpha\{=\}0in each panel\.
### 3\.6Ablation Studies

We test theα\\alpha\-robustness intuition behind DCD\.

Figure[5](https://arxiv.org/html/2608.12913#S3.F5)normalizesLLto 100% atα=0\\alpha\{=\}0\. This view separates robustness from absolute acceptance rate: SCD or CoS can start from a higherLLatα=0\\alpha\{=\}0, while the diagnostic tracks how quickly each proposal distribution drifts away from the contrastive target as the amateur penalty strengthens\.

DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}degrades more gradually than amateur\-coupled baselines in all four panels\. OnLlama\-3MMLU, the gap is moderate: atα=0\.5\\alpha\{=\}0\.5, DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}retains 79\.5% of itsα=0\\alpha\{=\}0accepted length, while SCD and CoS retain about 74%\. The separation becomes larger onQwen3MMLU, where SCD retains 62\.9% and DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}retains 77\.4%\. This empirical pattern is consistent with the alignment intuition: as the contrastive target moves away from the amateur distribution, an expert\-aligned lightweight drafter can lose acceptance more slowly than a proposal path tied to the amateur\.

The GSM8K panels show the same trend but with different severity\.Llama\-3GSM8K is relatively stable for all methods, yet DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}still remains closest to itsα=0\\alpha\{=\}0acceptance level atα=0\.5\\alpha\{=\}0\.5\(95\.1% versus about 88% for SCD/CoS\)\.Qwen3GSM8K is harder: the amateur\-coupled curves fall to 65\.8%, while DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}stays at 81\.9%\. The cross\-task consistency matters because the main speedups in Table[2](https://arxiv.org/html/2608.12913#S3.T2)come from both proposal cost and acceptance; robustness inLLprevents the serial\-cost advantage from being erased at largerα\\alpha\. Appendix Figure[11](https://arxiv.org/html/2608.12913#A5.F11)and Table[14](https://arxiv.org/html/2608.12913#A5.T14)give theγ\\gammaand throughput\-grid details\.

## 4Related Work

Speculative Decoding\.Speculative decoding accelerates generation with draft\-then\-verify\([Leviathan et al\. 2023](https://arxiv.org/html/2608.12913#bib.bib14);[Chen et al\. 2023](https://arxiv.org/html/2608.12913#bib.bib2)\)\. Recent work improves proposals through parallel heads, speculation trees and trajectories, self\-speculation, distillation, and graph\-structured acceptance\([Cai et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib1);[Miao et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib20);[Fu et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib8);[Elhoushi et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib6);[Zhang et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib28);[Zhou et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib31);[Gong et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib9)\)\. The EAGLE series\([Li et al\. 2024a](https://arxiv.org/html/2608.12913#bib.bib16);[Li et al\. 2024b](https://arxiv.org/html/2608.12913#bib.bib17);[Li et al\. 2025](https://arxiv.org/html/2608.12913#bib.bib18)\)drafts at the feature level\. A contrastive target raises a distinct question: should the drafter imitateπC​D\\pi\_\{CD\}, or should the amateur appear only in verification?

Contrastive Decoding\.Contrastive Decoding \(CD\)\([Li et al\. 2023](https://arxiv.org/html/2608.12913#bib.bib15)\)amplifies the expert–amateur gap\. Later work applies this idea to layer contrast, contextual understanding, noisy retrieval, vision\-language hallucination mitigation, and tuning\-by\-proxy\([Chuang et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib4);[Zhao et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib29);[Kim et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib12);[Leng et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib13);[Wang et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib25);[Mitchell et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib21);[Liu et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib19)\)\. CD still requires two model passes per token\.

Accelerating Contrastive Decoding\.Existing CD acceleration methods mostly retain the amateur in proposal generation\. SCD\([Yuan et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib27)\)and CoS\([Fu et al\. 2025](https://arxiv.org/html/2608.12913#bib.bib7)\)use it as a drafter and retain its serial cost\. Section[3\.1](https://arxiv.org/html/2608.12913#S3.SS1)finds no consistent benefit from adding the contrastive signal to a constrained lightweight proposer, motivating DCD’s expert\-aligned path\.

## 5Conclusion

Contrastive Decoding \(CD\) improves generation but adds an expensive amateur pass\. We ask whether speculative CD should use the contrastive signal during drafting or only verification\. Matched Cross\-α\\alphaand Approximate Dual\-Drafter diagnostics show that contrastive\-aware lightweight drafting does not consistently outperform expert\-aligned drafting: the correction is usually smaller than drafter error, which reconstruction can amplify\. Decoupled Contrastive Decoding \(DCD\) therefore uses an expert\-aligned lightweight proposer and reserves the amateur for unchanged CD verification, preserving vanilla CD’s output distribution\. Across the main 8B settings, EAGLE3\-based DCD yields 1\.65–1\.95×\\timesaverage greedy speedups and cuts MMLU proposal\-path latency by 5–12×\\timesrelative to amateur\-coupled proposal paths\.

## Limitations

Our evaluation follows the public benchmark mix and fixed decoding budgets used in the main study\. It does not exhaust every workload shape: very long contexts, multilingual or domain\-specific prompts, and adaptive choices ofα\\alpha, draft length, or the CD plausibility threshold may shift the best operating point\.

## References

- Cai et al\. \(2024\)Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D Lee, Deming Chen, and Tri Dao\. 2024\.MEDUSA: Simple LLM inference acceleration framework with multiple decoding heads\.In*Proceedings of the 41st International Conference on Machine Learning*, pages 5209–5235\.
- Chen et al\. \(2023\)Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean\-Baptiste Lespiau, Laurent Sifre, and John Jumper\. 2023\.Accelerating large language model decoding with speculative sampling\.*arXiv preprint arXiv:2302\.01318*\.
- Chen et al\. \(2021\)Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others\. 2021\.[Evaluating large language models trained on code](https://arxiv.org/abs/2107.03374)\.*Preprint*, arXiv:2107\.03374\.
- Chuang et al\. \(2024\)Yung\-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R\. Glass, and Pengcheng He\. 2024\.[DoLa: Decoding by contrasting layers improves factuality in large language models](https://openreview.net/forum?id=Th6NyL07na)\.In*The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024*\. OpenReview\.net\.
- Cobbe et al\. \(2021\)Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman\. 2021\.Training verifiers to solve math word problems\.*arXiv preprint arXiv:2110\.14168*\.
- Elhoushi et al\. \(2024\)Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed A Aly, Beidi Chen, and Carole\-Jean Wu\. 2024\.[LayerSkip: Enabling early exit inference and self\-speculative decoding](https://doi.org/10.18653/V1/2024.ACL-LONG.681)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2024, Bangkok, Thailand, August 11\-16, 2024*, pages 12622–12642\. Association for Computational Linguistics\.
- Fu et al\. \(2025\)Jiale Fu, Yuchu Jiang, Junkai Chen, Jiaming Fan, Xin Geng, and Xu Yang\. 2025\.Fast large language model collaborative decoding via speculation\.In*Forty\-second International Conference on Machine Learning*\.
- Fu et al\. \(2024\)Yichao Fu, Peter Bailis, Ion Stoica, and Hao Zhang\. 2024\.[Break the sequential dependency of LLM inference using lookahead decoding](https://openreview.net/forum?id=eDjvSFOkXw)\.In*Forty\-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21\-27, 2024*\. OpenReview\.net\.
- Gong et al\. \(2024\)Zhuocheng Gong, Jiahao Liu, Ziyue Wang, Pengfei Wu, Jingang Wang, Xunliang Cai, Dongyan Zhao, and Rui Yan\. 2024\.[Graph\-structured speculative decoding](https://doi.org/10.18653/V1/2024.FINDINGS-ACL.677)\.In*Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11\-16, 2024*, pages 11404–11415\. Association for Computational Linguistics\.
- Hendrycks et al\. \(2021\)Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt\. 2021\.Measuring massive multitask language understanding\.*Proceedings of the International Conference on Learning Representations \(ICLR\)*\.
- Huang et al\. \(2025\)Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu\. 2025\.[A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions](https://doi.org/10.1145/3703155)\.*ACM Trans\. Inf\. Syst\.*, 43\(2\):42:1–42:55\.
- Kim et al\. \(2024\)Youna Kim, Hyuhng Joon Kim, Cheonbok Park, Choonghyun Park, Hyunsoo Cho, Junyeob Kim, Kang Min Yoo, Sang\-goo Lee, and Taeuk Kim\. 2024\.[Adaptive contrastive decoding in retrieval\-augmented generation for handling noisy contexts](https://doi.org/10.18653/V1/2024.FINDINGS-EMNLP.136)\.In*Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12\-16, 2024*, pages 2421–2431\. Association for Computational Linguistics\.
- Leng et al\. \(2024\)Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing\. 2024\.Mitigating object hallucinations in large vision\-language models through visual contrastive decoding\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 13872–13882\.
- Leviathan et al\. \(2023\)Yaniv Leviathan, Matan Kalman, and Yossi Matias\. 2023\.Fast inference from transformers via speculative decoding\.In*International Conference on Machine Learning*, pages 19274–19286\. PMLR\.
- Li et al\. \(2023\)Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori B Hashimoto, Luke Zettlemoyer, and Mike Lewis\. 2023\.Contrastive decoding: Open\-ended text generation as optimization\.In*Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 12286–12312\.
- Li et al\. \(2024a\)Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang\. 2024a\.EAGLE\-2: Faster inference of language models with dynamic draft trees\.In*Empirical Methods in Natural Language Processing*\.
- Li et al\. \(2024b\)Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang\. 2024b\.EAGLE: Speculative sampling requires rethinking feature uncertainty\.In*International Conference on Machine Learning*\.
- Li et al\. \(2025\)Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang\. 2025\.EAGLE\-3: Scaling up inference acceleration of large language models via training\-time test\.In*Annual Conference on Neural Information Processing Systems*\.
- Liu et al\. \(2024\)Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi, and Noah A\. Smith\. 2024\.[Tuning language models by proxy](https://doi.org/10.48550/ARXIV.2401.08565)\.*CoRR*, abs/2401\.08565\.
- Miao et al\. \(2024\)Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, Chunan Shi, Zhuoming Chen, Daiyaan Arfeen, Reyna Abhyankar, and Zhihao Jia\. 2024\.SpecInfer: Accelerating large language model serving with tree\-based speculative inference and verification\.In*Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3*, pages 932–949\.
- Mitchell et al\. \(2024\)Eric Mitchell, Rafael Rafailov, Archit Sharma, Chelsea Finn, and Christopher D\. Manning\. 2024\.[An emulator for fine\-tuning large language models using small language models](https://openreview.net/forum?id=Eo7kv0sllr)\.In*The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024*\. OpenReview\.net\.
- O’Brien and Lewis \(2023\)Sean O’Brien and Mike Lewis\. 2023\.[Contrastive decoding improves reasoning in large language models](https://doi.org/10.48550/ARXIV.2309.09117)\.*CoRR*, abs/2309\.09117\.
- Plaat et al\. \(2024\)Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Bäck\. 2024\.[Reasoning with large language models, a survey](https://doi.org/10.48550/ARXIV.2407.11511)\.*CoRR*, abs/2407\.11511\.
- See et al\. \(2017\)Abigail See, Peter J\. Liu, and Christopher D\. Manning\. 2017\.[Get to the point: Summarization with pointer\-generator networks](https://doi.org/10.18653/v1/P17-1099)\.In*Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 1073–1083, Vancouver, Canada\. Association for Computational Linguistics\.
- Wang et al\. \(2024\)Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann\. 2024\.[Mitigating hallucinations in large vision\-language models with instruction contrastive decoding](https://doi.org/10.18653/V1/2024.FINDINGS-ACL.937)\.In*Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11\-16, 2024*, pages 15840–15853\. Association for Computational Linguistics\.
- Xia et al\. \(2024\)Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang, Yongqi Li, Tao Ge, Tianyu Liu, Wenjie Li, and Zhifang Sui\. 2024\.[Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding](https://doi.org/10.18653/v1/2024.findings-acl.456)\.In*Findings of the Association for Computational Linguistics: ACL 2024*, pages 7655–7671, Bangkok, Thailand and virtual meeting\. Association for Computational Linguistics\.
- Yuan et al\. \(2024\)Hongyi Yuan, Keming Lu, Fei Huang, Zheng Yuan, and Chang Zhou\. 2024\.Speculative contrastive decoding\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\)*, pages 56–64\.
- Zhang et al\. \(2024\)Jun Zhang, Jue Wang, Huan Li, Lidan Shou, Ke Chen, Gang Chen, and Sharad Mehrotra\. 2024\.[Draft& verify: Lossless large language model acceleration via self\-speculative decoding](https://doi.org/10.18653/V1/2024.ACL-LONG.607)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2024, Bangkok, Thailand, August 11\-16, 2024*, pages 11263–11282\. Association for Computational Linguistics\.
- Zhao et al\. \(2024\)Zheng Zhao, Emilio Monti, Jens Lehmann, and Haytham Assem\. 2024\.[Enhancing contextual understanding in large language models through contrastive decoding](https://doi.org/10.18653/V1/2024.NAACL-LONG.237)\.In*Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\), NAACL 2024, Mexico City, Mexico, June 16\-21, 2024*, pages 4225–4237\. Association for Computational Linguistics\.
- Zheng et al\. \(2024\)Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E\. Gonzalez, Clark W\. Barrett, and Ying Sheng\. 2024\.SGLang: Efficient execution of structured language model programs\.*Advances in Neural Information Processing Systems*, 37:62557–62583\.
- Zhou et al\. \(2024\)Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh, Sanjiv Kumar, Jean\-François Kagy, and Rishabh Agarwal\. 2024\.[DistillSpec: Improving speculative decoding via knowledge distillation](https://openreview.net/forum?id=rsY6J3ZaTF)\.In*The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024*\. OpenReview\.net\.

## Appendix AProposal\-Side Diagnostics

### A\.1Analysis of Contrastive\-Aligned Drafting Strategies

This subsection reports supplementary measurements for the contrastive\-aligned drafting results in Section[3\.1](https://arxiv.org/html/2608.12913#S3.SS1)\. It restates the matched\-control setup, then reports per\-position reconstruction checks and bucketed diagnostics over model groups, datasets, and contrastive strengths\. The matched control remains the two\-model\-pair, two\-dataset study in Section[3\.1](https://arxiv.org/html/2608.12913#S3.SS1); the later views are supplementary diagnostics\.

#### Measurement Methodology\.

We use two related measurement setups\. For the matched\-control KL and reconstruction checks referenced in the main text, we run per\-position forward passes on 200 GSM8K examples for theLlama\-3andQwen3model pairs under greedy decoding \(T=0T=0\) with draft lengthγ=5\\gamma=5\. We record full\-vocabulary logits from the expert modelℳp\\mathcal\{M\}\_\{p\}, the amateur modelℳq\\mathcal\{M\}\_\{q\}, and the expert\-aligned drafterEEinstantiated with EAGLE3\. We computeDK​L\(πp∥πC​D\)D\_\{KL\}\(\\pi\_\{p\}\\\|\\pi\_\{CD\}\)by constructingπC​D\\pi\_\{CD\}from Eq\.[1](https://arxiv.org/html/2608.12913#S2.E1)at each position, and we computeDK​L\(πe∥πp\)D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)directly from drafter and expert logits\.

For the supplementary bucketed regime analysis, we aggregate configurations across three model groups \(Llama\-3,Llama\-EFT, andQwen3\), two datasets \(GSM8K and MMLU\), and four contrastive strengths \(α∈\{0\.1,0\.3,0\.5,1\.0\}\\alpha\\in\\\{0\.1,0\.3,0\.5,1\.0\\\}\)\. We use the same shared evaluation convention as Section[3](https://arxiv.org/html/2608.12913#S3): 200 evaluation samples per dataset, greedy decoding \(T=0T=0\), and draft lengthγ=5\\gamma=5\. Herehhdenotes the current decoding history, andx⋆=arg⁡maxx​πC​D​\(x∣h\)x^\{\\star\}=\\arg\\max\_\{x\}\\pi\_\{CD\}\(x\\mid h\)denotes the CD top\-1 token at that position\. The per\-position contrastive\-signal statistic is the expert–amateur absolute log\-ratio at that token,\|log⁡πp​\(x⋆∣h\)πq​\(x⋆∣h\)\|\\left\|\\log\\frac\{\\pi\_\{p\}\(x^\{\\star\}\\mid h\)\}\{\\pi\_\{q\}\(x^\{\\star\}\\mid h\)\}\\right\|\. Table[7](https://arxiv.org/html/2608.12913#A1.T7)is count\-weighted: we sum the per\-bucket position counts over all configurations and divide by the total count\. The reported 81\.1% is therefore the sum of the first three buckets,\[0,0\.25\)\[0,0\.25\),\[0\.25,0\.5\)\[0\.25,0\.5\), and\[0\.5,1\)\[0\.5,1\)\.

#### Error Amplification in the Approximate Dual\-Drafter\.

A simple error decomposition explains why Path 2, the Approximate Dual\-Drafter, performs poorly\. Letπ^C​D​\(x\)∝πe​\(x\)1\+α⋅πf​\(x\)−α\\hat\{\\pi\}\_\{CD\}\(x\)\\propto\\pi\_\{e\}\(x\)^\{1\+\\alpha\}\\cdot\\pi\_\{f\}\(x\)^\{\-\\alpha\}be the combined distribution, whereπf\\pi\_\{f\}is the amateur\-aligned EAGLE head approximatingπq\\pi\_\{q\}\. Define the log approximation errorsϵp​\(x\)=log⁡\(πe​\(x\)/πp​\(x\)\)\\epsilon\_\{p\}\(x\)=\\log\(\\pi\_\{e\}\(x\)/\\pi\_\{p\}\(x\)\)andϵq​\(x\)=log⁡\(πf​\(x\)/πq​\(x\)\)\\epsilon\_\{q\}\(x\)=\\log\(\\pi\_\{f\}\(x\)/\\pi\_\{q\}\(x\)\)\.

###### Lemma A\.1\(Reconstruction Error Decomposition\)\.

The reconstructed distributionπ^C​D\\hat\{\\pi\}\_\{CD\}is an exponential tilt ofπC​D\\pi\_\{CD\}:

π^C​D​\(x\)\\displaystyle\\hat\{\\pi\}\_\{CD\}\(x\)=πC​D​\(x\)⋅eΔ⁡\(x\)ZΔ,\\displaystyle=\\frac\{\\pi\_\{CD\}\(x\)\\cdot e^\{\\Delta\(x\)\}\}\{Z\_\{\\Delta\}\},ZΔ\\displaystyle Z\_\{\\Delta\}=𝔼πC​D​\[eΔ⁡\(X\)\],\\displaystyle=\\mathbb\{E\}\_\{\\pi\_\{CD\}\}\[e^\{\\Delta\(X\)\}\],where the amplified combined error isΔ⁡\(x\)=\(1\+α\)​ϵp​\(x\)−α​ϵq​\(x\)\\Delta\(x\)=\(1\{\+\}\\alpha\)\\,\\epsilon\_\{p\}\(x\)\-\\alpha\\,\\epsilon\_\{q\}\(x\)\.

###### Proof\.

Direct expansion:

πe​\(x\)1\+α⋅πf​\(x\)−α\\displaystyle\\pi\_\{e\}\(x\)^\{1\+\\alpha\}\\cdot\\pi\_\{f\}\(x\)^\{\-\\alpha\}=\[πp​\(x\)​eϵp​\(x\)\]1\+α\\displaystyle=\\bigl\[\\pi\_\{p\}\(x\)\\,e^\{\\epsilon\_\{p\}\(x\)\}\\bigr\]^\{1\+\\alpha\}⋅\[πq​\(x\)​eϵq​\(x\)\]−α\\displaystyle\\quad\\cdot\\bigl\[\\pi\_\{q\}\(x\)\\,e^\{\\epsilon\_\{q\}\(x\)\}\\bigr\]^\{\-\\alpha\}=πp​\(x\)1\+α​πq​\(x\)−α\\displaystyle=\\pi\_\{p\}\(x\)^\{1\+\\alpha\}\\,\\pi\_\{q\}\(x\)^\{\-\\alpha\}⋅e\(1\+α\)​ϵp​\(x\)−α​ϵq​\(x\)\\displaystyle\\quad\\cdot e^\{\(1\+\\alpha\)\\epsilon\_\{p\}\(x\)\-\\alpha\\,\\epsilon\_\{q\}\(x\)\}=ZC​D​πC​D​\(x\)⋅eΔ⁡\(x\)\.\\displaystyle=Z\_\{CD\}\\,\\pi\_\{CD\}\(x\)\\cdot e^\{\\Delta\(x\)\}\.Normalizing yields the lemma\. ∎

The lemma makes the amplification mechanism explicit: the expert EAGLE log\-errorϵp\\epsilon\_\{p\}is scaled by\(1\+α\)\(1\{\+\}\\alpha\)and the amateur EAGLE errorϵq\\epsilon\_\{q\}byα\\alpha\. Even when the two heads have similar approximation quality \(DK​L\(πe∥πp\)≈2\.6D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)\\approx 2\.6vs\.DK​L\(πf∥πq\)≈2\.4D\_\{KL\}\(\\pi\_\{f\}\\\|\\pi\_\{q\}\)\\approx 2\.4on Llama\-3\), the combined errorΔ\\Deltagrows before normalization\. Consequently,DK​L\(π^C​D∥πC​D\)D\_\{KL\}\(\\hat\{\\pi\}\_\{CD\}\\\|\\pi\_\{CD\}\)becomes sensitive to the error distribution\.

#### Per\-Position Verification\.

We also measure the fraction of autoregressive positions at which the Approximate Dual\-Drafter reconstructionπ^C​D\\hat\{\\pi\}\_\{CD\}is closer toπC​D\\pi\_\{CD\}than the expert\-aligned draftπe\\pi\_\{e\}, that is, whereDK​L\(π^C​D∥πC​D\)<DK​L\(πe∥πC​D\)D\_\{KL\}\(\\hat\{\\pi\}\_\{CD\}\\\|\\pi\_\{CD\}\)<D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{CD\}\)\. On Llama\-3 \(GSM8K\), this holds for 39\.7% of positions atα=0\.1\\alpha=0\.1and 33\.1% atα=0\.5\\alpha=0\.5; Qwen3 shows 39\.2% and 31\.9%, respectively\. This position\-level check is consistent with the underperformance of Approximate Dual\-Drafter in the tested regime\.

Table[6](https://arxiv.org/html/2608.12913#A1.T6)reports the corresponding aggregate matched\-control results over accepted length, draft\-to\-target KL, and Top\-1 Match\.

α\\alphaMethodMean Accepted LengthLLAvg\.DK​L\(⋅∥πC​D\)D\_\{KL\}\(\\cdot\\\|\\pi\_\{CD\}\)Avg\. Top\-1Match \(%\)GSM8KMMLULlama\-3\(8B\-Ins / 1B\-Ins\)0\.1Expert\-aligned1\.9661\.4762\.9867\.7Approx\. Dual1\.9451\.4583\.1366\.70\.3Expert\-aligned1\.9321\.4243\.2666\.1Approx\. Dual1\.8531\.3573\.8462\.90\.5Expert\-aligned1\.8631\.3553\.5464\.4Approx\. Dual1\.7111\.2414\.7458\.6Qwen3\(8B / 1\.7B\)0\.1Expert\-aligned1\.7361\.1226\.4560\.6Approx\. Dual1\.7171\.1046\.6959\.70\.3Expert\-aligned1\.6561\.0276\.5558\.1Approx\. Dual1\.5930\.9737\.3955\.20\.5Expert\-aligned1\.5440\.9046\.6954\.7Approx\. Dual1\.4200\.8208\.3149\.3

Table 6:Approximate dual\-drafting vs\. expert\-aligned DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}\. GSM8K/MMLU columns report dataset\-specific mean accepted lengthLL\. The last two columns average over GSM8K and MMLU:DK​L\(⋅∥πC​D\)D\_\{KL\}\(\\cdot\\\|\\pi\_\{CD\}\)is the draft\-to\-target KL, and Top\-1 Match is the argmax match rate withπC​D\\pi\_\{CD\}\. The CD reconstruction route fails on all three diagnostics: it has lowerLL, higher draft\-to\-target KL, and lower Top\-1 Match, with larger gaps asα\\alphaincreases\.
#### Supplementary Aggregated Regime Analysis Beyond the Matched Control\.

Beyond the matched control above, we also aggregate configurations spanning three model groups \(Llama\-3,Llama\-EFT, andQwen3\), two datasets \(GSM8K and MMLU\), and four contrastive strengths \(α∈\{0\.1,0\.3,0\.5,1\.0\}\\alpha\\in\\\{0\.1,0\.3,0\.5,1\.0\\\}\)\. For each configuration, we record the Top\-1 gapΔ\\DeltaTop\-1==Top\-1\(contrastive\-aligned\)−\-Top\-1\(expert\-aligned\), where Top\-1 denotes the fraction of positions whose argmax matchesarg⁡maxx​πC​D​\(x∣h\)\\arg\\max\_\{x\}\\pi\_\{CD\}\(x\\mid h\), across bucketed views of signal,DK​L\(πe∥πp\)D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{p\}\), andDK​L\(πf∥πq\)D\_\{KL\}\(\\pi\_\{f\}\\\|\\pi\_\{q\}\)\. The main text now summarizes the signal distribution, the signal×\\timesDK​L\(πe∥πp\)D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)effect view, and the marginal KL\(πe∥πp\)\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)distribution; this appendix adds the complementary views from that broader diagnostic\.

Table[7](https://arxiv.org/html/2608.12913#A1.T7)gives the detailed count\-weighted signal\-mass breakdown behind the main\-text distribution panel\. In total, 81\.1% of positions fall below signal strength 1\.0, while 18\.9% lie at or above 1\.0\.

Signal bucketShareCumulative\[0,0\.25\)\[0,0\.25\)64\.4%64\.4%\[0\.25,0\.5\)\[0\.25,0\.5\)8\.4%72\.8%\[0\.5,1\)\[0\.5,1\)8\.3%81\.1%\[1,2\)\[1,2\)7\.0%88\.1%\[2,4\)\[2,4\)5\.2%93\.3%\[4,8\)\[4,8\)3\.6%96\.9%≥8\\geq 83\.1%100\.0%<1\.0<1\.081\.1%–≥1\.0\\geq 1\.018\.9%–Table 7:Count\-weighted share of autoregressive positions by contrastive\-signal bucket, aggregated over 24 configurations spanning three model groups, two datasets, and four contrastive strengths\.Main\-text Figure[2](https://arxiv.org/html/2608.12913#S3.F2)summarizes the count\-weighted signal distribution, the marginal KL\(πe∥πp\)\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)distribution, and the signal×\\timesDK​L\(πe∥πp\)D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)mechanism view\. The remaining appendix figures add complementary aggregated views\.

Figure[6](https://arxiv.org/html/2608.12913#A1.F6)provides the complementary configuration\-level view over KL\(πf∥πq\)\(\\pi\_\{f\}\\,\\\|\\,\\pi\_\{q\}\)buckets\. Along this axis, the values vary more by configuration and do not form a stable monotonic pattern\.

Llama\-3 / GSM8KLlama\-3 / MMLULlama\-EFT / GSM8KLlama\-EFT / MMLUQwen3 / GSM8KQwen3 / MMLU\[0,0\.01\)\[0,0\.01\)\[0\.01,0\.05\)\[0\.01,0\.05\)\[0\.05,0\.1\)\[0\.05,0\.1\)\[0\.1,0\.2\)\[0\.1,0\.2\)\[0\.2,0\.5\)\[0\.2,0\.5\)\[0\.5,1\)\[0\.5,1\)\[1,2\)\[1,2\)\[2,4\)\[2,4\)\[4,6\)\[4,6\)\[6,8\)\[6,8\)\[8,∞\)\[8,\\infty\)a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0KL\(πf∥πq\)\(\\pi\_\{f\}\\,\\\|\\,\\pi\_\{q\}\)\-0\.3\-0\.1500\.150\.3Δ\\DeltaTop\-1Figure 6:Configuration\-levelΔ\\DeltaTop\-1 across KL\(πf∥πq\)\(\\pi\_\{f\}\\,\\\|\\,\\pi\_\{q\}\)buckets\. The association is heterogeneous across model groups and contrastive strengths: forα≤0\.5\\alpha\\leq 0\.5, KL\-axis patterns are weak or mixed, whereas atα=1\.0\\alpha=1\.0the gap often becomes less negative at higher KL\. No configuration\-invariant positive\-gain regime emerges\.Figure[7](https://arxiv.org/html/2608.12913#A1.F7)provides a compact signal\-centric aggregation over mutually exclusive signal buckets and shows that the low\-signal failure pattern persists in most settings\.

Llama\-3 / GSM8KLlama\-3 / MMLULlama\-EFT / GSM8KLlama\-EFT / MMLUQwen3 / GSM8KQwen3 / MMLU\[0,0\.25\)\[0,0\.25\)\[0\.25,0\.5\)\[0\.25,0\.5\)\[0\.5,1\)\[0\.5,1\)\[1,2\)\[1,2\)\[2,4\)\[2,4\)\[4,8\)\[4,8\)≥8\\geq 8a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0a=0\.1a=0\.1a=0\.3a=0\.3a=0\.5a=0\.5a=1\.0a=1\.0Signal bucket\-0\.30\-0\.1500\.150\.30Δ\\DeltaTop\-1Figure 7:Compact regime heatmap for the expanded root\-level contrastive\-signal bucket analysis\. Here contrastive signal denotes the per\-position expert–amateur absolute log\-ratio\|log⁡πp​\(x∣h\)πq​\(x∣h\)\|\\left\|\\log\\frac\{\\pi\_\{p\}\(x\\mid h\)\}\{\\pi\_\{q\}\(x\\mid h\)\}\\right\|\. Rows are grouped by model family and dataset, with per\-row tick labels indicatingα\\alpha, and columns are mutually exclusive contrastive\-signal buckets\. Low\-signal buckets are mostly negative, while several higher\-signal buckets are positive\. Warm colors favor contrastive\-aligned drafting; cool colors favor expert\-aligned drafting\. To preserve visualization contrast for moderate variations, the color scale is truncated at±0\.3\\pm 0\.3\.Figure[8](https://arxiv.org/html/2608.12913#A1.F8)breaks this pattern down by model group and dataset\. Largerα\\alphausually pushes the low\-signal buckets further negative, while some higher\-signal buckets improve when the contrastive signal is informative enough\.

\[0,0\.25\)\[0,0\.25\)\[0\.25,0\.5\)\[0\.25,0\.5\)\[0\.5,1\)\[0\.5,1\)\[1,2\)\[1,2\)\[2,4\)\[2,4\)\[4,8\)\[4,8\)−0\.2\-0\.2−0\.1\-0\.1000\.10\.10\.20\.20\.30\.3Δ\\DeltaTop\-1Llama\-3\| GSM8K\[0,0\.25\)\[0,0\.25\)\[0\.25,0\.5\)\[0\.25,0\.5\)\[0\.5,1\)\[0\.5,1\)\[1,2\)\[1,2\)\[2,4\)\[2,4\)\[4,8\)\[4,8\)−0\.2\-0\.2−0\.1\-0\.1000\.10\.10\.20\.20\.30\.3Llama\-EFT\| GSM8K\[0,0\.25\)\[0,0\.25\)\[0\.25,0\.5\)\[0\.25,0\.5\)\[0\.5,1\)\[0\.5,1\)\[1,2\)\[1,2\)\[2,4\)\[2,4\)\[4,8\)\[4,8\)−0\.2\-0\.2−0\.1\-0\.1000\.10\.10\.20\.20\.30\.3Qwen3\| GSM8K\[0,0\.25\)\[0,0\.25\)\[0\.25,0\.5\)\[0\.25,0\.5\)\[0\.5,1\)\[0\.5,1\)\[1,2\)\[1,2\)\[2,4\)\[2,4\)\[4,8\)\[4,8\)−0\.2\-0\.2−0\.1\-0\.1000\.10\.10\.20\.20\.30\.3Signal BucketΔ\\DeltaTop\-1Llama\-3\| MMLU\[0,0\.25\)\[0,0\.25\)\[0\.25,0\.5\)\[0\.25,0\.5\)\[0\.5,1\)\[0\.5,1\)\[1,2\)\[1,2\)\[2,4\)\[2,4\)\[4,8\)\[4,8\)−0\.2\-0\.2−0\.1\-0\.1000\.10\.10\.20\.20\.30\.3Signal BucketLlama\-EFT\| MMLU\[0,0\.25\)\[0,0\.25\)\[0\.25,0\.5\)\[0\.25,0\.5\)\[0\.5,1\)\[0\.5,1\)\[1,2\)\[1,2\)\[2,4\)\[2,4\)\[4,8\)\[4,8\)−0\.2\-0\.2−0\.1\-0\.1000\.10\.10\.20\.20\.30\.3Signal BucketQwen3\| MMLUα=0\.1\\alpha=0\.1α=0\.3\\alpha=0\.3α=0\.5\\alpha=0\.5α=1\.0\\alpha=1\.0

Figure 8:Per\-groupΔ\\DeltaTop\-1 trends across contrastive\-signal buckets\. Here contrastive signal denotes the per\-position expert–amateur absolute log\-ratio\|log⁡πp​\(x∣h\)πq​\(x∣h\)\|\\left\|\\log\\frac\{\\pi\_\{p\}\(x\\mid h\)\}\{\\pi\_\{q\}\(x\\mid h\)\}\\right\|\. Each subplot fixes a model group and dataset, and each line corresponds to one contrastive strengthα\\alpha\. Largerα\\alphausually worsens low\-signal buckets but can amplify gains in higher\-signal buckets, especially for the Llama\-EFT settings\. For clarity of the moderate\-signal trends, the extreme outlier bucket \(≥8\\geq 8\) is excluded from the horizontal axis\.Across Figures[2](https://arxiv.org/html/2608.12913#S3.F2),[6](https://arxiv.org/html/2608.12913#A1.F6),[7](https://arxiv.org/html/2608.12913#A1.F7), and[8](https://arxiv.org/html/2608.12913#A1.F8), low\-signal buckets remain mostly negative, the dependence on expert\-side proposal error is strong, and the dependence on amateur\-side reconstruction error is noisier\.

#### Why Contrastive\-Aligned Training Also Underperforms in Our Setting \(Path 1\)\.

For Path 1, there is no inference\-time reconstruction step, but the same scale mismatch remains: the lightweight EAGLE head must learn both the dominant expert distributionπp\\pi\_\{p\}and the much smaller contrastive signalΔ∝πq−α\\Delta\\propto\\pi\_\{q\}^\{\-\\alpha\}\. This increases the modeling burden without reducing baseline proposal error\. Within our CD\-Aligned\-EAGLE family, Table[8](https://arxiv.org/html/2608.12913#A1.T8)therefore still favors expert\-aligned training \(train\-α=0\\alpha\{=\}0\) as the strongest default\.

### A\.2Cross\-alpha Experiment Details

Table[1](https://arxiv.org/html/2608.12913#S3.T1)in the main text summarizes the within\-family controlled single\-run ablation\. Table[8](https://arxiv.org/html/2608.12913#A1.T8)reports the full cross\-α\\alphamatrix for the Contrastive\-Aligned Dual\-Input Drafter in Section[3\.1](https://arxiv.org/html/2608.12913#S3.SS1)\(Path 1\), where rows vary train\-α\\alphaand columns vary infer\-α\\alpha\. All numeric variants share the same dual\-input architecture and ShareGPT training setup; only the target distribution differs\. We also include the Official checkpoint from our main experiments, evaluated under the same inference protocol, as an external reference\.

Train\-α\\alphaInfer\-α=0\.0\\alpha\{=\}0\.0Infer\-α=0\.1\\alpha\{=\}0\.1Infer\-α=0\.3\\alpha\{=\}0\.3Infer\-α=0\.5\\alpha\{=\}0\.5Llama\-3\(8B\-Ins / 1B\-Ins\)— GSM8KOfficial checkpoint1\.5551\.5371\.5461\.4760\.02\.0021\.9681\.9561\.8800\.12\.0081\.9881\.9771\.8920\.31\.9951\.9641\.9631\.8810\.51\.9361\.9151\.8961\.833Llama\-3\(8B\-Ins / 1B\-Ins\)— MMLUOfficial checkpoint1\.5231\.4281\.3441\.2140\.01\.6551\.5961\.5081\.4040\.11\.6631\.5761\.5031\.3700\.31\.6231\.5541\.4691\.3200\.51\.6181\.5521\.4661\.328Qwen3\(8B / 1\.7B\)— GSM8KOfficial checkpoint2\.0832\.0611\.9831\.8420\.01\.7321\.6911\.6021\.4770\.11\.7141\.6641\.5831\.4770\.31\.7151\.6781\.5751\.4740\.51\.6911\.6651\.5841\.487Qwen3\(8B / 1\.7B\)— MMLUOfficial checkpoint1\.6561\.6501\.5521\.3860\.01\.4381\.4261\.3171\.1890\.11\.4111\.4041\.2941\.1590\.31\.4181\.4111\.3171\.1760\.51\.3921\.3801\.2841\.156Table 8:Full cross\-α\\alphamatrix for the Contrastive\-Aligned Dual\-Input Drafter\. Numeric train\-α\\alpharows report our CD\-Aligned\-EAGLE family: rows vary the training target, columns vary infer\-α\\alpha, and cells report Mean Accepted Length \(L\)\. TheOfficial checkpointrow reports the publicly released model used in our main experiments, evaluated under the same inference protocol, as an external reference\. Train\-α=0\.0\\alpha\{=\}0\.0is our expert\-aligned baseline\.Highlighted rowsmark this baseline, andboldmarks the best value in each infer\-α\\alphacolumn over all displayed rows\.
### A\.3Optimistic Offline Diagnostic with a Stronger Full\-LM Proposer

This subsection extends the diagnosis in Section[3\.1](https://arxiv.org/html/2608.12913#S3.SS1)to a stronger same\-family proposer\. It tests whether the mechanism in Figure[2](https://arxiv.org/html/2608.12913#S3.F2)disappears once the proposal side is replaced by a substantially stronger full LM\.

#### Setup\.

We keep the expert and amateur fixed asLlama\-3\.1\-8B\-InstructandLlama\-3\.2\-1B\-Instruct, and replace the lightweight proposal proxy withLlama\-3\.2\-3B\-Instruct\. Letπr\\pi\_\{r\}denote the 3B proposer distribution\. For each shared response historyhh, we compare the true contrastive target

πC​D​\(x∣h\)∝πp​\(x∣h\)1\+α​πq​\(x∣h\)−α\\pi\_\{CD\}\(x\\mid h\)\\propto\\pi\_\{p\}\(x\\mid h\)^\{1\+\\alpha\}\\pi\_\{q\}\(x\\mid h\)^\{\-\\alpha\}with the stronger\-proposer approximation

π^C​Dr​\(x∣h\)∝πr​\(x∣h\)1\+α​πq​\(x∣h\)−α\.\\hat\{\\pi\}^\{r\}\_\{CD\}\(x\\mid h\)\\propto\\pi\_\{r\}\(x\\mid h\)^\{1\+\\alpha\}\\pi\_\{q\}\(x\\mid h\)^\{\-\\alpha\}\.We evaluate GSM8K and MMLU on 200 aligned examples each withα∈\{0\.1,0\.5\}\\alpha\\in\\\{0\.1,0\.5\\\}\. Reusing the true amateur logits and computing all statistics on shared response histories gives the proposer an optimistic offline diagnostic; online rollout performance is outside this check\.

\(a\) Contrastive signal

distribution

\[0,0\.25\)\[0,0\.25\)\[0\.25,0\.5\)\[0\.25,0\.5\)\[0\.5,1\)\[0\.5,1\)\[1,2\)\[1,2\)\[2,4\)\[2,4\)\[4,8\)\[4,8\)≥8\\geq 800202040406060Signal bucketShare \(%\)

\(b\) Effect by signal and proposal error

<0\.01<0\.010\.010\.01–0\.050\.050\.050\.05–0\.10\.10\.10\.1–0\.20\.20\.20\.2–0\.50\.50\.50\.5–1111–22≥2\\geq 2<0\.25<0\.250\.250\.25–0\.50\.50\.50\.5–1111–2222–4444–88≥8\\geq 8KL\(πr∥πp\)\(\\pi\_\{r\}\\,\\\|\\,\\pi\_\{p\}\)Signal\-0\.4\-0\.200\.20\.4Δ\\DeltaTop\-1

\(c\) KL\(πr∥πp\)\(\\pi\_\{r\}\\,\\\|\\,\\pi\_\{p\}\)

distribution

<0\.01<0\.010\.010\.01–0\.050\.050\.050\.05–0\.10\.10\.10\.1–0\.20\.20\.20\.2–0\.50\.50\.50\.5–1111–22≥2\\geq 20010102020303040405050KL\(πr∥πp\)\(\\pi\_\{r\}\\,\\\|\\,\\pi\_\{p\}\)bucketShare \(%\)

Figure 9:Full offline 3B proposer diagnostic\.Δ\\DeltaTop\-1 is the CD\-aware reconstruction’s Top\-1 agreement gain over the unmodified 3B proposer, measured againstarg⁡maxx​πC​D​\(x∣h\)\\arg\\max\_\{x\}\\pi\_\{CD\}\(x\\mid h\)\. Low\-signal rows remain mostly negative; setting\-level meanΔ\\DeltaTop\-1 values are slightly negative in all four settings \(GSM8K \(α=0\.1\\alpha=0\.1\) \-0\.0028; GSM8K \(α=0\.5\\alpha=0\.5\) \-0\.0123; MMLU \(α=0\.1\\alpha=0\.1\) \-0\.0030; MMLU \(α=0\.5\\alpha=0\.5\) \-0\.0144\)\.Figure[9](https://arxiv.org/html/2608.12913#A1.F9)reports the same three\-part diagnostic as Figure[2](https://arxiv.org/html/2608.12913#S3.F2)\. Under this optimistic proposer replacement, favorable gains remain concentrated in higher\-signal, lower\-error regions, while low\-signal or high\-proposer\-error buckets remain unfavorable\. The stronger proposer therefore enlarges the favorable pocket but does not overturn the conclusion that proposal\-side approximation error limits contrastive\-aligned drafting\.

## Appendix BTheory for Expert\-Aligned Drafting in Contrastive Acceleration

This appendix provides the robustness analysis summarized in Section[2\.3](https://arxiv.org/html/2608.12913#S2.SS3)\. It characterizes the expert\-aligned lightweight\-drafter regime used by DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}and connects the analysis to the observed relative degradation trends; the empirical comparisons remain the primary evidence\.

### B\.1Distribution Alignment and Robustness Analysis

We analyze how draft distributions align with the contrastive decoding target distributionπC​D\\pi\_\{CD\}and why expert\-aligned drafting is more robust toα\\alphaunder the stated alignment condition\. This analysis supports the empirical finding that mean accepted length decays more gradually in the expert\-aligned DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}instantiation as contrastive strength increases\.

#### KL Divergence Decomposition\.

Letπp\\pi\_\{p\},πq\\pi\_\{q\}, andπe\\pi\_\{e\}denote the expert, amateur, and lightweight\-drafter distributions, respectively\. The target distribution is defined asπC​D​\(x\)=1ZC​D​πp​\(x\)1\+α​πq​\(x\)−α\\pi\_\{CD\}\(x\)=\\frac\{1\}\{Z\_\{CD\}\}\\pi\_\{p\}\(x\)^\{1\+\\alpha\}\\pi\_\{q\}\(x\)^\{\-\\alpha\}\.

###### Lemma B\.1\(KL Divergence to Target\)\.

DK​L\(πp∥πC​D\)\\displaystyle D\_\{KL\}\(\\pi\_\{p\}\\\|\\pi\_\{CD\}\)\\\!=logZC​D−αDK​L\(πp∥πq\),\\displaystyle=\\\!\\log Z\_\{CD\}\-\\alpha D\_\{KL\}\(\\pi\_\{p\}\\\|\\pi\_\{q\}\),DK​L\(πq∥πC​D\)\\displaystyle D\_\{KL\}\(\\pi\_\{q\}\\\|\\pi\_\{CD\}\)\\\!=log⁡ZC​D\\displaystyle=\\\!\\log Z\_\{CD\}\+\(1\+α\)DK​L\(πq∥πp\),\\displaystyle\\quad\+\(1\+\\alpha\)D\_\{KL\}\(\\pi\_\{q\}\\\|\\pi\_\{p\}\),DK​L\(πe∥πC​D\)\\displaystyle D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{CD\}\)\\\!=log⁡ZC​D\\displaystyle=\\\!\\log Z\_\{CD\}\+\(1\+α\)DK​L\(πe∥πp\)\\displaystyle\\quad\+\(1\+\\alpha\)D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)−αDK​L\(πe∥πq\)\.\\displaystyle\\quad\-\\alpha D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{q\}\)\.

###### Proof\.

Substituting the definition ofπC​D\\pi\_\{CD\}intoDK​L\(⋅∥πC​D\)D\_\{KL\}\(\\cdot\\\|\\pi\_\{CD\}\)and expanding the logarithms gives the result\. ∎

#### Static Analysis\.

We first examine the absolute distance between each draft distribution and the targetπC​D\\pi\_\{CD\}at a fixedα\\alpha\.

From Lemma[B\.1](https://arxiv.org/html/2608.12913#A2.Thmtheorem1), the KL divergence gap between the amateur model and the drafter relative to the target is:

DK​L\(πq∥πC​D\)−DK​L\(πe∥πC​D\)=\(1\+α\)\[DK​L\(πq∥πp\)−DK​L\(πe∥πp\)\]\+αDK​L\(πe∥πq\)\.\\begin\{split\}&D\_\{KL\}\(\\pi\_\{q\}\\\|\\pi\_\{CD\}\)\-D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{CD\}\)\\\\ &\\quad=\(1\+\\alpha\)\[D\_\{KL\}\(\\pi\_\{q\}\\\|\\pi\_\{p\}\)\-D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)\]\\\\ &\\qquad\+\\alpha D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{q\}\)\.\\end\{split\}\(2\)This identity follows directly from Lemma[B\.1](https://arxiv.org/html/2608.12913#A2.Thmtheorem1), where thelog⁡ZC​D\\log Z\_\{CD\}terms cancel\.

Remark \(Static Capacity Gap\)\.In practice, if the absolute distance from the drafter to the target is larger than the corresponding distance from the amateur \(DK​L\(πe∥πC​D\)\>DK​L\(πq∥πC​D\)D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{CD\}\)\>D\_\{KL\}\(\\pi\_\{q\}\\\|\\pi\_\{CD\}\)\), it implies:

DK​L\(πe∥πp\)\>DK​L\(πq∥πp\)\+α1\+αDK​L\(πe∥πq\)\.\\begin\{split\}D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)\>&D\_\{KL\}\(\\pi\_\{q\}\\\|\\pi\_\{p\}\)\\\\ &\+\\frac\{\\alpha\}\{1\+\\alpha\}D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{q\}\)\.\\end\{split\}
This inequality quantifies thestatic capacity gap\. Although the lightweight drafter is trained to mimic the expert, its approximation errorDK​L\(πe∥πp\)D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{p\}\)can still exceed the amateur’s baseline error by a margin large enough to offset the alignment term\. This interpretation matches our acceptance\-rate comparisons, in which the lightweight drafter can start from a lower acceptance baseline than the amateur model\.

#### Dynamic Analysis: Robustness to Contrastive Penalty\.

Althoughπe\\pi\_\{e\}may be farther from the target in absolute terms, its response to increasingα\\alphacan still be more favorable under the alignment condition below\.

To keep the analysis tied to a meaningful draft model, we use the following alignment condition\.

###### Definition B\.2\(Effective Draft Alignment\)\.

A draft distributionπe\\pi\_\{e\}is effectively aligned if:

𝔼x∼πe​\[log⁡πp​\(x\)πq​\(x\)\]\>𝔼x∼πq​\[log⁡πp​\(x\)πq​\(x\)\]\.\\mathbb\{E\}\_\{x\\sim\\pi\_\{e\}\}\\left\[\\log\\frac\{\\pi\_\{p\}\(x\)\}\{\\pi\_\{q\}\(x\)\}\\right\]\>\\mathbb\{E\}\_\{x\\sim\\pi\_\{q\}\}\\left\[\\log\\frac\{\\pi\_\{p\}\(x\)\}\{\\pi\_\{q\}\(x\)\}\\right\]\.

This condition is equivalent to

∑x\(πe​\(x\)−πq​\(x\)\)​log⁡πp​\(x\)πq​\(x\)\>0\.\\sum\_\{x\}\(\\pi\_\{e\}\(x\)\-\\pi\_\{q\}\(x\)\)\\log\\frac\{\\pi\_\{p\}\(x\)\}\{\\pi\_\{q\}\(x\)\}\>0\.\(3\)
Remark\.The condition is practical\. Becauseπe\\pi\_\{e\}is trained to approximateπp\\pi\_\{p\}, it tends to move probability mass toward tokens on which the expert has a larger log\-likelihood advantage over the amateur\. Condition \([3](https://arxiv.org/html/2608.12913#A2.E3)\) only requires the drafter to separate expert\-preferred tokens from the rest at least as well as the amateur does\.

###### Theorem B\.3\(Slope Comparison under Effective Draft Alignment\)\.

Assumingπe\\pi\_\{e\}satisfies the Effective Draft Alignment condition, the distance from the amateur distributionπq\\pi\_\{q\}to the target distributionπC​D\\pi\_\{CD\}grows faster than the distance from the lightweight drafterπe\\pi\_\{e\}asα\\alphaincreases:

∂∂αDK​L\(πq∥πC​D\)\>∂∂αDK​L\(πe∥πC​D\)\.\\frac\{\\partial\}\{\\partial\\alpha\}D\_\{KL\}\(\\pi\_\{q\}\\\|\\pi\_\{CD\}\)\>\\frac\{\\partial\}\{\\partial\\alpha\}D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{CD\}\)\.

#### Proof of Theorem[B\.3](https://arxiv.org/html/2608.12913#A2.Thmtheorem3)\.

Empirically, SCD often has a higher acceptance rate atα=0\\alpha=0, whereas expert\-aligned drafting typically deteriorates more slowly asα\\alphaincreases\. We explain this trend by tracking how the distance to the target changes withα\\alpha\.

###### Proof\.

The target distribution is:

πC​D​\(x\)=1ZC​D​πp​\(x\)1\+α​πq​\(x\)−α\.\\pi\_\{CD\}\(x\)=\\frac\{1\}\{Z\_\{CD\}\}\\pi\_\{p\}\(x\)^\{1\+\\alpha\}\\pi\_\{q\}\(x\)^\{\-\\alpha\}\.Substituting its log\-form into the KL divergence formulaDK​L\(⋅∥πC​D\)D\_\{KL\}\(\\cdot\\\|\\pi\_\{CD\}\)and differentiating with respect toα\\alpha:

For the amateur modelℳq\\mathcal\{M\}\_\{q\}:

Slopeq\\displaystyle\\text\{Slope\}\_\{q\}=∂∂αDK​L\(πq∥πC​D\)\\displaystyle=\\frac\{\\partial\}\{\\partial\\alpha\}D\_\{KL\}\(\\pi\_\{q\}\\\|\\pi\_\{CD\}\)=∂∂α𝔼x∼πq\[logπq\(x\)\\displaystyle=\\\!\\frac\{\\partial\}\{\\partial\\alpha\}\\mathbb\{E\}\_\{x\\sim\\pi\_\{q\}\}\[\\log\\pi\_\{q\}\(x\)−\(\(1\+α\)​log⁡πp​\(x\)CLOSE\\displaystyle\\quad\-\(\(1\+\\alpha\)\\log\\pi\_\{p\}\(x\)−αlogπq\(x\)−logZC​D\)\]\\displaystyle\\quad\-\\alpha\\log\\pi\_\{q\}\(x\)\-\\log Z\_\{CD\}\)\]=𝔼x∼πq​\[log⁡πq​\(x\)−log⁡πp​\(x\)\]\\displaystyle=\\\!\\mathbb\{E\}\_\{x\\sim\\pi\_\{q\}\}\[\\log\\pi\_\{q\}\(x\)\-\\log\\pi\_\{p\}\(x\)\]\+∂log⁡ZC​D∂α\.\\displaystyle\\quad\+\\frac\{\\partial\\log Z\_\{CD\}\}\{\\partial\\alpha\}\.
For the drafterπe\\pi\_\{e\}:

Slopee\\displaystyle\\text\{Slope\}\_\{e\}=∂∂αDK​L\(πe∥πC​D\)\\displaystyle=\\frac\{\\partial\}\{\\partial\\alpha\}D\_\{KL\}\(\\pi\_\{e\}\\\|\\pi\_\{CD\}\)=∂∂α𝔼x∼πe\[logπe\(x\)\\displaystyle=\\frac\{\\partial\}\{\\partial\\alpha\}\\mathbb\{E\}\_\{x\\sim\\pi\_\{e\}\}\[\\log\\pi\_\{e\}\(x\)−\(\(1\+α\)​log⁡πp​\(x\)CLOSE\\displaystyle\\quad\-\(\(1\+\\alpha\)\\log\\pi\_\{p\}\(x\)−αlogπq\(x\)−logZC​D\)\]\\displaystyle\\quad\-\\alpha\\log\\pi\_\{q\}\(x\)\-\\log Z\_\{CD\}\)\]=𝔼x∼πe​\[log⁡πq​\(x\)−log⁡πp​\(x\)\]\\displaystyle=\\\!\\mathbb\{E\}\_\{x\\sim\\pi\_\{e\}\}\\\!\[\\log\\pi\_\{q\}\(x\)\\\!\-\\\!\\log\\pi\_\{p\}\(x\)\]\+∂log⁡ZC​D∂α\.\\displaystyle\\quad\+\\frac\{\\partial\\log Z\_\{CD\}\}\{\\partial\\alpha\}\.
Considering the slope differenceΔslope\\Delta\_\{\\text\{slope\}\}, the normalization term∂log⁡ZC​D∂α\\frac\{\\partial\\log Z\_\{CD\}\}\{\\partial\\alpha\}cancels out:

Δslope\\displaystyle\\Delta\_\{\\text\{slope\}\}=Slopeq−Slopee\\displaystyle=\\text\{Slope\}\_\{q\}\-\\text\{Slope\}\_\{e\}=𝔼x∼πq​\[log⁡πq​\(x\)−log⁡πp​\(x\)\]\\displaystyle=\\mathbb\{E\}\_\{x\\sim\\pi\_\{q\}\}\[\\log\\pi\_\{q\}\(x\)\-\\log\\pi\_\{p\}\(x\)\]−𝔼x∼πe​\[log⁡πq​\(x\)−log⁡πp​\(x\)\]\\displaystyle\\quad\-\\mathbb\{E\}\_\{x\\sim\\pi\_\{e\}\}\[\\log\\pi\_\{q\}\(x\)\-\\log\\pi\_\{p\}\(x\)\]=∑xπq​\(x\)​log⁡πq​\(x\)πp​\(x\)\\displaystyle=\\sum\_\{x\}\\pi\_\{q\}\(x\)\\log\\frac\{\\pi\_\{q\}\(x\)\}\{\\pi\_\{p\}\(x\)\}−∑xπe\(x\)logπq​\(x\)πp​\(x\)\\displaystyle\\quad\-\\sum\_\{x\}\\pi\_\{e\}\(x\)\\log\\frac\{\\pi\_\{q\}\(x\)\}\{\\pi\_\{p\}\(x\)\}=∑x\(πq​\(x\)−πe​\(x\)\)​log⁡πq​\(x\)πp​\(x\)\\displaystyle=\\sum\_\{x\}\(\\pi\_\{q\}\(x\)\-\\pi\_\{e\}\(x\)\)\\log\\frac\{\\pi\_\{q\}\(x\)\}\{\\pi\_\{p\}\(x\)\}=∑x\(πe​\(x\)−πq​\(x\)\)​log⁡πp​\(x\)πq​\(x\)\.\\displaystyle=\\sum\_\{x\}\(\\pi\_\{e\}\(x\)\\\!\-\\\!\\pi\_\{q\}\(x\)\)\\log\\frac\{\\pi\_\{p\}\(x\)\}\{\\pi\_\{q\}\(x\)\}\.\(4\)
ByDefinition[B\.2](https://arxiv.org/html/2608.12913#A2.Thmtheorem2), the expression in \([4](https://arxiv.org/html/2608.12913#A2.E4)\) is strictly positive\. ThusΔslope\>0\\Delta\_\{\\text\{slope\}\}\>0\.

∎

Conclusion and Discussion\.Asα\\alphaincreases, the amateur modelℳq\\mathcal\{M\}\_\{q\}used in SCD diverges from the contrastive target faster than the lightweight drafter under Effective Draft Alignment\. This explains the acceptance\-side robustness of expert\-aligned drafting in DCD\. The ablations in Section[3\.6](https://arxiv.org/html/2608.12913#S3.SS6)show the empirical counterpart: the relative mean accepted length of DCD declines more slowly than that of SCD asα\\alphaincreases\.

#### Synthesis\.

This analysis clarifies the empirical pattern: although the lightweight drafter may begin farther from the target because of its capacity limits \(Eq\.[2](https://arxiv.org/html/2608.12913#A2.E2)\), it has a more favorable dynamic slope under Theorem[B\.3](https://arxiv.org/html/2608.12913#A2.Thmtheorem3)\. This split between static gap and dynamic robustness explains the acceptance\-side degradation trends observed in the experiments\.

### B\.2Efficiency Advantage Condition

Letτm\\tau\_\{m\}andTmT\_\{m\}denote the mean output tokens and serial time per iteration for methodmm\. Under the throughput surrogateSm=τm/TmS\_\{m\}=\\tau\_\{m\}/T\_\{m\}, DCD is faster than an amateur\-coupled baselinebbif and only if

τDCDτb\>TDCDTb\.\\frac\{\\tau\_\{\\text\{DCD\}\}\}\{\\tau\_\{b\}\}\>\\frac\{T\_\{\\text\{DCD\}\}\}\{T\_\{b\}\}\.\(5\)Thus, a lower accepted\-token yield can be offset by a sufficiently large reduction in serial cost\. Appendix[D](https://arxiv.org/html/2608.12913#A4)expandsTmT\_\{m\}for DCD, SCD, and CoS and reports the measured decomposition used in Section[3\.5](https://arxiv.org/html/2608.12913#S3.SS5)\.

### B\.3Proof of Lossless Property

DCD inherits the lossless guarantee of speculative decoding by construction\. It changes only thechoiceof proposal distribution while leaving the verification procedure unchanged\.

###### Theorem B\.4\(Lossless Property of DCD\)\.

DCD produces outputs distributed identically to autoregressive decoding fromπC​D\\pi\_\{CD\}\.

###### Proof\.

Speculative decoding is lossless for any target distributionπtarget\\pi\_\{\\mathrm\{target\}\}that is a valid probability distribution\([Leviathan et al\. 2023](https://arxiv.org/html/2608.12913#bib.bib14);[Chen et al\. 2023](https://arxiv.org/html/2608.12913#bib.bib2)\)\. DCD instantiatesπtarget=πC​D\\pi\_\{\\mathrm\{target\}\}=\\pi\_\{CD\}, where:

πC​D​\(x\)=πp​\(x\)1\+α​πq​\(x\)−α∑x′πp​\(x′\)1\+α​πq​\(x′\)−α\.\\pi\_\{CD\}\(x\)=\\frac\{\\pi\_\{p\}\(x\)^\{1\+\\alpha\}\\pi\_\{q\}\(x\)^\{\-\\alpha\}\}\{\\sum\_\{x^\{\\prime\}\}\\pi\_\{p\}\(x^\{\\prime\}\)^\{1\+\\alpha\}\\pi\_\{q\}\(x^\{\\prime\}\)^\{\-\\alpha\}\}\.Since softmax outputs satisfyπp​\(x\),πq​\(x\)\>0\\pi\_\{p\}\(x\),\\pi\_\{q\}\(x\)\>0for allx∈𝒱x\\in\\mathcal\{V\}, the numerator is strictly positive and the normalization is well defined\. ThusπC​D\\pi\_\{CD\}is a valid distribution, and the lossless property follows immediately\. ∎

## Appendix CAlignment Gap Estimation Methodology

Table[9](https://arxiv.org/html/2608.12913#A3.T9)summarizes the empirical evidence for Definition[B\.2](https://arxiv.org/html/2608.12913#A2.Thmtheorem2)\. We measure the estimates in four greedy settings:Qwen3/GSM8K,Qwen3/MMLU,Llama\-3/GSM8K, andLlama\-3/MMLU\. Following the shared evaluation protocol in Section[3](https://arxiv.org/html/2608.12913#S3), we use 200 evaluation samples per dataset, greedy decoding \(T=0T=0\), and a base draft length ofγ=5\\gamma=5\. This appendix describes how we estimate𝔼πe​\[log⁡\(πp/πq\)\]\\mathbb\{E\}\_\{\\pi\_\{e\}\}\[\\log\(\\pi\_\{p\}/\\pi\_\{q\}\)\]and𝔼πq​\[log⁡\(πp/πq\)\]\\mathbb\{E\}\_\{\\pi\_\{q\}\}\[\\log\(\\pi\_\{p\}/\\pi\_\{q\}\)\]from draft trees collected during DCD inference\.

SettingLHSRHSGapQwen3/ GSM8K0\.392−\-0\.3480\.740Qwen3/ MMLU0\.181−\-0\.5710\.753Llama\-3/ GSM8K−\-0\.305−\-0\.3320\.028Llama\-3/ MMLU−\-0\.274−\-0\.4850\.211Table 9:Empirical alignment\-gap summary in four greedy settings\. Here LHS=𝔼πe​\[log⁡\(πp/πq\)\]=\\mathbb\{E\}\_\{\\pi\_\{e\}\}\[\\log\(\\pi\_\{p\}/\\pi\_\{q\}\)\], RHS=𝔼πq​\[log⁡\(πp/πq\)\]=\\mathbb\{E\}\_\{\\pi\_\{q\}\}\[\\log\(\\pi\_\{p\}/\\pi\_\{q\}\)\], and gap=LHS−RHS=\\mathrm\{LHS\}\-\\mathrm\{RHS\}; positive gaps support Effective Draft Alignment \(Definition[B\.2](https://arxiv.org/html/2608.12913#A2.Thmtheorem2)\)\.#### Sampling\.

For each evaluation sample in the reported settings, we collect the EAGLE3 draft tree generated by greedy DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}inference withα=0\.1\\alpha=0\.1\. At each node, we record full\-vocabulary logits from the expert modelℳp\\mathcal\{M\}\_\{p\}, the amateur modelℳq\\mathcal\{M\}\_\{q\}, and the drafterEE\.

#### Estimator\.

Letr⁡\(x\)=log⁡πp​\(x\)−log⁡πq​\(x\)r\(x\)=\\log\\pi\_\{p\}\(x\)\-\\log\\pi\_\{q\}\(x\)\. We then compute:

𝔼πe​\[log⁡πpπq\]\\displaystyle\\mathbb\{E\}\_\{\\pi\_\{e\}\}\\left\[\\log\\frac\{\\pi\_\{p\}\}\{\\pi\_\{q\}\}\\right\]=∑x∈𝒱πe​\(x\)⋅r⁡\(x\),\\displaystyle=\\sum\_\{x\\in\\mathcal\{V\}\}\\pi\_\{e\}\(x\)\\cdot r\(x\),𝔼πq​\[log⁡πpπq\]\\displaystyle\\mathbb\{E\}\_\{\\pi\_\{q\}\}\\left\[\\log\\frac\{\\pi\_\{p\}\}\{\\pi\_\{q\}\}\\right\]=∑x∈𝒱πq​\(x\)⋅r⁡\(x\),\\displaystyle=\\sum\_\{x\\in\\mathcal\{V\}\}\\pi\_\{q\}\(x\)\\cdot r\(x\),These expectations are evaluated over the full vocabulary at each position and then averaged across all positions and samples\.

## Appendix DThroughput Analysis

This section makes explicit the coarse throughput surrogate used in Section[3\.5](https://arxiv.org/html/2608.12913#S3.SS5)\. The formulas below use measured latency components to form analytic approximations to runtime throughput\.

### D\.1DCD

DCD decouples proposal generation from the amateur model through an amateur\-independent proposerEE\.

At each iteration, the proposer generatesγ\\gammadraft tokens, and the expert and amateur then run forward passes on the same draft sequence for verification\.

We approximate the serial time per iteration by

TDCD=γ⋅te\+tp\+tq,T\_\{\\text\{DCD\}\}=\\gamma\\cdot t\_\{e\}\+t\_\{p\}\+t\_\{q\},wheretet\_\{e\},tpt\_\{p\}, andtqt\_\{q\}denote the latency components reported in Section[3\.5](https://arxiv.org/html/2608.12913#S3.SS5)for the proposer, expert path, and amateur path, respectively\.

Under the same coarse surrogateS≈\(L\+1\)/TiterS\\approx\(L\+1\)/T\_\{\\text\{iter\}\}, the throughput approximation is

SpeedDCD=L\+1γ⋅te\+tp\+tq,\\text\{Speed\}\_\{\\text\{DCD\}\}=\\frac\{L\+1\}\{\\gamma\\cdot t\_\{e\}\+t\_\{p\}\+t\_\{q\}\},whereLLdenotes the average number of accepted tokens per iteration, excluding the additional token\.

When the proposer is lightweight or retrieval\-based,te≪tqt\_\{e\}\\ll t\_\{q\}, so the proposal overhead of DCD is much lower than that of SCD\.

### D\.2SCD with Delayed Drafting

SCD\([Yuan et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib27)\)with delayed drafting uses conditional drafting to reduce unnecessary draft generation\.

In each iteration, the amateur model first generatesγ\\gammadraft tokens\. An additional draft token is generated only when allγ\\gammadraft tokens are accepted by the contrastive distribution, which occurs with probabilityργ\\rho^\{\\gamma\}\.

The expected number of draft operations per iteration is therefore

Ndraft=γ\+ργ,N\_\{\\text\{draft\}\}=\\gamma\+\\rho^\{\\gamma\},whereρ\\rhodenotes the single\-step acceptance rate\. The corresponding throughput approximation is

SpeedSCD=L\+1\(γ\+ργ\)⋅tq\+tp,\\text\{Speed\}\_\{\\text\{SCD\}\}=\\frac\{L\+1\}\{\(\\gamma\+\\rho^\{\\gamma\}\)\\cdot t\_\{q\}\+t\_\{p\}\},whereLLdenotes the average number of accepted tokens per iteration, excluding the additional token\.

In practice, we infer an effective acceptance rateρ\\rhofrom the observedLLvalue using the local relationL≈∑i=1γρiL\\approx\\sum\_\{i=1\}^\{\\gamma\}\\rho^\{i\}\.

### D\.3CoS

CoS\([Fu et al\. 2025](https://arxiv.org/html/2608.12913#bib.bib7)\)uses an alternating proposal mechanism to reduce draft steps\.

If, in the previous iteration, all amateur\-model draft tokens and the expert proposal token are accepted, the current iteration can skip one amateur\-model draft step\.

The probability of saving a draft step is

Psave=ρpγp⋅ρqγq,P\_\{\\text\{save\}\}=\\rho\_\{p\}^\{\\gamma\_\{p\}\}\\cdot\\rho\_\{q\}^\{\\gamma\_\{q\}\},whereρp\\rho\_\{p\}denotes the per\-token acceptance rate for the expert proposal \(γp\\gamma\_\{p\}is typically set to 1\), andρq\\rho\_\{q\}denotes the per\-token acceptance rate for the amateur model, so thatρqγq\\rho\_\{q\}^\{\\gamma\_\{q\}\}is the probability that allγq\\gamma\_\{q\}draft tokens are accepted\.

Conversely, when all tokens in the current iteration are accepted, an additional amateur\-model draft step is required with probabilityρqγq\\rho\_\{q\}^\{\\gamma\_\{q\}\}to verify the expert proposal\.

The expected number of draft steps per iteration is therefore

Ndraft=γ−ρpγp⋅ρqγq\+ρqγq\.N\_\{\\text\{draft\}\}=\\gamma\-\\rho\_\{p\}^\{\\gamma\_\{p\}\}\\cdot\\rho\_\{q\}^\{\\gamma\_\{q\}\}\+\\rho\_\{q\}^\{\\gamma\_\{q\}\}\.The corresponding throughput approximation is

SpeedCoS=L\+1\(γ−ρpγp⋅ρqγq\+ρqγq\)⋅tq\+tp,\\text\{Speed\}\_\{\\text\{CoS\}\}=\\frac\{L\+1\}\{\(\\gamma\-\\rho\_\{p\}^\{\\gamma\_\{p\}\}\\cdot\\rho\_\{q\}^\{\\gamma\_\{q\}\}\+\\rho\_\{q\}^\{\\gamma\_\{q\}\}\)\\cdot t\_\{q\}\+t\_\{p\}\},whereLLdenotes the average number of accepted tokens per iteration, excluding the additional token\.

In practice, we infer an effective amateur acceptance rateρq\\rho\_\{q\}from the observedLLvalue using the same local relation\. Becauseρp\\rho\_\{p\}is not measured directly, we set it to 0\.9 as a calibration constant\.

### D\.4Multiplicative Throughput Decomposition

The speed formulas above yield the same coarse multiplicative decomposition for the advantage of DCD over any amateur\-coupled baselinebbevaluated at the same operating point:

SpeedDCDSpeedb≈LDCD\+1Lb\+1⋅TbTDCD\.\\frac\{\\mathrm\{Speed\}\_\{\\mathrm\{DCD\}\}\}\{\\mathrm\{Speed\}\_\{b\}\}\\approx\\frac\{L\_\{\\mathrm\{DCD\}\}\+1\}\{L\_\{b\}\+1\}\\cdot\\frac\{T\_\{b\}\}\{T\_\{\\mathrm\{DCD\}\}\}\.The first term captures acceptance and the second captures serial cost\. This decomposition is useful because it separates token acceptance from generation latency\. In our measured regime, the acceptance term is typically below 1, whereas the serial\-cost term is well above 1 and dominates the product\.

ModelBaseAcc\.CostPred\.Meas\.Llama\-3SCD0\.711\.621\.14x1\.12xCoS0\.711\.621\.15x1\.12xQwen3SCD0\.712\.061\.46x1\.41xCoS0\.712\.011\.43x1\.38xLlama\-EFTSCD0\.682\.301\.55x1\.49xCoS0\.682\.501\.69x1\.61xTable 10:Multiplicative decomposition of the speed advantage of DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}on MMLU \(T=0T\{=\}0,α=0\.1\\alpha\{=\}0\.1,γ=5\\gamma\{=\}5\)\. For an amateur\-coupled baselinebb, the predicted ratio is approximated by an acceptance factor\(LDCDeagle3\+1\)/\(Lb\+1\)\(L\_\{\\mathrm\{DCD\}\_\{\\textsc\{eagle3\}\}\}\+1\)/\(L\_\{b\}\+1\)times a serial\-cost factorTb/TDCDeagle3T\_\{b\}/T\_\{\\mathrm\{DCD\}\_\{\\textsc\{eagle3\}\}\}\. Acc\. below 1 means DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}accepts fewer tokens per speculative round; Cost above 1 means DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}has a cheaper serial proposal path\. Across all model families, the cost factor is the dominant term\.

## Appendix EAdditional Results and Ablations

### E\.1Main\-Result Breakdowns

This subsection reports the full throughput tables underlying[Table2](https://arxiv.org/html/2608.12913#S3.T2)\.

[Table11](https://arxiv.org/html/2608.12913#A5.T11)reports the greedy results forα∈\{0\.1,0\.5\}\\alpha\\in\\\{0\.1,0\.5\\\}, including raw tokens/sec and reported standard deviations\.

[Table12](https://arxiv.org/html/2608.12913#A5.T12)reports the corresponding sampling results atT=1T=1\.

MethodHumanEvalGSM8KMMLUCNN/DMAvg\.Llama\-3 \(Llama\-3\.1\-8B\-Instruct / Llama\-3\.2\-1B\-Instruct\)α=0\.1\\alpha=0\.1CD91\.3±\\pm4\.3 \(1\.00×\\times\)86\.6±\\pm5\.6 \(1\.00×\\times\)93\.1±\\pm2\.7 \(1\.00×\\times\)84\.0±\\pm0\.6 \(1\.00×\\times\)88\.7 \(1\.00×\\times\)SCD139\.2±\\pm4\.7 \(1\.52×\\times\)132\.1±\\pm2\.4 \(1\.53×\\times\)117\.4±\\pm1\.3 \(1\.26×\\times\)117\.4±\\pm2\.5 \(1\.40×\\times\)126\.5 \(1\.43×\\times\)CoS156\.5±\\pm16\.9 \(1\.71×\\times\)153\.7±\\pm2\.6 \(1\.77×\\times\)119\.9±\\pm4\.7 \(1\.29×\\times\)115\.0±\\pm6\.4 \(1\.37×\\times\)136\.3 \(1\.54×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}135\.0±\\pm1\.3 \(1\.48×\\times\)124\.0±\\pm1\.4 \(1\.43×\\times\)120\.5±\\pm0\.1 \(1\.29×\\times\)173\.6±\\pm3\.0 \(2\.07×\\times\)138\.3 \(1\.56×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}189\.2±\\pm2\.5 \(2\.07×\\times\)159\.7±\\pm1\.6 \(1\.84×\\times\)149\.0±\\pm7\.2 \(1\.60×\\times\)151\.4±\\pm2\.1 \(1\.80×\\times\)162\.3 \(1\.83×\\times\)α=0\.5\\alpha=0\.5CD95\.7±\\pm0\.6 \(1\.00×\\times\)91\.7±\\pm3\.7 \(1\.00×\\times\)83\.0±\\pm0\.7 \(1\.00×\\times\)90\.0±\\pm3\.5 \(1\.00×\\times\)90\.1 \(1\.00×\\times\)SCD152\.2±\\pm0\.7 \(1\.59×\\times\)130\.8±\\pm1\.0 \(1\.43×\\times\)100\.1±\\pm0\.6 \(1\.21×\\times\)100\.5±\\pm2\.3 \(1\.12×\\times\)120\.9 \(1\.34×\\times\)CoS159\.6±\\pm0\.2 \(1\.67×\\times\)142\.8±\\pm0\.9 \(1\.56×\\times\)99\.7±\\pm0\.6 \(1\.20×\\times\)101\.6±\\pm3\.4 \(1\.13×\\times\)126\.0 \(1\.40×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}133\.3±\\pm0\.7 \(1\.39×\\times\)122\.7±\\pm0\.2 \(1\.34×\\times\)128\.3±\\pm2\.3 \(1\.55×\\times\)188\.8±\\pm1\.4 \(2\.10×\\times\)143\.3 \(1\.59×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}181\.4±\\pm1\.1 \(1\.89×\\times\)152\.1±\\pm2\.6 \(1\.66×\\times\)119\.5±\\pm0\.2 \(1\.44×\\times\)143\.2±\\pm3\.5 \(1\.59×\\times\)149\.0 \(1\.65×\\times\)Qwen3 \(Qwen3\-8B / Qwen3\-1\.7B\)α=0\.1\\alpha=0\.1CD59\.9±\\pm5\.7 \(1\.00×\\times\)65\.7±\\pm1\.8 \(1\.00×\\times\)65\.7±\\pm0\.4 \(1\.00×\\times\)61\.0±\\pm4\.1 \(1\.00×\\times\)63\.1 \(1\.00×\\times\)SCD82\.8±\\pm3\.9 \(1\.38×\\times\)88\.5±\\pm0\.9 \(1\.35×\\times\)64\.8±\\pm0\.2 \(0\.99×\\times\)64\.0±\\pm3\.6 \(1\.05×\\times\)75\.0 \(1\.19×\\times\)CoS87\.1±\\pm2\.1 \(1\.45×\\times\)87\.4±\\pm6\.5 \(1\.33×\\times\)76\.6±\\pm0\.4 \(1\.17×\\times\)69\.3±\\pm2\.4 \(1\.14×\\times\)80\.1 \(1\.27×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}87\.1±\\pm0\.8 \(1\.45×\\times\)88\.9±\\pm1\.2 \(1\.35×\\times\)77\.1±\\pm1\.4 \(1\.17×\\times\)69\.6±\\pm2\.4 \(1\.14×\\times\)80\.7 \(1\.28×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}137\.8±\\pm0\.8 \(2\.30×\\times\)131\.9±\\pm4\.1 \(2\.01×\\times\)108\.8±\\pm4\.8 \(1\.66×\\times\)113\.0±\\pm1\.2 \(1\.85×\\times\)122\.9 \(1\.95×\\times\)α=0\.5\\alpha=0\.5CD64\.1±\\pm4\.1 \(1\.00×\\times\)66\.9±\\pm0\.2 \(1\.00×\\times\)64\.4±\\pm2\.0 \(1\.00×\\times\)63\.2±\\pm3\.3 \(1\.00×\\times\)64\.6 \(1\.00×\\times\)SCD78\.8±\\pm0\.7 \(1\.23×\\times\)83\.1±\\pm1\.4 \(1\.24×\\times\)56\.8±\\pm6\.0 \(0\.88×\\times\)51\.9±\\pm0\.9 \(0\.82×\\times\)67\.6 \(1\.05×\\times\)CoS80\.1±\\pm2\.6 \(1\.25×\\times\)87\.7±\\pm0\.9 \(1\.31×\\times\)62\.1±\\pm2\.2 \(0\.97×\\times\)49\.7±\\pm2\.9 \(0\.79×\\times\)69\.9 \(1\.08×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}84\.5±\\pm0\.2 \(1\.32×\\times\)85\.3±\\pm0\.5 \(1\.28×\\times\)75\.2±\\pm1\.8 \(1\.17×\\times\)63\.4±\\pm1\.2 \(1\.00×\\times\)77\.1 \(1\.19×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}127\.1±\\pm0\.7 \(1\.98×\\times\)123\.4±\\pm4\.8 \(1\.85×\\times\)103\.0±\\pm4\.5 \(1\.60×\\times\)88\.1±\\pm3\.3 \(1\.39×\\times\)110\.4 \(1\.71×\\times\)Llama\-EFT \(Llama\-3\.1\-8B\-Instruct / Llama\-3\.1\-8B\)α=0\.1\\alpha=0\.1CD75\.9±\\pm1\.5 \(1\.00×\\times\)75\.0±\\pm0\.8 \(1\.00×\\times\)73\.1±\\pm3\.6 \(1\.00×\\times\)69\.1±\\pm0\.5 \(1\.00×\\times\)73\.3 \(1\.00×\\times\)SCD91\.4±\\pm2\.1 \(1\.20×\\times\)87\.1±\\pm1\.0 \(1\.16×\\times\)77\.5±\\pm3\.1 \(1\.06×\\times\)71\.4±\\pm1\.1 \(1\.03×\\times\)81\.9 \(1\.12×\\times\)CoS98\.0±\\pm3\.5 \(1\.29×\\times\)91\.5±\\pm1\.3 \(1\.22×\\times\)73\.9±\\pm1\.3 \(1\.01×\\times\)73\.2±\\pm5\.5 \(1\.06×\\times\)84\.1 \(1\.15×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}108\.3±\\pm0\.5 \(1\.43×\\times\)97\.9±\\pm1\.3 \(1\.31×\\times\)96\.2±\\pm0\.4 \(1\.32×\\times\)142\.1±\\pm1\.8 \(2\.06×\\times\)111\.1 \(1\.52×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}154\.5±\\pm5\.2 \(2\.03×\\times\)130\.1±\\pm0\.4 \(1\.74×\\times\)113\.3±\\pm1\.7 \(1\.55×\\times\)120\.6±\\pm8\.0 \(1\.74×\\times\)129\.6 \(1\.77×\\times\)α=0\.5\\alpha=0\.5CD70\.7±\\pm2\.9 \(1\.00×\\times\)72\.3±\\pm2\.8 \(1\.00×\\times\)73\.4±\\pm3\.4 \(1\.00×\\times\)71\.6±\\pm3\.6 \(1\.00×\\times\)72\.0 \(1\.00×\\times\)SCD84\.4±\\pm3\.7 \(1\.19×\\times\)82\.5±\\pm0\.7 \(1\.14×\\times\)70\.1±\\pm1\.9 \(0\.96×\\times\)65\.8±\\pm1\.0 \(0\.92×\\times\)75\.7 \(1\.05×\\times\)CoS92\.8±\\pm0\.6 \(1\.31×\\times\)83\.2±\\pm1\.1 \(1\.15×\\times\)71\.6±\\pm1\.8 \(0\.98×\\times\)69\.3±\\pm1\.6 \(0\.97×\\times\)79\.2 \(1\.10×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}109\.3±\\pm0\.6 \(1\.55×\\times\)97\.3±\\pm0\.9 \(1\.35×\\times\)96\.0±\\pm1\.4 \(1\.31×\\times\)151\.1±\\pm0\.9 \(2\.11×\\times\)113\.4 \(1\.58×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}145\.5±\\pm5\.8 \(2\.06×\\times\)116\.8±\\pm6\.0 \(1\.62×\\times\)117\.9±\\pm5\.1 \(1\.61×\\times\)115\.7±\\pm6\.4 \(1\.61×\\times\)124\.0 \(1\.72×\\times\)Table 11:Per\-dataset greedy breakdown forα∈\{0\.1,0\.5\}\\alpha\\in\\\{0\.1,0\.5\\\}\. Each cell reports throughput in tokens/sec with reported std and speedup over vanilla CD at the sameα\\alpha\. The DCDngram\{\}\_\{\\textsc\{ngram\}\}rows use the matched greedy N\-gram proposer setting\.MethodHumanEvalGSM8KMMLUCNN/DMAvg\.Llama\-3 \(Llama\-3\.1\-8B\-Instruct / Llama\-3\.2\-1B\-Instruct\)α=0\.1\\alpha=0\.1CD92\.4±\\pm3\.3 \(1\.00×\\times\)90\.5±\\pm6\.6 \(1\.00×\\times\)90\.3±\\pm3\.3 \(1\.00×\\times\)88\.2±\\pm2\.6 \(1\.00×\\times\)90\.3 \(1\.00×\\times\)SCD120\.7±\\pm1\.6 \(1\.31×\\times\)125\.3±\\pm0\.9 \(1\.39×\\times\)104\.3±\\pm4\.9 \(1\.16×\\times\)107\.8±\\pm3\.2 \(1\.22×\\times\)114\.6 \(1\.27×\\times\)CoS156\.3±\\pm5\.9 \(1\.69×\\times\)139\.3±\\pm2\.4 \(1\.54×\\times\)109\.2±\\pm2\.3 \(1\.21×\\times\)108\.7±\\pm5\.2 \(1\.23×\\times\)128\.4 \(1\.42×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}128\.9±\\pm2\.0 \(1\.39×\\times\)114\.4±\\pm1\.5 \(1\.26×\\times\)108\.4±\\pm0\.9 \(1\.20×\\times\)142\.1±\\pm0\.9 \(1\.61×\\times\)123\.4 \(1\.37×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}161\.3±\\pm8\.9 \(1\.75×\\times\)139\.5±\\pm0\.6 \(1\.54×\\times\)119\.6±\\pm5\.2 \(1\.32×\\times\)140\.9±\\pm1\.7 \(1\.60×\\times\)140\.3 \(1\.55×\\times\)α=0\.5\\alpha=0\.5CD93\.3±\\pm3\.3 \(1\.00×\\times\)90\.8±\\pm5\.0 \(1\.00×\\times\)86\.0±\\pm5\.0 \(1\.00×\\times\)93\.1±\\pm2\.0 \(1\.00×\\times\)90\.8 \(1\.00×\\times\)SCD126\.6±\\pm2\.0 \(1\.36×\\times\)116\.8±\\pm2\.6 \(1\.29×\\times\)91\.8±\\pm0\.8 \(1\.07×\\times\)93\.5±\\pm5\.1 \(1\.00×\\times\)107\.2 \(1\.18×\\times\)CoS134\.7±\\pm9\.3 \(1\.44×\\times\)112\.4±\\pm0\.4 \(1\.24×\\times\)85\.0±\\pm3\.1 \(0\.99×\\times\)96\.8±\\pm5\.4 \(1\.04×\\times\)107\.2 \(1\.18×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}126\.7±\\pm1\.3 \(1\.36×\\times\)114\.5±\\pm1\.3 \(1\.26×\\times\)104\.9±\\pm1\.8 \(1\.22×\\times\)147\.4±\\pm1\.3 \(1\.58×\\times\)123\.4 \(1\.36×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}183\.1±\\pm2\.4 \(1\.96×\\times\)148\.1±\\pm4\.4 \(1\.63×\\times\)132\.6±\\pm1\.9 \(1\.54×\\times\)165\.2±\\pm0\.1 \(1\.77×\\times\)157\.2 \(1\.73×\\times\)Qwen3 \(Qwen3\-8B / Qwen3\-1\.7B\)α=0\.1\\alpha=0\.1CD65\.1±\\pm3\.4 \(1\.00×\\times\)63\.9±\\pm0\.2 \(1\.00×\\times\)64\.2±\\pm2\.2 \(1\.00×\\times\)61\.5±\\pm2\.1 \(1\.00×\\times\)63\.7 \(1\.00×\\times\)SCD76\.2±\\pm7\.6 \(1\.17×\\times\)85\.3±\\pm0\.7 \(1\.34×\\times\)67\.2±\\pm3\.0 \(1\.05×\\times\)63\.4±\\pm2\.8 \(1\.03×\\times\)73\.0 \(1\.15×\\times\)CoS84\.8±\\pm1\.7 \(1\.30×\\times\)89\.8±\\pm1\.5 \(1\.41×\\times\)73\.2±\\pm0\.9 \(1\.14×\\times\)66\.8±\\pm1\.5 \(1\.09×\\times\)78\.7 \(1\.24×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}85\.0±\\pm0\.9 \(1\.31×\\times\)87\.6±\\pm1\.2 \(1\.37×\\times\)77\.1±\\pm0\.6 \(1\.20×\\times\)69\.3±\\pm0\.6 \(1\.13×\\times\)79\.7 \(1\.25×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}133\.6±\\pm1\.4 \(2\.05×\\times\)125\.2±\\pm5\.6 \(1\.96×\\times\)106\.9±\\pm5\.4 \(1\.67×\\times\)103\.8±\\pm2\.4 \(1\.69×\\times\)117\.4 \(1\.84×\\times\)α=0\.5\\alpha=0\.5CD64\.6±\\pm2\.0 \(1\.00×\\times\)65\.9±\\pm0\.3 \(1\.00×\\times\)61\.5±\\pm2\.5 \(1\.00×\\times\)60\.4±\\pm0\.3 \(1\.00×\\times\)63\.1 \(1\.00×\\times\)SCD69\.0±\\pm0\.8 \(1\.07×\\times\)79\.7±\\pm1\.5 \(1\.21×\\times\)57\.7±\\pm3\.7 \(0\.94×\\times\)49\.3±\\pm1\.5 \(0\.82×\\times\)63\.9 \(1\.01×\\times\)CoS74\.3±\\pm2\.9 \(1\.15×\\times\)84\.4±\\pm1\.2 \(1\.28×\\times\)61\.0±\\pm3\.3 \(0\.99×\\times\)48\.1±\\pm2\.2 \(0\.80×\\times\)67\.0 \(1\.06×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}80\.4±\\pm0\.7 \(1\.25×\\times\)85\.5±\\pm0\.8 \(1\.30×\\times\)72\.9±\\pm1\.4 \(1\.19×\\times\)64\.8±\\pm0\.2 \(1\.07×\\times\)75\.9 \(1\.20×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}127\.0±\\pm0\.2 \(1\.97×\\times\)118\.0±\\pm10\.4 \(1\.79×\\times\)103\.3±\\pm6\.5 \(1\.68×\\times\)92\.4±\\pm3\.1 \(1\.53×\\times\)110\.2 \(1\.75×\\times\)Llama\-EFT \(Llama\-3\.1\-8B\-Instruct / Llama\-3\.1\-8B\)α=0\.1\\alpha=0\.1CD70\.4±\\pm2\.3 \(1\.00×\\times\)72\.8±\\pm2\.3 \(1\.00×\\times\)67\.7±\\pm1\.0 \(1\.00×\\times\)67\.7±\\pm3\.9 \(1\.00×\\times\)69\.7 \(1\.00×\\times\)SCD79\.2±\\pm5\.6 \(1\.12×\\times\)77\.4±\\pm2\.4 \(1\.06×\\times\)69\.9±\\pm2\.0 \(1\.03×\\times\)61\.8±\\pm0\.9 \(0\.91×\\times\)72\.1 \(1\.03×\\times\)CoS84\.8±\\pm1\.9 \(1\.20×\\times\)78\.9±\\pm4\.7 \(1\.08×\\times\)71\.4±\\pm5\.1 \(1\.06×\\times\)69\.3±\\pm4\.9 \(1\.02×\\times\)76\.1 \(1\.09×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}105\.3±\\pm1\.0 \(1\.49×\\times\)91\.4±\\pm0\.6 \(1\.25×\\times\)86\.1±\\pm1\.2 \(1\.27×\\times\)114\.8±\\pm2\.4 \(1\.70×\\times\)99\.4 \(1\.43×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}137\.5±\\pm7\.2 \(1\.95×\\times\)115\.9±\\pm0\.5 \(1\.59×\\times\)99\.9±\\pm5\.8 \(1\.48×\\times\)112\.2±\\pm3\.1 \(1\.66×\\times\)116\.4 \(1\.67×\\times\)α=0\.5\\alpha=0\.5CD66\.0±\\pm0\.9 \(1\.00×\\times\)75\.0±\\pm0\.6 \(1\.00×\\times\)72\.8±\\pm1\.0 \(1\.00×\\times\)68\.4±\\pm4\.4 \(1\.00×\\times\)70\.6 \(1\.00×\\times\)SCD76\.4±\\pm0\.8 \(1\.16×\\times\)65\.1±\\pm0\.9 \(0\.87×\\times\)57\.3±\\pm0\.6 \(0\.79×\\times\)63\.6±\\pm0\.6 \(0\.93×\\times\)65\.6 \(0\.93×\\times\)CoS78\.6±\\pm1\.9 \(1\.19×\\times\)73\.6±\\pm1\.9 \(0\.98×\\times\)62\.4±\\pm2\.6 \(0\.86×\\times\)63\.7±\\pm2\.3 \(0\.93×\\times\)69\.6 \(0\.99×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}104\.5±\\pm1\.2 \(1\.58×\\times\)92\.8±\\pm1\.1 \(1\.24×\\times\)87\.2±\\pm0\.9 \(1\.20×\\times\)120\.9±\\pm2\.2 \(1\.77×\\times\)101\.4 \(1\.44×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}142\.9±\\pm8\.9 \(2\.17×\\times\)129\.8±\\pm2\.5 \(1\.73×\\times\)117\.5±\\pm2\.1 \(1\.61×\\times\)131\.0±\\pm8\.7 \(1\.91×\\times\)130\.3 \(1\.85×\\times\)Table 12:Per\-dataset sampling breakdown forT=1T=1andα∈\{0\.1,0\.5\}\\alpha\\in\\\{0\.1,0\.5\\\}\. Each per\-dataset cell reports throughput in tokens/sec as mean±\\pmstandard deviation across three independent runs, with speedup over vanilla CD at the sameα\\alpha\. The Avg\. column reports the unweighted mean of the per\-dataset throughputs; its parenthesized multiplier is the ratio of this mean to the corresponding vanilla\-CD mean\. The DCDngram\{\}\_\{\\textsc\{ngram\}\}rows use the matchedT=1T=1N\-gram proposer setting\.
### E\.2Full Accuracy Sweep Details

This subsection documents the full evaluation configuration underlying Section[3\.4](https://arxiv.org/html/2608.12913#S3.SS4)\.

#### Setup\.

We evaluate the Llama\-3\.1\-8B\-Instruct and Llama\-3\.2\-1B\-Instruct pair on GSM8K and MMLU with 1000 examples per setting, draft lengthγ=5\\gamma\{=\}5, andα∈\{0\.0,0\.1,…,1\.0\}\\alpha\\in\\\{0\.0,0\.1,\\ldots,1\.0\\\}\. We report DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}, SCD, CoS, vanilla CD, and the expert\-only autoregressive baseline under both temperatures, with the matched DCDngram\{\}\_\{\\textsc\{ngram\}\}reference overlaid where available\. Error bars denote 95% confidence intervals\.

[Figure10](https://arxiv.org/html/2608.12913#A5.F10)presents the complete accuracy sweep for both greedy decoding \(T=0T\{=\}0\) and sampling \(T=1T\{=\}1\)\. The matched DCDngram\{\}\_\{\\textsc\{ngram\}\}reference curve is overlaid on both temperature settings, alongside the main EAGLE3\-based DCD trajectory\.

000\.20\.20\.40\.40\.60\.60\.80\.811757578788181848487879090Contrastive Strength \(α\\alpha\)Accuracy \(%\)GSM8K,T=0T\{=\}0

000\.20\.20\.40\.40\.60\.60\.80\.8115555606065657070757580808585Contrastive Strength \(α\\alpha\)GSM8K,T=1T\{=\}1

000\.20\.20\.40\.40\.60\.60\.80\.811606064646868727276768080Contrastive Strength \(α\\alpha\)MMLU,T=0T\{=\}0

000\.20\.20\.40\.40\.60\.60\.80\.8114040454550505555606065657070Contrastive Strength \(α\\alpha\)MMLU,T=1T\{=\}1

DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}DCDngram\{\}\_\{\\textsc\{ngram\}\}SCDCoSVanilla CDAR

Figure 10:Full task\-accuracy sweep across contrastive strengthsα\\alphaunder the actual SGLang setup\. The plot shows DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}, SCD, CoS, and vanilla CD on the Llama\-3\.1\-8B\-Instruct \+ Llama\-3\.2\-1B\-Instruct pair withγ=5\\gamma\{=\}5for GSM8K and MMLU under both greedy decoding \(T=0T\{=\}0\) and sampling \(T=1T\{=\}1\)\. A matched DCDngram\{\}\_\{\\textsc\{ngram\}\}reference is overlaid on both temperature settings where the N\-gram proposer run is available\. Error bars indicate 95% confidence intervals, and the dashed gray line shows the expert\-only autoregressive baseline\. From left to right: GSM8K with greedy decoding \(T=0T\{=\}0\), GSM8K with sampling \(T=1T\{=\}1\), MMLU with greedy decoding \(T=0T\{=\}0\), and MMLU with sampling \(T=1T\{=\}1\)\.

### E\.3Additional Hyperparameter Ablations

Figure[11](https://arxiv.org/html/2608.12913#A5.F11)reports the complementary draft\-length sweep referenced in the main text\. Table[13](https://arxiv.org/html/2608.12913#A5.T13)summarizes the same study in the fixed\-γ\\gammaversus best\-γ\\gammaformat discussed there\. Table[14](https://arxiv.org/html/2608.12913#A5.T14)reports the full DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}throughput grid atα=0\.1\\alpha\{=\}0\.1\. Each row corresponds to one evaluated combination of speculative steps, draft tokens, and top\-kk, and the last two columns report the resulting throughput on GSM8K and MMLU\.

11223344556677111\.51\.522Draft Tokens \(γ\\gamma\)Llama\-3MMLU11223344556677Draft Tokens \(γ\\gamma\)Qwen3MMLU11223344556677Draft Tokens \(γ\\gamma\)Llama\-3GSM8K11223344556677Draft Tokens \(γ\\gamma\)Qwen3GSM8KDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}SCDCoS

Figure 11:Appendix\-onlyγ\\gammaablation\. The y\-axis reports speedup relative to vanilla CD\. DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}stays efficient across a broader draft\-length range, while the amateur\-coupled baselines usually peak earlier as larger draft lengths make their heavier proposal paths less worthwhile\.ModelMethodFixedγ=5\\gamma\{=\}5Bestγ\\gammaBest speedupLlama\-3SCD1\.13×\\times31\.32×\\timesLlama\-3CoS1\.24×\\times21\.40×\\timesLlama\-3DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}1\.57×\\times51\.57×\\timesQwen3SCD1\.08×\\times21\.18×\\timesQwen3CoS1\.06×\\times11\.44×\\timesQwen3DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}1\.64×\\times31\.66×\\timesLlama\-EFTSCD1\.01×\\times31\.18×\\timesLlama\-EFTCoS1\.09×\\times11\.50×\\timesLlama\-EFTDCDeagle3\{\}\_\{\\textsc\{eagle3\}\}1\.60×\\times61\.70×\\timesTable 13:MMLU draft\-length tuning check atT=0T\{=\}0,α=0\.1\\alpha\{=\}0\.1\. Speedups are relative to vanilla CD for the same model pair\. Fixedγ=5\\gamma\{=\}5is the main operating point; Bestγ\\gammaselects the highest\-throughput setting fromγ∈\{1,…,7\}\\gamma\\in\\\{1,\\ldots,7\\\}using the existing chain\-drafting implementation\.ConfigurationThroughput \(tok/s\)Draft TokenskkGSM8KMMLUSteps = 341157\.06157\.06136\.84136\.8463163\.94163\.94145\.23145\.2365160\.91160\.91149\.53149\.53103170\.73170\.73154\.89154\.89105176\.24176\.24160\.24160\.24203178\.13178\.13163\.77163\.77205181\.14168\.51Steps = 561158\.44158\.44134\.78134\.7863160\.45160\.45142\.24142\.2465158\.72158\.72147\.62147\.62103178\.15178\.15156\.10156\.10105169\.37169\.37157\.24157\.24203188\.02188\.02165\.43165\.43205191\.84170\.47Steps = 891144\.19144\.19126\.62126\.6293152\.58152\.58137\.00137\.0095154\.11154\.11138\.35138\.35103156\.26156\.26139\.18139\.18105157\.35157\.35139\.96139\.96203169\.32169\.32148\.55148\.55205173\.45155\.76Table 14:DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}throughput grid atT=0T\{=\}0andα=0\.1\\alpha\{=\}0\.1\. Blocks fix speculative steps; rows vary draft tokens and top\-kk\. Bold marks the block\-wise best throughput for each dataset, and shading denotes the configuration that is best on both GSM8K and MMLU within a block\.

## Appendix FExtended System Evaluation

### F\.1Multi\-Request Speedups on GSM8K

Table[15](https://arxiv.org/html/2608.12913#A6.T15)reports multi\-request serving speedups on GSM8K\. The DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}, SCD, and matched DCDngram\{\}\_\{\\textsc\{ngram\}\}rows use the same serving configuration: five speculative steps with chain\-style greedy drafting and six draft tokens per speculative round\. Each entry reports throughput normalized by vanilla CD at the same request count\.

Method2481624324856SCD1\.291\.331\.321\.281\.161\.441\.210\.87DCDngram\{\}\_\{\\textsc\{ngram\}\}1\.371\.381\.190\.911\.040\.991\.110\.72DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}1\.561\.511\.291\.481\.361\.521\.321\.37Table 15:Configuration\-matched multi\-request serving speedups on the 1000\-question GSM8K sweep\. Columns denote the number of concurrent requests at 2, 4, 8, 16, 24, 32, 48, and 56\. DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}, SCD, and DCDngram\{\}\_\{\\textsc\{ngram\}\}all use 5 speculative steps with chain\-style greedy drafting and 6 draft tokens per speculative round\. Each entry reports the throughput of the corresponding method divided by the throughput of vanilla CD at the same request count; therefore, the vanilla CD baseline is 1\.0 in every column\.
### F\.2Runtime KV\-Cache Occupancy

We examine runtime KV\-cache occupancy in the representativeLlama\-38B setting from the main paper, whereLlama\-3\.1\-8B\-Instructis the expert,Llama\-3\.2\-1B\-Instructis the amateur, and DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}uses the same EAGLE3 drafter\. AR serves only the expert, vanilla CD and SCD keep both the expert and amateur active, and DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}changes only the proposal path through a lightweight feature\-level drafter\. In both the single\-request and shared\-batching settings, vanilla CD, SCD, and DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}show nearly identical steady\-state KV\-cache occupancy because the persistent verification\-time KV state remains dominated by the expert and amateur decoders\.

### F\.3Spec\-Bench Evaluation

To test whether the same speed pattern extends beyond the four main datasets, we also report results on Spec\-Bench\([Xia et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib26)\), a standardized benchmark for speculative decoding with six subtasks: multi\-turn conversation \(MT\-Bench\), translation, summarization, question answering, mathematical reasoning, and retrieval\-augmented generation\. Each subtask contains 80 prompts, giving 480 prompts in total\. Table[16](https://arxiv.org/html/2608.12913#A6.T16)reports per\-subtask throughput and average speedup under this setting\.

MethodMT\-BenchTranslationSumm\.QAMathRAGAvg\. \(Speedup\)Llama\-3\(8B\-Ins / 1B\-Ins\)CD95\.192\.795\.194\.295\.988\.293\.8 \(1\.00×\\times\)SCD132\.6105\.7114\.7122\.6151\.8119\.3125\.6 \(1\.34×\\times\)CoS138\.0107\.2116\.9125\.6159\.0123\.0129\.7 \(1\.38×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}118\.3104\.592\.9107\.4123\.896\.8108\.9 \(1\.16×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}156\.3109\.4146\.4121\.7166\.3142\.3142\.6\(1\.52×\\times\)Qwen3\(8B / 1\.7B\)CD65\.962\.465\.866\.866\.965\.765\.6 \(1\.00×\\times\)SCD76\.275\.566\.266\.097\.269\.875\.3 \(1\.15×\\times\)CoS77\.976\.666\.165\.7101\.870\.076\.6 \(1\.17×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}77\.658\.363\.171\.792\.672\.573\.4 \(1\.12×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}128\.099\.9110\.3114\.6152\.1124\.7122\.5\(1\.87×\\times\)Table 16:Spec\-Bench evaluation results \(tokens/sec\)\. Spec\-Bench\([Xia et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib26)\)is a standardized benchmark for speculative decoding with six sub\-tasks\. All methods useα=0\.1\\alpha\{=\}0\.1,γ=5\\gamma\{=\}5, and greedy decoding\. Speedup is relative to vanilla CD\. The matched DCDngram\{\}\_\{\\textsc\{ngram\}\}reference uses the same setting as an appendix row, while DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}has the highest average throughput for both model families and leads on most sub\-tasks, but not every individual column\.
### F\.470B\-Class Results

We report a greedy\-only 70B extension withLlama\-3\.3\-70B\-Instructas the expert andLlama\-3\.1\-8B\-Instructas the amateur\. The sweep uses the same four datasets andα∈\{0\.0,0\.1,0\.5\}\\alpha\\in\\\{0\.0,0\.1,0\.5\\\}; these results are single\-run diagnostics for checking whether the main trend extends to a larger expert\.

Table[17](https://arxiv.org/html/2608.12913#A6.T17)reports the per\-dataset throughput breakdown for the EAGLE3 and matched N\-gram proposer settings\. Table[18](https://arxiv.org/html/2608.12913#A6.T18)reports the corresponding accepted\-length robustness, normalized to each method atα=0\.0\\alpha=0\.0\.

Methodα\\alphaHumanEvalGSM8KMMLUCNN/DMAvg\.Llama\-3\.3\-70B\-Instruct / Llama\-3\.1\-8B\-Instruct, greedy decodingα=0\.0\\alpha=0\.0CD0\.030\.4 \(1\.00×\\times\)30\.2 \(1\.00×\\times\)30\.2 \(1\.00×\\times\)29\.8 \(1\.00×\\times\)30\.2 \(1\.00×\\times\)SCD0\.061\.6 \(2\.02×\\times\)59\.3 \(1\.96×\\times\)49\.5 \(1\.64×\\times\)50\.8 \(1\.70×\\times\)55\.3 \(1\.83×\\times\)CoS0\.066\.0 \(2\.17×\\times\)62\.6 \(2\.07×\\times\)50\.3 \(1\.67×\\times\)53\.0 \(1\.78×\\times\)58\.0 \(1\.92×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}0\.046\.8 \(1\.54×\\times\)42\.0 \(1\.39×\\times\)38\.0 \(1\.26×\\times\)50\.2 \(1\.69×\\times\)44\.3 \(1\.47×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}0\.079\.6 \(2\.62×\\times\)62\.7 \(2\.07×\\times\)47\.4 \(1\.57×\\times\)55\.5 \(1\.86×\\times\)61\.3 \(2\.03×\\times\)α=0\.1\\alpha=0\.1CD0\.130\.2 \(1\.00×\\times\)30\.1 \(1\.00×\\times\)30\.3 \(1\.00×\\times\)29\.8 \(1\.00×\\times\)30\.1 \(1\.00×\\times\)SCD0\.159\.3 \(1\.96×\\times\)58\.2 \(1\.93×\\times\)48\.7 \(1\.61×\\times\)49\.8 \(1\.67×\\times\)54\.0 \(1\.79×\\times\)CoS0\.162\.3 \(2\.06×\\times\)62\.2 \(2\.07×\\times\)50\.4 \(1\.66×\\times\)52\.2 \(1\.75×\\times\)56\.8 \(1\.89×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}0\.146\.5 \(1\.54×\\times\)40\.5 \(1\.35×\\times\)38\.0 \(1\.26×\\times\)50\.3 \(1\.69×\\times\)43\.8 \(1\.46×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}0\.178\.1 \(2\.58×\\times\)62\.0 \(2\.06×\\times\)47\.5 \(1\.57×\\times\)55\.3 \(1\.85×\\times\)60\.7 \(2\.02×\\times\)α=0\.5\\alpha=0\.5CD0\.530\.2 \(1\.00×\\times\)30\.3 \(1\.00×\\times\)30\.5 \(1\.00×\\times\)29\.8 \(1\.00×\\times\)30\.2 \(1\.00×\\times\)SCD0\.552\.2 \(1\.73×\\times\)57\.2 \(1\.89×\\times\)45\.6 \(1\.50×\\times\)49\.0 \(1\.64×\\times\)51\.0 \(1\.69×\\times\)CoS0\.554\.2 \(1\.79×\\times\)60\.0 \(1\.98×\\times\)47\.4 \(1\.56×\\times\)50\.0 \(1\.68×\\times\)52\.9 \(1\.75×\\times\)DCDngram\{\}\_\{\\textsc\{ngram\}\}0\.546\.1 \(1\.52×\\times\)41\.2 \(1\.36×\\times\)37\.3 \(1\.22×\\times\)49\.5 \(1\.66×\\times\)43\.5 \(1\.44×\\times\)DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}0\.573\.6 \(2\.44×\\times\)60\.8 \(2\.01×\\times\)46\.6 \(1\.53×\\times\)55\.1 \(1\.85×\\times\)59\.1 \(1\.96×\\times\)Table 17:70B\-class extension under greedy decoding\. Values report throughput \(tokens/sec\); speedup in parentheses is relative to vanilla CD at the sameα\\alpha\. All results are single\-run diagnostics using the Llama\-3\.3\-70B\-Instruct expert, the Llama\-3\.1\-8B\-Instruct amateur, and either the corresponding EAGLE3 70B drafter or the matched N\-gram proposer setting\.MethodRel\.@0\.10\.1Rel\.@0\.50\.5Drop to0\.50\.5SCD97\.1%89\.3%\-10\.7%CoS97\.1%89\.3%\-10\.7%DCDngram\{\}\_\{\\textsc\{ngram\}\}99\.2%97\.0%\-3\.0%DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}98\.5%93\.7%\-6\.3%Extra drop vs\. DCDeagle3\{\}\_\{\\textsc\{eagle3\}\}: SCD = \+4\.4 pts, CoS = \+4\.4 pts\.

Table 18:Relative mean accepted length as contrastive strength increases in the 70B\-class extension\. Each ratio is normalized to the same method atα=0\.0\\alpha\{=\}0\.0and averaged across the four datasets under greedy decoding, including the matched N\-gram proposer setting\. Drop is the relative decrease fromα=0\.0\\alpha\{=\}0\.0toα=0\.5\\alpha\{=\}0\.5\.

## Appendix GModel Details

[Table19](https://arxiv.org/html/2608.12913#A7.T19)lists the off\-the\-shelf models and EAGLE3 proposer checkpoints used in the deployment experiments\. The controlled proposal\-side diagnostics that train dual\-input or auxiliary EAGLE heads are described separately in Section[3\.1](https://arxiv.org/html/2608.12913#S3.SS1)and Appendix[A\.2](https://arxiv.org/html/2608.12913#A1.SS2)\.

ConfigurationComponentModelLlama\-3ExpertLlama\-3\.1\-8B\-InstructAmateurLlama\-3\.2\-1B\-InstructDrafterEAGLE3\-LLaMA3\.1\-Instruct\-8BaQwen3ExpertQwen3\-8BAmateurQwen3\-1\.7BDrafterQwen3\-8B\-EAGLE3bLlama\-EFTExpertLlama\-3\.1\-8B\-InstructAmateurLlama\-3\.1\-8BDrafterEAGLE3\-LLaMA3\.1\-Instruct\-8BaLlama\-370B ExtensionExpertLlama\-3\.3\-70B\-InstructAmateurLlama\-3\.1\-8B\-InstructDrafterEAGLE3\-LLaMA3\.3\-Instruct\-70Bc
- ayuhuili/EAGLE3\-LLaMA3\.1\-Instruct\-8B
- bTengyunw/qwen3\_8b\_eagle3
- cyuhuili/EAGLE3\-LLaMA3\.3\-Instruct\-70B

Table 19:Off\-the\-shelf model configurations used in the deployment experiments\.
## Appendix HImplementation Details

#### Experimental Environment\.

Unless noted otherwise, benchmarks run on a single NVIDIA H200 141GB GPU under the SGLang\([Zheng et al\. 2024](https://arxiv.org/html/2608.12913#bib.bib30)\)inference framework\.

#### Artifacts and Intended Use\.

We use publicly released benchmark datasets, model checkpoints, EAGLE3 proposer checkpoints, and the SGLang inference framework under their respective licenses and access terms\. These artifacts are used only for research benchmarking of decoding efficiency and task\-level consistency, and we do not redistribute third\-party datasets or model weights\.

#### Inference Configuration\.

To measure latency cleanly, the main single\-request deployment and diagnostic experiments use single\-card serial execution\. Table[20](https://arxiv.org/html/2608.12913#A8.T20)summarizes the key single\-request runtime parameters\. Runtime KV\-cache occupancy under this setup is examined in Appendix[F\.2](https://arxiv.org/html/2608.12913#A6.SS2); multi\-request serving uses the dedicated concurrent\-request settings in Appendix[F\.1](https://arxiv.org/html/2608.12913#A6.SS1)\. For EAGLE3,speculative\-eagle\-topk=1=1implements chain\-style greedy drafting\. For DCDngram\{\}\_\{\\textsc\{ngram\}\}, the proposer uses a draft\-token budget of 5, an N\-gram match\-window range from 1 to 12, BFS breadth 1 for chain\-style proposal selection, and branch length 18\.

ParameterValuemax\_running\_requests1disable\_cuda\_graphTruedtypebfloat16context\_length4096speculative\-num\-steps5speculative\-eagle\-topk1Table 20:Main single\-request inference configuration\.
#### Dataset and Prompt Configuration\.

For the main deployment and task\-accuracy evaluations, we use the official chat templates for all models and evaluate two settings:

- •Main Experiments \(Speed Analysis\):We use 200 samples per dataset with a maximum generation length of 256 tokens\.
- •Performance Tests \(Accuracy Verification\):We use 1000 samples with a maximum generation length of 1024 tokens and task\-specific zero\-shot templates with structured output formats, for example, “Final Answer:”, to support automated evaluation\.