Understanding Target Probability Reuse in Speculative Decoding

# Understanding Target Probability Reuse in Speculative Decoding
Source: [https://arxiv.org/html/2610.29210](https://arxiv.org/html/2610.29210)
Giyoon JangAffiliation:Soongsil Universityomanma1928@korea\.ac\.krjyjang1130@gmail\.comskacofls26@naver\.comrmawldn1204@naver\.comsihyeon2988@gmail\.comcnrywns1234@naver\.comsongjh041207@gmail\.comeunseosong4916@gmail\.comchanjun\.park@ssu\.ac\.krChailin NamAffiliation:Soongsil Universityomanma1928@korea\.ac\.krjyjang1130@gmail\.comskacofls26@naver\.comrmawldn1204@naver\.comsihyeon2988@gmail\.comcnrywns1234@naver\.comsongjh041207@gmail\.comeunseosong4916@gmail\.comchanjun\.park@ssu\.ac\.krJiwoo KumAffiliation:Soongsil Universityomanma1928@korea\.ac\.krjyjang1130@gmail\.comskacofls26@naver\.comrmawldn1204@naver\.comsihyeon2988@gmail\.comcnrywns1234@naver\.comsongjh041207@gmail\.comeunseosong4916@gmail\.comchanjun\.park@ssu\.ac\.krSihyeon LeeAffiliation:Soongsil Universityomanma1928@korea\.ac\.krjyjang1130@gmail\.comskacofls26@naver\.comrmawldn1204@naver\.comsihyeon2988@gmail\.comcnrywns1234@naver\.comsongjh041207@gmail\.comeunseosong4916@gmail\.comchanjun\.park@ssu\.ac\.krKyojun ChooAffiliation:Soongsil Universityomanma1928@korea\.ac\.krjyjang1130@gmail\.comskacofls26@naver\.comrmawldn1204@naver\.comsihyeon2988@gmail\.comcnrywns1234@naver\.comsongjh041207@gmail\.comeunseosong4916@gmail\.comchanjun\.park@ssu\.ac\.krJihun SongAffiliation:Soongsil Universityomanma1928@korea\.ac\.krjyjang1130@gmail\.comskacofls26@naver\.comrmawldn1204@naver\.comsihyeon2988@gmail\.comcnrywns1234@naver\.comsongjh041207@gmail\.comeunseosong4916@gmail\.comchanjun\.park@ssu\.ac\.krEunseo SongAffiliation:Soongsil Universityomanma1928@korea\.ac\.krjyjang1130@gmail\.comskacofls26@naver\.comrmawldn1204@naver\.comsihyeon2988@gmail\.comcnrywns1234@naver\.comsongjh041207@gmail\.comeunseosong4916@gmail\.comchanjun\.park@ssu\.ac\.krChanjun Park††thanks:Corresponding author\.Affiliation:Soongsil Universityomanma1928@korea\.ac\.krjyjang1130@gmail\.comskacofls26@naver\.comrmawldn1204@naver\.comsihyeon2988@gmail\.comcnrywns1234@naver\.comsongjh041207@gmail\.comeunseosong4916@gmail\.comchanjun\.park@ssu\.ac\.kr

###### Abstract

Speculative decoding uses a target model to verify a tree of draft tokens in parallel\. We study whether the resulting next\-token distributions can help select the next tree\. For unrestricted target models, we show that guaranteed exact reuse requires a distribution computed at the next round’s complete context\. We characterize when retaining this distribution requires sacrificing accepted tokens and establish conditions under which this loss persists across decoding rounds\. Across six targets, we find that reranking first\-token branches with a separately computed distribution at the current context improves expected acceptance over draft ranking, especially with small tree budgets\. Drawing the first token from this distribution before allocating the tree further improves expected acceptance over pruning a preselected tree, even at equal mean tree size\. In our experiments, both selection rules also decode faster than draft ranking when the separate target pass overlaps standalone drafting\. Our findings show that making target probabilities available before tree allocation can improve the efficiency of speculative decoding\. Our code is available at[https://github\.com/js\-lee\-AI/TargetRows](https://github.com/js-lee-AI/TargetRows)\.

![Refer to caption](https://arxiv.org/html/2610.29210v1/roundmap.png)Figure 1:Address matching across decoding rounds\. The dashed link identifies the scored prefixc​a​acaawith the next root; its row isp\(⋅∣caa\)p\(\\cdot\\mid caa\)\.## 1Introduction

Large language models typically generate text one token at a time\([Lee et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib46);[Lee et al\., 2026d](https://arxiv.org/html/2610.29210#bib.bib48);[Lee et al\., 2026e](https://arxiv.org/html/2610.29210#bib.bib47);[Lee et al\., 2026f](https://arxiv.org/html/2610.29210#bib.bib50);[Kang et al\., 2026](https://arxiv.org/html/2610.29210#bib.bib49)\)\. Speculative decoding accelerates this process by letting a cheap drafter propose tokens for the target model to verify in parallel\([Leviathan et al\., 2023](https://arxiv.org/html/2610.29210#bib.bib1);[Chen et al\., 2023](https://arxiv.org/html/2610.29210#bib.bib2);[Lee et al\., 2026c](https://arxiv.org/html/2610.29210#bib.bib51)\)\. Tree\-based methods extend this approach by arranging several candidate continuations into branches and verifying them in a single target pass\([Miao et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib3);[Cai et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib4);[Li et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib5);[Li et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib6);[Lee and Eo, 2026](https://arxiv.org/html/2610.29210#bib.bib52)\)\. To keep the tree within a fixed node budget, the drafter must choose which branches to retain\. The target’s probabilities could guide this choice, yet the tree is usually selected before verification reveals them\. The previous pass offers another source of information because it has already computed next\-token distributions at candidate prefixes\. We call these distributions rows\.

We ask when one of these rows can provide exact next\-token probabilities for selecting the next tree without another target evaluation\. We find that, for an unrestricted target, such reuse is guaranteed only when the row was computed at the next round’s complete context, which we call its address\. Two addresses may end in the same token even when they contain different sequences of preceding tokens\. A cache indexed by the final token can still provide draft candidates\([Luo et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib16)\), but a token match alone does not guarantee exact reuse at the next round’s address\.

A round has*address coverage*when its verification pass has already computed a row at the next round’s address\. Figure[1](https://arxiv.org/html/2610.29210#S0.F1)makes this condition concrete\. Starting from contextcc, suppose verification acceptsaaand then emits anotheraaas its continuation\. The next round starts atc​a​acaa, so it needs the row computed there\. The row atc​acahas a different address because it conditions on the context before the secondaa\. Once retained, the matching row supplies exact first\-token probabilities for selecting the next tree, provided the target computation is unchanged\.

We find an exact trade\-off between address coverage and accepted tokens when a pass scores only its candidate tree\. The result holds for rounds that accept one tree path, emit one continuation token, and preserve the target distribution for each chosen tree\. Coverage occurs exactly when the accepted path plus its continuation remains inside the tree\. Exhaustive traversal instead keeps accepting tokens until a draw leaves the tree, then emits that draw as its continuation\. The next root is therefore unscored\. A covered round could have accepted its continuation as another draft token, so it accepts at least one fewer token than exhaustive traversal on the same target\-token sequence\.

Scoring prefixes one step beyond the candidate tree can cover the next root even after exhaustive traversal\. Adding those same prefixes as candidates would instead let traversal accept one more token on each covered round\.

We then compare coverage over whole runs\. Within this verifier class, consider a decoder with one target pass in each round whose decisions use target information only along its emitted path\. We construct a decoder that is never covered, scores no more prefixes and emits at least as many tokens after every round\. Thus these decoders benefit from the next\-root row by computing it in a separate target pass\.

We evaluate the root row’s value for tree selection separately from the cost of obtaining it\. Our experiments use EAGLE\-3 dynamic trees\([Li et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib6)\)for six targets from 4B to 32B, seven task types and two decoding temperatures\. Root selection ranks candidate paths using target probabilities for their first token and draft probabilities for later tokens\. We find that it increases the expected number of accepted draft tokens over draft ranking, especially with small trees\. Most gains occur when the models’ most likely first tokens differ\.

Root\-draw selection instead draws the first token from the root row before allocating the tree beneath it\. We find higher expected acceptance than pruning a preselected tree to that token’s branch\([Liu et al\., 2025a](https://arxiv.org/html/2610.29210#bib.bib7)\), even at equal mean tree size\. Both selection rules also decode faster than draft ranking when standalone drafting on a second GPU overlaps the separate root pass\.

## 2Related Work

#### Speculation and verification\.

Speculative sampling uses acceptance and correction steps to preserve the target distribution\([Leviathan et al\., 2023](https://arxiv.org/html/2610.29210#bib.bib1);[Chen et al\., 2023](https://arxiv.org/html/2610.29210#bib.bib2)\)\. Tree\-based methods verify several candidate paths in parallel\([Miao et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib3);[Cai et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib4);[Li et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib5);[Li et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib6)\)\. Related studies bound expected acceptance in draft trees\([Pankratov and Alistarh, 2025](https://arxiv.org/html/2610.29210#bib.bib11)\)and analyze how jointly verifying multiple tokens or drafts can improve acceptance while preserving the target distribution\([Sun et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib12);[Hu et al\., 2025b](https://arxiv.org/html/2610.29210#bib.bib13);[Weng et al\., 2026](https://arxiv.org/html/2610.29210#bib.bib14)\)\. We instead characterize when a verification pass has already computed a row at the next round’s complete context and relate this availability to the number of accepted tokens\.

#### Target information and reuse\.

LogitSpec predicts the token after the next one from last\-token logits and uses it to retrieve drafts\([Liu et al\., 2025b](https://arxiv.org/html/2610.29210#bib.bib15)\)\. FlexDraft calibrates draft logits with the resolved continuation token\([Zhang et al\., 2026a](https://arxiv.org/html/2610.29210#bib.bib17)\), while Saguaro drafts ahead for the verification outcomes it predicts\([Kumar et al\., 2026](https://arxiv.org/html/2610.29210#bib.bib18)\)\. Token Recycling keeps the top candidates of every target row computed during verification under that row’s input token and retrieves the next draft tree from this store\([Luo et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib16)\)\. We evaluate full rows indexed by the final token as inputs to tree selection\.

#### Drafting and compute allocation\.

Work on draft generation includes proposer training\([Hu et al\., 2025a](https://arxiv.org/html/2610.29210#bib.bib21);[Zou et al\., 2026](https://arxiv.org/html/2610.29210#bib.bib22);[Lei et al\., 2026](https://arxiv.org/html/2610.29210#bib.bib23);[Lee et al\., 2026g](https://arxiv.org/html/2610.29210#bib.bib53)\), parallel drafting\([Hui et al\., 2026](https://arxiv.org/html/2610.29210#bib.bib24)\), block drafting\([Chen et al\., 2026](https://arxiv.org/html/2610.29210#bib.bib25);[Sandler et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib26);[Liu et al\., 2026](https://arxiv.org/html/2610.29210#bib.bib27);[Lee et al\., 2026a](https://arxiv.org/html/2610.29210#bib.bib33)\)and retrieval\([Zhang et al\., 2026b](https://arxiv.org/html/2610.29210#bib.bib19)\)\. EAGLE\-2, OPT\-Tree, SMART and Sequoia adapt candidate trees to draft confidence or hardware cost\([Li et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib5);[Wang et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib9);[Wang and Zhou, 2026](https://arxiv.org/html/2610.29210#bib.bib20);[Chen et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib10)\)\. Beyond tree construction, agreement between two drafts can guide whether a reasoning model invokes extended thinking\([Lee et al\., 2026b](https://arxiv.org/html/2610.29210#bib.bib34)\)\.

Pipelined methods overlap drafting with target verification\([Liu et al\., 2025a](https://arxiv.org/html/2610.29210#bib.bib7);[McDanel et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib8)\)\. For example, PEARL verifies the first draft token while the drafter proposes the rest, making the root row available during drafting\. We isolate the value of target rows for tree selection by comparing rules on the same draft candidate pools\.

## 3Setting and Reuse Criterion

At roundtt, a target modelppand drafterqqshare committed contextctc\_\{t\}\. The drafter selects a finite, prefix\-closed candidate trieTtT\_\{t\}with at mostBBnon\-root nodes\. Each node represents the token sequenceuufrom the root, with full contextct​uc\_\{t\}u\. The rootϵ\\epsilonrepresents the empty sequence; siblings with the same token are merged\.

One target pass scores a setQtQ\_\{t\}of prefixes, possibly outsideTtT\_\{t\}, and computes rows

ft\(u\)=p\(⋅∣ctu\),u∈Qt\.f\_\{t\}\(u\)=p\(\\cdot\\mid c\_\{t\}u\),\\qquad u\\in Q\_\{t\}\.\(1\)
Letata\_\{t\}be the accepted draft\-token sequence andbtb\_\{t\}the single continuation token\. The next committed context isct\+1=ct​at​btc\_\{t\+1\}=c\_\{t\}a\_\{t\}b\_\{t\}\.

###### Definition 1\(Exact address coverage\)\.

A round is covered whenat​bt∈Qta\_\{t\}b\_\{t\}\\in Q\_\{t\}\. On this event,ft\(atbt\)=p\(⋅∣ct\+1\)f\_\{t\}\(a\_\{t\}b\_\{t\}\)=p\(\\cdot\\mid c\_\{t\+1\}\)is the next round’s root row\.

Without coverage, the rows computed in this pass cannot guarantee exact recovery of the next root row for an unrestricted target\. Two targets can agree on all these rows but differ at the next root\.

For acceptance analysis, a*path\-aligned*verifier accepts only a root path,at∈Tta\_\{t\}\\in T\_\{t\}\.*Conditional exactness*requires the target law given the tree and prior history; marginal exactness requires it after averaging over tree selection\.

## 4Coverage and Verification

### 4\.1Exhaustive traversal and early stopping

For a fixed trieTT, consider a target\-distributed continuationX1,X2,…X\_\{1\},X\_\{2\},\\ldotsof contextcc\. LetLTL\_\{T\}be the number of initial tokens that remain inside the trie\. Its expectation is the sum of the target probabilities of the non\-root paths,

LT\\displaystyle L\_\{T\}=max\{d≥0:X1:d∈T\},\\displaystyle=\\max\\\{d\\geq 0:X\_\{1:d\}\\in T\\\},\(2\)τ∗​\(T\)\\displaystyle\\tau^\{\*\}\(T\)=∑u∈T∖\{ϵ\}p⁡\(u∣c\)\.\\displaystyle=\\sum\_\{u\\in T\\setminus\\\{\\epsilon\\\}\}p\(u\\mid c\)\.The path probabilityp⁡\(u∣c\)p\(u\\mid c\)is the product of target transitions alonguu\. For greedy decoding, the target law is concentrated on the argmax path\.

Exhaustive target sampling accepts target draws until one leaves the trie, then emits that draw as the continuation\. It acceptsLTL\_\{T\}tokens with expected acceptanceτ∗​\(T\)\\tau^\{\*\}\(T\)\([Pankratov and Alistarh, 2025](https://arxiv.org/html/2610.29210#bib.bib11)\)\.

We compare a verifier with exhaustive traversal by extending its output with fresh target draws\. When the verifier is conditionally exact givenTT, the resulting stream follows the target law\. If it is also path\-aligned, itsAAaccepted tokens form an initial path inTT, soA≤LTA\\leq L\_\{T\}; its continuation isXA\+1X\_\{A\+1\}\.

###### Theorem 1\(Coverage and unused path length\)\.

Suppose the verifier is path\-aligned and conditionally exact givenTT, exactly one continuation is emitted, and the scored set is the candidate trie,Q=TQ=T\. Then coverageC=\{X1:A\+1∈Q\}C=\\\{X\_\{1:A\+1\}\\in Q\\\}satisfies

C=\{A<LT\},τ∗​\(T\)−𝔼⁡\[A\]=𝔼⁡\[\(LT−A\)​𝟏C\]\.\\begin\{gathered\}C=\\\{A<L\_\{T\}\\\},\\\\ \\tau^\{\*\}\(T\)\-\\mathbb\{E\}\[A\]=\\mathbb\{E\}\\\!\\left\[\(L\_\{T\}\-A\)\\mathbf\{1\}\_\{C\}\\right\]\.\\end\{gathered\}\(3\)Consequently, exhaustive target sampling has zero coverage\. Positive coverage is equivalent to strictly smaller expected acceptance within this class\.

On the same target stream, coverage means that the accepted path followed by the continuation remains inTT\. Exhaustive traversal would therefore accept at least one more token\. Without coverage, both verifiers acceptLTL\_\{T\}tokens\. We thus obtain a lower bound on the expected acceptance gap:

𝔼⁡\[LT−A\]≥Pr⁡\(C\)\.\\mathbb\{E\}\[L\_\{T\}\-A\]\\geq\\Pr\(C\)\.\(4\)The full coupling proof is given in Appendix[A\.2](https://arxiv.org/html/2610.29210#A1.SS2)\.

#### Scoring beyond the trie\.

We find that scoring outsideTTcan cover the next root even with exhaustive traversal\. LetFFdenote the scored frontier: scored prefixes outsideTTwhose parents lie inTT\. Coverage occurs exactly when the first exit prefix lies inFF, giving

Pr⁡\(C\)\\displaystyle\\Pr\(C\)=∑u∈Fp⁡\(u∣c\)\\displaystyle=\\sum\_\{u\\in F\}p\(u\\mid c\)\(5\)=τ∗​\(T∪F\)−τ∗​\(T\)\.\\displaystyle=\\tau^\{\*\}\(T\\cup F\)\-\\tau^\{\*\}\(T\)\.On the same target stream, exhaustive traversal ofT∪FT\\cup Faccepts one more token exactly on the covered rounds\. Each such prefix could therefore extend the accepted path as a candidate\. More generally, for any scored trieQ⊇TQ\\supseteq T, the same exactness and stopping conditions giveτ∗​\(Q\)−𝔼⁡\[A\]≥Pr⁡\(C\)\\tau^\{\*\}\(Q\)\-\\mathbb\{E\}\[A\]\\geq\\Pr\(C\)\. Proofs are given in Appendix[A\.3](https://arxiv.org/html/2610.29210#A1.SS3)and finite examples in Appendix[B\.1](https://arxiv.org/html/2610.29210#A2.SS1)\.

#### Rejection\-based verifiers\.

We find that speculative sampling\([Leviathan et al\., 2023](https://arxiv.org/html/2610.29210#bib.bib1);[Chen et al\., 2023](https://arxiv.org/html/2610.29210#bib.bib2)\)and its multi\-step form for token trees\([Miao et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib3)\)never cover the next root whenQ=TQ=T\. Their rejection rule suffices without conditional exactness given the tree\. They test a node’s children in turn; rejected tokens retain zero probability in every later residual distribution\. The continuation therefore leaves the trie after all children are rejected, or at a leaf\. Point\-mass proposals, as in EAGLE\-3, give exhaustive target sampling\. Appendix[A\.4](https://arxiv.org/html/2610.29210#A1.SS4)gives the residual argument\.

#### Covering over a run\.

A covered round hands the next round its root row, so Equation[4](https://arxiv.org/html/2610.29210#S4.E4)alone does not settle whether covering pays off over a run\.

###### Theorem 2\(Covering over a run\)\.

Let a decoder’s only target computation after a shared prefill be one pass in each roundttover a trieQt⊇TtQ\_\{t\}\\supseteq T\_\{t\}, and let its verifier be path\-aligned, emit one continuation and be conditionally exact given the selection inputs\.

Suppose the decoder choosesTtT\_\{t\},QtQ\_\{t\}and where each round stops only from its emitted and drawn tokens and from the rows and other target outputs on its emitted path\. Its only other inputs are its drafter, independent randomness and a first token drawn from its held root row\.

Then some decoder preserves the target law, is never covered and scores no more prefixes per round\. On a common target stream, it has emitted no fewer tokens than the original after each round, and more after the original’s covered rounds\.

We construct the comparator on the same target stream with shared independent randomness\. It reconstructs the original tries but scores only its root and their prefixes extending its emitted context\. Exhaustive traversal takes it past every stream prefix scored by the original\. The comparator has scored these prefixes and drawn their next tokens\. It thus holds the rows, hidden states and draws needed to reproduce the original’s choices\.

We extend the comparison to decoders using off\-path target outputs\. Obtaining them may require scoring branches diverging within the comparator’s emitted context\. The extension applies when the cache attention mask permits such scoring\. Appendix[A\.7](https://arxiv.org/html/2610.29210#A1.SS7)gives the construction and proof\.

### 4\.2Adaptive selection preserves exactness

Retained rows can guide tree selection without changing the target distribution\. Letℋt\\mathcal\{H\}\_\{t\}contain the emitted history, retained target rows and drafter information after roundtt\. The next tree may depend onℋt\\mathcal\{H\}\_\{t\}, provided verification is conditionally exact givenℋt\\mathcal\{H\}\_\{t\}and that tree\. Each emitted token then follows the target conditional at its actual prefix\. Multiplying these conditional probabilities gives the target joint distribution for any fixed number of output tokens\.

Retained rows from other addresses can therefore guide tree selection, provided selection does not depend on the verifier’s future random draws\. The verifier must still preserve the target conditional at each emitted prefix\. Directly sampling from a row at another address can change the output distribution, as illustrated in Appendix[A\.6](https://arxiv.org/html/2610.29210#A1.SS6)\.

## 5Experiments

Table 1:Coverage, expected acceptance and shortfall on EAGLE\-3’s5959\-node trees, computed as expectations from the same recorded target rows for every rule\. Scored sets keep exhaustive acceptance on the tree and add the nextM−59M\-59candidates in draft order\.### 5\.1Setup

#### Targets and prompts\.

The six targets are Llama\-3\.1\-8B\-Instruct\([Grattafiori et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib30)\), Qwen3\-4B and Qwen3\-8B\([Yang et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib28)\), and Qwen2\.5\-7B, 14B and 32B\-Instruct\([Yang et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib29)\)\. Each uses a public EAGLE\-3 drafter trained for it\([Li et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib6);[Cen et al\., 2026](https://arxiv.org/html/2610.29210#bib.bib38);[Pan et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib39)\)and its own chat template, with Qwen3 in non\-thinking mode\. We evaluate164164HumanEval problems\([Chen et al\., 2021](https://arxiv.org/html/2610.29210#bib.bib32)\)and480480Spec\-Bench prompts\([Xia et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib35)\), including MT\-bench\([Zheng et al\., 2023](https://arxiv.org/html/2610.29210#bib.bib36)\)\. These cover seven task types\. Each target decodes the644644prompts under a default limit of256256new tokens, with greedy decoding and untruncated sampling at temperature11\.

#### Trees and verification\.

The EAGLE\-3 drafter grows710710candidates at depths one to eight\. Draft selection keeps the top5959non\-root candidates by cumulative draft score, forming EAGLE\-3’s default6060\-token tree including the root\. Verification uses exhaustive target sampling, which matches EAGLE\-3’s joint law for accepted tokens and the continuation\.

To evaluate alternative trees, the pass also scores expanded nodes and the top160160candidates\. Every candidate’s parent is scored, supplying target probabilities for every pool edge\. The verified tree’s rows remain unchanged up to rounding\. Checkpoints, tasks, hardware and the measurement protocol are detailed in Appendix[C](https://arxiv.org/html/2610.29210#A3)\.

Table 2:Expected accepted draft tokens and root\-draw gains over the PEARL\-style decoder at shared contexts\. Same cap uses the statedBB; Equal mean enlarges PEARL\-style preselection to match the mean verified node count\.Figure 2:Gain over draft selection when exact target rows replace draft factors through increasing depths, averaged over six targets\. The upper edge of each band is one selection rule\. The dotted line marks EAGLE\-3’s tree\.Figure 3:Share of the oracle advantage atB=59B=59recovered by root selection, split by whether the drafter’s most probable first token is the target’s most probable token\. Labels give the share of rounds in which the two differ\.
#### Selection values\.

We compare rules at the same recorded contexts and candidate pools\. For trees chosen before the first draw, each rule keeps itsBBhighest\-ranked non\-root candidates, breaking ties by depth and then draft rank\. This setSScontains their non\-root ancestors\. Its expected number of accepted draft tokens under exhaustive traversal is

V⁡\(S\)=∑u∈Sp⁡\(u∣c\)\.V\(S\)=\\sum\_\{u\\in S\}p\(u\\mid c\)\.\(6\)Under greedy decoding,ppplaces all mass on the target argmax at each prefix, in bothVVand scores usingpp\. ThusVVcounts selected nodes on the greedy path\.

Foru=\(x1,…,xd\)u=\(x\_\{1\},\\ldots,x\_\{d\}\), root selection replaces only the first draft factor with the exact root probability,

wroot​\(u\)=p⁡\(x1∣c\)​∏i=2dqi,w\_\{\\mathrm\{root\}\}\(u\)=p\(x\_\{1\}\\mid c\)\\prod\_\{i=2\}^\{d\}q\_\{i\},\(7\)whereqiq\_\{i\}is the drafter’s probability ofxix\_\{i\}\. Depth\-two selection also uses the exact second factor\. The oracle ranks byp⁡\(u∣c\)p\(u\\mid c\)and maximizesVVamong trees fixed before verification\([Wang et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib9);[Chen et al\., 2024](https://arxiv.org/html/2610.29210#bib.bib10)\)\.

Root\-draw selection draws the first token from the root row, then selects it and up toB−1B\-1descendants in draft order\. In contrast, a PEARL\-style decoder\([Liu et al\., 2025a](https://arxiv.org/html/2610.29210#bib.bib7)\)prunes draft selection’s preselected tree after the same draw\. It keeps the drawn token’s subtree, or that token alone if absent from the tree’s first level\. If the pool’s first level lacks the token, both verify it alone\.

Both decoders accept the first draw; their expected acceptance includes that token and averages over the root draw\. Under greedy decoding, root\-draw and root selection have equal expected acceptance when the pool’s first level contains the target argmax\. Reported95%95\\%percentile intervals use2,0002\{,\}000paired prompt\-bootstrap resamples\.

### 5\.2Coverage on EAGLE\-3 trees

Table[1](https://arxiv.org/html/2610.29210#S5.T1)compares stopping rules and scored sets\. We find no coverage under exhaustive traversal with only the tree scored\. Cap 1 accepts at most one draft token; sampled\-leaf stopping emits a draw reaching a leaf as its continuation, accepting one fewer token than exhaustive traversal on covered rounds\.

Extra scoring preserves the original tree’s exhaustive acceptance\. Coverage requires a scored prefix just outside it, which could instead extend the accepted path by one token\.

Table 3:Online decoding atB=59B=59with a separate root pass\. Gain is root\-draw selection’s gain in accepted draft tokens over the PEARL\-style decoder, with95%95\\%intervals\. Nodes gives mean non\-root counts as root draw / PEARL\-style at each decoder’s own contexts\.Table 4:Expected accepted draft tokens at shared contexts for draft selection, cross\-fitted rerankers and root selection\. Base uses draft and contextual features; Rows adds features from retained rows at other addresses\.Table 5:Root selection’s expected\-acceptance gain over draft ranking under a1,0241\{,\}024\-token limit, in percent, for rounds starting once the default output limit is reached\. Brackets give95%95\\%prompt\-bootstrap intervals\.
### 5\.3Rows at matched addresses

Additional scoring provides rows at some complete emitted addresses\. We find that these rows closely match the next root row: mean total variation is below0\.010\.01in every run\. By contrast, rows sharing only the final token have mean total variation between0\.380\.38and0\.470\.47\. The remaining variation for address\-matched rows is comparable to evaluating the same row in batches of different sizes\.

Frequent token matches in the previous tree therefore do not guarantee an exact row for the next root\. Address counts and row agreement are detailed in Appendix[D\.11](https://arxiv.org/html/2610.29210#A4.SS11)\.

### 5\.4Value of exact rows for tree selection

Table[2](https://arxiv.org/html/2610.29210#S5.T2)compares selection rules at shared contexts\. We find that root selection increases expected acceptance over draft ranking by up to8\.0%8\.0\\%at the default budget of5959nodes, with larger gains at88nodes\. Under greedy decoding, root selection cannot shorten the accepted path relative to draft selection\.

Figure[2](https://arxiv.org/html/2610.29210#S5.F2)compares gains over draft ranking from exact target probabilities at increasing depths\. We recover a substantial share of the oracle’s gain with the root row alone, while exact second\-token probabilities further improve allocation within first\-token branches\. Our results show that exact target probabilities are especially useful for small trees, since draft ranking already retains many likely target paths in larger trees\.

#### First\-token agreement\.

Figure[3](https://arxiv.org/html/2610.29210#S5.F3)groups rounds by whether the draft and target have the same most probable first token and compares root selection’s recovery of the oracle advantage\. We find that the less frequent disagreement rounds account for most of the gain\. Root selection rescales candidate scores within each first\-token branch by a common factor, shifting node allocation across branches while leaving later\-token draft probabilities unchanged\. The detailed comparison is given in Appendix[D\.7](https://arxiv.org/html/2610.29210#A4.SS7)\.

Table 6:Root selection’s expected\-acceptance gain over draft ranking atB=59B=59, in percent\. Each cell gives greedy decoding / temperature11\. All aggregates the complete prompt set\.
#### Drawing the first token\.

Table[2](https://arxiv.org/html/2610.29210#S5.T2)compares root\-draw selection and PEARL\-style pruning with the same root row and first draw\. Under the same cap, pruning discards other first\-token branches and retains fewer nodes on average\.

We therefore enlarge the baseline’s preselected tree to match mean verified sizes\. At a5959\-node root\-draw cap, we find up to2\.4%2\.4\\%higher expected acceptance than the enlarged baseline\. This baseline averages more verified nodes than root\-draw selection when the drafter’s most probable first token is drawn, but fewer for other candidate first tokens\. Matching details and intervals are in Appendix[D\.2](https://arxiv.org/html/2610.29210#A4.SS2)\.

Figure 4:Root\-draw selection’s decoding\-speed gain over the PEARL\-style decoder on two RTX A6000 GPUs\. Left: both caps are88\. Right: the PEARL\-style cap is5959and root\-draw caps are4040–4343\.
#### Online decoding\.

Table[3](https://arxiv.org/html/2610.29210#S5.T3)compares the mean number of draft tokens accepted across rounds at each decoder’s own contexts\. We find that root\-draw selection exceeds the PEARL\-style decoder on all six targets at both temperatures, consistent with the shared\-context comparison\. Separately, root\-draw selection retains higher expected acceptance when both trees are evaluated at its contexts with equal mean sizes\. Protocols and additional results appear in Appendix[D\.3](https://arxiv.org/html/2610.29210#A4.SS3)\.

#### Standalone drafters\.

Figure[4](https://arxiv.org/html/2610.29210#S5.F4)shows root\-draw selection’s speed gains over PEARL\-style decoding with standalone drafters\. Llama\-3\.2\-1B\-Instruct\([Meta, 2024](https://arxiv.org/html/2610.29210#bib.bib45)\)drafts for Llama\-3\.1\-8B and Qwen3\-0\.6B\([Yang et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib28)\)for Qwen3\-8B\. On two RTX A6000 GPUs, the root pass finishes during drafting\. Root and root\-draw selection also decode faster than draft selection\.

To bring mean verified tree sizes closer, we lower root\-draw selection’s cap to4040–4343and keep the PEARL\-style cap at5959\. We find that root\-draw selection decodes up to2\.2%2\.2\\%faster than the PEARL\-style decoder while verifying at least as many nodes on average\. These results show that allocating the tree after drawing the first token can improve decoding speed at similar mean tree sizes\. Timing and acceptance details appear in Appendix[D\.14](https://arxiv.org/html/2610.29210#A4.SS14)\.

#### Longer outputs\.

Table[5](https://arxiv.org/html/2610.29210#S5.T5)tests whether root selection’s gains persist beyond the default output limit\. We extend the limit to1,0241\{,\}024tokens for Llama\-3\.1\-8B and Qwen3\-8B and evaluate rounds starting once the default output limit is reached\. We find higher expected acceptance than draft ranking at both temperatures and budgets, with all95%95\\%intervals above zero\. These gains show that exact root probabilities remain useful for tree selection in later decoding rounds\. Counts and whole\-output comparisons appear in Appendix[D\.5](https://arxiv.org/html/2610.29210#A4.SS5)\.

#### Task coverage\.

Table[6](https://arxiv.org/html/2610.29210#S5.T6)reports root selection’s expected\-acceptance gains over draft ranking by task\. We find gains for every evaluated target and task at both temperatures\. Since tasks and prompts contribute different numbers of decoding rounds, we also compute the overall gain with equal task weights or equal prompt weights\. The gain remains positive under both alternatives\. Task\-stratified intervals and weighting details are given in Appendix[D\.4](https://arxiv.org/html/2610.29210#A4.SS4)\.

### 5\.5Rows at other addresses

Table[4](https://arxiv.org/html/2610.29210#S5.T4)tests whether rows from other addresses can improve tree selection when used as features\. We compare two rerankers: Base predicts target probabilities for first\-token candidates from draft and contextual features, while Rows adds features from retained target rows\. Both rerankers replace only the first draft\-probability factor with their predictions\. We train on one half of the prompts and evaluate on the other, then swap their roles\.

We find that Rows improves expected acceptance over both draft ranking and Base for every evaluated target at both temperatures and budgets, with all paired95%95\\%intervals above zero\. Root selection yields higher expected acceptance than either reranker\. By contrast, directly substituting a retained row for the first draft factor lowers expected acceptance in these tests\. Thus, rows from other addresses can guide tree allocation through learned predictions without being used as exact next\-token distributions\. Fitting details and direct\-substitution tests are given in Appendices[D\.8](https://arxiv.org/html/2610.29210#A4.SS8)and[D\.12](https://arxiv.org/html/2610.29210#A4.SS12), respectively\.

## 6Conclusion

We show that covering the next root costs accepted tokens when verification scores only its candidate tree\. The result holds for verifiers that accept one path, emit one continuation and preserve the target distribution conditional on the chosen tree\. For any such decoder using one target pass in each round and target information only along the emitted path, we construct an exact decoder that never covers the next root\. Compared with the original decoder, it scores no more prefixes in each round and has emitted at least as many tokens after every round\.

We also find that a separately computed root row increases expected acceptance over draft ranking, especially when the draft and target favor different first tokens\. Drawing the first token before allocating the tree improves expected acceptance over pruning a preselected tree, even at equal mean verified tree size\. In our measured deployment, the separate pass overlaps a standalone drafter on a second GPU, and both selection rules decode faster than draft selection\. These findings motivate evaluating the cost of obtaining target rows together with the tree\-selection gains they enable\.

## Limitations

Our acceptance study covers six targets with public EAGLE\-3 drafters and recorded target rows\. The timing results apply to batch\-one decoding with a standalone drafter on a second RTX A6000 GPU, where drafting outlasts the separate target pass\.

The coverage identity concerns path\-aligned, conditionally exact verification with one continuation and identical scored and candidate sets\. The run\-level comparison uses target information on the emitted path; its extension to off\-path outputs under a suitable cache attention mask is described in Appendix[A\.7](https://arxiv.org/html/2610.29210#A1.SS7)\.

## Ethics Statement

The experiments analyze model probabilities on public benchmark prompts from Spec\-Bench and HumanEval\. The models and datasets retain their licenses and usage restrictions\. The selection rules we study keep the target’s output distribution and change only how many tokens a round emits\.

## References

- Bojaret al\.\(2014\)O\. Bojar, C\. Buck, C\. Federmann, B\. Haddow, P\. Koehn, J\. Leveling, C\. Monz, P\. Pecina, M\. Post, H\. Saint\-Amand, R\. Soricut, L\. Specia, and A\. TamchynaFindings of the 2014 Workshop on Statistical Machine Translation\.InProceedings of the Ninth Workshop on Statistical Machine Translation,External Links:[Document](https://dx.doi.org/10.3115/v1/w14-3302),[Link](https://doi.org/10.3115/v1/w14-3302)Cited by:[Appendix C](https://arxiv.org/html/2610.29210#A3.SS0.SSS0.Px1.p3.1)\.
- Caiet al\.\(2024\)T\. Cai, Y\. Li, Z\. Geng, H\. Peng, J\. D\. Lee, D\. Chen, and T\. DaoMedusa: simple LLM inference acceleration framework with multiple decoding heads\.InInternational Conference on Machine Learning,External Links:2401\.10774,[Link](https://proceedings.mlr.press/v235/cai24b.html)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1),[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px1.p1.1)\.
- Cenet al\.\(2026\)R\. Cen, Q\. Hu, H\. Huang, H\. Liu, S\. Liu, X\. Luo, L\. Niu, Y\. Tan, D\. Wu, L\. Xie, R\. Yang, G\. Yu, and J\. ZhuAngelSlim: a more accessible, comprehensive, and efficient toolkit for large model compression\.arXiv preprint arXiv:2602\.21233\.External Links:2602\.21233,[Link](https://arxiv.org/abs/2602.21233)Cited by:[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px1.p1.1)\.
- Chenet al\.\(2023\)C\. Chen, S\. Borgeaud, G\. Irving, J\. Lespiau, L\. Sifre, and J\. JumperAccelerating large language model decoding with speculative sampling\.arXiv preprint arXiv:2302\.01318\.External Links:2302\.01318,[Link](https://arxiv.org/abs/2302.01318)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1),[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.29210#S4.SS1.SSS0.Px2.p1.1)\.
- Chenet al\.\(2026\)J\. Chen, Y\. Liang, and Z\. LiuDFlash: block diffusion for flash speculative decoding\.arXiv preprint arXiv:2602\.06036\.External Links:2602\.06036,[Link](https://arxiv.org/abs/2602.06036)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. de Oliveira Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, B\. Chan, S\. Gray, N\. Ryder, M\. Pavlov, A\. Power, L\. Kaiser, M\. Bavarian, C\. Winter, P\. Tillet, F\. P\. Such, D\. Cummings, M\. Plappert, F\. Chantzis, E\. Barnes, A\. Herbert\-Voss, W\. H\. Guss, A\. Nichol, A\. Paino, N\. Tezak, J\. Tang, I\. Babuschkin, S\. Balaji, S\. Jain, W\. Saunders, C\. Hesse, A\. N\. Carr, J\. Leike, J\. Achiam, V\. Misra, E\. Morikawa, A\. Radford, M\. Knight, M\. Brundage, M\. Murati, K\. Mayer, P\. Welinder, B\. McGrew, D\. Amodei, S\. McCandlish, I\. Sutskever, and W\. ZarembaEvaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.External Links:2107\.03374,[Link](https://arxiv.org/abs/2107.03374)Cited by:[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px1.p1.1)\.
- Chenet al\.\(2024\)Z\. Chen, A\. May, R\. Svirschevski, Y\. Huang, M\. Ryabinin, Z\. Jia, and B\. ChenSequoia: scalable and robust speculative decoding\.InAdvances in Neural Information Processing Systems,External Links:2402\.12374,[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/ea1f5f0878d43ff4fb8bf64ef4a2326c-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1),[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px3.p2.2)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining Verifiers to Solve Math Word Problems\.External Links:2110\.14168,[Link](http://arxiv.org/abs/2110.14168v2)Cited by:[Appendix C](https://arxiv.org/html/2610.29210#A3.SS0.SSS0.Px1.p3.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, G\. \(\. Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Y\. \(\. Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. MaThe Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px1.p1.1)\.
- Hermannet al\.\(2015\)K\. M\. Hermann, T\. Kočiský, E\. Grefenstette, L\. Espeholt, W\. Kay, M\. Suleyman, and P\. BlunsomTeaching Machines to Read and Comprehend\.External Links:[Document](https://dx.doi.org/10.48550/arxiv.1506.03340),1506\.03340,[Link](https://arxiv.org/abs/1506.03340v3)Cited by:[Appendix C](https://arxiv.org/html/2610.29210#A3.SS0.SSS0.Px1.p3.1)\.
- Huet al\.\(2025a\)S\. Hu, J\. Li, Z\. Lu, and P\. ZhouBridging draft policy misalignment: group tree optimization for speculative decoding\.arXiv preprint arXiv:2509\.22134\.External Links:2509\.22134,[Link](https://arxiv.org/abs/2509.22134)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Huet al\.\(2025b\)Z\. Hu, T\. Zheng, V\. Viswanathan, Z\. Chen, R\. A\. Rossi, Y\. Wu, D\. Manocha, and H\. HuangTowards optimal multi\-draft speculative decoding\.InInternational Conference on Learning Representations,External Links:2502\.18779,[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/0907335ecf28faf15be54485dbcbe70e-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px1.p1.1)\.
- Huiet al\.\(2026\)M\. Hui, X\. Huang, J\. Campos Salas, Y\. Sun, N\. Pemberton, X\. Song, A\. Khetan, and G\. KarypisP\-EAGLE: parallel\-drafting EAGLE with scalable training\.arXiv preprint arXiv:2602\.01469\.External Links:2602\.01469,[Link](https://arxiv.org/abs/2602.01469)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Kanget al\.\(2026\)M\. Kang, J\. Lee, J\. Seo, and H\. LimThinkFuse: trajectory\-aware test\-time fusion for small reasoning models\.External Links:2610\.07803,[Link](https://arxiv.org/abs/2610.07803)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1)\.
- Karpukhinet al\.\(2020\)V\. Karpukhin, B\. Oguz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. YihDense Passage Retrieval for Open\-Domain Question Answering\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.550),[Link](https://doi.org/10.18653/v1/2020.emnlp-main.550)Cited by:[Appendix C](https://arxiv.org/html/2610.29210#A3.SS0.SSS0.Px1.p3.1)\.
- Kumaret al\.\(2026\)T\. Kumar, T\. Dao, and A\. MaySpeculative speculative decoding\.arXiv preprint arXiv:2603\.03251\.External Links:2603\.03251,[Link](https://arxiv.org/abs/2603.03251)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px2.p1.1)\.
- Kwiatkowskiet al\.\(2019\)T\. Kwiatkowski, J\. Palomaki, O\. Redfield, M\. Collins, A\. Parikh, C\. Alberti, D\. Epstein, I\. Polosukhin, J\. Devlin, K\. Lee, K\. Toutanova, L\. Jones, M\. Kelcey, M\. Chang, A\. M\. Dai, J\. Uszkoreit, Q\. Le, and S\. PetrovNatural Questions: A Benchmark for Question Answering Research\.Transactions of the Association for Computational Linguistics\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00276),[Link](https://doi.org/10.1162/tacl_a_00276)Cited by:[Appendix C](https://arxiv.org/html/2610.29210#A3.SS0.SSS0.Px1.p3.1)\.
- Lee and Eo \(2026\)J\. Lee and S\. EoCAST: cost\-aware speculative trees from one\-pass block drafters\.External Links:2610\.00321,[Link](https://arxiv.org/abs/2610.00321)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1)\.
- Leeet al\.\(2026a\)J\. Lee, S\. Hong, D\. J\. Lee, C\. Park, J\. Seo, S\. Eo, and H\. LimVision is not overhead: one\-pass block drafting for lossless speculative decoding in vision\-language models\.arXiv preprint arXiv:2609\.00355\.External Links:2609\.00355,[Link](https://arxiv.org/abs/2609.00355)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Leeet al\.\(2026b\)J\. Lee, S\. Hong, S\. Lee, J\. Seo, J\. Son, S\. Eo, C\. Park, H\. Park, H\. Moon, and H\. LimDART: draft\-agreement routing for training\-free adaptive thinking budgets in hybrid reasoning models\.arXiv preprint arXiv:2606\.23181\.External Links:2606\.23181,[Link](https://arxiv.org/abs/2606.23181)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Leeet al\.\(2025\)J\. Lee, S\. Hong, H\. Moon, and H\. LimCross\-lingual optimization for language transfer in large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 15100–15119\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.734),[Link](https://aclanthology.org/2025.acl-long.734/)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1)\.
- Leeet al\.\(2026c\)J\. Lee, D\. J\. Lee, C\. Park, S\. Eo, and H\. LimFaster block\-diffusion serving with distribution\-free risk guarantees\.External Links:2609\.33887,[Link](https://arxiv.org/abs/2609.33887)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1)\.
- Leeet al\.\(2026d\)J\. Lee, S\. Lee, S\. Hong, S\. Eo, and H\. LimWhere activation sparsity and KV\-cache sparsity cross in LLM decoding\.External Links:2609\.33889,[Link](https://arxiv.org/abs/2609.33889)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1)\.
- Leeet al\.\(2026e\)J\. Lee, S\. Lee, S\. Hong, M\. Kim, C\. Park, and H\. LimBeyond penalizing mistakes: stabilizing efficiency training in large reasoning models via adaptive correct\-only rewards\.External Links:2606\.22716,[Link](https://arxiv.org/abs/2606.22716)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1)\.
- Leeet al\.\(2026f\)J\. Lee, S\. Lee, S\. Son, D\. J\. Lee, S\. Han, S\. Eo, and H\. LimAnswer\-conditioned chains of thought degrade verifiable\-reasoning distillation in large language models\.External Links:2607\.14552,[Link](https://arxiv.org/abs/2607.14552)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1)\.
- Leeet al\.\(2026g\)J\. Lee, C\. Park, S\. Eo, and H\. MoonRecovering off\-policy supervision for speculative decoding\.External Links:2609\.38795,[Link](https://arxiv.org/abs/2609.38795)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Leiet al\.\(2026\)H\. Lei, Y\. Li, H\. Zhang, S\. Zhang, Q\. Cheng, X\. Qu, G\. Cui, B\. Zhou, N\. Ding, Y\. Luo, and Y\. ChengDraft\-OPD: on\-policy distillation for speculative draft models\.arXiv preprint arXiv:2605\.29343\.External Links:2605\.29343,[Link](https://arxiv.org/abs/2605.29343)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Leviathanet al\.\(2023\)Y\. Leviathan, M\. Kalman, and Y\. MatiasFast inference from transformers via speculative decoding\.InInternational Conference on Machine Learning,External Links:2211\.17192,[Link](https://proceedings.mlr.press/v202/leviathan23a.html)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1),[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.29210#S4.SS1.SSS0.Px2.p1.1)\.
- Liet al\.\(2024\)Y\. Li, F\. Wei, C\. Zhang, and H\. ZhangEAGLE\-2: faster inference of language models with dynamic draft trees\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.422),2406\.16858,[Link](https://aclanthology.org/2024.emnlp-main.422/)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1),[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Liet al\.\(2025\)Y\. Li, F\. Wei, C\. Zhang, and H\. ZhangEAGLE\-3: scaling up inference acceleration of large language models via training\-time test\.InAdvances in Neural Information Processing Systems,Vol\.38\.External Links:[Document](https://dx.doi.org/10.52202/085713-4562)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1),[§1](https://arxiv.org/html/2610.29210#S1.p7.1),[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px1.p1.1)\.
- Liuet al\.\(2026\)F\. Liu, X\. Li, K\. Zhao, Y\. Gao, Z\. Zhou, Z\. Zhang, Z\. Wang, W\. Dou, S\. Zhong, and C\. TianDART: diffusion\-inspired speculative decoding for fast LLM inference\.arXiv preprint arXiv:2601\.19278\.External Links:2601\.19278,[Link](https://arxiv.org/abs/2601.19278)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Liuet al\.\(2025a\)T\. Liu, Y\. Li, Q\. Lv, K\. Liu, J\. Zhu, W\. Hu, and X\. SunPEARL: parallel speculative decoding with adaptive draft length\.InInternational Conference on Learning Representations,External Links:2408\.11850,[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/03b1043052700b1a471996b0baf309d4-Abstract-Conference.html)Cited by:[§D\.14](https://arxiv.org/html/2610.29210#A4.SS14.p1.1),[§D\.2](https://arxiv.org/html/2610.29210#A4.SS2.p4.1),[§1](https://arxiv.org/html/2610.29210#S1.p8.1),[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p2.1),[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px3.p3.1)\.
- Liuet al\.\(2025b\)T\. Liu, Q\. Lv, H\. Li, X\. Gao, X\. Sun, and X\. SunLogitSpec: accelerating retrieval\-based speculative decoding via next next token speculation\.arXiv preprint arXiv:2507\.01449\.External Links:2507\.01449,[Link](https://arxiv.org/abs/2507.01449)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px2.p1.1)\.
- Luoet al\.\(2025\)X\. Luo, Y\. Wang, Q\. Zhu, Z\. Zhang, X\. Zhang, Q\. Yang, and D\. XuTurning trash into treasure: accelerating inference of large language models with token recycling\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 6816–6831\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.338),2408\.08696,[Link](https://aclanthology.org/2025.acl-long.338/)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p2.1),[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px2.p1.1)\.
- McDanelet al\.\(2025\)B\. McDanel, S\. Q\. Zhang, Y\. Hu, and Z\. LiuPipeSpec: breaking stage dependencies in hierarchical LLM decoding\.arXiv preprint arXiv:2505\.01572\.External Links:2505\.01572,[Link](https://arxiv.org/abs/2505.01572)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p2.1)\.
- Meta \(2024\)MetaLlama\-3\.2\-1B\-Instruct\.External Links:[Link](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)Cited by:[§D\.13](https://arxiv.org/html/2610.29210#A4.SS13.p1.1),[§5\.4](https://arxiv.org/html/2610.29210#S5.SS4.SSS0.Px4.p1.1)\.
- Miaoet al\.\(2024\)X\. Miao, G\. Oliaro, Z\. Zhang, X\. Cheng, Z\. Wang, Z\. Zhang, R\. Y\. Y\. Wong, A\. Zhu, L\. Yang, X\. Shi, C\. Shi, Z\. Chen, D\. Arfeen, R\. Abhyankar, and Z\. JiaSpecInfer: accelerating large language model serving with tree\-based speculative inference and verification\.InProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3,External Links:[Document](https://dx.doi.org/10.1145/3620666.3651335),2305\.09781,[Link](https://doi.org/10.1145/3620666.3651335)Cited by:[§1](https://arxiv.org/html/2610.29210#S1.p1.1),[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.29210#S4.SS1.SSS0.Px2.p1.1)\.
- Nallapatiet al\.\(2016\)R\. Nallapati, B\. Zhou, C\. dos Santos, C\. Gulcehre, and B\. XiangAbstractive Text Summarization using Sequence\-to\-sequence RNNs and Beyond\.InProceedings of The 20th SIGNLL Conference on Computational Natural Language Learning,External Links:[Document](https://dx.doi.org/10.18653/v1/k16-1028),[Link](https://doi.org/10.18653/v1/k16-1028)Cited by:[Appendix C](https://arxiv.org/html/2610.29210#A3.SS0.SSS0.Px1.p3.1)\.
- Panet al\.\(2025\)R\. Pan, Z\. Chen, H\. Liu, A\. Krishnamurthy, and R\. NetravaliFail fast, win big: rethinking the drafting strategy in speculative decoding via diffusion LLMs\.arXiv preprint arXiv:2512\.20573\.External Links:2512\.20573,[Link](https://arxiv.org/abs/2512.20573)Cited by:[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px1.p1.1)\.
- Pankratov and Alistarh \(2025\)S\. Pankratov and D\. AlistarhSpeculative decoding speed\-of\-light: optimal lower bounds via branching random walks\.arXiv preprint arXiv:2512\.11718\.External Links:2512\.11718,[Link](https://arxiv.org/abs/2512.11718)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.29210#S4.SS1.p2.1)\.
- Paszkeet al\.\(2019\)A\. Paszke, S\. Gross, F\. Massa, A\. Lerer, J\. Bradbury, G\. Chanan, T\. Killeen, Z\. Lin, N\. Gimelshein, L\. Antiga, A\. Desmaison, A\. Kopf, E\. Yang, Z\. DeVito, M\. Raison, A\. Tejani, S\. Chilamkurthy, B\. Steiner, L\. Fang, J\. Bai, and S\. ChintalaPyTorch: an imperative style, high\-performance deep learning library\.InAdvances in Neural Information Processing Systems,Vol\.32\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2019/hash/bdbca288fee7f92f2bfa9f7012727740-Abstract.html)Cited by:[Appendix C](https://arxiv.org/html/2610.29210#A3.SS0.SSS0.Px1.p1.1)\.
- Sandleret al\.\(2025\)J\. Sandler, J\. K\. Christopher, T\. Hartvigsen, and F\. FiorettoSpecDiff\-2: scaling diffusion drafter alignment for faster speculative decoding\.arXiv preprint arXiv:2511\.00606\.External Links:2511\.00606,[Link](https://arxiv.org/abs/2511.00606)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Sunet al\.\(2025\)Z\. Sun, U\. Mendlovic, Y\. Leviathan, A\. Aharoni, J\. H\. Ro, A\. Beirami, and A\. T\. SureshBlock verification accelerates speculative decoding\.InInternational Conference on Learning Representations,External Links:2403\.10444,[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/3e710b42b1a9ed898f607ec0f4fcc971-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2025\)J\. Wang, Y\. Su, J\. Li, Q\. Xia, Z\. Ye, X\. Duan, Z\. Wang, and M\. ZhangOPT\-Tree: speculative decoding with adaptive draft tree structure\.Transactions of the Association for Computational Linguistics\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00735),2406\.17276,[Link](https://aclanthology.org/2025.tacl-1.8/)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1),[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px3.p2.2)\.
- Wang and Zhou \(2026\)L\. Wang and P\. ZhouSMART: when is it actually worth expanding a speculative tree?\.arXiv preprint arXiv:2604\.09731\.External Links:2604\.09731,[Link](https://arxiv.org/abs/2604.09731)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Wenget al\.\(2026\)Y\. Weng, Q\. Hu, and T\. YairiUniVer: a unified perspective for multi\-step and multi\-draft speculative decoding\.arXiv preprint arXiv:2605\.04543\.External Links:2605\.04543,[Link](https://arxiv.org/abs/2605.04543)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px1.p1.1)\.
- Xiaet al\.\(2024\)H\. Xia, Z\. Yang, Q\. Dong, P\. Wang, Y\. Li, T\. Ge, T\. Liu, W\. Li, and Z\. SuiUnlocking efficiency in large language model inference: a comprehensive survey of speculative decoding\.InFindings of the Association for Computational Linguistics: ACL 2024,Bangkok, Thailand,pp\. 7655–7671\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.456),[Link](https://aclanthology.org/2024.findings-acl.456/)Cited by:[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px1.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. QiuQwen3 technical report\.arXiv preprint arXiv:2505\.09388\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§D\.13](https://arxiv.org/html/2610.29210#A4.SS13.p1.1),[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px1.p1.1),[§5\.4](https://arxiv.org/html/2610.29210#S5.SS4.SSS0.Px4.p1.1)\.
- Yanget al\.\(2024\)A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. QiuQwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2026a\)Y\. Zhang, J\. Huang, J\. Ke, Y\. Han, Y\. Long, T\. Zhao, B\. Qi, and L\. ZhangFlexDraft: flexible speculative decoding via attention tuning and bonus\-guided calibration\.arXiv preprint arXiv:2605\.20022\.External Links:2605\.20022,[Link](https://arxiv.org/abs/2605.20022)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2026b\)Z\. Zhang, Z\. Li, L\. Zhang, P\. Wang, and H\. ZhaoRACER: retrieval\-augmented contextual rapid speculative decoding\.arXiv preprint arXiv:2604\.14885\.External Links:2604\.14885,[Link](https://arxiv.org/abs/2604.14885)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.
- Zhenget al\.\(2023\)L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing, H\. Zhang, J\. Gonzalez, and I\. StoicaJudging LLM\-as\-a\-judge with MT\-bench and chatbot arena\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 46595–46623\.Cited by:[Appendix C](https://arxiv.org/html/2610.29210#A3.SS0.SSS0.Px1.p3.1),[§5\.1](https://arxiv.org/html/2610.29210#S5.SS1.SSS0.Px1.p1.1)\.
- Zouet al\.\(2026\)X\. Zou, J\. Li, J\. Huang, and P\. ZhouVariational speculative decoding: rethinking draft training from token likelihood to sequence acceptance\.arXiv preprint arXiv:2602\.05774\.External Links:2602\.05774,[Link](https://arxiv.org/abs/2602.05774)Cited by:[§2](https://arxiv.org/html/2610.29210#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix AProofs and Scope

### A\.1Cache\-only recovery

Fix a finite set of queried prefix addressesQQand a requested addressz∉Qz\\notin Q\. Define two autoregressive targetsp0,p1p\_\{0\},p\_\{1\}that assign the same next\-token row at every queried address, but different rows atzz\. Such targets exist because the class imposes no relation between transition rows at different prefixes\. They produce identical row\-query transcripts\. A deterministic recovery function must return the same answer on both transcripts, so it cannot equal bothp0\(⋅∣z\)p\_\{0\}\(\\cdot\\mid z\)andp1\(⋅∣z\)p\_\{1\}\(\\cdot\\mid z\)\. Randomization cannot provide exact recovery with probability one for both targets either\.

The query model imposes no structure linking rows\. Known finite\-state structure, a context\-independent distribution or an exactly matching drafter can determine a missing row using additional information\. A learned approximation can also improve selection without exact recovery\.

### A\.2Fixed\-tree ceiling and coverage identity

Fix the prior context and candidate trieTT\. Extend theA\+1A\+1tokens that the round emits with fresh target draws at the growing context\. Conditional exactness givenTTmakes every emitted token target\-distributed given the tokens before it\. The fresh draws have the same property by construction\.

The extended sequence is therefore an infinite target\-distributed streamXXwithTTheld fixed\. Its firstAAtokens are accepted, andXA\+1X\_\{A\+1\}is the continuation\. Exactness must hold given the realized tree becauseLTL\_\{T\}is computed on that tree\. Path alignment givesA≤LTA\\leq L\_\{T\}pointwise\. The tail\-sum identity now gives

𝔼⁡\[LT\]\\displaystyle\\mathbb\{E\}\[L\_\{T\}\]=∑d≥1Pr\(X1:d∈T\)\\displaystyle=\\sum\_\{d\\geq 1\}\\Pr\(X\_\{1:d\}\\in T\)\(8\)=∑u∈T∖\{ϵ\}p⁡\(u∣c\)=τ∗​\(T\)\.\\displaystyle=\\sum\_\{u\\in T\\setminus\\\{\\epsilon\\\}\}p\(u\\mid c\)=\\tau^\{\*\}\(T\)\.At each depth, distinct trie paths represent disjoint events\. The trie is finite, so the sum is finite\. Exhaustive target sampling accepts the maximal initial path and attains this expectation\.

WhenQ=TQ=T, the continuation is covered exactly whenX1:A\+1X\_\{1:A\+1\}remains insideTT\. Prefix closure makes this equivalent toA<LTA<L\_\{T\}\. ThusLT−AL\_\{T\}\-Avanishes outside coverage, proving Equation[3](https://arxiv.org/html/2610.29210#S4.E3)\. Since the difference is a nonnegative integer, its expectation is positive exactly when coverage has positive probability\.

BesidesQ=TQ=T, the proof uses three properties of the verifier, namely path alignment, conditional exactness given the tree and a single continuation\. Appendix[C](https://arxiv.org/html/2610.29210#A3)shows that EAGLE\-3’s tree verifier has all three\.

### A\.3Scored frontier

Let the scored setQQstrictly containTT, and letFFcontain the prefixes inQ∖TQ\\setminus Twhose parents lie inTT\. Exhaustive traversal onTTacceptsX1:LTX\_\{1:L\_\{T\}\}and emits the addressX1:LT\+1X\_\{1:L\_\{T\}\+1\}\. This address lies outsideTT, but its parent lies inTT\. The round is therefore covered exactly when the address belongs toFF\.

Conversely, ifX1:\|u\|=uX\_\{1:\|u\|\}=ufor someu∈Fu\\in F, every proper prefix ofuulies inTT, so traversal exits exactly atuu\. Two distinct frontier elements cannot both be prefixes ofXX: the shorter would be a prefix of the longer one’s parent and would then lie inTT\. These exit events are disjoint, giving

Pr⁡\(C\)\\displaystyle\\Pr\(C\)=∑u∈FPr\(X1:\|u\|=u\)\\displaystyle=\\sum\_\{u\\in F\}\\Pr\(X\_\{1:\|u\|\}=u\)\(9\)=∑u∈Fp⁡\(u∣c\)\.\\displaystyle=\\sum\_\{u\\in F\}p\(u\\mid c\)\.Every element ofFFhas its parent inTT, soT∪FT\\cup Fis a trie\. Its newly added nodes have no children inT∪FT\\cup F\. On the same stream, exhaustive traversal of this enlarged trie therefore accepts one additional token exactly on coverage:

LT∪F\\displaystyle L\_\{T\\cup F\}=LT\+𝟏C,\\displaystyle=L\_\{T\}\+\\mathbf\{1\}\_\{C\},\(10\)τ∗​\(T∪F\)−τ∗​\(T\)\\displaystyle\\tau^\{\*\}\(T\\cup F\)\-\\tau^\{\*\}\(T\)=Pr⁡\(C\)\.\\displaystyle=\\Pr\(C\)\.The second line takes expectations of the first using the tail\-sum identity\.

Now let a verifier with the three properties acceptAAtokens onTT, and letQ⊇TQ\\supseteq Tbe a scored trie\. On coverage,X1:A\+1∈QX\_\{1:A\+1\}\\in Q, soLQ≥A\+1L\_\{Q\}\\geq A\+1\. Otherwise,LQ≥LT≥AL\_\{Q\}\\geq L\_\{T\}\\geq A\. Hence

LQ−A\\displaystyle L\_\{Q\}\-A≥𝟏C,\\displaystyle\\geq\\mathbf\{1\}\_\{C\},\(11\)τ∗​\(Q\)−𝔼​\[A\]\\displaystyle\\tau^\{\*\}\(Q\)\-\\mathbb\{E\}\[A\]≥Pr⁡\(C\)\.\\displaystyle\\geq\\Pr\(C\)\.Equality holds for exhaustive traversal whenQ=T∪FQ=T\\cup F\.

### A\.4Rejection\-based verifiers

Speculative sampling and its multi\-step form for token trees try a node’s children in turn\. A childxxproposed byqqis accepted with probabilitymin⁡\{1,r⁡\(x\)/q⁡\(x\)\}\\min\\\{1,r\(x\)/q\(x\)\\\}, whererrstarts as the node’s target row\. Each rejection replacesrrwith the normalizedmax⁡\{r−q,0\}\\max\\\{r\-q,0\\\}\.

Rejectingxxrequiresr⁡\(x\)<q⁡\(x\)r\(x\)<q\(x\), soxxhas zero mass under every later residual row\. The continuation drawn after the last rejection is therefore none of the node’s children, and a continuation drawn at a leaf also leaves the trie\. When the pass scores only the trie, the next root is thus never scored, although these verifiers need not be conditionally exact given the tree\.

### A\.5Adaptive selection

Letccbe the initial context andX1,X2,…X\_\{1\},X\_\{2\},\\ldotsthe stream emitted across rounds\. Condition on the full emitted history before tokenjjand on the past information used to select the current tree\. The tree is a function of that information\. Thus conditional exactness given the information and the tree gives

Pr⁡\(Xj=x∣X<j,past information\)\\displaystyle\\Pr\(X\_\{j\}=x\\mid X\_\{<j\},\\text\{past information\}\)\(12\)=p⁡\(x∣c​X<j\)\.\\displaystyle=p\(x\\mid cX\_\{<j\}\)\.Taking conditional expectations and multiplying these token\-level factors yields the target autoregressive law at any fixed horizon\. The grouping into rounds can depend on token values without changing this factorization\.

### A\.6A row at the wrong branch

Consider a tree whose root has childrenaaandbb\. Let the target be deterministic with transitionsc↦ac\\mapsto a,c​a↦bca\\mapsto b,c​a​b↦acab\\mapsto a, andc​b↦bcb\\mapsto b\. A rule that verifies the siblingsa,ba,bas one sequence can accept both by consuming rows atccandc​aca, then take its continuation from the row scored atc​bcb\. It emitsa​b​babb\. Autoregressive decoding emitsa​b​aaba, because the third row must be conditioned onc​a​bcab\. Exactness of every scored row at its own address therefore does not ensure exactness of a sequence assembled from incorrect addresses\.

### A\.7Covering over a run

###### Proof of Theorem[2](https://arxiv.org/html/2610.29210#Thmtheorem2)\.

*Common target stream\.*Every scored row is exact\. Extend each round’s output with fresh target draws as above\. The result is a target stream whose firstA\+1A\+1tokens are the round’s output\. When the first token is drawn before selection, it comes from the held root row and the verifier is exact given that draw, so the same construction applies\.

To realize this coupling, draw target tokens in order\. At eachXjX\_\{j\}, declare it the continuation with the round’s conditional stopping probability givenX1:jX\_\{1:j\}, the tree, the history and survival to that position\. This decision does not depend on later draws, which are fresh target draws after a stop\. With randomness independent of the streamXX, the first decoder therefore emitsX1:ntX\_\{1:n\_\{t\}\}after roundtt, starting fromn0=0n\_\{0\}=0and the shared prefill contextcc\.

*Comparator construction\.*Let the second decoder emitX1:nt′X\_\{1:n^\{\\prime\}\_\{t\}\}, withn0′=0n^\{\\prime\}\_\{0\}=0\. Assume inductively thatnt′≥ntn^\{\\prime\}\_\{t\}\\geq n\_\{t\}\. The extra tokens it has already emitted form the stringut=Xnt\+1:nt′u\_\{t\}=X\_\{n\_\{t\}\+1:n^\{\\prime\}\_\{t\}\}, which is empty when the lengths agree\.

Using the same independent randomness, the second decoder computes the first decoder’s triesTt\+1T\_\{t\+1\}andQt\+1Q\_\{t\+1\}atcX1:ntcX\_\{1:n\_\{t\}\}\. It then scores and verifies the trie

Qt\+1′=\{ϵ\}∪\{w≠ϵ:ut​w∈Qt\+1\}\.\\displaystyle Q^\{\\prime\}\_\{t\+1\}=\\\{\\epsilon\\\}\\cup\\\{w\\neq\\epsilon:u\_\{t\}w\\in Q\_\{t\+1\}\\\}\.\(13\)These are the root and the scored prefixes that extend its own emitted context\. It traversesQt\+1′Q^\{\\prime\}\_\{t\+1\}exhaustively, taking draws fromXX\. Its scored and candidate tries coincide, so Theorem[1](https://arxiv.org/html/2610.29210#Thmtheorem1)gives zero coverage\.

*Emitted lengths and query count\.*LetLLbe the largestddwithXnt\+1:nt\+d∈Qt\+1X\_\{n\_\{t\}\+1:n\_\{t\}\+d\}\\in Q\_\{t\+1\}\. The first decoder accepts only withinTt\+1⊆Qt\+1T\_\{t\+1\}\\subseteq Q\_\{t\+1\}, so its accepted length satisfiesA≤LA\\leq Land

nt\+1=nt\+A\+1≤nt\+L\+1\.n\_\{t\+1\}=n\_\{t\}\+A\+1\\leq n\_\{t\}\+L\+1\.\(14\)If its round is covered, the continuation also lies inQt\+1Q\_\{t\+1\}, giving the stronger boundnt\+1≤nt\+Ln\_\{t\+1\}\\leq n\_\{t\}\+L\.

Givennt′≥ntn^\{\\prime\}\_\{t\}\\geq n\_\{t\}, every remaining stream prefix through positionnt\+Ln\_\{t\}\+Llies inQt\+1′Q^\{\\prime\}\_\{t\+1\}\. The next token exitsQt\+1Q\_\{t\+1\}\. Ifnt′≤nt\+Ln^\{\\prime\}\_\{t\}\\leq n\_\{t\}\+L, exhaustive traversal therefore emits throughXnt\+L\+1X\_\{n\_\{t\}\+L\+1\}\. Otherwise it emits one token\. Consequently,

nt\+1′\\displaystyle n^\{\\prime\}\_\{t\+1\}=max⁡\(nt′,nt\+L\)\+1\\displaystyle=\\max\(n^\{\\prime\}\_\{t\},\\,n\_\{t\}\+L\)\+1\(15\)≥nt\+L\+1\.\\displaystyle\\geq n\_\{t\}\+L\+1\.This is at leastnt\+1n\_\{t\+1\}and strictly larger after a covered round, completing the length induction\.

The mapw↦ut​ww\\mapsto u\_\{t\}winjects the nonempty strings ofQt\+1′Q^\{\\prime\}\_\{t\+1\}intoQt\+1∖\{ϵ\}Q\_\{t\+1\}\\setminus\\\{\\epsilon\\\}\. Including the root, the comparator scores at most\|Qt\+1\|\|Q\_\{t\+1\}\|prefixes\. WhenQt\+1=Tt\+1Q\_\{t\+1\}=T\_\{t\+1\}, it also has at mostBBnon\-root nodes\.

*Available information\.*It remains to check, in the same induction, that the comparator uses only its own past\. After roundtt, it has scored every prefixX1:jX\_\{1:j\}withj<nt′j<n^\{\\prime\}\_\{t\}, as a tree node or a round’s root\. If the first decoder scoredX1:jX\_\{1:j\}in rounds≤ts\\leq t, thenXns−1\+1:j∈QsX\_\{n\_\{s\-1\}\+1:j\}\\in Q\_\{s\}\. Thusj−ns−1j\-n\_\{s\-1\}is at most that round’sLL, and Equation[15](https://arxiv.org/html/2610.29210#A1.E15)givesnt′≥ns′\>jn^\{\\prime\}\_\{t\}\\geq n^\{\\prime\}\_\{s\}\>j\.

The comparator therefore holds every target output that the first decoder holds onXX\. Whenever the first decoder holds its root row, the comparator has also emitted the first token drawn from that row\. For a root row retained by the shared prefill, both decoders hold it initially; the comparator drawsX1X\_\{1\}as the first step of its exhaustive traversal whenever the first decoder does\.

Sincent\+1≤nt\+1′n\_\{t\+1\}\\leq n^\{\\prime\}\_\{t\+1\}, the first decoder’s round ends within the comparator’s emitted stream\. The comparator can therefore replay its stopping decisions using tokens and target outputs already held, together with the same independent randomness\. Its own traversal uses exact target conditionals, so the token factorization above preserves the target law\. ∎

#### Off\-path target outputs\.

Now allow the first decoder to read outputs at scored prefixes offXX, and allow a pass to score prefixes that branch before the committed context ends\. For such a prefix, the attention mask gives its new tokens their positions in that prefix and hides later cached positions\. The returned rows remain exact because cached keys and values depend only on earlier tokens\.

Keep the notationut=Xnt\+1:nt′u\_\{t\}=X\_\{n\_\{t\}\+1:n^\{\\prime\}\_\{t\}\}\. In roundt\+1t\+1, the comparator additionally scorescX1:ntvcX\_\{1:n\_\{t\}\}vfor everyv∈Qt\+1v\\in Q\_\{t\+1\}that is neither a prefix nor an extension ofutu\_\{t\}\. Each suchvvis nonempty and lies outside the image ofw↦ut​ww\\mapsto u\_\{t\}w\. Thus the total still does not exceed\|Qt\+1\|\|Q\_\{t\+1\}\|\. These extra prefixes leaveXX, whereas the comparator’s next context lies onXXand outsideQt\+1′Q^\{\\prime\}\_\{t\+1\}, so it remains uncovered\.

An off\-path prefixcX1:ntvcX\_\{1:n\_\{t\}\}vcannot havevvas a prefix ofutu\_\{t\}, since that would place it onXX\. Ifvvextendsutu\_\{t\}, the comparator scores it throughQt\+1′Q^\{\\prime\}\_\{t\+1\}; otherwise it is one of the additional prefixes\. The comparator therefore holds every off\-path target output available to the first decoder as well\.

If the first decoder also scores prefixes that branch before its committed context ends, the comparator scores each of them too\. They leaveXX, so the query count, zero coverage and availability of target outputs follow in the same way\. The remainder of the coupling is unchanged\.

## Appendix BStopping Policies and Finite Examples

### B\.1Stopping placement and row cost

Consider an IID uniform binary target on

T∖\{ϵ\}=\{\\displaystyle T\\setminus\\\{\\epsilon\\\}=\\\{a,a​a,a​a​a,a​a​b,\\displaystyle a,aa,aaa,aab,\(16\)ab,b,ba\},\\displaystyle ab,b,ba\\\},withQ=TQ=Tand eight scored rows including the root\. A capkkstops at the first tree exit or afterkkaccepted tokens and emits the next unaccepted draw\. Sampled\-leaf stopping emits the current draw when it labels a leaf child, otherwise following matching children until exit\. The decision uses only the current draw and trie topology\.

Every rule scores the same eight rows before stopping\. Exhaustive traversal has coverage00and expected acceptance22\. Cap 1 has coverage3/43/4, expected acceptance11and shortfall11, sampled\-leaf stopping has coverage3/43/4, expected acceptance5/45/4and shortfall3/43/4, and cap 2 has coverage1/41/4, expected acceptance7/47/4and shortfall1/41/4\.

Cap 1 and sampled\-leaf stopping have equal coverage but defer different path lengths\. For cap 1, covered continuation addresses area​aaa,a​babandb​aba, each with probability1/41/4\. The first leaves two accepted tokens unused and the others one\. Sampled\-leaf stopping instead ends ata​a​a,a​a​b,a​b,b​aaaa,aab,ab,bawith probabilities1/8,1/8,1/4,1/41/8,1/8,1/4,1/4, each leaving one token unused\.

#### Scored prefixes beyond the trie\.

Let the candidate trie beT=\{ϵ,a\}T=\\\{\\epsilon,a\\\}and the scored set beQ=\{ϵ,a,b\}Q=\\\{\\epsilon,a,b\\\}under the same target\. Exhaustive traversal has expected acceptance1/21/2, coverage1/21/2and zero shortfall relative to its own trie\. Scoring only the two rows inTTgives zero coverage\. Makingbba candidate raises expected acceptance to11, and the added token is accepted exactly in the covered rounds\.

Table 7:Target checkpoints and their paired EAGLE\-3 and standalone drafters\. Checkpoint names link to the evaluated Hugging Face revisions\.Table 8:Mean tokens emitted in each round, counting accepted draft tokens and the continuation, by task\.

## Appendix CMeasurement Protocol

#### Checkpoints and prompts\.

Table[7](https://arxiv.org/html/2610.29210#A2.T7)lists the targets and drafters\. Apart from the pipelined runs of Appendix[D\.14](https://arxiv.org/html/2610.29210#A4.SS14), all runs use the EAGLE\-3 reference implementation with PyTorch 2\.7\.1\([Paszke et al\., 2019](https://arxiv.org/html/2610.29210#bib.bib37)\)and bfloat16 weights\. These runs use eager attention, except that the standalone drafters use scaled dot\-product attention as described in Appendix[D\.13](https://arxiv.org/html/2610.29210#A4.SS13)\. Each non\-pipelined run uses one NVIDIA A100 80GB, H100 NVL or RTX A6000 GPU, except that the Qwen2\.5\-32B runs of root and root\-draw selection in Tables[3](https://arxiv.org/html/2610.29210#S5.T3)and[14](https://arxiv.org/html/2610.29210#A4.T14), and their second runs at temperature11, use two RTX A6000 GPUs\.

Prompts use each target’s chat template with one user turn and no added system message, so the Qwen2\.5 and Llama\-3\.1 templates insert their default system text, and the Qwen3 templates use their non\-thinking mode\.

Spec\-Bench supplies the first turns of8080MT\-bench conversations\([Zheng et al\., 2023](https://arxiv.org/html/2610.29210#bib.bib36)\), ten from each of its eight categories, and8080prompts each for translation\([Bojar et al\., 2014](https://arxiv.org/html/2610.29210#bib.bib40)\), summarization\([Hermann et al\., 2015](https://arxiv.org/html/2610.29210#bib.bib31);[Nallapati et al\., 2016](https://arxiv.org/html/2610.29210#bib.bib41)\), question answering\([Kwiatkowski et al\., 2019](https://arxiv.org/html/2610.29210#bib.bib42)\), mathematical reasoning\([Cobbe et al\., 2021](https://arxiv.org/html/2610.29210#bib.bib44)\)and retrieval\-augmented generation\([Kwiatkowski et al\., 2019](https://arxiv.org/html/2610.29210#bib.bib42);[Karpukhin et al\., 2020](https://arxiv.org/html/2610.29210#bib.bib43)\)\. HumanEval supplies164164function signatures with docstrings, each used verbatim as the user turn\.

Generation stops after the round in which an end\-of\-sequence token appears or the output reaches the default limit, so an output holds at most264264tokens\. The twelve default\-length runs contain365,922365\{,\}922verification rounds\. The runs of Appendix[D\.5](https://arxiv.org/html/2610.29210#A4.SS5)use a limit of1,0241\{,\}024new tokens, so their outputs hold at most1,0321\{,\}032tokens\.

#### Draft pool and tree\.

The drafter first proposes its ten most probable tokens and expands all of them\. At each of seven further steps, it adds the ten most probable children of every expanded node to the pool, and after each of the first six it expands the ten new candidates with the highest cumulative draft log probability\. The pool thus holds1010candidates at depth one and100100at each depth from two to eight,710710in total, and7070of them are expanded\.

Candidates are ranked by cumulative draft log probability, then by depth and then by pool index\. The verified tree holds the5959highest\-ranked candidates, and the parent of every candidate is the root or an expanded node\. Selection rules that keep draft factors take them from the same cumulative scores\. Every rule’s score is non\-increasing along a path, so with ties broken by depth itsBBhighest\-ranked candidates form an ancestor\-closed set of exactlyBBnodes, and the oracle’s set maximizesVVamong all ancestor\-closed sets ofBBcandidates\.

#### Verification\.

Under greedy decoding, traversal accepts the child labeled by the target argmax until the argmax labels no child, and that token becomes the continuation\. Under sampling, each reached node draws a token from its full target row, without top\-k or top\-p truncation, with a generator seeded for each prompt and run, and proceeds in the same way\.

EAGLE\-3’s sampling verifier visits the distinct children of a node in order and accepts each with its probability under the current row, otherwise removing it from the row and renormalizing\. The probability of accepting thekk\-th visited child is thus the product of the earlier rejection probabilities and its renormalized probability, and this product telescopes to its target probability\. If every child is rejected, the continuation is drawn from the row restricted to the remaining tokens\. This is the law of exhaustive target sampling\.

The acceptance test uses no draft probability, so this law holds given the realized tree\. The accepted tokens also form a root path of that tree, and exactly one continuation is emitted, so the verifier has the three properties that Theorem[1](https://arxiv.org/html/2610.29210#Thmtheorem1)requires of it\. The key\-value cache then keeps the root and the accepted path, and the continuation becomes the next root\.

#### Scored rows\.

The verification pass scores the root, the verified tree, all expanded nodes and the160160highest\-ranked candidates under a tree attention mask, about170170prefixes in each round\. Each position attends only to its own prefix, so the additional prefixes leave the rows of the verified tree unchanged up to rounding\.

EAGLE\-3’s decoder in Appendix[D\.10](https://arxiv.org/html/2610.29210#A4.SS10)runs online at its own contexts with a pass that scores only its tree\. AtB=59B=59, the expected acceptance of EAGLE\-3’s tree on the recorded rounds lies inside the95%95\\%prompt\-bootstrap interval of this decoder’s realized acceptance in every run\. The relative difference is at most1\.1%1\.1\\%\.

Every candidate’s parent is scored, so the pass gives the target probability of all710710candidate edges\. Scored sets of other sizes are defined by address membership, since the row at a covered address is the next round’s root row\.

#### Retained rows\.

After each round, we retain the row at the emitted address when the verification pass scored it, together with up to2424rows, taken in pool order, at other scored addresses whose final token equals the continuation\. When the root token equals the continuation, the root row joins the second group without counting toward the2424\. Each retained row is compared with the next round’s root row, computed at the complete emitted context\. Total variation and argmax agreement use full\-vocabulary softmax probabilities in float32\.

Figure 5:Share of the oracle advantage over draft selection recovered by exact rows at increasing depths, for each target and temperature\. The dotted line marks EAGLE\-3’s5959\-node tree\.
#### Rows at other addresses\.

A second pass replays each recorded generation through its target\. Each round scores the recorded prefixes under the same tree mask and then commits the recorded accepted path and continuation, so contexts, pools and outputs are those of the recorded run\. The replayed root rows equal the recorded ones in the nine runs replayed on the capture’s GPU type\. In the other three, which ran on another GPU type, the probability of a first\-level candidate differs from the recorded one by at most0\.0070\.007on average in each run\.

Before the first round, the token\-keyed cache holds, for every prompt token, the row computed at its latest occurrence\. After each round, it stores the row of every verified prefix under the prefix’s final token, keeping the highest\-ranked prefix when several end in the same token, and then stores the root’s row under the root token\.

The same\-token row is the previous root’s row when the previous root ends in the continuation token, and otherwise the row of the highest\-ranked prefix of the previous verified tree that ends in it\. The preceding row is the row at the last accepted node, or at the root when no draft token is accepted\. Its probability of the continuation is set to zero before its first\-level candidates are ranked and before it is compared with the next root row\. At every budget, includingB=8B=8, the three rows come from this replay, whose verified trees are EAGLE\-3’s5959\-node trees\.

#### Compute\.

The twelve runs take about99GPU hours, including model loading, and the replay takes about88more\. Online decoding in Appendix[D\.3](https://arxiv.org/html/2610.29210#A4.SS3)and the covering comparison in Appendix[D\.10](https://arxiv.org/html/2610.29210#A4.SS10)together take about121121GPU hours, including the PEARL\-style and covering decoders, the second runs at temperature11and the further runs of root\-draw selection\. The1,0241\{,\}024\-token runs take about44more GPU hours\. The standalone\-drafter runs in Appendix[D\.13](https://arxiv.org/html/2610.29210#A4.SS13)take about77GPU hours, and their pipelined decoding in Appendix[D\.14](https://arxiv.org/html/2610.29210#A4.SS14)takes about5050, counting both GPUs\.

Table 9:Selection gains in expected accepted draft tokens and recovered shares of the oracle advantage\. Depth 2 under Expected gives acceptance with two exact factors\. Brackets give95%95\\%prompt\-bootstrap intervals\.Table 10:Gains over draft selection atB=59B=59under round weighting and equal task weighting\. Brackets give95%95\\%intervals from1,0001\{,\}000resamples of prompts within tasks\.

## Appendix DAdditional Results

### D\.1Acceptance length

Table[8](https://arxiv.org/html/2610.29210#A2.T8)gives the mean number of tokens each round emits, its accepted draft tokens and continuation\. Rounds emit3\.223\.22to5\.515\.51tokens under greedy decoding and3\.083\.08to4\.534\.53under sampling\. Counted in emitted tokens, root selection’s gain over draft selection is2\.3%2\.3\\%to5\.5%5\.5\\%atB=59B=59, with a median of3\.5%3\.5\\%, and6\.1%6\.1\\%to14\.2%14\.2\\%atB=8B=8\. Root\-draw selection’s gain over the PEARL\-style decoder is2\.9%2\.9\\%to4\.9%4\.9\\%atB=59B=59and7\.7%7\.7\\%to9\.0%9\.0\\%atB=8B=8\. All four gains divide the difference in expected accepted draft tokens by the baseline’s expected emitted tokens, continuation included, so each interval also lies above zero\.

M=100M=100M=200M=200M=400M=400M=710M=710TargetAdvantageRoot, %AdvantageRoot, %AdvantageRoot, %AdvantageRoot, %Greedy decodingLlama\-3\.1\-8B0\.186580\.321480\.380430\.39043Qwen3\-4B0\.135630\.279520\.359470\.38046Qwen3\-8B0\.125670\.247580\.319530\.34352Qwen2\.5\-7B0\.147590\.267510\.327470\.34346Qwen2\.5\-14B0\.164580\.293490\.348450\.36044Qwen2\.5\-32B0\.159580\.284500\.341450\.35444Sampling at temperature11Llama\-3\.1\-8B0\.163440\.289340\.343310\.35530Qwen3\-4B0\.134590\.275480\.354430\.37442Qwen3\-8B0\.122640\.241540\.311490\.33548Qwen2\.5\-7B0\.139490\.257410\.315370\.33336Qwen2\.5\-14B0\.149480\.272380\.329340\.34334Qwen2\.5\-32B0\.147460\.270370\.328330\.34233Table 11:Oracle advantage over draft selection atB=59B=59when the pool holds theMMhighest\-ranked candidates, and the share that root selection recovers\.
### D\.2Selection values with intervals

Table[9](https://arxiv.org/html/2610.29210#A3.T9)gives the gains of root, root\-draw and oracle selection over draft selection and the share of the oracle advantage that root and depth\-two selection recover\. The root\-draw value of a round is one for the drawn token plus the path mass, summed over first\-level candidates, of the candidates that each keeps beneath it\. When the pool’s first level lacks the drawn token, the tree holds that token alone\. This happens in9%9\\%to16%16\\%of greedy rounds and with probability11%11\\%to19%19\\%at temperature11, and exhaustive traversal of draft selection’s tree, which lies in the same pool, then accepts no draft token, while root\-draw selection accepts the drawn one\.

Each bootstrap draw resamples prompts with all their rounds and keeps every rule and budget together\. Every interval lies above zero at both budgets, as does every interval of a greedy root row entered with its softmax probabilities\.

Table[13](https://arxiv.org/html/2610.29210#A4.T13)expresses the root gain in nodes\. For each run, it gives the smallest budget at which draft selection reaches root selection’s value at a reference budget and the smallest at which root selection reaches draft selection’s, searching every budget up to710710within the same pool of depths one to eight and repeating the search in every resample\. AtB=8B=8, draft selection needs1111to2222nodes to reach root selection’s value, and root selection reaches draft selection’s with44to77\.

A PEARL\-style decoder\([Liu et al\., 2025a](https://arxiv.org/html/2610.29210#bib.bib7)\)uses the root row only for the first draw\. It takes the same draw as root\-draw selection and counts it as one accepted token, then keeps the part of draft selection’s tree beneath the drawn token, or the drawn token alone when that tree lacks it\. Its value exceeds that of draft selection by the probability that the drawn token is missing from the tree’s first level\. Draft selection keeps at mostB−1B\-1descendants of a kept first\-level token, and these are its highest\-ranked descendants\. Root\-draw selection keeps theB−1B\-1highest\-ranked descendants of the drawn token, or all of them when there are fewer, so its value is at least the PEARL\-style value in every round\. Under the same cap, both decoders keep at mostBBnodes\.

Table[12](https://arxiv.org/html/2610.29210#A4.T12)shows that root\-draw selection improves expected acceptance over the PEARL\-style decoder by up to14\.2%14\.2\\%atB=8B=8and7\.0%7\.0\\%atB=59B=59under the same node cap\. All confidence intervals lie above zero\. We also find that root\-draw selection retains larger trees on average than the PEARL\-style decoder:4242to5252versus2626to3838nodes atB=59B=59, and6\.66\.6to7\.47\.4versus4\.14\.1to5\.85\.8atB=8B=8\.

A variant closer to PEARL, whose drafter commits to one first token, keeps the drafter’s most probable first token with itsB−1B\-1highest\-ranked descendants and verifies the drawn token alone when the draw differs\. Its value lies below the PEARL\-style value in every run, by0\.5%0\.5\\%to1\.5%1\.5\\%atB=8B=8and6\.7%6\.7\\%to9\.1%9\.1\\%atB=59B=59with every interval excluding zero, so Table[12](https://arxiv.org/html/2610.29210#A4.T12)compares root\-draw selection with the stronger of the two\.

Table[12](https://arxiv.org/html/2610.29210#A4.T12)also compares root\-draw selection with a PEARL\-style decoder whose pre\-draw budget is raised, separately in each run, until its pruned trees hold as many nodes on average as root\-draw selection’s\. The raised budget mixes the two nearest whole budgets and comes to8181to9696candidates atB=59B=59and10\.410\.4to14\.014\.0atB=8B=8\. The pruned trees then hold more thanBBnodes in an expected38%38\\%to53%53\\%of rounds atB=59B=59and40%40\\%to56%56\\%atB=8B=8, while root\-draw selection’s trees never do\.

After mean tree sizes are matched, root\-draw selection still improves expected acceptance by up to4\.8%4\.8\\%atB=8B=8and2\.4%2\.4\\%atB=59B=59\. Every confidence interval remains above zero when matching is repeated in each resample\. The draw is the drafter’s most probable first token in an expected52%52\\%to76%76\\%of rounds\. AtB=59B=59the pruned trees hold6666to7474nodes on average after such a draw, more than root\-draw selection’s5959, and only1212to1515after a draw of another first\-level token, where root\-draw selection keeps3939to4949\. AtB=8B=8the pruned trees hold9\.29\.2to11\.211\.2and2\.12\.1to2\.42\.4nodes after the two kinds of draw, while root\-draw selection’s trees hold88after either\.

B=8B=8B=59B=59TargetGain, %Equal size, %Share, %Gain, %Equal size, %Share, %Greedy decodingLlama\-3\.1\-8B10\.0 \[9\.4, 10\.5\]2\.9 \[2\.6, 3\.3\]713\.5 \[3\.3, 3\.7\]0\.9 \[0\.8, 1\.0\]64Qwen3\-4B12\.8 \[12\.4, 13\.3\]4\.5 \[4\.2, 4\.8\]556\.7 \[6\.5, 6\.9\]2\.4 \[2\.2, 2\.5\]60Qwen3\-8B13\.8 \[13\.4, 14\.2\]4\.4 \[4\.0, 4\.7\]486\.7 \[6\.5, 6\.9\]2\.2 \[2\.1, 2\.4\]47Qwen2\.5\-7B10\.7 \[10\.2, 11\.2\]4\.1 \[3\.8, 4\.4\]554\.2 \[4\.0, 4\.4\]1\.3 \[1\.2, 1\.4\]46Qwen2\.5\-14B11\.1 \[10\.6, 11\.6\]4\.0 \[3\.7, 4\.4\]594\.2 \[4\.0, 4\.4\]1\.2 \[1\.1, 1\.3\]51Qwen2\.5\-32B10\.6 \[10\.1, 11\.1\]3\.9 \[3\.6, 4\.2\]584\.1 \[3\.9, 4\.3\]1\.2 \[1\.1, 1\.4\]51Sampling at temperature11Llama\-3\.1\-8B12\.2 \[11\.7, 12\.7\]3\.1 \[2\.8, 3\.4\]574\.7 \[4\.5, 5\.0\]1\.4 \[1\.3, 1\.5\]51Qwen3\-4B13\.1 \[12\.6, 13\.5\]4\.7 \[4\.4, 4\.9\]536\.8 \[6\.6, 7\.0\]2\.4 \[2\.2, 2\.5\]57Qwen3\-8B14\.2 \[13\.8, 14\.6\]4\.8 \[4\.5, 5\.0\]457\.0 \[6\.8, 7\.2\]2\.3 \[2\.2, 2\.5\]44Qwen2\.5\-7B12\.6 \[12\.1, 13\.0\]4\.8 \[4\.6, 5\.1\]485\.2 \[5\.1, 5\.4\]1\.8 \[1\.7, 1\.9\]42Qwen2\.5\-14B13\.0 \[12\.5, 13\.4\]4\.7 \[4\.4, 4\.9\]535\.3 \[5\.2, 5\.5\]1\.8 \[1\.7, 1\.9\]48Qwen2\.5\-32B12\.2 \[11\.7, 12\.6\]4\.5 \[4\.2, 4\.8\]525\.2 \[5\.0, 5\.3\]1\.8 \[1\.7, 1\.9\]48Table 12:Gain of root\-draw selection over the PEARL\-style decoder\. In Equal size, it preselects more candidates until mean tree sizes are equal\. Brackets give95%95\\%intervals\. Share is the part of root\-draw selection’s gain over draft selection that Gain makes up\.Table 13:Smallest budget at which draft selection reaches root selection’s expected accepted draft tokens at the reference budget, and at which root selection reaches draft selection’s\. Brackets give95%95\\%prompt\-bootstrap intervals\.Table 14:Online decoding atB=59B=59\. Accepted is the realized mean acceptance, and EAGLE\-3 that of the recorded runs\. Gain over EAGLE\-3 has95%95\\%prompt\-bootstrap intervals in brackets, and Offline is the gain over draft selection at the recorded contexts\.First tokens differ, %Oracle advantageRoot recovery, %Depth\-two recovery, %TargetRoundsRoot gainAgreeDifferAgreeDifferAgreeDifferGreedy decodingLlama\-3\.1\-8B24600\.3710\.44824926196Qwen3\-4B36750\.3710\.39519936297Qwen3\-8B45780\.3520\.33320946197Qwen2\.5\-7B34690\.3400\.34922926296Qwen2\.5\-14B31670\.3580\.36422936297Qwen2\.5\-32B31680\.3450\.37521936197Sampling at temperature11Llama\-3\.1\-8B31570\.3710\.32018624577Qwen3\-4B37750\.3750\.37417866094Qwen3\-8B46780\.3510\.31619855994Qwen2\.5\-7B37680\.3480\.30617725185Qwen2\.5\-14B34640\.3540\.32118684882Qwen2\.5\-32B33630\.3540\.31817674982Table 15:Selection atB=59B=59split by whether the drafter’s most probable first token is the target’s most probable token\. Oracle advantage is in expected accepted draft tokens, and recovery is the share of it that each rule attains\.
### D\.3Online decoding

The online decoders of Section[5\.4](https://arxiv.org/html/2610.29210#S5.SS4)that take the root row from a separate target pass run on all six targets, with the checkpoints, prompts, pools, output limit and seeding of Appendix[C](https://arxiv.org/html/2610.29210#A3)\. In each round, the drafter grows its pool from the committed context, and a first target pass scores the root alone to give the root row\. The pool does not depend on this row, so running the first pass while the drafter works would leave every acceptance value unchanged, as the pipelined runs of Appendix[D\.14](https://arxiv.org/html/2610.29210#A4.SS14)confirm for the standalone drafters\.

The rule selects the tree with that row, and a second pass scores the root and the tree under the tree mask\. Exhaustive target sampling then traverses the tree, taking its first draw from the root row\. Root\-draw selection makes this draw before it selects the tree, and the PEARL\-style decoder makes it before it cuts draft selection’s5959\-node tree to the part beneath the drawn token, or to the drawn token alone when that tree lacks it\. The key\-value cache keeps the root and the accepted path, and the continuation becomes the next root\.

The two passes compute the root row in batches of different sizes, and the two computations share their argmax in98\.6%98\.6\\%to99\.6%99\.6\\%of rounds and, in the PEARL\-style runs, differ by a mean total variation of0\.00410\.0041to0\.01290\.0129\. For Qwen3\-4B and Qwen2\.5\-7B, whose recorded runs and the runs above all used RTX A6000 GPUs, the matched pairs of Section[5\.3](https://arxiv.org/html/2610.29210#S5.SS3)share their argmax more often than these two computations in every decoder at both temperatures, and they differ by a smaller mean total variation than the two computations in the PEARL\-style runs\.

The two rows of a matched pair have the same token, position and key\-value cache, and the replay of Appendix[C](https://arxiv.org/html/2610.29210#A3), run with the recorded batches on the capture’s GPU type, reproduces the recorded root rows exactly, so the two rows differ only because different batches compute them\.

Table[14](https://arxiv.org/html/2610.29210#A4.T14)gives the realized acceptance of root and root\-draw selection, counting the drawn token under root\-draw selection, and Table[16](https://arxiv.org/html/2610.29210#A4.T16)that of root\-draw selection and the PEARL\-style decoder behind Table[3](https://arxiv.org/html/2610.29210#S5.T3)\. Realized acceptance divides the accepted tokens of all rounds by the number of rounds\. Each decoder processes one prompt at a time, with a generator seeded for each prompt, run and decoder\. An online run and the recorded run follow different outputs after their first differing token, so each resample draws prompts jointly for both\.

Under greedy decoding, the two outputs agree up to a first token where bfloat16 rounding changes an argmax, and38%38\\%to59%59\\%of the emitted tokens precede it\. On this shared text, a round of either decoder that starts where a recorded round starts has the recorded context, and its acceptance equals its rule’s value on the recorded rows in97\.2%97\.2\\%to98\.5%98\.5\\%of such rounds\. On the same text, root\-draw decoders start6\.8%6\.8\\%to12\.9%12\.9\\%of their rounds whereEAGLE\-3accepted no draft token, against9\.9%9\.9\\%to17\.8%17\.8\\%ofEAGLE\-3’s rounds, a relative reduction of25%25\\%to31%31\\%\.

At temperature11the outputs diverge within the first rounds, and the sampling noise of two separate outputs enters the comparison\. A second run of each root decoder at temperature11therefore scores the union of its tree and EAGLE\-3’s draft\-ranked tree, and the union gives the expected acceptance of both trees at the same contexts\. At these contexts, the root gain over draft ranking is3\.0%3\.0\\%for Llama\-3\.1\-8B,7\.1%7\.1\\%for Qwen3\-4B,7\.5%7\.5\\%for Qwen3\-8B,4\.2%4\.2\\%for Qwen2\.5\-7B,4\.1%4\.1\\%for Qwen2\.5\-14B and3\.9%3\.9\\%for Qwen2\.5\-32B, with every interval above2\.8%2\.8\\%\. Each such run’s realized mean acceptance is within0\.010\.01tokens of its trees’ expected acceptance\.

A further run of root\-draw selection on every target at both temperatures also scores, in each tree pass, the drawn token’s160160highest\-ranked pool descendants\. The verifier still sees only root\-draw selection’s tree, and the extra prefixes leave its rows unchanged up to rounding\. The run uses the seeds of the first root\-draw run, but that rounding can make the two outputs differ\. The target probabilities of these descendants give, at this run’s contexts, the expected acceptance of the PEARL\-style tree for every pre\-draw budget up to161161candidates\.

As in Table[12](https://arxiv.org/html/2610.29210#A4.T12), the budget is raised until the mean tree sizes are equal, which takes8181to9595candidates, and the enlarged trees then hold more than5959nodes in39%39\\%to54%54\\%of rounds\. Table[18](https://arxiv.org/html/2610.29210#A4.T18)gives root\-draw selection’s gain over these trees,1\.0%1\.0\\%to2\.4%2\.4\\%against0\.9%0\.9\\%to2\.4%2\.4\\%at the recorded contexts, and with the budget matched again in every resample, each interval lies above zero\. Each run’s realized mean acceptance is within0\.020\.02tokens of its trees’ expected acceptance\.

Root\-draw selection and the PEARL\-style decoder also run online with the same prompts and seeding under a cap of88nodes, where the PEARL\-style decoder cuts draft selection’s88\-node tree\. In Table[17](https://arxiv.org/html/2610.29210#A4.T17), root\-draw selection’s realized gain is10\.7%10\.7\\%to13\.7%13\.7\\%, with every interval above9\.8%9\.8\\%, and each run’s gain is within0\.80\.8percentage points of its offline gain\.

Table 16:Online decoding atB=59B=59\. Accepted is realized mean acceptance\. Gain compares root\-draw selection with the PEARL\-style decoder; brackets give95%95\\%intervals\. Offline evaluates shared recorded contexts\. Root TV compares the two passes’ root rows by mean total variation\.Table 17:Online decoding atB=8B=8\. Gain compares root\-draw selection with the PEARL\-style decoder in realized acceptance; brackets give95%95\\%intervals\. Offline evaluates shared recorded contexts\. Root TV compares the two passes’ root rows by mean total variation\.Table 18:Gain in expected acceptance of root\-draw selection over the PEARL\-style tree enlarged to equal mean size atB=59B=59, with95%95\\%intervals\. Online evaluates both trees at online contexts; Offline uses the shared recorded contexts\.Table 19:Expected accepted draft tokens after cross\-fitting calibrated or reranked draft factors on complementary prompt halves\. Base uses draft and contextual features; Rows adds retained rows from other addresses\. Blue cells mark selection with the exact root row\.Table 20:Root selection’s gains over draft ranking under a1,0241\{,\}024\-token limit, in percent, with95%95\\%prompt\-bootstrap intervals in brackets\. Late is the share of outputs with rounds starting once the default output limit is reached; Rounds counts these rounds\.
### D\.4Tasks, weighting and targets

Round averages weight long generations more heavily\. Averaging the rounds of each prompt first, so that every prompt weighs the same, gives relative root gains of2\.9%2\.9\\%to7\.9%7\.9\\%atB=59B=59and8\.6%8\.6\\%to23\.4%23\.4\\%atB=8B=8, with every interval above zero\. Table[10](https://arxiv.org/html/2610.29210#A3.T10)instead averages the seven tasks equally, with intervals from1,0001\{,\}000resamples of prompts within each task\. With prompts resampled within each task, every task interval of the root and root\-draw gains lies above zero at both budgets in every run\.

Figure[5](https://arxiv.org/html/2610.29210#A3.F5)shows the recovered share of the oracle advantage for each target at the budgets evaluated in Figure[2](https://arxiv.org/html/2610.29210#S5.F2),B=2B=2,44,88,1212,1616,2424,3232,4848,5959,8080,100100,150150and200200\.

### D\.5Longer outputs

Llama\-3\.1\-8B and Qwen3\-8B also decode all644644prompts under a1,0241\{,\}024\-token limit at both temperatures, on RTX A6000 GPUs and with the code and settings of Appendix[C](https://arxiv.org/html/2610.29210#A3)otherwise unchanged\. A late round starts with an output length at or above the default limit, so the default\-length runs contain none\. Late rounds occur in41%41\\%to48%48\\%of the outputs, and HumanEval prompts account for about half of these outputs\. At most2\.5%2\.5\\%of all outputs reach the limit\.

Table[20](https://arxiv.org/html/2610.29210#A4.T20)gives the late\-round and whole\-output gains of root selection over draft ranking\. Whole\-output gains remain within0\.30\.3percentage points of the default\-length runs atB=59B=59and within1\.11\.1atB=8B=8\. Root\-draw selection also improves late\-round expected acceptance over draft ranking at both budgets, with no interval below3\.4%3\.4\\%\.

### D\.6Pool size

Table[11](https://arxiv.org/html/2610.29210#A4.T11)restricts every rule to theMMhighest\-ranked candidates of the pool and reports the oracle advantage atB=59B=59with the share that root selection recovers\. The advantage grows with the pool, reaching0\.120\.12to0\.190\.19tokens atM=100M=100and0\.330\.33to0\.390\.39atM=710M=710, and root selection recovers44%44\\%to67%67\\%of it atM=100M=100\.

Table 21:Coverage of exhaustive traversal when the pass also scores the firstEEfrontier candidates in draft order\. Frontier size is the mean number of pool candidates outside the tree whose parents lie in it\.
### D\.7First\-token agreement

Table[15](https://arxiv.org/html/2610.29210#A4.T15)gives the first\-token split of Section[5\.4](https://arxiv.org/html/2610.29210#S5.SS4.SSS0.Px1)for each run\. The first tokens agree when the drafter’s most probable first token is the target’s most probable token at the current root\. In those rounds, rows through depth two recover45%45\\%to62%62\\%of the oracle advantage atB=59B=59\. In every run, the rounds in which the first tokens differ carry a larger share of the root gain than of the rounds\.

The root gain is also larger where the target is uncertain\. AtB=59B=59, rounds in the upper two quintiles of the root row’s softmax entropy carry1\.61\.6to12\.812\.8times the gain of the lower two\. These quintiles are taken over the rounds of each run, and under greedy decoding the mean root gain rises from each quintile to the next in every run\.

Table 22:Successor rounds whose root’s final token or root address appears among the prefixes scored in the previous round\. Tree rows are EAGLE\-3’s6060verified prefixes, and all scored rows are the roughly170170prefixes of our verification pass\.Table 23:Agreement between retained rows and the next round’s root row\. Matched rows share the complete emitted prefix, while token\-only rows share only its final token\. Total variation uses softmax probabilities\.Table 24:Standalone drafters in pools of EAGLE\-3’s shape\. Draft is draft selection’s expected acceptance\. Root is root selection’s gain over draft selection, and Root draw root\-draw selection’s gain over the PEARL\-style decoder, with95%95\\%prompt\-bootstrap intervals in brackets\.Table 25:Gains over draft selection in expected accepted draft tokens, with95%95\\%intervals, when the softmax probabilities of the exact root row or of a row at another address replace the first draft factor\. Rounds without one keep the draft ranking\.Available, %Mean TVArgmax agree, %TargetCacheSame tokenCacheSame tokenPrecedingCacheSame tokenPrecedingGreedy decodingLlama\-3\.1\-8B76400\.460\.360\.9454646Qwen3\-4B71270\.510\.360\.9549645Qwen3\-8B71290\.520\.350\.9449656Qwen2\.5\-7B74310\.500\.430\.9451575Qwen2\.5\-14B72300\.490\.390\.9451605Qwen2\.5\-32B72300\.490\.400\.9451595Sampling at temperature11Llama\-3\.1\-8B71350\.470\.360\.9253647Qwen3\-4B70260\.520\.360\.9549645Qwen3\-8B69280\.520\.360\.9448646Qwen2\.5\-7B69280\.520\.430\.9348576Qwen2\.5\-14B68280\.500\.400\.9449596Qwen2\.5\-32B68270\.500\.390\.9449606Table 26:Agreement of rows at other addresses with the next round’s root row\. Availability is the share of roots with such a row, and agreement is computed over those roots\. The preceding row is always available\. Total variation uses softmax probabilities\.Figure 6:Gain over draft selection, averaged over six targets, when the softmax probabilities of the exact root row or of a row at another address replace the first draft factor\. The dotted line marks EAGLE\-3’s tree\.
### D\.8Calibrated and reranked draft scores

Table[19](https://arxiv.org/html/2610.29210#A4.T19)compares two calibrated controls and two rerankers with draft and root selection\. The first control replacesq1q\_\{1\}with an isotonic estimate of the target probability; the second calibrates every draft factor with a separate map for each depth\. Each map is fitted over200200equal\-count bins of draft probability on the even\-numbered prompts and applied to the odd\-numbered ones, and vice versa\.

The reranker uses gradient\-boosted trees fitted on one half of the prompts and applied to the other to predict the target probability of each first\-level candidate, and under greedy decoding whether it is the target’s token\. Its inputs, all known before verification, are the draft probabilities of the candidate and its siblings, the draft distribution and pool subtree below the candidate, a token prior estimated on the training half, the task, the prompt length, the round’s position, the previous round’s acceptance and whether the candidate repeats the root’s token\.

The token prior averages the target over candidates with the same token and shrinks the average toward the overall mean with a pseudo\-count of2020\. Within the training half, the candidates are split in alternation, and each one’s prior comes from the other part, so no prior contains its own label\. The model is a histogram gradient\-boosted classifier under greedy decoding and a regressor under sampling, with400400iterations, learning rate0\.050\.05and at most6363leaves\. The prediction replaces the first draft factor as the root row does\.

Its top\-ranked first\-level candidate is the target’s most probable one in60\.1%60\.1\\%to82\.8%82\.8\\%of rounds, against58\.7%58\.7\\%to82\.7%82\.7\\%for the draft ranking, where greedy runs count only rounds whose first level contains the target’s token\.

The row features come from the token\-keyed cache, a same\-token row from the previous tree and the row that generated the continuation\. For each available row, the reranker reads the candidate’s log probability and its difference from the highest first\-level log probability\. It also reads the number of rounds since the cache row was stored and the number of previous\-tree prefixes that end in the root’s token\. Each of these inputs is missing in rounds without its row, and the preceding row’s entry for the continuation token is always missing\.

This reranker recovers at most29%29\\%of the root gain atB=8B=8and33%33\\%atB=59B=59, and in every run its held\-out squared error on the first\-level targets is below that of the draft probabilities and of the reranker without rows\. In every run at both budgets, the95%95\\%intervals of its gain over draft selection and of its gain over the reranker without rows lie above zero\. Their lower ends are at least0\.27%0\.27\\%and0\.30%0\.30\\%of draft selection’s value atB=59B=59, and0\.81%0\.81\\%and0\.88%0\.88\\%atB=8B=8\.

### D\.9Coverage from the scored frontier

Scoring theMMhighest\-ranked candidates includes candidates whose parents lie outside the tree\. Such a candidate cannot be the emitted address after exhaustive traversal\. Table[21](https://arxiv.org/html/2610.29210#A4.T21)instead scores only frontier candidates, which lie outside the tree but have their parents in it, in draft order\. Forty frontier candidates cover12%12\\%to17%17\\%of next roots, more than the4141candidates added by the top\-100100set, and the complete frontier,380380to418418candidates on average, covers28%28\\%to35%35\\%, the same share as the whole pool\.

### D\.10Selection with reused rows

Without a separate root pass, a decoder selects with the exact root row only after a covered round and keeps the draft ranking otherwise, including in the first round of each generation\. Each round verifies the tree of the rule it uses and scores theMMhighest\-ranked candidates in that rule’s order, so its coverage is that of its own tree\. Under greedy decoding a round is covered when the greedy token after its accepted path labels a scored candidate\. At temperature11a round holds the row with the probability that the previous round was covered, and that round’s coverage is the scored frontier mass of Equation[5](https://arxiv.org/html/2610.29210#S4.E5)\. For EAGLE\-3’s tree this computation gives the recorded coverage in every greedy round\.

At the recorded contexts, root selection in rounds that hold the row raises expected acceptance over draft ranking atB=59B=59by0\.5%0\.5\\%to0\.9%0\.9\\%when verification scores the100100highest\-ranked candidates and by1\.1%1\.1\\%to2\.3%2\.3\\%when it scores the whole pool, and atB=8B=8by4\.9%4\.9\\%to10\.1%10\.1\\%and5\.2%5\.2\\%to11\.1%11\.1\\%, with every interval above zero\.

With the100100candidates,41%41\\%to56%56\\%of rounds hold the row atB=8B=8, against10%10\\%to15%15\\%atB=59B=59, since the smaller trees exit closer to the root and leave9292of the100100scored candidates outside the tree\. Scoring the whole pool,710710prefixes, gives the largest coverage that any scored subset of the pool reaches\. By Theorem[2](https://arxiv.org/html/2610.29210#Thmtheorem2), whatever the gain in rounds that hold the row, a decoder that is never covered and scores no more prefixes in each round has emitted at least as many tokens as such a decoder after every round\.

Tokens for each passTargetBBEAGLE\-3CoveringChange, %Covered, %GainGreedy decodingLlama\-3\.1\-8B84\.1713\.859−\-7\.5 \[−\-8\.2,−\-6\.8\]490\.62595\.5055\.059−\-8\.1 \[−\-8\.6,−\-7\.6\]520\.25Qwen3\-4B82\.6542\.476−\-6\.7 \[−\-7\.1,−\-6\.3\]310\.49593\.3233\.106−\-6\.5 \[−\-6\.9,−\-6\.2\]290\.28Qwen3\-8B82\.5692\.395−\-6\.8 \[−\-7\.2,−\-6\.4\]330\.57593\.2312\.998−\-7\.2 \[−\-7\.6,−\-6\.8\]310\.34Qwen2\.5\-7B83\.3683\.151−\-6\.4 \[−\-7\.1,−\-5\.8\]350\.61594\.3023\.987−\-7\.3 \[−\-8\.1,−\-6\.6\]360\.32Qwen2\.5\-14B83\.4463\.229−\-6\.3 \[−\-6\.8,−\-5\.8\]360\.61594\.4304\.138−\-6\.6 \[−\-7\.1,−\-6\.0\]380\.29Qwen2\.5\-32B83\.4703\.248−\-6\.4 \[−\-6\.8,−\-6\.0\]360\.59594\.4464\.152−\-6\.6 \[−\-7\.1,−\-6\.0\]370\.30Sampling at temperature11Llama\-3\.1\-8B83\.4763\.198−\-8\.0 \[−\-10\.2,−\-5\.8\]420\.67594\.4934\.124−\-8\.2 \[−\-11\.2,−\-5\.1\]450\.32Qwen3\-4B82\.5942\.419−\-6\.8 \[−\-7\.3,−\-6\.3\]310\.50593\.2413\.018−\-6\.9 \[−\-7\.5,−\-6\.3\]290\.30Qwen3\-8B82\.4772\.298−\-7\.2 \[−\-7\.9,−\-6\.5\]320\.59593\.1042\.869−\-7\.5 \[−\-8\.3,−\-6\.8\]300\.37Qwen2\.5\-7B82\.9622\.803−\-5\.4 \[−\-6\.6,−\-4\.0\]300\.64593\.7543\.551−\-5\.4 \[−\-6\.8,−\-3\.8\]320\.36Qwen2\.5\-14B83\.0382\.835−\-6\.7 \[−\-7\.7,−\-5\.6\]320\.65593\.8663\.630−\-6\.1 \[−\-7\.4,−\-4\.9\]340\.35Qwen2\.5\-32B83\.0312\.866−\-5\.5 \[−\-6\.6,−\-4\.3\]310\.64593\.8983\.651−\-6\.3 \[−\-7\.8,−\-4\.8\]340\.36Table 27:Covering decoder against EAGLE\-3’s decoder, in tokens emitted for each target pass\. Change is the relative change with95%95\\%intervals, Covered the share of covered rounds, and Gain root\-draw selection’s token gain over draft selection after covered rounds\.We also run a covering decoder online, at its own contexts\. It drafts EAGLE\-3’s pool and verifies its tree by exhaustive target sampling, except that a draw labeling a leaf of the verified tree, including a tree that holds only the drawn token, is emitted as the continuation instead of being accepted\. The leaf’s row from the same pass is then the next root row, from which the next round draws its first token before it selects its tree by root\-draw selection, and other rounds keep EAGLE\-3’s tree\. EAGLE\-3’s decoder runs on the same prompts with the same output limit\.

Both run one target pass in each round, so Table[27](https://arxiv.org/html/2610.29210#A4.T27)compares the tokens they emit for each target pass, counted up to the first end\-of\-sequence token\. In the rounds that hold a covered row, the pass also scores the trees that draft selection and root selection would keep, for measurement only\. The covering decoder emits fewer tokens for each target pass at both budgets, although selection improves in the rounds that hold the root row\.

### D\.11Token and address matches

Among the retained rows of Appendix[C](https://arxiv.org/html/2610.29210#A3), the row at the emitted address is a matched row and the others are token\-only rows\. Table[23](https://arxiv.org/html/2610.29210#A4.T23)gives the agreement of matched and token\-only rows with the next round’s root row in each run, and Table[22](https://arxiv.org/html/2610.29210#A4.T22)the token and address match rates behind Section[5\.3](https://arxiv.org/html/2610.29210#S5.SS3)\. Matched pairs have median total variation below0\.00040\.0004in every run\. Among token\-only rows, those on the accepted path have mean total variation of0\.720\.72to0\.790\.79from the next root row, above that of the other token\-only rows in every run, as they condition on an earlier instance of the continuation token\.

### D\.12Rows at other addresses

Table[25](https://arxiv.org/html/2610.29210#A4.T25)reports direct first\-factor substitution at both budgets, and Figure[6](https://arxiv.org/html/2610.29210#A4.F6)varies the budget\. All three substitutes lower acceptance relative to draft selection in these tests\. Under greedy decoding, this comparison uses softmax probabilities even for the Exact column, whereas root selection uses the point\-mass target law\. Table[26](https://arxiv.org/html/2610.29210#A4.T26)reports availability and distribution agreement\. The replay and row construction are described in Appendix[C](https://arxiv.org/html/2610.29210#A3)\.

The cache and preceding rows can also be mixed geometrically with the first draft factor, choosing the weight from0\.250\.25,0\.50\.5and0\.750\.75on the other half of the prompts\. The preceding\-row mixture still lowers acceptance, while the cache mixture yields no consistently positive gain\.

### D\.13Standalone drafters

Llama\-3\.1\-8B and Qwen3\-8B also decode all644644prompts at both temperatures with a standalone drafter that shares the target’s tokenizer, Llama\-3\.2\-1B\-Instruct\([Meta, 2024](https://arxiv.org/html/2610.29210#bib.bib45)\)for Llama\-3\.1\-8B and Qwen3\-0\.6B\([Yang et al\., 2025](https://arxiv.org/html/2610.29210#bib.bib28)\)for Qwen3\-8B\. The drafter runs in bfloat16 with scaled dot\-product attention instead of eager attention\. It constructs the candidate pool in the same way as the EAGLE\-3 drafter\. Target settings, verification, scoring, prompts, output limit and seeding follow the protocol in Appendix[C](https://arxiv.org/html/2610.29210#A3)\. The selection rules of Section[5\.1](https://arxiv.org/html/2610.29210#S5.SS1)therefore apply unchanged, and Table[24](https://arxiv.org/html/2610.29210#A4.T24)gives their values\.

The relative root gain grows as the budget shrinks in every run, and the interval of the root gain lies above zero at each budget from22to200200\. AtB=59B=59, the root row recovers35%35\\%to57%57\\%of the oracle’s advantage, and the PEARL\-style decoder exceeds the variant whose drafter commits to its most probable first token\. Table[24](https://arxiv.org/html/2610.29210#A4.T24)therefore compares root\-draw selection with the stronger baseline\.

Against the PEARL\-style decoder enlarged as in Appendix[D\.2](https://arxiv.org/html/2610.29210#A4.SS2)until the mean tree sizes are equal in each run, root\-draw selection still gains0\.5%0\.5\\%to1\.3%1\.3\\%atB=59B=59and4\.2%4\.2\\%to5\.2%5\.2\\%atB=8B=8, with every interval above zero\.

Table 28:Gain in decoding speed atB=8B=8with each standalone drafter in its own process on a second GPU, over draft selection and over the PEARL\-style decoder, with95%95\\%prompt\-bootstrap intervals in brackets\.Table 29:Root\-draw selection at its lowered cap against the PEARL\-style decoder atB=59B=59\. Nodes and Tree pass are means over rounds\. Expected is its gain in expected acceptance on the recorded rounds, Speed its gain in decoding speed, with95%95\\%intervals\.
### D\.14Pipelined decoding

Llama\-3\.1\-8B and Qwen3\-8B decode all644644prompts again with the standalone drafters of Appendix[D\.13](https://arxiv.org/html/2610.29210#A4.SS13), now with the drafter in its own process on a second RTX A6000 GPU, as in PEARL’s deployment\([Liu et al\., 2025a](https://arxiv.org/html/2610.29210#bib.bib7)\)\. Target and drafter run at batch size one with a static key\-value cache, and every call after the prompt prefill replays a captured CUDA graph\. In each round, the target process sends the newly committed tokens to the drafter, which grows its pool and returns it\. The drafter waits while the target verifies, so no decoder uses PEARL’s post\-verification, which drafts during verification\.

Draft selection waits for the pool and verifies the continuation token and itsBB\-node tree in one target pass\. The PEARL\-style decoder, root\-draw selection and root selection run a one\-token target pass on the continuation token while the drafter works\. They draw the first token from the returned root row and then verify their trees as described in Appendix[D\.3](https://arxiv.org/html/2610.29210#A4.SS3)\. Root selection ranks the pool with this row and verifies its whole tree\. When the drawn token is not on the tree’s first level, which happens in1\.2%1\.2\\%to8\.4%8\.4\\%of its rounds atB=59B=59, the drawn token becomes the round’s continuation and the second pass is skipped\.

The settings areT∈\{0,1\}T\\in\\\{0,1\\\}andB∈\{8,59\}B\\in\\\{8,59\\\}, and a decode stops at its first end\-of\-sequence token or after the round in which its output reaches the default limit\. After warm\-up decodes on two prompts, each prompt runs all sixteen combinations of decoder,TTandBBonce, in an order rotated by its position in a seeded prompt order, so resampling prompts also resamples the timing noise of single decodes\.

A decode’s speed counts the tokens emitted up to the first end\-of\-sequence token over the time from the start of the target prefill to the last commit, so it includes the prefill, and a run’s speed divides the summed tokens by the summed time\. Gains compare the speeds of two decoders on the same prompts, with95%95\\%percentile intervals from2,0002\{,\}000paired bootstrap resamples of prompts\. All prompts are retained in the timing comparison\.

Drafting takes a median of5050to5151ms a round and the one\-token target pass2626to2828ms, so that pass ends while the drafter works, and the median round of root\-draw or root selection lies within0\.40\.4ms of draft selection’s\. AtB=59B=59, root selection decodes3\.2%3\.2\\%to6\.2%6\.2\\%faster than draft selection, root\-draw selection3\.5%3\.5\\%to9\.6%9\.6\\%faster and the PEARL\-style decoder4\.3%4\.3\\%to8\.3%8\.3\\%faster, and no lower end of their intervals falls below2\.7%2\.7\\%\. AtB=8B=8, root selection decodes9\.6%9\.6\\%to17\.3%17\.3\\%faster than draft selection, and Table[28](https://arxiv.org/html/2610.29210#A4.T28)gives the gains of root\-draw selection and the PEARL\-style decoder, whose intervals also exclude zero\.

In these runs, root\-draw selection’s trees hold1616to1818more nodes on average than the PEARL\-style decoder’s atB=59B=59and1\.41\.4to2\.02\.0more atB=8B=8\. Enlarging the PEARL\-style tree atB=59B=59to equal mean size would leave more than6464tokens to verify after pruning, beyond the largest captured CUDA graph, in45%45\\%to51%51\\%of rounds\. To time the two decoders at matched tree sizes, we lower root\-draw selection’s cap instead, in each run to the smallest value at which its mean expected tree size on the recorded rounds of Appendix[D\.13](https://arxiv.org/html/2610.29210#A4.SS13)still reaches the PEARL\-style decoder’s atB=59B=59, between4040and4343nodes\.

Both decoders then decode all644644prompts again in the same deployment, interleaved as before, with two sampling seeds atT=1T=1and all prompts retained\. In Table[29](https://arxiv.org/html/2610.29210#A4.T29), root\-draw selection verifies at least as many nodes on average as the PEARL\-style decoder and decodes1\.3%1\.3\\%to2\.2%2\.2\\%faster\. Its expected acceptance at these caps on the recorded rounds is1\.1%1\.1\\%to2\.5%2\.5\\%higher\. These offline gains exceed the equal\-size gains of Appendix[D\.13](https://arxiv.org/html/2610.29210#A4.SS13), which match the two decoders at root\-draw selection’s larger trees\.