普林斯顿大学与 Polymathic AI 发布 FlashMap：流映射推测解码

# FlashMap: Flow Map Speculative Decoding
Source: [https://arxiv.org/html/2610.24365](https://arxiv.org/html/2610.24365)
Jeff Shen††thanks:Corresponding author\.shenjeff@princeton\.eduZhejian ZhouAffiliation:USC Information Sciences InstituteRyan LeeAffiliation:USC Information Sciences InstituteJonathan MayAffiliation:USC Information Sciences InstituteShirley HoAffiliation:Polymathic AIAffiliation:Princeton UniversityAffiliation:New York UniversityAffiliation:Flatiron Institute

###### Abstract

Speculative decoding accelerates large language model \(LLM\) inference by using a lightweight drafter to propose a block of tokens that the target model verifies in one parallel pass\. Autoregressive drafters generate this block one token at a time, whereas block diffusion drafters such as DFlash generate it in one pass, making drafting faster\. However, these diffusion drafters predict each token independently, ignoring dependencies between tokens in the block\. We introduceFlashMap, a continuous flow drafter that conditions a flow map language model on the target model’s hidden states and generates the whole block in one forward pass while capturing inter\-token dependencies\. On Qwen3\-8B,FlashMapis faster than a matched DFlash baseline on every benchmark, with 7% higher average speedup and up to

6\.1×6\.1\\timeslossless acceleration\.

[Code \(Github\)](https://github.com/al-jshen/flashmap)

\(a\)TargetLLM\(frozen\)welandedinFlashMapdrafterallKKslots in one passnoisex0∼𝒩⁡\(0,I\)x\_\{0\}\\sim\\mathcal\{N\}\(0,I\)KKEmbedEESelf\-AttnCross\-AttnMLP×L\\times LadaLN\(s,t\)\(s,t\)Projhidden statesLM headlogitsnewyorkonfridaynighty^1:K\\hat\{y\}\_\{1:K\}argmaxfrozentrainedTargetLLMverifiesallKKslotsin one passnext round: accepted tokens extend the context\(b\)Drafting two slots afterwe landed in; the target continues withnew yorkorsan diego, each with probability12\\tfrac\{1\}\{2\}\.new?sanyork?diegoslot 1slot 2new?sanyork?diegoslot 1slot 2DFlash\. Each slot is predicted independently, so the draft followsq1​\(y1\)​q2​\(y2\)q\_\{1\}\(y\_\{1\}\)\\,q\_\{2\}\(y\_\{2\}\)\. Half of the drafts mix the modes \(new diego,san york\) and are rejected at slot 2\.FlashMap\. Each noise samplex0x\_\{0\}is sent to a single blockX0,1​\(x0\)X\_\{0,1\}\(x\_\{0\}\), so the noise selects the mode for both slots jointly\. Every draft isnew yorkorsan diego\.

\(c\)

Figure 1:\(a\) One round of speculative decoding: the drafter reads hidden features from the target and proposes the nextKKtokens in parallel with a flow map, and the target verifies them in one pass\. \(b\) Sampling two correlated slots in the few\-step regime \(adapted from Figure 3 of FMLM\): DFlash predicts each slot independently, so half of its drafts pair tokens from different modes, whileFlashMapcan represent a joint over all slots\. \(c\) Speedup \(×\\times\) on Qwen3\-8B, withFlashMapdrafting in 1 or 2 flow\-map steps\.Boldmarks the best speedup\.## 1Introduction

Speculative decoding drafts a block of future tokens with a lightweight model and verifies it in one parallel pass of the larger target LLM\. As a result, speedup depends on how many drafted tokens are accepted and how cheaply, in terms of compute, they are drafted\. EAGLE\-3\([Li et al\., 2025](https://arxiv.org/html/2610.24365#bib.bib8)\)achieves high acceptance by conditioning on the hidden states of the target model but drafts autoregressively so cost grows with block length\.

DFlash\([Chen et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib1)\)instead drafts an entire block in one forward pass using a lightweight block diffusion model, yielding over 6×\\timeslossless acceleration\. However, a single masked\-denoising pass predicts each position in the block independently\. Prior work finds that draft tokens from this model can be individually plausible but jointly inconsistent, leading to smaller accepted prefixes, and acceptance decays at later positions\([Cheng et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib9)\)\.

By changing the drafter’s generative model, we no longer draft each token independently \(known as the*factorized approximation*\)\. Flow Map Language Models\([Lee et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib5)\)distill a continuous flow over one\-hot token embeddings into a flow map that transports Gaussian noise to a full sequence in one network evaluation\. Because the map acts on the whole block jointly, it can represent dependencies between tokens that a drafter predicting each position independently cannot, while keeping DFlash’s one\-pass cost \(Figure[1](https://arxiv.org/html/2610.24365#S0.F1)\)\.

#### Contributions\.

We evaluateFlashMapon Qwen3\-8B across nine math, code, and chat benchmarks \(Qwen3\-4B in Appendix[C](https://arxiv.org/html/2610.24365#A3)\), against EAGLE\-3 and DFlash retrained under identical data and settings\. Our contributions are:

\(1\)Flow map drafting\.We introduceFlashMap, the first speculative decoding drafter built on a continuous flow\.FlashMapcombines DFlash’s target\-conditioning, single\-pass block drafting with flow map language models, allowing it to draft a whole block that carries inter\-token dependencies in a way that DFlash cannot\.\(2\)Analysis of flow map drafts\.We study what drivesFlashMap’s acceptance length, using proposal\-length ablations, conditional acceptance curves, and a shuffle test that separates the contribution of per\-slot predictions from that of inter\-token dependencies\.

## 2Background

#### Speculative decoding\.

Letppbe an autoregressive target model over a vocabulary𝒱\\mathcal\{V\}andccthe current context\. In each round, a drafter proposes a blocky^1:K\\hat\{y\}\_\{1:K\}from a proposalq\(⋅∣c\)q\(\\cdot\\mid c\), and the target scores allKKpositions in one forward pass\. The target accepts tokens left to right with the rejection rule of[Leviathan et al\. \(2023\)](https://arxiv.org/html/2610.24365#bib.bib11), which keeps the output distributed exactly aspp\. At the first rejection, the target resamples that token from the residual distribution and discards the rest of the block\. Under greedy decoding \(T=0T=0\), the rule reduces to keeping the longest prefix that matches the target’s argmax\. The acceptance lengthγ∈\[1,K\+1\]\\gamma\\in\[1,K\+1\]is the expected number of tokens committed per round, including the one contributed by the target\. The per\-token latency is\(Tdraft\+Tverify\)/γ\(T\_\{\\text\{draft\}\}\+T\_\{\\text\{verify\}\}\)/\\gamma\([Chen et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib1)\), whereTverifyT\_\{\\text\{verify\}\}is roughly constant because decoding is memory\-bound\([Chen et al\., 2023](https://arxiv.org/html/2610.24365#bib.bib12)\)\.

#### DFlash\.

DFlash\([Chen et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib1)\)drafts with a block discrete diffusion model\. It attends to hidden states from several target layers and denoises a fully masked block in a single forward pass, soTdraftT\_\{\\text\{draft\}\}is independent ofKK\. Its token embedding and language modeling \(LM\) head are copied from the target and kept frozen, so only a few transformer layers are trained\. Training samples random anchor positions in target\-generated responses and weights the per\-slot cross\-entropy by exponentially decaying weights over block positions\.

#### Factorization error\.

Discrete diffusion samples tokens at every denoising step\. Sampling several tokens in one step would require their joint distribution over𝒱K\\mathcal\{V\}^\{K\}, whose size is exponential inKKand thus intractable, so few\-step inference with discrete diffusion typically relies on the factorized approximation: each position is instead predicted independently given the noisy state\([Lee et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib5)\)\. A one\-pass drafter, whose input is the fully masked block, therefore proposes

q\(y1:K∣c\)=∏k=1Kqk\(yk∣c\)\.q\(y\_\{1:K\}\\mid c\)=\\prod\_\{k=1\}^\{K\}q\_\{k\}\(y\_\{k\}\\mid c\)\.\(1\)Per\-slot cross\-entropy is minimized by the target’s marginalsqk=p⁡\(yk∣c\)q\_\{k\}=p\(y\_\{k\}\\mid c\)\. The remaining gap is the total correlationKL\(p\(y1:K∣c\)∥∏kp\(yk∣c\)\)\\mathrm\{KL\}\\big\(p\(y\_\{1:K\}\\mid c\)\\,\\\|\\,\\prod\_\{k\}p\(y\_\{k\}\\mid c\)\\big\), which no amount of model capacity removes\. For example, if the continuation is “new york” or “san diego” with equal probability, the drafter proposes “new diego” or “san york” half of the time\([Lee et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib5)\)\. This is a problem that plagues many non\-autoregressive methods\([Gu et al\., 2018](https://arxiv.org/html/2610.24365#bib.bib35), e\.g\.,\)\. Taking a large number of steps at inference time recovers these dependencies, but is antithetical to the goal of a fast, few\-pass drafter\.

#### Flows over one\-hot tokens\.

Flow Map Language Models \(FMLM;[Lee et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib5)\) instead model language with a continuous flow\. The flow’s only randomness is the initial noise, which a deterministic trajectory transports to the whole block jointly\. A sequencey∈𝒱Ly\\in\\mathcal\{V\}^\{L\}is represented by its one\-hot encodingx1∈ℝL×\|𝒱\|x\_\{1\}\\in\\mathbb\{R\}^\{L\\times\|\\mathcal\{V\}\|\}and decoded with an argmax\. Following rectified flow\([Lipman et al\., 2023](https://arxiv.org/html/2610.24365#bib.bib13);[Liu et al\., 2022](https://arxiv.org/html/2610.24365#bib.bib14);[Albergo et al\., 2025](https://arxiv.org/html/2610.24365#bib.bib15)\), noise and data are connected by

It=\(1−t\)​x0\+t​x1,x0∼𝒩⁡\(0,I\),I\_\{t\}=\(1\-t\)\\,x\_\{0\}\+t\\,x\_\{1\},\\qquad x\_\{0\}\\sim\\mathcal\{N\}\(0,I\),\(2\)and the velocityvt​\(x\)=𝔼⁡\[x1−x0∣It=x\]v\_\{t\}\(x\)=\\mathbb\{E\}\[x\_\{1\}\-x\_\{0\}\\mid I\_\{t\}=x\]transportsx0x\_\{0\}att=0t=0to data att=1t=1\. FMLM predicts the clean data rather than the velocity, since the clean data are low\-rank while the velocity is noisy and full\-rank\([Li and He, 2026](https://arxiv.org/html/2610.24365#bib.bib19)\)\. A denoiserDtθD^\{\\theta\}\_\{t\}outputs the clean\-data prediction, and the velocity is recovered as

vtθ​\(x\)=Dtθ​\(x\)−x1−t\.v^\{\\theta\}\_\{t\}\(x\)=\\frac\{D^\{\\theta\}\_\{t\}\(x\)\-x\}\{1\-t\}\.\(3\)Because one\-hot vectors are the vertices of the simplex, the true denoiserDt\(x\)l=p\(yl=⋅∣It=x\)D\_\{t\}\(x\)^\{l\}=p\(y\_\{l\}=\\cdot\\mid I\_\{t\}=x\)lies on the simplex at every position\. FMLM therefore parameterizesDtθD^\{\\theta\}\_\{t\}with a softmax and trains it with cross\-entropy,

ℒCE\(θ\)=𝔼t,x0,x1\[−∑l=1Lx1l⋅logDtθ\(It\)l\],\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\(\\theta\)=\\mathbb\{E\}\_\{t,x\_\{0\},x\_\{1\}\}\\Big\[\-\\sum\_\{l=1\}^\{L\}x\_\{1\}^\{l\}\\cdot\\log D^\{\\theta\}\_\{t\}\(I\_\{t\}\)^\{l\}\\Big\],\(4\)whose unique minimizer is the true denoiser\. Sampling integrates the flow with a numerical ODE solver, typically over tens of steps\.

#### Flow maps\.

The flow mapXs,tX\_\{s,t\}sends the ODE’s state at timessto its state at timett\([Boffi et al\., 2025a](https://arxiv.org/html/2610.24365#bib.bib16);[Boffi et al\., 2025b](https://arxiv.org/html/2610.24365#bib.bib17)\)\. It satisfiesXs,s=idX\_\{s,s\}=\\mathrm\{id\}and the semigroup propertyXu,t∘Xs,u=Xs,tX\_\{u,t\}\\circ X\_\{s,u\}=X\_\{s,t\}\. FMLM parameterizes it through a two\-time denoiserδs,t\\delta\_\{s,t\}on the simplex,

Xs,t​\(x\)=1−t1−s​x\+t−s1−s​δs,t​\(x\),X\_\{s,t\}\(x\)=\\frac\{1\-t\}\{1\-s\}\\,x\+\\frac\{t\-s\}\{1\-s\}\\,\\delta\_\{s,t\}\(x\),\(5\)which reduces to the denoiser on the diagonal,δs,s=Ds\\delta\_\{s,s\}=D\_\{s\}\([Lee et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib5), Prop\. 3\.3\)\. A single evaluationX0,1​\(x0\)=δ0,1​\(x0\)X\_\{0,1\}\(x\_\{0\}\)=\\delta\_\{0,1\}\(x\_\{0\}\)thus maps noise to data\. The mapδ\\deltais obtained by distilling a pretrained flow \(§[3\.2](https://arxiv.org/html/2610.24365#S3.SS2)\)\.

## 3Method

Our drafter is a flow over one\-hot draft blocks, conditioned on the target in the same way as DFlash\. We first train it as a flow \(§[3\.1](https://arxiv.org/html/2610.24365#S3.SS1)\) and then distill it into a two\-time denoiser that drafts a whole block in a single network evaluation \(§[3\.2](https://arxiv.org/html/2610.24365#S3.SS2)\)\. Letaabe the position of the anchoryay\_\{a\}, the last token committed by the target\. The drafter proposes the nextKKtokens, and we index the block by slotk=1,…,Kk=1,\\dots,K, so that slotkkholdsya\+ky\_\{a\+k\}\. The drafter operates on a noisy block, which we call the*state*x∈ℝK×\|𝒱\|x\\in\\mathbb\{R\}^\{K\\times\|\\mathcal\{V\}\|\}: the point on the flow’s trajectory between Gaussian noise \(t=0t=0\) and the one\-hot encoding of theKKdraft tokens \(t=1t=1\)\. During training the state is the interpolantItI\_\{t\}, and during drafting it is the current iterate of the flow\. We writexk∈ℝ\|𝒱\|x^\{k\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\}for the state at slotkk, andccfor the target features available at drafting time\.

### 3\.1Continuous Flow Drafter

The drafter’s denoiserDtθ​\(x,c\)D^\{\\theta\}\_\{t\}\(x;c\)is trained as in FMLM \(Eqs\.[2](https://arxiv.org/html/2610.24365#S2.E2)–[4](https://arxiv.org/html/2610.24365#S2.E4)\), with changes to the architecture, the input featurization, and the loss weighting\.

#### Architecture\.

The backbone is a conditional diffusion transformer \(DiT;[Peebles and Xie, 2023](https://arxiv.org/html/2610.24365#bib.bib18)\)\. Each block applies self\-attention among theKKslots, cross\-attention to context features from the target, and an MLP, each modulated by adaLN\-Zero on the timett\. The context features are the target’s hidden states from a fixed set of layers, concatenated and linearly projected at each position before the anchor, together with an embedding of the anchor tokenyay\_\{a\}\. Both vocabulary projections are copied from the target and kept frozen: the state enters the network through the target’s embedding matrixE∈ℝ\|𝒱\|×dE\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\\times d\}, and the output passes through the target’s LM head, with a softmax over the vocabulary at each slot givingDtθ​\(x,c\)D^\{\\theta\}\_\{t\}\(x;c\)\.

#### Training\.

We train on target\-generated responses with cached target features and sample many anchor positions per sequence, as in DFlash\. Each block receives its own timettand noisex0x\_\{0\}and is noised toIt=\(1−t\)x0\+t⋅onehot\(ya\+1:a\+K\)I\_\{t\}=\(1\-t\)\\,x\_\{0\}\+t\\cdot\\mathrm\{onehot\}\(y\_\{a\+1:a\+K\}\)\. Following DFlash, we weight the cross\-entropy of Eq\. \([4](https://arxiv.org/html/2610.24365#S2.E4)\) at slotkkbywk=e−\(k−1\)/κw\_\{k\}=e^\{\-\(k\-1\)/\\kappa\}\. The weights emphasize early slots, since an early error invalidates every later token\.

#### Time reparameterization\.

With a large vocabulary \(e\.g\.,\|𝒱\|=151,936\|\\mathcal\{V\}\|=151\{,\}936for Qwen3\), the state reveals the clean tokens only within a narrow window ofttnear11\([Lee et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib5)\)\. Samplingttuniformly therefore spends most of training at times where the state reveals almost nothing or has already revealed everything\. We adopt FMLM’s decoding\-error reparameterization, which measures time by how many tokens have been resolved\. LetPe​\(t\)P\_\{e\}\(t\)be the probability that decoding the state at a slot by its argmax gives the wrong token; it falls from1−1/\|𝒱\|1\-1/\|\\mathcal\{V\}\|under pure noise to00att=1t=1\. The reparameterized timeτ⁡\(t\)=1−Pe​\(t\)/Pe​\(0\)\\tau\(t\)=1\-P\_\{e\}\(t\)/P\_\{e\}\(0\)is the fraction of this initial error removed by timett, so equal steps inτ\\taumake equal progress in resolving tokens\. For our interpolant,Pe​\(t\)P\_\{e\}\(t\)has the closed form

Pe​\(t\)=1−𝔼z∼𝒩⁡\(0,1\)​\[Φ​\(z\+t1−t\)\|𝒱\|−1\],P\_\{e\}\(t\)=1\-\\mathbb\{E\}\_\{z\\sim\\mathcal\{N\}\(0,1\)\}\\Big\[\\Phi\\big\(z\+\\tfrac\{t\}\{1\-t\}\\big\)^\{\|\\mathcal\{V\}\|\-1\}\\Big\],\(6\)whereΦ\\Phiis the standard normal CDF\. We sampleτ∼𝒰⁡\[0,1\]\\tau\\sim\\mathcal\{U\}\[0,1\]during training and map it back tottby invertingτ⁡\(t\)\\tau\(t\); at inference, we use the gridtn=τ−1​\(n/N\)t\_\{n\}=\\tau^\{\-1\}\(n/N\)\. Drafting directly from the flow takesNNEuler steps on this grid, and thus costsNNNFEs per drafting round, typically𝒪⁡\(100\)\\mathcal\{O\}\(100\)\.

### 3\.2Distillation into a Flow Map

Drafting with𝒪⁡\(100\)\\mathcal\{O\}\(100\)network evaluations per round is expensive, and taking fewer Euler steps reduces this cost at the price of a less accurate draft\. In the extreme of a single step fromt=0t=0, the draft loses all joint structure: it is the denoiser’s prediction from pure noise, which is trained on noise drawn independently of the data\. Its best value is therefore the per\-slot marginals whatever the noise, i\.e\., the factorized drafter of Eq\. \([1](https://arxiv.org/html/2610.24365#S2.E1)\)\. The noise shapes the sample only through integration, as each step moves the state in a noise\-dependent direction that later evaluations resolve into one coherent block\.

A flow map\([Boffi et al\., 2025a](https://arxiv.org/html/2610.24365#bib.bib16);[Boffi et al\., 2025b](https://arxiv.org/html/2610.24365#bib.bib17)\)is the solution operator of the flow’s ODE:Xs,tX\_\{s,t\}sends the state at any timessto the state the ODE reaches at any later timett\. Learning it replaces numerical integration over\[s,t\]\[s,t\]with a single network evaluation, so the exact map preserves the flow’s joint distribution over the block at any number of jumps, including one\. In practice, a learned map approximates long jumps less accurately than short ones, so at 1\-NFE it captures this joint structure only partially \(§[6\.2](https://arxiv.org/html/2610.24365#S6.SS2)\)\. Following the flow map distillation framework of[Boffi et al\. \(2025a\)](https://arxiv.org/html/2610.24365#bib.bib16), we distill the trained flow \(the teacher\) into a flow map \(the student\), a two\-time denoiserδs,tϕ​\(x,c\)\\delta^\{\\phi\}\_\{s,t\}\(x;c\)that estimates the jump fromsstott\. With a single jump, the draft isX0,1​\(x0\)=δ0,1ϕ​\(x0,c\)X\_\{0,1\}\(x\_\{0\}\)=\\delta^\{\\phi\}\_\{0,1\}\(x\_\{0\};c\); more jumps can be composed at the cost of one evaluation each\([Boffi et al\., 2025a](https://arxiv.org/html/2610.24365#bib.bib16)\)\.

#### Semigroup distillation\.

We train the student on the semigroup property of flow maps\([Boffi et al\., 2025a](https://arxiv.org/html/2610.24365#bib.bib16);[Boffi et al\., 2025b](https://arxiv.org/html/2610.24365#bib.bib17)\): a jump fromsstottmust land in the same place as a jump fromssto an intermediate timeuufollowed by a jump fromuutott\. For zero\-length jumps \(s=ts=t\), the target is the frozen flow’s denoiserDsθ​\(Is\)D^\{\\theta\}\_\{s\}\(I\_\{s\}\), which enforces the tangent conditionδs,s=Ds\\delta\_\{s,s\}=D\_\{s\}\([Boffi et al\., 2025b](https://arxiv.org/html/2610.24365#bib.bib17)\)and anchors the student to the teacher\. For longer jumps \(s<ts<t\), the target is the student’s own estimate from splitting the jump at the midpointuu, which in terms of the two\-time denoiser is a weighted average\([Lee et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib5)\):

δ¯s,t=sg⁡\(λ​δs,uϕ​\(Is\)\+\(1−λ\)​δu,tϕ​\(Xs,uϕ​\(Is\)\)\),\\bar\{\\delta\}\_\{s,t\}=\\sg\\Big\(\\lambda\\,\\delta^\{\\phi\}\_\{s,u\}\(I\_\{s\}\)\+\(1\-\\lambda\)\\,\\delta^\{\\phi\}\_\{u,t\}\\big\(X^\{\\phi\}\_\{s,u\}\(I\_\{s\}\)\\big\)\\Big\),\(7\)whereλ=\(1−t\)​\(u−s\)\(1−u\)​\(t−s\)\\lambda=\\frac\{\(1\-t\)\(u\-s\)\}\{\(1\-u\)\(t\-s\)\},Xs,uϕX^\{\\phi\}\_\{s,u\}is the student’s first jump \(Eq\.[5](https://arxiv.org/html/2610.24365#S2.E5)\), andsg\\sgdenotes the stop\-gradient, since the target is computed from the student itself\. Training toward this target bootstraps from short jumps, which are anchored to the teacher, to progressively longer ones\([Boffi et al\., 2025b](https://arxiv.org/html/2610.24365#bib.bib17)\)\. Following FMLM, half of the blocks train zero\-length jumps and the rest train jumps of random start and length, including a small fraction of full jumps\(0,1\)\(0,1\)\. The resulting map is valid for any pair of times, so it can also draft with several shorter jumps\. Appendix[B](https://arxiv.org/html/2610.24365#A2)describes an optimization for multi\-step jumps\.

Table 1:Acceptance lengthγ\\gammaon Qwen3\-8B\. For each temperature and benchmark, bold marks the best result\. Speedups are in Figure[1](https://arxiv.org/html/2610.24365#S0.F1)\(c\)\.

## 4Experimental Setup

#### Models\.

We use Qwen3\-4B and Qwen3\-8B\([Yang et al\., 2025](https://arxiv.org/html/2610.24365#bib.bib22)\)as target models and implement all drafters in DeepSpec111[https://github\.com/deepseek\-ai/DeepSpec](https://github.com/deepseek-ai/DeepSpec)\. We extract the target model’s hidden states at layers 1, 9, 17, 25, and 33, to be used as inputs to all drafters\.

FlashMapuses 6 DiT blocks\([Peebles and Xie, 2023](https://arxiv.org/html/2610.24365#bib.bib18)\), embedding dimension 2048, 16 attention heads, QK\-norm, and GELU activations in the MLPs\. We note that DFlash uses the same embedding dimension as the target model\. Thus, for Qwen3\-8B, DFlash’s embedding dimension is 4096, and the drafter has a total of∼\{\\sim\}1B parameters, whileFlashMaphas∼\{\\sim\}0\.5B parameters\. For EAGLE\-3 we use training\-time test of TTT=7, top\-k of 10, and a tree size of 16 to ensure that all methods have the same verification budget\.

#### Training\.

We train on 500K samples from Open\-PerfectBlend222[https://huggingface\.co/datasets/mlabonne/open\-perfectblend](https://huggingface.co/datasets/mlabonne/open-perfectblend), an open\-source version of PerfectBlend\([Xu et al\., 2024](https://arxiv.org/html/2610.24365#bib.bib23)\); we use only a subset of the full dataset because of storage and compute limits\. We train with block sizeK=16K=16, and set the slot loss decaywkw\_\{k\}toκ=7\\kappa=7\. We use the AdamW optimizer\([Loshchilov and Hutter, 2019](https://arxiv.org/html/2610.24365#bib.bib24)\)with\(β1,β2\)=\(0\.9,0\.95\)\(\\beta\_\{1\},\\beta\_\{2\}\)=\(0\.9,0\.95\)and weight decay0\.10\.1, which we find important for stable training\. We use a cosine learning rate schedule with 4% warmup and a peak learning rate of6×10−46\\times 10^\{\-4\}\. We train in bf16 with gradient\-norm clipping at1\.01\.0and a global batch size of 256 for 20 epochs \(39K steps\)\. The EAGLE\-3 and DFlash baselines are retrained with the same data and under the same settings for controlled comparison\.

#### Distillation\.

We initialize the flow map from the weights of the flow teacher\. A fraction1/161/16of the jumps are full jumps\(s,t\)=\(0,1\)\(s,t\)=\(0,1\), and we use a warmup\-stable\-decay learning rate schedule with 4% warmup, peak learning rate of5×10−55\\times 10^\{\-5\}, and 20% decay\. We use no weight decay and train for 12 epochs \(23k steps\)\. All other settings match those of the teacher\.

#### Evaluation\.

We evaluate on nine benchmarks in three domains: math \(GSM8K\([Cobbe et al\., 2021](https://arxiv.org/html/2610.24365#bib.bib25)\), MATH\([Hendrycks et al\., 2021](https://arxiv.org/html/2610.24365#bib.bib26)\), and AIME25\([Zhang and Math\-AI, 2025](https://arxiv.org/html/2610.24365#bib.bib27)\)\), code \(MBPP\([Austin et al\., 2021](https://arxiv.org/html/2610.24365#bib.bib28)\), HumanEval\([Chen et al\., 2021](https://arxiv.org/html/2610.24365#bib.bib29)\), and LiveCodeBench \(LCB;[Jain et al\. \(2024\)](https://arxiv.org/html/2610.24365#bib.bib30)\)\), and chat \(MT\-Bench\([Bai et al\., 2024](https://arxiv.org/html/2610.24365#bib.bib31)\), Alpaca\([Li et al\., 2023](https://arxiv.org/html/2610.24365#bib.bib32)\), and Arena\-Hard\([Li et al\., 2024](https://arxiv.org/html/2610.24365#bib.bib33)\)\)\. We report the acceptance lengthγ\\gammaand the end\-to\-end speedup over autoregressive decoding with the target, at temperaturesT=0T=0andT=1T=1\.FlashMapdrafts with one or two steps, i\.e\., jumps of the flow map\. All evaluations use SGLang at concurrency 1 and speedups are measured on an NVIDIA H200 NVL GPU, unless otherwise specified\.

Figure 2:Acceptance lengthγ\\gammaagainst the proposal lengthKKused at inference on Qwen3\-8B atT=0T=0, forFlashMapwith one and two steps and for DFlash, retrained with the same data and settings, on GSM8K, HumanEval, and MT\-Bench \(80 prompts each\)\. All drafters were trained atK=16K=16and run at smallerKKwithout retraining\. Each point is a single run, so no uncertainty is shown\.

## 5Results

We show the benefit of speculative decoding methods over plain autoregressive decoding with the target model in terms of speedup \(Figure[1](https://arxiv.org/html/2610.24365#S0.F1)\(c\)\) and acceptance lengthγ\\gamma\(Table[1](https://arxiv.org/html/2610.24365#S3.T1)\) for various draft models, as applied to Qwen3\-8B\. We show analogous results for Qwen3\-4B in Appendix[C](https://arxiv.org/html/2610.24365#A3)\. Across all models, speedups are largest for math and code tasks, where text is highly structured, and smaller for chat tasks, which are more open\-ended; in chat, a given piece of text can have many possible relevant continuations\.

#### One stepFlashMapis the fastest drafter\.

At bothT=0T=0andT=1T=1,FlashMapat 1 NFE leads to the highest speedup over the base model across every task and domain, with speedups of up to6\.14×6\.14\\timesfor GSM8K atT=0T=0\. As compared to DFlash,FlashMapat 1 NFE is7\.4%7\.4\\%faster on average across all tasks\.

#### One stepFlashMapalso drafts more accurately than DFlash\.

AlthoughFlashMapat 1 NFE does not have the highest acceptance length in all, or even a majority of the tasks, it does have higher acceptance length than DFlash in 15/18 settings, despite having half the parameters of DFlash\. This indicates that the wins in speedup are not coming from a smaller model alone \(i\.e\., reduced latency\)—FlashMapat 1 NFE is also a highly accurate model\. Our gains over DFlash are largest in the chat tasks—the most difficult ones\. In particular, atT=0T=0, while acceptance length/speedup on math tasks is only 2\.3%/5\.5% higher than for DFlash, on chat tasks this increases to 7\.4%/11\.1%\.

#### AR drafters are accurate but slow\.

On the acceptance length front,FlashMapat 2 NFE and EAGLE\-3 are both stronger thanFlashMapat 1 NFE, especially atT=1T=1\. In particular, atT=1T=1, EAGLE\-3 is the drafter with the highest acceptance length in 6/9 tasks\. However, EAGLE\-3, being an autoregressive drafter, requires multiple sequential passes to produce a single draft, resulting in very high latency\. Given that total speedup depends both on acceptance length and latency, the small edge that EAGLE\-3 has in terms of acceptance length does not offset its higher latency, leading to EAGLE\-3 being not competitive with parallel drafters\. Indeed, atT=1T=1, averaged across tasks, EAGLE\-3 has 9% higher acceptance length but is a whopping 35% slower thanFlashMapat 1 NFE\.

#### Two stepFlashMaphas the best acceptance of any parallel drafter\.

FlashMapat 2 NFE beats DFlash in acceptance length by an average of10%10\\%, and wins 18/18 settings \(atT=0T=0andT=1T=1\)\. It also remains slightly faster than DFlash, winning 13/18 settings, despite two draft passes \(Appendix[B](https://arxiv.org/html/2610.24365#A2)\)\.

As an alternative to more function evaluations, another way to spend more drafting compute is to draft several blocks from independent noise draws and verify them together as a tree, which we explore in Table[4](https://arxiv.org/html/2610.24365#A4.T4); we find that this can lead to speedups of∼11%\{\\sim\}11\\%over single\-sample drafting with standard chain verification\.

Figure 3:Conditional acceptance rate on Qwen3\-8B atT=0T=0for the drafters of Figure[2](https://arxiv.org/html/2610.24365#S4.F2): the probability that the draft token at positionkkof aK=16K=16block is accepted given that all earlier tokens in the block were accepted, pooled over the math, code, and chat tasks\. Positions are 0\-indexed, so positionkkis slotk\+1k\+1; shaded bands are 95% Wilson intervals\.

## 6Discussion

The results above show thatFlashMapat one step is the fastest drafter we evaluate\. We now ask where its gains come from, whether its drafts use dependencies between tokens, and what limits one\-step drafting\.

### 6\.1Where is acceptance lost?

A parallel drafter proposes a whole block in one pass, so a longer block adds little drafting cost\. We investigate how acceptance length varies as a function of block size\. Figure[2](https://arxiv.org/html/2610.24365#S4.F2)shows acceptance length when the drafters, all trained atK=16K=16, propose blocks ofK=2K=2to1616tokens on Qwen3\-8B \([Chen et al\. 2026](https://arxiv.org/html/2610.24365#bib.bib1)show that the penalty of such a training\-inference mismatch inKKis∼0\.15\{\\sim\}0\.15tokens\), and Figure[3](https://arxiv.org/html/2610.24365#S5.F3)shows the conditional acceptance rate at each slot of a 16\-token block \(the acceptance rate at a given slot, assuming that all tokens before that position were accepted\)\.

Acceptance length generally grows withKKup to the trained block length, withγ\\gammagrowing more slowly for chat than for math and code \(note the different y\-axes\)\. The conditional acceptance curves show where the drafters differ\. One\-stepFlashMapmatches or exceeds DFlash at the first slot in every domain; however, the conditional acceptance decays more rapidly forFlashMapat 1 NFE than for DFlash\. This indicates that the later slots in the block are less likely to be accepted\. ForFlashMapwith two steps, the rate at slot 16 is higher than for DFlash across all domains\. Because a gain at an early slot counts toward every block that reaches it, while a loss at a late slot affects only the few blocks that get that far, one\-stepFlashMapstill accepts more than DFlash overall \(Table[1](https://arxiv.org/html/2610.24365#S3.T1)\), but it gains less than DFlash from its last slots\.

Notably, two\-stepFlashMap’s conditional acceptance does not decay within the block for any domain\. The conditional acceptance curve stays flat, or even rises, matching what is observed in models that apply sequential postprocessing to re\-capture inter\-token dependencies\([Cheng et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib9)\)\. This indicates that it may gain more than DFlash or one\-stepFlashMapfrom blocks even longer than 16 tokens, but testing this requires drafters trained at largerKK\. Why one\-step drafts, which come from a joint model, fall below a factorized drafter at late positions in the block is the question we turn to next\.

### 6\.2DoesFlashMapcapture joint structure?

Whether a draft is accepted depends on two properties of the drafter’s distribution over blocks: its per\-slot marginals, i\.e\., how often each slot’s token matches the target on its own, and the dependence between slots, i\.e\., whether the tokens in a block are consistent with each other\. DFlash has only marginals: it predicts each slot independently and, atT=0T=0, drafts the most likely token of each\.FlashMapdrafts the whole block from one noise draw, which can make the block consistent, but each slot’s token then depends on the noise, so it is not necessarily the most likely token under that slot’s marginal\. The conditional acceptance rates in Figure[3](https://arxiv.org/html/2610.24365#S5.F3)reflect both properties and cannot separate them\. We separate them with a shuffle test: we compare drafts whose slots all come from one shared noise draw \(shared\) with drafts using a different, independent noise draw per slot \(shuffled\)\. Shuffling keeps each slot’s marginal but removes the dependence between slots, soγshuffled\\gamma\_\{\\mathrm\{shuffled\}\}measures the marginals alone andΔ=γshared−γshuffled\\Delta=\\gamma\_\{\\mathrm\{shared\}\}\-\\gamma\_\{\\mathrm\{shuffled\}\}is the acceptance gain due to the draft’s joint structure\. A drafter whose proposal does not depend on the noise, such as DFlash, hasΔ=0\\Delta=0by construction, whereas a drafter that exploits joint structure hasΔ\>0\\Delta\>0\. We show the results of the shuffle test in Table[2](https://arxiv.org/html/2610.24365#S6.T2)\.

#### Integration creates joint structure\.

Joint structure is modelled along a flow’s trajectory\. Indeed, when performing only a single evaluation of the flow model, we find that there is no difference in acceptance length between the shared noise configuration and the shuffled noise configuration, indicating that the flow captures no joint structure at 1 NFE\. However, the same flow, when integrated for 16 steps, hasΔ\>0\\Delta\>0, and indeed, can attribute almost an entire token’s worth of acceptance to joint structure\. This indicates that it is integration of the flow that resolves each noise draw into one consistent block\. It is this integration that we try to distill into a few\-step flow map inFlashMap\.

#### Two steps capture most of it\.

At 1 NFE,FlashMapretains only a small fraction of theΔ\\Deltain the 16\-step teacher, while at 2\-NFE,FlashMaprecovers most of theΔ\\Deltain the 16\-step teacher\. This indicates that a single jump of the flow map from pure noise underutilizes the noise, and produces a draft that, while not totally factorized, does not fully capture all inter\-token dependencies—consistent with its falling conditional acceptance in Figure[3](https://arxiv.org/html/2610.24365#S5.F3)\. In contrast, the 2 NFE flow map heavily exploits inter\-token dependencies, which explains its flat or increasing conditional acceptance along the block \(Figure[3](https://arxiv.org/html/2610.24365#S5.F3)\)\.

Table 2:Shuffle test on Qwen3\-8B atT=0T=0: accepted draft tokens per block, for a draft with shared noise across slots \(shared\) and for a draft with independent per\-slot noise \(shuffled\)\.Δ\\Deltais the difference between the two, with a larger value indicating a greater acceptance due to joint structure\. Results for Qwen3\-4B are in Table[5](https://arxiv.org/html/2610.24365#A6.T5)\.

### 6\.3What limits one\-step drafting?

At both 1 NFE and 2 NFE,FlashMaptrails the flow teacher at 16 NFE in terms of acceptance length under the shared setting, which is the usual inference setting\. A better distillation strategy would reduce this gap and make the 1 NFE flow map competitive with the many\-NFE flow teacher\. Further,FlashMapalso trails the teacher in the shuffled setting, indicating that the per\-position marginals are also less well estimated\.

While in theory flow map distillation enables few\-step flow maps to capture inter\-token dependencies, these results indicate that there is still a gap between few\-step flow map models and one\-step models\. Indeed, some recent works on flow map language models find that while performance of such models is strong in the few\-step regime, one\-step models still suffer from high generative perplexity or collapsed entropy\([Potaptchik et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib6);[Tang and Wang, 2026](https://arxiv.org/html/2610.24365#bib.bib34), e\.g\.,\)\. Our results also indicate potential for future work to improve, both on the theoretical and empirical sides, the distillation strategies for discrete flow maps, particularly for inference at 1 NFE\.

## 7Related Work

#### Speculative decoding\.

Speculative decoding drafts tokens with a cheap model and verifies them with the target in one parallel pass, while keeping the target’s output distribution\([Leviathan et al\., 2023](https://arxiv.org/html/2610.24365#bib.bib11);[Chen et al\., 2023](https://arxiv.org/html/2610.24365#bib.bib12)\)\. Most drafters are autoregressive\. EAGLE\-3\([Li et al\., 2025](https://arxiv.org/html/2610.24365#bib.bib8)\)conditions a lightweight drafter on the target’s hidden states and reaches high acceptance, but it drafts one token per forward pass, so its drafting cost grows with block length\.

#### Diffusion drafters\.

Discrete diffusion drafters instead propose a whole block in parallel\. DiffuSpec\([Li et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib2)\)and SpecDiff\-2\([Sandler et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib3)\)repurpose large pretrained diffusion LMs, adding causal path search and verifier calibration respectively, but their drafter size limits practical speedups\. DFlash\([Chen et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib1)\)trains a lightweight block diffusion drafter on the target’s hidden states and drafts a block in a single forward pass\. Recent work adds draft trees on top of DFlash to accelerate sampling\([Ringel and Romano, 2026](https://arxiv.org/html/2610.24365#bib.bib4)\)\. Since a one\-pass discrete drafter like DFlash predicts each position independently \(Eq\.[1](https://arxiv.org/html/2610.24365#S2.E1)\), follow\-up works attempt to recover joint structure from this factorized output through autoregressive postprocessing\([Cheng et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib9);[Huang et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib21);[Zheng and Li, 2026](https://arxiv.org/html/2610.24365#bib.bib20)\)or low\-rank latent mixtures\([Karimi et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib10)\)\. These methods add search or sequential computation on top of the parallel draft, or restrict the joint to a low\-rank family\.

#### Flow maps for discrete data\.

Alongside Flow Map Language Models\([Lee et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib5)\), Discrete Flow Maps\([Potaptchik et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib6)\)and Categorical Flow Maps\([Roos et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib7)\)learn flow maps that respect the geometry of the probability simplex, achieving strong few\-step generation of text and other categorical data\. Instead of standalone generation, we focus on drafting for a target model\.

#### Flow models as drafters\.

To our knowledge,FlashMapis the first speculative decoding drafter built on a continuous flow\. Rather than correcting a factorized proposal after drafting, it changes the drafter’s generative model, so that dependencies between slots can be represented within the drafting pass itself\.FlashMapkeeps DFlash’s conditioning on target hidden states and its frozen embedding and LM head, and replaces its discrete diffusion model with a continuous flow\.

## Limitations

We evaluate two target models from one family, Qwen3\-4B and Qwen3\-8B; other model families and larger targets are left to future work\. Because of compute limits, all drafters, including the baselines, are trained on a 500K\-sample subset of Open\-PerfectBlend, so absolute acceptance lengths may differ from those of drafters trained on the full data\. Speedups are measured at concurrency 1 on a single GPU, and batched serving is not evaluated\. We draft blocks of 16 tokens and compare against EAGLE\-3 at the same 16\-token verification budget\. Our baselines are EAGLE\-3 and DFlash; we do not evaluate follow\-ups that build on DFlash’s draft, such as draft trees\([Ringel and Romano, 2026](https://arxiv.org/html/2610.24365#bib.bib4)\), autoregressive postprocessing\([Cheng et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib9)\), or low\-rank latent mixtures\([Karimi et al\., 2026](https://arxiv.org/html/2610.24365#bib.bib10)\), althoughFlashMapis also compatible with draft trees \(Appendix[D](https://arxiv.org/html/2610.24365#A4)\)\. Our benchmarks cover English math, code, and chat at temperatures 0 and 1\. It is also possible that our models are undertrained; for instance,[Lee et al\. 2026](https://arxiv.org/html/2610.24365#bib.bib5)train their flow language model over 1M steps and distill over 100K steps, while we train and distill only for 39K and 23K steps, respectively, due to compute constraints\.

## Ethical Considerations

#### Risks\.

FlashMapis a drafter for speculative decoding, which produces a system that preserves the sampling distribution of the target model\. The risks of the target model, such as biased, harmful, or factually wrong outputs, therefore carry over\. As with any inference speedup, lowering the cost of generation also lowers the cost of misuse at scale\.

#### Artifacts and licenses\.

We use only publicly available models, data, and software, and cite their creators \(§[4](https://arxiv.org/html/2610.24365#S4)\)\. Qwen3\-4B and Qwen3\-8B and SGLang are released under the Apache 2\.0 license, Open\-PerfectBlend under Apache 2\.0, and DeepSpec under MIT\. The evaluation benchmarks are used for evaluation only, consistent with their intended use and licenses \(Appendix[G](https://arxiv.org/html/2610.24365#A7)\)\.

#### Data\.

We do not collect new data or use human annotators\. Open\-PerfectBlend is a public mixture of existing instruction\-tuning datasets, mostly in English, covering math, code, and chat\.

#### Use of AI assistants\.

We used AI assistants to help write experiment code and to edit the text of this paper\. All ideas, experiments, and results are the authors’ own, and the authors reviewed and verified all AI\-assisted code and text\.

## Acknowledgments

J\.S\. is supported by the Natural Sciences and Engineering Research Council of Canada \(NSERC\), funding reference number 587652\. We would like to acknowledge the support of the Simons Foundation, Schmidt Sciences, and Coefficient Giving\. This work was supported in part by the AI2050 program at Schmidt Sciences \(Grant G\-25\-70028\)\. Additionally, computations were run at facilities supported by the Scientific Computing Core at the Flatiron Institute\. The Flatiron Institute is a division of the Simons Foundation\.

## References

- Albergoet al\.\(2025\)M\. S\. Albergo, N\. M\. Boffi, and E\. Vanden\-EijndenStochastic interpolants: a unifying framework for flows and diffusions\.External Links:2303\.08797,[Link](https://arxiv.org/abs/2303.08797)Cited by:[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px4.p1.1)\.
- Austinet al\.\(2021\)J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le, and C\. SuttonProgram synthesis with large language models\.External Links:2108\.07732,[Link](https://arxiv.org/abs/2108.07732)Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px4.p1.1)\.
- Baiet al\.\(2024\)G\. Bai, J\. Liu, X\. Bu, Y\. He, J\. Liu, Z\. Zhou, Z\. Lin, W\. Su, T\. Ge, B\. Zheng, and W\. OuyangMT\-bench\-101: a fine\-grained benchmark for evaluating large language models in multi\-turn dialogues\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 7421–7454\.External Links:[Link](http://dx.doi.org/10.18653/v1/2024.acl-long.401),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.401)Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px4.p1.1)\.
- Boffiet al\.\(2025a\)N\. M\. Boffi, M\. S\. Albergo, and E\. Vanden\-EijndenFlow map matching with stochastic interpolants: a mathematical framework for consistency models\.External Links:2406\.07507,[Link](https://arxiv.org/abs/2406.07507)Cited by:[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px5.p1.1),[§3\.2](https://arxiv.org/html/2610.24365#S3.SS2.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2610.24365#S3.SS2.p2.1)\.
- Boffiet al\.\(2025b\)N\. M\. Boffi, M\. S\. Albergo, and E\. Vanden\-EijndenHow to build a consistency model: learning flow maps via self\-distillation\.External Links:2505\.18825,[Link](https://arxiv.org/abs/2505.18825)Cited by:[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px5.p1.1),[§3\.2](https://arxiv.org/html/2610.24365#S3.SS2.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2610.24365#S3.SS2.SSS0.Px1.p1.2),[§3\.2](https://arxiv.org/html/2610.24365#S3.SS2.p2.1)\.
- Chenet al\.\(2023\)C\. Chen, S\. Borgeaud, G\. Irving, J\. Lespiau, L\. Sifre, and J\. JumperAccelerating large language model decoding with speculative sampling\.External Links:2302\.01318,[Link](https://arxiv.org/abs/2302.01318)Cited by:[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px1.p1.1),[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2026\)J\. Chen, Y\. Liang, and Z\. LiuDFlash: Block Diffusion for Flash Speculative Decoding\.arXiv\.Note:arXiv:2602\.06036 \[cs\.CL\]External Links:[Link](http://arxiv.org/abs/2602.06036),[Document](https://dx.doi.org/10.48550/arXiv.2602.06036)Cited by:[§1](https://arxiv.org/html/2610.24365#S1.p2.1),[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px2.p1.1),[§6\.1](https://arxiv.org/html/2610.24365#S6.SS1.p1.1),[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. de Oliveira Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, B\. Chan, S\. Gray, N\. Ryder, M\. Pavlov, A\. Power, L\. Kaiser, M\. Bavarian, C\. Winter, P\. Tillet, F\. P\. Such, D\. Cummings, M\. Plappert, F\. Chantzis, E\. Barnes, A\. Herbert\-Voss, W\. H\. Guss, A\. Nichol, A\. Paino, N\. Tezak, J\. Tang, I\. Babuschkin, S\. Balaji, S\. Jain, W\. Saunders, C\. Hesse, A\. N\. Carr, J\. Leike, J\. Achiam, V\. Misra, E\. Morikawa, A\. Radford, M\. Knight, M\. Brundage, M\. Murati, K\. Mayer, P\. Welinder, B\. McGrew, D\. Amodei, S\. McCandlish, I\. Sutskever, and W\. ZarembaEvaluating large language models trained on code\.External Links:2107\.03374Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px4.p1.1)\.
- Chenget al\.\(2026\)X\. Cheng, X\. Yu, C\. Shao, J\. Li, Y\. Xiong, Y\. Qian, J\. Zhu, S\. Ma, X\. Zhang, J\. Ye, Q\. Chen, C\. Deng, J\. Yu, D\. Dai, Z\. Zhang, Y\. Wei, Y\. Tan, W\. Yang, R\. Xu, Y\. Wu, Z\. Xu, X\. Wang, M\. Chen, R\. Tian, X\. Bi, Z\. Hao, S\. Chen, H\. Cao, W\. Zhang, A\. Xu, H\. Zhang, D\. Zhao, and W\. LiangDSpark: Confidence\-Scheduled Speculative Decoding with Semi\-Autoregressive Generation\.arXiv\.Note:arXiv:2607\.05147 \[cs\.AI\]External Links:[Link](http://arxiv.org/abs/2607.05147),[Document](https://dx.doi.org/10.48550/arXiv.2607.05147)Cited by:[§1](https://arxiv.org/html/2610.24365#S1.p2.1),[§6\.1](https://arxiv.org/html/2610.24365#S6.SS1.p3.1),[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px2.p1.1),[Limitations](https://arxiv.org/html/2610.24365#Sx1.p1.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.External Links:2110\.14168,[Link](https://arxiv.org/abs/2110.14168)Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px4.p1.1)\.
- Guet al\.\(2018\)J\. Gu, J\. Bradbury, C\. Xiong, V\. O\. K\. Li, and R\. SocherNon\-autoregressive neural machine translation\.External Links:1711\.02281,[Link](https://arxiv.org/abs/1711.02281)Cited by:[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px3.p1.2)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the math dataset\.External Links:2103\.03874,[Link](https://arxiv.org/abs/2103.03874)Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px4.p1.1)\.
- Huanget al\.\(2026\)J\. Huang, Y\. Zhang, Q\. Zhang, H\. Lin, H\. Xu, and L\. ZhangDomino: decoupling causal modeling from autoregressive drafting in speculative decoding\.External Links:2605\.29707,[Link](https://arxiv.org/abs/2605.29707)Cited by:[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px2.p1.1)\.
- Jainet al\.\(2024\)N\. Jain, K\. Han, A\. Gu, W\. Li, F\. Yan, T\. Zhang, S\. Wang, A\. Solar\-Lezama, K\. Sen, and I\. StoicaLiveCodeBench: holistic and contamination free evaluation of large language models for code\.External Links:2403\.07974,[Link](https://arxiv.org/abs/2403.07974)Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px4.p1.1)\.
- Karimiet al\.\(2026\)A\. Karimi, C\. Gao, and N\. HassanpourDBLAST: Dependent Block Drafting for Stochastic Speculative Decoding\.\(en\)\.External Links:[Link](https://arxiv.org/abs/2608.05448v1)Cited by:[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px2.p1.1),[Limitations](https://arxiv.org/html/2610.24365#Sx1.p1.1)\.
- Leeet al\.\(2026\)C\. Lee, J\. Yoo, M\. Agarwal, S\. Shah, J\. Huang, A\. Raghunathan, S\. Hong, N\. M\. Boffi, and J\. KimFlow Map Language Models: One\-step Language Modeling via Continuous Denoising\.arXiv\.Note:arXiv:2602\.16813 \[cs\.CL\]External Links:[Link](http://arxiv.org/abs/2602.16813),[Document](https://dx.doi.org/10.48550/arXiv.2602.16813)Cited by:[§1](https://arxiv.org/html/2610.24365#S1.p3.1),[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px3.p1.2),[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px4.p1.1),[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px5.p1.2),[§3\.1](https://arxiv.org/html/2610.24365#S3.SS1.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2610.24365#S3.SS2.SSS0.Px1.p1.1),[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px3.p1.1),[Limitations](https://arxiv.org/html/2610.24365#Sx1.p1.1)\.
- Leviathanet al\.\(2023\)Y\. Leviathan, M\. Kalman, and Y\. MatiasFast inference from transformers via speculative decoding\.External Links:2211\.17192,[Link](https://arxiv.org/abs/2211.17192)Cited by:[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px1.p1.1),[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px1.p1.1)\.
- Liet al\.\(2026\)G\. Li, Z\. Fu, M\. Fang, Q\. Zhao, M\. Tang, C\. Yuan, and J\. WangDiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding\.InFindings of the Association for Computational Linguistics: ACL 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 20896–20910\.External Links:ISBN 979\-8\-89176\-395\-1,[Link](https://aclanthology.org/2026.findings-acl.1048/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.1048)Cited by:[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px2.p1.1)\.
- Li and He \(2026\)T\. Li and K\. HeBack to basics: let denoising generative models denoise\.External Links:2511\.13720,[Link](https://arxiv.org/abs/2511.13720)Cited by:[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px4.p1.2)\.
- Liet al\.\(2024\)T\. Li, W\. Chiang, E\. Frick, L\. Dunlap, T\. Wu, B\. Zhu, J\. E\. Gonzalez, and I\. StoicaFrom crowdsourced data to high\-quality benchmarks: arena\-hard and benchbuilder pipeline\.External Links:2406\.11939,[Link](https://arxiv.org/abs/2406.11939)Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px4.p1.1)\.
- Liet al\.\(2023\)X\. Li, T\. Zhang, Y\. Dubois, R\. Taori, I\. Gulrajani, C\. Guestrin, P\. Liang, and T\. B\. HashimotoAlpacaEval: an automatic evaluator of instruction\-following models\.GitHub\.Note:[https://github\.com/tatsu\-lab/alpaca\_eval](https://github.com/tatsu-lab/alpaca_eval)Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px4.p1.1)\.
- Liet al\.\(2025\)Y\. Li, F\. Wei, C\. Zhang, and H\. ZhangEAGLE\-3: Scaling up Inference Acceleration of Large Language Models via Training\-Time Test\.\(en\)\.External Links:[Link](https://arxiv.org/abs/2503.01840v1)Cited by:[§1](https://arxiv.org/html/2610.24365#S1.p1.1),[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px1.p1.1)\.
- Lipmanet al\.\(2023\)Y\. Lipman, R\. T\. Q\. Chen, H\. Ben\-Hamu, M\. Nickel, and M\. LeFlow matching for generative modeling\.External Links:2210\.02747,[Link](https://arxiv.org/abs/2210.02747)Cited by:[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px4.p1.1)\.
- Liuet al\.\(2022\)X\. Liu, C\. Gong, and Q\. LiuFlow straight and fast: learning to generate and transfer data with rectified flow\.External Links:2209\.03003,[Link](https://arxiv.org/abs/2209.03003)Cited by:[§2](https://arxiv.org/html/2610.24365#S2.SS0.SSS0.Px4.p1.1)\.
- Loshchilov and Hutter \(2019\)I\. Loshchilov and F\. HutterDecoupled weight decay regularization\.External Links:1711\.05101,[Link](https://arxiv.org/abs/1711.05101)Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px2.p1.1)\.
- Peebles and Xie \(2023\)W\. Peebles and S\. XieScalable diffusion models with transformers\.External Links:2212\.09748,[Link](https://arxiv.org/abs/2212.09748)Cited by:[§3\.1](https://arxiv.org/html/2610.24365#S3.SS1.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px1.p2.1)\.
- Potaptchiket al\.\(2026\)P\. Potaptchik, J\. Yim, A\. Saravanan, P\. Holderrieth, E\. Vanden\-Eijnden, and M\. S\. AlbergoDiscrete Flow Maps\.\(en\)\.External Links:[Link](https://arxiv.org/abs/2604.09784v2)Cited by:[§6\.3](https://arxiv.org/html/2610.24365#S6.SS3.p2.1),[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px3.p1.1)\.
- Ringel and Romano \(2026\)L\. Ringel and Y\. RomanoAccelerating Speculative Decoding with Block Diffusion Draft Trees\.arXiv\.Note:arXiv:2604\.12989 \[cs\.CL\]External Links:[Link](http://arxiv.org/abs/2604.12989),[Document](https://dx.doi.org/10.48550/arXiv.2604.12989)Cited by:[Appendix D](https://arxiv.org/html/2610.24365#A4.p1.1),[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px2.p1.1),[Limitations](https://arxiv.org/html/2610.24365#Sx1.p1.1)\.
- Rooset al\.\(2026\)D\. Roos, O\. Davis, F\. Eijkelboom, M\. Bronstein, M\. Welling, İ\. İ\. Ceylan, L\. Ambrogioni, and J\. van de MeentCategorical Flow Maps\.arXiv\(en\)\.Note:Version Number: 1External Links:[Link](https://arxiv.org/abs/2602.12233),[Document](https://dx.doi.org/10.48550/ARXIV.2602.12233)Cited by:[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px3.p1.1)\.
- Sandleret al\.\(2026\)J\. Sandler, J\. Christopher, T\. Hartvigsen, and F\. FiorettoSpecDiff\-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding\.Proceedings of Machine Learning and Systems8,pp\. 1128–1147\(en\)\.External Links:[Link](https://proceedings.mlsys.org/paper_files/paper/2026/hash/041dad5ed2191b44ba3ed0e00cdc3187-Abstract-Conference.html)Cited by:[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px2.p1.1)\.
- Tang and Wang \(2026\)S\. Tang and S\. WangDiscrete beckmann transport models for one\-step language modeling and reasoning\.External Links:2609\.15903,[Link](https://arxiv.org/abs/2609.15903)Cited by:[§6\.3](https://arxiv.org/html/2610.24365#S6.SS3.p2.1)\.
- Xuet al\.\(2024\)T\. Xu, E\. Helenowski, K\. A\. Sankararaman, D\. Jin, K\. Peng, E\. Han, S\. Nie, C\. Zhu, H\. Zhang, W\. Zhou, Z\. Zeng, Y\. He, K\. Mandyam, A\. Talabzadeh, M\. Khabsa, G\. Cohen, Y\. Tian, H\. Ma, S\. Wang, and H\. FangThe perfect blend: redefining rlhf with mixture of judges\.External Links:2409\.20370,[Link](https://arxiv.org/abs/2409.20370)Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px2.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. QiuQwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px1.p1.1)\.
- Zhang and Math\-AI \(2025\)Y\. Zhang and T\. Math\-AIAmerican invitational mathematics examination \(aime\) 2025\.Cited by:[§4](https://arxiv.org/html/2610.24365#S4.SS0.SSS0.Px4.p1.1)\.
- Zheng and Li \(2026\)H\. Zheng and P\. LiDeLS\-spec: decoupled long\-short contexts for parallel speculative drafting\.External Links:2607\.07409,[Link](https://arxiv.org/abs/2607.07409)Cited by:[§7](https://arxiv.org/html/2610.24365#S7.SS0.SSS0.Px2.p1.1)\.

## Appendix AInput featurization

A frozen input projection constrains how the state can enter the network\. The natural choice, the linear read\-outxk​Ex^\{k\}Eof the state at each slot, sums the embeddings of all tokens, each weighted by its noisy coordinate\. The clean token contributes only one term and every other token contributes noise, so the signal is drowned out\. For the flow model, we therefore embed the posterior over the clean token given the state, which provides a better signal\. Under a uniform prior, the posterior over classes given a Gaussian\-noised one\-hot is

πt​\(xk\)=softmax⁡\(t​xk\(1−t\)2\)\.\\pi\_\{t\}\(x^\{k\}\)=\\mathrm\{softmax\}\\left\(\\frac\{t\\,x^\{k\}\}\{\(1\-t\)^\{2\}\}\\right\)\.\(8\)The network inputhk=RMSNorm⁡\(πt​\(xk\)​E\)h^\{k\}=\\mathrm\{RMSNorm\}\(\\pi\_\{t\}\(x^\{k\}\)\\,E\)is then dominated by the embeddings of the few tokens consistent with the state\. We find, in small\-scale tests, that this improves the performance of the flow teacher model as compared to linear featurization\.

Ats=0s=0, however, this posterior inputπ0​\(xk\)\\pi\_\{0\}\(x^\{k\}\)is uniform for everyxkx^\{k\}\. For a flow map model that will be evaluated with 1 NFE \(i\.e\., for\(s,t\)=\(0,1\)\(s,t\)=\(0,1\)\), we require that the model has different random noise to anchor its joint prediction\. That is, in the case where we only feed in the posterior featurization to the flow map model and evaluate it at 1 NFE, the flow map model will learn the per\-slot marginals and revert back to a deterministic factorized drafter\. The student therefore also reads the raw state through a linear path,

hk=πs​\(xk\)​E\+\(t−s\)​xk​E,h^\{k\}=\\pi\_\{s\}\(x^\{k\}\)\\,E\+\(t\-s\)\\,x^\{k\}E,\(9\)where thet−st\-sterm ensures that the student matches the teacher on the diagonal, and each term is normalized with RMSNorm\.

## Appendix BFast multi\-step drafting

Drafting with more than one jump embeds a vocabulary\-sized state at every slot:x0​Ex\_\{0\}Eat the first jump, andπu​\(xu\)​E\\pi\_\{u\}\(x\_\{u\}\)\\,Eandxu​Ex\_\{u\}Eat the second, each a full pass overEE\. With two jumps at times0<u<10<u<1, the state entering the second jump isxu=\(1−u\)​x0\+u​δ^x\_\{u\}=\(1\-u\)\\,x\_\{0\}\+u\\,\\hat\{\\delta\}, whereδ^=δ0,uϕ​\(x0,c\)\\hat\{\\delta\}=\\delta^\{\\phi\}\_\{0,u\}\(x\_\{0\};c\), so its posterior isπu​\(xuk\)∝exp⁡\(α​x0k\+α2​δ^k\)\\pi\_\{u\}\(x\_\{u\}^\{k\}\)\\propto\\exp\(\\alpha x\_\{0\}^\{k\}\+\\alpha^\{2\}\\hat\{\\delta\}^\{k\}\)withα=u/\(1−u\)\\alpha=u/\(1\-u\)\. Sinceδ^k\\hat\{\\delta\}^\{k\}is concentrated on a few tokens, we split each token’s weight into a noise\-only part and a correction on the top tokensTkT^\{k\}ofδ^k\\hat\{\\delta\}^\{k\}:

πu​\(xuk\)​E≈W⁡\(x0k\)\+∑v∈Tkrv​EvS⁡\(x0k\)\+∑v∈Tkrv,W\(z\)=∑v∈𝒱eα​zvEv,S\(z\)=∑v∈𝒱eα​zv,\\begin\{gathered\}\\pi\_\{u\}\(x\_\{u\}^\{k\}\)\\,E\\approx\\frac\{W\(x\_\{0\}^\{k\}\)\+\\sum\_\{v\\in T^\{k\}\}r\_\{v\}E\_\{v\}\}\{S\(x\_\{0\}^\{k\}\)\+\\sum\_\{v\\in T^\{k\}\}r\_\{v\}\},\\\\ W\(z\)=\\sum\_\{v\\in\\mathcal\{V\}\}e^\{\\alpha z\_\{v\}\}E\_\{v\},\\qquad S\(z\)=\\sum\_\{v\\in\\mathcal\{V\}\}e^\{\\alpha z\_\{v\}\},\\end\{gathered\}\(10\)whererv=eα​x0,vk​\(eα2​δ^vk−1\)r\_\{v\}=e^\{\\alpha x^\{k\}\_\{0,v\}\}\\big\(e^\{\\alpha^\{2\}\\hat\{\\delta\}^\{k\}\_\{v\}\}\-1\\big\); likewisexuk​E≈\(1−u\)​x0k​E\+u​∑v∈Tkδ^vk​Evx\_\{u\}^\{k\}E\\approx\(1\-u\)\\,x\_\{0\}^\{k\}E\+u\\sum\_\{v\\in T^\{k\}\}\\hat\{\\delta\}^\{k\}\_\{v\}E\_\{v\}\. The parts over the whole vocabulary,x0​Ex\_\{0\}E,WW, andSS, depend only on the noise and onα\\alpha, which we can precompute for a bank of 8,192 noise draws\. At drafting time each slot selects a random bank entry, sox0​Ex\_\{0\}E,WW, andSSare already precomputed, and we only require computing the correction\. The correction needs the noise onTkT^\{k\}, which we redraw; this perturbs only\|Tk\|\|T^\{k\}\|of the\|𝒱\|\|\\mathcal\{V\}\|terms inWWandSS\. With\|Tk\|=16\|T^\{k\}\|=16, the second jump reads 16 rows ofEEper slot instead of all of them, reducing the added cost of the second jump over one\-jump drafting by about 70% on Qwen3\-8B, while acceptance stays essentially unchanged as compared to the two\-step draft without this optimization\.

## Appendix CSpeedup and acceptance length on Qwen3\-4B

Table[3](https://arxiv.org/html/2610.24365#A3.T3)gives speedup and acceptance length on Qwen3\-4B\. These results are analogous to the ones presented in Figure[1](https://arxiv.org/html/2610.24365#S0.F1)\(c\) and Table[1](https://arxiv.org/html/2610.24365#S3.T1)\.

Table 3:Speedup \(×\\times\) over autoregressive decoding with the target model \(top\) and acceptance lengthγ\\gamma\(bottom\) on Qwen3\-4B\. For each metric, temperature and benchmark, bold marks the best result and underline the second best\.
## Appendix DMulti\-Sample Drafting with Tree Verification

BecauseFlashMapdrafts from noise, it can draft several blocks from independent noise draws in a single batched forward pass, merge them into a trie, and verify the trie as a tree in one target pass\. This is similar to what is done in DDTree\([Ringel and Romano, 2026](https://arxiv.org/html/2610.24365#bib.bib4)\), but here every branch comes from a separate noise draw\. We ablateFlashMap’s performance at various sample sizes and tree verification sizes and report the results for Qwen3\-8B atT=0T=0in Table[4](https://arxiv.org/html/2610.24365#A4.T4)\. With four samples merged into a 16\-token tree—the same verification budget as a single sample—speedup rises on every benchmark, by 4–24% \(11% on average\) at one step and by 6–15% \(11% on average\) at two steps\.

Table 4:Multi\-sample drafting withFlashMapon Qwen3\-8B atT=0T=0: speedup \(×\\times\) over the target and acceptance lengthγ\\gammafor various numbers of steps, sample counts, and tree verification sizes\. For each metric and benchmark, bold marks the best result\. Calculated on one H100 SXM with 32 prompts per benchmark \(30 for AIME25\)\.
## Appendix EAdditional conditional acceptance results

We show conditional acceptance curves for Qwen3\-8B atT=1T=1in Figure[4](https://arxiv.org/html/2610.24365#A5.F4)\. This is the counterpart to theT=0T=0results in Figure[3](https://arxiv.org/html/2610.24365#S5.F3)\. Similarly, we show conditional acceptance curves for Qwen3\-4B at bothT=0T=0andT=1T=1in Figure[5](https://arxiv.org/html/2610.24365#A5.F5)\.

![Refer to caption](https://arxiv.org/html/2610.24365v1/figures/decay_8b_t1.png)Figure 4:Same as Figure[3](https://arxiv.org/html/2610.24365#S5.F3)but atT=1T=1\.![Refer to caption](https://arxiv.org/html/2610.24365v1/figures/cond_decay_4b_t0.0.png)

![Refer to caption](https://arxiv.org/html/2610.24365v1/figures/cond_decay_4b_t1.0.png)

Figure 5:Same as Figure[3](https://arxiv.org/html/2610.24365#S5.F3), but for Qwen3\-4B atT=0T=0\(top row\) andT=1T=1\(bottom row\)\.
## Appendix FShuffle test on Qwen3\-4B

Table[5](https://arxiv.org/html/2610.24365#A6.T5)shows the results of the shuffle test, as described in Section[6\.2](https://arxiv.org/html/2610.24365#S6.SS2), with Qwen3\-4B as the target model\. These results are analogous to the ones in Table[2](https://arxiv.org/html/2610.24365#S6.T2), and the conclusions are largely the same\.

Table 5:Same as Table[2](https://arxiv.org/html/2610.24365#S6.T2)but for Qwen3\-4B\.
## Appendix GImplementation and compute details

#### Model sizes\.

On Qwen3\-8B,FlashMaphas 500M trainable parameters, DFlash has 1049M and EAGLE\-3 has 302M\. On Qwen3\-4B,FlashMaphas 475M trainable parameters, DFlash has 537M, and EAGLE\-3 has 149M\.

#### Compute\.

We train on 4 NVIDIA H200 SXM5 nodes with 8 GPUs each \(total of 32 GPUs\)\. Training oneFlashMapflow teacher took∼25\{\\sim\}25hours for Qwen3\-8B and∼17\{\\sim\}17hours for Qwen3\-4B\. Distillation into a flow map took∼9\{\\sim\}9hours\. The retrained DFlash took∼27\{\\sim\}27hours for Qwen3\-8B and∼18\{\\sim\}18hours for Qwen3\-4B\. The retrained EAGLE\-3 took∼21\{\\sim\}21hours for Qwen3\-8B and∼16\{\\sim\}16hours for Qwen3\-4B\. Across all target models, drafters, and ablations, the total is about 4600 GPU hours\. Speedups are measured on one NVIDIA H200 NVL GPU \(one H100 SXM for Table[4](https://arxiv.org/html/2610.24365#A4.T4)\)\.

#### Software\.

We implement all drafters in DeepSpec 005e03b with PyTorch and evaluate with SGLang 0\.5\.20 at concurrency 1, with 2048 max new tokens and thinking mode off\.

#### Evaluation data\.

Licenses: GSM8K, MATH, LiveCodeBench and HumanEval \(MIT\); AIME25, MBPP, MT\-Bench, AlpacaEval, and Arena\-Hard \(Apache 2\.0\)\.

#### Variance\.

All speedups and acceptance lengths are from a single run per configuration\. AtT=0T=0, re\-running a configuration on a different GPU changed acceptance length by at most about 1%\. Conditional acceptance curves show 95% Wilson intervals\.