LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding

# LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding
Source: [https://arxiv.org/html/2608.20530](https://arxiv.org/html/2608.20530)
Matan RusanovskyYoav MironRoy UzielOmer BelhasinRan ZilbersteinMaor AshkenaziMichael Elad\[2pt\] NVIDIA

###### Abstract

Speculative decoding accelerates language\-model inference by drafting future tokens that the target model verifies in parallel\. A diffusion\-style block head such as DFlash is an attractive drafter, predicting an entire block of future tokens in a single forward pass\. However, it is trained on per\-position marginals rather than the joint block distribution, so the tokens it emits are individually plausible yet jointly incoherent\. We introduce LiLiCorr, aLightweightLikelihood\-based model thatCorrelates the per\-position marginal distributions such a drafter already produces\. It keeps the top\-KKtokens at each position as*candidates*and processes them jointly, producing for each an*in*and an*out*vector\. These vectors link candidates at consecutive positions: a pair matches when the earlier candidate’s*out*vector has high cosine similarity with the later candidate’s*in*vector\. These matches capture the block’s joint structure without ever materializing the full joint distribution, which is exponential in the block length\. Training makes the correct pairings score highest while pushing competing ones down, so coherent blocks outscore incoherent ones\. One lightweight network pass produces all the vectors, and the pairwise scores are then computed in parallel as batched matrix operations, leaving only a cheap greedy walk sequential\. We further co\-train the DFlash drafter with LiLiCorr, so it learns to propose candidates that correlate into longer accepted sequences\. Over the vanilla DFlash drafter, LiLiCorr raises acceptance length on every benchmark, by99to19%19\\%, while its single\-pass scoring head accounts for only about2\.8%2\.8\\%of the per\-block latency\. Against DFlash and two concurrently developed methods that also restore coherence at draft time, LiLiCorr delivers the highest throughput in7070of7272settings: nine benchmarks at two target sizes under greedy and temperature\-one decoding, and a throughput sweep over six concurrencies, two input lengths and three entropy tiers, with all systems equally optimized on a common serving stack\. Extending LiLiCorr to inputs an order of magnitude longer than it was trained on preserves that lead\.

††\*Correspondence to Matan Rusanovsky,mrusanovsky@nvidia\.com\.## 1 Introduction

Speculative decoding accelerates autoregressive language\-model inference by letting a small draft model propose several future tokens that the target model verifies in parallel, keeping the longest correct prefix\([8](https://arxiv.org/html/2608.20530#bib.bib1),[3](https://arxiv.org/html/2608.20530#bib.bib2)\)\. Because decoding is memory\-bandwidth bound, verifying a multitude of tokens together costs little more than generating just one token\. The resulting speedup depends on two quantities: the*acceptance length*, the expected number of draft tokens accepted per step, and the cost of the drafter itself\. A good drafter must raise the former without inflating the latter\. Much effort therefore goes into raising acceptance length, with designs ranging from feature\-autoregressive heads such as EAGLE\([10](https://arxiv.org/html/2608.20530#bib.bib4),[9](https://arxiv.org/html/2608.20530#bib.bib5),[11](https://arxiv.org/html/2608.20530#bib.bib6)\)to multi\-head predictors such as Medusa\([2](https://arxiv.org/html/2608.20530#bib.bib3)\)\. A particularly promising approach predicts an entire block of future tokens in a single forward pass via a parallel diffusion\-style drafter, such as DFlash\([4](https://arxiv.org/html/2608.20530#bib.bib7)\), offering a long speculative window at very low cost\.

In this work we build on DFlash, which predicts an entire block of future tokens in one forward pass\. For each position in the block, henceforth a*slot*, it produces a distribution over the full vocabulary\. Although DFlash attends bidirectionally across slots, so its representations can in principle correlate different positions, it is optimized with a per\-position cross\-entropy that supervises only the token*marginal*at each slot\. This leaves the*joint*distribution over the block largely unconstrained, so DFlash predicts each slot well yet can assemble the tokens into an incoherent block\.

This marginals problem is well known in the discrete diffusion literature, and its origin is combinatorial: the joint distribution over a block ofBBpositions, each drawn from a vocabulary of sizeVV, hasVBV^\{B\}possible realizations, so materializing it is intractable even for very short blocks\. Diffusion language models therefore never model the joint explicitly, and both diffusion and speculative\-decoding literature offer ways to recover coherence without it\. Masked diffusion models such as LLaDA, MDLM, and Dream\([13](https://arxiv.org/html/2608.20530#bib.bib14),[17](https://arxiv.org/html/2608.20530#bib.bib15),[21](https://arxiv.org/html/2608.20530#bib.bib16)\)correlate the sequence through iterative denoising, re\-masking a subset of the predicted tokens and re\-predicting them over several rounds\. In a speculative\-decoding setting, however, each such round is another full drafter pass, which is too expensive\. Tree\-based reranking, such as DDTree\([16](https://arxiv.org/html/2608.20530#bib.bib10)\), expands several candidate paths from the per\-slot marginals and lets the target model select among them\. This shifts the correlation work onto the target, whose passes speculative decoding is designed to save\. The trade is especially unfavorable in the compute\-bound, high\-concurrency regime typical of real\-world serving\. In contrast, Domino\([7](https://arxiv.org/html/2608.20530#bib.bib8)\)and DSpark\([5](https://arxiv.org/html/2608.20530#bib.bib9)\), two works developed concurrently with ours, act at draft time while avoiding extra target passes, but they do so through a sequence of auto\-regressive neural\-network passes, one on each draft slot\.

DFlash: one forward passmarginals over the full vocabularyslot 1slot 2slot 3argmax of each slotThedogmeowssweetly×\\times“The dog meows sweetly”top\-KKcandidates per slotdogcatbirdmeowsbarkssingssweetlyloudlysoftlyLiLiCorr: one bidirectional passinout0\.340\.810\.22Thedogcatbirdmeowsbarkssingssweetlyloudlysoftlyall links are scored in parallel✓\\checkmark“The cat meows softly”one network pass, all slots at onceargmax walk, no network weights\(a\)the marginal drafter is jointly wrong\(b\)one pass over all candidates\(c\)cheap greedy decoding

Figure 1:Overview of LiLiCorr\.\(a\)DFlash produces a distribution logit over the vocabulary at each slot in one forward pass, and taking the most probable token of each slot independently yields a sequence that is locally probable but jointly incoherent\.\(b\)LiLiCorr retains the top\-KKtokens of every slot and scores them jointly in a single pass, emitting an*in*and an*out*vector per candidate\.\(c\)Adjacent candidates are coupled by the alignment of the earlier*out*vector with the later*in*vector, so each slot pair reduces to one matrix product and the whole block is scored in parallel\. The block is then committed left to right by a greedy argmax operation that touches no network weights\.We introduce LiLiCorr, a lightweight likelihood\-based model that recovers a coherent sequence from these per\-slot marginals in a*single*network pass\. Figure[1](https://arxiv.org/html/2608.20530#S1.F1)sets out the method end to end\. LiLiCorr takes the top\-KKcandidate tokens at each slot and produces two vectors per candidate:*in*and*out*\. These capture how candidates chain together: two adjacent candidates are compatible when the*out*vector of the first aligns, in cosine similarity, with the*in*vector of the second\. Training aligns the*out*and*in*vectors of consecutive ground\-truth tokens and separates the competing pairings\. At inference, a cheap greedy walk commits the block left to right, at each position taking the candidate that best matches its committed predecessor\. The central novelty is that this correlation logic reduces to*matrix operations that are easy to optimize*\. All candidate interactions follow from one forward pass and are computed in parallel\. Only the greedy path selection remains sequential, unlike prior draft\-time approaches, which run the correlation network once per slot\.

LiLiCorr is trained jointly with DFlash rather than on top of a frozen drafter, so the drafter learns to propose candidate sets that LiLiCorr can turn into longer accepted sequences\. Against a vanilla DFlash drafter trained on the same data, this raises acceptance length by99to19%19\\%on every benchmark, for 8B and 4B target sizes and both greedy and temperature\-11decoding\. We further compare against the two concurrent works above, Domino and DSpark, which likewise tune the drafter alongside their head, giving every system the same optimization effort on a common serving stack\. LiLiCorr achieves longer accepted lengths than Domino on 83% of the benchmarks and on the benchmark average in every setting, and it turns its drafts into tokens more efficiently than either baseline: it serves faster than both in all thirty\-six settings, leading the stronger of the two by as much as5\.3%5\.3\\%on the benchmark average\. The reason is structural: both baselines spend a network pass on every draft slot, whereas LiLiCorr scores the entire candidate lattice at once\. This results in a head roughly2\.4×2\.4\\timescheaper than Domino’s once both are fully optimized \(Table[3](https://arxiv.org/html/2608.20530#A1.T3)\)\. The lead persists as serving concurrency rises, and it carries to inputs an order of magnitude longer than the drafter was trained on, once the positional encoding is extended at inference\.

##### Contributions\.

1. \(1\)A novel likelihood\-based formulation of the marginals problem\.We cast the recovery of a coherent sequence from per\-slot marginals as a learned scoring model over candidate paths\. Rather than reason in the full vocabulary, the model attends over all candidates of all slots by assigning two low\-dimensional vectors to each, which encode how neighbors are matched\.
2. \(2\)Correlation as efficient parallel matrix operations\.At decoding time, a single network pass precomputes all of the correlations, as the pairwise candidate interactions reduce to matrix operations that are easy to optimize\. This is followed by a cheap greedy walk that remains sequential\. Prior draft\-time approaches instead run the correlation network once per slot\.
3. \(3\)State\-of\-the\-art results\.LiLiCorr sets a new state\-of\-the\-art for draft\-time correlation in speculative decoding, trained jointly with the drafter and given the same data, training budget, and optimization effort as the two concurrent works\([7](https://arxiv.org/html/2608.20530#bib.bib8),[5](https://arxiv.org/html/2608.20530#bib.bib9)\)\. We evaluate on nine benchmarks, at two target sizes and under both greedy and temperature\-one decoding, and on a throughput sweep over six serving concurrencies, two input lengths, and three output\-entropy tiers\. LiLiCorr holds the highest throughput of any system compared in7070of these7272settings, and the lead survives inputs longer than the drafter was trained on\.

## 2 Related Work

##### Speculative decoding and parallel drafters\.

Speculative decoding accelerates inference by proposing draft tokens and verifying them with the target model under a rule that preserves its distribution\([8](https://arxiv.org/html/2608.20530#bib.bib1),[3](https://arxiv.org/html/2608.20530#bib.bib2)\)\. Most work along these lines improves the drafter, from multi\-head predictors such as Medusa\([2](https://arxiv.org/html/2608.20530#bib.bib3)\)to feature\-autoregressive heads such as the EAGLE family\([10](https://arxiv.org/html/2608.20530#bib.bib4),[9](https://arxiv.org/html/2608.20530#bib.bib5),[11](https://arxiv.org/html/2608.20530#bib.bib6)\)\. A separate line of work drafts an entire block in one forward pass: parallel and diffusion\-style block heads such as DFlash\([4](https://arxiv.org/html/2608.20530#bib.bib7)\)and diffusion\-based self\-speculation\([6](https://arxiv.org/html/2608.20530#bib.bib13)\)handle a long speculative window cheaply\. D\-PACE\([19](https://arxiv.org/html/2608.20530#bib.bib11)\)improves DFlash with an acceptance\-aware training objective\. Their parallel prediction mirrors that of diffusion language models, such as LLaDA\([13](https://arxiv.org/html/2608.20530#bib.bib14)\), MDLM\([17](https://arxiv.org/html/2608.20530#bib.bib15)\), Dream\([21](https://arxiv.org/html/2608.20530#bib.bib16)\), and Nemotron\-Labs\-Diffusion\([6](https://arxiv.org/html/2608.20530#bib.bib13)\)\. These models exhibit a well\-known marginals limitation: although attention is bidirectional, the block is factorized into independent per\-position predictions, so the sampled tokens are not guaranteed to be jointly coherent\. This is addressed via an iterative refinement, re\-masking and re\-predicting subsets of tokens so that later rounds condition on earlier ones\. Parallel drafters inherit the same limitation, but in speculative decoding each refinement round adds a full drafter pass, which negates the latency gains that motivate parallel drafting\. Recovering a coherent block from these marginals, without paying for repeated drafter passes, is the problem LiLiCorr addresses\.

##### Recovering coherent drafts\.

Several recent methods recover a coherent sequence from the per\-slot marginals that a parallel drafter such as DFlash produces\. One direction expands these marginals into a draft tree of several continuations that the target verifies in a single pass\. DDTree\([16](https://arxiv.org/html/2608.20530#bib.bib10)\)is training\-free, ranking candidate prefixes by their marginal product, whereas[14](https://arxiv.org/html/2608.20530#bib.bib12)first learn a small autoregressive adapter before building the tree\. Both push the correlation work onto the target, spending the compute speculative decoding aims to save, and both rely on tree\-structured attention, which is typically less efficient than standard causal attention\. An alternative direction reintroduces the dependencies at draft time: Domino\([7](https://arxiv.org/html/2608.20530#bib.bib8)\)and DSpark\([5](https://arxiv.org/html/2608.20530#bib.bib9)\), the two works concurrent with ours, condition each slot on the previously sampled tokens, which avoids extra target passes but runs a correction network once at every slot\. Both also tune the drafter jointly with their correction head, as we do\. LiLiCorr also acts at draft time, but needs only a single network pass: it produces all candidate representations at once, computes their pairwise correlations in parallel, and leaves only a cheap greedy argmax over the resulting scores to run sequentially\.

## 3 Method

### 3\.1 Setup: prefix acceptance and the marginal drafter

LetMMbe a frozen target language model\. A parallel block drafter proposes, in a single forward pass, a block ofBBpositions conditioned on the verified context, of whichB−1B\-1are future tokens\. The first block position is the last token the target has committed, which serves as the*anchor*, slot00\. The remainingB−1B\-1positions are the speculative*slots*1,…,B−11,\\dots,B\-1\. At each slot the drafter produces a distribution \(logit\) over the vocabulary, a per\-slot*marginal*\. Standard decoding resolves each slot independently, taking its most probable token\. Once the draft is complete, the target verifies the whole block in a single pass and accepts a*prefix*of it, discarding everything from the first rejected position onward\. With a greedy target a draft token is accepted when it coincides with the target’s own choice\. Under a sampling target, acceptance instead follows the standard rejection test, which preserves the target distribution exactly\([8](https://arxiv.org/html/2608.20530#bib.bib1),[3](https://arxiv.org/html/2608.20530#bib.bib2)\)\. In both regimes acceptance is prefix\-structured, and it is this structure that LiLiCorr is built around\.

### 3\.2 A scoring model over the candidate lattice

The drafter is supervised on per\-slot marginals, so the tokens it selects are individually probable but not necessarily compatible with one another\. Selecting tokens that are jointly compatible, and not only locally probable, can lengthen the accepted prefix, and doing so at draft time could avoid extra target passes\.

LiLiCorr addresses this goal by correlating the slots over a small pool of alternatives that the drafter’s own marginal already places near the top\. At each slotssit retains the drafter’s top\-KKcandidate tokenscs,1,…,cs,Kc\_\{s,1\},\\dots,c\_\{s,K\}rather than its single most probable one\. Each candidate is a token, and we represent it by a vector we call its*node*, formed by summing four terms of equal width: the target’s frozen input embedding of the token, the drafter’s hidden state at that slot, a small network over a few scalar features of how confident the drafter is in that candidate, listed in Appendix[A](https://arxiv.org/html/2608.20530#A1), and learned encodings of the slot and of the candidate’s rank\. The hidden state belongs to the slot and is shared by itsKKcandidates, so what separates them is the token itself, its confidence, and its rank\. Across theB−1B\-1slots these nodes form a\(B−1\)×K\(B\\\!\-\\\!1\)\\times Klattice\.

A small Transformer processes the\(B−1\)×K\(B\\\!\-\\\!1\)\\times Knodes jointly\. Its self\-attention lets every candidate exchange information with every other, both the competing candidates within a slot and the candidates at neighboring slots, so each is represented in the context of the whole block rather than on its own\. Each attended node is then fused with a projection of the target’s hidden state that produced the last verified token: the candidate’s own state, that projection, and their elementwise product are concatenated and mapped back to the node width by a single layer\. Though one vector common to the whole block, it enters every candidate separately, so that the verifier’s view of the context, and not the drafter’s alone, shapes each of them\. For each node LiLiCorr emits two vectors inℝde\\mathbb\{R\}^\{d\_\{e\}\}, an*out*vector𝐨\\mathbf\{o\}and an*in*vector𝐢\\mathbf\{i\}, each normalized to unit norm\. The dimensionded\_\{e\}is given in Appendix[A](https://arxiv.org/html/2608.20530#A1)\. The anchor, which precedes the first slot, is encoded far more thinly: the anchor projection itself, mapped by its own head to one further out vector\. Indexing the anchor as slot00, we write this vector𝐨0\\mathbf\{o\}\_\{0\}\. Adjacent slotsssands\+1s\+1are then coupled by a single*matching*term, the cosine similarity between the out and in vectors:

ϕp​\(s,k,k′\)=σ​cos⁡\(𝐨s,k,𝐢s\+1,k′\)for​s=0,…,B−2,1≤k≤Ks,1≤k′≤K,\\phi^\{\\mathrm\{p\}\}\(s,k,k^\{\\prime\}\)=\\sigma\\,\\cossim\(\\mathbf\{o\}\_\{s,k\},\\,\\mathbf\{i\}\_\{s\+1,k^\{\\prime\}\}\)~~~~\\mbox\{for\}~~~s=0,\\dots,B\-2,~~~1\\leq k\\leq K\_\{s\},~1\\leq k^\{\\prime\}\\leq K,\(1\)wherecos⁡\(⋅,⋅\)\\cossim\(\\cdot,\\cdot\)is the cosine similarity andσ\\sigmais a fixed scaling constant, withK0=1K\_\{0\}\\\!=\\\!1at the anchor, which carries a single out vector, andKs=KK\_\{s\}\\\!=\\\!Kat every speculative slot\. Figure[1](https://arxiv.org/html/2608.20530#S1.F1)\(b\) shows the lattice and the pair of vectors each candidate carries\.

The factorization is local, but the information behind it is not\. Only adjacent slots are coupled by a score, yet the vectors that form that score depend on the entire lattice, so a pair at slotsssands\+1s\\\!\+\\\!1may be judged compatible on evidence lying outside them, among the candidates that precedessor follows\+1s\\\!\+\\\!1\. We hypothesize that this is the more useful place to spend capacity: rather than enrich the coupling itself, we keep it pairwise and let the vectors it multiplies see the whole block\. The pairwise form is what leaves the scores computable as batched matrix products\.

Casting each score as a cosine is a deliberate stabilizer: it removes vector magnitude as an unconstrained degree of freedom\. Without this bound we observe runaway score magnitudes that corrupt, and in practice destroy, the shared DFlash backbone under joint training\. The cosine instead confines every learned factor to\[−σ,σ\]\[\-\\sigma,\\sigma\]while avoiding saturating nonlinearities whose gradients vanish\. We use a fixed scaleσ\\sigmato separate the candidate scores to discriminate among them\. Its value, and the architecture of the head, are given in Appendix[A](https://arxiv.org/html/2608.20530#A1)\.

### 3\.3 Training the reranker

##### Local normalization aligned with prefix acceptance\.

A natural formulation scores each full path by summing its matching terms across the block and selects the path that maximizes this whole\-chain score\. We argue that this is the wrong target for speculative decoding\. The global optimum may sacrifice an early slot to win a richer tail, yet under prefix acceptance that tail is never reached\. Section[3\.5](https://arxiv.org/html/2608.20530#S3.SS5)and Appendix[E](https://arxiv.org/html/2608.20530#A5)take up this comparison at decoding time\. Optimizing the whole chain thus spends capacity on regions the metric never credits\. We therefore score each slot*locally*, conditioned on the candidate committed just before it\. Writing the per\-slot*node score*of candidatekkaszs​\[k\]z\_\{s\}\[k\], the chain rule factorizes the path distribution as

P\(cs∣cs−1\)=softmaxk\(zs\[k\]\),zs\[k\]=ϕp\(s−1,cs−1,k\),s=1,…,B−1,P\(c\_\{s\}\\mid c\_\{s\-1\}\)=\\softmax\_\{k\}\\big\(z\_\{s\}\[k\]\\big\),\\qquad z\_\{s\}\[k\]=\\phi^\{\\mathrm\{p\}\}\(s\\\!\-\\\!1,\\,c\_\{s\-1\},\\,k\),\\qquad s=1,\\dots,B\-1,\(2\)wherecsc\_\{s\}is the candidate committed at slotssandc0c\_\{0\}is the anchor\. Each conditional still depends on the previous pick through the matching term inzs​\[k\]z\_\{s\}\[k\], so locality avoids scoring the whole chain without reducing the model to a per\-slot reranker\.

##### Oracle\-prefix supervision\.

We train LiLiCorr as a teacher\-forced reranker over the drafter’s marginals, so the supervision must reflect how it is served\. Two conditions decide which slots carry a learning signal\. The first is coverage: a slot is useful only when its ground\-truth token is among the drafter’sKKcandidates, since otherwise no reranking can recover it\. The second follows from prefix acceptance\. We condition each transition on the ground\-truth predecessorgs−1g\_\{s\-1\}: a path that has already left the ground truth is discarded by the verifier, so the transitions leaving it are never exercised in serving\. A slot is therefore supervised only when every earlier slot is covered as well\. Together these define the*oracle prefix*, the longest leading run of slots that each contain the ground\-truth token among theirKKcandidates\. Let𝒮\\mathcal\{S\}collect these slots across a training batch\. We teacher\-force the per\-slot cross\-entropy on them:

ℒCE=−1\|𝒮\|∑s∈𝒮logP\(cs=gs∣cs−1=gs−1\),\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\\;=\\;\-\\frac\{1\}\{\|\\mathcal\{S\}\|\}\\\!\\\!\\sum\_\{s\\in\\mathcal\{S\}\}\\log P\\big\(c\_\{s\}=g\_\{s\}\\mid c\_\{s\-1\}=g\_\{s\-1\}\\big\),\(3\)wheregsg\_\{s\}is the ground\-truth token at slotss\. The softmax of Eq\. \([2](https://arxiv.org/html/2608.20530#S3.E2)\) runs over allKKcandidates at slotss, so each supervised transition raises the ground\-truth successor and pushes itsK−1K\\\!\-\\\!1competitors down\. A block whose very first slot already misses contributes zero loss\. The effect resembles the per\-slot decay used to train DFlash, though it arises from the coverage mask rather than from a designed weighting: supervision concentrates on earlier slots\.

##### Target\-weighted distractor penalty\.

The per\-slot cross\-entropy raises the ground\-truth probability but is indifferent to how the residual mass is distributed among the remaining candidates\. Under cross\-entropy supervision alone, we observed that LiLiCorr’s errors were typically candidates the target scored well\-below the ground truth, a mismatch with the verifier rather than a genuine ambiguity in the target distribution\. Since the target is itself the verifier, its logits provide a graded measure of candidate compatibility that cross\-entropy ignores\. Scoring against the target lets us direct the negative gradient at these high\-scoring but target\-incompatible competitors, widening the ground truth’s margin over the candidates most likely to trigger an early verification failure\.

Letqkq\_\{k\}denote the target probability of candidatekk, andgsg\_\{s\}the ground\-truth candidate at slotss\. Bothqgsq\_\{g\_\{s\}\}and theqkq\_\{k\}are indexed out of the teacher\-forced target pass that already supplies the drafter’s supervision\. We define

Ωs=∑kPθ​\(cs=k∣cs−1=gs−1\)​ReLU⁡\(log⁡qgs−log⁡qk\),\\Omega\_\{s\}=\\sum\_\{k\}P\_\{\\theta\}\\\!\\left\(c\_\{s\}=k\\mid c\_\{s\-1\}=g\_\{s\-1\}\\right\)\\relu\\\!\\left\(\\log q\_\{g\_\{s\}\}\-\\log q\_\{k\}\\right\),\(4\)the expected target\-relative cost under the LiLiCorr distribution\. Candidates the target ranks belowgsg\_\{s\}are penalized in proportion to the gap\. Whengsg\_\{s\}is the target’s top candidate, this term shares its gradient with a target cross\-entropy over the candidate pool, and the rectifier extends it safely to the rare positions wheregsg\_\{s\}is not the target’s top candidate, as detailed in Appendix[D](https://arxiv.org/html/2608.20530#A4)\.

Averaging the per\-slot penalties over the supervised slots gives the distractor loss

ℒdist=1\|𝒮\|​∑s∈𝒮Ωs\.\\mathcal\{L\}\_\{\\mathrm\{dist\}\}=\\frac\{1\}\{\|\\mathcal\{S\}\|\}\\sum\_\{s\\in\\mathcal\{S\}\}\\Omega\_\{s\}\.\(5\)We keep it alongside the hard cross\-entropy because the two supply complementary gradients\. Cross\-entropy makes LiLiCorr rank the ground truth first even when the target assigns it only modest probability, so it remains the selected token however soft the target is\. The distractor penalty complements this by pushing the confusers the target disfavors further down\. It is therefore cost\-aware shaping of cross\-entropy, not a replacement, as detailed in Appendix[D](https://arxiv.org/html/2608.20530#A4)\. It is the term that carries the short\-prompt benchmarks, raising acceptance length on every one of them \(Appendix[C](https://arxiv.org/html/2608.20530#A3)\)\.

### 3\.4 Co\-adapting the drafter and the reranker

We tune DFlash jointly with LiLiCorr, so that the drafter learns to expose candidate pools and hidden representations from which LiLiCorr can assemble longer accepted prefixes\. Joint training must cross the top\-KKselection that forms the candidate pool, which is not differentiable\. We keep the selected candidate indices fixed and retain gradients through their draft log\-probabilities and the hidden states from which they were produced,111The candidate token embeddings remain frozen\.both of which are inputs to LiLiCorr\. The acceptance\-oriented loss therefore reaches the DFlash backbone without differentiating pool membership, and inference is unchanged\.

The drafter keeps its own DFlash training loss, a per\-slot block cross\-entropy over the drafter marginals that weights earlier slots more heavily\([4](https://arxiv.org/html/2608.20530#bib.bib7)\),

ℒDFlash=−∑s=1B−1wslogps\(gs\),ws=exp\(−s−1β\),\\mathcal\{L\}\_\{\\text\{DFlash \}\}=\-\\sum\_\{s=1\}^\{B\-1\}w\_\{s\}\\,\\log p\_\{s\}\(g\_\{s\}\),\\qquad w\_\{s\}=\\exp\\\!\\left\(\-\\frac\{s\-1\}\{\\beta\}\\right\),\(6\)wherepsp\_\{s\}is the drafter’s marginal at slotssandβ\\betasets the decay\. The drafter and LiLiCorr are trained jointly to minimize the drafter loss together with the two reranker terms,

ℒ=ℒDFlash\+γ⁡\[ℒCE\+λ​ℒdist\]\.\\mathcal\{L\}\\;=\\;\\mathcal\{L\}\_\{\\text\{DFlash \}\}\\;\+\\;\\gamma\\left\[\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\\;\+\\;\\lambda\\,\\mathcal\{L\}\_\{\\mathrm\{dist\}\}\\right\]\.\(7\)Hereγ\\gammasets the weight of the reranker objective as a whole andλ\\lambdascales the distractor penalty within it\. These weights and the decayβ\\betaare given in Appendix[A](https://arxiv.org/html/2608.20530#A1), and Appendix[C](https://arxiv.org/html/2608.20530#A3)isolates the contribution of the distractor penalty\.

### 3\.5 Single\-pass decoding

Decoding has a parallel\- and a sequential stage, and the network runs only once\. In the parallel stage, a single LiLiCorr pass produces every outgoing and incoming vector, including the anchor out vector\. The block hasB−1B\-1adjacent slot pairs, and within each pair every candidate is matched against every candidate in the next slot\. This givesK2K^\{2\}comparisons for each of theB−2B\\\!\-\\\!2interior pairs andKKfor the anchor pair,\(B−2\)​K2\+K\(B\\\!\-\\\!2\)K^\{2\}\+Kmatching terms in total\. Evaluating them reduces to one matrix multiplication per slot pair, batched across the full block, which keeps the computation static in shape and well suited to kernel fusion and graph capture\. In the sequential stage, the block is committed greedily from left to right: at slotsswe select

c^s=arg​maxk⁡zs​\[k\]=arg​maxk⁡ϕp​\(s−1,c^s−1,k\),\\hat\{c\}\_\{s\}\\;=\\;\\argmax\_\{k\}\\,z\_\{s\}\[k\]\\;=\\;\\argmax\_\{k\}\\,\\phi^\{\\mathrm\{p\}\}\(s\\\!\-\\\!1,\\,\\hat\{c\}\_\{s\-1\},\\,k\),\(8\)conditioning each slot on the candidate just committed, a walk traced in Figure[1](https://arxiv.org/html/2608.20530#S1.F1)\(c\)\. This greedy rule is exactly the inference behavior the local objective in Eq\. \([2](https://arxiv.org/html/2608.20530#S3.E2)\) was trained to optimize\. An alternative is to commit the combination of candidates that maximizes the sum of the matching terms along the whole path\. Appendix[E](https://arxiv.org/html/2608.20530#A5)evaluates an efficient dynamic program that recovers this globally best\-scoring path exactly, and finds that it decodes worse than the greedy left\-to\-right rule in the speculative\-decoding setting\. Only the argmax walk is sequential, and it touches no network weights; the expensive computation is the single parallel pass\. This is the key distinction from iterative\-refinement drafters, which invoke a network once per position: LiLiCorr confines all learning to one pass and reduces the per\-position work to an argmax over precomputed matching scores\.

## 4 Experiments

### 4\.1 Experimental setup

##### Models and evaluation\.

We use frozen Qwen3\-8B and Qwen3\-4B targets\([20](https://arxiv.org/html/2608.20530#bib.bib17)\)with their reasoning mode disabled\. Evaluation spans eight public datasets across three domains: math \(GSM8K, MATH\-500, AIME 2025\), code \(HumanEval, MBPP, LiveCodeBench\), and chat \(Alpaca, MT\-Bench\)\. We additionally report on SPEED\-Bench\([1](https://arxiv.org/html/2608.20530#bib.bib19)\), hereafter SPEED, whose qualitative split holds 880 prompts across eleven categories, from math and coding to multilingual text, summarization, and roleplay\. We evaluate two decoding regimes: a deterministic one with a greedy target, and a sampling one at temperature11over the untruncated target distribution, verified by exact rejection sampling so that the target distribution is preserved and averaged over three seeds\. We measure the acceptance lengthτ\\tau, computed block\-weighted as the total number of accepted tokens divided by the number of verification blocks, together with output throughput and the resulting speedup over the target’s own autoregressive decoding on the same hardware\.

##### Baselines\.

We compare LiLiCorr against two concurrent works to ours, Domino\([7](https://arxiv.org/html/2608.20530#bib.bib8)\)and DSpark\([5](https://arxiv.org/html/2608.20530#bib.bib9)\), and against the DFlash drafter it builds on\. Both baselines likewise recover a coherent block from the per\-slot marginals at draft time, but resolve them through a sequence of per\-slot network passes rather than a single scoring pass\. Like LiLiCorr, both also tune the DFlash drafter as they train their own head\. The DFlash drafter takes each slot’s most probable token independently and performs no reranking\. At temperature11, LiLiCorr and Domino propose a single block deterministically and leave exactness to the verifier, whereas DSpark supports that contract and a sampled one; we report it under both\. All systems are trained on the same corpus for the same number of epochs, at the same global batch size and sequence length, then evaluated under one regime, so acceptance length and throughput are directly comparable\.

##### Training data\.

All three systems are trained on roughly1\.41\.4M examples drawn from the code, mathematics, STEM, and general\-chat splits of the Nemotron Post\-Training Dataset V2\([12](https://arxiv.org/html/2608.20530#bib.bib18)\), with the multilingual splits excluded\. For each conversation we keep the original system and user messages and regenerate every assistant turn with the target model, decoding greedily with reasoning disabled and a cap of4,0964\{,\}096completion tokens\.

##### Implementation details\.

We serve every system on the same deployment\-representative SGLang backend, with block size 16 on a single H100 and concurrency one unless stated otherwise\. Following DSpark, each speculative head is folded into the drafter and optimized with fused kernels and graph capture, and we port Domino onto the same stack, so that no system is disadvantaged by its implementation\. Fully optimized, the LiLiCorr head costs0\.280\.28ms per block against Domino’s0\.670\.67ms,2\.4×2\.4\\timescheaper, and accounts for about2\.8%2\.8\\%of the per\-block wall\.222DSpark folds its head into the drafter by construction and so leaves no separate stage to measure; it is compared on end\-to\-end throughput throughout\.Remaining setup and training details, and the per\-stage decomposition behind these figures, are deferred to Appendix[A](https://arxiv.org/html/2608.20530#A1)\.

### 4\.2 Main results

Table[1](https://arxiv.org/html/2608.20530#S4.T1)reports speedup over autoregressive decoding on the nine benchmarks, for both target sizes and both decoding regimes; the acceptance lengths and absolute throughputs behind it are deferred to Appendix[B](https://arxiv.org/html/2608.20530#A2)\. The three draft\-time methods reach comparable acceptance lengths, and throughput is what separates them\. Under greedy decoding LiLiCorr is the fastest system in all eighteen settings, leading the strongest baseline by up to4\.9%4\.9\\%on the benchmark average\. Against the vanilla DFlash drafter, which reranks nothing, it improves on both axes everywhere, raising acceptance length by1212to19%19\\%and throughput by44to13%13\\%\. Tables[7](https://arxiv.org/html/2608.20530#A2.T7)and[8](https://arxiv.org/html/2608.20530#A2.T8)resolve SPEED into its eleven categories\. LiLiCorr leads on every one, and by the widest margin on multilingual, by11\.0%11\.0\\%on Qwen3\-8B and16\.8%16\.8\\%on Qwen3\-4B\. This is the only domain held out of the training corpus, which suggests a capacity to generalize out of distribution\.

Sampling does not reorder the systems\. At temperature one, verified by exact rejection sampling, accepted prefixes shorten for all of them, as expected once the target no longer commits to its own argmax, and LiLiCorr’s lead over Domino widens\. DSpark’s sampled arm accepts longer prefixes than its single\-block one but serves slower on both targets\. LiLiCorr is the fastest system in seventeen of the eighteen settings, and the fastest of those proposing a single block in all eighteen\.

Table 1:Speedup over autoregressive decoding on the nine benchmarks, both target sizes, single H100, concurrency one\.T=1T\\\!=\\\!1entries are means over three seeds, and DSpark appears there under both of its proposal contracts\. Average is the unweighted mean over the nine benchmarks\. Best per row and target size in bold; acceptance lengths in Tables[5](https://arxiv.org/html/2608.20530#A2.T5)and[6](https://arxiv.org/html/2608.20530#A2.T6)\.AR \(TPS\)LiLiCorrDominoDSparkDSpark \(samp\.\)Vanilla DFlashBenchmark8B4B8B4B8B4B8B4B8B4B8B4BGreedy,T=0T\\\!=\\\!0GSM8K1502355\.30×\\times4\.95×\\times4\.97×\\times4\.62×\\times5\.07×\\times4\.58×\\times––4\.70×\\times4\.49×\\timesMATH\-5001502306\.59×\\times6\.16×\\times6\.35×\\times5\.87×\\times6\.31×\\times5\.72×\\times––6\.00×\\times5\.73×\\timesAIME 20251502335\.98×\\times5\.86×\\times5\.77×\\times5\.64×\\times5\.69×\\times5\.39×\\times––5\.43×\\times5\.48×\\timesHumanEval1462274\.15×\\times3\.95×\\times3\.95×\\times3\.79×\\times4\.06×\\times3\.80×\\times––3\.88×\\times3\.80×\\timesMBPP1502404\.15×\\times3\.82×\\times4\.00×\\times3\.66×\\times4\.11×\\times3\.68×\\times––3\.77×\\times3\.58×\\timesLiveCodeBench1492325\.38×\\times5\.20×\\times5\.20×\\times4\.94×\\times5\.34×\\times4\.92×\\times––5\.02×\\times4\.95×\\timesAlpacaEval1502402\.69×\\times2\.55×\\times2\.58×\\times2\.45×\\times2\.61×\\times2\.42×\\times––2\.46×\\times2\.40×\\timesMT\-Bench1502392\.91×\\times2\.77×\\times2\.82×\\times2\.67×\\times2\.85×\\times2\.63×\\times––2\.65×\\times2\.61×\\timesSPEED1492323\.58×\\times3\.50×\\times3\.42×\\times3\.29×\\times3\.44×\\times3\.25×\\times––3\.27×\\times3\.24×\\timesAverage1502344\.51×\\times4\.29×\\times4\.32×\\times4\.09×\\times4\.37×\\times4\.03×\\times––4\.11×\\times4\.02×\\timesSampling,T=1T\\\!=\\\!1GSM8K1482294\.57×\\times4\.43×\\times4\.32×\\times4\.14×\\times4\.43×\\times4\.14×\\times4\.12×\\times3\.60×\\times4\.16×\\times4\.08×\\timesMATH\-5001482294\.55×\\times4\.45×\\times4\.42×\\times4\.26×\\times4\.45×\\times4\.20×\\times4\.55×\\times4\.03×\\times4\.32×\\times4\.25×\\timesAIME 20251472293\.61×\\times3\.48×\\times3\.37×\\times3\.21×\\times3\.45×\\times3\.25×\\times3\.80×\\times3\.36×\\times3\.39×\\times3\.32×\\timesHumanEval1442224\.11×\\times4\.00×\\times3\.92×\\times3\.82×\\times4\.04×\\times3\.83×\\times3\.49×\\times3\.18×\\times3\.83×\\times3\.82×\\timesMBPP1482313\.57×\\times3\.50×\\times3\.41×\\times3\.36×\\times3\.49×\\times3\.40×\\times3\.33×\\times2\.92×\\times3\.28×\\times3\.30×\\timesLiveCodeBench1462264\.11×\\times3\.80×\\times3\.90×\\times3\.60×\\times4\.10×\\times3\.62×\\times3\.74×\\times3\.15×\\times3\.79×\\times3\.58×\\timesAlpacaEval1482322\.18×\\times2\.22×\\times2\.10×\\times2\.14×\\times2\.13×\\times2\.12×\\times2\.06×\\times1\.88×\\times2\.05×\\times2\.13×\\timesMT\-Bench1482302\.48×\\times2\.48×\\times2\.37×\\times2\.38×\\times2\.45×\\times2\.37×\\times2\.31×\\times2\.07×\\times2\.30×\\times2\.35×\\timesSPEED1472282\.52×\\times2\.63×\\times2\.41×\\times2\.48×\\times2\.46×\\times2\.49×\\times2\.41×\\times2\.25×\\times2\.38×\\times2\.48×\\timesAverage1472293\.52×\\times3\.43×\\times3\.36×\\times3\.25×\\times3\.44×\\times3\.26×\\times3\.31×\\times2\.93×\\times3\.28×\\times3\.24×\\times
### 4\.3 Scaling with concurrency

Deployed systems run batched, a regime in which decoding moves from memory\-bandwidth bound towards compute bound and the cost of speculation is weighed differently\. SPEED provides a second, throughput split for this setting, which holds the input length fixed and varies serving concurrency\([1](https://arxiv.org/html/2608.20530#bib.bib19)\)\. We evaluate on its11K and22K input\-length blocks, in each of their three output\-entropy tiers\. Every drafter is trained on sequences of3,0723\{,\}072tokens \(Table[4](https://arxiv.org/html/2608.20530#A1.T4)\), and these are the blocks that a1,0241\{,\}024\-token generation keeps within that length, so all systems are served in the regime they were trained for\.

Table[2](https://arxiv.org/html/2608.20530#S4.T2)covers concurrencies fromc=1c\\\!=\\\!1toc=32c\\\!=\\\!32\. LiLiCorr attains the highest throughput at thirty\-five of the thirty\-six operating points, by up to7\.2%7\.2\\%over the strongest baseline, and trails DSpark by0\.3%0\.3\\%at the remaining one,11K mixed atc=32c\\\!=\\\!32\. Acceptance lengths are again close: LiLiCorr exceeds Domino everywhere, by1\.61\.6to10\.4%10\.4\\%, and stays within1\.4%1\.4\\%of DSpark, which edges it on the high\-entropy and mixed tiers\. Speedups compress for all systems as the batch grows, from2\.12\.1–3\.8×3\.8\\timesatc=1c\\\!=\\\!1to1\.21\.2–1\.8×1\.8\\timesatc=32c\\\!=\\\!32, and the ordering is preserved throughout\. Acceptance lengths and absolute throughputs are given in Appendix[F](https://arxiv.org/html/2608.20530#A6)\.

Table 2:Speedup over autoregressive decoding as serving concurrencyccgrows, on the11K and22K blocks of the SPEED throughput split\([1](https://arxiv.org/html/2608.20530#bib.bib19)\), Qwen3\-8B target, greedy decoding, single H100\. Each output\-entropy tier, low \(L\), mixed \(M\) and high \(H\), holds512512requests with1,0241\{,\}024generated tokens\. Best per tier and row in bold; acceptance lengths in Table[13](https://arxiv.org/html/2608.20530#A6.T13)\.#### 4\.3\.1 Inputs beyond the training range

The throughput split also holds88K,1616K and3232K blocks, which carry every drafter past the range it was trained on\. We serve these with the positional encoding extended at inference by YaRN\([15](https://arxiv.org/html/2608.20530#bib.bib20)\), applied identically to all systems\. Without it every drafter attends over positions outside its training range and gives up acceptance length\. We find that adding a hinge on the reranker’s decision gap during training makes LiLiCorr more robust at these lengths\. Trained this way, it attains the highest throughput at every one of the ninety operating points of the throughput split, within the training range and beyond, leading the strongest baseline by up to10\.0%10\.0\\%\. The regularizer, the full sweep, and the comparison against LiLiCorr trained without it are given in Appendix[G](https://arxiv.org/html/2608.20530#A7)\.

## 5 Conclusion

LiLiCorr correlates the per\-slot marginals of a parallel block drafter in a single network pass\. Assigning each candidate an*in*and an*out*vector reduces the correlation to batched matrix products, leaving only a greedy walk that touches no network weights\. Trained jointly with the drafter, LiLiCorr improves on the vanilla DFlash drafter on every benchmark we test, and a draft of comparable quality at a fraction of the cost of correcting one slot at a time makes it the faster system almost everywhere we measure, across benchmarks, serving concurrencies and input lengths\.

One limitation is inherent to the design: LiLiCorr reranks only the top\-KKtokens the drafter proposes, so a slot whose correct token is absent from that pool cannot be recovered, and the achievable acceptance length is bounded by the drafter’s coverage\. Coupling slots beyond adjacent pairs is a natural next step, and whether longer\-range couplings repay their cost under prefix acceptance remains open\.

## References

- \[1\]T\. Abramovich, M\. Ashkenazi, I\. Putterman, B\. Chislett, T\. Mitra, B\. D\. Rouhani, R\. Zilberstein, and Y\. Geifman\(2026\)SPEED\-bench: a unified and diverse benchmark for speculative decoding\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=Rl2uQlCoQX)Cited by:[Table 7](https://arxiv.org/html/2608.20530#A2.T7),[§4\.1](https://arxiv.org/html/2608.20530#S4.SS1.SSS0.Px1.p1.1),[§4\.3](https://arxiv.org/html/2608.20530#S4.SS3.p1.1),[Table 2](https://arxiv.org/html/2608.20530#S4.T2)\.
- \[2\]\(2024\)Medusa: simple LLM inference acceleration framework with multiple decoding heads\.InForty\-first International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=PEpbUobfJv)Cited by:[§1](https://arxiv.org/html/2608.20530#S1.p1.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1)\.
- \[3\]C\. Chen, S\. Borgeaud, G\. Irving, J\. Lespiau, L\. Sifre, and J\. Jumper\(2023\)Accelerating large language model decoding with speculative sampling\.arXiv preprint arXiv:2302\.01318\.Cited by:[§1](https://arxiv.org/html/2608.20530#S1.p1.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2608.20530#S3.SS1.p1.1)\.
- \[4\]J\. Chen, Y\. Liang, and Z\. Liu\(2026\)DFlash: block diffusion for flash speculative decoding\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=Oz335dV48X)Cited by:[§1](https://arxiv.org/html/2608.20530#S1.p1.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1),[§3\.4](https://arxiv.org/html/2608.20530#S3.SS4.p2.1)\.
- \[5\]X\. Cheng, X\. Yu, C\. Shao, J\. Li, Y\. Xiong, Y\. Qian, J\. Zhu, S\. Ma, X\. Zhang, J\. Ye,et al\.\(2026\)DSpark: confidence\-scheduled speculative decoding with semi\-autoregressive generation\.arXiv preprint arXiv:2607\.05147\.Cited by:[item \(3\)](https://arxiv.org/html/2608.20530#S1.I1.i3.p1.1),[§1](https://arxiv.org/html/2608.20530#S1.p3.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2608.20530#S4.SS1.SSS0.Px2.p1.1)\.
- \[6\]Y\. Fu, L\. Whalen, A\. Garg, C\. Wu, M\. Khadkevich, N\. Oswald, E\. Xie, D\. Egert, S\. T\. Sreenivas, S\. Diao,et al\.\(2026\)Nemotron\-labs\-diffusion: a tri\-mode language model unifying autoregressive, diffusion, and self\-speculation decoding\.arXiv preprint arXiv:2607\.05722\.Cited by:[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1)\.
- \[7\]J\. Huang, Y\. Zhang, Q\. Zhang, H\. Lin, H\. Xu, and L\. Zhang\(2026\)Domino: decoupling causal modeling from autoregressive drafting in speculative decoding\.arXiv preprint arXiv:2605\.29707\.Cited by:[item \(3\)](https://arxiv.org/html/2608.20530#S1.I1.i3.p1.1),[§1](https://arxiv.org/html/2608.20530#S1.p3.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2608.20530#S4.SS1.SSS0.Px2.p1.1)\.
- \[8\]Y\. Leviathan, M\. Kalman, and Y\. Matias\(2023\)Fast inference from transformers via speculative decoding\.InInternational conference on machine learning,pp\. 19274–19286\.Cited by:[§1](https://arxiv.org/html/2608.20530#S1.p1.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2608.20530#S3.SS1.p1.1)\.
- \[9\]Y\. Li, F\. Wei, C\. Zhang, and H\. Zhang\(2024\)Eagle\-2: faster inference of language models with dynamic draft trees\.InProceedings of the 2024 conference on empirical methods in natural language processing,pp\. 7421–7432\.Cited by:[§1](https://arxiv.org/html/2608.20530#S1.p1.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1)\.
- \[10\]Y\. Li, F\. Wei, C\. Zhang, and H\. Zhang\(2024\)EAGLE: speculative sampling requires rethinking feature uncertainty\.InForty\-first International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=1NdN7eXyb4)Cited by:[§1](https://arxiv.org/html/2608.20530#S1.p1.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1)\.
- \[11\]Y\. Li, F\. Wei, C\. Zhang, and H\. Zhang\(2026\)Eagle\-3: scaling up inference acceleration of large language models via training\-time test\.Advances in Neural Information Processing Systems38,pp\. 136737–136756\.Cited by:[§1](https://arxiv.org/html/2608.20530#S1.p1.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1)\.
- \[12\]Nemotron\-Post\-Training\-Dataset\-v2External Links:[Link](https://huggingface.co/datasets/nvidia/Nemotron-Post-Training-Dataset-v2)Cited by:[§4\.1](https://arxiv.org/html/2608.20530#S4.SS1.SSS0.Px3.p1.1)\.
- \[13\]S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. Li\(2026\)Large language diffusion models\.Advances in Neural Information Processing Systems38,pp\. 50608–50646\.Cited by:[§1](https://arxiv.org/html/2608.20530#S1.p3.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1)\.
- \[14\]Y\. Oda, R\. Mathieu, R\. Knyazhitskiy, and A\. Chakhvadze\(2026\)Trees from marginals: autoregressive drafting with factorized priors\.arXiv preprint arXiv:2607\.06763\.Cited by:[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px2.p1.1)\.
- \[15\]B\. Peng, J\. Quesnelle, H\. Fan, and E\. Shippole\(2024\)Yarn: efficient context window extension of large language models\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 31932–31951\.Cited by:[Appendix F](https://arxiv.org/html/2608.20530#A6.SS0.SSS0.Px3.p1.1),[§4\.3\.1](https://arxiv.org/html/2608.20530#S4.SS3.SSS1.p1.1)\.
- \[16\]L\. Ringel and Y\. Romano\(2026\)Accelerating speculative decoding with block diffusion draft trees\.arXiv preprint arXiv:2604\.12989\.Cited by:[§1](https://arxiv.org/html/2608.20530#S1.p3.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px2.p1.1)\.
- \[17\]S\. S\. Sahoo, M\. Arriola, Y\. Schiff, A\. Gokaslan, E\. Marroquin, J\. T\. Chiu, A\. Rush, and V\. Kuleshov\(2024\)Simple and effective masked diffusion language models\.Advances in Neural Information Processing Systems37,pp\. 130136–130184\.Cited by:[§1](https://arxiv.org/html/2608.20530#S1.p3.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1)\.
- \[18\]A\. Viterbi\(1967\)Error bounds for convolutional codes and an asymptotically optimum decoding algorithm\.IEEE transactions on Information Theory13\(2\),pp\. 260–269\.Cited by:[Appendix E](https://arxiv.org/html/2608.20530#A5.SS0.SSS0.Px1.p1.2)\.
- \[19\]T\. Wu, Y\. Yao, Z\. Qi, H\. Zheng, Z\. Wang, H\. Ma, L\. Liao, H\. Lakkaraju, J\. Li, and Y\. Du\(2026\)D\-pace: dynamic position\-aware cross\-entropy for parallel speculative drafting\.arXiv preprint arXiv:2605\.18810\.Cited by:[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1)\.
- \[20\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§4\.1](https://arxiv.org/html/2608.20530#S4.SS1.SSS0.Px1.p1.1)\.
- \[21\]J\. Ye, Z\. Xie, L\. Zheng, J\. Gao, Z\. Wu, X\. Jiang, Z\. Li, and L\. KongDream 7b: diffusion large language models, 2025\.URL https://arxiv\. org/abs/2508\.15487\.Cited by:[§1](https://arxiv.org/html/2608.20530#S1.p3.1),[§2](https://arxiv.org/html/2608.20530#S2.SS0.SSS0.Px1.p1.1)\.

## Appendix AImplementation and Training Details

##### Architecture\.

Each candidate enters the head as one vector of the head’s hidden width, formed by summing four terms\. The first is the target’s input embedding of the candidate token, held frozen and projected down\. The second is the drafter’s hidden state at that slot, projected down and shared by the slot’sKKcandidates\. The third is a two\-layer network over five scalars describing the drafter’s confidence in the candidate, all read from its distribution at that slot: the log\-probability and the probability under the full vocabulary, the log\-probability gap to the slot’s most likely token, the rank normalized to\[0,1\]\[0,1\], and an indicator for being the slot’s top choice\. The fourth is a pair of learned tables, one indexed by slot and one by rank\. The target state that every candidate is later fused with is formed separately: the target’s hidden states that produced the last verified token, collapsed across five captured layers, normalized, and projected to the same width\.

The head itself is a two\-layer bidirectional Transformer over the\(B−1\)×K\(B\\\!\-\\\!1\)\\times Kcandidate lattice, attending over all of it at once, with hidden width10241024, eight attention heads, and a feed\-forward width of20482048, using RMSNorm and SiLU activations\. Beyond the slot table, position enters through two attention biases learned per head and added to the attention logits, one indexed by the signed slot distance between a pair and one marking pairs of candidates that occupy the same slot\. Only after the Transformer is each candidate combined with that target state, by concatenating the candidate state, the target state, and their elementwise product, and projecting the result back down to the head’s width through a single linear layer followed by a SiLU\. This is the one concatenation in the head, everything else being a sum\. Two linear heads then emit the out and in vectors of Section[3\.2](https://arxiv.org/html/2608.20530#S3.SS2), each of dimensionded\_\{e\}and normalized to unit norm, and a third emits the anchor’s single out vector\. Table[4](https://arxiv.org/html/2608.20530#A1.T4)presents the settings\.

##### Objective\.

The DFlash cross\-entropy of Eq\. \([6](https://arxiv.org/html/2608.20530#S3.E6)\) is applied at every slot, whereas the LiLiCorr terms are evaluated only on the oracle prefix, the maximal run of leading slots in which the ground\-truth token is present in the candidate pool\. Slots beyond the first coverage miss contribute to none of the three terms\.

##### Serving\.

We fuse top\-KKcandidate extraction and vocabulary\-wide logit normalization into a single kernel, compile the matching and selection step, and use wide CUDA graphs that fold candidate pooling and reranking into a single replay\. The same level of optimization is applied to Domino’s correction loop\. Table[3](https://arxiv.org/html/2608.20530#A1.T3)gives the stage decomposition for both heads before and after it, and is the source of the per\-block figures quoted in Section[4\.1](https://arxiv.org/html/2608.20530#S4.SS1)\.

LiLiCorr and Domino differ only in the speculative head, which each places on a shared DFlash drafter in place of its per\-slot argmax\. The serving stack folds that head into the drafter, and every other result we report is measured that way\. To isolate the head, this measurement alone is taken on a pipeline that keeps it separate, which is why the per\-block wall it reports is not the one behind the throughputs of Section[4\.2](https://arxiv.org/html/2608.20530#S4.SS2)\. DSpark folds its head by construction and leaves no separate stage to measure, so its comparison rests entirely on end\-to\-end throughput\.

Table 3:Per\-block stage decomposition in milliseconds, MT\-Bench, single H100\. The systems differ only in the speculative head\. Lowest head cost and per\-block wall of the two added heads in bold\.
##### Baselines\.

Domino is served in its widened configuration, with512512anchors and the same block size of1616\. Its correction head is a recurrent cell over the drafter’s hidden width followed by a vocabulary projection\. DSpark is served in a width\-matched configuration, also with512512anchors, and at temperature11under each of its two proposal contracts, the deterministic one that LiLiCorr and Domino use and a sampled one\. All systems are trained on the same regenerated corpus for the same six epochs, at the same global batch size and sequence length\. LiLiCorr and Domino differ only in the head placed on top of a shared DFlash drafter and in the losses that train it\.

Table 4:Configuration of LiLiCorr and of the joint training run\. Symbols follow Section[3](https://arxiv.org/html/2608.20530#S3)\.SymbolQuantityValueBBBlock size, of which drafted slots1616, of which1515KKCandidates retained per slot88ded\_\{e\}Dimension of the in and out vectors10241024σ\\sigmaCosine scale8\.08\.0Scoring Transformer layers22Hidden width, heads, feed\-forward width10241024,88,20482048Parameters in the LiLiCorr head36\.7636\.76M \(8B\),32\.0432\.04M \(4B\)β\\betaDecay of the DFlash cross\-entropy7\.07\.0λ\\lambdaWeight of the distractor penalty1\.01\.0γ\\gammaWeight of the reranker objective, Eq\. \([7](https://arxiv.org/html/2608.20530#S3.E7)\)0\.250\.25OptimizerAdamWLearning rate, drafter and LiLiCorr head6×10−46\\times 10^\{\-4\},1\.5×10−41\.5\\times 10^\{\-4\}Weight decay, drafter and LiLiCorr head00,0\.010\.01Schedulecosine,4%4\\%warmupGradient\-norm clipping1\.01\.0Global batch size, sequence length6464,30723072Epochs66Precisionbf16, fp32 master weightsHardwareH100

## Appendix BAdditional Results

##### Acceptance length and absolute throughput\.

Table[1](https://arxiv.org/html/2608.20530#S4.T1)reports speedup, which is the quantity the comparison turns on\. Tables[5](https://arxiv.org/html/2608.20530#A2.T5)and[6](https://arxiv.org/html/2608.20530#A2.T6)give the two underlying measurements for each target, the block\-weighted acceptance length and the output throughput in tokens per second, under the same protocol\. They are the source of the acceptance\-length statements of Section[4\.2](https://arxiv.org/html/2608.20530#S4.SS2)\.

Table 5:Acceptance length \(τ\\tau, block\-weighted\) and output throughput behind Table[1](https://arxiv.org/html/2608.20530#S4.T1), Qwen3\-8B target\. Cells reportτ\\tau/ TPS / speedup over autoregressive decoding, which averages150150and147147TPS at the two temperatures\. Best per row and temperature in bold\.Table 6:As Table[5](https://arxiv.org/html/2608.20530#A2.T5), with a Qwen3\-4B target\. Autoregressive decoding averages234234TPS atT=0T\\\!=\\\!0and229229atT=1T\\\!=\\\!1\.
##### Per\-category results on SPEED\.

Tables[7](https://arxiv.org/html/2608.20530#A2.T7)and[8](https://arxiv.org/html/2608.20530#A2.T8)break the SPEED aggregate of Section[4\.2](https://arxiv.org/html/2608.20530#S4.SS2)into its eleven categories, under the same protocol\. The per\-benchmark conclusions carry over: LiLiCorr serves faster than every system compared in forty\-one of the forty\-four cells, and accepts more tokens per block than Domino in thirty\-seven of them\. The multilingual lead noted in Section[4\.2](https://arxiv.org/html/2608.20530#S4.SS2)is the clearest of any category on throughput, reaching8\.0%8\.0\\%in acceptance length and16\.8%16\.8\\%in throughput on Qwen3\-4B under greedy decoding\. Sample standard deviations across the threeT=1T\\\!=\\\!1seeds stay below0\.280\.28on acceptance length in every category and system, with two exceptions, both on Qwen3\-4B under DSpark’s sampled contract\.

##### Seed variability atT=1T\\\!=\\\!1\.

Tables[9](https://arxiv.org/html/2608.20530#A2.T9)and[10](https://arxiv.org/html/2608.20530#A2.T10)give the mean and sample standard deviation over seeds4242,4343, and4444that stand behind theT=1T\\\!=\\\!1panels of Tables[5](https://arxiv.org/html/2608.20530#A2.T5)and[6](https://arxiv.org/html/2608.20530#A2.T6)\. Dispersion is small throughout: the standard deviation ofτ\\taustays below0\.280\.28on every benchmark and system, and below0\.160\.16on all but two\. What scatter remains concentrates on the small splits, above all AIME 2025, which at3030prompts is an order of magnitude smaller than any other\. On the two splits of more than a thousand prompts no standard deviation exceeds0\.140\.14\.

Table 7:Per\-category results on SPEED\([1](https://arxiv.org/html/2608.20530#bib.bib19)\)with a Qwen3\-8B target, reported asτ\\tau/ TPS and otherwise as in Table[5](https://arxiv.org/html/2608.20530#A2.T5)\.Table 8:As Table[7](https://arxiv.org/html/2608.20530#A2.T7), with a Qwen3\-4B target\.Table 9:Seed variability behind theT=1T\\\!=\\\!1panel of Table[5](https://arxiv.org/html/2608.20530#A2.T5), Qwen3\-8B target\. Mean over seeds4242,4343and4444, sample standard deviation in parentheses\.Table 10:As Table[9](https://arxiv.org/html/2608.20530#A2.T9), with a Qwen3\-4B target\.

## Appendix CLoss\-Term Ablation

We isolate the distractor penalty by removing it and holding everything else fixed, the total weight of the reranker objective included, so that the comparison reallocates the same budget rather than enlarging it\. Table[11](https://arxiv.org/html/2608.20530#A3.T11)reports acceptance length on the evaluation suite at concurrency one\.

The penalty carries the suite\. It raises acceptance length on all nine benchmarks, by1\.51\.5to8\.9%8\.9\\%and by3\.2%3\.2\\%on the nine\-benchmark average, with its largest gain on HumanEval and its smallest on MATH\-500\. This is the term that turns a reranker trained only against the ground\-truth label into one trained against the candidates that actually cost acceptances, Appendix[D](https://arxiv.org/html/2608.20530#A4)giving the sense in which it is an expected target cost rather than a second cross\-entropy\.

Table 11:Loss\-term ablation, Qwen3\-8B target, greedy decoding, concurrency one, acceptance lengthτ\\tau\. The two rows share the architecture, all other settings, and the total reranker weight, so the second reallocates that budget rather than enlarging it\. It is the model of Section[4](https://arxiv.org/html/2608.20530#S4)\. Best per column in bold\.
## Appendix DDerivations for the Reranker Objective

##### The distractor penalty is an expected target cost\.

WritePθ​\(k\)P\_\{\\theta\}\(k\)forPθ​\(cs=k∣cs−1=gs−1\)P\_\{\\theta\}\(c\_\{s\}=k\\mid c\_\{s\-1\}=g\_\{s\-1\}\), letck=−log⁡qkc\_\{k\}=\-\\log q\_\{k\}be the cost the target assigns to candidatekk, low where the target is confident, and letwk=ReLU⁡\(ck−cgs\)w\_\{k\}=\\relu\(c\_\{k\}\-c\_\{g\_\{s\}\}\)be that cost relative to the ground truth after rectification\. The penalty of Eq\. \([4](https://arxiv.org/html/2608.20530#S3.E4)\) is thenΩs=∑kPθ​\(k\)​wk\\Omega\_\{s\}=\\sum\_\{k\}P\_\{\\theta\}\(k\)\\,w\_\{k\}, the expected rectified cost of the candidate LiLiCorr commits to\. Whengsg\_\{s\}is the target’s preferred candidate in the pool no cost is negative, the rectifier is inactive, and since∑kPθ​\(k\)=1\\sum\_\{k\}P\_\{\\theta\}\(k\)=1,

Ωs=∑kPθ​\(k\)​\(ck−cgs\)=H⁡\(Pθ,q\)−cgs,H⁡\(Pθ,q\)=∑kPθ​\(k\)​ck\.\\Omega\_\{s\}\\;=\\;\\sum\_\{k\}P\_\{\\theta\}\(k\)\\,\\left\(c\_\{k\}\-c\_\{g\_\{s\}\}\\right\)\\;=\\;H\(P\_\{\\theta\},q\)\-c\_\{g\_\{s\}\},\\qquad H\(P\_\{\\theta\},q\)=\\sum\_\{k\}P\_\{\\theta\}\(k\)\\,c\_\{k\}\.\(9\)H⁡\(Pθ,q\)H\(P\_\{\\theta\},q\)is the expected target cost underPθP\_\{\\theta\}, and the offsetcgsc\_\{g\_\{s\}\}does not depend on the LiLiCorr parameters, so∇θΩs=∇θH​\(Pθ,q\)\\nabla\_\{\\theta\}\\Omega\_\{s\}=\\nabla\_\{\\theta\}H\(P\_\{\\theta\},q\): the penalty and a cross\-entropy against the target have the same gradient\. The penalty involves the target only through cost differences, so it is unchanged ifqqis renormalized over the pool\.

##### Why this is not distribution matching\.

The target is frozen, so the costsckc\_\{k\}are constants andΩs\\Omega\_\{s\}is linear inPθP\_\{\\theta\}\. A linear function on the probability simplex attains its minimum at a vertex, that is, at a distribution placing all of its mass on one candidate, and the minimizing vertex is the candidate of least cost\. Under the assumption above that candidate isgsg\_\{s\}, so minimizing the penalty drives LiLiCorr to commit to the target’s preferred candidate rather than to reproduce the target’s distribution over the pool\. Committing is what the verifier rewards, since a block is accepted on the tokens the target would itself emit and mass placed elsewhere is not recovered\.

##### The rectifier\.

At the occasional slot wheregsg\_\{s\}is not the target’s preferred candidate, the rectifier clips the negative costs that would otherwise reward a candidate the target ranks above the supervised token\. Such slots are rare, and arise from numerical differences between the offline generation of the supervision and the teacher\-forced target pass used during training, rather than from a genuine target preference\.

##### Complementary gradients\.

The hard cross\-entropy and the distractor penalty act on the ground\-truth logitzgz\_\{g\}through complementary gradients\. For cross\-entropy,

∂ℒCE∂zg=Pθ​\(g\)−1,\\frac\{\\partial\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\}\{\\partial z\_\{g\}\}=P\_\{\\theta\}\(g\)\-1,\(10\)which remains strong whenPθ​\(g\)P\_\{\\theta\}\(g\)is small, exactly the hardest mistakes\. For the penalty,

∂Ωs∂zg=−Pθ​\(g\)​𝔼Pθ​\[w\],\\frac\{\\partial\\Omega\_\{s\}\}\{\\partial z\_\{g\}\}=\-\\,P\_\{\\theta\}\(g\)\\,\\mathbb\{E\}\_\{P\_\{\\theta\}\}\[w\],\(11\)which vanishes asPθ​\(g\)→0P\_\{\\theta\}\(g\)\\to 0, so the penalty alone is weak precisely when the ground truth is badly misranked\. It is also indifferent among candidates of zero cost, including target ties and candidates the target ranks abovegg, since the rectifier clips their costs to zero\. Cross\-entropy therefore robustly pulls the exact ground\-truth token upward, whileΩs\\Omega\_\{s\}preferentially pushes target\-rejected confusers downward, so the penalty is best viewed as cost\-aware shaping of cross\-entropy rather than a replacement for it\. Both terms are evaluated only on the oracle prefix and averaged over the same slots, so slots after the first coverage miss contribute to neither\.

## Appendix EGreedy Selection Versus the Highest\-Scoring Path

Sections[3\.3](https://arxiv.org/html/2608.20530#S3.SS3)and[3\.5](https://arxiv.org/html/2608.20530#S3.SS5)argue that prefix acceptance rewards a greedy left\-to\-right decode over a search for the best\-scoring block\. We test the decode rule in isolation, holding the trained model fixed and varying only the rule that selects a path through the candidate lattice\.

##### Path selection\.

Letc=\(c1,…,cB−1\)c=\(c\_\{1\},\\dots,c\_\{B\-1\}\)denote a path, wherecsc\_\{s\}is the candidate committed at slotssandc0c\_\{0\}is the anchor\. Since LiLiCorr couples only adjacent slots, a path scores as the sum of its matching terms,

g⁡\(c\)=∑s=1B−1ϕp​\(s−1,cs−1,cs\)\.g\(c\)\\;=\\;\\sum\_\{s=1\}^\{B\-1\}\\phi^\{\\mathrm\{p\}\}\\\!\\left\(s\\\!\-\\\!1,\\,c\_\{s\-1\},\\,c\_\{s\}\\right\)\.\(12\)The greedy rule of Section[3\.5](https://arxiv.org/html/2608.20530#S3.SS5)commits each slot to the best continuation of what precedes it and never revisits an earlier choice\. The alternative is to return the path that maximizesgg\. Enumeration is infeasible, as the lattice holdsKB−1K^\{B\-1\}paths, but the pairwise structure of Eq\. \([12](https://arxiv.org/html/2608.20530#A5.E12)\) makes the exact maximizer reachable inO⁡\(B​K2\)O\(B\\,K^\{2\}\)time by dynamic programming\. Sweeping left to right, one records for each candidate the score of the best path ending there, then traces the stored predecessors back from the best final candidate\. This is the Viterbi algorithm\([18](https://arxiv.org/html/2608.20530#bib.bib21)\), a strictly stronger optimizer ofggthan the greedy walk\.

##### Why the stronger optimizer is the wrong one\.

Under prefix acceptance every slot contributes only while all the slots before it have matched, so the early ones carry most of the expected acceptance length, and a block that loses its first slot gains nothing from the rest\. Maximizingggtargets the block as a whole, which is essentially the probability that the entire block survives verification\. Giving up an early slot to raise the total is rational for that objective and self\-defeating here, because the improved tail is never reached\. The greedy walk instead maximizes the leading decision exactly, and each later one conditioned on what was committed\.

Table 12:Decode\-rule ablation, Qwen3\-8B target, greedy decoding, concurrency one, acceptance lengthτ\\tau\. Both arms serve the same checkpoint and differ only in the rule that selects a path through the lattice, each as a single fused kernel in the same captured graph\.Δ\\Deltais the change from greedy to exact, so negative favors greedy\. Best per row in bold\.
##### Results\.

Table[12](https://arxiv.org/html/2608.20530#A5.T12)shows the exact optimizer decodes worse\. It lowers acceptance length on all nine benchmarks, by0\.900\.90to4\.81%4\.81\\%and by2\.44%2\.44\\%on average\. Throughput follows, since the two rules are equally cheap to serve: each is a single fused kernel folded into the same captured draft graph, and the block rate they sustain differs by at most0\.62%0\.62\\%on any benchmark and by0\.005%0\.005\\%on average\. The stronger optimizer is therefore had for free and still loses\. Its worst case is AIME 2025, though at3030problems that split carries the direction rather than the magnitude; every other benchmark lies between−0\.90%\-0\.90\\%and−3\.27%\-3\.27\\%\. On the multi\-domain SPEED suite greedy wins all eleven categories\.

## Appendix FInput\-Length Sweep on the SPEED Throughput Split

Table[13](https://arxiv.org/html/2608.20530#A6.T13)gives the acceptance lengths and absolute throughputs behind Table[2](https://arxiv.org/html/2608.20530#S4.T2), and Table[14](https://arxiv.org/html/2608.20530#A6.T14)extends the sweep to the88K,1616K and3232K blocks under the positional extension of Section[4\.3\.1](https://arxiv.org/html/2608.20530#S4.SS3.SSS1)\. Each block holds512512requests at a fixed input length with1,0241\{,\}024generated tokens, in each of the three output\-entropy tiers, and is swept fromc=1c\\\!=\\\!1toc=32c\\\!=\\\!32\. At3232K the few prompts whose templated input exceeds39,90439\{,\}904tokens are right\-truncated so that the full generation budget fits within the target’s40,96040\{,\}960\-token context\. Shorter inputs are left unchanged\.

##### Within the training range\.

On the11K and22K blocks the acceptance\-length gap to the strongest baseline is small, from1\.4%1\.4\\%behind to3\.2%3\.2\\%ahead, and the lead LiLiCorr holds is carried by throughput, by up to7\.2%7\.2\\%\. Every acceptance\-length exception is a loss to DSpark on the high\-entropy or mixed tier; against Domino the block is longer at all thirty\-six operating points\.

##### Beyond the training range\.

On the88K,1616K and3232K blocks LiLiCorr serves faster than every baseline at forty\-six of the fifty\-four operating points, by up to10\.8%10\.8\\%, and attains the highest acceptance length at thirty\-eight of them\. Every exception is a loss to DSpark on the1616K and3232K blocks, costing at most0\.91%0\.91\\%of throughput and4\.2%4\.2\\%of acceptance length\. Appendix[G](https://arxiv.org/html/2608.20530#A7)shows that the hinge regularizer of Section[4\.3\.1](https://arxiv.org/html/2608.20530#S4.SS3.SSS1)removes all eight, leaving LiLiCorr with the highest throughput at every one of the ninety operating points of the throughput split, within the training range and beyond\.

##### Positional extension\.

Served without it, every drafter loses acceptance length on these blocks, and loses it consistently, since the positions it is then asked to attend over lie outside the range it was trained on\. The extension is YaRN\([15](https://arxiv.org/html/2608.20530#bib.bib20)\), applied identically to every system, and all four absorb it: past the training range every drafter still accepts roughly between2\.82\.8and5\.85\.8tokens per block, a band comparable to the one it reaches at11K and22K\.

##### Memory\-bound operating points\.

Beyond88K the highest concurrencies measure offered load rather than resident concurrency\. The key\-value cache of a single H100 admits roughly1818sequences at a1616K input and99at3232K, so throughput ceases to improve once the sweep exceeds that capacity\. At1616K thec=32c\\\!=\\\!32row runs against the memory limit rather than on a larger batch, and at3232K thec=16c\\\!=\\\!16andc=32c\\\!=\\\!32rows describe the same operating point\. These rows are marked with a dagger\. All systems meet the same ceiling, so the comparison between them remains valid, but the sweep no longer varies the resident batch size it is intended to control\. The11K through88K blocks stay within memory across the entire sweep\.

Table 13:Acceptance length and absolute throughput behind Table[2](https://arxiv.org/html/2608.20530#S4.T2), on the11K and22K blocks of the SPEED throughput split\. Cells reportτ\\tau/ TPS / speedup over autoregressive decoding at the same concurrency, which does not speculate and so has noτ\\tau\. Best per row in bold\.Table 14:The88K,1616K and3232K blocks of the SPEED throughput split, all served with the positional extension of Section[4\.3\.1](https://arxiv.org/html/2608.20530#S4.SS3.SSS1), otherwise as in Table[13](https://arxiv.org/html/2608.20530#A6.T13)\. Rows marked†\\daggeroffer more concurrency than the key\-value cache holds\.

## Appendix GExtending to Inputs Beyond the Training Range

Section[4\.3\.1](https://arxiv.org/html/2608.20530#S4.SS3.SSS1)reports the88K,1616K and3232K blocks of the throughput split under two changes, a positional extension applied at inference to every system alike and a hinge regularizer added to the reranker objective during training\. This appendix gives the regularizer and the full evidence for both\.

##### Margin regularization\.

Cross\-entropy maximizes the likelihood of the ground\-truth token, but the served rule is an argmax: a slot is decided by which candidate scores highest, not by how much probability the ground truth carries\. Once the ground truth is ranked first, cross\-entropy keeps spending gradient on decisions that are already settled\. Contested slots, where the decision gap is narrow, are exactly the ones a longer prompt produces more of\. We therefore add a hinge on the decision gapΔs=zs​\[gs\]−maxk≠gs⁡zs​\[k\]\\Delta\_\{s\}=z\_\{s\}\[g\_\{s\}\]\-\\max\_\{k\\neq g\_\{s\}\}z\_\{s\}\[k\],

ℒmargin=1\|𝒮\|​∑s∈𝒮ReLU⁡\(ρ−Δs\),\\mathcal\{L\}\_\{\\mathrm\{margin\}\}=\\frac\{1\}\{\|\\mathcal\{S\}\|\}\\sum\_\{s\\in\\mathcal\{S\}\}\\relu\\\!\\left\(\\rho\-\\Delta\_\{s\}\\right\),\(13\)with a fixed marginρ\>0\\rho\>0, and blend it against the hard label at a fixed total reranker weight, so that Eq\. \([7](https://arxiv.org/html/2608.20530#S3.E7)\) becomesℒDFlash\+γ⁡\[\(1−μ\)​ℒCE\+λ​ℒdist\+μ​ℒmargin\]\\mathcal\{L\}\_\{\\text\{DFlash \}\}\+\\gamma\[\(1\-\\mu\)\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\+\\lambda\\mathcal\{L\}\_\{\\mathrm\{dist\}\}\+\\mu\\mathcal\{L\}\_\{\\mathrm\{margin\}\}\]\. The hinge is inactive once the gap clearsρ\\rho, so its gradient falls only on the slots still in contention, and it leaves inference unchanged\. We write LiLiCorr \+ margin for the model trained this way and setμ=0\.5\\mu\\\!=\\\!0\.5andρ=2\.0\\rho\\\!=\\\!2\.0, with every other setting as in Table[4](https://arxiv.org/html/2608.20530#A1.T4)\.

##### Effect on the evaluation suite\.

Table[15](https://arxiv.org/html/2608.20530#A7.T15)compares the two on the nine benchmarks\. They are within a few tenths of a percent of each other: on the unweighted average LiLiCorr \+ margin is behind by0\.10%0\.10\\%on Qwen3\-8B under greedy decoding and ahead by0\.040\.04,0\.100\.10and0\.52%0\.52\\%in the other three panels\. The regularizer is therefore neither bought nor paid for on short prompts, which is what makes it a long\-input measure rather than a general improvement\.

##### Effect beyond the training range\.

Tables[14](https://arxiv.org/html/2608.20530#A6.T14)and[17](https://arxiv.org/html/2608.20530#A7.T17)give the88K,1616K and3232K blocks for the two models in the same format\. LiLiCorr serves faster than every baseline at forty\-six of the fifty\-four operating points; LiLiCorr \+ margin does so at all fifty\-four, and raises the count on acceptance length from thirty\-eight to forty\-four\. Its acceptance\-length gain over LiLiCorr grows with the prompt:−0\.1%\-0\.1\\%at88K,\+1\.3%\+1\.3\\%at1616K and\+2\.8%\+2\.8\\%at3232K on the mean over each block\. Inside the training range the ordering reverses and the regularizer costs a little,0\.4%0\.4\\%on the mean at11K and22K \(Table[16](https://arxiv.org/html/2608.20530#A7.T16)\)\.

Table 15:LiLiCorr and LiLiCorr \+ margin on the nine benchmarks, asτ\\tau/ speedup, each speedup against the autoregressive baseline of its own campaign\. Otherwise as in Table[1](https://arxiv.org/html/2608.20530#S4.T1)\.Table 16:LiLiCorr \+ margin on the11K and22K blocks of the SPEED throughput split, reported as in Table[13](https://arxiv.org/html/2608.20530#A6.T13)\.Table 17:LiLiCorr \+ margin on the88K,1616K and3232K blocks of the SPEED throughput split, reported as in Table[14](https://arxiv.org/html/2608.20530#A6.T14)and directly comparable to it\. Rows marked†\\daggeroffer more concurrency than the key\-value cache holds\.