Uncertainty-gated selection for block-sparse attention

# Uncertainty-gated selection for block-sparse attentionCode, data, and reproduction scripts: https://github.com/ThomasRossi/uncertainty-gated-block-sparse-attention. Persistent archive: doi:10.5281/zenodo.20630587 (concept DOI; always resolves to the latest version). A value-of-information view of the SSA-style selector
Source: [https://arxiv.org/html/2607.07724](https://arxiv.org/html/2607.07724)
###### Abstract

Block\-sparse attention scales long\-context language models by replacing theO​\(N2\)O\(N^\{2\}\)softmax with a per\-query top\-kkselection over key blocks\. This cutoff is myopic: when thekk\-th and\(k\+1\)\(k\{\+\}1\)\-th blocks are nearly tied in score, the selector commits without spending extra budget, and a dropped block carrying answer evidence is unrecoverable downstream\. We propose a value\-of\-information*router*that measures, for each query, how decisively the top\-kkcut was made, and doubles the kept set for the queries where that gap is smallest; the rule is backbone\-agnostic and stacks with existing block\-scoring methods such as Quest\. On LongBench\-v2 medium atn=215n=215\(the entire dataset subset\), router\-on\-Quest reaches paired recall0\.750\.75vs\. top\-kk0\.470\.47–\+28\+28pp over the SSA\-style baseline \(McNemarp<0\.01p<0\.01\) – and lands within22pp of dense on RULER NIAH multikey at the same context\. The lift reproduces on four models from three architectures \(Qwen2\.5, Mistral\-Nemo, Qwen3\.6\)\. At128128K, the router preserves0\.810\.81and0\.890\.89of dense accuracy on Qwen2\.5\-7B\-1M and Qwen3\.6 \(vs\. SSA\-style top\-kkat0\.090\.09on the former\) while the fused selection\-plus\-kernel pipeline runs at0\.62×0\.62\\timesand0\.80×0\.80\\timesdense wall time\.

## 1Introduction

Long\-context language models increasingly use*block\-sparse attention*as a drop\-in replacement for theO​\(N2\)O\(N^\{2\}\)softmax\. The idea is shared by Quest\(Tang et al\.,[2024](https://arxiv.org/html/2607.07724#bib.bib2)\), H2O\(Zhang et al\.,[2023](https://arxiv.org/html/2607.07724#bib.bib3)\), SnapKV\(Li et al\.,[2024](https://arxiv.org/html/2607.07724#bib.bib4)\), MInference\(Jiang et al\.,[2024](https://arxiv.org/html/2607.07724#bib.bib5)\), NSA\(Yuan et al\.,[2025](https://arxiv.org/html/2607.07724#bib.bib6)\), MoBA\(Lu et al\.,[2025](https://arxiv.org/html/2607.07724#bib.bib7)\), and Subquadratic’s SSA\(Subquadratic,[2025](https://arxiv.org/html/2607.07724#bib.bib8)\): a cheap per\-query*selector*pickskkout ofNBN\_\{\\\!B\}key blocks; exact attention then runs only on the selected set\. The selector is the lever – it decides where the model attends\.

This is a myopic decision\. When thekk\-th and\(k\+1\)\(k\{\+\}1\)\-th block are nearly tied, top\-kksilently breaks the tie and moves on; if the dropped block contained an evidence token, the answer is gone, and no downstream layer can recover it\. The failure mode is structural, not noisy: it bites hardest on*multi\-hop*and*query\-latent*retrieval, where the relevance of a block depends on what was learned earlier in the same forward pass and is not visible to the selector’s surface query–key match\. SSA’s own reported NIAH multi\-key recall degrades sharply as the number of keys grows\(Subquadratic,[2025](https://arxiv.org/html/2607.07724#bib.bib8)\)\.

This paper\.We treat the top\-kkcutoff as a*value\-of\-information*\(VoI\) decision and add a single layer of policy on top of it\. For each Q\-tile and head, we compute the normalised cutoff margin

σ=s\(k−1\)−s\(k\)s\(0\)−s\(k\)∈\[0,1\],\\sigma\\;=\\;\\frac\{s\_\{\(k\-1\)\}\-s\_\{\(k\)\}\}\{s\_\{\(0\)\}\-s\_\{\(k\)\}\}\\;\\in\\;\[0,1\],wheres\(⋅\)s\_\{\(\\cdot\)\}are the sorted block scores\. A smallσ\\sigmameans the cutoff is high\-risk; we then*route*that tile to a2×2\\timesexpandedkv\_idx– it gets to attend to more blocks – while confident tiles keep the baselinekbudgetk\_\{\\text\{budget\}\}\. The expansion is paid for selectively: we trigger on the bottomqq\-fraction of tiles per layer, so the average attended set grows by only1\+q1\+qblocks per row\.

The router is backbone\-agnostic\.The cutoff marginσ\\sigmais a function of the sorted block scores; it does not depend on how those scores are computed\. Existing selectors differ in their block\-scoring backbone – SSA pools keys with a mean \(k¯b=1Bn​∑jkj\\bar\{k\}\_\{b\}=\\frac\{1\}\{B\_\{n\}\}\\sum\_\{j\}k\_\{j\}\), Quest uses a min/max upper bound \(s=∑dmax⁡\(qd​kb,dmax,qd​kb,dmin\)s=\\sum\_\{d\}\\max\(q\_\{d\}k^\{\\max\}\_\{b,d\},q\_\{d\}k^\{\\min\}\_\{b,d\}\)\)\. The router can sit on top of any of them\. This turns the contribution from*a replacement for top\-kk*into*a generic budget\-allocation layer that stacks on whichever scoring backbone is best for the task*\. We validate this empirically:better scoring \(Quest\) and better budget allocation \(router\) are orthogonal directions; combining them strictly dominates either aloneon both benchmarks we test\.

Contributions\.

1. 1\.A VoI formulation of the selector cutoff that adds one scalar per tile and one quantile threshold per layer, independent of the block\-scoring backbone\. No retraining, no extra parameters, no per\-row keep tensor\.
2. 2\.Empirical demonstration that the router composes with two distinct scoring backbones \(SSA\-style K\-mean and Quest’s K\-min/K\-max\) and that, on every model in the panel, the router\-lifted version of the*winning*backbone dominates the unlifted version of the other\. The result holds on two standardised benchmarks \(RULER NIAH multi\-key and LongBench\-v2 medium\), four models from three architecture classes, and contexts from3232K to128128K\. Which backbone wins is model\-dependent – QK\-Norm flips the winner from Quest’s K\-max to the SSA\-style K\-mean – but the router lifts whichever wins\. Full panel in Section[4](https://arxiv.org/html/2607.07724#S4)\.
3. 3\.A fused selection\-plus\-kernel implementation that produceskv\_idxdirectly per Q\-tile and handles the routed expansion in the same code path, keeping kernel dispatch shape\-uniform\. All four sparse policies \(top\-kk, router, Quest, router\-on\-Quest\) run on this same kernel; they differ only in the selection step\. Wall\-time profile crosses dense between3232K and6464K on Qwen2\.5\-7B\-1M \(0\.87×0\.87\\timesdense at6464K,0\.62×0\.62\\timesat128128K\) and between6464K and128128K on the hybrid Qwen3\.6 \(0\.80×0\.80\\timesat128128K\); the crossover regime is characterised via an Amdahl decomposition of the prefill\.
4. 4\.A custom diagnostic benchmark, the*Pointer\-Chase Haystack*\(PCH; Appendix[B](https://arxiv.org/html/2607.07724#A2)\), used during method development to isolate selector quality from model capability\.
5. 5\.A negative\-control result \(LongBench\-v1\) that pins down*when*the router helps: only when the selector’s per\-query budget \(the blocks it keeps\) is small relative to where the answer\-relevant evidence lives\.

## 2Background and related work

### 2\.1Block\-sparse attention

A standard decoder layer\(Vaswani et al\.,[2017](https://arxiv.org/html/2607.07724#bib.bib1)\)maps hidden stateh\(L\)∈ℝN×dmodelh^\{\(L\)\}\\in\\mathbb\{R\}^\{N\\times d\_\{\\text\{model\}\}\}toh\(L\+1\)h^\{\(L\+1\)\}via

h′=h\(L\)\+Attn⁡\(LN⁡\(h\(L\)\)\),h\(L\+1\)=h′\+MLP⁡\(LN⁡\(h′\)\)\.h^\{\\prime\}=h^\{\(L\)\}\+\\operatorname\{Attn\}\\bigl\(\\operatorname\{LN\}\(h^\{\(L\)\}\)\\bigr\),\\qquad h^\{\(L\+1\)\}=h^\{\\prime\}\+\\operatorname\{MLP\}\\bigl\(\\operatorname\{LN\}\(h^\{\\prime\}\)\\bigr\)\.The attention block, for query positioniion headhh, computes

Attn\(x\)i,h=∑j≤iexp⁡\(qi,h⋅kj,h/dhead\)∑j′≤iexp⁡\(qi,h⋅kj′,h/dhead\)vj,h\.\\operatorname\{Attn\}\(x\)\_\{i,h\}\\;=\\;\\sum\_\{j\\leq i\}\\frac\{\\exp\\\!\\bigl\(q\_\{i,h\}\\\!\\cdot\\\!k\_\{j,h\}/\\sqrt\{d\_\{\\text\{head\}\}\}\\bigr\)\}\{\\sum\_\{j^\{\\prime\}\\leq i\}\\exp\\\!\\bigl\(q\_\{i,h\}\\\!\\cdot\\\!k\_\{j^\{\\prime\},h\}/\\sqrt\{d\_\{\\text\{head\}\}\}\\bigr\)\}\\;v\_\{j,h\}\.*Block\-sparse attention*replaces\{j≤i\}\\\{j\\leq i\\\}by a small selected subsetSiS\_\{i\}, chosen per query at inference time on frozen weights\.

### 2\.2The selector landscape

Concrete selectors differ in \(a\) how they pool keys into a per\-block summary and \(b\) how they score blocks against the query, butthey all reduce, at the per\-query selection step, to a top\-kkrule over a block score:

- •SSA\(Subquadratic,[2025](https://arxiv.org/html/2607.07724#bib.bib8)\)\(Subquadratic Sparse Attention\) – per\-Q\-tile top\-kkover mean\-pooled keys,sb=q⋅k¯bs\_\{b\}=q\\cdot\\bar\{k\}\_\{b\}withk¯b=1Bn​∑jkj\\bar\{k\}\_\{b\}=\\frac\{1\}\{B\_\{n\}\}\\sum\_\{j\}k\_\{j\}\. Cheap to compute; mean\-pooling blurs single hot keys, which is exactly the multi\-key NIAH degradation discussed above\.
- •Quest\(Tang et al\.,[2024](https://arxiv.org/html/2607.07724#bib.bib2)\)– per\-block elementwiseKbmin,KbmaxK^\{\\min\}\_\{b\},K^\{\\max\}\_\{b\}summaries; score is an upper bound onmaxj∈b⁡q⋅kj\\max\_\{j\\in b\}q\\cdot k\_\{j\}, computed coordinate\-wise \(see Eq\.[2](https://arxiv.org/html/2607.07724#S3.E2)\)\. Recovers sharp single\-key needles that mean\-pooling loses\.
- •H2O\(Zhang et al\.,[2023](https://arxiv.org/html/2607.07724#bib.bib3)\),SnapKV\(Li et al\.,[2024](https://arxiv.org/html/2607.07724#bib.bib4)\),MInference\(Jiang et al\.,[2024](https://arxiv.org/html/2607.07724#bib.bib5)\)– top\-kkover learned or attention\-history\-based block scores\.
- •NSA\(Yuan et al\.,[2025](https://arxiv.org/html/2607.07724#bib.bib6)\),MoBA\(Lu et al\.,[2025](https://arxiv.org/html/2607.07724#bib.bib7)\)– end\-to\-end learned gating, but still resolves to a top\-kkover block scores at the per\-query selection step\.

What none of these methods do is treat the cutoff as a*decision under uncertainty*: whens\(k−1\)≈s\(k\)s\_\{\(k\-1\)\}\\approx s\_\{\(k\)\}, the selector commits without spending extra budget\. That step is the lever this paper pulls\. Because the lever is on the cutoff \(not on howssis computed\), it composes with any of the scoring backbones above\. In the experiments we evaluate it on top of both the SSA\-style K\-mean backbone and Quest’s K\-max upper bound\.

### 2\.3Long\-context evaluation

The published benchmarks fall into two families\.Synthetic / diagnostic:RULER\(Hsieh et al\.,[2024](https://arxiv.org/html/2607.07724#bib.bib10)\)\(NIAH, VT\), BABILong, MRCR\. Designed for controlled stress tests of long\-context recall\. Their failure modes are interpretable but they are*not*predictive of downstream performance, as HELMET\(Yen et al\.,[2024](https://arxiv.org/html/2607.07724#bib.bib13)\)has documented\.Real\-task:LongBench\(Bai et al\.,[2023](https://arxiv.org/html/2607.07724#bib.bib11),[2024](https://arxiv.org/html/2607.07724#bib.bib12)\), HELMET\(Yen et al\.,[2024](https://arxiv.org/html/2607.07724#bib.bib13)\), NoCha\. These cover multi\-hop QA, summarisation, code, and so on\. LongBench\-v2 in particular has a*medium*split with native prompt lengths typically≥\\geq100K words, designed to stress long\-context selection\. We useRULER NIAH\(synthetic, standardised\) andLongBench\-v1 \+ v2 medium\(real\-task, standardised\) as the headline benchmarks, with a custom diagnostic benchmark \(PCH, Appendix[B](https://arxiv.org/html/2607.07724#A2)\) used during method development\.

## 3Method

We describe the working method end\-to\-end\. Each step is motivated by the previous one and lifts a specific weakness\. The full per\-equation derivation lives in Appendix[A](https://arxiv.org/html/2607.07724#A1); here we give the path the reader needs to follow to understand the experiments\.

### 3\.1Step 1: block scoring

Keys are grouped into contiguous blocks ofBLOCKN=64\\text\{BLOCK\}\_\{N\}=64tokens\. We evaluate the router on top of two block\-scoring rules\.

The first is the SSA\-style mean\-pooled\-key inner product: at layerLL, the selector scores blockb∈\{0,…,NB−1\}b\\in\\\{0,\\dots,N\_\{\\\!B\}\-1\\\}via

s\(L\)​\[i,h,b\]=qi\(L,h\)⋅k¯b\(L,h\)dhead,k¯b\(L,h\)=1BLOCKN​∑j∈block​bkj\(L,h\)\.s^\{\(L\)\}\[i,h,b\]\\;=\\;\\frac\{q\_\{i\}^\{\(L,h\)\}\\cdot\\bar\{k\}\_\{b\}^\{\(L,h\)\}\}\{\\sqrt\{d\_\{\\text\{head\}\}\}\},\\qquad\\bar\{k\}\_\{b\}^\{\(L,h\)\}\\;=\\;\\frac\{1\}\{\\text\{BLOCK\}\_\{N\}\}\\sum\_\{j\\,\\in\\,\\text\{block \}b\}k\_\{j\}^\{\(L,h\)\}\.\(1\)Mean pooling blurs single\-key signals: a block containing one hot key adjacent to noise scores like a block of all noise, and gets dropped\.

The second is Quest’s K\-min/K\-max upper bound\(Tang et al\.,[2024](https://arxiv.org/html/2607.07724#bib.bib2)\), which replacesk¯b\\bar\{k\}\_\{b\}with an elementwise pair\(Kbmin,Kbmax\)\(K^\{\\min\}\_\{b\},K^\{\\max\}\_\{b\}\)and scores

sbquest=∑d=1dheadmax⁡\(qd⋅Kb,dmax,qd⋅Kb,dmin\),s^\{\\text\{quest\}\}\_\{b\}\\;=\\;\\sum\_\{d=1\}^\{d\_\{\\text\{head\}\}\}\\,\\max\\\!\\bigl\(q\_\{d\}\\cdot K^\{\\max\}\_\{b,d\},\\;q\_\{d\}\\cdot K^\{\\min\}\_\{b,d\}\\bigr\),\(2\)a coordinate\-wise upper bound onmaxj∈b⁡q⋅kj\\max\_\{j\\in b\}q\\cdot k\_\{j\}that preserves the single\-key signals the mean averages away\.

We treat the choice between Eqs\. \([1](https://arxiv.org/html/2607.07724#S3.E1)\) and \([2](https://arxiv.org/html/2607.07724#S3.E2)\) as a*backbone hyperparameter*; everything downstream \(per\-tile selection, the cutoff marginσ\\sigma, the trigger\) is identical\.

### 3\.2Step 2: per\-tile selection

A naive per\-row top\-kkemits a boolean keep tensor of shape\[B,H,M,NB\]\[B,H,M,N\_\{\\\!B\}\]that downstream must sort and union across rows of a Q\-tile; this becomes the dominant cost at long context\.

Following the SSA recipe, we groupBLOCKM=64\\text\{BLOCK\}\_\{M\}=64consecutive query rows into a*Q\-tile*ttand make one selection decision per tile, shared by all rows in that tile:

tile\_score​\[t,h,b\]=maxr∈tile​t⁡s​\[r,h,b\],kv\_idx​\[t,h\]=top−⁡k​\(tile\_score​\[t,h,⋅\]\)\.\\text\{tile\\\_score\}\[t,h,b\]\\;=\\;\\max\_\{r\\,\\in\\,\\text\{tile \}t\}s\[r,h,b\],\\qquad\\text\{kv\\\_idx\}\[t,h\]\\;=\\;\\operatorname\*\{top\-\}k\\bigl\(\\text\{tile\\\_score\}\[t,h,\\cdot\]\\bigr\)\.\(3\)The sink block0and the tile’s own block are forced intokv\_idxby adding\+∞\+\\inftyto their scores*before*the top\-kk, which avoids a downstream concat\-and\-dedup\. The output is a single integer tensor of shape\[B,H,Qt,kbudget\]\[B,H,Q\_\{t\},k\_\{\\text\{budget\}\}\]\(whereQt=M/BLOCKMQ\_\{t\}=M/\\text\{BLOCK\}\_\{M\}\), passed straight to the attention kernel without a per\-row inner mask\. Wall time for selection scales asO​\(N​NB/BLOCKM\)O\(NN\_\{\\\!B\}/\\text\{BLOCK\}\_\{M\}\)rather thanO​\(N​NB\)O\(NN\_\{\\\!B\}\)for the per\-row version\.

This is coarser than per\-row by construction: a block that one row in the tile strongly wants can be outranked by a block that several rows weakly want, since the tile score is a max over rows of an inner product\. The router \(Steps 3–5\) addresses this – it spends a controlled amount of extra budget on the tiles where the top\-kkcut was ambiguous, while leaving confident tiles untouched\.

### 3\.3Step 3: the cutoff is a decision – read its uncertainty

The top\-kkin Eq\. \([3](https://arxiv.org/html/2607.07724#S3.E3)\) is a decision: keep block\(k−1\)\(k\{\-\}1\), drop block\(k\)\(k\)\. Its*quality*is naturally measured by how decisive that ranking is\. Lets\(0\)≥s\(1\)≥⋯s\_\{\(0\)\}\\geq s\_\{\(1\)\}\\geq\\cdotsbe the sorted tile scores for tilett, headhh\. We define the normalised cutoff margin

σ​\[t,h\]=s\(k−1\)−s\(k\)s\(0\)−s\(k\)∈\[0,1\]\.\\sigma\[t,h\]\\;=\\;\\frac\{s\_\{\(k\-1\)\}\-s\_\{\(k\)\}\}\{s\_\{\(0\)\}\-s\_\{\(k\)\}\}\\;\\;\\in\\;\[0,1\]\.\(4\)The interpretation is direct:

- •σ→1\\sigma\\to 1– the kept set is far above the rejected tail; the cutoff is unambiguous\.
- •σ→0\\sigma\\to 0– the kept set’s last element is tied with the first rejected element; the cutoff is a coin flip\.

σ\\sigmais computed from a top\-\(k\+1\)\(k\{\+\}1\)partial sort – one extra element beyond the top\-kkthat produceskv\_idx– at no extra asymptotic cost\.

#### Why this is a value\-of\-information signal\.

If we were to commit to the top\-kk, the expected loss from the cutoff decision is bounded by the probability mass the softmax would have placed on the dropped block\. When thekk\-th and\(k\+1\)\(k\{\+\}1\)\-th scores are close, that mass is large; when they are well\-separated, it is small\. The cutoff marginσ\\sigmais a cheap, dimensionless proxy for this expected loss\. The formal anchor is in Appendix[D](https://arxiv.org/html/2607.07724#A4): under a Gaussian noise model on the block scores,σ\\sigmais the dimensionless analogue of the best\-arm\-identification exploration index ofGarivier and Kaufmann \([2016](https://arxiv.org/html/2607.07724#bib.bib14)\); expanding the kept set never increases the identification error, pointwise; and theσ\\sigma\-quantile router satisfies a finite\-sample regret bound against the best trigger rule of the same budget \(Proposition[1](https://arxiv.org/html/2607.07724#Thmproposition1)\)\.

### 3\.4Step 4: aggregateσ\\sigmaacross heads and trigger on the bottomqq\-fraction

We aggregate the per\-head margin to a per\-tile signal via a uniform mean,σ¯​\[t\]=1H​∑h=1Hσ​\[t,h\]\\bar\{\\sigma\}\[t\]=\\frac\{1\}\{H\}\\sum\_\{h=1\}^\{H\}\\sigma\[t,h\]: the tile’s overall decision confidence, low when many heads simultaneously face an ambiguous cutoff, high when most heads agree the cutoff is clean\. Aggregation is necessary – gating directly on the per\-cellσ​\[t,h\]\\sigma\[t,h\]does not work \(Appendix[C](https://arxiv.org/html/2607.07724#A3)\): heads in the same tile are highly correlated, so per\-cell gating is dominated by per\-head idiosyncratic noise that has no relation to whether the tile is actually ambiguous\. Averaging across heads recovers the signal\.

We then turnσ¯\\bar\{\\sigma\}into a binary decision via a per\-layer empirical quantile:

trigger​\[t\]=1​\[σ¯​\[t\]≤quantileq⁡\(σ¯​\[⋅\]\)\]\.\\text\{trigger\}\[t\]\\;=\\;\\mathbb\{1\}\\\!\\Bigl\[\\bar\{\\sigma\}\[t\]\\;\\leq\\;\\operatorname\{quantile\}\_\{q\}\\bigl\(\\bar\{\\sigma\}\[\\,\\cdot\\,\]\\bigr\)\\Bigr\]\.\(5\)qqis the only hyperparameter introduced by the router; intuitively it is the per\-layer fraction of tiles that are deemed risky enough to spend extra budget on\. We use the sameqqacross all layers and find it stable\.

#### Choosingqq\.

The working valueq=0\.40q=0\.40was selected from a sweep on RULER NIAH\-multikey atn=30n=30, ctx=32K \(Appendix[C](https://arxiv.org/html/2607.07724#A3)\), where paired recall rises monotonically withqqover the tested range\{0\.10,0\.20,0\.30,0\.40\}\\\{0\.10,0\.20,0\.30,0\.40\\\}and plateaus near0\.400\.40\. We use the same value unchanged across all benchmarks, models, and contexts in Section[4](https://arxiv.org/html/2607.07724#S4); the sweep was not repeated per task\. A more thoroughρ×q\\rho\\times qPareto study remains open \(Section[4\.5](https://arxiv.org/html/2607.07724#S4.SS5)\)\.

#### Why a quantile, not an absolute threshold\.

The absolute scale ofσ\\sigmavaries across layers \(early layers are more peaked than later layers\)\. A quantile is the simplest layer\-conditional normalisation\. We did test absolute and per\-cell quantile thresholds; they fail \(Appendix[C](https://arxiv.org/html/2607.07724#A3)\)\.

### 3\.5Step 5: expandkv\_idxuniformly

Triggered tiles get aρ×\\rho\\timesexpanded top\-kkovertile\_score\(we useρ=2\\rho=2\)\. The selection operation here is the same top\-kkthat every selector in the literature reduces to;tile\_scoreinherits whichever block\-scoring backbone of §[3](https://arxiv.org/html/2607.07724#S3)is in play \(Eq\. \([1](https://arxiv.org/html/2607.07724#S3.E1)\) or Eq\. \([2](https://arxiv.org/html/2607.07724#S3.E2)\)\), so the same expansion rule applies to both “router” and “router\-on\-Quest” without modification\.

To preserve a fixed kernel dispatch shape across triggered and non\-triggered tiles, we allocate\[B,H,Qt,ρ​kbudget\]\[B,H,Q\_\{t\},\\rho k\_\{\\text\{budget\}\}\]uniformly: triggered tiles fill allρ​kbudget\\rho k\_\{\\text\{budget\}\}slots from the expanded top\-ρ​k\\rho k, while non\-triggered tiles keep their top\-kkand pad the remaining slots with the kernel’sNBN\_\{\\\!B\}\-sentinel \(the exact fill rule is step \(A\.9\) of Appendix[A](https://arxiv.org/html/2607.07724#A1)\)\. The kernel skipsb=NBb=N\_\{\\\!B\}on a fast check, so non\-triggered tiles cost the same as plain top\-kk\. The end\-to\-end forward is otherwise unchanged: a standard FlashAttention\-style online softmax, iterating each tile’skv\_idxinstead of allNBN\_\{\\\!B\}blocks\.

#### Cost accounting\.

The average kept blocks per row iskbudget​\(1\+q​\(ρ−1\)\)k\_\{\\text\{budget\}\}\(1\+q\(\\rho\-1\)\)\. Atq=0\.4,ρ=2q=0\.4,\\rho=2this is1\.4​kbudget1\.4\\,k\_\{\\text\{budget\}\}\. The measured wall\-time cost is sub\-proportional to this block\-count factor: router/top\-kkkernel\-time ratio is≈1\.53\\approx 1\.53at 32K and≈1\.47\\approx 1\.47at 64K \(Section[4\.3](https://arxiv.org/html/2607.07724#S4.SS3)\)\.

## 4Experiments

### 4\.1Setup

Models\.All weights frozen; no retraining, no fine\-tuning\. We evaluate on four instruction\-tuned models spanning three architecture classes \(dense Qwen2\.5, dense Mistral, hybrid\+MoE Qwen3\.6\):

- •*Qwen2\.5\-14B\-Instruct*–4040Q /88KV heads,dhead=128d\_\{\\text\{head\}\}\{=\}128,4848layers, trained at3232K context\. The headline model\.
- •*Qwen2\.5\-7B\-Instruct\-1M*–2828Q /44KV heads,dhead=128d\_\{\\text\{head\}\}\{=\}128,2828layers, trained at11M context\. Same family as the headline model, but a smaller parameter count and a much longer training window; used for 128K\-context experiments \(Section[4\.3](https://arxiv.org/html/2607.07724#S4.SS3)\) since the 14B model’s3232K training window precludes them\.
- •*Mistral\-Nemo\-Instruct\-2407*–3232Q /88KV heads,dhead=128d\_\{\\text\{head\}\}\{=\}128,4040layers, trained at128128K context\. Different architecture family, used to test cross\-family transfer\. The Mistral tokenizer requiresfix\_mistral\_regex=TrueonAutoTokenizer\.from\_pretrained; otherwise the load path is unchanged\.
- •*Qwen3\.6\-35B\-A3B*– hybrid\+MoE:4040layers organised as\(linear\_attn×\\times3→\\tofull\_attn×\\times1\)so only10/4010/40layers run softmax attention; full\-attention heads are1616Q /22KV,dhead=256d\_\{\\text\{head\}\}\{=\}256with partial RoPE on the first6464dims and per\-head QK\-Norm; MLP is sparse MoE \(128128experts, top\-88activated;33B active params of3535B total\); trained at128128K context\.

The router code path and the fused selection\-plus\-kernel implementation run unchanged across all four models; the only model\-dependent code is the tokenizer flag above and, for Qwen3\.6, two harness\-side prompt tweaks \(disablingenable\_thinkingon the chat template and prefilling the assistant turn with"The correct answer is \("for multiple\-choice\)\.

Hardware\.A100\-80GB via Modal for the three Qwen2\.5 / Mistral\-Nemo models; H200 for Qwen3\.6\-35B\-A3B \(does not fit on A100\)\. The Qwen3\.6 image additionally installsflash\-linear\-attentionandcausal\-conv1dso that the3030linear\-attention \(Gated DeltaNet\) layers and their conv branch run on fused kernels\.

Sparse path\.Custom Triton block\-sparse attention kernel with fused per\-tile selection\.

Baselines and policies\.

- •*dense*– FlashAttention\(Dao et al\.,[2022](https://arxiv.org/html/2607.07724#bib.bib9)\)via PyTorch SDPA, the ceiling\.
- •*top\-kk*– per\-tile top\-kkon the K\-mean backbone \(Eq\.[1](https://arxiv.org/html/2607.07724#S3.E1)\)\. SSA\-style baseline; uses only Step 2\.
- •*Quest*– per\-tile top\-kkon the Quest K\-max upper\-bound backbone \(Eq\.[2](https://arxiv.org/html/2607.07724#S3.E2)\)\. Strong baseline on sharp\-needle tasks\.
- •*router*– Steps 2–5 on top of the K\-mean backbone,q=0\.40,ρ=2q=0\.40,\\rho=2\. Uncertainty\-gated expansion only\.
- •*router\-on\-Quest*– Steps 2–5 on top of the Quest K\-max backbone, sameq,ρq,\\rho\. Combines better scoring with budget allocation\.

All four sparse policies share the same kernel and selection code; they differ only in \(a\) which backbone scores blocks and \(b\) whether the router expansion is applied\. Wall\-time and quality numbers are therefore directly comparable across them\.

Selector budget\.Fixed atkbudget=33k\_\{\\text\{budget\}\}=33blocks per tile, corresponding to roughly2​K2\\text\{K\}attended tokens per row \(the PCH diagnostic in Appendix[B](https://arxiv.org/html/2607.07724#A2)uses slightly different budgets, noted in\-place\)\.

#### Metrics\.

The primary metric isaccuracy\(or task score, e\.g\. F1 on extractive QA\): the unconditional success rate – fraction of examples the policy answers correctly\. This is the standard reporting unit in the long\-context literature \(Quest, H2O, SnapKV, MInference, NSA, SSA\)\. Headline tables lead with it\.

The secondary, diagnostic metric ispaired recall: among the examples dense answers correctly, the fraction the sparse policy also answers correctly,

paired recall=\|\{i:densei​correct∧sparsei​correct\}\|/\|\{i:densei​correct\}\|\.\\text\{paired recall\}\\;=\\;\\bigl\|\\\{i:\\text\{dense\}\_\{i\}\\text\{ correct\}\\wedge\\text\{sparse\}\_\{i\}\\text\{ correct\}\\\}\\bigr\|\\,/\\,\\bigl\|\\\{i:\\text\{dense\}\_\{i\}\\text\{ correct\}\\\}\\bigr\|\.We writennfor the total number of examples andnd​c≤nn\_\{dc\}\\leq nfor the number dense answers correctly;nd​cn\_\{dc\}is the denominator of paired recall, notnn\. We report paired recall alongside accuracy because raw accuracy conflates two failure modes – the selector dropped a needed block vs\. the model could not have answered even with full attention – and paired conditions on\{i:densei​correct\}\\\{i:\\text\{dense\}\_\{i\}\\text\{ correct\}\\\}to isolate the first\. On RULER NIAH dense accuracy is1\.001\.00, so the two metrics coincide cell\-by\-cell; on LongBench\-v2 they diverge \(raw accuracy clusters tightly across sparse policies while paired recall ranges widely\), and that divergence is itself analysed in Section[4\.2](https://arxiv.org/html/2607.07724#S4.SS2)\.

Hit rate \(fraction of gold\-evidence blocks the selector keeps\) is used only in the diagnostic experiments of Appendix[B](https://arxiv.org/html/2607.07724#A2), where the gold blocks are known by construction\.

### 4\.2Quality at 32K context across benchmarks

Setup\.Two standardised long\-context benchmarks at fixed context3232K and selector budgetkbudget=33k\_\{\\text\{budget\}\}=33,q=0\.40q=0\.40,ρ=2\\rho=2, on all four panel models:

- •*RULER NIAH\-multikey*\(3 keys\),n=100n=100\. Single\-hot\-key retrieval; dense answers≥96%\\geq 96\\%of items on every model in the panel, so the accuracy ceiling is essentially1\.001\.00and accuracy coincides with paired recall cell\-by\-cell\.
- •*LongBench\-v2 medium*,n=215n=215\(the entire medium\-length subset of the dataset; no further filter\)\. Multi\-hop multiple\-choice \(A/B/C/D\); native item length≥100\\geq 100K words, middle\-truncated to3232K so the selector budget binds\. Dense accuracy varies across models from0\.160\.16to0\.410\.41, so accuracy and paired recall diverge – both are reported\.

Result\.

Table 1:RULER NIAH\-multikey atn=100n=100, ctx = 32K\. Values are unconditionalaccuracy\. Dense accuracy is≥0\.96\\geq 0\.96on every model so accuracy and paired recall coincide cell\-by\-cell \(Nemo differs by≤0\.01\\leq 0\.01on Quest\)\. Backbone column indicates the block\-scoring rule \(K\-mean = Eq\.[1](https://arxiv.org/html/2607.07724#S3.E1); K\-max = Quest, Eq\.[2](https://arxiv.org/html/2607.07724#S3.E2)\)\. “–” indicates a configuration not run\.Table 2:LongBench\-v2 medium atn=215n=215, ctx = 32K\. Each model column reports unconditionalaccuracy\(acc\) andpaired recall\(paired\); dense paired is1\.001\.00by construction\.nd​cn\_\{dc\}on the dense row is the dense\-correct count and the denominator of paired\. Note the metric divergence: sparse accuracies cluster tightly near dense accuracy while paired recall ranges widely – the selector\-quality signal lives entirely in the paired column\.Read\.Two observations across the panel:

*\(i\) The router improves the accuracy of whichever backbone it is applied to, on every model and benchmark in the panel\.*On RULER NIAH: K\-mean lift on Qwen\-14B0\.51→0\.630\.51\\to 0\.63\(McNemarp<0\.01p<0\.01\), Qwen\-7B\-1M0\.28→0\.390\.28\\to 0\.39, Qwen3\.60\.94→0\.980\.94\\to 0\.98; K\-max lift on Qwen\-14B0\.93→0\.980\.93\\to 0\.98, Qwen\-7B\-1M0\.94→1\.000\.94\\to 1\.00, Nemo0\.50→0\.660\.50\\to 0\.66\(p<0\.001p<0\.001\), Qwen3\.60\.85→0\.970\.85\\to 0\.97\. On LongBench\-v2 the same pattern holds on both backbones across all four models \(Table[2](https://arxiv.org/html/2607.07724#S4.T2)\); the LB\-v2 accuracy spread is small on Qwen2\.5 \+ Nemo \(saturated within standard error around dense0\.160\.16–0\.270\.27\) but materially separates on Qwen3\.6 \(0\.270\.27–0\.410\.41\)\.

*\(ii\) Which backbone is the better one depends on the model\.*On Qwen2\.5 \(both sizes\) and Mistral\-Nemo, the K\-max \(Quest\) backbone dominates K\-mean on both benchmarks, sometimes by very large margins \(RULER NIAH: Qwen\-14B Quest beats top\-kkby\+42\+42pp; Qwen\-7B\-1M by\+66\+66pp\)\. On Qwen3\.6 the ordering inverts: K\-mean dominates K\-max \(RULER NIAH top\-kk0\.940\.94vs Quest0\.850\.85; LB\-v2 paired0\.900\.90vs Quest0\.330\.33\)\. The mechanism is structural: Qwen3\.6 applies per\-head RMSNorm toQQandKKpost\-projection \(QK\-Norm\), which regularises away the per\-head magnitude variance the K\-min/K\-max upper bound exploits, so the Quest envelope flattens and mean\-pool becomes at least as informative\. Testable prediction: any future QK\-Norm model should reproduce this reversal\. Combining \(i\) and \(ii\), the best policy is router\-on\-the\-winning\-backbone, model by model: router\-on\-Quest on Qwen2\.5 \+ Nemo, router\-on\-K\-mean on Qwen3\.6\.

Paired recall as a selector diagnostic\.The literature universally reports unconditional accuracy on these benchmarks; we add paired recall as a secondary, selector\-isolated reading\. On RULER NIAH dense accuracy is≥0\.96\\geq 0\.96so the two metrics coincide cell\-by\-cell and the accuracy column in Table[1](https://arxiv.org/html/2607.07724#S4.T1)reads as both\. On LongBench\-v2 medium dense itself answers only0\.160\.16–0\.410\.41of items correctly \(the benchmark is intentionally hard at 32K truncation\), so raw accuracy is dominated by which questions the model can answer at all, not by which blocks the selector kept\. Paired recall conditions on the dense\-correct subset: when the sparse policy makes the same selections that let dense answer, paired is high; when it drops needed blocks, paired drops\. The paired column in Table[2](https://arxiv.org/html/2607.07724#S4.T2)therefore shows the same selector\-quality ordering as accuracy but at higher contrast — e\.g\. on Qwen\-14B accuracy is0\.190\.19–0\.210\.21across all sparse policies \(within standard error of0\.0270\.027\) while paired ranges0\.470\.47–0\.750\.75\. See Table[4](https://arxiv.org/html/2607.07724#S4.T4)for the matched\-budget decomposition of how much of the router lift is budget vs\. selectivity\.

VT\.We do not include RULER VT \(variable tracking, 3\-hop\) as a quality benchmark: dense itself fully solves≤5/100\\leq 5/100items at 32K on every model we tested – Qwen\-14B5/1005/100, Qwen\-7B\-1M0/1000/100, Nemo0/1000/100, and a smoke on the reasoning\-tuned Qwen3\.6\-35B\-A3B \(n=30n=30\) also gave0/300/30– which leaves the paired metric undefined or vanishingly thin on all but Qwen\-14B\. VT at hop=3=3with33distractor chains is harder than the current panel can handle at this parameter scale, including the MoE reasoning model; we defer VT to a future revision with a stronger reasoning base\.

### 4\.3Speed–quality Pareto:3232K→128\\to 128K

Setup\.We fix the model and vary the context length on the two panel models with training windows past 32K: Qwen2\.5\-7B\-Instruct\-1M \(dense softmax\) and Qwen3\.6\-35B\-A3B \(hybrid \+ MoE, QK\-Norm\)\. Configuration unchanged from §[4\.2](https://arxiv.org/html/2607.07724#S4.SS2)\(kbudget=33k\_\{\\text\{budget\}\}=33,q=0\.40q=0\.40,ρ=2\\rho=2\)\. Quality is RULER NIAH\-multikey accuracy atn=100n=100; speed is per\-prefill CUDA\-event wall time atn=8n=8\(decode is the same dense SDPA path for every policy, so policy\-dependent wall time comes purely from prefill\)\.

Table 3:Joint speed\-quality Pareto on the*winning*backbone for each model \(router\-on\-Quest for Qwen2\.5\-7B\-1M, router\-on\-K\-mean for Qwen3\.6, from the QK\-Norm reversal in §[4\.2](https://arxiv.org/html/2607.07724#S4.SS2)\)\. “router accuracy” is RULER NIAH\-multikey atn=100n=100; “wall vs\. dense” is the prefill\-forward wall\-time ratio atn=8n=8\. Dense accuracy is1\.001\.00at every cell\. Bold marks Pareto wins \(router beats dense on speed at quality≥0\.80\\geq 0\.80\)\.Read\.Two observations:

*\(i\) The winning backbone holds most of its accuracy as context grows; the losing one collapses\.*Table[3](https://arxiv.org/html/2607.07724#S4.T3)reports the winning backbone only – in the same data, the*losing*backbone on each model breaks: top\-kkon K\-mean falls0\.28→0\.090\.28\\to 0\.09on Qwen2\.5\-7B\-1M, and Quest on K\-max falls0\.85→0\.500\.85\\to 0\.50on Qwen3\.6\. The QK\-Norm reversal from §[4\.2](https://arxiv.org/html/2607.07724#S4.SS2)is robust at every context, and the gap between winning and losing backbone widens with context on both architectures\.

*\(ii\) Sparse runs slower than dense at short context and faster at long context; the crossover is structural \(Amdahl\)\.*Attention’s share of dense prefill grows with context while the rest\-of\-model floor is invariant\. On Qwen2\.5\-7B\-1M attention shares are31%/45%/63%31\\%/45\\%/63\\%at3232K/6464K/128128K, and the router crosses dense between3232K and6464K\. On Qwen3\.6 the attention share is much smaller \(11%/19%/37%11\\%/19\\%/37\\%\) because30/4030/40layers are linear\-attention rather than softmax, so the crossover shifts right, between6464K and128128K\. The Pareto win materialises on both architectures; the context at which it activates is set by the architecture’s attention share\.

### 4\.4Where the lift comes from: budget vs\. selectivity

The router atq=0\.40,ρ=2q=0\.40,\\rho=2spends on average92\.492\.4effective blocks per tile vs\.6666for Quest atkbudget=33k\_\{\\text\{budget\}\}=33–≈1\.4×\\approx 1\.4\\timesQuest’s budget\. Some of the router’s lift over Quest could therefore come from spending the marginal budget*selectively*\(the design intent\), or simply from spending*more*budget on average\. To separate the two we ran uniform Quest at a budget\-matched control \(kbudget=47k\_\{\\text\{budget\}\}=47, eff\.9494\) and at intermediate budgets on the same RULER NIAH\-multikey examples used in Table[1](https://arxiv.org/html/2607.07724#S4.T1), on three models\.

Table 4:Budget\-match ablation on RULER NIAH\-multikey, ctx=32K,n=100n=100\. Values are accuracy \(= paired recall up to≤0\.01\\leq 0\.01on Nemo where dense=0\.96=0\.96\)\. Effective budget includes the sink and self\-block forced into everykv\_idx\.Questkbudget=47\\text\{Quest\}\_\{k\_\{\\text\{budget\}\}=47\}is the budget\-matched control for the router atq=0\.40q=0\.40\. Qwen\-14B and Qwen3\.6 are near the ceiling on their respective*winning*backbones; Nemo is off\-ceiling\.Read\.The router’s lift over Questkbudget=33\{\}\_\{k\_\{\\text\{budget\}\}=33\}splits into a*budget*term \(Questkbudget=47\\text\{Quest\}\_\{k\_\{\\text\{budget\}\}=47\}minusQuestkbudget=33\\text\{Quest\}\_\{k\_\{\\text\{budget\}\}=33\}, what uniform expansion buys\) and a*selectivity*term \(router minusQuestkbudget=47\\text\{Quest\}\_\{k\_\{\\text\{budget\}\}=47\}\)\. The split is regime\-dependent:

- •*Qwen\-14B \(near\-ceiling on its winning backbone\):*\+5\+5pp total=\+3=\+3budget\+\+2\+\+2selectivity \(≈2/3\\approx 2/3budget,1/31/3selectivity\)\. Uniform Quest plateaus at0\.970\.97fromkbudget=52k\_\{\\text\{budget\}\}=52; the router lands one point above the asymptote – directionally consistent, inside noise atn=100n=100\(McNemar router vs\. matched\-Quest:33vs\.11discordant,p=0\.625p=0\.625\)\.
- •*Nemo \(off\-ceiling\):*\+15\+15pp=\+6=\+6budget\+\+9\+\+9selectivity \(≈2/5\\approx 2/5budget,3/53/5selectivity\)\. Uniform Quest does not plateau in the tested range, and the router at eff\.92\.492\.4Pareto\-dominates uniformQuestkbudget=66\\text\{Quest\}\_\{k\_\{\\text\{budget\}\}=66\}at eff\.132132by\+6\+6pp – a separation no amount of uniform budget expansion within this range can close \(one\-sided sign testp≤0\.004p\\leq 0\.004\)\.
- •*Qwen3\.6 \(near\-ceiling on K\-mean, the*other*backbone\):*on the K\-max sweep the model looks like Qwen\-14B \(near\-ceiling, budget\-dominated\), but K\-max is the wrong backbone here – uniform Quest at2×2\\timeseff\. does not catch router\-on\-K\-mean \(0\.980\.98at1×1\\timeseff\.\)\. The Pareto domination on Qwen3\.6 is across backbones, not across budgets\.

Reading the three columns together: when uncertainty at the cutoff is rare \(sharp scores\), budget expansion dominates the lift; when it is common \(diffuse scores\), selective expansion dominates\. The three columns bracket the regime in which the router operates\.

When does the lift activate? LongBench\-v1 as a negative control\.The decomposition above answers*how*the router’s lift breaks down once it exists; it does not say*when*it exists at all\. To pin that down we ran the LongBench\-v1 panel \(musique\-2hop, musique\-4hop, narrativeqa, qasper;n=30n=30, 32K context,kbudget=33k\_\{\\text\{budget\}\}=33,q=0\.40q=0\.40\)\. LB\-v1 native prompt lengths are55–3030K, so truncation to 32K is essentially a no\-op and the selector budget is generous relative to needle density\. Result: dense, top\-kk, and router essentially tie on F1 across all four tasks; top\-kkalready preserves100%100\\%of dense\-correct examples on musique\-2hop, musique\-4hop, and qasper – no headroom\. One small lift on narrativeqa \(router paired1\.001\.00vs\. top\-kk0\.890\.89, F1\+3\.1\+3\.1pp\)\.The router’s lift activates only when the per\-query budget is small relative to where the answer\-relevant evidence lives\.LB\-v2 medium puts the budget under pressure \(100100K source→32\\to 32K window\); LB\-v1 does not\.

### 4\.5Limitations

1. 1\.Per\-tile granularity\.Per\-tile selection structurally loses needles that one row strongly wants but other rows in the tile ignore\. The router recovers part of this gap but not all\. A per\-row scatter\-into\-bitmap kernel would close the rest of the gap without re\-materialising the\[B,H,M,NB\]\[B,H,M,N\_\{\\\!B\}\]keep tensor; not yet built\.
2. 2\.Cross\-architecture coverage\.The panel covers four models from three architecture classes \(dense Qwen2\.5, dense Mistral, hybrid\+MoE Qwen3\.6\); broader coverage on additional families \(Llama\-3, MiniMax, Gemma\) and additional standardised benchmarks \(e\.g\. HELMET\-RAG\) would strengthen the cross\-family claim\.
3. 3\.Engineering improvements\.Several speed optimisations remain on the table \(a fused MoE kernel on hybrid\+MoE; block\-selection fused into the attention kernel for≥256\\geq 256K; per\-row scatter for the per\-tile granularity above\)\. All are engineering work, not method changes; we leave them to future iterations\.
4. 4\.Hyperparameter sweeps incomplete\.The trigger fractionq=0\.40q=0\.40and expansion factorρ=2\\rho=2were selected on a RULER NIAH\-multikey sweep atn=30n=30and used unchanged everywhere; we did not re\-sweep per task or per architecture\. The diagnostic in Appendix[D\.4](https://arxiv.org/html/2607.07724#A4.SS4)opens a complementary path:*predicting*the Bayes\-optimalq⋆q^\{\\star\}per task from the measurable noise scale rather than searching for it; this is not yet wired into the harness\.
5. 5\.What’s bounded, what’s owed\.Appendix[D](https://arxiv.org/html/2607.07724#A4)proves a finite\-sample regret bound for theσ\\sigma\-quantile trigger vs\. the best trigger rule of the same budget, under Gaussian noise on block scores \(Proposition[1](https://arxiv.org/html/2607.07724#Thmproposition1)\)\. The bound is exponentially tight when secondary score spacings dominate the noise scale, but degrades to a trivial cap in the noise\-dominated regime – which is where our measured diagnostics place Qwen\-14B at 32K, so the closed\-form certificate is currently weak precisely where we operate; the pending sharp path is a Monte\-Carlo evaluation of the regret from the same score dump \(§[D\.4](https://arxiv.org/html/2607.07724#A4.SS4)\)\. The bound is also a*top\-kkidentification*regret; converting it to an attention\-output bound requires a softmax\-sensitivity step that we sketch but do not prove \(§[D\.5](https://arxiv.org/html/2607.07724#A4.SS5)\)\.

## Conclusion

We presented an uncertainty\-gated value\-of\-information*router*for block\-sparse attention selectors: a per\-tile cutoff\-margin signalσ\\sigmatriggers aρ×\\rho\\timesexpansion of the kept set on the bottomqq\-fraction of tiles per layer, independent of how block scores are computed and stackable with existing scoring backbones such as Quest\.

The router delivers measurable accuracy gains across the panel: on every model and benchmark tested, it lifts whichever scoring backbone wins on that model\. Which backbone that is proves model\-dependent – QK\-Norm \(Qwen3\.6\) flips the winner from Quest’s K\-max to the SSA\-style K\-mean – but the composition claim holds in every cell of the panel\. The lift activates only when the selector budget binds relative to where the answer\-relevant evidence lives: LongBench\-v1, whose native prompts fit comfortably in the context window, shows no headroom and serves as the negative control\. At128128K the router preserves0\.810\.81and0\.890\.89of dense accuracy on Qwen2\.5\-7B\-1M and Qwen3\.6 respectively, while running at0\.62×0\.62\\timesand0\.80×0\.80\\timesdense wall time\.

Three natural extensions: third\-architecture validation \(Llama\-3 / MiniMax / Gemma\) and HELMET\-RAG; a fused MoE kernel on hybrid\+MoE to push Qwen3\.6’s128128K speed ratio toward0\.5×0\.5\\timesdense; and composing the BAI top\-kkidentification bound of Appendix[D](https://arxiv.org/html/2607.07724#A4)with an attention\-output sensitivity step\.

## Appendix AStep\-by\-step derivation ofσ\\sigmaand the trigger rule

This appendix walks through Section[3](https://arxiv.org/html/2607.07724#S3)at one equation per step, so a reader who has not followed the body can reproduce the implementation\. Notation:BBbatch,HHheads,MMquery rows,NNkeys,NB=N/BLOCKNN\_\{\\\!B\}=N/\\text\{BLOCK\}\_\{N\}key blocks,Qt=M/BLOCKMQ\_\{t\}=M/\\text\{BLOCK\}\_\{M\}query tiles,dhead=dmodel/Hd\_\{\\text\{head\}\}=d\_\{\\text\{model\}\}/H\.

#### \(A\.1\) Block\-pooled keys\.

k¯b\(L,h\)=1BLOCKN​∑j∈block​bkj\(L,h\),b∈\{0,…,NB−1\}\.\\bar\{k\}\_\{b\}^\{\(L,h\)\}\\;=\\;\\frac\{1\}\{\\text\{BLOCK\}\_\{N\}\}\\sum\_\{j\\,\\in\\,\\text\{block \}b\}k\_\{j\}^\{\(L,h\)\},\\qquad b\\in\\\{0,\\dots,N\_\{\\\!B\}\-1\\\}\.\(6\)

#### \(A\.2\) Per\-row block scores\.

s\(L\)​\[i,h,b\]=qi\(L,h\)⋅k¯b\(L,h\)dhead,i∈\{0,…,M−1\}\.s^\{\(L\)\}\[i,h,b\]\\;=\\;\\frac\{q\_\{i\}^\{\(L,h\)\}\\cdot\\bar\{k\}\_\{b\}^\{\(L,h\)\}\}\{\\sqrt\{d\_\{\\text\{head\}\}\}\},\\qquad i\\in\\\{0,\\dots,M\-1\\\}\.\(7\)

#### \(A\.3\) Per\-tile reduction\.

For Q\-tilettcovering rows\{t​BLOCKM,…,\(t\+1\)​BLOCKM−1\}\\\{t\\text\{BLOCK\}\_\{M\},\\dots,\(t\{\+\}1\)\\text\{BLOCK\}\_\{M\}\{\-\}1\\\},

tile\_score​\[t,h,b\]=maxr∈tile​t⁡s​\[r,h,b\]\.\\text\{tile\\\_score\}\[t,h,b\]\\;=\\;\\max\_\{r\\,\\in\\,\\text\{tile \}t\}s\[r,h,b\]\.\(8\)We use max\-pool because dropping a block the most aggressive row wants is worse than dropping one only weak rows want; mean\-pool was tested and underperforms\.

#### \(A\.4\) Forcing the sink and self\-block\.

Letbsink=0b\_\{\\text\{sink\}\}=0andbself​\(t\)=⌊t​BLOCKM/BLOCKN⌋b\_\{\\text\{self\}\}\(t\)=\\lfloor t\\text\{BLOCK\}\_\{M\}/\\text\{BLOCK\}\_\{N\}\\rfloor\(the key block containing the tile’s own rows\)\. We add\+∞\+\\inftyto the corresponding entries oftile\_scorebefore the top\-kk:

tile\_score~​\[t,h,b\]=tile\_score​\[t,h,b\]\+∞⋅𝟙​\[b∈\{bsink,bself​\(t\)\}\]\.\\widetilde\{\\text\{tile\\\_score\}\}\[t,h,b\]\\;=\\;\\text\{tile\\\_score\}\[t,h,b\]\+\\infty\\cdot\\mathbb\{1\}\\\!\\bigl\[b\\in\\\{b\_\{\\text\{sink\}\},b\_\{\\text\{self\}\}\(t\)\\\}\\bigr\]\.\(9\)This guarantees both blocks land in the kept set without a separate concat\-and\-dedup\.

#### \(A\.5\) Top\-\(k\+1\)\(k\{\+\}1\)and the sorted prefix\.

We compute

idxsorted​\[t,h,⋅\],s~sorted​\[t,h,⋅\]=top−⁡\(k\+1\)⁡\(tile\_score~​\[t,h,⋅\]\),\\text\{idx\}\_\{\\text\{sorted\}\}\[t,h,\\,\\cdot\\,\],\\;\\;\\widetilde\{s\}\_\{\\text\{sorted\}\}\[t,h,\\,\\cdot\\,\]\\;=\\;\\operatorname\*\{top\-\}\{\(k\{\+\}1\)\}\\\!\\bigl\(\\widetilde\{\\text\{tile\\\_score\}\}\[t,h,\\cdot\]\\bigr\),\(10\)in descending order\. The firstkkindices arekv\_idx​\[t,h\]\\text\{kv\\\_idx\}\[t,h\]; the\(k\+1\)\(k\{\+\}1\)\-th score is what we need forσ\\sigma\.

#### \(A\.6\) Normalised cutoff margin\.

σ​\[t,h\]=s~sorted​\[t,h,k−1\]−s~sorted​\[t,h,k\]s~sorted​\[t,h,0\]−s~sorted​\[t,h,k\]\.\\sigma\[t,h\]\\;=\\;\\frac\{\\widetilde\{s\}\_\{\\text\{sorted\}\}\[t,h,k\{\-\}1\]\-\\widetilde\{s\}\_\{\\text\{sorted\}\}\[t,h,k\]\}\{\\widetilde\{s\}\_\{\\text\{sorted\}\}\[t,h,0\]\-\\widetilde\{s\}\_\{\\text\{sorted\}\}\[t,h,k\]\}\.\(11\)Edge cases: if the denominator is zero \(all top scores tied\) we setσ=0\\sigma=0\(treat as maximally uncertain\);\+∞\+\\inftyentries from \(A\.4\) participate in the sorted prefix on top, sos~sorted​\[t,h,0\]\\widetilde\{s\}\_\{\\text\{sorted\}\}\[t,h,0\]is always\+∞\+\\infty– in practice we therefore exclude the forced indices from theσ\\sigmacomputation and apply Eq\. \(A\.6\) to the remaining tile scores\.

#### \(A\.7\) Head aggregation\.

σ¯​\[t\]=1H​∑h=1Hσ​\[t,h\]\.\\bar\{\\sigma\}\[t\]\\;=\\;\\frac\{1\}\{H\}\\sum\_\{h=1\}^\{H\}\\sigma\[t,h\]\.\(12\)

#### \(A\.8\) Per\-layer empirical quantile\.

τ\(L\)=quantileq⁡\(σ¯​\[⋅\]\),trigger​\[t\]=1​\[σ¯​\[t\]≤τ\(L\)\]\.\\tau^\{\(L\)\}\\;=\\;\\operatorname\{quantile\}\_\{q\}\\\!\\bigl\(\\bar\{\\sigma\}\[\\,\\cdot\\,\]\\bigr\),\\qquad\\text\{trigger\}\[t\]\\;=\\;\\mathbb\{1\}\\\!\\bigl\[\\bar\{\\sigma\}\[t\]\\leq\\tau^\{\(L\)\}\\bigr\]\.\(13\)

#### \(A\.9\) Padded expansion\.

Working budgetkbudgetk\_\{\\text\{budget\}\}, expansion factorρ\\rho, sentinelb⊥=NBb\_\{\\bot\}=N\_\{\\\!B\}\. We allocatekv\_idxfinal∈ℤB×H×Qt×ρ​kbudget\\text\{kv\\\_idx\}\_\{\\text\{final\}\}\\in\\mathbb\{Z\}^\{B\\times H\\times Q\_\{t\}\\times\\rho k\_\{\\text\{budget\}\}\}and fill

kv\_idxfinal​\[t,h\]=\{\[idxsorted​\[t,h,0\],…,idxsorted​\[t,h,ρ​k−1\]\]trigger​\[t\]=1,\[idxsorted​\[t,h,0\],…,idxsorted​\[t,h,k−1\],b⊥,…,b⊥\]otherwise\.\\text\{kv\\\_idx\}\_\{\\text\{final\}\}\[t,h\]\\;=\\;\\begin\{cases\}\\bigl\[\\text\{idx\}\_\{\\text\{sorted\}\}\[t,h,0\],\\dots,\\text\{idx\}\_\{\\text\{sorted\}\}\[t,h,\\rho k\{\-\}1\]\\bigr\]&\\text\{trigger\}\[t\]=1,\\\\\[2\.5pt\] \\bigl\[\\text\{idx\}\_\{\\text\{sorted\}\}\[t,h,0\],\\dots,\\text\{idx\}\_\{\\text\{sorted\}\}\[t,h,k\{\-\}1\],\\,b\_\{\\bot\},\\dots,b\_\{\\bot\}\\bigr\]&\\text\{otherwise\.\}\\end\{cases\}\(14\)The triggered case requires a top\-ρ​k\\rho krather than a top\-\(k\+1\)\(k\{\+\}1\), which we batch with the top\-\(k\+1\)\(k\{\+\}1\)used forσ\\sigmaas a single top\-max⁡\(ρ​k,k\+1\)=ρ​k\\max\(\\rho k,k\{\+\}1\)=\\rho kwhenρ≥2\\rho\\geq 2\.

#### \(A\.10\) Kernel iteration\.

The Triton kernel’s inner loop iterateskv\_idxfinal​\[t,h\]\\text\{kv\\\_idx\}\_\{\\text\{final\}\}\[t,h\]and skips any entry equal tob⊥b\_\{\\bot\}on a single compare\. The online softmax is unchanged from FlashAttention\.

#### Average attended budget\.

A row in tilettattends tokkblocks iftrigger​\[t\]=0\\text\{trigger\}\[t\]=0andρ​k\\rho kiftrigger​\[t\]=1\\text\{trigger\}\[t\]=1\. The expected per\-row attended budget is therefore

𝔼​\[attended\]=kbudget⋅\(1\+q​\(ρ−1\)\),\\mathbb\{E\}\[\\text\{attended\}\]\\;=\\;k\_\{\\text\{budget\}\}\\cdot\\bigl\(1\+q\(\\rho\-1\)\\bigr\),\(15\)which atq=0\.4,ρ=2q=0\.4,\\rho=2is1\.4​kbudget1\.4\\,k\_\{\\text\{budget\}\}\. This is the headline cost of the router\.

## Appendix BPointer\-Chase Haystack: construction and diagnostic results

The Pointer\-Chase Haystack \(PCH\) is a custom diagnostic benchmark we built during method development to isolate selector quality from model capability\. We retain it here for transparency on the development process; the headline empirical results in Section[4](https://arxiv.org/html/2607.07724#S4)use the standardised benchmarks RULER NIAH and LongBench\-v2 medium\.

Design goals\.We needed a task that \(i\) has a dense\-attention ceiling of≈100%\\approx 100\\%, so failures are unambiguously selector failures, and \(ii\) is*query\-latent*, meaning the relevant tokens cannot be identified by surface matching against the query alone\. RULER NIAH multi\-key satisfies \(i\) but not \(ii\); RULER VT satisfies \(ii\) but not \(i\) \(Qwen 7B fails it\)\.

Construction\.A pointer chase of depthhhis a sequence of indexed entries

> EntryID0\\text\{ID\}\_\{0\}: continue at entryID1\\text\{ID\}\_\{1\}\. EntryID1\\text\{ID\}\_\{1\}: continue at entryID2\\text\{ID\}\_\{2\}\. ⋮\\vdots EntryIDh\\text\{ID\}\_\{h\}: the recorded value isVV\.

with allIDk\\text\{ID\}\_\{k\}drawn from a private namespace\. A PCH instance contains one gold chain plusDDdistractor chains, all entries shuffled into a haystack of target token lengthNN\. The query givesID0\\text\{ID\}\_\{0\}and asks forVV\. Hop0is query\-identifiable; hops1,…,h1,\\dots,hare query\-latent\. Compounding is by construction: missing entrykkpermanently blocks every entryk\+1,…,hk\{\+\}1,\\dots,h\.

Grounding\.The pointer chase is the canonical communication\-complexity problem for unavoidablekk\-round sequential dependency: it provably cannot be collapsed into fewer rounds\. Every per\-hop operation is trivial \(follow a link, read a value\), keeping a capable dense model at the∼100%\\sim 100\\%ceiling\.

Metric\.We report*hit rate*: the fraction of gold\-chain blocks the selector keeps inkv\_idx\. Hit rate is well\-defined only when the gold blocks are known by construction, which is the case here\.

Result\.Table[5](https://arxiv.org/html/2607.07724#A2.T5)reports hit rate ath=3h=3, fixedkbudget=40k\_\{\\text\{budget\}\}=40blocks/row \(32K, 64K\) and2929\(8K\), on Qwen2\.5\-14B\-Instruct\.

Table 5:PCH hop=3 hit rate\. Hit rate is the fraction of gold\-chain blocks the selector keeps\. Steady\-state wall time is reported for orientation; the main efficiency analysis is in Section[4\.3](https://arxiv.org/html/2607.07724#S4.SS3)\.Router lifts hit rate over plain top\-kkby\+12\+12–\+15\+15pp across scales \(and recovers dense’s1\.001\.00at 8K\)\. The gain is monotonic with context length, consistent with the structural cost of per\-tile selection increasing withNNthat motivates the router in the first place\.

## Appendix CPer\-row vs\. per\-cell aggregation: why heads are pooled before thresholding

We tested two natural alternatives to Eq\. \([5](https://arxiv.org/html/2607.07724#S3.E5)\) and they both fail\. We report them here so the design choice in Section[3](https://arxiv.org/html/2607.07724#S3)can be understood as a tested decision rather than an arbitrary one\.

#### Per\-cell quantile\.

Trigger on𝟙​\[σ​\[t,h\]≤quantileq⁡\(σ​\[⋅,⋅\]\)\]\\mathbb\{1\}\\\!\\bigl\[\\sigma\[t,h\]\\leq\\operatorname\{quantile\}\_\{q\}\(\\sigma\[\\,\\cdot\\,,\\cdot\]\)\\bigr\], i\.e\. flag the bottom\-qq*cells*, not tiles\. Result: the rescue set is dominated by per\-head idiosyncratic noise; the gain over plain top\-kkvanishes for everyqqwe tested\. Mechanism: heads in the same tile are highly correlated \(PCH cross\-head agreement matches peakedness byte\-for\-byte across candidates\), so per\-headσ\\sigmavalues carry mostly the same signal plus head\-private noise; quantile\-by\-cell amplifies the noise\.

#### Absolute thresholdτ\\tauonσ\\sigma\.

Trigger on𝟙​\[σ​\[t,h\]≤τ\]\\mathbb\{1\}\\\!\\bigl\[\\sigma\[t,h\]\\leq\\tau\\bigr\]for a fixedτ\\tau\. Result: works at one layer’sσ\\sigmadistribution but not across the stack; we observe systematic distribution shift early\-vs\-late layers\. A per\-layer quantile is the simplest layer\-conditional fix and dominated the absolute\-τ\\tausweep\.

#### Row\-grainqqsweep\.

On PCH at 32K, row\-grainq=0\.10q=0\.10reaches near\-top\-kkeffective budget at hit rate0\.780\.78, vs\. plain top\-kkat0\.630\.63\. On RULER NIAH\-multikey atn=30n=30, ctx=32K \(fixedkbudget=33k\_\{\\text\{budget\}\}=33, kernel\-v2, row\-grain quantile router\), we sweptq∈\{0\.10,0\.20,0\.30,0\.40\}q\\in\\\{0\.10,0\.20,0\.30,0\.40\\\}and observed paired recall0\.38,0\.35,0\.47,0\.530\.38,0\.35,0\.47,0\.53respectively, vs\. top\-kkbaseline0\.350\.35\. The trend is monotonic inqqover the tested range and plateaus nearq=0\.40q=0\.40\. We useq=0\.40q=0\.40for the headline results on RULER and LB\-v2; the regime whereqqshould differ across tasks \(because the budget\-vs\-evidence Pareto shifts\) is consistent with the smaller value chosen for PCH, but a per\-task sweep on LB\-v2 medium remains open\.

## Appendix Dσ\\sigmaas a best\-arm\-identification exploration index

The Method section introducesσ\\sigmaas a value\-of\-information proxy and validates it empirically\. This appendix gives the formal anchor: under a Gaussian noise model on the per\-block relevance signal,σ\\sigmais a dimensionless exploration index in the sense of best\-arm identification \(BAI\); expanding the kept set never increases the identification error, pointwise; and theσ\\sigma\-quantile router admits a finite\-TTregret bound against the best trigger rule of the same budget, with explicit constants that are exponentially small when the score spacings outside the cutoff dominate the noise scale, and an honest trivial\-cap fallback when they do not\.

### D\.1The BAI intuition that motivatesσ\\sigma

The classical fixed\-confidence top\-kkidentification problem ofKaufmann et al\. \([2016](https://arxiv.org/html/2607.07724#bib.bib15)\); Garivier and Kaufmann \([2016](https://arxiv.org/html/2607.07724#bib.bib14)\)asks forS^\\widehat\{S\}such thatℙ​\(S^≠S∗​\(μ\)\)≤δ\\mathbb\{P\}\(\\widehat\{S\}\\neq S^\{\*\}\(\\mu\)\)\\leq\\deltausing as few arm pulls as possible\. The asymptotically optimal Track\-and\-Stop algorithm pullsb⋆=arg⁡maxb⁡\[wb⋆​\(μ^\)−Tb/T\]b^\{\\star\}=\\arg\\max\_\{b\}\[w^\{\\star\}\_\{b\}\(\\hat\{\\mu\}\)\-T\_\{b\}/T\]where the optimal allocationw⋆w^\{\\star\}solves a max\-min programme whose minimiser concentrates on the cutoff pair\(k−1,k\)\(k\{\-\}1,k\): any alternative world in which the top\-kkdiffers fromμ^\\hat\{\\mu\}’s is achieved most cheaply by perturbing those two arms\. Under sub\-Gaussian noise, the per\-step exploration value at the cutoff scales as

VTaS​\(μ^\)∝\(μ^\(k−1\)−μ^\(k\)\)−2,V\_\{\\text\{TaS\}\}\(\\hat\{\\mu\}\)\\;\\propto\\;\\bigl\(\\hat\{\\mu\}\_\{\(k\-1\)\}\-\\hat\{\\mu\}\_\{\(k\)\}\\bigr\)^\{\-2\},\(16\)since KL divergence between Gaussians of equal variance scales as the squared mean gap\.Small empirical gap at the cutoff⟺\\Longleftrightarrowhigh exploration value⟺\\Longleftrightarrowallocate budget here\.

Our setting differs from classical BAI in three ways: \(a\) we observe one sample per arm per tile, not a sequence; \(b\) the budget decision is binary per tile \(expand toρ​k\\rho kor keep atkk\), not a continuous allocation; \(c\) we do not knowτ\\tau\. The dimensionless normalisationσt=gt/rt\\sigma\_\{t\}=g\_\{t\}/r\_\{t\}– withgt:=st,\(k−1\)−st,\(k\)g\_\{t\}:=s\_\{t,\(k\-1\)\}\-s\_\{t,\(k\)\}andrt:=st,\(0\)−st,\(k\)r\_\{t\}:=s\_\{t,\(0\)\}\-s\_\{t,\(k\)\}– replaces the missingτ\\tauwith the per\-tile spread, on the heuristic that the spread concentrates around its layer\-conditional mean and absorbsτ\\tau\. An absolute\-threshold rule \(Appendix[C](https://arxiv.org/html/2607.07724#A3)\) fails because the marginal distribution ofssshifts across layers;σ\\sigmais the simplest layer\-conditional and tile\-conditional normaliser\.

The rest of this appendix turns this heuristic into a regret bound\.

### D\.2Setting and policies

For each Q\-tilet∈\[T\]t\\in\[T\]\(we suppress the head index throughout\) and blockb∈\[NB\]b\\in\[N\_\{\\\!B\}\],

st,b=μt,b\+ηt,b,ηt,b∼iid𝒩​\(0,τ2\)\.s\_\{t,b\}\\;=\\;\\mu\_\{t,b\}\+\\eta\_\{t,b\},\\qquad\\eta\_\{t,b\}\\stackrel\{\{\\scriptstyle\\text\{iid\}\}\}\{\{\\sim\}\}\\mathcal\{N\}\(0,\\tau^\{2\}\)\.\(17\)We work in the compound\-decision \(empirical\-Bayes\) frame: the relevance vectorμt∈ℝNB\\mu\_\{t\}\\in\\mathbb\{R\}^\{N\_\{\\\!B\}\}is modelled as drawn from an improper flat prior, so that conditional on the observed scores,μt∣st∼𝒩​\(st,τ2​I\)\\mu\_\{t\}\\mid s\_\{t\}\\sim\\mathcal\{N\}\(s\_\{t\},\\tau^\{2\}I\)\. Sort the scores of tilettin decreasing order,st,\(0\)≥st,\(1\)≥⋯s\_\{t,\(0\)\}\\geq s\_\{t,\(1\)\}\\geq\\cdots, and define

gt:=st,\(k−1\)−st,\(k\),Gt:=st,\(k−1\)−st,\(ρ​k\),rt:=st,\(0\)−st,\(k\),σt:=gt/rt\.g\_\{t\}:=s\_\{t,\(k\-1\)\}\-s\_\{t,\(k\)\},\\qquad G\_\{t\}:=s\_\{t,\(k\-1\)\}\-s\_\{t,\(\\rho k\)\},\\qquad r\_\{t\}:=s\_\{t,\(0\)\}\-s\_\{t,\(k\)\},\\qquad\\sigma\_\{t\}:=g\_\{t\}/r\_\{t\}\.\(18\)gtg\_\{t\}is the cutoff gap the router reads;Gt≥gtG\_\{t\}\\geq g\_\{t\}is the*expansion gap*, from the last kept rank down to the first rank excluded even after expansion\. \(An earlier draft used the local spacingst,\(ρ​k−1\)−st,\(ρ​k\)s\_\{t,\(\\rho k\-1\)\}\-s\_\{t,\(\\rho k\)\}here; that is the wrong quantity – the error event of the expanded set is governed by the cumulative gapGtG\_\{t\}, see Lemma[1](https://arxiv.org/html/2607.07724#Thmlemma1)\.\) LetS^m​\(st\):=\{b:st,b≥st,\(m−1\)\}\\widehat\{S\}\_\{m\}\(s\_\{t\}\):=\\\{b:s\_\{t,b\}\\geq s\_\{t,\(m\-1\)\}\\\}be the empirical top\-mmset andS∗​\(μt\)S^\{\*\}\(\\mu\_\{t\}\)the true top\-kkset ofμt\\mu\_\{t\}\. The per\-tile identification error of a kept setS^\\widehat\{S\}isℰt​\(S^\):=ℙ​\(S^⊉S∗​\(μt\)∣st\)\\mathcal\{E\}\_\{t\}\(\\widehat\{S\}\):=\\mathbb\{P\}\(\\widehat\{S\}\\not\\supseteq S^\{\*\}\(\\mu\_\{t\}\)\\mid s\_\{t\}\), and the per\-tile*expansion value*is

Vt∗:=ℰt​\(S^k​\(st\)\)−ℰt​\(S^ρ​k​\(st\)\)\.V^\{\*\}\_\{t\}\\;:=\\;\\mathcal\{E\}\_\{t\}\(\\widehat\{S\}\_\{k\}\(s\_\{t\}\)\)\-\\mathcal\{E\}\_\{t\}\(\\widehat\{S\}\_\{\\rho k\}\(s\_\{t\}\)\)\.\(19\)
###### Observation 1\(expansion never harms, pointwise\)\.

S^k⊆S^ρ​k\\widehat\{S\}\_\{k\}\\subseteq\\widehat\{S\}\_\{\\rho k\}, so\{S^ρ​k⊉S∗\}⊆\{S^k⊉S∗\}\\\{\\widehat\{S\}\_\{\\rho k\}\\not\\supseteq S^\{\*\}\\\}\\subseteq\\\{\\widehat\{S\}\_\{k\}\\not\\supseteq S^\{\*\}\\\}, henceVt∗≥0V^\{\*\}\_\{t\}\\geq 0for every realisation ofsts\_\{t\}\. No distributional assumption is needed\.

A*trigger policy*choosesE⊆\[T\]E\\subseteq\[T\]with\|E\|=⌈q​T⌉\|E\|=\\lceil qT\\rceil, keepsS^ρ​k​\(st\)\\widehat\{S\}\_\{\\rho k\}\(s\_\{t\}\)fort∈Et\\in EandS^k​\(st\)\\widehat\{S\}\_\{k\}\(s\_\{t\}\)otherwise; its riskℰ​\(Π\)\\mathcal\{E\}\(\\Pi\)is the average over tiles of the per\-tile error of the kept set\. The action space is deliberately restricted to\{empirical top\-​k,empirical top\-​ρ​k\}\\\{\\text\{empirical top\-\}k,\\ \\text\{empirical top\-\}\\rho k\\\}: a fully Bayes\-optimal agent could also re\-select*which*blocks to keep from the posterior, and we claim nothing about that larger class\. Within the class, minimisingℰ​\(Π\)\\mathcal\{E\}\(\\Pi\)is equivalent to maximising∑t∈EVt∗\\sum\_\{t\\in E\}V^\{\*\}\_\{t\}, so the optimal trigger isΠq∗:=\\Pi\_\{q\}^\{\*\}:=the⌈q​T⌉\\lceil qT\\rceiltiles of largestVt∗V^\{\*\}\_\{t\}\(Neyman–Pearson onV∗V^\{\*\}\)\. The router is theσ\\sigma\-quantile ruleΠqσ:=\\Pi\_\{q\}^\{\\sigma\}:=the⌈q​T⌉\\lceil qT\\rceiltiles of smallestσt\\sigma\_\{t\}\. The regret is

ℛq:=ℰ​\(Πqσ\)−ℰ​\(Πq∗\)=1T​\[∑t∈Πq∗Vt∗−∑t∈ΠqσVt∗\]≥0\.\\mathcal\{R\}\_\{q\}\\;:=\\;\\mathcal\{E\}\(\\Pi\_\{q\}^\{\\sigma\}\)\-\\mathcal\{E\}\(\\Pi\_\{q\}^\{\*\}\)\\;=\\;\\frac\{1\}\{T\}\\Bigl\[\\sum\_\{t\\in\\Pi\_\{q\}^\{\*\}\}V^\{\*\}\_\{t\}\\;\-\\;\\sum\_\{t\\in\\Pi\_\{q\}^\{\\sigma\}\}V^\{\*\}\_\{t\}\\Bigr\]\\;\\geq\\;0\.\(20\)
Two structural assumptions, both with measurable proxies \(§[D\.4](https://arxiv.org/html/2607.07724#A4.SS4)\):

- \(A1\)*Secondary\-spacing floor\.*Every spacing other than the cutoff spacing is at leastΔ2≥0\\Delta\_\{2\}\\geq 0:st,\(j\)−st,\(j\+1\)≥Δ2s\_\{t,\(j\)\}\-s\_\{t,\(j\+1\)\}\\geq\\Delta\_\{2\}for allj≠k−1j\\neq k\-1\. \(If a fractionπ2\\pi\_\{2\}of tiles violates the floor, those tiles cost at most the trivial per\-tile cap, addingπ2\\pi\_\{2\}to the bound additively\.\)
- \(A2\)*Bounded spread variation\.*rt=r¯​\(1\+ξt\)r\_\{t\}=\\bar\{r\}\\,\(1\+\\xi\_\{t\}\)withr¯:=1T​∑trt\\bar\{r\}:=\\frac\{1\}\{T\}\\sum\_\{t\}r\_\{t\}and\|ξt\|≤β<1\|\\xi\_\{t\}\|\\leq\\beta<1for alltt\. The measuredCV​\(r\):=Vart​rt/r¯≤β\\mathrm\{CV\}\(r\):=\\sqrt\{\\mathrm\{Var\}\_\{t\}\\,r\_\{t\}\}/\\bar\{r\}\\leq\\betais its observable proxy\.

### D\.3Theorem and proof

Throughout,Φ\\Phiandϕ\\phiare the standard normal CDF and density, andV¯​\(g\):=Φ​\(−g/\(2​τ\)\)\\bar\{V\}\(g\):=\\Phi\\bigl\(\-g/\(\\sqrt\{2\}\\,\\tau\)\\bigr\);V¯\\bar\{V\}is decreasing withV¯​\(0\)=12\\bar\{V\}\(0\)=\\tfrac\{1\}\{2\}\.

###### Lemma 1\(two\-sided cutoff localisation\)\.

Letχ:=∑m≥1\(m\+1\)​e−m2​Δ22/\(4​τ2\)\\chi:=\\sum\_\{m\\geq 1\}\(m\+1\)\\,e^\{\-m^\{2\}\\Delta\_\{2\}^\{2\}/\(4\\tau^\{2\}\)\}; noteχ≤2\.2​e−Δ22/\(4​τ2\)\\chi\\leq 2\.2\\,e^\{\-\\Delta\_\{2\}^\{2\}/\(4\\tau^\{2\}\)\}onceΔ2≥2​τ\\Delta\_\{2\}\\geq 2\\tau\. Under \([17](https://arxiv.org/html/2607.07724#A4.E17)\) and \(A1\),

V¯​\(gt\)≤ℰt​\(S^k\)\\displaystyle\\bar\{V\}\(g\_\{t\}\)\\;\\leq\\;\\mathcal\{E\}\_\{t\}\(\\widehat\{S\}\_\{k\}\)\\;≤V¯​\(gt\)​\(1\+χ\),\\displaystyle\\leq\\;\\bar\{V\}\(g\_\{t\}\)\\,\(1\+\\chi\),\(21\)0≤ℰt​\(S^ρ​k\)\\displaystyle 0\\;\\leq\\;\\mathcal\{E\}\_\{t\}\(\\widehat\{S\}\_\{\\rho k\}\)\\;≤V¯​\(Gt\)​\(1\+χ\),\\displaystyle\\leq\\;\\bar\{V\}\(G\_\{t\}\)\\,\(1\+\\chi\),\(22\)and consequently, withε¯:=χ\+\(1\+χ\)​exp⁡\(−\(\(ρ−1\)​k​Δ2\)2/\(4​τ2\)\)\\bar\{\\varepsilon\}:=\\chi\+\(1\+\\chi\)\\,\\exp\\bigl\(\-\\bigl\(\(\\rho\-1\)k\\,\\Delta\_\{2\}\\bigr\)^\{2\}/\(4\\tau^\{2\}\)\\bigr\),

\(1−ε¯\)​V¯​\(gt\)≤Vt∗≤\(1\+ε¯\)​V¯​\(gt\)\.\(1\-\\bar\{\\varepsilon\}\)\\,\\bar\{V\}\(g\_\{t\}\)\\;\\leq\\;V^\{\*\}\_\{t\}\\;\\leq\\;\(1\+\\bar\{\\varepsilon\}\)\\,\\bar\{V\}\(g\_\{t\}\)\.\(23\)

###### Proof\.

*Lower bound in \([21](https://arxiv.org/html/2607.07724#A4.E21)\)\.*Letbk−1,bkb\_\{k\-1\},b\_\{k\}be the arms at empirical ranksk−1k\{\-\}1andkk\. On the eventA:=\{μbk\>μbk−1\}A:=\\\{\\mu\_\{b\_\{k\}\}\>\\mu\_\{b\_\{k\-1\}\}\\\}the kept set fails: ifbk∈S∗b\_\{k\}\\in S^\{\*\}we are done, sincebk∉S^kb\_\{k\}\\notin\\widehat\{S\}\_\{k\}; otherwise at leastkkarms exceedμbk\\mu\_\{b\_\{k\}\}, hence alsoμbk−1\\mu\_\{b\_\{k\-1\}\}, sobk−1∉S∗b\_\{k\-1\}\\notin S^\{\*\}and thekkarms ofS∗S^\{\*\}cannot fit in thek−1k\-1remaining slots ofS^k∖\{bk−1\}\\widehat\{S\}\_\{k\}\\setminus\\\{b\_\{k\-1\}\\\}\. Under the posterior the two means differ bygtg\_\{t\}with total variance2​τ22\\tau^\{2\}, soℙ​\(A∣st\)=Φ​\(−gt/\(2​τ\)\)=V¯​\(gt\)\\mathbb\{P\}\(A\\mid s\_\{t\}\)=\\Phi\(\-g\_\{t\}/\(\\sqrt\{2\}\\tau\)\)=\\bar\{V\}\(g\_\{t\}\)\.

*Upper bound in \([21](https://arxiv.org/html/2607.07724#A4.E21)\)\.*IfS^k⊉S∗\\widehat\{S\}\_\{k\}\\not\\supseteq S^\{\*\}, someb′∉S^kb^\{\\prime\}\\notin\\widehat\{S\}\_\{k\}hasμb′\\mu\_\{b^\{\\prime\}\}among the topkk, so at mostk−1k\-1arms beat it inμ\\mu; sinceS^k\\widehat\{S\}\_\{k\}haskkarms, someb∈S^kb\\in\\widehat\{S\}\_\{k\}hasμb<μb′\\mu\_\{b\}<\\mu\_\{b^\{\\prime\}\}\. Union\-bound over pairs at empirical ranks\(i,j\)\(i,j\),i≤k−1<k≤ji\\leq k\-1<k\\leq j\. By \(A1\) the pair at distancem:=\(k−1−i\)\+\(j−k\)≥0m:=\(k\-1\-i\)\+\(j\-k\)\\geq 0has score gap at leastgt\+m​Δ2g\_\{t\}\+m\\Delta\_\{2\}, there are at mostm\+1m\+1pairs at distancemm, and the Gaussian tail ratioΦ​\(−x−a\)≤Φ​\(−x\)​e−a​x−a2/2\\Phi\(\-x\-a\)\\leq\\Phi\(\-x\)\\,e^\{\-ax\-a^\{2\}/2\}\(valid fora,x≥0a,x\\geq 0\) givesΦ​\(−\(gt\+m​Δ2\)/\(2​τ\)\)≤V¯​\(gt\)​e−m2​Δ22/\(4​τ2\)\\Phi\\bigl\(\-\(g\_\{t\}\+m\\Delta\_\{2\}\)/\(\\sqrt\{2\}\\tau\)\\bigr\)\\leq\\bar\{V\}\(g\_\{t\}\)\\,e^\{\-m^\{2\}\\Delta\_\{2\}^\{2\}/\(4\\tau^\{2\}\)\}\. Summing overm≥1m\\geq 1and adding the cutoff pair \(m=0m=0\) yieldsV¯​\(gt\)​\(1\+χ\)\\bar\{V\}\(g\_\{t\}\)\(1\+\\chi\)\.

*Upper bound in \([22](https://arxiv.org/html/2607.07724#A4.E22)\)\.*Same union bound with the excluded arm now outside the*expanded*set: pairs\(i,j\)\(i,j\)withi≤k−1i\\leq k\-1,j≥ρ​kj\\geq\\rho k\. The dominant pair is\(k−1,ρ​k\)\(k\{\-\}1,\\rho k\)at gapGtG\_\{t\}; every other pair is at gap≥Gt\+m​Δ2\\geq G\_\{t\}\+m\\Delta\_\{2\}withm:=\(k−1−i\)\+\(j−ρ​k\)m:=\(k\-1\-i\)\+\(j\-\\rho k\)\. No matching lower bound is claimed at levelρ​k\\rho k, and none is needed\.

*Sandwich \([23](https://arxiv.org/html/2607.07724#A4.E23)\)\.*Upper:Vt∗≤ℰt​\(S^k\)≤\(1\+χ\)​V¯​\(gt\)V^\{\*\}\_\{t\}\\leq\\mathcal\{E\}\_\{t\}\(\\widehat\{S\}\_\{k\}\)\\leq\(1\+\\chi\)\\bar\{V\}\(g\_\{t\}\)\. Lower:Vt∗≥V¯​\(gt\)−\(1\+χ\)​V¯​\(Gt\)V^\{\*\}\_\{t\}\\geq\\bar\{V\}\(g\_\{t\}\)\-\(1\+\\chi\)\\bar\{V\}\(G\_\{t\}\); sinceGt−gt≥\(ρ−1\)​k​Δ2G\_\{t\}\-g\_\{t\}\\geq\(\\rho\-1\)k\\,\\Delta\_\{2\}by \(A1\), the same tail ratio givesV¯​\(Gt\)≤V¯​\(gt\)​e−\(\(ρ−1\)​k​Δ2\)2/\(4​τ2\)\\bar\{V\}\(G\_\{t\}\)\\leq\\bar\{V\}\(g\_\{t\}\)\\,e^\{\-\(\(\\rho\-1\)k\\Delta\_\{2\}\)^\{2\}/\(4\\tau^\{2\}\)\}\. ∎

###### Proposition 1\(σ\\sigma\-quantile regret, finiteTT\)\.

Letσ\(q\)\\sigma\_\{\(q\)\}be the largestσt\\sigma\_\{t\}insideΠqσ\\Pi\_\{q\}^\{\\sigma\}\(the empirical quantile the router thresholds on\) andγq:=σ\(q\)​r¯\\gamma\_\{q\}:=\\sigma\_\{\(q\)\}\\,\\bar\{r\}itsgg\-scale\. Under \([17](https://arxiv.org/html/2607.07724#A4.E17)\), \(A1\) and \(A2\),

ℛq≤min⁡\(q,1−q\)⋅Cq,Cq:=min⁡\{1,1\+ε¯2,2​β​γqτ​ϕ​\(γq​\(1−β\)2​τ\)\+ε¯\}\.\\mathcal\{R\}\_\{q\}\\;\\leq\\;\\min\(q,\\,1\-q\)\\cdot C\_\{q\},\\qquad C\_\{q\}:=\\min\\biggl\\\{\\,1,\\;\\;\\frac\{1\+\\bar\{\\varepsilon\}\}\{2\},\\;\\;\\sqrt\{2\}\\,\\frac\{\\beta\\gamma\_\{q\}\}\{\\tau\}\\;\\phi\\\!\\Bigl\(\\frac\{\\gamma\_\{q\}\(1\-\\beta\)\}\{\\sqrt\{2\}\\,\\tau\}\\Bigr\)\+\\bar\{\\varepsilon\}\\,\\biggr\\\}\.\(24\)The bound is exact at finiteTT: there is no asymptotic remainder, and every quantity is an empirical functional of the observed scores except the model parameters\(τ,Δ2,β\)\(\\tau,\\Delta\_\{2\},\\beta\), which have measurable proxies \(§[D\.4](https://arxiv.org/html/2607.07724#A4.SS4)\)\.

###### Proof\.

\|Πq∗\|=\|Πqσ\|=⌈q​T⌉\|\\Pi\_\{q\}^\{\*\}\|=\|\\Pi\_\{q\}^\{\\sigma\}\|=\\lceil qT\\rceil, so the common tiles cancel in \([20](https://arxiv.org/html/2607.07724#A4.E20)\) and the two difference sets have equal cardinalityD≤min⁡\(⌈q​T⌉,T−⌈q​T⌉\)D\\leq\\min\(\\lceil qT\\rceil,\\,T\-\\lceil qT\\rceil\)\. Pair them arbitrarily:

ℛq=1T​∑pairs​\(t,t′\)\(Vt∗−Vt′∗\),t∈Πq∗∖Πqσ,t′∈Πqσ∖Πq∗\.\\mathcal\{R\}\_\{q\}\\;=\\;\\frac\{1\}\{T\}\\sum\_\{\\text\{pairs \}\(t,t^\{\\prime\}\)\}\\bigl\(V^\{\*\}\_\{t\}\-V^\{\*\}\_\{t^\{\\prime\}\}\\bigr\),\\qquad t\\in\\Pi\_\{q\}^\{\*\}\\setminus\\Pi\_\{q\}^\{\\sigma\},\\quad t^\{\\prime\}\\in\\Pi\_\{q\}^\{\\sigma\}\\setminus\\Pi\_\{q\}^\{\*\}\.Only theσ\\sigma\-side constraints are needed\.t∉Πqσt\\notin\\Pi\_\{q\}^\{\\sigma\}forcesσt≥σ\(q\)\\sigma\_\{t\}\\geq\\sigma\_\{\(q\)\}, so by \(A2\)gt=σt​rt≥σ\(q\)​r¯​\(1−β\)=γq​\(1−β\)g\_\{t\}=\\sigma\_\{t\}r\_\{t\}\\geq\\sigma\_\{\(q\)\}\\,\\bar\{r\}\\,\(1\-\\beta\)=\\gamma\_\{q\}\(1\-\\beta\); symmetrically,t′∈Πqσt^\{\\prime\}\\in\\Pi\_\{q\}^\{\\sigma\}forcesgt′≤γq​\(1\+β\)g\_\{t^\{\\prime\}\}\\leq\\gamma\_\{q\}\(1\+\\beta\)\. By the sandwich \([23](https://arxiv.org/html/2607.07724#A4.E23)\) and monotonicity ofV¯\\bar\{V\},

Vt∗−Vt′∗\\displaystyle V^\{\*\}\_\{t\}\-V^\{\*\}\_\{t^\{\\prime\}\}\\;≤\(1\+ε¯\)​V¯​\(γq​\(1−β\)\)−\(1−ε¯\)​V¯​\(γq​\(1\+β\)\)\\displaystyle\\leq\\;\(1\+\\bar\{\\varepsilon\}\)\\,\\bar\{V\}\\bigl\(\\gamma\_\{q\}\(1\-\\beta\)\\bigr\)\-\(1\-\\bar\{\\varepsilon\}\)\\,\\bar\{V\}\\bigl\(\\gamma\_\{q\}\(1\+\\beta\)\\bigr\)=V¯​\(γq​\(1−β\)\)−V¯​\(γq​\(1\+β\)\)⏟boundary\-band width in​V¯\+ε¯​\[V¯​\(γq​\(1−β\)\)\+V¯​\(γq​\(1\+β\)\)\]⏟≤1\.\\displaystyle=\\;\\underbrace\{\\bar\{V\}\\bigl\(\\gamma\_\{q\}\(1\-\\beta\)\\bigr\)\-\\bar\{V\}\\bigl\(\\gamma\_\{q\}\(1\+\\beta\)\\bigr\)\}\_\{\\text\{boundary\-band width in \}\\bar\{V\}\}\\;\+\\;\\bar\{\\varepsilon\}\\,\\underbrace\{\\bigl\[\\bar\{V\}\(\\gamma\_\{q\}\(1\-\\beta\)\)\+\\bar\{V\}\(\\gamma\_\{q\}\(1\+\\beta\)\)\\bigr\]\}\_\{\\leq\\,1\}\.The band\-width term integrates the slopeϕ\(⋅/\(2τ\)\)/\(2τ\)\\phi\(\\cdot/\(\\sqrt\{2\}\\tau\)\)/\(\\sqrt\{2\}\\tau\)over an interval of length2​β​γq2\\beta\\gamma\_\{q\}, andϕ\\phiis decreasing on the positive axis, giving at most2​β​γq​ϕ​\(γq​\(1−β\)/\(2​τ\)\)/τ\\sqrt\{2\}\\,\\beta\\gamma\_\{q\}\\,\\phi\\bigl\(\\gamma\_\{q\}\(1\-\\beta\)/\(\\sqrt\{2\}\\tau\)\\bigr\)/\\tau\. The two caps come fromVt∗≤ℰt​\(S^k\)≤1V^\{\*\}\_\{t\}\\leq\\mathcal\{E\}\_\{t\}\(\\widehat\{S\}\_\{k\}\)\\leq 1andVt∗≤\(1\+ε¯\)​V¯​\(gt\)≤1\+ε¯2V^\{\*\}\_\{t\}\\leq\(1\+\\bar\{\\varepsilon\}\)\\bar\{V\}\(g\_\{t\}\)\\leq\\tfrac\{1\+\\bar\{\\varepsilon\}\}\{2\}, withVt′∗≥0V^\{\*\}\_\{t^\{\\prime\}\}\\geq 0\(Observation[1](https://arxiv.org/html/2607.07724#Thmobservation1)\)\. Multiplying the per\-pair cost byD/T≤min⁡\(q,1−q\)D/T\\leq\\min\(q,1\-q\)completes the proof\. ∎

*Population\-quantile version\.*Thresholding on the distributionalqq\-quantile ofσ\\sigmainstead of the empiricalσ\(q\)\\sigma\_\{\(q\)\}adds the standard Dvoretzky–Kiefer–Wolfowitz fluctuation termO​\(log⁡\(1/δ\)/T\)O\(\\sqrt\{\\log\(1/\\delta\)/T\}\)\(Dvoretzky et al\.,[1956](https://arxiv.org/html/2607.07724#bib.bib16); Massart,[1990](https://arxiv.org/html/2607.07724#bib.bib17)\); we state the empirical version because that is what the router computes\.

### D\.4Empirical estimation of the bound

Every input to Proposition[1](https://arxiv.org/html/2607.07724#Thmproposition1)is measurable from a small diagnostic run\. The recipe is:

1. 1\.For each layerℓ\\elland headhh, dump the per\-tile block\-score vector\(st,b\)b∈\[NB\]\(s\_\{t,b\}\)\_\{b\\in\[N\_\{\\\!B\}\]\}during prefill on a small example panel \(n=10n=10is sufficient: a 32K context withBLOCKM=64\\text\{BLOCK\}\_\{M\}=64yields∼512\\sim 512Q\-tiles per \(layer, head, example\), so∼5000\\sim 5000tiles per \(layer, head\) overall\)\.
2. 2\.From the sorted scores computegt,Gt,rt,σtg\_\{t\},G\_\{t\},r\_\{t\},\\sigma\_\{t\}; aggregate per layerCV​\(r\)\\mathrm\{CV\}\(r\)\(the observable proxy forβ\\beta\), the boundary scaleγq=σ\(q\)​r¯\\gamma\_\{q\}=\\sigma\_\{\(q\)\}\\bar\{r\}, the band massmq​\(β\)m\_\{q\}\(\\beta\)of Remark[1](https://arxiv.org/html/2607.07724#Thmremark1), and the secondary\-spacing quantiles that stand in forΔ2\\Delta\_\{2\}\.
3. 3\.Estimateτ\\taufrom the first\-difference dispersion of per\-row block scores along the query dimension \(underμt\+1,b≈μt,b\\mu\_\{t\+1,b\}\\approx\\mu\_\{t,b\}for adjacent rows,𝔼​\[\(st\+1,b−st,b\)2\]=2​τ2\\mathbb\{E\}\[\(s\_\{t\+1,b\}\-s\_\{t,b\}\)^\{2\}\]=2\\tau^\{2\}\)\. Caveat: this measures row\-to\-row score roughness – a proxy for, not the same object as, the score\-vs\-relevance noise in \([17](https://arxiv.org/html/2607.07724#A4.E17)\); it is the simplest estimator consistent with the smoothness heuristic\.
4. 4\.Evaluate \([24](https://arxiv.org/html/2607.07724#A4.E24)\); and, independently of \(A1\)–\(A2\), estimateℰt​\(S^k\)\\mathcal\{E\}\_\{t\}\(\\widehat\{S\}\_\{k\}\),ℰt​\(S^ρ​k\)\\mathcal\{E\}\_\{t\}\(\\widehat\{S\}\_\{\\rho k\}\),Vt∗V^\{\*\}\_\{t\}andℛq\\mathcal\{R\}\_\{q\}itself by Monte Carlo, samplingμ∼𝒩​\(st,τ^2​I\)\\mu\\sim\\mathcal\{N\}\(s\_\{t\},\\hat\{\\tau\}^\{2\}I\)and checking containment \(Remark[2](https://arxiv.org/html/2607.07724#Thmremark2)\)\. The MC estimate ofℛq\\mathcal\{R\}\_\{q\}is the sharpest number this appendix can produce; the closed\-form bound is its certificate in the spacing\-dominated regime\.

#### Measured quantities on Qwen\-14B, RULER NIAH n=10, 32K\.

The diagnostic collects∼\\sim5\.7M\(B,H,Qt\)\(B,H,Q\_\{t\}\)tile cells per scoring backbone \(10 examples×\\times48 layers×\\times40 heads×\\times∼\\sim512 tiles, masked to causally\-valid tiles with≥ρ​k\+1\\geq\\rho k\+1visible blocks\), plus the per\-call noise\-scale estimateτ^\\hat\{\\tau\}of step 3\. Aggregated over the 48 Qwen\-14B layers and 40 heads:

whererqr\_\{q\}is theqq\-quantile ofrtr\_\{t\}andgq\+g\_\{q\}^\{\+\}is the4040th percentile ofgtg\_\{t\}on the informative subset\{t:gt\>0\}\\\{t:g\_\{t\}\>0\\\}\(the𝚋𝚏𝟷𝟼\\mathtt\{bf16\}floor for Quest scores yields exact zeros on∼83%\\sim 83\\%of tiles, where the cutoff lands in the flat tail of the score distribution; conditioning ongt\>0g\_\{t\}\>0is the honest active\-set restriction\)\. Four observations\.

*\(i\) The measured regime is tie\- and noise\-dominated at the cutoff\.*Even on the active set,gq\+/τ^≈0\.03g\_\{q\}^\{\+\}/\\hat\{\\tau\}\\approx 0\.03: the cutoff is buried in the noise, and the83%83\\%atom atgt=0g\_\{t\}=0means the secondary\-spacing floor \(A1\) holds only withΔ2≈0\\Delta\_\{2\}\\approx 0\. This places the diagnostic squarely in the noise\-dominated regime of Remark[2](https://arxiv.org/html/2607.07724#Thmremark2):ε¯\\bar\{\\varepsilon\}is not small, and the closed\-form certificate \([24](https://arxiv.org/html/2607.07724#A4.E24)\) falls back to its trivial branch \(min⁡\(q,1−q\)=40\\min\(q,1\-q\)=40pp atq=0\.40q=0\.40\)\. The atom also means the band mass of Remark[1](https://arxiv.org/html/2607.07724#Thmremark1)must be computed on the active set\.

*\(ii\) The measured scales are consistent with a small true regret\.*Within the two\-Φ\\Phirepresentation of Lemma[1](https://arxiv.org/html/2607.07724#Thmlemma1)– heuristic here, sinceε¯\\bar\{\\varepsilon\}is uncontrolled in this regime – the per\-pair cost at the boundary is at mostϕ​\(0\)⋅\(boundaryg\-scale\)/\(2​τ^\)≈0\.4×0\.03/1\.41≲1\\phi\(0\)\\cdot\(\\text\{boundary $g$\-scale\}\)/\(\\sqrt\{2\}\\,\\hat\{\\tau\}\)\\approx 0\.4\\times 0\.03/1\.41\\lesssim 1pp\. This is consistent with the observed22pp router\-vs\-dense paired gap on RULER NIAHn=100n=100\(router\-on\-Quest0\.980\.98vs\. dense1\.001\.00\), which itself upper\-bounds the end\-task manifestation of the regret: dense is the unconstrained policy, soℰ​\(Πq∗\)≥ℰ​\(dense\)=0\\mathcal\{E\}\(\\Pi\_\{q\}^\{\*\}\)\\geq\\mathcal\{E\}\(\\text\{dense\}\)=0\.

*\(iii\) Positivity of the expansion value is structural, not empirical\.*An earlier draft validated that the local spacing at rankρ​k\\rho kexceedsgtg\_\{t\}in expectation, across all48×348\\times 3\(layer, backbone\) cells, as an “expansion helps in expectation” check\. Under the corrected definition \([18](https://arxiv.org/html/2607.07724#A4.E18)\) no such check is needed:Gt≥gtG\_\{t\}\\geq g\_\{t\}holds by construction, andVt∗≥0V^\{\*\}\_\{t\}\\geq 0holds pointwise by Observation[1](https://arxiv.org/html/2607.07724#Thmobservation1)\.

*\(iv\) Pending re\-analysis\.*The quantities the corrected bound actually consumes –γq\\gamma\_\{q\}, the band massmq​\(β\)m\_\{q\}\(\\beta\), the secondary\-spacing quantiles standing in forΔ2\\Delta\_\{2\}, and above all the Monte\-Carlo estimate ofℛq\\mathcal\{R\}\_\{q\}– are computable from the same raw dump with an updatedanalyze\_block\_scores\.pypass; no new GPU run is required\. Until that pass lands, this section reports measured scales and regime identification, not a numerical certificate\.

Diagnostic produced bydump\_block\_scores\.py\(cache\-safe runtime monkey\-patch; lives outsidecode\_version\) andanalyze\_block\_scores\.py\. Raw pickle and per\-layer table available with the release\. Total diagnostic cost:∼\\sim$2 across the twon=10n\{=\}10runs\.

### D\.5What \([24](https://arxiv.org/html/2607.07724#A4.E24)\) does*not*bound

The proposition controls top\-kk*identification*error – the probability that the kept setS^\\widehat\{S\}misses an element ofS∗​\(μ\)S^\{\*\}\(\\mu\)\. The downstream quantity is the*attention\-output*error,

‖oS^−ofull‖2≤\(∑b∉S^αb∗\)⋅maxj⁡‖vj‖,\\bigl\\\|\\,o^\{\\widehat\{S\}\}\-o^\{\\mathrm\{full\}\}\\,\\bigr\\\|\_\{2\}\\;\\leq\\;\\Bigl\(\\sum\_\{b\\notin\\widehat\{S\}\}\\alpha^\{\*\}\_\{b\}\\Bigr\)\\cdot\\max\_\{j\}\\\|v\_\{j\}\\\|,whereα∗\\alpha^\{\*\}is the dense softmax\. The dropped softmax mass is itselfΦ\\Phi\-tailed in the cutoff gap – the Lemma[1](https://arxiv.org/html/2607.07724#Thmlemma1)argument applied at the softmax temperature1/d1/\\sqrt\{d\}instead ofτ\\tau– so composing gives a bound proportional toΦ​\(−gt/τsoftmax\)⋅maxj⁡‖vj‖\\Phi\(\-g\_\{t\}/\\tau\_\{\\mathrm\{softmax\}\}\)\\cdot\\max\_\{j\}\\\|v\_\{j\}\\\|\. The two temperatures are different constants and the composition is one paper of additional work; we record it as the natural follow\-up theorem\.

Bought\.Pointwise “expansion never harms” with no distributional assumption \(Observation[1](https://arxiv.org/html/2607.07724#Thmobservation1)\); a two\-sided localisation of the identification error to the cutoff and expansion gaps, with explicit constants \(Lemma[1](https://arxiv.org/html/2607.07724#Thmlemma1)\); and a finite\-TT, assumption\-explicit regret bound for theσ\\sigma\-quantile trigger against the best trigger rule of the same budget, with a measurable band refinement \(Proposition[1](https://arxiv.org/html/2607.07724#Thmproposition1), Remarks[1](https://arxiv.org/html/2607.07724#Thmremark1)–[2](https://arxiv.org/html/2607.07724#Thmremark2)\)\.

Owed\.The Monte\-Carlo evaluation ofℛq\\mathcal\{R\}\_\{q\}from the existing score dump \(analysis\-only, §[D\.4](https://arxiv.org/html/2607.07724#A4.SS4)\); a sharper treatment of the noise\-dominated regime than the trivial cap; the composite attention\-output bound \(one further composition with a softmax\-sensitivity inequality\); and a treatment of LB\-v2 medium that distinguishes “the bound is loose” from “the budget is binding” \(budget sweep, Section[4\.5](https://arxiv.org/html/2607.07724#S4.SS5)\)\.

## References

- Vaswani et al\. \(2017\)A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin\.Attention is all you need\.In*NeurIPS*, 2017\.[https://arxiv\.org/abs/1706\.03762](https://arxiv.org/abs/1706.03762)\.
- Tang et al\. \(2024\)J\. Tang, Y\. Zhao, K\. Zhu, G\. Xiao, B\. Kasikci, and S\. Han\.Quest: Query\-aware sparsity for efficient long\-context LLM inference\.In*MLSys*, 2024\.[https://arxiv\.org/abs/2406\.10774](https://arxiv.org/abs/2406.10774)\.
- Zhang et al\. \(2023\)Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng, R\. Cai, Z\. Song, Y\. Tian, C\. Ré, C\. Barrett, Z\. Wang, and B\. Chen\.H2O: Heavy\-hitter oracle for efficient generative inference of large language models\.In*NeurIPS*, 2023\.[https://arxiv\.org/abs/2306\.14048](https://arxiv.org/abs/2306.14048)\.
- Li et al\. \(2024\)Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. Chen\.SnapKV: LLM knows what you are looking for before generation\.In*NeurIPS*, 2024\.[https://arxiv\.org/abs/2404\.14469](https://arxiv.org/abs/2404.14469)\.
- Jiang et al\. \(2024\)H\. Jiang, Y\. Li, C\. Zhang, Q\. Wu, X\. Luo, S\. Ahn, Z\. Han, A\. H\. Abdi, D\. Li, C\.\-Y\. Lin, Y\. Yang, and L\. Qiu\.MInference 1\.0: Accelerating pre\-filling for long\-context LLMs via dynamic sparse attention\.In*NeurIPS*, 2024\.[https://arxiv\.org/abs/2407\.02490](https://arxiv.org/abs/2407.02490)\.
- Yuan et al\. \(2025\)J\. Yuan, H\. Gao, D\. Dai, J\. Luo, L\. Zhao, Z\. Zhang, Z\. Xie, Y\. X\. Wei, L\. Wang, Z\. Xiao, et al\.Native sparse attention: Hardware\-aligned and natively trainable sparse attention\.arXiv preprint, 2025\.[https://arxiv\.org/abs/2502\.11089](https://arxiv.org/abs/2502.11089)\.
- Lu et al\. \(2025\)E\. Lu, Z\. Jiang, J\. Liu, Y\. Du, T\. Jiang, C\. Hong, S\. Liu, W\. He, E\. Yuan, Y\. Wang, et al\.MoBA: Mixture of block attention for long\-context LLMs\.arXiv preprint, 2025\.[https://arxiv\.org/abs/2502\.13189](https://arxiv.org/abs/2502.13189)\.
- Subquadratic \(2025\)Subquadratic\.How SSA makes long\-context practical\.2025\.[https://subq\.ai/how\-ssa\-makes\-long\-context\-practical](https://subq.ai/how-ssa-makes-long-context-practical)\.
- Dao et al\. \(2022\)T\. Dao, D\. Y\. Fu, S\. Ermon, A\. Rudra, and C\. Ré\.FlashAttention: Fast and memory\-efficient exact attention with IO\-awareness\.In*NeurIPS*, 2022\.[https://arxiv\.org/abs/2205\.14135](https://arxiv.org/abs/2205.14135)\.
- Hsieh et al\. \(2024\)C\.\-P\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, Y\. Zhang, and B\. Ginsburg\.RULER: What’s the real context size of your long\-context language models?In*COLM*, 2024\.[https://arxiv\.org/abs/2404\.06654](https://arxiv.org/abs/2404.06654)\.
- Bai et al\. \(2023\)Y\. Bai, X\. Lv, J\. Zhang, H\. Lyu, J\. Tang, Z\. Huang, Z\. Du, X\. Liu, A\. Zeng, L\. Hou, Y\. Dong, J\. Tang, and J\. Li\.LongBench: A bilingual, multitask benchmark for long context understanding\.In*ACL*, 2024\.[https://arxiv\.org/abs/2308\.14508](https://arxiv.org/abs/2308.14508)\.
- Bai et al\. \(2024\)Y\. Bai, S\. Tu, J\. Zhang, H\. Peng, X\. Wang, X\. Lv, S\. Cao, J\. Xu, L\. Hou, Y\. Dong, J\. Tang, and J\. Li\.LongBench v2: Towards deeper understanding and reasoning on realistic long\-context multitasks\.arXiv preprint, 2024\.[https://arxiv\.org/abs/2412\.15204](https://arxiv.org/abs/2412.15204)\.
- Yen et al\. \(2024\)H\. Yen, T\. Gao, M\. Hou, K\. Ding, D\. Fleischer, P\. Izsak, M\. Wasserblat, and D\. Chen\.HELMET: How to evaluate long\-context language models effectively and thoroughly\.arXiv preprint, 2024\.[https://arxiv\.org/abs/2410\.02694](https://arxiv.org/abs/2410.02694)\.
- Garivier and Kaufmann \(2016\)A\. Garivier and E\. Kaufmann\.Optimal best arm identification with fixed confidence\.In*COLT*, 2016\.
- Kaufmann et al\. \(2016\)E\. Kaufmann, O\. Cappé, and A\. Garivier\.On the complexity of best\-arm identification in multi\-armed bandit models\.*JMLR*, 17\(1\):1–42, 2016\.
- Dvoretzky et al\. \(1956\)A\. Dvoretzky, J\. Kiefer, and J\. Wolfowitz\.Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator\.*Annals of Mathematical Statistics*, 27\(3\):642–669, 1956\.
- Massart \(1990\)P\. Massart\.The tight constant in the Dvoretzky–Kiefer–Wolfowitz inequality\.*Annals of Probability*, 18\(3\):1269–1283, 1990\.