The risk of KV cache compression

# The risk of KV cache compression
Source: [https://arxiv.org/html/2607.01520](https://arxiv.org/html/2607.01520)
Lukas Haverbeck RWTH Aachen University lukas\.haverbeck@rwth\-aachen\.de &Carmen Amo Alonso Stanford University camoalon@stanford\.edu Andres Felipe Posada\-Moreno RWTH Aachen University andres\.posada@dsme\.rwth\-aachen\.de &Sebastian Trimpe RWTH Aachen University trimpe@dsme\.rwth\-aachen\.de &Marco Pavone Stanford University pavone@stanford\.edu

###### Abstract

Transformer inference on long sequences is expensive because softmax attention repeatedly reads from a large KV cache\. The prevalent approach to this bottleneck is*KV cache compression*, which replaces the full cache with a compact summary\. Despite its practical importance, the design of such summaries is largely driven by empirical experimentation\. On the theoretical side, existing results show that KV cache compression can be impossible in the worst case, but offer little systematic guidance for designing algorithms in regimes where accurate compression is possible\. We bridge this gap by characterizing the*minimax risk of KV cache compression*in terms of the intrinsic compressibility of a cache, revealing when and how accurate compression is possible\. These results yield novel design principles for KV cache compression under causal masking that map efficiently to prefill and autoregressive decoding while achieving minimax\-optimal risk\. We instantiate these principles in a practical algorithm and report promising performance on LongBench in targeted experiments\. Overall, our results provide a principled avenue for practical KV cache compression with theoretical guarantees\.

## 1Introduction

Transformers have become the dominant architecture for processing and generating long sequences across various domains\[[14](https://arxiv.org/html/2607.01520#bib.bib14),[5](https://arxiv.org/html/2607.01520#bib.bib5),[22](https://arxiv.org/html/2607.01520#bib.bib22),[4](https://arxiv.org/html/2607.01520#bib.bib4)\]due to their excellent ability to model complex interactions between tokens\. During inference, these interactions are mediated through the*key–value \(KV\) cache*storing all previous tokens, and each new token is computed from this cache via softmax attention\[[23](https://arxiv.org/html/2607.01520#bib.bib23)\]\. The KV cache therefore serves as the model’s working memory of the past\. While this persistent access to past data makes Transformers powerful, it is also costly\. As the sequence grows, the cache grows with it, and each new step must read from an increasingly large history\. For long\-context applications, this creates a major bottleneck in both memory use and runtime\.

An attractive remedy is*KV compression*: replacing a long history of key–value pairs by a compact summary that approximately preserves the attention outputs of the original cache\.111The term also refers to numerical compression of the stored tensors\. We focus on reducing the number of key–value pairs\.Such a summary is useful only if future attention cannot reliably distinguish the compressed cache from the full history\. This makes it challenging to decide which distinctions in the past can be safely forgotten, because the importance of a token is not intrinsic to the token itself\. A token that can be safely discarded in one context and for one set of future queries may be essential for another, while several distinct tokens may be interchangeable if attention uses them in the same way\. Thus, compression is safe only to the extent that the summary preserves the context that future queries can actually use\.

Despite its practical importance, KV compression lacks a systematic design theory and remains largely guided by empirical exploration\[[20](https://arxiv.org/html/2607.01520#bib.bib20),[27](https://arxiv.org/html/2607.01520#bib.bib27),[24](https://arxiv.org/html/2607.01520#bib.bib24),[19](https://arxiv.org/html/2607.01520#bib.bib19),[10](https://arxiv.org/html/2607.01520#bib.bib10),[1](https://arxiv.org/html/2607.01520#bib.bib1)\], using heuristic proxies for future relevance such as recency, accumulated attention mass, and attention sinks\. These methods show substantial compression is often possible, but they do not explain which properties of a cache make compression safe, how the achievable error should scale with the compression budget, or what an optimal summary should preserve\. Conversely, theoretical results show that accurate KV compression can be impossible in the worst case\[[16](https://arxiv.org/html/2607.01520#bib.bib16)\], and identify structural assumptions under which compression is possible\[[18](https://arxiv.org/html/2607.01520#bib.bib18),[11](https://arxiv.org/html/2607.01520#bib.bib11),[26](https://arxiv.org/html/2607.01520#bib.bib26)\]\. This leaves open the intermediate regime most relevant to practical algorithm design: caches that are not adversarial, but also not prescribed by a fixed model class\.

To bridge this gap, we provide a graded account of KV cache compressibility in terms of the intrinsic interaction between a cache and future queries through softmax attention\. Specifically, we characterize the*minimax risk of KV compression*, asking: for a fixed compression budget, how much error is unavoidable for the way a given cache can be probed by future queries? Beyond quantitative rates, this characterization identifies what an optimal summary must preserve\. It captures the continuum between easy regimes, where many past tokens are effectively redundant and a small summary can preserve attention accurately, and hard regimes, where the cache contains many separately retrievable pieces of information and any small summary must lose something, and yields concrete design criteria for algorithms to attain minimax\-optimal risk\.

Concretely, we make the following contributions\.\(1\)We recast KV compression as sparse approximation of a measure, strictly generalizing token eviction, and formalize its minimax risk\.\(2\)We prove tight upper and lower bounds on this risk in terms of an intrinsic complexity measure capturing how future queries probe the cache, and show a sharp separation between algorithms with and without access to the future\-query distribution\.\(3\)We derive design principles for efficient KV compression during causally masked prefill and autoregressive decoding with minimax\-optimal compression risk\.\(4\)We instantiate these principles in a concrete KV compression algorithm and evaluate it through targeted experiments on LongBench, where it shows promising performance\. Put together, our results advance KV compression towards practical methods with theoretical guarantees\.

## 2Related work

A large body of recent work reduces KV cache size by selecting tokens according to empirical proxies for future relevance\. These proxies include recency and attention sinks\[[24](https://arxiv.org/html/2607.01520#bib.bib24),[27](https://arxiv.org/html/2607.01520#bib.bib27)\], accumulated or persistent attention mass\[[20](https://arxiv.org/html/2607.01520#bib.bib20),[27](https://arxiv.org/html/2607.01520#bib.bib27),[1](https://arxiv.org/html/2607.01520#bib.bib1)\], and attention patterns observed during prefill or across layers\[[19](https://arxiv.org/html/2607.01520#bib.bib19),[10](https://arxiv.org/html/2607.01520#bib.bib10)\]\. The strong empirical performance of these methods suggests that real Transformer caches often contain substantial redundancy\. At the same time, these proxies do not by themselves explain which properties of a cache make compression safe, how the achievable error should scale with the compression budget, or what information an optimal summary must preserve\. Our work addresses this gap by studying the minimax risk of KV compression algorithms broadly, aiming to inform practical compression beyond empirical exploration\.

Existing theory has mainly identified two endpoints: impossibility results for sharp token retrieval\[[3](https://arxiv.org/html/2607.01520#bib.bib3),[16](https://arxiv.org/html/2607.01520#bib.bib16)\], and positive approximation guarantees under additional structure\[[26](https://arxiv.org/html/2607.01520#bib.bib26),[18](https://arxiv.org/html/2607.01520#bib.bib18),[11](https://arxiv.org/html/2607.01520#bib.bib11),[2](https://arxiv.org/html/2607.01520#bib.bib2)\]\. These results identify important limitations and tractable regimes, but they do not give a graded characterization from lossless compression to worst\-case incompressibility\. Our upper bounds require no a priori structural model of the data beyond bounded values, which is weaker than bounded keys and queries\. Instead, we deliberately characterize the compression risk through the interaction between caches and softmax attention itself\. Our lower\-bound assumptions are close in spirit to the sharp\-attention regimes considered in prior hardness results\. However, we employ them to characterize when compression remains possible, even under sharp attention, not to establish impossibility alone\.

Finally, we formalize the minimax risk of KV compression as sparse approximation of a measure\. Measure\-based views of softmax attention and Transformers are not new, and have led to useful perspectives in several settings\[[15](https://arxiv.org/html/2607.01520#bib.bib15),[9](https://arxiv.org/html/2607.01520#bib.bib9)\]\. To the best of our knowledge, our recasting is new for KV compression, and it strictly generalizes token eviction from subset selection to sparse reweighting\.

## 3Problem setup

We study KV compression: the problem of replacing the context seen by a single softmax attention head with a compact summary while preserving its outputs on future queries, in order to reduce memory use and computational cost\. Since softmax attention depends on the context only through the induced weighting of its key–value pairs, we formalize KV compression as sparse approximation of a probability measure over tokens\. Our goal is to characterize the achievable error for this approximation problem as a function of the context’s*intrinsic compressibility*under a query distribution, which we formalize as a minimax risk in terms of a data\-dependent complexity measure\.

### 3\.1KV compression as sparse approximation of a probability measure

In their seminal work,Vaswani et al\. \[[23](https://arxiv.org/html/2607.01520#bib.bib23)\]define softmax attention as

Att⁡\(q∣k1,…,kn,v1,…,vn\)=∑i=1nexp⁡\(⟨q,ki⟩/dk\)​vi∑i=1nexp⁡\(⟨q,ki⟩/dk\),\\operatorname\{Att\}\\bigl\(q\\mid k\_\{1\},\\dots,k\_\{n\},v\_\{1\},\\dots,v\_\{n\}\\bigr\)=\\frac\{\\sum\_\{i=1\}^\{n\}\\exp\\left\(\\left\\langle q,k\_\{i\}\\right\\rangle/\\sqrt\{d\_\{k\}\}\\right\)\\,v\_\{i\}\}\{\\sum\_\{i=1\}^\{n\}\\exp\\left\(\\left\\langle q,k\_\{i\}\\right\\rangle/\\sqrt\{d\_\{k\}\}\\right\)\},\(1\)whereq∈𝒬q\\in\\mathcal\{Q\}is a*query*,ki∈𝒦k\_\{i\}\\in\\mathcal\{K\}are*keys*, andvi∈𝒱v\_\{i\}\\in\\mathcal\{V\}are*values*, in their respective spaces𝒦,𝒬⊆ℝdk\\mathcal\{K\},\\mathcal\{Q\}\\subseteq\\mathbb\{R\}^\{d\_\{k\}\}and𝒱⊆ℝdv\\mathcal\{V\}\\subseteq\\mathbb\{R\}^\{d\_\{v\}\}\. We write𝒳≔𝒦×𝒱\\mathcal\{X\}\\coloneqq\\mathcal\{K\}\\times\\mathcal\{V\}for the key–value space, and assume throughout that values are bounded in norm by a constantV\>0V\>0, which is usually not restrictive in practice\.

The collection\(ki,vi\)i=1n\(k\_\{i\},v\_\{i\}\)\_\{i=1\}^\{n\}of key–value pairs in \([1](https://arxiv.org/html/2607.01520#S3.E1)\) is called the*KV cache*\. We study the problem of*KV compression*: finding a compact representation of this collection that approximately preserves the attention map in \([1](https://arxiv.org/html/2607.01520#S3.E1)\)\. Since \([1](https://arxiv.org/html/2607.01520#S3.E1)\) depends on the KV cache only through a weighted combination, we represent the KV cache as a finitely supported probability measurePPand define

Att⁡\(q\|P\)≔∫𝒳ak​\(q∣P\)​v​d​P​\(k,v\),ak​\(q∣P\)≔κ​\(q,k\)∫𝒳κ​\(q,k′\)​d​P​\(k′,v′\)\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\\coloneqq\\int\_\{\\mathcal\{X\}\}a\_\{k\}\(q\\mid P\)\\,v\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\),\\qquad a\_\{k\}\(q\\mid P\)\\coloneqq\\frac\{\\kappa\(q,k\)\}\{\\int\_\{\\mathcal\{X\}\}\\kappa\(q,k^\{\\prime\}\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k^\{\\prime\},v^\{\\prime\}\)\}\(2\)for a Gaussian kernelκ​\(q,k\)≔exp⁡\(−‖q−k‖22/\(2​dk\)\)\\kappa\(q,k\)\\coloneqq\\exp\(\-\\\|q\-k\\\|\_\{2\}^\{2\}/\(2\\sqrt\{d\_\{k\}\}\)\)onℝdk\\mathbb\{R\}^\{d\_\{k\}\}with scale parameterdk\\sqrt\{d\_\{k\}\}\. This definition coincides with \([1](https://arxiv.org/html/2607.01520#S3.E1)\) wheneverP∝∑i=1nexp⁡\(‖ki‖2/\(2​dk\)\)⋅δ\(ki,vi\)P\\propto\\sum\_\{i=1\}^\{n\}\\exp\\bigl\(\\\|k\_\{i\}\\\|^\{2\}/\(2\\sqrt\{d\_\{k\}\}\)\\bigr\)\\cdot\\delta\_\{\(k\_\{i\},v\_\{i\}\)\}\. For the remainder of this paper, we work with attention as defined in \([2](https://arxiv.org/html/2607.01520#S3.E2)\) so that attention depends on the KV cache only through the*context measure*PP\.

To formalize KV compression from this viewpoint, we introduce the notation𝒫​\(𝒮\)\\mathcal\{P\}\(\\mathcal\{S\}\)for the probability measures on a subset𝒮⊆ℋ\\mathcal\{S\}\\subseteq\\mathcal\{H\}of a Hilbert spaceℋ\\mathcal\{H\}, and denote by𝒫fin​\(𝒮\)\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{S\}\)those with finite support and by𝒫K​\(𝒮\)\\mathcal\{P\}\_\{K\}\(\\mathcal\{S\}\)those supported on at mostKKpoints\. KV compression then becomes the following problem\.

Problem statement I \(KV compression\)Given a budgetK≥1K\\geq 1, a context measureP∈𝒫fin​\(𝒳\)P\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\), and a query distributionν∈𝒫​\(𝒬\)\\nu\\in\\mathcal\{P\}\(\\mathcal\{Q\}\), find a compressed context measureP^∈𝒫K​\(𝒳\)\\hat\{P\}\\in\\mathcal\{P\}\_\{K\}\(\\mathcal\{X\}\)minimizing the mean squared errorℰP,ν​\(P^\)≔𝔼q∼ν⁡‖Att⁡\(q∣P\)−Att⁡\(q∣P^\)‖22\.\\textstyle\\mathcal\{E\}\_\{P,\\nu\}\\bigl\(\\hat\{P\}\\bigr\)\\coloneqq\\operatorname\{\\mathbb\{E\}\}\_\{q\\sim\\nu\}\\bigl\\\|\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\-\\operatorname\{Att\}\\bigl\(q\\,\|\\,\\hat\{P\}\\bigr\)\\bigr\\\|\_\{2\}^\{2\}\.

[Section˜3\.1](https://arxiv.org/html/2607.01520#S3.SS1)strictly generalizes token eviction\[[20](https://arxiv.org/html/2607.01520#bib.bib20),[27](https://arxiv.org/html/2607.01520#bib.bib27),[24](https://arxiv.org/html/2607.01520#bib.bib24),[19](https://arxiv.org/html/2607.01520#bib.bib19),[10](https://arxiv.org/html/2607.01520#bib.bib10)\], which seeks a small, unweighted subset ofKKtokens that approximately preserve the attention map\. In our setting, these tokens form the support ofP^\\hat\{P\}\. However, we allow arbitrary sparse reweightings of the cache, which, to the best of our knowledge, provides a new view on KV compression\.

### 3\.2Capturing the intrinsic compressibility of a context

To understand when accurate compression is possible, we identify how changes toPPaffect softmax attention\. SincePPandP^\\hat\{P\}are both probability measures, their differenceΔ≔P^−P\\Delta\\coloneqq\\hat\{P\}\-Pis a signed, zero\-mass measure\. Compression can therefore be viewed as a reweighting of the original context, and its error is determined by how this reweighting perturbs the softmax numerator and normalizer\.

Indeed, a direct computation shows that, for every queryq∈𝒬q\\in\\mathcal\{Q\},

Att⁡\(q∣P^\)−Att⁡\(q∣P\)=∫𝒳ak​\(q∣P\)​\(v−Att⁡\(q∣P\)\)​d​Δ​\(k,v\)1\+∫𝒳\(ak​\(q∣P\)−1\)​d​Δ​\(k,v\)\.\\operatorname\{Att\}\(q\\mid\\hat\{P\}\)\-\\operatorname\{Att\}\(q\\mid P\)=\\frac\{\\int\_\{\\mathcal\{X\}\}a\_\{k\}\(q\\mid P\)\\bigl\(v\-\\operatorname\{Att\}\(q\\mid P\)\\bigr\)\\,\\mathop\{\}\\\!\\mathrm\{d\}\\Delta\(k,v\)\}\{1\+\\int\_\{\\mathcal\{X\}\}\\bigl\(a\_\{k\}\(q\\mid P\)\-1\\bigr\)\\,\\mathop\{\}\\\!\\mathrm\{d\}\\Delta\(k,v\)\}\.\(3\)Thus, every compression perturbation enters attention through two linear functionals ofΔ\\Delta: a centered numerator term and a centered renormalization term\. This identifies the pair of query\-indexed coefficients with which each\(k,v\)∈𝒳\(k,v\)\\in\\mathcal\{X\}contributes to the compression\-induced reweighting of tokens\. We collect these coefficients with the following definition\.

###### Definition 3\.1\(Response profile\)\.

LetP∈𝒫fin​\(𝒳\)P\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)be a context measure andν∈𝒫​\(𝒬\)\\nu\\in\\mathcal\{P\}\(\\mathcal\{Q\}\)a query distribution\. We call

ΓP​\(k,v\):𝒬→ℝdv⊕ℝ,q↦\(ak​\(q∣P\)​\(v−Att⁡\(q∣P\)\),V​\(ak​\(q∣P\)−1\)\)\\Gamma\_\{P\}\(k,v\):\\mathcal\{Q\}\\to\\mathbb\{R\}^\{d\_\{v\}\}\\oplus\\mathbb\{R\},\\qquad q\\mapsto\\Bigl\(a\_\{k\}\(q\\mid P\)\\bigl\(v\-\\operatorname\{Att\}\(q\\mid P\)\\bigr\),\\;V\\bigl\(a\_\{k\}\(q\\mid P\)\-1\\bigr\)\\Bigr\)\(4\)the*response profile*ofPP\. We viewΓP​\(k,v\)\\Gamma\_\{P\}\(k,v\)as an element of the Hilbert spaceℋν≔L2​\(ν,ℝdv⊕ℝ\)\\mathcal\{H\}\_\{\\nu\}\\coloneqq L\_\{2\}\(\\nu,\\mathbb\{R\}^\{d\_\{v\}\}\\oplus\\mathbb\{R\}\)and define the covariance operator

ΣP,ν≔∫𝒳ΓP​\(k,v\)⊗ΓP​\(k,v\)​d​P​\(k,v\)\.\\textstyle\\Sigma\_\{P,\\nu\}\\coloneqq\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\}\(k,v\)\\otimes\\Gamma\_\{P\}\(k,v\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\)\.

The first coordinate ofΓP​\(k,v\)\\Gamma\_\{P\}\(k,v\)records the token’s contribution to the centered numerator perturbation in \([3](https://arxiv.org/html/2607.01520#S3.E3)\)\. The second coordinate records its contribution to the centered denominator perturbation, scaled by the value boundVVso that both coordinates have the same units\. The query distribution enters through the norm onℋν\\mathcal\{H\}\_\{\\nu\}: Directions that matter on frequent queries are weighted heavily, while directions that matter only on rare queries are weighted lightly\. Consequently,ΣP,ν\\Sigma\_\{P,\\nu\}measures the amount and geometry of query\-visible variation in the context\. While we do not claim thatΣP,ν\\Sigma\_\{P,\\nu\}is the*only*object to capture the intrinsic redundancy of a context, we argue that it is a meaningful choice\. To substantiate this, we give three examples illustrating that the spectral decay ofΣP,ν\\Sigma\_\{P,\\nu\}tracks the intuitive difficulty of compression in several natural regimes\.

\{restatable\}

\[Needles in a haystack\]exampleexpneedles Suppose a contextPPconsists ofmmapproximately equal “hay tokens” andn≪mn\\ll m“needle tokens” that are well separated from all other tokens and dominate underν\\nu\. Then,ΣP,ν\\Sigma\_\{P,\\nu\}has few large eigenvalues\.\{restatable\}\[Waning clusters\]exampleexpclusters Suppose a contextPPconsists ofmmwell\-separated clusters of nearly interchangeable tokens, and thejj\-th cluster dominates underν\\nuwith frequency of orderj−\(1\+α\)j^\{\-\(1\+\\alpha\)\}\. Then, the eigenvalues ofΣP,ν\\Sigma\_\{P,\\nu\}decay asr−αr^\{\-\\alpha\}\.\{restatable\}\[Random lookup\]exampleexplookup Suppose a contextPPconsists ofmmpairwise well\-separated tokens, and the query distributionν\\nuis diffuse\. Then,ΣP,ν\\Sigma\_\{P,\\nu\}has many large eigenvalues\.

### 3\.3Characterizing the achievable error of KV compression algorithms

Our goal in this paper is to characterize the fundamental limits of KV compression\. Specifically, we seek minimax rates in terms of the intrinsic compressibility of a context by asking:*What error does the best compression algorithm incur on its worst input\(P,ν\)\(P,\\nu\), as a function ofΣP,ν\\Sigma\_\{P,\\nu\}?*To make this precise, we first specify what information a compression algorithm may use\.

###### Definition 3\.2\(Compressor\)\.

AKK\-compressor is a mapA:𝒫fin​\(𝒳\)×𝒫​\(𝒬\)×Ω→𝒫K​\(𝒳\)A:\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)\\times\\mathcal\{P\}\(\\mathcal\{Q\}\)\\times\\Omega\\to\\mathcal\{P\}\_\{K\}\(\\mathcal\{X\}\)over a probability space\(Ω,ℱ,ℙ\)\(\\Omega,\\mathcal\{F\},\\mathbb\{P\}\)of random seeds\. We write𝒜Kaw\\mathcal\{A\}\_\{K\}^\{\\mathrm\{aw\}\}for the class of all such compressors\.

This definition allows compression algorithms to be randomized\. It also allows the compressor to depend on the query distributionν\\nu\. We call this the*query\-aware*setting\. In many applications, however, the compressed summary must be formed before the future query distribution is known, which we call the*query\-agnostic*setting, which we capture through the following restricted class\.

###### Definition 3\.3\(Query\-agnostic compressor\)\.

We call aKK\-compressorA∈𝒜KawA\\in\\mathcal\{A\}\_\{K\}^\{\\mathrm\{aw\}\}query\-agnostic ifA​\(P,ν,ω\)=A​\(P,ρ,ω\)A\(P,\\nu,\\omega\)=A\(P,\\rho,\\omega\)for allν,ρ∈𝒫​\(𝒬\)\\nu,\\rho\\in\\mathcal\{P\}\(\\mathcal\{Q\}\),P∈𝒫fin​\(𝒳\)P\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\), andω∈Ω\\omega\\in\\Omega\. We write𝒜Kag\\mathcal\{A\}\_\{K\}^\{\\mathrm\{ag\}\}for the class of all query\-agnosticKK\-compressors\.

We measure the performance of a class of compressors by its worst\-case expected mean squared error over a prescribed family of contexts and query distributions\.

###### Definition 3\.4\(Minimax risk\)\.

FixK≥1K\\geq 1\. For a family of compressors𝒞⊆𝒜Kaw\\mathcal\{C\}\\subseteq\\mathcal\{A\}\_\{K\}^\{\\mathrm\{aw\}\}, and a family of context and query measuresℱ⊆𝒫fin​\(𝒳\)×𝒫​\(𝒬\)\\mathcal\{F\}\\subseteq\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)\\times\\mathcal\{P\}\(\\mathcal\{Q\}\), define the minimax compression risk

ℜ​\(𝒞,ℱ\)≔infA∈𝒞sup\(P,ν\)∈ℱ𝔼ω∼ℙ⁡\[ℰP,ν​\(A​\(P,ν,ω\)\)\]\.\\mathfrak\{R\}\\\!\\left\(\\mathcal\{C\},\\mathcal\{F\}\\right\)\\coloneqq\\inf\_\{A\\in\\mathcal\{C\}\}\\sup\_\{\(P,\\nu\)\\in\\mathcal\{F\}\}\\operatorname\{\\mathbb\{E\}\}\_\{\\omega\\sim\\mathbb\{P\}\}\\\!\\left\[\\mathcal\{E\}\_\{P,\\nu\}\\bigl\(A\(P,\\nu,\\omega\)\\bigr\)\\right\]\.

Our objective is to characterizeℜ​\(𝒜Kaw,ℱ\)\\mathfrak\{R\}\\\!\\left\(\\mathcal\{A\}\_\{K\}^\{\\mathrm\{aw\}\},\\mathcal\{F\}\\right\)andℜ​\(𝒜Kag,ℱ\)\\mathfrak\{R\}\\\!\\left\(\\mathcal\{A\}\_\{K\}^\{\\mathrm\{ag\}\},\\mathcal\{F\}\\right\)through the response covariancesΣP,ν\\Sigma\_\{P,\\nu\}realized by\(P,ν\)∈ℱ\(P,\\nu\)\\in\\mathcal\{F\}, and to use this characterization to identify what compressed summaries must preserve in order to attain minimax\-optimal risk\.

## 4Minimax rates for KV compression

We now study the minimax risk of KV compression\.[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)illustrates our approach: compressing a KV cache amounts to replacing the context measurePPby a sparse reweighting, so that many tokens receive weight0while the remaining tokens are reweighted to preserve the effect of the full context on the attention map\. To track this effect, we represent each key–value pair\(k,v\)∈𝒳\(k,v\)\\in\\mathcal\{X\}by its response profileΓP​\(k,v\)∈ℋν\\Gamma\_\{P\}\(k,v\)\\in\\mathcal\{H\}\_\{\\nu\}, which records the contribution of\(k,v\)\(k,v\)to the attention numerator and denominator under reweightings ofPP\. These individual contributions combine through their*barycenter*inℋν\\mathcal\{H\}\_\{\\nu\}\. As we show, reweightings causing small barycenter displacement also cause small attention error, recasting KV compression as a balancing problem in Hilbert space: given a context measurePP, reweightPPso that its support is limited to at mostKKtokens while the barycenter of the response profiles remains nearly fixed\.[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)formulates this problem abstractly\.

𝒳\\mathcal\{X\}\(k,v\)\(k,v\)ℋν\\mathcal\{H\}\_\{\\nu\}ΓP​\(k,v\)\\Gamma\_\{P\}\(k,v\)𝒳\\mathcal\{X\}ℋν\\mathcal\{H\}\_\{\\nu\}𝒳\\mathcal\{X\}ℋν\\mathcal\{H\}\_\{\\nu\}𝒳\\mathcal\{X\}ℋν\\mathcal\{H\}\_\{\\nu\}supinf‖Att⁡\(q\|\)−Att⁡\(q\|\)‖L2​\(ν\)≍‖‖ℋν\\displaystyle\\sup\_\{\\mathord\{\\vbox\{\\hbox\{ \\hbox to6\.92pt\{\\vbox to4\.34pt\{\\pgfpicture\\makeatletter\\hbox\{\\thinspace\\lower\-0\.175pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\}\{\}\{\{\}\{\}\{\{\}\}\{\} \{\}\{\{\}\}\{\}\{\}\{\}\{\}\{\{\}\}\{\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\color\[rgb\]\{0\.65,0\.65,0\.65\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.65,0\.65,0\.65\}\\pgfsys@color@gray@stroke\{0\.65\}\\pgfsys@invoke\{ \}\\pgfsys@color@gray@fill\{0\.65\}\\pgfsys@invoke\{ \}\\definecolor\{pgffillcolor\}\{rgb\}\{0\.65,0\.65,0\.65\}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\}\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@moveto\{0\.0pt\}\{0\.8pt\}\\pgfsys@lineto\{0\.0pt\}\{3\.19415pt\}\\pgfsys@curveto\{0\.0pt\}\{3\.63599pt\}\{0\.35817pt\}\{3\.99416pt\}\{0\.8pt\}\{3\.99416pt\}\\pgfsys@lineto\{5\.77103pt\}\{3\.99416pt\}\\pgfsys@curveto\{6\.21286pt\}\{3\.99416pt\}\{6\.57103pt\}\{3\.63599pt\}\{6\.57103pt\}\{3\.19415pt\}\\pgfsys@lineto\{6\.57103pt\}\{0\.8pt\}\\pgfsys@curveto\{6\.57103pt\}\{0\.35817pt\}\{6\.21286pt\}\{0\.0pt\}\{5\.77103pt\}\{0\.0pt\}\\pgfsys@lineto\{0\.8pt\}\{0\.0pt\}\\pgfsys@curveto\{0\.35817pt\}\{0\.0pt\}\{0\.0pt\}\{0\.35817pt\}\{0\.0pt\}\{0\.8pt\}\\pgfsys@closepath\\pgfsys@moveto\{6\.57103pt\}\{3\.99416pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\\par \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{1,0\.6,0\.2\}\\pgfsys@color@rgb@fill\{1\}\{0\.6\}\{0\.2\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{1,0\.6,0\.2\}\\pgfsys@color@rgb@fill\{1\}\{0\.6\}\{0\.2\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{0\.92448pt\}\{3\.08139pt\}\\pgfsys@lineto\{0\.82774pt\}\{3\.08139pt\}\\pgfsys@curveto\{0\.68968pt\}\{3\.08139pt\}\{0\.57774pt\}\{2\.96945pt\}\{0\.57774pt\}\{2\.83139pt\}\\pgfsys@lineto\{0\.57774pt\}\{2\.73465pt\}\\pgfsys@curveto\{0\.57774pt\}\{2\.59659pt\}\{0\.68968pt\}\{2\.48465pt\}\{0\.82774pt\}\{2\.48465pt\}\\pgfsys@lineto\{0\.92448pt\}\{2\.48465pt\}\\pgfsys@curveto\{1\.06255pt\}\{2\.48465pt\}\{1\.17448pt\}\{2\.59659pt\}\{1\.17448pt\}\{2\.73465pt\}\\pgfsys@lineto\{1\.17448pt\}\{2\.83139pt\}\\pgfsys@curveto\{1\.17448pt\}\{2\.96945pt\}\{1\.06255pt\}\{3\.08139pt\}\{0\.92448pt\}\{3\.08139pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.57774pt\}\{2\.48465pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{0\.87611pt\}\{2\.78302pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,0\.3,1\}\\pgfsys@color@rgb@fill\{0\.3\}\{0\.3\}\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,0\.3,1\}\\pgfsys@color@rgb@fill\{0\.3\}\{0\.3\}\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{2\.16142pt\}\{3\.44215pt\}\\pgfsys@lineto\{2\.06468pt\}\{3\.44215pt\}\\pgfsys@curveto\{1\.92662pt\}\{3\.44215pt\}\{1\.81468pt\}\{3\.33022pt\}\{1\.81468pt\}\{3\.19215pt\}\\pgfsys@lineto\{1\.81468pt\}\{3\.09541pt\}\\pgfsys@curveto\{1\.81468pt\}\{2\.95735pt\}\{1\.92662pt\}\{2\.84541pt\}\{2\.06468pt\}\{2\.84541pt\}\\pgfsys@lineto\{2\.16142pt\}\{2\.84541pt\}\\pgfsys@curveto\{2\.29948pt\}\{2\.84541pt\}\{2\.41142pt\}\{2\.95735pt\}\{2\.41142pt\}\{3\.09541pt\}\\pgfsys@lineto\{2\.41142pt\}\{3\.19215pt\}\\pgfsys@curveto\{2\.41142pt\}\{3\.33022pt\}\{2\.29948pt\}\{3\.44215pt\}\{2\.16142pt\}\{3\.44215pt\}\\pgfsys@closepath\\pgfsys@moveto\{1\.81468pt\}\{2\.84541pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{2\.11305pt\}\{3\.14378pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,1,1\}\\pgfsys@color@cmyk@fill\{0\.7\}\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,1,1\}\\pgfsys@color@cmyk@fill\{0\.7\}\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{5\.51132pt\}\{2\.35986pt\}\\pgfsys@lineto\{5\.41458pt\}\{2\.35986pt\}\\pgfsys@curveto\{5\.27652pt\}\{2\.35986pt\}\{5\.16458pt\}\{2\.24792pt\}\{5\.16458pt\}\{2\.10986pt\}\\pgfsys@lineto\{5\.16458pt\}\{2\.01312pt\}\\pgfsys@curveto\{5\.16458pt\}\{1\.87506pt\}\{5\.27652pt\}\{1\.76312pt\}\{5\.41458pt\}\{1\.76312pt\}\\pgfsys@lineto\{5\.51132pt\}\{1\.76312pt\}\\pgfsys@curveto\{5\.64938pt\}\{1\.76312pt\}\{5\.76132pt\}\{1\.87506pt\}\{5\.76132pt\}\{2\.01312pt\}\\pgfsys@lineto\{5\.76132pt\}\{2\.10986pt\}\\pgfsys@curveto\{5\.76132pt\}\{2\.24792pt\}\{5\.64938pt\}\{2\.35986pt\}\{5\.51132pt\}\{2\.35986pt\}\\pgfsys@closepath\\pgfsys@moveto\{5\.16458pt\}\{1\.76312pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{5\.46295pt\}\{2\.0615pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.7,0\.7,0\}\\pgfsys@color@cmyk@fill\{0\}\{0\}\{0\.7\}\{0\.3\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.7,0\.7,0\}\\pgfsys@color@cmyk@fill\{0\}\{0\}\{0\.7\}\{0\.3\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{1\.64601pt\}\{1\.01988pt\}\\pgfsys@lineto\{1\.54927pt\}\{1\.01988pt\}\\pgfsys@curveto\{1\.41121pt\}\{1\.01988pt\}\{1\.29927pt\}\{0\.90794pt\}\{1\.29927pt\}\{0\.76988pt\}\\pgfsys@lineto\{1\.29927pt\}\{0\.67314pt\}\\pgfsys@curveto\{1\.29927pt\}\{0\.53508pt\}\{1\.41121pt\}\{0\.42314pt\}\{1\.54927pt\}\{0\.42314pt\}\\pgfsys@lineto\{1\.64601pt\}\{0\.42314pt\}\\pgfsys@curveto\{1\.78407pt\}\{0\.42314pt\}\{1\.89601pt\}\{0\.53508pt\}\{1\.89601pt\}\{0\.67314pt\}\\pgfsys@lineto\{1\.89601pt\}\{0\.76988pt\}\\pgfsys@curveto\{1\.89601pt\}\{0\.90794pt\}\{1\.78407pt\}\{1\.01988pt\}\{1\.64601pt\}\{1\.01988pt\}\\pgfsys@closepath\\pgfsys@moveto\{1\.29927pt\}\{0\.42314pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{1\.59764pt\}\{0\.72151pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.8125,0\.5,0\.1875\}\\pgfsys@color@rgb@fill\{0\.8125\}\{0\.5\}\{0\.1875\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.8125,0\.5,0\.1875\}\\pgfsys@color@rgb@fill\{0\.8125\}\{0\.5\}\{0\.1875\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{4\.04253pt\}\{3\.00409pt\}\\pgfsys@lineto\{3\.94579pt\}\{3\.00409pt\}\\pgfsys@curveto\{3\.80772pt\}\{3\.00409pt\}\{3\.69579pt\}\{2\.89215pt\}\{3\.69579pt\}\{2\.75409pt\}\\pgfsys@lineto\{3\.69579pt\}\{2\.65735pt\}\\pgfsys@curveto\{3\.69579pt\}\{2\.51929pt\}\{3\.80772pt\}\{2\.40735pt\}\{3\.94579pt\}\{2\.40735pt\}\\pgfsys@lineto\{4\.04253pt\}\{2\.40735pt\}\\pgfsys@curveto\{4\.18059pt\}\{2\.40735pt\}\{4\.29253pt\}\{2\.51929pt\}\{4\.29253pt\}\{2\.65735pt\}\\pgfsys@lineto\{4\.29253pt\}\{2\.75409pt\}\\pgfsys@curveto\{4\.29253pt\}\{2\.89215pt\}\{4\.18059pt\}\{3\.00409pt\}\{4\.04253pt\}\{3\.00409pt\}\\pgfsys@closepath\\pgfsys@moveto\{3\.69579pt\}\{2\.40735pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{3\.99416pt\}\{2\.70572pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.825,0\.3,0\.475\}\\pgfsys@color@rgb@fill\{0\.825\}\{0\.3\}\{0\.475\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.825,0\.3,0\.475\}\\pgfsys@color@rgb@fill\{0\.825\}\{0\.3\}\{0\.475\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{2\.93445pt\}\{1\.71565pt\}\\pgfsys@lineto\{2\.8377pt\}\{1\.71565pt\}\\pgfsys@curveto\{2\.69965pt\}\{1\.71565pt\}\{2\.5877pt\}\{1\.60371pt\}\{2\.5877pt\}\{1\.46565pt\}\\pgfsys@lineto\{2\.5877pt\}\{1\.36891pt\}\\pgfsys@curveto\{2\.5877pt\}\{1\.23085pt\}\{2\.69965pt\}\{1\.11891pt\}\{2\.8377pt\}\{1\.11891pt\}\\pgfsys@lineto\{2\.93445pt\}\{1\.11891pt\}\\pgfsys@curveto\{3\.07251pt\}\{1\.11891pt\}\{3\.18445pt\}\{1\.23085pt\}\{3\.18445pt\}\{1\.36891pt\}\\pgfsys@lineto\{3\.18445pt\}\{1\.46565pt\}\\pgfsys@curveto\{3\.18445pt\}\{1\.60371pt\}\{3\.07251pt\}\{1\.71565pt\}\{2\.93445pt\}\{1\.71565pt\}\\pgfsys@closepath\\pgfsys@moveto\{2\.5877pt\}\{1\.11891pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{2\.88608pt\}\{1\.41728pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0,0\.6,0\}\\pgfsys@color@rgb@fill\{0\}\{0\.6\}\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0,0\.6,0\}\\pgfsys@color@rgb@fill\{0\}\{0\.6\}\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{4\.84135pt\}\{1\.14874pt\}\\pgfsys@lineto\{4\.74461pt\}\{1\.14874pt\}\\pgfsys@curveto\{4\.60655pt\}\{1\.14874pt\}\{4\.49461pt\}\{1\.0368pt\}\{4\.49461pt\}\{0\.89874pt\}\\pgfsys@lineto\{4\.49461pt\}\{0\.802pt\}\\pgfsys@curveto\{4\.49461pt\}\{0\.66394pt\}\{4\.60655pt\}\{0\.552pt\}\{4\.74461pt\}\{0\.552pt\}\\pgfsys@lineto\{4\.84135pt\}\{0\.552pt\}\\pgfsys@curveto\{4\.97942pt\}\{0\.552pt\}\{5\.09135pt\}\{0\.66394pt\}\{5\.09135pt\}\{0\.802pt\}\\pgfsys@lineto\{5\.09135pt\}\{0\.89874pt\}\\pgfsys@curveto\{5\.09135pt\}\{1\.0368pt\}\{4\.97942pt\}\{1\.14874pt\}\{4\.84135pt\}\{1\.14874pt\}\\pgfsys@closepath\\pgfsys@moveto\{4\.49461pt\}\{0\.552pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{4\.79298pt\}\{0\.85037pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\}\}\}\\;\\inf\_\{\\mathord\{\\vbox\{\\hbox\{ \\hbox to6\.92pt\{\\vbox to4\.34pt\{\\pgfpicture\\makeatletter\\hbox\{\\thinspace\\lower\-0\.175pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\}\{\}\{\{\}\{\}\{\{\}\}\{\} \{\}\{\{\}\}\{\}\{\}\{\}\{\}\{\{\}\}\{\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\color\[rgb\]\{0\.65,0\.65,0\.65\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.65,0\.65,0\.65\}\\pgfsys@color@gray@stroke\{0\.65\}\\pgfsys@invoke\{ \}\\pgfsys@color@gray@fill\{0\.65\}\\pgfsys@invoke\{ \}\\definecolor\{pgffillcolor\}\{rgb\}\{0\.65,0\.65,0\.65\}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\}\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@moveto\{0\.0pt\}\{0\.8pt\}\\pgfsys@lineto\{0\.0pt\}\{3\.19415pt\}\\pgfsys@curveto\{0\.0pt\}\{3\.63599pt\}\{0\.35817pt\}\{3\.99416pt\}\{0\.8pt\}\{3\.99416pt\}\\pgfsys@lineto\{5\.77103pt\}\{3\.99416pt\}\\pgfsys@curveto\{6\.21286pt\}\{3\.99416pt\}\{6\.57103pt\}\{3\.63599pt\}\{6\.57103pt\}\{3\.19415pt\}\\pgfsys@lineto\{6\.57103pt\}\{0\.8pt\}\\pgfsys@curveto\{6\.57103pt\}\{0\.35817pt\}\{6\.21286pt\}\{0\.0pt\}\{5\.77103pt\}\{0\.0pt\}\\pgfsys@lineto\{0\.8pt\}\{0\.0pt\}\\pgfsys@curveto\{0\.35817pt\}\{0\.0pt\}\{0\.0pt\}\{0\.35817pt\}\{0\.0pt\}\{0\.8pt\}\\pgfsys@closepath\\pgfsys@moveto\{6\.57103pt\}\{3\.99416pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\\par \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{1,0\.6,0\.2\}\\pgfsys@color@rgb@fill\{1\}\{0\.6\}\{0\.2\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{1,0\.6,0\.2\}\\pgfsys@color@rgb@fill\{1\}\{0\.6\}\{0\.2\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{0\.92448pt\}\{3\.08139pt\}\\pgfsys@lineto\{0\.82774pt\}\{3\.08139pt\}\\pgfsys@curveto\{0\.68968pt\}\{3\.08139pt\}\{0\.57774pt\}\{2\.96945pt\}\{0\.57774pt\}\{2\.83139pt\}\\pgfsys@lineto\{0\.57774pt\}\{2\.73465pt\}\\pgfsys@curveto\{0\.57774pt\}\{2\.59659pt\}\{0\.68968pt\}\{2\.48465pt\}\{0\.82774pt\}\{2\.48465pt\}\\pgfsys@lineto\{0\.92448pt\}\{2\.48465pt\}\\pgfsys@curveto\{1\.06255pt\}\{2\.48465pt\}\{1\.17448pt\}\{2\.59659pt\}\{1\.17448pt\}\{2\.73465pt\}\\pgfsys@lineto\{1\.17448pt\}\{2\.83139pt\}\\pgfsys@curveto\{1\.17448pt\}\{2\.96945pt\}\{1\.06255pt\}\{3\.08139pt\}\{0\.92448pt\}\{3\.08139pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.57774pt\}\{2\.48465pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{0\.87611pt\}\{2\.78302pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,0\.3,1\}\\pgfsys@color@rgb@fill\{0\.3\}\{0\.3\}\{1\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,0\.3,1\}\\pgfsys@color@rgb@fill\{0\.3\}\{0\.3\}\{1\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{2\.16142pt\}\{3\.44215pt\}\\pgfsys@lineto\{2\.06468pt\}\{3\.44215pt\}\\pgfsys@curveto\{1\.92662pt\}\{3\.44215pt\}\{1\.81468pt\}\{3\.33022pt\}\{1\.81468pt\}\{3\.19215pt\}\\pgfsys@lineto\{1\.81468pt\}\{3\.09541pt\}\\pgfsys@curveto\{1\.81468pt\}\{2\.95735pt\}\{1\.92662pt\}\{2\.84541pt\}\{2\.06468pt\}\{2\.84541pt\}\\pgfsys@lineto\{2\.16142pt\}\{2\.84541pt\}\\pgfsys@curveto\{2\.29948pt\}\{2\.84541pt\}\{2\.41142pt\}\{2\.95735pt\}\{2\.41142pt\}\{3\.09541pt\}\\pgfsys@lineto\{2\.41142pt\}\{3\.19215pt\}\\pgfsys@curveto\{2\.41142pt\}\{3\.33022pt\}\{2\.29948pt\}\{3\.44215pt\}\{2\.16142pt\}\{3\.44215pt\}\\pgfsys@closepath\\pgfsys@moveto\{1\.81468pt\}\{2\.84541pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{2\.11305pt\}\{3\.14378pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,1,1\}\\pgfsys@color@cmyk@fill\{0\.7\}\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,1,1\}\\pgfsys@color@cmyk@fill\{0\.7\}\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{5\.51132pt\}\{2\.35986pt\}\\pgfsys@lineto\{5\.41458pt\}\{2\.35986pt\}\\pgfsys@curveto\{5\.27652pt\}\{2\.35986pt\}\{5\.16458pt\}\{2\.24792pt\}\{5\.16458pt\}\{2\.10986pt\}\\pgfsys@lineto\{5\.16458pt\}\{2\.01312pt\}\\pgfsys@curveto\{5\.16458pt\}\{1\.87506pt\}\{5\.27652pt\}\{1\.76312pt\}\{5\.41458pt\}\{1\.76312pt\}\\pgfsys@lineto\{5\.51132pt\}\{1\.76312pt\}\\pgfsys@curveto\{5\.64938pt\}\{1\.76312pt\}\{5\.76132pt\}\{1\.87506pt\}\{5\.76132pt\}\{2\.01312pt\}\\pgfsys@lineto\{5\.76132pt\}\{2\.10986pt\}\\pgfsys@curveto\{5\.76132pt\}\{2\.24792pt\}\{5\.64938pt\}\{2\.35986pt\}\{5\.51132pt\}\{2\.35986pt\}\\pgfsys@closepath\\pgfsys@moveto\{5\.16458pt\}\{1\.76312pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{5\.46295pt\}\{2\.0615pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.7,0\.7,0\}\\pgfsys@color@cmyk@fill\{0\}\{0\}\{0\.7\}\{0\.3\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.7,0\.7,0\}\\pgfsys@color@cmyk@fill\{0\}\{0\}\{0\.7\}\{0\.3\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{1\.64601pt\}\{1\.01988pt\}\\pgfsys@lineto\{1\.54927pt\}\{1\.01988pt\}\\pgfsys@curveto\{1\.41121pt\}\{1\.01988pt\}\{1\.29927pt\}\{0\.90794pt\}\{1\.29927pt\}\{0\.76988pt\}\\pgfsys@lineto\{1\.29927pt\}\{0\.67314pt\}\\pgfsys@curveto\{1\.29927pt\}\{0\.53508pt\}\{1\.41121pt\}\{0\.42314pt\}\{1\.54927pt\}\{0\.42314pt\}\\pgfsys@lineto\{1\.64601pt\}\{0\.42314pt\}\\pgfsys@curveto\{1\.78407pt\}\{0\.42314pt\}\{1\.89601pt\}\{0\.53508pt\}\{1\.89601pt\}\{0\.67314pt\}\\pgfsys@lineto\{1\.89601pt\}\{0\.76988pt\}\\pgfsys@curveto\{1\.89601pt\}\{0\.90794pt\}\{1\.78407pt\}\{1\.01988pt\}\{1\.64601pt\}\{1\.01988pt\}\\pgfsys@closepath\\pgfsys@moveto\{1\.29927pt\}\{0\.42314pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{1\.59764pt\}\{0\.72151pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.8125,0\.5,0\.1875\}\\pgfsys@color@rgb@fill\{0\.8125\}\{0\.5\}\{0\.1875\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.8125,0\.5,0\.1875\}\\pgfsys@color@rgb@fill\{0\.8125\}\{0\.5\}\{0\.1875\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{4\.04253pt\}\{3\.00409pt\}\\pgfsys@lineto\{3\.94579pt\}\{3\.00409pt\}\\pgfsys@curveto\{3\.80772pt\}\{3\.00409pt\}\{3\.69579pt\}\{2\.89215pt\}\{3\.69579pt\}\{2\.75409pt\}\\pgfsys@lineto\{3\.69579pt\}\{2\.65735pt\}\\pgfsys@curveto\{3\.69579pt\}\{2\.51929pt\}\{3\.80772pt\}\{2\.40735pt\}\{3\.94579pt\}\{2\.40735pt\}\\pgfsys@lineto\{4\.04253pt\}\{2\.40735pt\}\\pgfsys@curveto\{4\.18059pt\}\{2\.40735pt\}\{4\.29253pt\}\{2\.51929pt\}\{4\.29253pt\}\{2\.65735pt\}\\pgfsys@lineto\{4\.29253pt\}\{2\.75409pt\}\\pgfsys@curveto\{4\.29253pt\}\{2\.89215pt\}\{4\.18059pt\}\{3\.00409pt\}\{4\.04253pt\}\{3\.00409pt\}\\pgfsys@closepath\\pgfsys@moveto\{3\.69579pt\}\{2\.40735pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{3\.99416pt\}\{2\.70572pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.825,0\.3,0\.475\}\\pgfsys@color@rgb@fill\{0\.825\}\{0\.3\}\{0\.475\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.825,0\.3,0\.475\}\\pgfsys@color@rgb@fill\{0\.825\}\{0\.3\}\{0\.475\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{2\.93445pt\}\{1\.71565pt\}\\pgfsys@lineto\{2\.8377pt\}\{1\.71565pt\}\\pgfsys@curveto\{2\.69965pt\}\{1\.71565pt\}\{2\.5877pt\}\{1\.60371pt\}\{2\.5877pt\}\{1\.46565pt\}\\pgfsys@lineto\{2\.5877pt\}\{1\.36891pt\}\\pgfsys@curveto\{2\.5877pt\}\{1\.23085pt\}\{2\.69965pt\}\{1\.11891pt\}\{2\.8377pt\}\{1\.11891pt\}\\pgfsys@lineto\{2\.93445pt\}\{1\.11891pt\}\\pgfsys@curveto\{3\.07251pt\}\{1\.11891pt\}\{3\.18445pt\}\{1\.23085pt\}\{3\.18445pt\}\{1\.36891pt\}\\pgfsys@lineto\{3\.18445pt\}\{1\.46565pt\}\\pgfsys@curveto\{3\.18445pt\}\{1\.60371pt\}\{3\.07251pt\}\{1\.71565pt\}\{2\.93445pt\}\{1\.71565pt\}\\pgfsys@closepath\\pgfsys@moveto\{2\.5877pt\}\{1\.11891pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{2\.88608pt\}\{1\.41728pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0,0\.6,0\}\\pgfsys@color@rgb@fill\{0\}\{0\.6\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0,0\.6,0\}\\pgfsys@color@rgb@fill\{0\}\{0\.6\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{4\.84135pt\}\{1\.14874pt\}\\pgfsys@lineto\{4\.74461pt\}\{1\.14874pt\}\\pgfsys@curveto\{4\.60655pt\}\{1\.14874pt\}\{4\.49461pt\}\{1\.0368pt\}\{4\.49461pt\}\{0\.89874pt\}\\pgfsys@lineto\{4\.49461pt\}\{0\.802pt\}\\pgfsys@curveto\{4\.49461pt\}\{0\.66394pt\}\{4\.60655pt\}\{0\.552pt\}\{4\.74461pt\}\{0\.552pt\}\\pgfsys@lineto\{4\.84135pt\}\{0\.552pt\}\\pgfsys@curveto\{4\.97942pt\}\{0\.552pt\}\{5\.09135pt\}\{0\.66394pt\}\{5\.09135pt\}\{0\.802pt\}\\pgfsys@lineto\{5\.09135pt\}\{0\.89874pt\}\\pgfsys@curveto\{5\.09135pt\}\{1\.0368pt\}\{4\.97942pt\}\{1\.14874pt\}\{4\.84135pt\}\{1\.14874pt\}\\pgfsys@closepath\\pgfsys@moveto\{4\.49461pt\}\{0\.552pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{4\.79298pt\}\{0\.85037pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\}\}\}\\;\\left\\\|\\operatorname\{Att\}\\\!\\left\(q\\,\\middle\|\\,\\mathord\{\\vbox\{\\hbox\{ \\hbox to9\.74pt\{\\vbox to6\.06pt\{\\pgfpicture\\makeatletter\\hbox\{\\thinspace\\lower\-0\.175pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\}\{\}\{\{\}\{\}\{\{\}\}\{\} \{\}\{\{\}\}\{\}\{\}\{\}\{\}\{\{\}\}\{\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\color\[rgb\]\{0\.65,0\.65,0\.65\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.65,0\.65,0\.65\}\\pgfsys@color@gray@stroke\{0\.65\}\\pgfsys@invoke\{ \}\\pgfsys@color@gray@fill\{0\.65\}\\pgfsys@invoke\{ \}\\definecolor\{pgffillcolor\}\{rgb\}\{0\.65,0\.65,0\.65\}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\}\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@moveto\{0\.0pt\}\{0\.8pt\}\\pgfsys@lineto\{0\.0pt\}\{4\.90593pt\}\\pgfsys@curveto\{0\.0pt\}\{5\.34776pt\}\{0\.35817pt\}\{5\.70593pt\}\{0\.8pt\}\{5\.70593pt\}\\pgfsys@lineto\{8\.58717pt\}\{5\.70593pt\}\\pgfsys@curveto\{9\.029pt\}\{5\.70593pt\}\{9\.38718pt\}\{5\.34776pt\}\{9\.38718pt\}\{4\.90593pt\}\\pgfsys@lineto\{9\.38718pt\}\{0\.8pt\}\\pgfsys@curveto\{9\.38718pt\}\{0\.35817pt\}\{9\.029pt\}\{0\.0pt\}\{8\.58717pt\}\{0\.0pt\}\\pgfsys@lineto\{0\.8pt\}\{0\.0pt\}\\pgfsys@curveto\{0\.35817pt\}\{0\.0pt\}\{0\.0pt\}\{0\.35817pt\}\{0\.0pt\}\{0\.8pt\}\\pgfsys@closepath\\pgfsys@moveto\{9\.38718pt\}\{5\.70593pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\\par \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{1,0\.6,0\.2\}\\pgfsys@color@rgb@fill\{1\}\{0\.6\}\{0\.2\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{1,0\.6,0\.2\}\\pgfsys@color@rgb@fill\{1\}\{0\.6\}\{0\.2\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{1\.42784pt\}\{4\.40198pt\}\\pgfsys@lineto\{1\.07536pt\}\{4\.40198pt\}\\pgfsys@curveto\{0\.9373pt\}\{4\.40198pt\}\{0\.82536pt\}\{4\.29004pt\}\{0\.82536pt\}\{4\.15198pt\}\\pgfsys@lineto\{0\.82536pt\}\{3\.7995pt\}\\pgfsys@curveto\{0\.82536pt\}\{3\.66144pt\}\{0\.9373pt\}\{3\.5495pt\}\{1\.07536pt\}\{3\.5495pt\}\\pgfsys@lineto\{1\.42784pt\}\{3\.5495pt\}\\pgfsys@curveto\{1\.5659pt\}\{3\.5495pt\}\{1\.67784pt\}\{3\.66144pt\}\{1\.67784pt\}\{3\.7995pt\}\\pgfsys@lineto\{1\.67784pt\}\{4\.15198pt\}\\pgfsys@curveto\{1\.67784pt\}\{4\.29004pt\}\{1\.5659pt\}\{4\.40198pt\}\{1\.42784pt\}\{4\.40198pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.82536pt\}\{3\.5495pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{1\.2516pt\}\{3\.97574pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,0\.3,1\}\\pgfsys@color@rgb@fill\{0\.3\}\{0\.3\}\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,0\.3,1\}\\pgfsys@color@rgb@fill\{0\.3\}\{0\.3\}\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{3\.19489pt\}\{4\.91736pt\}\\pgfsys@lineto\{2\.8424pt\}\{4\.91736pt\}\\pgfsys@curveto\{2\.70435pt\}\{4\.91736pt\}\{2\.5924pt\}\{4\.80542pt\}\{2\.5924pt\}\{4\.66736pt\}\\pgfsys@lineto\{2\.5924pt\}\{4\.31488pt\}\\pgfsys@curveto\{2\.5924pt\}\{4\.17682pt\}\{2\.70435pt\}\{4\.06488pt\}\{2\.8424pt\}\{4\.06488pt\}\\pgfsys@lineto\{3\.19489pt\}\{4\.06488pt\}\\pgfsys@curveto\{3\.33295pt\}\{4\.06488pt\}\{3\.44489pt\}\{4\.17682pt\}\{3\.44489pt\}\{4\.31488pt\}\\pgfsys@lineto\{3\.44489pt\}\{4\.66736pt\}\\pgfsys@curveto\{3\.44489pt\}\{4\.80542pt\}\{3\.33295pt\}\{4\.91736pt\}\{3\.19489pt\}\{4\.91736pt\}\\pgfsys@closepath\\pgfsys@moveto\{2\.5924pt\}\{4\.06488pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{3\.01865pt\}\{4\.49112pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,1,1\}\\pgfsys@color@cmyk@fill\{0\.7\}\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,1,1\}\\pgfsys@color@cmyk@fill\{0\.7\}\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{7\.98045pt\}\{3\.37123pt\}\\pgfsys@lineto\{7\.62798pt\}\{3\.37123pt\}\\pgfsys@curveto\{7\.48991pt\}\{3\.37123pt\}\{7\.37798pt\}\{3\.2593pt\}\{7\.37798pt\}\{3\.12123pt\}\\pgfsys@lineto\{7\.37798pt\}\{2\.76875pt\}\\pgfsys@curveto\{7\.37798pt\}\{2\.63069pt\}\{7\.48991pt\}\{2\.51875pt\}\{7\.62798pt\}\{2\.51875pt\}\\pgfsys@lineto\{7\.98045pt\}\{2\.51875pt\}\\pgfsys@curveto\{8\.11852pt\}\{2\.51875pt\}\{8\.23045pt\}\{2\.63069pt\}\{8\.23045pt\}\{2\.76875pt\}\\pgfsys@lineto\{8\.23045pt\}\{3\.12123pt\}\\pgfsys@curveto\{8\.23045pt\}\{3\.2593pt\}\{8\.11852pt\}\{3\.37123pt\}\{7\.98045pt\}\{3\.37123pt\}\\pgfsys@closepath\\pgfsys@moveto\{7\.37798pt\}\{2\.51875pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{7\.80421pt\}\{2\.94499pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.7,0\.7,0\}\\pgfsys@color@cmyk@fill\{0\}\{0\}\{0\.7\}\{0\.3\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.7,0\.7,0\}\\pgfsys@color@cmyk@fill\{0\}\{0\}\{0\.7\}\{0\.3\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{2\.45859pt\}\{1\.45697pt\}\\pgfsys@lineto\{2\.10611pt\}\{1\.45697pt\}\\pgfsys@curveto\{1\.96805pt\}\{1\.45697pt\}\{1\.85611pt\}\{1\.34503pt\}\{1\.85611pt\}\{1\.20697pt\}\\pgfsys@lineto\{1\.85611pt\}\{0\.85449pt\}\\pgfsys@curveto\{1\.85611pt\}\{0\.71643pt\}\{1\.96805pt\}\{0\.60449pt\}\{2\.10611pt\}\{0\.60449pt\}\\pgfsys@lineto\{2\.45859pt\}\{0\.60449pt\}\\pgfsys@curveto\{2\.59665pt\}\{0\.60449pt\}\{2\.70859pt\}\{0\.71643pt\}\{2\.70859pt\}\{0\.85449pt\}\\pgfsys@lineto\{2\.70859pt\}\{1\.20697pt\}\\pgfsys@curveto\{2\.70859pt\}\{1\.34503pt\}\{2\.59665pt\}\{1\.45697pt\}\{2\.45859pt\}\{1\.45697pt\}\\pgfsys@closepath\\pgfsys@moveto\{1\.85611pt\}\{0\.60449pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{2\.28235pt\}\{1\.03073pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.8125,0\.5,0\.1875\}\\pgfsys@color@rgb@fill\{0\.8125\}\{0\.5\}\{0\.1875\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.8125,0\.5,0\.1875\}\\pgfsys@color@rgb@fill\{0\.8125\}\{0\.5\}\{0\.1875\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{5\.88217pt\}\{4\.29155pt\}\\pgfsys@lineto\{5\.5297pt\}\{4\.29155pt\}\\pgfsys@curveto\{5\.39163pt\}\{4\.29155pt\}\{5\.2797pt\}\{4\.17961pt\}\{5\.2797pt\}\{4\.04155pt\}\\pgfsys@lineto\{5\.2797pt\}\{3\.68907pt\}\\pgfsys@curveto\{5\.2797pt\}\{3\.55101pt\}\{5\.39163pt\}\{3\.43907pt\}\{5\.5297pt\}\{3\.43907pt\}\\pgfsys@lineto\{5\.88217pt\}\{3\.43907pt\}\\pgfsys@curveto\{6\.02023pt\}\{3\.43907pt\}\{6\.13217pt\}\{3\.55101pt\}\{6\.13217pt\}\{3\.68907pt\}\\pgfsys@lineto\{6\.13217pt\}\{4\.04155pt\}\\pgfsys@curveto\{6\.13217pt\}\{4\.17961pt\}\{6\.02023pt\}\{4\.29155pt\}\{5\.88217pt\}\{4\.29155pt\}\\pgfsys@closepath\\pgfsys@moveto\{5\.2797pt\}\{3\.43907pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{5\.70593pt\}\{3\.86531pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.825,0\.3,0\.475\}\\pgfsys@color@rgb@fill\{0\.825\}\{0\.3\}\{0\.475\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.825,0\.3,0\.475\}\\pgfsys@color@rgb@fill\{0\.825\}\{0\.3\}\{0\.475\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{4\.29921pt\}\{2\.45093pt\}\\pgfsys@lineto\{3\.94673pt\}\{2\.45093pt\}\\pgfsys@curveto\{3\.80867pt\}\{2\.45093pt\}\{3\.69673pt\}\{2\.33899pt\}\{3\.69673pt\}\{2\.20093pt\}\\pgfsys@lineto\{3\.69673pt\}\{1\.84845pt\}\\pgfsys@curveto\{3\.69673pt\}\{1\.71039pt\}\{3\.80867pt\}\{1\.59845pt\}\{3\.94673pt\}\{1\.59845pt\}\\pgfsys@lineto\{4\.29921pt\}\{1\.59845pt\}\\pgfsys@curveto\{4\.43727pt\}\{1\.59845pt\}\{4\.54921pt\}\{1\.71039pt\}\{4\.54921pt\}\{1\.84845pt\}\\pgfsys@lineto\{4\.54921pt\}\{2\.20093pt\}\\pgfsys@curveto\{4\.54921pt\}\{2\.33899pt\}\{4\.43727pt\}\{2\.45093pt\}\{4\.29921pt\}\{2\.45093pt\}\\pgfsys@closepath\\pgfsys@moveto\{3\.69673pt\}\{1\.59845pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{4\.12297pt\}\{2\.02469pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0,0\.6,0\}\\pgfsys@color@rgb@fill\{0\}\{0\.6\}\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0,0\.6,0\}\\pgfsys@color@rgb@fill\{0\}\{0\.6\}\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{7\.02335pt\}\{1\.64105pt\}\\pgfsys@lineto\{6\.67087pt\}\{1\.64105pt\}\\pgfsys@curveto\{6\.5328pt\}\{1\.64105pt\}\{6\.42087pt\}\{1\.52911pt\}\{6\.42087pt\}\{1\.39105pt\}\\pgfsys@lineto\{6\.42087pt\}\{1\.03857pt\}\\pgfsys@curveto\{6\.42087pt\}\{0\.90051pt\}\{6\.5328pt\}\{0\.78857pt\}\{6\.67087pt\}\{0\.78857pt\}\\pgfsys@lineto\{7\.02335pt\}\{0\.78857pt\}\\pgfsys@curveto\{7\.1614pt\}\{0\.78857pt\}\{7\.27335pt\}\{0\.90051pt\}\{7\.27335pt\}\{1\.03857pt\}\\pgfsys@lineto\{7\.27335pt\}\{1\.39105pt\}\\pgfsys@curveto\{7\.27335pt\}\{1\.52911pt\}\{7\.1614pt\}\{1\.64105pt\}\{7\.02335pt\}\{1\.64105pt\}\\pgfsys@closepath\\pgfsys@moveto\{6\.42087pt\}\{0\.78857pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{6\.8471pt\}\{1\.21481pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\}\}\\right\)\-\\operatorname\{Att\}\\\!\\left\(q\\,\\middle\|\\,\\mathord\{\\vbox\{\\hbox\{ \\hbox to9\.74pt\{\\vbox to6\.06pt\{\\pgfpicture\\makeatletter\\hbox\{\\thinspace\\lower\-0\.175pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\}\{\}\{\{\}\{\}\{\{\}\}\{\} \{\}\{\{\}\}\{\}\{\}\{\}\{\}\{\{\}\}\{\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\color\[rgb\]\{0\.65,0\.65,0\.65\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.65,0\.65,0\.65\}\\pgfsys@color@gray@stroke\{0\.65\}\\pgfsys@invoke\{ \}\\pgfsys@color@gray@fill\{0\.65\}\\pgfsys@invoke\{ \}\\definecolor\{pgffillcolor\}\{rgb\}\{0\.65,0\.65,0\.65\}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\}\\pgfsys@moveto\{0\.0pt\}\{0\.0pt\}\\pgfsys@moveto\{0\.0pt\}\{0\.8pt\}\\pgfsys@lineto\{0\.0pt\}\{4\.90593pt\}\\pgfsys@curveto\{0\.0pt\}\{5\.34776pt\}\{0\.35817pt\}\{5\.70593pt\}\{0\.8pt\}\{5\.70593pt\}\\pgfsys@lineto\{8\.58717pt\}\{5\.70593pt\}\\pgfsys@curveto\{9\.029pt\}\{5\.70593pt\}\{9\.38718pt\}\{5\.34776pt\}\{9\.38718pt\}\{4\.90593pt\}\\pgfsys@lineto\{9\.38718pt\}\{0\.8pt\}\\pgfsys@curveto\{9\.38718pt\}\{0\.35817pt\}\{9\.029pt\}\{0\.0pt\}\{8\.58717pt\}\{0\.0pt\}\\pgfsys@lineto\{0\.8pt\}\{0\.0pt\}\\pgfsys@curveto\{0\.35817pt\}\{0\.0pt\}\{0\.0pt\}\{0\.35817pt\}\{0\.0pt\}\{0\.8pt\}\\pgfsys@closepath\\pgfsys@moveto\{9\.38718pt\}\{5\.70593pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\\par \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{1,0\.6,0\.2\}\\pgfsys@color@rgb@fill\{1\}\{0\.6\}\{0\.2\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{1,0\.6,0\.2\}\\pgfsys@color@rgb@fill\{1\}\{0\.6\}\{0\.2\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{1\.42784pt\}\{4\.40198pt\}\\pgfsys@lineto\{1\.07536pt\}\{4\.40198pt\}\\pgfsys@curveto\{0\.9373pt\}\{4\.40198pt\}\{0\.82536pt\}\{4\.29004pt\}\{0\.82536pt\}\{4\.15198pt\}\\pgfsys@lineto\{0\.82536pt\}\{3\.7995pt\}\\pgfsys@curveto\{0\.82536pt\}\{3\.66144pt\}\{0\.9373pt\}\{3\.5495pt\}\{1\.07536pt\}\{3\.5495pt\}\\pgfsys@lineto\{1\.42784pt\}\{3\.5495pt\}\\pgfsys@curveto\{1\.5659pt\}\{3\.5495pt\}\{1\.67784pt\}\{3\.66144pt\}\{1\.67784pt\}\{3\.7995pt\}\\pgfsys@lineto\{1\.67784pt\}\{4\.15198pt\}\\pgfsys@curveto\{1\.67784pt\}\{4\.29004pt\}\{1\.5659pt\}\{4\.40198pt\}\{1\.42784pt\}\{4\.40198pt\}\\pgfsys@closepath\\pgfsys@moveto\{0\.82536pt\}\{3\.5495pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{1\.2516pt\}\{3\.97574pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,0\.3,1\}\\pgfsys@color@rgb@fill\{0\.3\}\{0\.3\}\{1\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,0\.3,1\}\\pgfsys@color@rgb@fill\{0\.3\}\{0\.3\}\{1\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{3\.19489pt\}\{4\.91736pt\}\\pgfsys@lineto\{2\.8424pt\}\{4\.91736pt\}\\pgfsys@curveto\{2\.70435pt\}\{4\.91736pt\}\{2\.5924pt\}\{4\.80542pt\}\{2\.5924pt\}\{4\.66736pt\}\\pgfsys@lineto\{2\.5924pt\}\{4\.31488pt\}\\pgfsys@curveto\{2\.5924pt\}\{4\.17682pt\}\{2\.70435pt\}\{4\.06488pt\}\{2\.8424pt\}\{4\.06488pt\}\\pgfsys@lineto\{3\.19489pt\}\{4\.06488pt\}\\pgfsys@curveto\{3\.33295pt\}\{4\.06488pt\}\{3\.44489pt\}\{4\.17682pt\}\{3\.44489pt\}\{4\.31488pt\}\\pgfsys@lineto\{3\.44489pt\}\{4\.66736pt\}\\pgfsys@curveto\{3\.44489pt\}\{4\.80542pt\}\{3\.33295pt\}\{4\.91736pt\}\{3\.19489pt\}\{4\.91736pt\}\\pgfsys@closepath\\pgfsys@moveto\{2\.5924pt\}\{4\.06488pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{3\.01865pt\}\{4\.49112pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,1,1\}\\pgfsys@color@cmyk@fill\{0\.7\}\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.3,1,1\}\\pgfsys@color@cmyk@fill\{0\.7\}\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{7\.98045pt\}\{3\.37123pt\}\\pgfsys@lineto\{7\.62798pt\}\{3\.37123pt\}\\pgfsys@curveto\{7\.48991pt\}\{3\.37123pt\}\{7\.37798pt\}\{3\.2593pt\}\{7\.37798pt\}\{3\.12123pt\}\\pgfsys@lineto\{7\.37798pt\}\{2\.76875pt\}\\pgfsys@curveto\{7\.37798pt\}\{2\.63069pt\}\{7\.48991pt\}\{2\.51875pt\}\{7\.62798pt\}\{2\.51875pt\}\\pgfsys@lineto\{7\.98045pt\}\{2\.51875pt\}\\pgfsys@curveto\{8\.11852pt\}\{2\.51875pt\}\{8\.23045pt\}\{2\.63069pt\}\{8\.23045pt\}\{2\.76875pt\}\\pgfsys@lineto\{8\.23045pt\}\{3\.12123pt\}\\pgfsys@curveto\{8\.23045pt\}\{3\.2593pt\}\{8\.11852pt\}\{3\.37123pt\}\{7\.98045pt\}\{3\.37123pt\}\\pgfsys@closepath\\pgfsys@moveto\{7\.37798pt\}\{2\.51875pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{7\.80421pt\}\{2\.94499pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.7,0\.7,0\}\\pgfsys@color@cmyk@fill\{0\}\{0\}\{0\.7\}\{0\.3\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.7,0\.7,0\}\\pgfsys@color@cmyk@fill\{0\}\{0\}\{0\.7\}\{0\.3\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{2\.45859pt\}\{1\.45697pt\}\\pgfsys@lineto\{2\.10611pt\}\{1\.45697pt\}\\pgfsys@curveto\{1\.96805pt\}\{1\.45697pt\}\{1\.85611pt\}\{1\.34503pt\}\{1\.85611pt\}\{1\.20697pt\}\\pgfsys@lineto\{1\.85611pt\}\{0\.85449pt\}\\pgfsys@curveto\{1\.85611pt\}\{0\.71643pt\}\{1\.96805pt\}\{0\.60449pt\}\{2\.10611pt\}\{0\.60449pt\}\\pgfsys@lineto\{2\.45859pt\}\{0\.60449pt\}\\pgfsys@curveto\{2\.59665pt\}\{0\.60449pt\}\{2\.70859pt\}\{0\.71643pt\}\{2\.70859pt\}\{0\.85449pt\}\\pgfsys@lineto\{2\.70859pt\}\{1\.20697pt\}\\pgfsys@curveto\{2\.70859pt\}\{1\.34503pt\}\{2\.59665pt\}\{1\.45697pt\}\{2\.45859pt\}\{1\.45697pt\}\\pgfsys@closepath\\pgfsys@moveto\{1\.85611pt\}\{0\.60449pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{2\.28235pt\}\{1\.03073pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.8125,0\.5,0\.1875\}\\pgfsys@color@rgb@fill\{0\.8125\}\{0\.5\}\{0\.1875\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.8125,0\.5,0\.1875\}\\pgfsys@color@rgb@fill\{0\.8125\}\{0\.5\}\{0\.1875\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{5\.88217pt\}\{4\.29155pt\}\\pgfsys@lineto\{5\.5297pt\}\{4\.29155pt\}\\pgfsys@curveto\{5\.39163pt\}\{4\.29155pt\}\{5\.2797pt\}\{4\.17961pt\}\{5\.2797pt\}\{4\.04155pt\}\\pgfsys@lineto\{5\.2797pt\}\{3\.68907pt\}\\pgfsys@curveto\{5\.2797pt\}\{3\.55101pt\}\{5\.39163pt\}\{3\.43907pt\}\{5\.5297pt\}\{3\.43907pt\}\\pgfsys@lineto\{5\.88217pt\}\{3\.43907pt\}\\pgfsys@curveto\{6\.02023pt\}\{3\.43907pt\}\{6\.13217pt\}\{3\.55101pt\}\{6\.13217pt\}\{3\.68907pt\}\\pgfsys@lineto\{6\.13217pt\}\{4\.04155pt\}\\pgfsys@curveto\{6\.13217pt\}\{4\.17961pt\}\{6\.02023pt\}\{4\.29155pt\}\{5\.88217pt\}\{4\.29155pt\}\\pgfsys@closepath\\pgfsys@moveto\{5\.2797pt\}\{3\.43907pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{5\.70593pt\}\{3\.86531pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.825,0\.3,0\.475\}\\pgfsys@color@rgb@fill\{0\.825\}\{0\.3\}\{0\.475\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0\.825,0\.3,0\.475\}\\pgfsys@color@rgb@fill\{0\.825\}\{0\.3\}\{0\.475\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{1\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{4\.29921pt\}\{2\.45093pt\}\\pgfsys@lineto\{3\.94673pt\}\{2\.45093pt\}\\pgfsys@curveto\{3\.80867pt\}\{2\.45093pt\}\{3\.69673pt\}\{2\.33899pt\}\{3\.69673pt\}\{2\.20093pt\}\\pgfsys@lineto\{3\.69673pt\}\{1\.84845pt\}\\pgfsys@curveto\{3\.69673pt\}\{1\.71039pt\}\{3\.80867pt\}\{1\.59845pt\}\{3\.94673pt\}\{1\.59845pt\}\\pgfsys@lineto\{4\.29921pt\}\{1\.59845pt\}\\pgfsys@curveto\{4\.43727pt\}\{1\.59845pt\}\{4\.54921pt\}\{1\.71039pt\}\{4\.54921pt\}\{1\.84845pt\}\\pgfsys@lineto\{4\.54921pt\}\{2\.20093pt\}\\pgfsys@curveto\{4\.54921pt\}\{2\.33899pt\}\{4\.43727pt\}\{2\.45093pt\}\{4\.29921pt\}\{2\.45093pt\}\\pgfsys@closepath\\pgfsys@moveto\{3\.69673pt\}\{1\.59845pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{4\.12297pt\}\{2\.02469pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0,0\.6,0\}\\pgfsys@color@rgb@fill\{0\}\{0\.6\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{ \{\}\{\}\}\}\{ \{\}\{\}\} \{\{\}\{\{\}\}\}\{\{\}\{\}\}\{\}\{\{\}\{\}\}\{\}\{\}\{\}\{\}\{\} \{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0,0\.6,0\}\\pgfsys@color@rgb@fill\{0\}\{0\.6\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@fill@opacity\{0\}\\pgfsys@invoke\{ \}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\{\{\}\{\}\{\{\}\}\}\{\{\}\{\}\{\{\}\}\}\{\}\{\}\\pgfsys@moveto\{7\.02335pt\}\{1\.64105pt\}\\pgfsys@lineto\{6\.67087pt\}\{1\.64105pt\}\\pgfsys@curveto\{6\.5328pt\}\{1\.64105pt\}\{6\.42087pt\}\{1\.52911pt\}\{6\.42087pt\}\{1\.39105pt\}\\pgfsys@lineto\{6\.42087pt\}\{1\.03857pt\}\\pgfsys@curveto\{6\.42087pt\}\{0\.90051pt\}\{6\.5328pt\}\{0\.78857pt\}\{6\.67087pt\}\{0\.78857pt\}\\pgfsys@lineto\{7\.02335pt\}\{0\.78857pt\}\\pgfsys@curveto\{7\.1614pt\}\{0\.78857pt\}\{7\.27335pt\}\{0\.90051pt\}\{7\.27335pt\}\{1\.03857pt\}\\pgfsys@lineto\{7\.27335pt\}\{1\.39105pt\}\\pgfsys@curveto\{7\.27335pt\}\{1\.52911pt\}\{7\.1614pt\}\{1\.64105pt\}\{7\.02335pt\}\{1\.64105pt\}\\pgfsys@closepath\\pgfsys@moveto\{6\.42087pt\}\{0\.78857pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\{\{\{\{\}\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\pgfsys@transformcm\{1\.0\}\{0\.0\}\{0\.0\}\{1\.0\}\{6\.8471pt\}\{1\.21481pt\}\\pgfsys@invoke\{ \}\\hbox\{\{\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\hbox\{\{\\hbox\{\{\}\}\}\} \}\}\\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\}\}\\right\)\\right\\\|\_\{L\_\{2\}\(\\nu\)\}\\;\\asymp\\;\\left\\\|\\mathord\{\\vbox\{\\hbox\{ \\hbox to7\.97pt\{\\vbox to7\.97pt\{\\pgfpicture\\makeatletter\\hbox\{\\thinspace\\lower\-0\.0297pt\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\definecolor\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@rgb@stroke\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{0\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\nullfont\\hbox to0\.0pt\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\}\{\}\{\{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\}\}\{\}\{\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \{\}\{\{\}\}\\hbox\{\\hbox\{\{\\pgfsys@beginscope\\pgfsys@invoke\{ \}\{\{\}\{\{\}\}\{\}\{\}\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\}\}\} \\par\{\{\}\}\{\}\{\{\}\}\{\}\{\{\{\}\} \{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\} \}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\pgfsys@color@rgb@stroke\{1\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{1\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{1,0,0\}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\pgfsys@setdash\{\\pgf@temp\}\{\\the\\pgf@x\}\\pgfsys@invoke\{ \}\\pgfsys@roundcap\\pgfsys@invoke\{ \}\{\}\\pgfsys@moveto\{1\.6953pt\}\{1\.6953pt\}\\pgfsys@moveto\{3\.1953pt\}\{1\.6953pt\}\\pgfsys@curveto\{3\.1953pt\}\{2\.52373pt\}\{2\.52373pt\}\{3\.1953pt\}\{1\.6953pt\}\{3\.1953pt\}\\pgfsys@curveto\{0\.86687pt\}\{3\.1953pt\}\{0\.1953pt\}\{2\.52373pt\}\{0\.1953pt\}\{1\.6953pt\}\\pgfsys@curveto\{0\.1953pt\}\{0\.86687pt\}\{0\.86687pt\}\{0\.1953pt\}\{1\.6953pt\}\{0\.1953pt\}\\pgfsys@curveto\{2\.52373pt\}\{0\.1953pt\}\{3\.1953pt\}\{0\.86687pt\}\{3\.1953pt\}\{1\.6953pt\}\\pgfsys@closepath\\pgfsys@moveto\{1\.6953pt\}\{1\.6953pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\\par\{\{\}\}\{\}\{\{\}\}\{\}\{\}\{\{\}\}\{\}\{\{\}\}\{\}\{\{\}\}\{\} \{\}\{\}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\color\[rgb\]\{1,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{1,0,0\}\\pgfsys@color@rgb@stroke\{1\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@rgb@fill\{1\}\{0\}\{0\}\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{1,0,0\}\\pgfsys@setlinewidth\{\\the\\pgflinewidth\}\\pgfsys@invoke\{ \}\\pgfsys@setdash\{\\pgf@temp\}\{\\the\\pgf@x\}\\pgfsys@invoke\{ \}\\pgfsys@roundcap\\pgfsys@invoke\{ \}\{\}\\pgfsys@moveto\{2\.7396pt\}\{2\.7396pt\}\\pgfsys@lineto\{5\.49277pt\}\{5\.49277pt\}\\pgfsys@stroke\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\\par\{\{\}\}\{\}\{\{\}\}\{\}\{\{\{\}\} \{\}\{\}\{\}\{\}\{\}\{\}\{\}\{\} \}\\pgfsys@beginscope\\pgfsys@invoke\{ \}\\color\[rgb\]\{0,0,0\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0,0,0\}\\pgfsys@color@gray@stroke\{0\}\\pgfsys@invoke\{ \}\\pgfsys@color@gray@fill\{0\}\\pgfsys@invoke\{ \}\\definecolor\[named\]\{pgffillcolor\}\{rgb\}\{0,0,0\}\\pgfsys@moveto\{6\.44212pt\}\{6\.44212pt\}\\pgfsys@moveto\{7\.94212pt\}\{6\.44212pt\}\\pgfsys@curveto\{7\.94212pt\}\{7\.27055pt\}\{7\.27055pt\}\{7\.94212pt\}\{6\.44212pt\}\{7\.94212pt\}\\pgfsys@curveto\{5\.6137pt\}\{7\.94212pt\}\{4\.94212pt\}\{7\.27055pt\}\{4\.94212pt\}\{6\.44212pt\}\\pgfsys@curveto\{4\.94212pt\}\{5\.6137pt\}\{5\.6137pt\}\{4\.94212pt\}\{6\.44212pt\}\{4\.94212pt\}\\pgfsys@curveto\{7\.27055pt\}\{4\.94212pt\}\{7\.94212pt\}\{5\.6137pt\}\{7\.94212pt\}\{6\.44212pt\}\\pgfsys@closepath\\pgfsys@moveto\{6\.44212pt\}\{6\.44212pt\}\\pgfsys@fill\\pgfsys@invoke\{ \} \\pgfsys@invoke\{ \}\\pgfsys@endscope\} \\pgfsys@invoke\{ \}\\pgfsys@endscope\{\}\{\}\{\}\\hss\}\\pgfsys@discardpath\\pgfsys@invoke\{ \}\\pgfsys@endscope\\hss\}\}\\endpgfpicture\}\}\}\}\}\\right\\\|\_\{\\mathcal\{H\}\_\{\\nu\}\}Figure 1:Illustration of our approach\.Left:We view the token sequence as a measurePPover key–value pairs\(k,v\)∈𝒳\(k,v\)\\in\\mathcal\{X\}, which we map to their response profilesΓP​\(k,v\)∈ℋν\\Gamma\_\{P\}\(k,v\)\\in\\mathcal\{H\}\_\{\\nu\}\.Right:CompressingPPamounts to a sparse reweighting of tokens, which moves the barycenter of the response profiles\. The size of this displacement depends on the retained tokens and their assigned weights, and controls the attention error incurred by compression\. KV compression therefore reduces to a balancing problem in Hilbert space: find aKK\-atomic reweighting that keeps the barycenter inℋν\\mathcal\{H\}\_\{\\nu\}nearly fixed\.Problem statement II \(Sparse balancing in Hilbert space\)GivenK≥1K\\geq 1, a Hilbert spaceℋ\\mathcal\{H\}, andμ∈𝒫fin​\(ℋ\)\\mu\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{H\}\), findμ^∈𝒫K​\(supp⁡\(μ\)\)\\hat\{\\mu\}\\in\\mathcal\{P\}\_\{K\}\(\\operatorname\{supp\}\(\\mu\)\)minimizingℰ~μ​\(μ^\)≔‖∫ℋg​d​μ^​\(g\)‖ℋ2\.\\textstyle\\widetilde\{\\mathcal\{E\}\}\_\{\\mu\}\(\\hat\{\\mu\}\)\\coloneqq\\bigl\\\|\\int\_\{\\mathcal\{H\}\}g\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{\\mu\}\(g\)\\bigr\\\|\_\{\\mathcal\{H\}\}^\{2\}\.

The rest of the section turns this reformulation into tight minimax bounds\.[Section˜4\.1](https://arxiv.org/html/2607.01520#S4.SS1)proves the transfer back to KV compression: aKK\-atomic reweighting whose squared barycenter displacement is at mostε\\varepsilonincurs attention error at mostC​εC\\varepsilon, for a universal constantC\>0C\>0\.[Section˜4\.2](https://arxiv.org/html/2607.01520#S4.SS2)then proves two balancing bounds for[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)and transfers them to the query\-aware and query\-agnostic minimax compression risk\. Finally,[Section˜4\.3](https://arxiv.org/html/2607.01520#S4.SS3)proves that these bounds are tight for sufficiently rich classes of contexts and query distributions\. For that, we work in the regime where softmax attention can sharply retrieve individual tokens, formalized by the following assumption\.

\{restatable\}

assumptionasprichness FixK≥1K\\geq 1and let𝖡d​\(r\)\\mathsf\{B\}\_\{d\}\(r\)denote the radius\-rrball inℝd\\mathbb\{R\}^\{d\}\. Assume that\(i\)𝒦∩𝒬⊇𝖡dk​\(8​dk1/4​log⁡K\)\\mathcal\{K\}\\cap\\mathcal\{Q\}\\supseteq\\mathsf\{B\}\_\{d\_\{k\}\}\\bigl\(8\\,d\_\{k\}^\{1/4\}\\sqrt\{\\log K\}\\bigr\)\(ii\)dk,dv≥10​log⁡Kd\_\{k\},d\_\{v\}\\geq 10\\log K\(iii\)𝒱=𝖡dv​\(V\)\\mathcal\{V\}=\\mathsf\{B\}\_\{d\_\{v\}\}\(V\)\.

Condition[\(i\)](https://arxiv.org/html/2607.01520#S4.I1.i1)gives the minimal norm growth needed for sharp attention amongKKtokens: resolving one token againstKKcompetitors requires logit gaps of orderlog⁡K\\log K, while radius\-BBkeys and queries give gaps at most2​B2/dk2B^\{2\}/\\sqrt\{d\_\{k\}\}, forcingB≳dk1/4​log⁡KB\\gtrsim d\_\{k\}^\{1/4\}\\sqrt\{\\log K\}\. Condition[\(ii\)](https://arxiv.org/html/2607.01520#S4.I1.i2)allows mild dimension growth with context length, and Condition[\(iii\)](https://arxiv.org/html/2607.01520#S4.I1.i3)ensures that the value norm boundVVis at the right scale\.

##### Notation\.

For a Hilbert spaceℋ\\mathcal\{H\}and a positive trace\-class operatorLLonℋ\\mathcal\{H\}, with eigenvaluesλ1​\(L\)≥λ2​\(L\)≥⋯≥0\\lambda\_\{1\}\(L\)\\geq\\lambda\_\{2\}\(L\)\\geq\\cdots\\geq 0, multiplicities counted, settailr⁡\(L\)≔∑j\>rλj​\(L\)\\operatorname\{tail\}\_\{r\}\(L\)\\coloneqq\\sum\_\{j\>r\}\\lambda\_\{j\}\(L\)\. Forx,y≥0x,y\\geq 0, writex≲yx\\lesssim yifx≤C​yx\\leq Cyandx≳yx\\gtrsim yifx≥c​yx\\geq cyfor universalC,c\>0C,c\>0, and writex≍yx\\asymp yif both hold\.

All proofs and constructions for this section are in[Appendices˜A](https://arxiv.org/html/2607.01520#A1)and[B](https://arxiv.org/html/2607.01520#A2)\.

### 4\.1KV compression reduces to sparse balancing in Hilbert space

Recall from[Section˜3](https://arxiv.org/html/2607.01520#S3)that, for a fixed context measurePPand query distributionν\\nu, each token affects attention outputs under context reweightings through its response profileΓP​\(k,v\)∈ℋν\\Gamma\_\{P\}\(k,v\)\\in\\mathcal\{H\}\_\{\\nu\}\. Having formalized the context as a measure, we can form the pushforwardμP,ν≔P∘ΓP−1,\\mu\_\{P,\\nu\}\\coloneqq P\\circ\\Gamma\_\{P\}^\{\-1\},which then describes the distribution of responses to the attention numerator and denominator under reweightings ofPP\. Moreover, everyKK\-atomic measureμ^\\hat\{\\mu\}approximatingμP,ν\\mu\_\{P,\\nu\}while remaining supported onsupp⁡\(μP,ν\)\\operatorname\{supp\}\(\\mu\_\{P,\\nu\}\)can be pulled back to aKK\-atomic context measureP^\\hat\{P\}\. Consequently, any solution of Problem[1](https://arxiv.org/html/2607.01520#S4.F1)for the measureμP,ν\\mu\_\{P,\\nu\}inℋν\\mathcal\{H\}\_\{\\nu\}directly yields a compressed KV cache\. The following proposition shows that error bounds survive this pullback, so that upper bounds for sparse balancing in Hilbert space immediately transfer to bounds on the KV compression error\.

\{restatable\}

propositionpropreduction LetK≥1K\\geq 1,P∈𝒫fin​\(𝒳\)P\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\), andν∈𝒫​\(𝒬\)\\nu\\in\\mathcal\{P\}\(\\mathcal\{Q\}\)\. DefineμP,ν≔P∘ΓP−1\\mu\_\{P,\\nu\}\\coloneqq P\\circ\\Gamma\_\{P\}^\{\-1\}\. Then,μP,ν\\mu\_\{P,\\nu\}is centered and has covariance operatorΣP,ν\\Sigma\_\{P,\\nu\}\. Moreover, any fixedμ^∈𝒫K​\(supp⁡\(μP,ν\)\)\\hat\{\\mu\}\\in\\mathcal\{P\}\_\{K\}\(\\operatorname\{supp\}\(\\mu\_\{P,\\nu\}\)\)can be pulled back to a measureP^∈𝒫K​\(𝒳\)\\hat\{P\}\\in\\mathcal\{P\}\_\{K\}\(\\mathcal\{X\}\)independently ofν\\nusuch thatℰP,ν​\(P^\)≲ℰ~μP,ν​\(μ^\)\\mathcal\{E\}\_\{P,\\nu\}\\bigl\(\\hat\{P\}\\bigr\)\\lesssim\\widetilde\{\\mathcal\{E\}\}\_\{\\mu\_\{P,\\nu\}\}\(\\hat\{\\mu\}\)\. In particular,

infP^∈𝒫K​\(𝒳\)ℰP,ν​\(P^\)≲infμ^∈𝒫K​\(supp⁡\(μP,ν\)\)ℰ~μP,ν​\(μ^\)\.\\inf\_\{\\hat\{P\}\\in\\mathcal\{P\}\_\{K\}\(\\mathcal\{X\}\)\}\\mathcal\{E\}\_\{P,\\nu\}\\bigl\(\\hat\{P\}\\bigr\)\\;\\;\\lesssim\\;\\;\\inf\_\{\\hat\{\\mu\}\\in\\mathcal\{P\}\_\{K\}\(\\operatorname\{supp\}\(\\mu\_\{P,\\nu\}\)\)\}\\widetilde\{\\mathcal\{E\}\}\_\{\\mu\_\{P,\\nu\}\}\\bigl\(\\hat\{\\mu\}\\bigr\)\.

### 4\.2Compression bounds from balancing bounds

By[Section˜4\.1](https://arxiv.org/html/2607.01520#S4.SS1), it suffices to bound the sparse balancing error in Hilbert space: prove a bound for balancing the barycenter of a probability measure in Hilbert space, and have the same bound apply pointwise to KV compression\. Standard results on sparse convex optimization already provide basic balancing bounds\. First, kernel herding and conditional\-gradient methods give a bound of orderB2/KB^\{2\}/Kwhensupp⁡\(μP,ν\)\\operatorname\{supp\}\(\\mu\_\{P,\\nu\}\)lies in a radius\-BBball\[[12](https://arxiv.org/html/2607.01520#bib.bib12),[17](https://arxiv.org/html/2607.01520#bib.bib17)\]\. This can be loose, however, because the radius is sensitive to worst\-case response profiles and does not measure data\-dependent redundancy\. Second, independent sampling fromPPgives a bound of the formtr⁡\(ΣP,ν\)/K\\operatorname\{tr\}\(\\Sigma\_\{P,\\nu\}\)/K\. This captures the total response variance but does not exploit potential anisotropy ofΣP,ν\\Sigma\_\{P,\\nu\}, which can be significant whenν\\nuconcentrates on a few important directions\.

#### 4\.2\.1Query\-aware upper bound

Since query\-aware compressors can explicitly adapt to anisotropic structure in the query distribution, we seek a bound in terms of the spectral decay ofΣP,ν\\Sigma\_\{P,\\nu\}, which kernel herding and random sampling do not provide\. Below, we provide such a bound, which is new to the best of our knowledge\. The construction works by preserving the barycenter exactly on the leading eigenspace while simultaneously sparsifying the measure on the orthogonal complement\.[Section˜4\.1](https://arxiv.org/html/2607.01520#S4.SS1)allows us to immediately transfer this bound to the minimax risk of query\-aware KV compression\.\{restatable\}theoremthmsparsehilbertapproximationfixedk FixK≥7K\\geq 7and a real Hilbert spaceℋ\\mathcal\{H\}\. Letμ∈𝒫fin​\(ℋ\)\\mu\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{H\}\)be centered, with covariance operatorΣ:ℋ→ℋ\\Sigma:\\mathcal\{H\}\\to\\mathcal\{H\}\. Then,‖∫ℋg​d​μ^​\(g\)‖ℋ2≲tailK/3⁡\(Σ\)K\\\|\\int\_\{\\mathcal\{H\}\}g\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{\\mu\}\(g\)\\\|\_\{\\mathcal\{H\}\}^\{2\}\\lesssim\\frac\{\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\)\}\{K\}for someμ^∈𝒫K​\(supp⁡\(μ\)\)\\hat\{\\mu\}\\in\\mathcal\{P\}\_\{K\}\(\\operatorname\{supp\}\(\\mu\)\)\.\{restatable\}corollarycorupperaware LetK≥7K\\geq 7andℱ⊆𝒫fin​\(𝒳\)×𝒫​\(𝒬\)\\mathcal\{F\}\\subseteq\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)\\times\\mathcal\{P\}\(\\mathcal\{Q\}\)\. Then,ℜ​\(𝒜Kaw,ℱ\)≲supP,ν∈ℱtailK/3⁡\(ΣP,ν\)K\\mathfrak\{R\}\\\!\\left\(\\mathcal\{A\}\_\{K\}^\{\\mathrm\{aw\}\},\\mathcal\{F\}\\right\)\\lesssim\\sup\_\{P,\\nu\\in\\mathcal\{F\}\}\\frac\{\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\_\{P,\\nu\}\)\}\{K\}\.

Thus, whenν\\nuis known, the compression budget can be spent on preserving the leading eigenspace of the response covariance, and the error is controlled only by the response variance outside that space\.

#### 4\.2\.2Query\-agnostic upper bound

The above approach relies on knowledge of the geometry ofΣP,ν\\Sigma\_\{P,\\nu\}\. In contrast, a query\-agnostic strategy has no access to this geometry, since it depends onν\\nu\. A simple balancing strategy that does not rely on knowledge of the Hilbert space geometry is to sampleKKatoms independently fromPPand take their empirical measure, which corresponds to averagingKKindependent centered draws fromμP,ν\\mu\_\{P,\\nu\}and yields the following, standardtr⁡\(ΣP,ν\)/K\\operatorname\{tr\}\(\\Sigma\_\{P,\\nu\}\)/Kbound for the query\-agnostic minimax risk via[Section˜4\.1](https://arxiv.org/html/2607.01520#S4.SS1)\.

\{restatable\}

theoremthmupperagnostic LetK≥1K\\geq 1andℱ⊆𝒫fin​\(𝒳\)×𝒫​\(𝒬\)\\mathcal\{F\}\\subseteq\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)\\times\\mathcal\{P\}\(\\mathcal\{Q\}\)\. Then,ℜ​\(𝒜Kag,ℱ\)≲supP,ν∈ℱtr⁡\(ΣP,ν\)/K\\mathfrak\{R\}\\\!\\left\(\\mathcal\{A\}\_\{K\}^\{\\mathrm\{ag\}\},\\mathcal\{F\}\\right\)\\lesssim\\sup\_\{P,\\nu\\in\\mathcal\{F\}\}\\operatorname\{tr\}\(\\Sigma\_\{P,\\nu\}\)/K\.

### 4\.3Compressors cannot beat the spectral barrier

We now show that the upper bounds from[Section˜4\.2\.1](https://arxiv.org/html/2607.01520#S4.SS2.SSS1)and[Section˜4\.2\.2](https://arxiv.org/html/2607.01520#S4.SS2.SSS2)are tight when attention can be sharp\. In this regime, it is known that accurate KV compression, and fast approximate attention more broadly, is impossible*in the worst case*\[[16](https://arxiv.org/html/2607.01520#bib.bib16),[3](https://arxiv.org/html/2607.01520#bib.bib3)\]\. We generalize these results by showing that sharpness alone does not determine whether a cache is compressible\. Instead, the minimax compression risk is governed by the spectrum ofΣP,ν\\Sigma\_\{P,\\nu\}throughout a nontrivial range, from essentially loss\-free compression to worst possible error scale under bounded values\.

Concretely, we construct admissible contexts, together with query distributions, on which the error for any compression is calibrated totailK/3⁡\(ΣP,ν\)\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\_\{P,\\nu\}\)andtr⁡\(ΣP,ν\)\\operatorname\{tr\}\(\\Sigma\_\{P,\\nu\}\), respectively\. Importantly, since an arbitrary finitely supported measure on𝒳\\mathcal\{X\}need not be induced by actual KV caches, our lower\-bound instances are chosen from the subclass𝒫K⋆​\(𝒳\)⊆𝒫K​\(𝒳\)\\mathcal\{P\}\_\{K\}^\{\\star\}\(\\mathcal\{X\}\)\\subseteq\\mathcal\{P\}\_\{K\}\(\\mathcal\{X\}\)of context measures induced by KV caches\(ki,vi\)i=1K\(k\_\{i\},v\_\{i\}\)\_\{i=1\}^\{K\}\. Moreover, the constructed contexts have length2​K2K, so that our lower bounds already apply when the original context is only a constant factor larger than the compression budget and do not rely on compressing extremely long contexts into tiny summaries\.

#### 4\.3\.1Query\-aware lower bound

When the compressor knowsν\\nu, we show that, at every scaleR∈\[0,V2​K\]R\\in\[0,V^\{2\}K\], there exist instances with covariancetailK/3⁡\(ΣP,ν\)≍R\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\_\{P,\\nu\}\)\\asymp R, on which everyKK\-atomic compressor incurs error at scaleR/KR/K\. Thus, the ratetailK/3⁡\(ΣP,ν\)/K\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\_\{P,\\nu\}\)/Kgoverns the query\-aware minimax risk throughout the full natural range, from exact compressibility at zero error to the maximum possible scaleV2V^\{2\}\.

\{restatable\}

theoremthmlowerboundaware LetK≥50K\\geq 50andR∈\[0,V2​K\]R\\in\[0,V^\{2\}K\]\. Suppose[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)holds\. Then, there existsℱ⊆𝒫2​K⋆​\(𝒳\)×𝒫​\(𝒬\)\\mathcal\{F\}\\subseteq\\mathcal\{P\}\_\{2K\}^\{\\star\}\(\\mathcal\{X\}\)\\times\\mathcal\{P\}\(\\mathcal\{Q\}\)withtailK/3⁡\(ΣP,ν\)≍R\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\_\{P,\\nu\}\)\\asymp Rfor all\(P,ν\)∈ℱ\(P,\\nu\)\\in\\mathcal\{F\}andℜ​\(𝒜Kaw,ℱ\)≳RK\\mathfrak\{R\}\\\!\\left\(\\mathcal\{A\}\_\{K\}^\{\\mathrm\{aw\}\},\\mathcal\{F\}\\right\)\\gtrsim\\frac\{R\}\{K\}\.

#### 4\.3\.2Query\-agnostic lower bound

When the compressor does not have access toν\\nu, it must choose a single summary to protect against many query distributions simultaneously\. In this harder setting, we obtain a sharper lower bound\. Specifically, at every scaleT∈\[V2,V2​K\]T\\in\[V^\{2\},V^\{2\}K\], there exist instances with covariancetr⁡\(ΣP,ν\)≍T\\operatorname\{tr\}\(\\Sigma\_\{P,\\nu\}\)\\asymp T, on which everyKK\-atomic compressor incurs error at scaleT/KT/Kon some query distribution\. At the same time, for each individual query distribution, perfect compression of the same contexts is possible, witnessing a genuine separation of the two regimes\. The price of hiding the query distribution is therefore the loss of access to anisotropic geometry inΣP,ν\\Sigma\_\{P,\\nu\}\. Query\-aware compressors can preserve the dominant directions and pay only for the tail, whereas query\-agnostic compressors must in general pay for the full trace, which can be significantly worse\.

\{restatable\}

theoremthmlowerboundagnostic LetK≥50K\\geq 50andT∈\[V2,V2​K\]T\\in\[V^\{2\},V^\{2\}K\]\. Suppose[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)holds\. Then, there existsℱ⊆𝒫2​K⋆​\(𝒳\)×𝒫​\(𝒬\)\\mathcal\{F\}\\subseteq\\mathcal\{P\}\_\{2K\}^\{\\star\}\(\\mathcal\{X\}\)\\times\\mathcal\{P\}\(\\mathcal\{Q\}\)withtr⁡\(ΣP,ν\)≍T\\operatorname\{tr\}\(\\Sigma\_\{P,\\nu\}\)\\asymp Tfor all\(P,ν\)∈ℱ\(P,\\nu\)\\in\\mathcal\{F\}andℜ​\(𝒜Kag,ℱ\)≳TK,\\mathfrak\{R\}\\\!\\left\(\\mathcal\{A\}\_\{K\}^\{\\mathrm\{ag\}\},\\mathcal\{F\}\\right\)\\gtrsim\\frac\{T\}\{K\},whileℜ​\(𝒜Kaw,ℱ\)=0\\mathfrak\{R\}\\\!\\left\(\\mathcal\{A\}\_\{K\}^\{\\mathrm\{aw\}\},\\mathcal\{F\}\\right\)=0\.

## 5Efficient KV compression algorithms with minimax\-optimal risk

We now study how the abstract minimax rates from the previous section can be attained algorithmically during practical Transformer inference, focusing on the causally masked setting of autoregressive sequence models\[[23](https://arxiv.org/html/2607.01520#bib.bib23),[13](https://arxiv.org/html/2607.01520#bib.bib13),[8](https://arxiv.org/html/2607.01520#bib.bib8)\]\. During*prefill*, the prompt is processed in parallel subject to causal attention\. During*autoregressive decoding*, tokens arrive sequentially and attend to the full past\. Our goal is to make efficient and minimax\-optimal KV compression compatible with both regimes\.

Since the future query distribution is typically unavailable at compression time, we work in the query\-agnostic setting\. Here, naive random sampling already achieves the minimax\-optimal risk\. Our objective is therefore to achieve this risk as a baseline, while improving on it under additional structure\. Recall from[Section˜4](https://arxiv.org/html/2607.01520#S4)that the query\-aware and query\-agnostic minimax risks differ because the geometry of the response profile spaceℋν\\mathcal\{H\}\_\{\\nu\}depends onν\\nu\. A query\-aware compressor can adapt to this geometry, whereas a query\-agnostic compressor must choose its summary beforeν\\nuis known and hence hedge against all admissible geometries\. We therefore restrict admissible query distributions to those whose response geometry is uniformly comparable to a data\-independent reference geometry\.

\{restatable\}

assumptionaspcommongeometry FixK≥1K\\geq 1, a Hilbert spaceℋ\\mathcal\{H\}, and a bounded feature mapΦ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}\. Letν∈𝒫​\(𝒬\)\\nu\\in\\mathcal\{P\}\(\\mathcal\{Q\}\)andP∈𝒫fin​\(𝒳\)P\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)\. Assume that,

‖∫𝒳Φ​d​σ‖ℋ2≲‖∫𝒳ΓP​d​σ‖ℋν2≲‖∫𝒳Φ​d​σ‖ℋ2\\textstyle\\bigl\\\|\\int\_\{\\mathcal\{X\}\}\\Phi\\mathop\{\}\\\!\\mathrm\{d\}\\sigma\\bigr\\\|\_\{\\mathcal\{H\}\}^\{2\}\\;\\lesssim\\;\\bigl\\\|\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\}\\mathop\{\}\\\!\\mathrm\{d\}\\sigma\\bigr\\\|\_\{\\mathcal\{H\}\_\{\\nu\}\}^\{2\}\\;\\lesssim\\;\\bigl\\\|\\int\_\{\\mathcal\{X\}\}\\Phi\\mathop\{\}\\\!\\mathrm\{d\}\\sigma\\bigr\\\|\_\{\\mathcal\{H\}\}^\{2\}for all signed measuresσ\\sigmasupported onsupp⁡\(P\)\\operatorname\{supp\}\(P\)withσ​\(𝒳\)=0\\sigma\(\\mathcal\{X\}\)=0\.

##### Desiderata\.

We seek algorithms that\(i\)reduce the cost of attention during prefill and autoregressive decoding\(ii\)reduce cache memory during autoregressive decoding\(iii\)respect causal masking\(iv\)attain the minimax\-optimal query\-agnostic risk\(v\)attain the query\-aware minimax\-optimal rate under[Section˜5](https://arxiv.org/html/2607.01520#S5)\.

##### Notation\.

We work with a stream\(ki,vi\)i≥1\(k\_\{i\},v\_\{i\}\)\_\{i\\geq 1\}of key–value pairs\. The firstnnpairs form the prompt; subsequent pairs arrive sequentially during decoding\. Fix a chunk sizeKK, and assumen=M​Kn=MKwithMMa power of two\. For any index setI⊆ℕI\\subseteq\\mathbb\{N\}, letPIP\_\{I\}denote the context measure on the indexed tokens, and define its unnormalized massw​\(I\)≔∑i∈Iexp⁡\(‖ki‖2/\(2​dk\)\)w\(I\)\\coloneqq\\sum\_\{i\\in I\}\\exp\(\\\|k\_\{i\}\\\|^\{2\}/\(2\\sqrt\{d\_\{k\}\}\)\)\. IfP,Q∈𝒫fin​\(𝒳\)P,Q\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)represent disjoint index setsIIandJJ, we write

P⊕Q≔w​\(I\)w​\(I\)\+w​\(J\)​P\+w​\(J\)w​\(I\)\+w​\(J\)​Q\\textstyle P\\oplus Q\\coloneqq\\frac\{w\(I\)\}\{w\(I\)\+w\(J\)\}P\+\\frac\{w\(J\)\}\{w\(I\)\+w\(J\)\}Qfor their union, suppressing the dependence onIIandJJwhen clear from context\.

All proofs and constructions for this section are in[Appendix˜C](https://arxiv.org/html/2607.01520#A3)\.

Algorithm 1Parallel prefix compression for causally masked prefill1:

M←n/KM\\leftarrow n/K,

L←log2⁡ML\\leftarrow\\log\_\{2\}M,

SL1←∅S\_\{L\}^\{1\}\\leftarrow\\emptyset
2:

Q00,…,QM−10←C1,…,CMQ\_\{0\}^\{0\},\.\.\.,Q\_\{M\-1\}^\{0\}\\leftarrow C\_\{1\},\.\.\.,C\_\{M\}⊳\\trianglerightpartition prompt into chunks of sizeKK

3:for

ℓ=1,…,L\\ell=1,\\ldots,Ldo⊳\\trianglerightupsweep stage

4:for all

j=1,…,M/2ℓj=1,\\ldots,M/2^\{\\ell\}in paralleldo

5:

Qjℓ←𝖱𝖤𝖣𝖴𝖢𝖤⁡\(Q2​j−1ℓ−1⊕Q2​jℓ−1\)Q\_\{j\}^\{\\ell\}\\leftarrow\\operatorname\{\\mathsf\{REDUCE\}\}\(Q\_\{2j\-1\}^\{\\ell\-1\}\\oplus Q\_\{2j\}^\{\\ell\-1\}\)⊳\\trianglerightsummary of tokens\(j−1\)​2ℓ​K\+1\(j\-1\)2^\{\\ell\}K\+1toj​2ℓ​Kj2^\{\\ell\}K

6:for

ℓ=L,L−1,…,1\\ell=L,L\-1,\\ldots,1do⊳\\trianglerightdownsweep stage

7:for all

j=1,…,M/2ℓj=1,\\ldots,M/2^\{\\ell\}in paralleldo

8:

S2​j−1ℓ−1←SjℓS\_\{2j\-1\}^\{\\ell\-1\}\\leftarrow S\_\{j\}^\{\\ell\}⊳\\trianglerightsummary of tokens11to\(2​j−2\)​2ℓ−1​K\(2j\-2\)2^\{\\ell\-1\}K

9:

S2​jℓ−1←𝖱𝖤𝖣𝖴𝖢𝖤⁡\(Sjℓ⊕Q2​j−1ℓ−1\)S\_\{2j\}^\{\\ell\-1\}\\leftarrow\\operatorname\{\\mathsf\{REDUCE\}\}\(S\_\{j\}^\{\\ell\}\\oplus Q\_\{2j\-1\}^\{\\ell\-1\}\)⊳\\trianglerightsummary of tokens11to\(2​j−1\)​2ℓ−1​K\(2j\-1\)2^\{\\ell\-1\}K

10:return

\(Sc←Sc0\)c=1M\(S\_\{c\}\\leftarrow S\_\{c\}^\{0\}\)\_\{c=1\}^\{M\}and

Q1LQ\_\{1\}^\{L\}⊳\\trianglerightexclusive prefix summaries and full\-prompt summary

Algorithm 2Streaming prefix compression for autoregressive decoding1:

ℬ0,ℬ1,…←∅\\mathcal\{B\}\_\{0\},\\mathcal\{B\}\_\{1\},\\ldots\\leftarrow\\emptyset,

ℬL←Q1L\\mathcal\{B\}\_\{L\}\\leftarrow Q\_\{1\}^\{L\},

ℓ←0\\ell\\leftarrow 0⊳\\trianglerightQ1LQ\_\{1\}^\{L\}is the summary produced by[Algorithm˜1](https://arxiv.org/html/2607.01520#alg1)

2:for

t=0,K,2​K,…t=0,K,2K,\\ldotsdo

3:

Q←P\{n\+t\+1,…,n\+t\+K\}Q\\leftarrow P\_\{\\\{n\+t\+1,\.\.\.,n\+t\+K\\\}\}
4:while

ℬℓ≠∅\\mathcal\{B\}\_\{\\ell\}\\neq\\emptysetdo⊳\\trianglerightupdate buckets

5:

Q←𝖱𝖤𝖣𝖴𝖢𝖤⁡\(ℬℓ⊕Q\)Q\\leftarrow\\operatorname\{\\mathsf\{REDUCE\}\}\(\\mathcal\{B\}\_\{\\ell\}\\oplus Q\),

ℬℓ←∅\\mathcal\{B\}\_\{\\ell\}\\leftarrow\\emptyset,

ℓ←ℓ\+1\\ell\\leftarrow\\ell\+1
6:

ℓ←0\\ell\\leftarrow 0,

ℬℓ←Q\\mathcal\{B\}\_\{\\ell\}\\leftarrow Q,

H←∅H\\leftarrow\\emptyset
7:for

ℓ\\ellin decreasing orderdo⊳\\trianglerightcompress buckets to sizeKK

8:

H←𝖱𝖤𝖣𝖴𝖢𝖤⁡\(H⊕ℬℓ\)H\\leftarrow\\operatorname\{\\mathsf\{REDUCE\}\}\(H\\oplus\\mathcal\{B\}\_\{\\ell\}\)
9:yield

HH

### 5\.1Efficient prefill and autoregressive decoding under causal masking

In causally masked prefill, tokenttmay attend only to the prefixPt≔P\{1,…,t\}P\_\{t\}\\coloneqq P\_\{\\\{1,\\dots,t\\\}\}, so compression must provide*prefix summaries*for each positiontt\. Since storing a separate summary for every prefix would defeat the purpose of compression, we summarize only at chunk boundaries\. Specifically, we fix a chunk sizeKK, partition the stream into chunksI1,I2,…I\_\{1\},I\_\{2\},\\ldotsof sizeKK, and maintain aKK\-atomic summaryScS\_\{c\}of the prefix for each chunkcc\. At positiont∈Ict\\in I\_\{c\}, attention is then computed against

P^t≔Sc−1⊕P\{min⁡Ic,…,t\},\\hat\{P\}\_\{t\}\\coloneqq S\_\{c\-1\}\\oplus P\_\{\\\{\\min I\_\{c\},\\dots,t\\\}\},\(5\)which keeps the local causal prefix exact and compresses only the long\-range past\. Hence, every attention computation uses at most2​K2Katoms while respecting causal masking\.

The summariesScS\_\{c\}are computed by recursively applying a randomized local reducer𝖱𝖤𝖣𝖴𝖢𝖤:𝒫2​K​\(𝒳\)→𝒫K​\(𝒳\)\\operatorname\{\\mathsf\{REDUCE\}\}:\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\)\\to\\mathcal\{P\}\_\{K\}\(\\mathcal\{X\}\)\.[Algorithm˜1](https://arxiv.org/html/2607.01520#alg1)implements this recursion as a parallel Blelloch scan over chunks, producing oneKK\-atomic summary for each chunk prefix and one summary of the full prompt\.[Algorithm˜2](https://arxiv.org/html/2607.01520#alg2)uses the same merge\-reduce primitive online: starting from the prompt summary, it updates the compressed history during autoregressive decoding and emits a newKK\-atomic summary at each chunk boundary\. Thus, both prefill and autoregressive decoding reduce to a small number of calls to𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}on inputs of size2​K2K\. Consequently, if𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}can be implemented efficiently, the full compression scheme meets our efficiency desiderata, as the following proposition shows\.

\{restatable\}

propositionproruntime[Algorithm˜1](https://arxiv.org/html/2607.01520#alg1)makes𝒪⁡\(n/K\)\\operatorname\{\\mathcal\{O\}\}\(n/K\)calls to𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}, and uses parallel depth𝒪⁡\(log⁡\(n/K\)\)\\operatorname\{\\mathcal\{O\}\}\(\\log\(n/K\)\)and𝒪⁡\(n\)\\operatorname\{\\mathcal\{O\}\}\(n\)memory\. Additionally, at timett,[Algorithm˜2](https://arxiv.org/html/2607.01520#alg2)stores𝒪⁡\(K​log⁡\(t/K\)\)\\operatorname\{\\mathcal\{O\}\}\(K\\,\\log\(t/K\)\)atoms, and makes𝒪⁡\(log⁡\(t/K\)/K\)\\operatorname\{\\mathcal\{O\}\}\(\\log\(t/K\)/K\)calls to𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}amortized per token\.

### 5\.2From local reducers to global risk guarantees

[Algorithms˜1](https://arxiv.org/html/2607.01520#alg1)and[2](https://arxiv.org/html/2607.01520#alg2)lift any local compressor𝖱𝖤𝖣𝖴𝖢𝖤:𝒫2​K​\(𝒳\)→𝒫K​\(𝒳\)\\operatorname\{\\mathsf\{REDUCE\}\}:\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\)\\to\\mathcal\{P\}\_\{K\}\(\\mathcal\{X\}\)to a global compression scheme for causally masked prefill and autoregressive decoding\. It remains to identify conditions on𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}that yield global risk guarantees\. We give such conditions below\.

\{restatable\}

\[Design criteria\]definitiondefdesigncriteria FixK≥1K\\geq 1, a Hilbert spaceℋ\\mathcal\{H\}, andΦ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}\. ForP∈𝒫fin​\(𝒳\)P\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)andQ∈𝒫2​K​\(𝒳\)Q\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\), define

μPΦ≔∫𝒳Φ​\(k,v\)​d​P​\(k,v\)andΣPΦ≔∫𝒳\(Φ​\(k,v\)−μPΦ\)⊗\(Φ​\(k,v\)−μPΦ\)​d​P​\(k,v\)\.\\textstyle\\mu\_\{P\}^\{\\Phi\}\\coloneqq\\int\_\{\\mathcal\{X\}\}\\Phi\(k,v\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\)\\qquad\\text\{and\}\\qquad\\Sigma\_\{P\}^\{\\Phi\}\\coloneqq\\int\_\{\\mathcal\{X\}\}\(\\Phi\(k,v\)\-\\mu\_\{P\}^\{\\Phi\}\)\\otimes\(\\Phi\(k,v\)\-\\mu\_\{P\}^\{\\Phi\}\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\)\.We call𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}\(Φ,r,τ\)\(\\Phi,r,\\tau\)\-admissible forr,τ≥0r,\\tau\\geq 0if, for allQ∈𝒫2​K​\(𝒳\)Q\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\),Q^≔𝖱𝖤𝖣𝖴𝖢𝖤⁡\(Q\)∈𝒫K​\(supp⁡\(Q\)\)\\hat\{Q\}\\coloneqq\\operatorname\{\\mathsf\{REDUCE\}\}\(Q\)\\in\\mathcal\{P\}\_\{K\}\(\\operatorname\{supp\}\(Q\)\)almost surely and the following hold:

1. \(i\)𝔼⁡\[Q^\]=Q\\operatorname\{\\mathbb\{E\}\}\[\\hat\{Q\}\]=Q
2. \(ii\)𝔼⁡\[‖∫𝒳Ψ​d​\(Q^−Q\)‖𝒢2\]≲1K​tr⁡\(ΣQΨ\)\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\\|\\int\_\{\\mathcal\{X\}\}\\Psi\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{Q\}\-Q\)\\\|\_\{\\mathcal\{G\}\}^\{2\}\\bigr\]\\lesssim\\frac\{1\}\{K\}\\operatorname\{tr\}\(\\Sigma\_\{Q\}^\{\\Psi\}\)for all Hilbert spaces𝒢\\mathcal\{G\}and boundedΨ:𝒳→𝒢\\Psi:\\mathcal\{X\}\\to\\mathcal\{G\}
3. \(iii\)𝔼⁡\[‖∫𝒳Φ​d​\(Q^−Q\)‖ℋ2\]≲1K​\(tailr⁡\(ΣQΦ\)\+τ\)\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\\|\\int\_\{\\mathcal\{X\}\}\\Phi\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{Q\}\-Q\)\\\|\_\{\\mathcal\{H\}\}^\{2\}\\bigr\]\\lesssim\\frac\{1\}\{K\}\\bigl\(\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{Q\}^\{\\Phi\}\)\+\\tau\\bigr\)

Such reducers exist\. We give concrete instantiations satisfying the conditions for any boundedΦ\\Phiin[Appendix˜C](https://arxiv.org/html/2607.01520#A3), together with a natural choice ofΦ\\Phiand illustrative examples in which[Section˜5](https://arxiv.org/html/2607.01520#S5)holds for thisΦ\\Phi\. When𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}satisfies these conditions, we recover the minimax\-optimal query\-agnostic trace bound without further assumptions, and obtain the spectral refinement of query\-aware compression under[Section˜5](https://arxiv.org/html/2607.01520#S5)\.

\{restatable\}

theoremthmalgguarantee LetK≥1K\\geq 1andν∈𝒫​\(𝒬\)\\nu\\in\\mathcal\{P\}\(\\mathcal\{Q\}\), and suppose that𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}is\(Φ,r,τ\)\(\\Phi,r,\\tau\)\-admissible\. LetP^t∈𝒫2​K​\(𝒳\)\\hat\{P\}\_\{t\}\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\)be as in \([5](https://arxiv.org/html/2607.01520#S5.E5)\)\. Then,𝔼⁡ℰPt,ν​\(P^t\)≲log⁡\(2\+t/K\)K​tr⁡\(ΣPt,ν\)\\operatorname\{\\mathbb\{E\}\}\\mathcal\{E\}\_\{P\_\{t\},\\nu\}\\bigl\(\\hat\{P\}\_\{t\}\\bigr\)\\lesssim\\frac\{\\log\(2\+t/K\)\}\{K\}\\operatorname\{tr\}\(\\Sigma\_\{P\_\{t\},\\nu\}\)\. If additionally[Section˜5](https://arxiv.org/html/2607.01520#S5)holds forν\\nu,PtP\_\{t\}, andΦ\\Phi, then𝔼⁡ℰPt,ν​\(P^t\)≲log⁡\(2\+t/K\)K​\(tailr⁡\(ΣPt,ν\)\+τ\)\\operatorname\{\\mathbb\{E\}\}\\mathcal\{E\}\_\{P\_\{t\},\\nu\}\\bigl\(\\hat\{P\}\_\{t\}\\bigr\)\\lesssim\\frac\{\\log\(2\+t/K\)\}\{K\}\\,\(\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{P\_\{t\},\\nu\}\)\+\\tau\)\.

## 6Experiments

We conduct a focused long\-context evaluation of the compression scheme from[Section˜5](https://arxiv.org/html/2607.01520#S5)on the “long” subset of LongBench\-v2\[[6](https://arxiv.org/html/2607.01520#bib.bib6)\], a multiple\-choice benchmark with contexts ranging from128​k128\\mathrm\{k\}to2​M2\\mathrm\{M\}words\. We evaluate Qwen3\-32B\[[25](https://arxiv.org/html/2607.01520#bib.bib25)\]\. As a baseline, we use attention over the entire available KV cache, following the official truncation protocol ofBai et al\. \[[6](https://arxiv.org/html/2607.01520#bib.bib6)\]whenever the full context exceeds the model’s context window\. We compare the baseline against our compression scheme from[Section˜5](https://arxiv.org/html/2607.01520#S5)and against ScissorHands\[[20](https://arxiv.org/html/2607.01520#bib.bib20)\], SnapKV\[[19](https://arxiv.org/html/2607.01520#bib.bib19)\], and StreamingLLM\[[24](https://arxiv.org/html/2607.01520#bib.bib24)\]from the literature, each applied independently to each attention head\.

Since ScissorHands and SnapKV do not provide efficient parallel inference methods for compressed prefill, we let the literature baselines \(including StreamingLLM\) use the full KV cache during prefill and only compress during decoding\. In contrast, our compression algorithm is used during both prefill and decoding, so that each head only ever attends to a small subset of the tokens seen by the full\-attention baseline\.

We evaluate our compression scheme for two local reducers\. The first is random sampling from the context measureQQ, which attains the query\-agnostic minimax\-optimal rate\. The second protects the leadingrrdirections ofΣQΦ\\Sigma\_\{Q\}^\{\\Phi\}and clusters tokens in the protected coordinates\. This reducer attains the spectral guarantee of[Section˜5\.2](https://arxiv.org/html/2607.01520#S5.SS2)under[Section˜5](https://arxiv.org/html/2607.01520#S5)\. We elaborate on the design of the clustering reducer and the choice of feature mapΦ\\Phiin[Appendix˜C](https://arxiv.org/html/2607.01520#A3)\.[Appendix˜D](https://arxiv.org/html/2607.01520#A4)has additional implementation details\.

[Table˜1](https://arxiv.org/html/2607.01520#S6.T1)reports accuracy with standard error for a compression rate of approximately95%95\\%relative to full context window\. Random sampling already provides a strong query\-agnostic baseline, consistent with its minimax optimality\. Admissible clustering improves accuracy atr=64r=64, and matches the uncompressed baseline atr=128r=128\. Furthermore, all our local reducers are competitive with the literature baselines\[[20](https://arxiv.org/html/2607.01520#bib.bib20),[19](https://arxiv.org/html/2607.01520#bib.bib19),[24](https://arxiv.org/html/2607.01520#bib.bib24)\]despite using significantly fewer tokens during prefill\. While we emphasize that standard errors overlap and further validation is warranted, these results suggest that the algorithmic design principles developed in[Section˜5](https://arxiv.org/html/2607.01520#S5)hold promise for practical long\-context compression at substantial KV budget reductions, during both decoding and prefill\.

Table 1:Accuracy and standard error on the “long” subset of LongBench\-v2\[[6](https://arxiv.org/html/2607.01520#bib.bib6)\]using Qwen3\-32B\[[25](https://arxiv.org/html/2607.01520#bib.bib25)\]\. “Full KV” refers to standard softmax attention using the full, uncompressed KV cache\. Literature baselines\[[20](https://arxiv.org/html/2607.01520#bib.bib20),[19](https://arxiv.org/html/2607.01520#bib.bib19),[24](https://arxiv.org/html/2607.01520#bib.bib24)\]use the full, uncompressed KV cache during prefill, then compress it once after prefill, and use the compressed cache during decoding\. Our methods compress during both prefill and decoding, using the scheme from[Section˜5](https://arxiv.org/html/2607.01520#S5)for different choices of𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}\.
## 7Conclusion

We posit KV compression as sparse balancing of a finite probability measure in Hilbert space, and characterize its minimax risk in terms of how a cache interacts with future queries through softmax attention\. This yields tight rates under natural assumptions and identifies what compressed summaries must preserve\. The same characterization reveals design principles for causal compression during prefill and autoregressive decoding, separating universal guarantees from structure\-exploiting refinements\. Together, our results provide a principled path toward practical KV cache compression with theoretical guarantees\.

## Acknowledgments and Disclosure of Funding

Funded by the European Union\. This work has received funding from the European High Performance Computing Joint Undertaking \(JU\) and from the German Federal Ministry of Research, Technology and Space \(BMFTR\), the Ministry of Culture and Science of North Rhine\-Westphalia \(MKW NRW\) and the Hessian Ministry of Science and Research, Arts and Culture \(HMWK\) under grant agreement No101250682\. LH was supported by the RWTH Research Ambassador Scholarship during his research stay at Stanford University\. CAA is supported by a Schmidt Science Fellowship\. Computations were performed with computing resources granted by RWTH Aachen University under projectthes2228\.

## References

- Adnan et al\. \[2024\]Muhammad Adnan, Akhil Arunkumar, Gaurav Jain, Prashant J Nair, Ilya Soloveychik, and Purushotham Kamath\.Keyformer: KV cache reduction through key tokens selection for efficient generative inference\.*Proceedings of Machine Learning and Systems*, 6:114–127, 2024\.
- Aliakbarpour et al\. \[2026\]Maryam Aliakbarpour, Vladimir Braverman, Junze Yin, and Haochen Zhang\.Support basis: Fast attention beyond bounded entries\.In*Proceedings of the 29th International Conference on Artificial Intelligence and Statistics*, 2026\.
- Alman and Song \[2023\]Josh Alman and Zhao Song\.Fast attention requires bounded entries\.*Advances in Neural Information Processing Systems*, 36:63117–63135, 2023\.
- Avsec et al\. \[2021\]Žiga Avsec, Vikram Agarwal, Daniel Visentin, Joseph R Ledsam, Agnieszka Grabska\-Barwinska, Kyle R Taylor, Yannis Assael, John Jumper, Pushmeet Kohli, and David R Kelley\.Effective gene expression prediction from sequence by integrating long\-range interactions\.*Nature Methods*, 18\(10\):1196–1203, 2021\.
- Baevski et al\. \[2020\]Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli\.wav2vec 2\.0: A framework for self\-supervised learning of speech representations\.*Advances in Neural Information Processing Systems*, 33:12449–12460, 2020\.
- Bai et al\. \[2025\]Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, et al\.LongBench v2: Towards deeper understanding and reasoning on realistic long\-context multitasks\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 3639–3664, 2025\.
- Bansal \[2024\]Nikhil Bansal\.On a generalization of iterated and randomized rounding\.*Theory of Computing*, 20\(6\):1–23, 2024\.
- Brown et al\. \[2020\]Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al\.Language models are few\-shot learners\.*Advances in Neural Information Processing Systems*, 33:1877–1901, 2020\.
- Burger et al\. \[2025\]Martin Burger, Samira Kabri, Yury Korolev, Tim Roith, and Lukas Weigand\.Analysis of mean\-field models arising from self\-attention dynamics in transformer architectures with layer normalization\.*Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences*, 383\(2298\), 2025\.
- Cai et al\. \[2024\]Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Junjie Hu, et al\.PyramidKV: Dynamic KV cache compression based on pyramidal information funneling\.*arXiv preprint arXiv:2406\.02069*, 2024\.
- Carrell et al\. \[2025\]Annabelle Michael Carrell, Albert Gong, Abhishek Shetty, Raaz Dwivedi, and Lester Mackey\.Low\-rank thinning\.In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste\-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu, editors,*Proceedings of the 42nd International Conference on Machine Learning*, volume 267 of*Proceedings of Machine Learning Research*, pages 6811–6848\. PMLR, 13–19 Jul 2025\.URL[https://proceedings\.mlr\.press/v267/carrell25a\.html](https://proceedings.mlr.press/v267/carrell25a.html)\.
- Chen et al\. \[2010\]Yutian Chen, Max Welling, and Alex Smola\.Super\-samples from kernel herding\.In*Proceedings of the Twenty\-Sixth Conference on Uncertainty in Artificial Intelligence*, pages 109–116, 2010\.
- Dai et al\. \[2019\]Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G Carbonell, Quoc Le, and Ruslan Salakhutdinov\.Transformer\-xl: Attentive language models beyond a fixed\-length context\.In*Proceedings of the 57th annual meeting of the association for computational linguistics*, pages 2978–2988, 2019\.
- Devlin et al\. \[2019\]Jacob Devlin, Ming\-Wei Chang, Kenton Lee, and Kristina Toutanova\.BERT: Pre\-training of deep bidirectional transformers for language understanding\.In*Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 \(long and short papers\)*, pages 4171–4186, 2019\.
- Geshkovski et al\. \[2025\]Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, and Philippe Rigollet\.A mathematical perspective on transformers\.*Bulletin of the American Mathematical Society*, 62\(3\):427–479, 2025\.
- Haris and Onak \[2025\]Themistoklis Haris and Krzysztof Onak\.Compression barriers for autoregressive transformers\.*arXiv preprint arXiv:2502\.15955*, 2025\.
- Jaggi \[2013\]Martin Jaggi\.Revisiting frank\-wolfe: Projection\-free sparse convex optimization\.In*International conference on machine learning*, pages 427–435\. PMLR, 2013\.
- Kochetkova et al\. \[2025\]Ekaterina Kochetkova, Kshiteej Sheth, Insu Han, Amir Zandieh, and Michael Kapralov\.Streaming attention approximation via discrepancy theory\.*arXiv preprint arXiv:2502\.07861*, 2025\.
- Li et al\. \[2024\]Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen\.SnapKV: LLM knows what you are looking for before generation\.*Advances in Neural Information Processing Systems*, 37:22947–22970, 2024\.
- Liu et al\. \[2023\]Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava\.ScissorHands: Exploiting the persistence of importance hypothesis for LLM KV cache compression at test time\.*Advances in Neural Information Processing Systems*, 36:52342–52364, 2023\.
- Peng et al\. \[2024\]Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole\.YaRN: Efficient context window extension of large language models\.In B\. Kim, Y\. Yue, S\. Chaudhuri, K\. Fragkiadaki, M\. Khan, and Y\. Sun, editors,*International Conference on Learning Representations*, volume 2024, pages 31932–31951, 2024\.URL[https://proceedings\.iclr\.cc/paper\_files/paper/2024/file/874a4d89f2d04b4bcf9a2c19545cf040\-Paper\-Conference\.pdf](https://proceedings.iclr.cc/paper_files/paper/2024/file/874a4d89f2d04b4bcf9a2c19545cf040-Paper-Conference.pdf)\.
- Rives et al\. \[2021\]Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al\.Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences\.*Proceedings of the National Academy of Sciences*, 118\(15\):e2016239118, 2021\.
- Vaswani et al\. \[2017\]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin\.Attention is all you need\.*Advances in Neural Information Processing Systems*, 30, 2017\.
- Xiao et al\. \[2024\]Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis\.Efficient streaming language models with attention sinks\.In B\. Kim, Y\. Yue, S\. Chaudhuri, K\. Fragkiadaki, M\. Khan, and Y\. Sun, editors,*International Conference on Learning Representations*, volume 2024, pages 21875–21895, 2024\.URL[https://proceedings\.iclr\.cc/paper\_files/paper/2024/file/5e5fd18f863cbe6d8ae392a93fd271c9\-Paper\-Conference\.pdf](https://proceedings.iclr.cc/paper_files/paper/2024/file/5e5fd18f863cbe6d8ae392a93fd271c9-Paper-Conference.pdf)\.
- Yang et al\. \[2025\]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al\.Qwen3 technical report\.*arXiv preprint arXiv:2505\.09388*, 2025\.
- Zandieh et al\. \[2024\]Amir Zandieh, Insu Han, Vahab Mirrokni, and Amin Karbasi\.SubGen: Token generation in sublinear time and memory\.*arXiv preprint arXiv:2402\.06082*, 2024\.
- Zhang et al\. \[2023\]Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, et al\.H2O: Heavy\-hitter oracle for efficient generative inference of large language models\.*Advances in Neural Information Processing Systems*, 36:34661–34710, 2023\.

## Appendix Contents

## Appendix ASparse barycenter balancing in Hilbert space

This appendix records an upper bound for the abstract balancing problem in[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)\.

###### Theorem A\.1\.

Letℋ\\mathcal\{H\}be a real Hilbert space, and let

μ=∑i=1npi​δxi∈𝒫n​\(ℋ\)\\textstyle\\mu=\\sum\_\{i=1\}^\{n\}p\_\{i\}\\,\\delta\_\{x\_\{i\}\}\\in\\mathcal\{P\}\_\{n\}\(\\mathcal\{H\}\)have mean zero and covariance operator

Σ≔∑i=1npi​xi⊗xi\.\\textstyle\\Sigma\\coloneqq\\sum\_\{i=1\}^\{n\}p\_\{i\}\\,x\_\{i\}\\otimes x\_\{i\}\.LetU⊆ℋU\\subseteq\\mathcal\{H\}be a finite\-dimensional subspace\. Letd≔dimUd\\coloneqq\\dim U, and letΠU:ℋ→ℋ\\Pi\_\{U\}:\\mathcal\{H\}\\to\\mathcal\{H\}be the orthogonal projector ontoUU\. Then, for everym∈ℕm\\in\\mathbb\{N\}, there existsμ^∈𝒫m\+2​d\+2​\(supp⁡\(μ\)\)\\hat\{\\mu\}\\in\\mathcal\{P\}\_\{m\+2d\+2\}\(\\operatorname\{supp\}\(\\mu\)\)with

∫ℋx​𝑑μ^∈U⟂and‖∫ℋx​𝑑μ^‖ℋ2≤209​m​tr⁡\(\(I−ΠU\)​Σ\)\.\\int\_\{\\mathcal\{H\}\}x\\,d\\hat\{\\mu\}\\in U^\{\\perp\}\\qquad\\text\{and\}\\qquad\\left\\\|\\int\_\{\\mathcal\{H\}\}x\\,d\\hat\{\\mu\}\\right\\\|\_\{\\mathcal\{H\}\}^\{2\}\\leq\\frac\{20\}\{9m\}\\operatorname\{tr\}\\bigl\(\(I\-\\Pi\_\{U\}\)\\Sigma\\bigr\)\.

###### Proof\.

Deferred to[Section˜A\.3](https://arxiv.org/html/2607.01520#A1.SS3)\. ∎

While[Theorem˜A\.1](https://arxiv.org/html/2607.01520#A1.Thmtheorem1)only claims existence, we give a constructive proof with concrete algorithms\. We later invoke this construction when we design KV compression algorithms in[Appendix˜C](https://arxiv.org/html/2607.01520#A3)\.

For our construction, we reduce sparsifying a finitely supported measureμ=∑i=1npi​δxi∈𝒫n​\(ℋ\)\\mu=\\sum\_\{i=1\}^\{n\}p\_\{i\}\\,\\delta\_\{x\_\{i\}\}\\in\\mathcal\{P\}\_\{n\}\(\\mathcal\{H\}\)to a*constrained rounding problem*: Given a vector\(p1,…,pn\)∈\[0,1\]n\(p\_\{1\},\.\.\.,p\_\{n\}\)\\in\[0,1\]^\{n\}, find a \(random\) rounded vector\(p^1,…,p^n\)∈\[0,1\]n\(\\hat\{p\}\_\{1\},\.\.\.,\\hat\{p\}\_\{n\}\)\\in\[0,1\]^\{n\}with at mostm\+2​d\+2m\+2d\+2fractional entries preserving the mass ofppwhile satisfying linear constraints\. As we show, mass preservation is sufficient to construct a sparse measure supported on at mostm\+2​d\+2m\+2d\+2points from such a constrained rounding procedure, and linear constraints are used to encode exact preservation of the barycenter in the subspaceUU\.

In[Section˜A\.1](https://arxiv.org/html/2607.01520#A1.SS1), we first describe a general algorithm for constrained rounding with mass preservation\.[Section˜A\.2](https://arxiv.org/html/2607.01520#A1.SS2)then specializes this algorithm to sparsification of measures with a protected subspace, and[Section˜A\.3](https://arxiv.org/html/2607.01520#A1.SS3)proves[Theorem˜A\.1](https://arxiv.org/html/2607.01520#A1.Thmtheorem1)from this\.

### A\.1Finite\-dimensional constrained rounding

We first isolate a finite\-dimensional constrained rounding procedure\. Starting from a fractional weight vector, the goal is to round most coordinates while preserving mass and satisfying linear constraints\. Such a procedure can be obtained as a straightforward application of the sub\-isotropic rounding walk ofBansal \[[7](https://arxiv.org/html/2607.01520#bib.bib7)\]\. Given a current pointx∈\[0,1\]nx\\in\[0,1\]^\{n\}, an alive setA⊆\[n\]A\\subseteq\[n\], and linear constraints, we write

x\+=SubIsoWalk\(x,A\|∑s=1nzs=m,∑s=1nzscs=0\)x^\{\+\}=\\textsc\{SubIsoWalk\}\\\!\\left\(x,A\\;\\middle\|\\;\\sum\_\{s=1\}^\{n\}z\_\{s\}=m,\\;\\sum\_\{s=1\}^\{n\}z\_\{s\}c\_\{s\}=0\\right\)for the stopped phase of Bansal’s walk, restricted to the coordinates inAA, that returns a random pointx\+∈\[0,1\]nx^\{\+\}\\in\[0,1\]^\{n\}satisfying the constraints

∑s=1nxs\+=m,∑s=1nxs\+​cs=0,\\sum\_\{s=1\}^\{n\}x^\{\+\}\_\{s\}=m,\\qquad\\sum\_\{s=1\}^\{n\}x^\{\+\}\_\{s\}c\_\{s\}=0,andxs\+=xsx^\{\+\}\_\{s\}=x\_\{s\}for alls∉As\\not\\in A, such that at least one coordinate inAAreaches0or11\.[Algorithm˜3](https://arxiv.org/html/2607.01520#alg3)simply applies this procedure iteratively until only2​d\+22d\+2fractional coordinates remain, which yields the following guarantees on the output\.

Algorithm 3ConstrainedPartialRound1:

z∈\[0,1\]Nz\\in\[0,1\]^\{N\}, vectors

c1,…,cN∈ℝdc\_\{1\},\\ldots,c\_\{N\}\\in\\mathbb\{R\}^\{d\}
2:

∑s=1Nzs=m\\sum\_\{s=1\}^\{N\}z\_\{s\}=mand

∑s=1Nzs​cs=0\\sum\_\{s=1\}^\{N\}z\_\{s\}\\,c\_\{s\}=0
3:

t←0,z\(0\)←zt\\leftarrow 0,\\quad z^\{\(0\)\}\\leftarrow z
4:

A←\{s∈\[N\]∣0<zs\(0\)<1\}A\\leftarrow\\\{s\\in\[N\]\\;\\mid\\;0<z\_\{s\}^\{\(0\)\}<1\\\}⊳\\triangleright“alive set”

5:while

\|A\|\>2​d\+2\|A\|\>2d\+2do

6:

t←t\+1t\\leftarrow t\+1
7:

z\(t\)←SubIsoWalk\(z\(t−1\),A∣∑s=1Nzs=m,∑s=1Nzscs=0\)z^\{\(t\)\}\\leftarrow\\textsc\{SubIsoWalk\}\\\!\\left\(z^\{\(t\-1\)\},A\\,\\mid\\,\\sum\_\{s=1\}^\{N\}z\_\{s\}=m,\\sum\_\{s=1\}^\{N\}z\_\{s\}c\_\{s\}=0\\right\)⊳\\trianglerightprovided by\[[7](https://arxiv.org/html/2607.01520#bib.bib7)\]

8:

A←\{s∈\[N\]∣0<zs\(t\)<1\}A\\leftarrow\\\{s\\in\[N\]\\;\\mid\\;0<z\_\{s\}^\{\(t\)\}<1\\\}
9:return

z\(t\)z^\{\(t\)\}

###### Lemma A\.2\.

Fixn,d,m∈ℕn,d,m\\in\\mathbb\{N\}\. Letz∈\[0,1\]nz\\in\[0,1\]^\{n\}andc1,…,cn∈ℝdc\_\{1\},\.\.\.,c\_\{n\}\\in\\mathbb\{R\}^\{d\}\. Assume that∑s=1nzs​cs=0\\sum\_\{s=1\}^\{n\}z\_\{s\}\\,c\_\{s\}=0andm≔∑s=1nzs∈ℕm\\coloneqq\\sum\_\{s=1\}^\{n\}z\_\{s\}\\in\\mathbb\{N\}\. LetZ≔ConstrainedPartialRound​\(z,c1,…,cn\)Z\\coloneqq\\textsc\{ConstrainedPartialRound\}\(z,\\,c\_\{1\},\.\.\.,c\_\{n\}\)be the random output of[Algorithm˜3](https://arxiv.org/html/2607.01520#alg3)\. Then, the following hold:

1. \(i\)∑s=1nZs=m\\sum\_\{s=1\}^\{n\}Z\_\{s\}=malmost surely
2. \(ii\)∑s=1nZs​cs=0\\sum\_\{s=1\}^\{n\}Z\_\{s\}\\,c\_\{s\}=0almost surely
3. \(iii\)\|\{s∈\[n\]∣0<Zs<1\}\|≤2​d\+2\|\\\{s\\in\[n\]\\mid 0<Z\_\{s\}<1\\\}\|\\leq 2d\+2almost surely
4. \(iv\)𝔼⁡\[Z\]=z\\operatorname\{\\mathbb\{E\}\}\[Z\]=z
5. \(v\)for everya∈ℝna\\in\\mathbb\{R\}^\{n\}, 𝔼⁡\[\(∑s=1n\(Zs−zs\)​a\)2\]≤209​∑s=1nas2​zs​\(1−zs\)\.\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\Bigl\(\\sum\_\{s=1\}^\{n\}\(Z\_\{s\}\-z\_\{s\}\)\\,a\\Bigr\)^\{2\}\\right\]\\leq\\frac\{20\}\{9\}\\sum\_\{s=1\}^\{n\}a\_\{s\}^\{2\}z\_\{s\}\(1\-z\_\{s\}\)\.

###### Proof\.

Forx∈\[0,1\]nx\\in\[0,1\]^\{n\}, let

A​\(x\)≔\{s∈\[n\]:xs∈\(0,1\)\}A\(x\)\\coloneqq\\\{s\\in\[n\]:x\_\{s\}\\in\(0,1\)\\\}be the alive set\. The algorithm runs the sub\-isotropic rounding walk ofBansal \[[7](https://arxiv.org/html/2607.01520#bib.bib7), Section 3\.1\]with the following deterministic schedule of active constraints\. As long as\|A​\(x\)\|\>2​d\+2\|A\(x\)\|\>2d\+2, we keep the constraints

∑s=1nxs=mand∑s=1nxs​cs=0\\sum\_\{s=1\}^\{n\}x\_\{s\}=m\\qquad\\text\{and\}\\qquad\\sum\_\{s=1\}^\{n\}x\_\{s\}c\_\{s\}=0active on the alive coordinates\. Once\|A​\(x\)\|≤2​d\+2\|A\(x\)\|\\leq 2d\+2, the algorithm stops\.

We first check that the active constraints satisfy the rank condition required byBansal \[[7](https://arxiv.org/html/2607.01520#bib.bib7)\]\. Restricted to the alive coordinates, the active system has at mostd\+1d\+1scalar constraints\. Whenever the loop is entered,\|A​\(x\)\|\>2​d\+2\|A\(x\)\|\>2d\+2, and therefore

d\+1≤\|A​\(x\)\|2\.d\+1\\leq\\frac\{\|A\(x\)\|\}\{2\}\.Thus the slack hypothesis in\[[7](https://arxiv.org/html/2607.01520#bib.bib7), Theorem 1\.2\]holds with parameterδ=1/2\\delta=1/2\.

Let\(z\(k\)\)k≥0\(z^\{\(k\)\}\)\_\{k\\geq 0\}denote the elementary iterates of the resulting walk, and let\(ℱk\)k≥0\(\\mathcal\{F\}\_\{k\}\)\_\{k\\geq 0\}be its natural filtration\. LetTTbe the terminal time of the full walk and write

X≔z\(T\)∈\{0,1\}nX\\coloneqq z^\{\(T\)\}\\in\\\{0,1\\\}^\{n\}for the final integral vector\. By the martingale property of Bansal’s walk,𝔼⁡\[X\]=z\\operatorname\{\\mathbb\{E\}\}\[X\]=z\. Let

τ≔inf\{k≥0:\|A​\(z\(k\)\)\|≤2​d\+2\}\.\\tau\\coloneqq\\inf\\left\\\{k\\geq 0:\\bigl\|A\(z^\{\(k\)\}\)\\bigr\|\\leq 2d\+2\\right\\\}\.Then,Z≔z\(τ\)Z\\coloneqq z^\{\(\\tau\)\}\.

Before timeτ\\tau, every increment lies in the nullspace of the active constraints\. Hence, the identities

∑s=1nzs\(k\)=mand∑s=1nzs\(k\)​cs=0\\sum\_\{s=1\}^\{n\}z\_\{s\}^\{\(k\)\}=m\\qquad\\text\{and\}\\qquad\\sum\_\{s=1\}^\{n\}z\_\{s\}^\{\(k\)\}c\_\{s\}=0are preserved pathwise for allk≤τk\\leq\\tau\. Since they hold at initialization, they hold forZZ, which proves

∑s=1nZs=m,∑s=1nZs​cs=0\.\\sum\_\{s=1\}^\{n\}Z\_\{s\}=m,\\qquad\\sum\_\{s=1\}^\{n\}Z\_\{s\}c\_\{s\}=0\.The definition ofτ\\taugives

\|\{s∈\[n\]∣0<Zs<1\}\|≤2​d\+2\.\\bigl\|\\\{s\\in\[n\]\\mid 0<Z\_\{s\}<1\\\}\\bigr\|\\leq 2d\+2\.
It remains to prove unbiasedness and the scalar second\-moment bound\. Fork≥1k\\geq 1, set

Δ\(k\)≔z\(k\)−z\(k−1\),\\Delta^\{\(k\)\}\\coloneqq z^\{\(k\)\}\-z^\{\(k\-1\)\},and putΔ\(k\)≔0\\Delta^\{\(k\)\}\\coloneqq 0fork\>Tk\>T\. In the notation ofBansal \[[7](https://arxiv.org/html/2607.01520#bib.bib7), Section 3\.1\], conditional onℱk−1\\mathcal\{F\}\_\{k\-1\}, the increment on the alive coordinates is of the formγk​Uk1/2​rk\\gamma\_\{k\}U\_\{k\}^\{1/2\}r\_\{k\}, whererkr\_\{k\}is a random sign vector andUkU\_\{k\}is positive semidefinite with range contained in the nullspace of the active constraints\. Embedding the conditional covariance intoℝn\\mathbb\{R\}^\{n\}and padding zeros outside the alive coordinates gives a matrixU~k\\widetilde\{U\}\_\{k\}such that

𝔼⁡\[Δ\(k\)∣ℱk−1\]=0,𝔼⁡\[Δ\(k\)​\(Δ\(k\)\)⊤∣ℱk−1\]=U~k\.\\operatorname\{\\mathbb\{E\}\}\[\\Delta^\{\(k\)\}\\mid\\mathcal\{F\}\_\{k\-1\}\]=0,\\qquad\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\Delta^\{\(k\)\}\(\\Delta^\{\(k\)\}\)^\{\\top\}\\mid\\mathcal\{F\}\_\{k\-1\}\\right\]=\\widetilde\{U\}\_\{k\}\.Moreover, the sub\-isotropic covariance estimate in the same construction gives, forδ=1/2\\delta=1/2,

U~k⪯209​diag⁡\(U~k\)\.\\widetilde\{U\}\_\{k\}\\preceq\\frac\{20\}\{9\}\\operatorname\{diag\}\(\\widetilde\{U\}\_\{k\}\)\.
Fixa∈ℝna\\in\\mathbb\{R\}^\{n\}\. Since\(⟨a,z\(k\)⟩\)k≥0\(\\langle a,z^\{\(k\)\}\\rangle\)\_\{k\\geq 0\}is a martingale, its increments are orthogonal inL2L\_\{2\}\. Hence,

𝔼⁡\[⟨a,X−z⟩2\]=∑k≥1𝔼⁡\[⟨a,Δ\(k\)⟩2\]\.\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\bigl\\langle a,X\-z\\bigr\\rangle^\{2\}\\right\]=\\sum\_\{k\\geq 1\}\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\bigl\\langle a,\\Delta^\{\(k\)\}\\bigr\\rangle^\{2\}\\right\]\.Using the conditional covariance estimate,

𝔼⁡\[⟨a,Δ\(k\)⟩2∣ℱk−1\]\\displaystyle\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\bigl\\langle a,\\Delta^\{\(k\)\}\\bigr\\rangle^\{2\}\\mid\\mathcal\{F\}\_\{k\-1\}\\right\]=a⊤​U~k​a\\displaystyle=a^\{\\top\}\\widetilde\{U\}\_\{k\}a≤209​∑s=1nas2​\(U~k\)s​s\\displaystyle\\leq\\frac\{20\}\{9\}\\sum\_\{s=1\}^\{n\}a\_\{s\}^\{2\}\(\\widetilde\{U\}\_\{k\}\)\_\{ss\}=209​∑s=1nas2​𝔼⁡\[\(Δs\(k\)\)2∣ℱk−1\]\.\\displaystyle=\\frac\{20\}\{9\}\\sum\_\{s=1\}^\{n\}a\_\{s\}^\{2\}\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\(\\Delta\_\{s\}^\{\(k\)\}\)^\{2\}\\mid\\mathcal\{F\}\_\{k\-1\}\\right\]\.Summing overkkand taking expectations yields

𝔼⁡\[⟨a,X−z⟩2\]≤209​∑s=1nas2​∑k≥1𝔼⁡\[\(Δs\(k\)\)2\]\.\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\bigl\\langle a,X\-z\\bigr\\rangle^\{2\}\\right\]\\leq\\frac\{20\}\{9\}\\sum\_\{s=1\}^\{n\}a\_\{s\}^\{2\}\\sum\_\{k\\geq 1\}\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\(\\Delta\_\{s\}^\{\(k\)\}\)^\{2\}\\right\]\.For each coordinatess, the process\(zs\(k\)\)k≥0\(z\_\{s\}^\{\(k\)\}\)\_\{k\\geq 0\}is a bounded martingale with terminal valueXs∈\{0,1\}X\_\{s\}\\in\\\{0,1\\\}\. Its increments are orthogonal inL2L\_\{2\}, and therefore

∑k≥1𝔼⁡\[\(Δs\(k\)\)2\]=𝔼⁡\[\(Xs−zs\)2\]=zs​\(1−zs\),\\sum\_\{k\\geq 1\}\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\(\\Delta\_\{s\}^\{\(k\)\}\)^\{2\}\\right\]=\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\(X\_\{s\}\-z\_\{s\}\)^\{2\}\\right\]=z\_\{s\}\(1\-z\_\{s\}\),where we usedXs2=XsX\_\{s\}^\{2\}=X\_\{s\}and𝔼⁡\[Xs\]=zs\\operatorname\{\\mathbb\{E\}\}\[X\_\{s\}\]=z\_\{s\}\. Consequently,

𝔼⁡\[⟨a,X−z⟩2\]≤209​∑s=1nas2​zs​\(1−zs\)\.\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\bigl\\langle a,X\-z\\bigr\\rangle^\{2\}\\right\]\\leq\\frac\{20\}\{9\}\\sum\_\{s=1\}^\{n\}a\_\{s\}^\{2\}z\_\{s\}\(1\-z\_\{s\}\)\.
Finally, the stopped process\(z\(k∧T\)\)k≥0\(z^\{\(k\\wedge T\)\}\)\_\{k\\geq 0\}is a bounded martingale andτ≤T\\tau\\leq Talmost surely\. Hence, optional stopping gives

Zs=zs\(τ\)=𝔼⁡\[Xs∣ℱτ\]Z\_\{s\}=z\_\{s\}^\{\(\\tau\)\}=\\operatorname\{\\mathbb\{E\}\}\[X\_\{s\}\\mid\\mathcal\{F\}\_\{\\tau\}\]for everys∈\[n\]s\\in\[n\]\. Taking expectations gives𝔼⁡\[Z\]=z\\operatorname\{\\mathbb\{E\}\}\[Z\]=z\. Moreover,

⟨a,Z−z⟩=𝔼⁡\[⟨a,X−z⟩∣ℱτ\],\\bigl\\langle a,Z\-z\\bigr\\rangle=\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\bigl\\langle a,X\-z\\bigr\\rangle\\mid\\mathcal\{F\}\_\{\\tau\}\\right\],and conditional Jensen yields

𝔼⁡\[⟨a,Z−z⟩2\]≤𝔼⁡\[⟨a,X−z⟩2\]≤209​∑s=1nas2​zs​\(1−zs\)\.\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\bigl\\langle a,Z\-z\\bigr\\rangle^\{2\}\\right\]\\leq\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\bigl\\langle a,X\-z\\bigr\\rangle^\{2\}\\right\]\\leq\\frac\{20\}\{9\}\\sum\_\{s=1\}^\{n\}a\_\{s\}^\{2\}z\_\{s\}\(1\-z\_\{s\}\)\.That proves the claim\. ∎

### A\.2Protected Hilbert rounding

We now apply the constrained rounding procedure to finitely supported measures on a Hilbert space\. The linear constraints are chosen to preserve the barycenter in a protected subspace, and mass preservation forces small support\.

Algorithm 4ProtectedSparsify1:

μ=∑j=1Spj​δxj∈𝒫fin​\(ℋ\)\\mu=\\sum\_\{j=1\}^\{S\}p\_\{j\}\\,\\delta\_\{x\_\{j\}\}\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{H\}\)
2:finite\-dimensional subspace

U⊆ℋU\\subseteq\\mathcal\{H\}with

d=dim\(U\)d=\\dim\(U\)
3:integer

m≥1m\\geq 1
4:

x¯←∫x​d​μ\\bar\{x\}\\leftarrow\\int x\\mathop\{\}\\\!\\mathrm\{d\}\\mu
5:

u1,…,ud←orthonormal basis ofUu\_\{1\},\.\.\.,u\_\{d\}\\leftarrow\\text\{orthonormal basis of \\ $U$\}
6:

s←0s\\leftarrow 0⊳\\trianglerightsplit atoms to have small mass

7:for

j=1,…,Sj=1,\\ldots,Sdo

8:

nj←⌈m​pj⌉n\_\{j\}\\leftarrow\\lceil mp\_\{j\}\\rceil
9:for

ℓ=1,…,nj\\ell=1,\\ldots,n\_\{j\}do

10:

s←s\+1s\\leftarrow s\+1
11:

ys←xjy\_\{s\}\\leftarrow x\_\{j\}
12:

ιs←j\\iota\_\{s\}\\leftarrow j
13:

αs←pj/nj\\alpha\_\{s\}\\leftarrow p\_\{j\}/n\_\{j\}
14:

N←sN\\leftarrow s
15:for

s=1,…,Ns=1,\\ldots,Ndo⊳\\trianglerightcompute protected coordinates

16:

ξs←ys−x¯\\xi\_\{s\}\\leftarrow y\_\{s\}\-\\bar\{x\}
17:

cs←\(⟨ξs,u1⟩,…,⟨ξs,ud⟩\)∈ℝdc\_\{s\}\\leftarrow\\bigl\(\\langle\\xi\_\{s\},u\_\{1\}\\rangle,\\ldots,\\langle\\xi\_\{s\},u\_\{d\}\\rangle\\bigr\)\\in\\mathbb\{R\}^\{d\}
18:

zs←m​αsz\_\{s\}\\leftarrow m\\alpha\_\{s\}
19:

Z←ConstrainedPartialRound​\(z,c1,…,cN\)Z\\leftarrow\\textsc\{ConstrainedPartialRound\}\\\!\\left\(z,\\,c\_\{1\},\\ldots,c\_\{N\}\\right\)⊳\\trianglerightsparsify

20:for

j=1,…,Sj=1,\\ldots,Sdo⊳\\trianglerightreassemble atoms

21:

p^j←1m​∑s:ιs=jZs\\hat\{p\}\_\{j\}\\leftarrow\\frac\{1\}\{m\}\\sum\_\{s:\\,\\iota\_\{s\}=j\}Z\_\{s\}
22:

μ^←∑j=1Sp^j​δxj\\hat\{\\mu\}\\leftarrow\\sum\_\{j=1\}^\{S\}\\hat\{p\}\_\{j\}\\,\\delta\_\{x\_\{j\}\}
23:return

μ^\\hat\{\\mu\}

###### Proposition A\.3\.

Fix a Hilbert spaceℋ\\mathcal\{H\}, a finite\-dimensional subspaceU⊆ℋU\\subseteq\\mathcal\{H\}withd≔dim\(U\)d\\coloneqq\\dim\(U\), and somem≥1m\\geq 1\. Letμ=∑j=1Spj​δxj∈𝒫fin​\(ℋ\)\\mu=\\sum\_\{j=1\}^\{S\}p\_\{j\}\\,\\delta\_\{x\_\{j\}\}\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{H\}\)and let

μ^≔ProtectedSparsify​\(μ,U,m\)=∑j=1Sp^j​δxj\\textstyle\\hat\{\\mu\}\\coloneqq\\textsc\{ProtectedSparsify\}\(\\mu,U,m\)=\\sum\_\{j=1\}^\{S\}\\hat\{p\}\_\{j\}\\,\\delta\_\{x\_\{j\}\}be the random output of[Algorithm˜4](https://arxiv.org/html/2607.01520#alg4)\. Write

x¯≔∫ℋx​d​μ,Σ≔∫ℋ\(x−x¯\)⊗\(x−x¯\)​d​μ,η≔∫ℋx​d​\(μ^−μ\)\.\\textstyle\\bar\{x\}\\coloneqq\\int\_\{\\mathcal\{H\}\}x\\mathop\{\}\\\!\\mathrm\{d\}\\mu,\\qquad\\Sigma\\coloneqq\\int\_\{\\mathcal\{H\}\}\(x\-\\bar\{x\}\)\\otimes\(x\-\\bar\{x\}\)\\mathop\{\}\\\!\\mathrm\{d\}\\mu,\\qquad\\eta\\coloneqq\\int\_\{\\mathcal\{H\}\}x\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{\\mu\}\-\\mu\)\.Then, the following hold:

1. \(i\)μ^∈𝒫m\+2​d\+2​\(supp⁡\(μ\)\)\\hat\{\\mu\}\\in\\mathcal\{P\}\_\{m\+2d\+2\}\(\\operatorname\{supp\}\(\\mu\)\)almost surely
2. \(ii\)∫ℋx​d​\(μ^−μ\)∈U⟂\\int\_\{\\mathcal\{H\}\}x\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{\\mu\}\-\\mu\)\\in U^\{\\perp\}almost surely
3. \(iii\)𝔼⁡\[μ^\]=μ\\operatorname\{\\mathbb\{E\}\}\[\\hat\{\\mu\}\]=\\mu
4. \(iv\)𝔼⁡\[η⊗η\]⪯209​m​\(I−ΠU\)​Σ​\(I−ΠU\)\\operatorname\{\\mathbb\{E\}\}\[\\eta\\otimes\\eta\]\\preceq\\frac\{20\}\{9m\}\(I\-\\Pi\_\{U\}\)\\Sigma\(I\-\\Pi\_\{U\}\)

Additionally, for every Hilbert space𝒢\\mathcal\{G\}and everyy1,…,yS∈𝒢y\_\{1\},\\ldots,y\_\{S\}\\in\\mathcal\{G\}, writing

y¯≔∑j=1Spj​yj,Σy≔∑j=1Spj​\(yj−y¯\)⊗\(yj−y¯\),\\textstyle\\bar\{y\}\\coloneqq\\sum\_\{j=1\}^\{S\}p\_\{j\}y\_\{j\},\\qquad\\Sigma\_\{y\}\\coloneqq\\sum\_\{j=1\}^\{S\}p\_\{j\}\(y\_\{j\}\-\\bar\{y\}\)\\otimes\(y\_\{j\}\-\\bar\{y\}\),one has

𝔼⁡‖∑j=1S\(p^j−pj\)​yj‖𝒢2≤209​m​tr⁡\(Σy\)\.\\operatorname\{\\mathbb\{E\}\}\\Bigl\\\|\\sum\_\{j=1\}^\{S\}\(\\hat\{p\}\_\{j\}\-p\_\{j\}\)y\_\{j\}\\Bigr\\\|\_\{\\mathcal\{G\}\}^\{2\}\\leq\\frac\{20\}\{9m\}\\operatorname\{tr\}\(\\Sigma\_\{y\}\)\.

###### Proof\.

Use the notation of[Algorithm˜4](https://arxiv.org/html/2607.01520#alg4)\. We verify the four guarantees in the order suggested by the construction: first that the finite\-dimensional rounding lemma applies, then support preservation and unbiasedness, then exact cancellation onUU, and finally the operator second\-moment bound\.

The splitting step gives

∑s=1Nαs​δys=μ,0<αs≤1m\.\\sum\_\{s=1\}^\{N\}\\alpha\_\{s\}\\delta\_\{y\_\{s\}\}=\\mu,\\qquad 0<\\alpha\_\{s\}\\leq\\frac\{1\}\{m\}\.Hencezs=m​αs∈\[0,1\]z\_\{s\}=m\\alpha\_\{s\}\\in\[0,1\]and∑szs=m\\sum\_\{s\}z\_\{s\}=m\. Since

∑s=1Nαs​ξs=∫ℋ\(x−x¯\)​d​μ=0,\\sum\_\{s=1\}^\{N\}\\alpha\_\{s\}\\xi\_\{s\}=\\int\_\{\\mathcal\{H\}\}\(x\-\\bar\{x\}\)\\mathop\{\}\\\!\\mathrm\{d\}\\mu=0,the protected coordinates satisfy

∑s=1Nzs​cs=0\.\\sum\_\{s=1\}^\{N\}z\_\{s\}c\_\{s\}=0\.Thus,ConstrainedPartialRoundapplies\. By[Lemma˜A\.2](https://arxiv.org/html/2607.01520#A1.Thmtheorem2), it returnsZ∈\[0,1\]NZ\\in\[0,1\]^\{N\}with

∑s=1NZs=m,∑s=1NZs​cs=0,𝔼⁡\[Z\]=z,\\sum\_\{s=1\}^\{N\}Z\_\{s\}=m,\\qquad\\sum\_\{s=1\}^\{N\}Z\_\{s\}c\_\{s\}=0,\\qquad\\operatorname\{\\mathbb\{E\}\}\[Z\]=z,at most2​d\+22d\+2fractional coordinates, and, for everya∈ℝNa\\in\\mathbb\{R\}^\{N\},

𝔼⁡\[\(∑s=1N\(Zs−zs\)​as\)2\]≤209​∑s=1Nzs​\(1−zs\)​as2\.\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\left\(\\sum\_\{s=1\}^\{N\}\(Z\_\{s\}\-z\_\{s\}\)a\_\{s\}\\right\)^\{2\}\\right\]\\leq\\frac\{20\}\{9\}\\sum\_\{s=1\}^\{N\}z\_\{s\}\(1\-z\_\{s\}\)a\_\{s\}^\{2\}\.
The non\-fractional positive coordinates ofZZare equal to11, and their count is at most∑sZs=m\\sum\_\{s\}Z\_\{s\}=m\. Together with the at most2​d\+22d\+2fractional coordinates, this shows that at mostm\+2​d\+2m\+2d\+2split copies receive positive mass\. After reassembling split copies, the same bound holds for the number of original atoms withp^j\>0\\hat\{p\}\_\{j\}\>0\. Hence,

μ^∈𝒫m\+2​d\+2​\(supp⁡\(μ\)\)\\hat\{\\mu\}\\in\\mathcal\{P\}\_\{m\+2d\+2\}\(\\operatorname\{supp\}\(\\mu\)\)almost surely\.

Moreover, for eachjj,

𝔼⁡\[p^j\]=1m​∑s:ιs=j𝔼⁡\[Zs\]=1m​∑s:ιs=jzs=∑s:ιs=jαs=pj\.\\operatorname\{\\mathbb\{E\}\}\[\\hat\{p\}\_\{j\}\]=\\frac\{1\}\{m\}\\sum\_\{s:\\,\\iota\_\{s\}=j\}\\operatorname\{\\mathbb\{E\}\}\[Z\_\{s\}\]=\\frac\{1\}\{m\}\\sum\_\{s:\\,\\iota\_\{s\}=j\}z\_\{s\}=\\sum\_\{s:\\,\\iota\_\{s\}=j\}\\alpha\_\{s\}=p\_\{j\}\.Therefore,

𝔼⁡\[μ^\]=∑j=1S𝔼⁡\[p^j\]​δxj=∑j=1Spj​δxj=μ\.\\operatorname\{\\mathbb\{E\}\}\[\\hat\{\\mu\}\]=\\sum\_\{j=1\}^\{S\}\\operatorname\{\\mathbb\{E\}\}\[\\hat\{p\}\_\{j\}\]\\delta\_\{x\_\{j\}\}=\\sum\_\{j=1\}^\{S\}p\_\{j\}\\delta\_\{x\_\{j\}\}=\\mu\.
By the reassembly definition ofp^j\\hat\{p\}\_\{j\},

η=∑j=1S\(p^j−pj\)​xj=1m​∑s=1N\(Zs−zs\)​ys\.\\eta=\\sum\_\{j=1\}^\{S\}\(\\hat\{p\}\_\{j\}\-p\_\{j\}\)x\_\{j\}=\\frac\{1\}\{m\}\\sum\_\{s=1\}^\{N\}\(Z\_\{s\}\-z\_\{s\}\)y\_\{s\}\.Since∑sZs=∑szs=m\\sum\_\{s\}Z\_\{s\}=\\sum\_\{s\}z\_\{s\}=m, the constant componentx¯\\bar\{x\}cancels, and hence

η=1m​∑s=1N\(Zs−zs\)​ξs\.\\eta=\\frac\{1\}\{m\}\\sum\_\{s=1\}^\{N\}\(Z\_\{s\}\-z\_\{s\}\)\\xi\_\{s\}\.Moreover,∑s\(Zs−zs\)​cs=0\\sum\_\{s\}\(Z\_\{s\}\-z\_\{s\}\)c\_\{s\}=0, and therefore

⟨η,uℓ⟩=1m​∑s=1N\(Zs−zs\)​⟨ξs,uℓ⟩=0\(ℓ=1,…,d\)\.\\langle\\eta,u\_\{\\ell\}\\rangle=\\frac\{1\}\{m\}\\sum\_\{s=1\}^\{N\}\(Z\_\{s\}\-z\_\{s\}\)\\langle\\xi\_\{s\},u\_\{\\ell\}\\rangle=0\\qquad\(\\ell=1,\\ldots,d\)\.Thusη∈U⟂\\eta\\in U^\{\\perp\}, and consequently

η=1m​∑s=1N\(Zs−zs\)​\(I−ΠU\)​ξs\.\\eta=\\frac\{1\}\{m\}\\sum\_\{s=1\}^\{N\}\(Z\_\{s\}\-z\_\{s\}\)\(I\-\\Pi\_\{U\}\)\\xi\_\{s\}\.
Fixh∈ℋh\\in\\mathcal\{H\}, and apply the scalar second\-moment bound withas=⟨h,\(I−ΠU\)​ξs⟩a\_\{s\}=\\langle h,\(I\-\\Pi\_\{U\}\)\\xi\_\{s\}\\rangle\. Sincezs=m​αsz\_\{s\}=m\\alpha\_\{s\},

𝔼⟨h,η⟩2\\displaystyle\\operatorname\{\\mathbb\{E\}\}\\langle h,\\eta\\rangle^\{2\}≤209​m2​∑s=1Nzs​\(1−zs\)​⟨h,\(I−ΠU\)​ξs⟩2\\displaystyle\\leq\\frac\{20\}\{9m^\{2\}\}\\sum\_\{s=1\}^\{N\}z\_\{s\}\(1\-z\_\{s\}\)\\langle h,\(I\-\\Pi\_\{U\}\)\\xi\_\{s\}\\rangle^\{2\}≤209​m​∑s=1Nαs​⟨h,\(I−ΠU\)​ξs⟩2\\displaystyle\\leq\\frac\{20\}\{9m\}\\sum\_\{s=1\}^\{N\}\\alpha\_\{s\}\\langle h,\(I\-\\Pi\_\{U\}\)\\xi\_\{s\}\\rangle^\{2\}=209​m​⟨h,\(I−ΠU\)​Σμ​\(I−ΠU\)​h⟩\.\\displaystyle=\\frac\{20\}\{9m\}\\left\\langle h,\(I\-\\Pi\_\{U\}\)\\Sigma\_\{\\mu\}\(I\-\\Pi\_\{U\}\)h\\right\\rangle\.Sinceh∈ℋh\\in\\mathcal\{H\}was arbitrary, the operator inequality follows\.

It remains to prove the auxiliary variance bound\. Since∑j\(p^j−pj\)=0\\sum\_\{j\}\(\\hat\{p\}\_\{j\}\-p\_\{j\}\)=0,

∑j=1S\(p^j−pj\)​yj=∑j=1S\(p^j−pj\)​\(yj−y¯\)\.\\sum\_\{j=1\}^\{S\}\(\\hat\{p\}\_\{j\}\-p\_\{j\}\)y\_\{j\}=\\sum\_\{j=1\}^\{S\}\(\\hat\{p\}\_\{j\}\-p\_\{j\}\)\(y\_\{j\}\-\\bar\{y\}\)\.Using the split copies,

∑j=1S\(p^j−pj\)​\(yj−y¯\)=1m​∑s=1N\(Zs−zs\)​\(yιs−y¯\)\.\\sum\_\{j=1\}^\{S\}\(\\hat\{p\}\_\{j\}\-p\_\{j\}\)\(y\_\{j\}\-\\bar\{y\}\)=\\frac\{1\}\{m\}\\sum\_\{s=1\}^\{N\}\(Z\_\{s\}\-z\_\{s\}\)\(y\_\{\\iota\_\{s\}\}\-\\bar\{y\}\)\.Fixg∈𝒢g\\in\\mathcal\{G\}and apply the scalar second\-moment bound with

as=⟨g,yιs−y¯⟩𝒢\.a\_\{s\}=\\langle g,y\_\{\\iota\_\{s\}\}\-\\bar\{y\}\\rangle\_\{\\mathcal\{G\}\}\.Then

𝔼⁡\[⟨g,∑j=1S\(p^j−pj\)​yj⟩2\]≤209​m2​∑s=1Nzs​\(1−zs\)​⟨g,yιs−y¯⟩2\.\\operatorname\{\\mathbb\{E\}\}\\left\[\\left\\langle g,\\sum\_\{j=1\}^\{S\}\(\\hat\{p\}\_\{j\}\-p\_\{j\}\)y\_\{j\}\\right\\rangle^\{2\}\\right\]\\leq\\frac\{20\}\{9m^\{2\}\}\\sum\_\{s=1\}^\{N\}z\_\{s\}\(1\-z\_\{s\}\)\\langle g,y\_\{\\iota\_\{s\}\}\-\\bar\{y\}\\rangle^\{2\}\.Sincezs=m​αs≤1z\_\{s\}=m\\alpha\_\{s\}\\leq 1,

zs​\(1−zs\)≤m​αs\.z\_\{s\}\(1\-z\_\{s\}\)\\leq m\\alpha\_\{s\}\.Therefore,

𝔼⟨g,∑j=1S\(p^j−pj\)yj⟩2≤209​m∑s=1Nαs⟨g,yιs−y¯⟩2\.\\operatorname\{\\mathbb\{E\}\}\\left\\langle g,\\sum\_\{j=1\}^\{S\}\(\\hat\{p\}\_\{j\}\-p\_\{j\}\)y\_\{j\}\\right\\rangle^\{2\}\\leq\\frac\{20\}\{9m\}\\sum\_\{s=1\}^\{N\}\\alpha\_\{s\}\\langle g,y\_\{\\iota\_\{s\}\}\-\\bar\{y\}\\rangle^\{2\}\.Reassembling the split copies gives

∑s=1Nαs​\(yιs−y¯\)⊗\(yιs−y¯\)=Σy\.\\sum\_\{s=1\}^\{N\}\\alpha\_\{s\}\(y\_\{\\iota\_\{s\}\}\-\\bar\{y\}\)\\otimes\(y\_\{\\iota\_\{s\}\}\-\\bar\{y\}\)=\\Sigma\_\{y\}\.Thus

𝔼⟨g,∑j=1S\(p^j−pj\)yj⟩2≤209​m⟨g,Σyg⟩\.\\operatorname\{\\mathbb\{E\}\}\\left\\langle g,\\sum\_\{j=1\}^\{S\}\(\\hat\{p\}\_\{j\}\-p\_\{j\}\)y\_\{j\}\\right\\rangle^\{2\}\\leq\\frac\{20\}\{9m\}\\langle g,\\Sigma\_\{y\}g\\rangle\.Taking traces yields the claim\. ∎

### A\.3Proof of[Theorem˜A\.1](https://arxiv.org/html/2607.01520#A1.Thmtheorem1)

[Theorem˜A\.1](https://arxiv.org/html/2607.01520#A1.Thmtheorem1)now follows by runningProtectedSparsifyon the measureμ\\muand subspaceUU, and taking one random realization on which the desired bound holds\.

###### Proof\.

Apply[Proposition˜A\.3](https://arxiv.org/html/2607.01520#A1.Thmtheorem3)to\(μ,U,m\)\(\\mu,U,m\), and letμ^\\hat\{\\mu\}denote the random output\. Sinceμ\\muis centered,

η≔∫ℋx​d​μ^=∫ℋx​d​\(μ^−μ\)\.\\eta\\coloneqq\\int\_\{\\mathcal\{H\}\}x\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{\\mu\}=\\int\_\{\\mathcal\{H\}\}x\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{\\mu\}\-\\mu\)\.Hence,[Proposition˜A\.3](https://arxiv.org/html/2607.01520#A1.Thmtheorem3)gives a randomμ^\\hat\{\\mu\}such that

1. \(i\)μ^∈𝒫m\+2​d\+2​\(supp⁡\(μ\)\)\\hat\{\\mu\}\\in\\mathcal\{P\}\_\{m\+2d\+2\}\(\\operatorname\{supp\}\(\\mu\)\)almost surely;
2. \(ii\)∫ℋx​d​μ^∈U⟂\\int\_\{\\mathcal\{H\}\}x\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{\\mu\}\\in U^\{\\perp\}almost surely; and
3. \(iii\)𝔼⁡\[η⊗η\]⪯209​m​\(I−ΠU\)​Σ​\(I−ΠU\)\\operatorname\{\\mathbb\{E\}\}\[\\eta\\otimes\\eta\]\\preceq\\frac\{20\}\{9m\}\(I\-\\Pi\_\{U\}\)\\Sigma\(I\-\\Pi\_\{U\}\)\.

Moreover, taking traces in[\(iii\)](https://arxiv.org/html/2607.01520#A1.I3.i3)gives

𝔼⁡‖∫ℋx​d​μ^‖ℋ2=𝔼⁡‖η‖ℋ2≤209​m​tr⁡\(\(I−ΠU\)​Σ​\(I−ΠU\)\)=209​m​tr⁡\(\(I−ΠU\)​Σ\)\.\\operatorname\{\\mathbb\{E\}\}\\left\\\|\\int\_\{\\mathcal\{H\}\}x\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{\\mu\}\\right\\\|\_\{\\mathcal\{H\}\}^\{2\}=\\operatorname\{\\mathbb\{E\}\}\\\|\\eta\\\|\_\{\\mathcal\{H\}\}^\{2\}\\leq\\frac\{20\}\{9m\}\\operatorname\{tr\}\\bigl\(\(I\-\\Pi\_\{U\}\)\\Sigma\(I\-\\Pi\_\{U\}\)\\bigr\)=\\frac\{20\}\{9m\}\\operatorname\{tr\}\\bigl\(\(I\-\\Pi\_\{U\}\)\\Sigma\\bigr\)\.Therefore some realization ofμ^\\hat\{\\mu\}satisfies the same bound\. ∎

## Appendix BMinimax rates

This appendix proves our minimax rates for KV compression in the query\-aware and query\-agnostic settings\.[Section˜B\.1](https://arxiv.org/html/2607.01520#A2.SS1)proves upper bounds and[Section˜B\.2](https://arxiv.org/html/2607.01520#A2.SS2)proves the matching lower bounds\.

##### Notation\.

We briefly recall the relevant notation from the main text\. Given a context measureP∈𝒫fin​\(𝒳\)P\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\), we write softmax attention as

Att⁡\(q\|P\)=∫𝒳aP​\(q,k\)​v​d​P​\(k,v\),ak​\(q∣P\)=κ​\(q,k\)∫𝒳κ​\(q,k′\)​d​P​\(k′,v′\)\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)=\\int\_\{\\mathcal\{X\}\}a\_\{P\}\(q,k\)\\,v\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\),\\qquad a\_\{k\}\(q\\mid P\)=\\frac\{\\kappa\(q,k\)\}\{\\int\_\{\\mathcal\{X\}\}\\kappa\(q,k^\{\\prime\}\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k^\{\\prime\},v^\{\\prime\}\)\}and study the mean squared error of compressed context measuresP^∈𝒫K​\(𝒳\)\\hat\{P\}\\in\\mathcal\{P\}\_\{K\}\(\\mathcal\{X\}\)

ℰP,ν​\(P^\)=𝔼q∼ν⁡‖Att⁡\(q∣P\)−Att⁡\(q∣P^\)‖22\\mathcal\{E\}\_\{P,\\nu\}\\bigl\(\\hat\{P\}\\bigr\)=\\operatorname\{\\mathbb\{E\}\}\_\{q\\sim\\nu\}\\\!\\left\\\|\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\-\\operatorname\{Att\}\\bigl\(q\\,\|\\,\\hat\{P\}\\bigr\)\\right\\\|\_\{2\}^\{2\}under a fixed query distributionν∈𝒫​\(𝒬\)\\nu\\in\\mathcal\{P\}\(\\mathcal\{Q\}\)\. The response profile of a key–value pair\(k,v\)∈𝒳\(k,v\)\\in\\mathcal\{X\}underPPis

ΓP​\(k,v\):𝒬→ℝdv⊕ℝ,q↦\(ak​\(q∣P\)​\(v−Att⁡\(q∣P\)\),V​\(ak​\(q∣P\)−1\)\),\\Gamma\_\{P\}\(k,v\):\\mathcal\{Q\}\\to\\mathbb\{R\}^\{d\_\{v\}\}\\oplus\\mathbb\{R\},\\qquad q\\mapsto\\Bigl\(a\_\{k\}\(q\\mid P\)\\bigl\(v\-\\operatorname\{Att\}\(q\\mid P\)\\bigr\),\\;V\\bigl\(a\_\{k\}\(q\\mid P\)\-1\\bigr\)\\Bigr\),with the corresponding covariance operator

ΣP,ν=∫𝒳ΓP​\(k,v\)⊗ΓP​\(k,v\)​d​P​\(k,v\),\\Sigma\_\{P,\\nu\}=\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\}\(k,v\)\\otimes\\Gamma\_\{P\}\(k,v\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\),whereΓP​\(k,v\)⊗ΓP​\(k,v\)\\Gamma\_\{P\}\(k,v\)\\otimes\\Gamma\_\{P\}\(k,v\)is understood as the rank\-one operator

h↦⟨ΓP​\(k,v\),h⟩ℋν⋅ΓP​\(k,v\)h\\mapsto\\left\\langle\\Gamma\_\{P\}\(k,v\),h\\right\\rangle\_\{\\mathcal\{H\}\_\{\\nu\}\}\\cdot\\Gamma\_\{P\}\(k,v\)andℋν=L2​\(ν,ℝdv⊕ℝ\)\\mathcal\{H\}\_\{\\nu\}=L\_\{2\}\(\\nu,\\mathbb\{R\}^\{d\_\{v\}\}\\oplus\\mathbb\{R\}\)\.

### B\.1Upper bounds

We derive upper bounds on the minimax compression risk in the query\-aware and query\-agnostic settings from a shared Hilbert space representation\. For fixed\(P,ν\)\(P,\\nu\), the response profile mapΓP\\Gamma\_\{P\}sends each token\(k,v\)\(k,v\)to an element of the Hilbert spaceℋν\\mathcal\{H\}\_\{\\nu\}, and thereby pushes the context measurePPforward to a finitely supported probability measure onℋν\\mathcal\{H\}\_\{\\nu\}\. This measure is centered and has covarianceΣP,ν\\Sigma\_\{P,\\nu\}, and the attention error of a compressed context is controlled by the squared norm of the corresponding barycenter inℋν\\mathcal\{H\}\_\{\\nu\}\. The query\-aware and query\-agnostic regimes differ only in how this barycenter is approximated: in the aware regime, we may choose a sparse approximation after seeingν\\nu, whereas in the agnostic regime we must rely on a query\-independent approximation\. Both regimes permit upper bounds on the sparse barycenter approximation problem, which we then translate back to bounds on the KV compression problem\.

#### B\.1\.1Reduction to barycenter sparsification

Recall the surrogate error definition

ℰ~μ​\(μ^\)=‖∫ℋg​d​μ^​\(g\)‖ℋ2\\widetilde\{\\mathcal\{E\}\}\_\{\\mu\}\(\\hat\{\\mu\}\)=\\left\\\|\\int\_\{\\mathcal\{H\}\}g\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{\\mu\}\(g\)\\right\\\|\_\{\\mathcal\{H\}\}^\{2\}between measuresμ\\muandμ^\\hat\{\\mu\}on a Hilbert spaceℋν\\mathcal\{H\}\_\{\\nu\}\. Our goal is to derive upper bounds on the compression errorℰP,ν​\(P^\)\\mathcal\{E\}\_\{P,\\nu\}\(\\hat\{P\}\)from upper bounds onℰ~​\(μ^\)\\widetilde\{\\mathcal\{E\}\}\(\\hat\{\\mu\}\)for the pushforwardμP,ν=P∘ΓP−1\\mu\_\{P,\\nu\}=P\\circ\\Gamma\_\{P\}^\{\-1\}by pulling back a sparse approximationμ^∈𝒫K​\(supp⁡\(μP,ν\)\)\\hat\{\\mu\}\\in\\mathcal\{P\}\_\{K\}\(\\operatorname\{supp\}\(\\mu\_\{P,\\nu\}\)\)ofμP,ν\\mu\_\{P,\\nu\}to a sparse approximationP^∈𝒫K​\(𝒳\)\\hat\{P\}\\in\\mathcal\{P\}\_\{K\}\(\\mathcal\{X\}\)ofPP\.[Lemma˜B\.1](https://arxiv.org/html/2607.01520#A2.Thmtheorem1)first shows that upper bounds onℰ~\\widetilde\{\\mathcal\{E\}\}indeed survive this pullback, and[Section˜4\.1](https://arxiv.org/html/2607.01520#S4.SS1)packages the full reduction principle\.

###### Lemma B\.1\.

LetP∈𝒫fin​\(𝒳\)P\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\),ν∈𝒫​\(𝒬\)\\nu\\in\\mathcal\{P\}\(\\mathcal\{Q\}\), and letP^∈𝒫​\(supp⁡\(P\)\)\\hat\{P\}\\in\\mathcal\{P\}\(\\operatorname\{supp\}\(P\)\)\. Then,

ℰP,ν​\(P^\)≤16​‖∫𝒳ΓP​\(k,v\)​d​P^​\(k,v\)‖ℋν2\.\\mathcal\{E\}\_\{P,\\nu\}\\bigl\(\\hat\{P\}\\bigr\)\\leq 16\\left\\\|\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\}\(k,v\)\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\(k,v\)\\right\\\|\_\{\\mathcal\{H\}\_\{\\nu\}\}^\{2\}\.

###### Proof of[Lemma˜B\.1](https://arxiv.org/html/2607.01520#A2.Thmtheorem1)\.

Fixq∈supp⁡\(ν\)q\\in\\operatorname\{supp\}\(\\nu\), and abbreviate

mP​\(q\)≔∫𝒳κ​\(q,k\)​d​P​\(k,v\),mP^​\(q\)≔∫𝒳κ​\(q,k\)​d​P^​\(k,v\)\.m\_\{P\}\(q\)\\coloneqq\\int\_\{\\mathcal\{X\}\}\\kappa\(q,k\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\),\\qquad m\_\{\\hat\{P\}\}\(q\)\\coloneqq\\int\_\{\\mathcal\{X\}\}\\kappa\(q,k\)\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\(k,v\)\.Define

U​\(q\)≔∫𝒳ak​\(q∣P\)​\(v−Att⁡\(q\|P\)\)​d​P^​\(k,v\),U\(q\)\\coloneqq\\int\_\{\\mathcal\{X\}\}a\_\{k\}\(q\\mid P\)\\bigl\(v\-\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\\bigr\)\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\(k,v\),and

t​\(q\)≔∫𝒳\(ak​\(q∣P\)−1\)​d​P^​\(k,v\)\.t\(q\)\\coloneqq\\int\_\{\\mathcal\{X\}\}\\bigl\(a\_\{k\}\(q\\mid P\)\-1\\bigr\)\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\(k,v\)\.SinceP^\\hat\{P\}is a probability measure,

1\+t​\(q\)=∫𝒳ak​\(q∣P\)​d​P^​\(k,v\)=mP^​\(q\)mP​\(q\)≥0\.1\+t\(q\)=\\int\_\{\\mathcal\{X\}\}a\_\{k\}\(q\\mid P\)\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\(k,v\)=\\frac\{m\_\{\\hat\{P\}\}\(q\)\}\{m\_\{P\}\(q\)\}\\geq 0\.Also,

Att⁡\(q\|P^\)\\displaystyle\\operatorname\{Att\}\\bigl\(q\\,\|\\,\\hat\{P\}\\bigr\)=∫𝒳κ​\(q,k\)​v​d​P^​\(k,v\)mP^​\(q\)=∫𝒳ak​\(q∣P\)​v​d​P^​\(k,v\)1\+t​\(q\)\\displaystyle=\\frac\{\\int\_\{\\mathcal\{X\}\}\\kappa\(q,k\)v\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\(k,v\)\}\{m\_\{\\hat\{P\}\}\(q\)\}=\\frac\{\\int\_\{\\mathcal\{X\}\}a\_\{k\}\(q\\mid P\)v\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\(k,v\)\}\{1\+t\(q\)\}=Att⁡\(q\|P\)\+U​\(q\)\+t​\(q\)​Att⁡\(q\|P\)1\+t​\(q\)=Att⁡\(q\|P\)\+U​\(q\)1\+t​\(q\)\.\\displaystyle=\\frac\{\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\+U\(q\)\+t\(q\)\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\}\{1\+t\(q\)\}=\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\+\\frac\{U\(q\)\}\{1\+t\(q\)\}\.Hence

Att⁡\(q\|P^\)−Att⁡\(q\|P\)=U​\(q\)1\+t​\(q\)\.\\operatorname\{Att\}\\bigl\(q\\,\|\\,\\hat\{P\}\\bigr\)\-\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)=\\frac\{U\(q\)\}\{1\+t\(q\)\}\.On the other hand,

∫𝒳ΓP​\(k,v\)​\(q\)​d​P^​\(k,v\)=\(U​\(q\),V​t​\(q\)\)∈ℝdv\+1\.\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\}\(k,v\)\(q\)\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\(k,v\)=\\bigl\(U\(q\),Vt\(q\)\\bigr\)\\in\\mathbb\{R\}^\{d\_\{v\}\+1\}\.
Since bothAtt⁡\(q\|P\)\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)andAtt⁡\(q\|P^\)\\operatorname\{Att\}\\bigl\(q\\,\|\\,\\hat\{P\}\\bigr\)are convex combinations of values, we have

‖Att⁡\(q∣P^\)−Att⁡\(q∣P\)‖2≤2​V\.\\left\\\|\\operatorname\{Att\}\\bigl\(q\\,\|\\,\\hat\{P\}\\bigr\)\-\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\\right\\\|\_\{2\}\\leq 2V\.If1\+t​\(q\)≥121\+t\(q\)\\geq\\frac\{1\}\{2\}, then

‖Att⁡\(q∣P^\)−Att⁡\(q∣P\)‖22=‖U​\(q\)1\+t​\(q\)‖22≤4​‖U​\(q\)‖22\.\\left\\\|\\operatorname\{Att\}\\bigl\(q\\,\|\\,\\hat\{P\}\\bigr\)\-\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\\right\\\|\_\{2\}^\{2\}=\\left\\\|\\frac\{U\(q\)\}\{1\+t\(q\)\}\\right\\\|\_\{2\}^\{2\}\\leq 4\\\|U\(q\)\\\|\_\{2\}^\{2\}\.If1\+t​\(q\)<121\+t\(q\)<\\frac\{1\}\{2\}, thent​\(q\)2≥14t\(q\)^\{2\}\\geq\\frac\{1\}\{4\}, and therefore

‖Att⁡\(q∣P^\)−Att⁡\(q∣P\)‖22≤4​V2≤16​V2​t​\(q\)2\.\\left\\\|\\operatorname\{Att\}\\bigl\(q\\,\|\\,\\hat\{P\}\\bigr\)\-\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\\right\\\|\_\{2\}^\{2\}\\leq 4V^\{2\}\\leq 16V^\{2\}t\(q\)^\{2\}\.Thus, in all cases,

‖Att⁡\(q∣P^\)−Att⁡\(q∣P\)‖22≤16​\(‖U​\(q\)‖22\+V2​t​\(q\)2\)=16​‖∫𝒳ΓP​\(k,v\)​\(q\)​d​P^​\(k,v\)‖ℝdv⊕ℝ2\.\\left\\\|\\operatorname\{Att\}\\bigl\(q\\,\|\\,\\hat\{P\}\\bigr\)\-\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\\right\\\|\_\{2\}^\{2\}\\leq 16\\bigl\(\\\|U\(q\)\\\|\_\{2\}^\{2\}\+V^\{2\}t\(q\)^\{2\}\\bigr\)=16\\left\\\|\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\}\(k,v\)\(q\)\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\(k,v\)\\right\\\|\_\{\\mathbb\{R\}^\{d\_\{v\}\}\\oplus\\mathbb\{R\}\}^\{2\}\.Averaging overq∼νq\\sim\\nuthen yields the claim\. ∎

###### Proof of[Section˜4\.1](https://arxiv.org/html/2607.01520#S4.SS1)\.

We first show thatμP,ν\\mu\_\{P,\\nu\}is centered\. For everyq∈𝒬q\\in\\mathcal\{Q\},

∫𝒳ΓP​\(k,v\)​\(q\)​d​P​\(k,v\)\\displaystyle\\phantom\{=\}\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\}\(k,v\)\(q\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\)=\(∫𝒳ak​\(q∣P\)​\(v−Att⁡\(q\|P\)\)​d​P​\(k,v\),V​\(∫𝒳ak​\(q∣P\)​d​P​\(k,v\)−1\)\)\\displaystyle=\\left\(\\int\_\{\\mathcal\{X\}\}a\_\{k\}\(q\\mid P\)\\bigl\(v\-\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\\bigr\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\),\\;V\\left\(\\int\_\{\\mathcal\{X\}\}a\_\{k\}\(q\\mid P\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\)\-1\\right\)\\right\)=\(0,0\),\\displaystyle=\(0,0\),since

∫𝒳ak​\(q∣P\)​d​P​\(k,v\)=1and∫𝒳ak​\(q∣P\)​v​d​P​\(k,v\)=Att⁡\(q\|P\)\.\\int\_\{\\mathcal\{X\}\}a\_\{k\}\(q\\mid P\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\)=1\\qquad\\text\{and\}\\qquad\\int\_\{\\mathcal\{X\}\}a\_\{k\}\(q\\mid P\)\\,v\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\)=\\operatorname\{Att\}\\bigl\(q\\,\|\\,P\\bigr\)\.Thus,∫ℋνg​d​μP,ν​\(g\)=0\\int\_\{\\mathcal\{H\}\_\{\\nu\}\}g\\mathop\{\}\\\!\\mathrm\{d\}\\mu\_\{P,\\nu\}\(g\)=0\.

For the covariance identity, we have

∫ℋνg⊗g​d​μP,ν​\(g\)=∫𝒳ΓP​\(k,v\)⊗ΓP​\(k,v\)​d​P​\(k,v\)=ΣP,ν\\displaystyle\\int\_\{\\mathcal\{H\}\_\{\\nu\}\}g\\otimes g\\,\\mathop\{\}\\\!\\mathrm\{d\}\\mu\_\{P,\\nu\}\(g\)=\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\}\(k,v\)\\otimes\\Gamma\_\{P\}\(k,v\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\(k,v\)=\\Sigma\_\{P,\\nu\}by the definition ofΣP,ν\\Sigma\_\{P,\\nu\}\.

Now, letμ^∈𝒫K​\(supp⁡\(μP,ν\)\)\\hat\{\\mu\}\\in\\mathcal\{P\}\_\{K\}\(\\operatorname\{supp\}\(\\mu\_\{P,\\nu\}\)\)\. Write

μ^=∑ℓ=1mβℓ​δgℓ,m≤K,\\hat\{\\mu\}=\\sum\_\{\\ell=1\}^\{m\}\\beta\_\{\\ell\}\\delta\_\{g\_\{\\ell\}\},\\qquad m\\leq K,withgℓ∈supp⁡\(μP,ν\)g\_\{\\ell\}\\in\\operatorname\{supp\}\(\\mu\_\{P,\\nu\}\),βℓ≥0\\beta\_\{\\ell\}\\geq 0, and∑ℓ=1mβℓ=1\\sum\_\{\\ell=1\}^\{m\}\\beta\_\{\\ell\}=1\.

For eachℓ∈\[m\]\\ell\\in\[m\], choose one atom\(k^ℓ,v^ℓ\)∈supp⁡\(P\)\(\\hat\{k\}\_\{\\ell\},\\hat\{v\}\_\{\\ell\}\)\\in\\operatorname\{supp\}\(P\)such that

ΓP​\(k^ℓ,v^ℓ\)=gℓ\.\\Gamma\_\{P\}\(\\hat\{k\}\_\{\\ell\},\\hat\{v\}\_\{\\ell\}\)=g\_\{\\ell\}\.This is possible becausegℓ∈supp⁡\(P∘ΓP−1\)g\_\{\\ell\}\\in\\operatorname\{supp\}\(P\\circ\\Gamma\_\{P\}^\{\-1\}\)\. Define

P^≔∑ℓ=1mβℓ​δ\(k^ℓ,v^ℓ\)\.\\hat\{P\}\\coloneqq\\sum\_\{\\ell=1\}^\{m\}\\beta\_\{\\ell\}\\delta\_\{\(\\hat\{k\}\_\{\\ell\},\\hat\{v\}\_\{\\ell\}\)\}\.Then,P^∈𝒫K​\(supp⁡\(P\)\)\\hat\{P\}\\in\\mathcal\{P\}\_\{K\}\(\\operatorname\{supp\}\(P\)\)depends onμ^\\hat\{\\mu\}but not onν\\nu, and

∫𝒳ΓP​\(k,v\)​d​P^​\(k,v\)=∑ℓ=1mβℓ​ΓP​\(k^ℓ,v^ℓ\)=∑ℓ=1mβℓ​gℓ=∫ℋνg​d​μ^​\(g\)\.\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\}\(k,v\)\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\(k,v\)=\\sum\_\{\\ell=1\}^\{m\}\\beta\_\{\\ell\}\\Gamma\_\{P\}\(\\hat\{k\}\_\{\\ell\},\\hat\{v\}\_\{\\ell\}\)=\\sum\_\{\\ell=1\}^\{m\}\\beta\_\{\\ell\}g\_\{\\ell\}=\\int\_\{\\mathcal\{H\}\_\{\\nu\}\}g\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{\\mu\}\(g\)\.
Therefore, by[Lemma˜B\.1](https://arxiv.org/html/2607.01520#A2.Thmtheorem1),

ℰP,ν​\(P^\)≤16​‖∫𝒳ΓP​\(k,v\)​d​P^​\(k,v\)‖ℋν2=16​‖∫ℋνg​d​μ^​\(g\)‖ℋν2\.\\mathcal\{E\}\_\{P,\\nu\}\\bigl\(\\hat\{P\}\\bigr\)\\leq 16\\left\\\|\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\}\(k,v\)\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\(k,v\)\\right\\\|\_\{\\mathcal\{H\}\_\{\\nu\}\}^\{2\}=16\\left\\\|\\int\_\{\\mathcal\{H\}\_\{\\nu\}\}g\\,\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{\\mu\}\(g\)\\right\\\|\_\{\\mathcal\{H\}\_\{\\nu\}\}^\{2\}\.Sinceμ^∈𝒫K​\(supp⁡\(μP,ν\)\)\\hat\{\\mu\}\\in\\mathcal\{P\}\_\{K\}\(\\operatorname\{supp\}\(\\mu\_\{P,\\nu\}\)\)was arbitrary, that proves the claim\. ∎

#### B\.1\.2Query\-aware upper bound

In the query\-aware regime, the compressor may depend onν\\nu, and therefore on the geometry ofℋν\\mathcal\{H\}\_\{\\nu\}\. After the reduction above, the problem is exactly to approximate the zero barycenter of the pushed\-forward measureμP,ν\\mu\_\{P,\\nu\}by aKK\-atomic measure onsupp⁡\(μP,ν\)\\operatorname\{supp\}\(\\mu\_\{P,\\nu\}\)\. This is exactly the setting of[Section˜4\.2\.1](https://arxiv.org/html/2607.01520#S4.SS2.SSS1), which provides a barycenter bound in terms of the spectral tail of its covariance, which here isΣP,ν\\Sigma\_\{P,\\nu\}\. Pulling the resulting sparse measure back tosupp⁡\(P\)\\operatorname\{supp\}\(P\)then gives the desired compression bound for KV compression\.

\\thmsparsehilbertapproximationfixedk

\*

###### Proof\.

Chooser≔⌊K/3⌋r\\coloneqq\\lfloor K/3\\rfloorandM≔K−2​r−2M\\coloneqq K\-2r\-2\. Further, chooseUUto be the span of the topr∗≔min⁡\{r,rank⁡\(Σ\)\}r\_\{\\ast\}\\coloneqq\\min\\\{r,\\operatorname\{rank\}\(\\Sigma\)\\\}eigenvectors ofΣ\\Sigma\. Then,dim\(U\)=r∗≤r\\dim\(U\)=r\_\{\\ast\}\\leq r\. Invoking[Theorem˜A\.1](https://arxiv.org/html/2607.01520#A1.Thmtheorem1)with subspaceUUand support size parameterMMyieldsμ^∈𝒫​\(supp⁡\(μ\)\)\\hat\{\\mu\}\\in\\mathcal\{P\}\(\\operatorname\{supp\}\(\\mu\)\)withsupp⁡\(μ^\)≤M\+2​r\+2≤K\\operatorname\{supp\}\(\\hat\{\\mu\}\)\\leq M\+2r\+2\\leq Kand

‖∫ℋx​𝑑μ^​\(x\)‖ℋ2≤209​M​tr⁡\(\(I−ΠU\)​Σ\)=209​M​tailr⁡\(Σ\)\.\\left\\\|\\int\_\{\\mathcal\{H\}\}x\\,d\\hat\{\\mu\}\(x\)\\right\\\|\_\{\\mathcal\{H\}\}^\{2\}\\leq\\frac\{20\}\{9M\}\\operatorname\{tr\}\\bigl\(\(I\-\\Pi\_\{U\}\)\\Sigma\\bigr\)=\\frac\{20\}\{9M\}\\operatorname\{tail\}\_\{r\}\(\\Sigma\)\.

###### Proof\.

This is immediate from[Section˜4\.1](https://arxiv.org/html/2607.01520#S4.SS1)and[Section˜4\.2\.1](https://arxiv.org/html/2607.01520#S4.SS2.SSS1)\. ∎

#### B\.1\.3Query\-agnostic upper bound

In the query\-agnostic regime, the compressor cannot adapt toν\\nu, so the sparse approximation argument from the aware case does not apply\. A natural query\-independent alternative is the empirical measure ofKKi\.i\.d\. samples fromPP\. Under the Hilbert embedding, this corresponds to averagingKKindependent copies of the centered random response profileG=ΓP​\(X\)G=\\Gamma\_\{P\}\(X\)\. The mean remains zero, and the mean squared norm of the empirical average is of ordertr⁡\(ΣP,ν\)/K\\operatorname\{tr\}\(\\Sigma\_\{P,\\nu\}\)/K\. Using[Lemma˜B\.1](https://arxiv.org/html/2607.01520#A2.Thmtheorem1)from the reduction, this yields the trace upper bound\.

###### Lemma B\.2\.

Letℋ\\mathcal\{H\}be a real Hilbert space, and letμ∈𝒫fin​\(ℋ\)\\mu\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{H\}\)be centered, with covariance operatorΣ\\Sigma\. LetG1,…,GK∼iidμG\_\{1\},\\dots,G\_\{K\}\\stackrel\{\{\\scriptstyle\\mathrm\{iid\}\}\}\{\{\\sim\}\}\\mu, and define

μ^K≔1K​∑i=1KδGi\.\\hat\{\\mu\}\_\{K\}\\coloneqq\\frac\{1\}\{K\}\\sum\_\{i=1\}^\{K\}\\delta\_\{G\_\{i\}\}\.Then,

𝔼⁡\[ℰ~μ​\(μ^K\)\]=tr⁡\(Σ\)K\.\\operatorname\{\\mathbb\{E\}\}\\left\[\\widetilde\{\\mathcal\{E\}\}\_\{\\mu\}\(\\hat\{\\mu\}\_\{K\}\)\\right\]=\\frac\{\\operatorname\{tr\}\(\\Sigma\)\}\{K\}\.

###### Proof\.

Sinceμ\\muis centered, theGiG\_\{i\}are independent and mean zero\. Hence

𝔼⁡\[ℰ~μ​\(μ^K\)\]=𝔼⁡‖1K​∑i=1KGi‖ℋ2=1K2​∑i=1K𝔼⁡‖Gi‖ℋ2=tr⁡\(Σ\)K\.\\operatorname\{\\mathbb\{E\}\}\\left\[\\widetilde\{\\mathcal\{E\}\}\_\{\\mu\}\(\\hat\{\\mu\}\_\{K\}\)\\right\]=\\operatorname\{\\mathbb\{E\}\}\\left\\\|\\frac\{1\}\{K\}\\sum\_\{i=1\}^\{K\}G\_\{i\}\\right\\\|\_\{\\mathcal\{H\}\}^\{2\}=\\frac\{1\}\{K^\{2\}\}\\sum\_\{i=1\}^\{K\}\\operatorname\{\\mathbb\{E\}\}\\\|G\_\{i\}\\\|\_\{\\mathcal\{H\}\}^\{2\}=\\frac\{\\operatorname\{tr\}\(\\Sigma\)\}\{K\}\.∎

###### Proof\.

LetP∈𝒫fin​\(𝒳\)P\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)andν∈𝒫​\(𝒬\)\\nu\\in\\mathcal\{P\}\(\\mathcal\{Q\}\)\. LetX1,…,XK∼iidPX\_\{1\},\\dots,X\_\{K\}\\stackrel\{\{\\scriptstyle\\mathrm\{iid\}\}\}\{\{\\sim\}\}P, set

Gi=ΓP​\(Xi\),μ^K=1K​∑i=1KδGi\.G\_\{i\}=\\Gamma\_\{P\}\(X\_\{i\}\),\\qquad\\hat\{\\mu\}\_\{K\}=\\frac\{1\}\{K\}\\sum\_\{i=1\}^\{K\}\\delta\_\{G\_\{i\}\}\.By[Section˜4\.1](https://arxiv.org/html/2607.01520#S4.SS1),μ^K\\hat\{\\mu\}\_\{K\}can be pulled back to the empirical context measure

P^=1K​∑i=1KδXi\\hat\{P\}=\\frac\{1\}\{K\}\\sum\_\{i=1\}^\{K\}\\delta\_\{X\_\{i\}\}with

ℰP,ν​\(P^\)≲ℰ~μP,ν​\(μ^K\)\.\\mathcal\{E\}\_\{P,\\nu\}\\bigl\(\\hat\{P\}\\bigr\)\\lesssim\\widetilde\{\\mathcal\{E\}\}\_\{\\mu\_\{P,\\nu\}\}\(\\hat\{\\mu\}\_\{K\}\)\.Taking expectations and applying[Lemma˜B\.2](https://arxiv.org/html/2607.01520#A2.Thmtheorem2)withμ=μP,ν\\mu=\\mu\_\{P,\\nu\}andΣ=ΣP,ν\\Sigma=\\Sigma\_\{P,\\nu\}gives

𝔼⁡\[ℰP,ν​\(P^\)\]≲tr⁡\(ΣP,ν\)K\.\\operatorname\{\\mathbb\{E\}\}\[\\mathcal\{E\}\_\{P,\\nu\}\\bigl\(\\hat\{P\}\\bigr\)\]\\lesssim\\frac\{\\operatorname\{tr\}\(\\Sigma\_\{P,\\nu\}\)\}\{K\}\.The claim then follows because random sampling fromPPcan be implemented by a randomized query\-agnostic compressor\. ∎

### B\.2Lower bounds

We now provide lower bounds on the query\-aware and query\-agnostic minimax compression risk matching the upper bounds from[Section˜B\.1](https://arxiv.org/html/2607.01520#A2.SS1)\. Both bounds are derived from a shared, explicit construction of an instance of a contextPPand a query distributionν\\nuthat are realizable from actual keys, queries, and values under[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)\. This instance is constructed so that any significant compression of the context incurs large error over the chosen query distribution\. We then specialize this construction to the query\-aware and query\-agnostic regime by calibrating the error incurred under compression against the spectrum ofΣP,ν\\Sigma\_\{P,\\nu\}, obtaining lower bounds that faithfully match their corresponding upper bounds throughout the entire claimed range of redundancy levels captured bytr⁡\(ΣP,ν\)\\operatorname\{tr\}\(\\Sigma\_\{P,\\nu\}\)andtailK/3⁡\(ΣP,ν\)\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\_\{P,\\nu\}\), respectively\.

Throughout this section, fix universal constants

ρ0≔99100andγ0≔910\.\\textstyle\\rho\_\{0\}\\coloneqq\\frac\{99\}\{100\}\\qquad\\text\{and\}\\qquad\\gamma\_\{0\}\\coloneqq\\frac\{9\}\{10\}\.
#### B\.2\.1Hard instance construction

We begin by constructing a pairP,νP,\\nuthat is impossible to compress significantly without incurring large error\. The main idea is to use keys that can be sharply distinguished from each other, and have them carry almost orthogonal values\. We call such an instance a*lookup instance*because querying a key of the context sharply retrieves the associated value\.[Definition˜B\.3](https://arxiv.org/html/2607.01520#A2.Thmtheorem3)formalizes the properties of a lookup instance formally, and[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4)shows that such instances can indeed be constructed from actual keys, queries, and values under[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)\. In particular,[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)provides enough ambient dimension to choose well\-separated keys and values from Hamming packings, and enough key/query norm budget to address the keys sharply through softmax attention\.

###### Definition B\.3\(Lookup instance\)\.

Fix an even integerN≥2N\\geq 2and let

PN≔1N​∑i=1Nδ\(ki,vi\),νN≔1N​∑i=1Nδki\.\\textstyle P\_\{N\}\\coloneqq\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\delta\_\{\(k\_\{i\},v\_\{i\}\)\},\\qquad\\nu\_\{N\}\\coloneqq\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\delta\_\{k\_\{i\}\}\.We call the pair\(PN,νN\)\(P\_\{N\},\\nu\_\{N\}\)a*lookup instance of sizeNN*if

1. \(membership\)ki∈𝒦∩𝒬k\_\{i\}\\in\\mathcal\{K\}\\cap\\mathcal\{Q\}, andvi∈𝒱v\_\{i\}\\in\\mathcal\{V\}for alli∈\[N\]i\\in\[N\];
2. \(centering\)1N​∑i=1Nvi=0;\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}v\_\{i\}=0;
3. \(boundary\)‖vi‖2=ρ0​V\\\|v\_\{i\}\\\|\_\{2\}=\\rho\_\{0\}Vfor alli∈\[N\]i\\in\[N\];
4. \(separation\)⟨vi,vj⟩≤γ0​ρ02​V2\\left\\langle v\_\{i\},v\_\{j\}\\right\\rangle\\leq\\gamma\_\{0\}\\rho\_\{0\}^\{2\}V^\{2\}for alli≠ji\\neq j;
5. \(sharpness\)1N​∑j≠iakj​\(ki∣PN\)≤N−3/2\\frac\{1\}\{N\}\\sum\_\{j\\neq i\}a\_\{k\_\{j\}\}\(k\_\{i\}\\mid P\_\{N\}\)\\leq N^\{\-3/2\}for alli∈\[N\]i\\in\[N\]\.

###### Lemma B\.4\.

LetK≥50K\\geq 50, setN≔2​KN\\coloneqq 2K, and suppose[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)holds\. Then, there exists a lookup instance\(PN,νN\)∈𝒫N⋆​\(𝒳\)×𝒫​\(𝒬\)\\bigl\(P\_\{N\},\\nu\_\{N\}\\bigr\)\\in\\mathcal\{P\}\_\{N\}^\{\\star\}\(\\mathcal\{X\}\)\\times\\mathcal\{P\}\(\\mathcal\{Q\}\)of sizeNNsuch that

1. \(i\)\|supp⁡\(νN\)\|=\|supp⁡\(PN\)\|=N\|\\operatorname\{supp\}\(\\nu\_\{N\}\)\|=\|\\operatorname\{supp\}\(P\_\{N\}\)\|=N;
2. \(ii\)dimspan⁡\{v1,…,vN\}≤K/6\\dim\\operatorname\{span\}\\\{v\_\{1\},\\ldots,v\_\{N\}\\\}\\leq K/6; and
3. \(iii\)‖k‖2=5​dk1/4​log⁡N\\\|k\\\|\_\{2\}=5d\_\{k\}^\{1/4\}\\sqrt\{\\log N\}for allk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\)\.

###### Proof\.

Forr≥1r\\geq 1, set

Br≔∑ℓ=0⌊r/20⌋\(rℓ\)\.B\_\{r\}\\coloneqq\\sum\_\{\\ell=0\}^\{\\lfloor r/20\\rfloor\}\\binom\{r\}\{\\ell\}\.We first construct a Hamming packing\. Greedily select words fromx∈\{±1\}rx\\in\\\{\\pm 1\\\}^\{r\}\. After selectingxx, delete the Hamming balls of radius⌊r/20⌋\\lfloor r/20\\rflooraroundxxand around−x\-x\. Each step deletes at most2​Br2B\_\{r\}words, so the set𝖢r⊆\{±1\}r\\mathsf\{C\}\_\{r\}\\subseteq\\\{\\pm 1\\\}^\{r\}selected by the greedy procedure satisfies

\|𝖢r\|≥2r2​Br\.\|\\mathsf\{C\}\_\{r\}\|\\geq\\frac\{2^\{r\}\}\{2B\_\{r\}\}\.Denote bydHd\_\{H\}the Hamming distance\. Ifx,x′∈𝖢rx,x^\{\\prime\}\\in\\mathsf\{C\}\_\{r\}are distinct, then

dH​\(x,x′\)\>⌊r/20⌋anddH​\(x,−x′\)\>⌊r/20⌋\.d\_\{H\}\(x,x^\{\\prime\}\)\>\\lfloor r/20\\rfloor\\qquad\\text\{and\}\\qquad d\_\{H\}\(x,\-x^\{\\prime\}\)\>\\lfloor r/20\\rfloor\.Since⟨x,x′⟩=r−2​dH​\(x,x′\)\\left\\langle x,x^\{\\prime\}\\right\\rangle=r\-2d\_\{H\}\(x,x^\{\\prime\}\), this implies

\|⟨x,x′⟩\|≤γ0​r\.\|\\left\\langle x,x^\{\\prime\}\\right\\rangle\|\\leq\\gamma\_\{0\}r\.
Using the entropy bound for Hamming balls, forr≥20r\\geq 20,

Br≤exp⁡\(r​\[−120​log⁡120−1920​log⁡1920\]\)≤exp⁡\(0\.2​r\)\.B\_\{r\}\\leq\\exp\\\!\\left\(r\\left\[\-\\frac\{1\}\{20\}\\log\\frac\{1\}\{20\}\-\\frac\{19\}\{20\}\\log\\frac\{19\}\{20\}\\right\]\\right\)\\leq\\exp\(0\.2r\)\.Hence, forr≥20r\\geq 20,

\|𝖢r\|≥12​exp⁡\(\(log⁡2−0\.2\)​r\)\.\|\\mathsf\{C\}\_\{r\}\|\\geq\\frac\{1\}\{2\}\\exp\(\(\\log 2\-0\.2\)r\)\.\(6\)In particular, ifr≥10​log⁡Kr\\geq 10\\log K, then

\|𝖢r\|≥12​K4\.9≥2​K,\|\\mathsf\{C\}\_\{r\}\|\\geq\\frac\{1\}\{2\}K^\{4\.9\}\\geq 2K,\(7\)where the last inequality usesK≥50K\\geq 50\. Also, ifr≥8r\\geq 8, then

\|𝖢r\|≥6​r\+5\.\|\\mathsf\{C\}\_\{r\}\|\\geq 6r\+5\.\(8\)Indeed, for8≤r<208\\leq r<20, this follows fromBr=1B\_\{r\}=1and2r−1≥6​r\+52^\{r\-1\}\\geq 6r\+5\. Forr≥20r\\geq 20,[Equation˜6](https://arxiv.org/html/2607.01520#A2.E6)gives

\|𝖢r\|≥12​exp⁡\(\(log⁡2−0\.2\)​r\)≥6​r\+5\.\|\\mathsf\{C\}\_\{r\}\|\\geq\\frac\{1\}\{2\}\\exp\(\(\\log 2\-0\.2\)r\)\\geq 6r\+5\.
LetN≔2​KN\\coloneqq 2K\. Choose distinctξ1,…,ξN∈𝖢dk\\xi\_\{1\},\\ldots,\\xi\_\{N\}\\in\\mathsf\{C\}\_\{d\_\{k\}\}, and define

Rk≔5​dk1/4​log⁡N,andki≔Rk​ξidkR\_\{k\}\\coloneqq 5d\_\{k\}^\{1/4\}\\sqrt\{\\log N\},\\qquad\\text\{and\}\\qquad k\_\{i\}\\coloneqq R\_\{k\}\\frac\{\\xi\_\{i\}\}\{\\sqrt\{d\_\{k\}\}\}fori∈\[N\]i\\in\[N\]\. Sincedk≥10​log⁡Kd\_\{k\}\\geq 10\\log K, the code𝖢dk\\mathsf\{C\}\_\{d\_\{k\}\}has at least2​K=N2K=Nwords\. Moreover,

‖ki‖2=Rk=5​dk1/4​log⁡N≤8​dk1/4​log⁡K,\\\|k\_\{i\}\\\|\_\{2\}=R\_\{k\}=5d\_\{k\}^\{1/4\}\\sqrt\{\\log N\}\\leq 8d\_\{k\}^\{1/4\}\\sqrt\{\\log K\},soki∈𝒦∩𝒬k\_\{i\}\\in\\mathcal\{K\}\\cap\\mathcal\{Q\}by[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)\.

Set

rv≔min⁡\{dv,⌊K/6⌋\}\.r\_\{v\}\\coloneqq\\min\\\{d\_\{v\},\\lfloor K/6\\rfloor\\\}\.Thenrv≥8r\_\{v\}\\geq 8, sincedv≥10​log⁡Kd\_\{v\}\\geq 10\\log KandK≥50K\\geq 50\. The code𝖢rv\\mathsf\{C\}\_\{r\_\{v\}\}has at leastKKwords\. Ifrv=dvr\_\{v\}=d\_\{v\}, this follows from[Equation˜7](https://arxiv.org/html/2607.01520#A2.E7), because thenrv≥10​log⁡Kr\_\{v\}\\geq 10\\log K\. Ifrv=⌊K/6⌋r\_\{v\}=\\lfloor K/6\\rfloor, it follows from[Equation˜8](https://arxiv.org/html/2607.01520#A2.E8), because thenrv≥8r\_\{v\}\\geq 8and6​rv\+5≥K6r\_\{v\}\+5\\geq K\. Choose distinctζ1,…,ζK∈𝖢rv\\zeta\_\{1\},\\ldots,\\zeta\_\{K\}\\in\\mathsf\{C\}\_\{r\_\{v\}\}, and letE:ℝrv→ℝdvE:\\mathbb\{R\}^\{r\_\{v\}\}\\to\\mathbb\{R\}^\{d\_\{v\}\}be the canonical coordinate embedding\. Define

uℓ≔E​ζℓrv,vℓ≔ρ0​V​uℓ,vK\+ℓ≔−ρ0​V​uℓu\_\{\\ell\}\\coloneqq E\\frac\{\\zeta\_\{\\ell\}\}\{\\sqrt\{r\_\{v\}\}\},\\qquad v\_\{\\ell\}\\coloneqq\\rho\_\{0\}Vu\_\{\\ell\},\\qquad v\_\{K\+\\ell\}\\coloneqq\-\\rho\_\{0\}Vu\_\{\\ell\}forℓ∈\[K\]\\ell\\in\[K\]\. Then,vi∈𝒱=𝖡dv​\(V\)v\_\{i\}\\in\\mathcal\{V\}=\\mathsf\{B\}\_\{d\_\{v\}\}\(V\),

1N​∑i=1Nvi=0,and‖vi‖2=ρ0​V\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}v\_\{i\}=0,\\qquad\\text\{and\}\\qquad\\\|v\_\{i\}\\\|\_\{2\}=\\rho\_\{0\}Vfor alli∈\[N\]i\\in\[N\]\. Fori≠ji\\neq j,

⟨vi,vj⟩≤γ0​ρ02​V2\.\\left\\langle v\_\{i\},v\_\{j\}\\right\\rangle\\leq\\gamma\_\{0\}\\rho\_\{0\}^\{2\}V^\{2\}\.Also,

dimspan⁡\{v1,…,vN\}≤dimim⁡\(E\)≤rv≤K/6\.\\dim\\operatorname\{span\}\\\{v\_\{1\},\\ldots,v\_\{N\}\\\}\\leq\\dim\\operatorname\{im\}\(E\)\\leq r\_\{v\}\\leq K/6\.
Define

PN≔1N​∑i=1Nδ\(ki,vi\),andνN≔1N​∑i=1Nδki\.P\_\{N\}\\coloneqq\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\delta\_\{\(k\_\{i\},v\_\{i\}\)\},\\qquad\\text\{and\}\\qquad\\nu\_\{N\}\\coloneqq\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\delta\_\{k\_\{i\}\}\.Sinceki∈𝒦∩𝒬k\_\{i\}\\in\\mathcal\{K\}\\cap\\mathcal\{Q\}and‖vi‖2≤V\\\|v\_\{i\}\\\|\_\{2\}\\leq Vfor alli∈\[N\]i\\in\[N\],PN∈𝒫N⋆​\(𝒳\)P\_\{N\}\\in\\mathcal\{P\}\_\{N\}^\{\\star\}\(\\mathcal\{X\}\)andνN∈𝒫​\(𝒬\)\\nu\_\{N\}\\in\\mathcal\{P\}\(\\mathcal\{Q\}\)\.

The keyskik\_\{i\}are distinct, so

\|supp⁡\(PN\)\|=\|supp⁡\(νN\)\|=N\.\|\\operatorname\{supp\}\(P\_\{N\}\)\|=\|\\operatorname\{supp\}\(\\nu\_\{N\}\)\|=N\.
It remains to verify sharpness\. Fori≠ji\\neq j,

⟨ki,kj⟩dk≤25​γ0​log⁡N,\\frac\{\\left\\langle k\_\{i\},k\_\{j\}\\right\\rangle\}\{\\sqrt\{d\_\{k\}\}\}\\leq 25\\gamma\_\{0\}\\log N,while

‖ki‖22dk=25​log⁡N\.\\frac\{\\\|k\_\{i\}\\\|\_\{2\}^\{2\}\}\{\\sqrt\{d\_\{k\}\}\}=25\\log N\.Hence, forj≠ij\\neq i,

akj​\(ki∣PN\)=N​exp⁡\(⟨ki,kj⟩/dk\)∑m=1Nexp⁡\(⟨ki,km⟩/dk\)≤N​exp⁡\(−25​\(1−γ0\)​log⁡N\)=N−3/2\.a\_\{k\_\{j\}\}\(k\_\{i\}\\mid P\_\{N\}\)=\\frac\{N\\exp\(\\left\\langle k\_\{i\},k\_\{j\}\\right\\rangle/\\sqrt\{d\_\{k\}\}\)\}\{\\sum\_\{m=1\}^\{N\}\\exp\(\\left\\langle k\_\{i\},k\_\{m\}\\right\\rangle/\\sqrt\{d\_\{k\}\}\)\}\\leq N\\exp\(\-25\(1\-\\gamma\_\{0\}\)\\log N\)=N^\{\-3/2\}\.Therefore,

1N​∑j≠iakj​\(ki∣PN\)≤N−3/2\\frac\{1\}\{N\}\\sum\_\{j\\neq i\}a\_\{k\_\{j\}\}\(k\_\{i\}\\mid P\_\{N\}\)\\leq N^\{\-3/2\}for alli∈\[N\]i\\in\[N\]\.

Thus,\(PN,νN\)\(P\_\{N\},\\nu\_\{N\}\)is a lookup instance of sizeNN, and it satisfies \(i\)–\(iii\)\. ∎

#### B\.2\.2Approximating lookup instances requires almost the full context

We now show that lookup instances are hard to compress\. Indeed, querying a key of such an instance retrieves the associated value almost perfectly, and the retrievable values are separated by a constant fraction of the value norm\. As a consequence, compressing lookup instances to even half their context size incurs compression error at the maximum possible scaleV2V^\{2\}\.

###### Lemma B\.5\.

Fix an even integerN≥100N\\geq 100, and let\(PN,νN\)\(P\_\{N\},\\nu\_\{N\}\)be the lookup instance of sizeNNfrom[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4)\. Then,Att⁡\(0∣PN\)=0\\operatorname\{Att\}\(0\\mid P\_\{N\}\)=0, and, for every\(k,v\)∈supp⁡\(PN\)\(k,v\)\\in\\operatorname\{supp\}\(P\_\{N\}\),

‖Att⁡\(k∣PN\)−v‖2≤2​ρ0​V​N−3/2\.\\\|\\operatorname\{Att\}\(k\\mid P\_\{N\}\)\-v\\\|\_\{2\}\\leq 2\\rho\_\{0\}VN^\{\-3/2\}\.Consequently,

‖Att⁡\(k∣PN\)‖2≤249250​Vand‖Att⁡\(k∣PN\)−Att⁡\(k′∣PN\)‖2≥V4\\\|\\operatorname\{Att\}\(k\\mid P\_\{N\}\)\\\|\_\{2\}\\leq\\frac\{249\}\{250\}V\\qquad\\text\{and\}\\quad\\\|\\operatorname\{Att\}\(k\\mid P\_\{N\}\)\-\\operatorname\{Att\}\(k^\{\\prime\}\\mid P\_\{N\}\)\\\|\_\{2\}\\geq\\frac\{V\}\{4\}for allk,k′∈supp⁡\(νN\)k,k^\{\\prime\}\\in\\operatorname\{supp\}\(\\nu\_\{N\}\)withk≠k′k\\neq k^\{\\prime\}\.

###### Lemma B\.6\.

Fix an even integerN≥100N\\geq 100, and let\(PN,νN\)\(P\_\{N\},\\nu\_\{N\}\)be the lookup instance of sizeNNfrom[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4)\. Assume‖v‖2≤V\\\|v\\\|\_\{2\}\\leq Vfor allv∈𝒱v\\in\\mathcal\{V\}\. Then, for everyP^∈𝒫N/2​\(𝒳\)\\hat\{P\}\\in\\mathcal\{P\}\_\{N/2\}\(\\mathcal\{X\}\),

ℰPN,νN​\(P^\)≳V2\.\\mathcal\{E\}\_\{P\_\{N\},\\nu\_\{N\}\}\\bigl\(\\hat\{P\}\\bigr\)\\gtrsim V^\{2\}\.

###### Proof of[Lemma˜B\.5](https://arxiv.org/html/2607.01520#A2.Thmtheorem5)\.

Write

PN=1N​∑i=1Nδ\(ki,vi\),νN=1N​∑i=1Nδki,P\_\{N\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\delta\_\{\(k\_\{i\},v\_\{i\}\)\},\\qquad\\nu\_\{N\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\delta\_\{k\_\{i\}\},as in[Definition˜B\.3](https://arxiv.org/html/2607.01520#A2.Thmtheorem3), and set

wj​\(q\)≔1N​akj​\(q∣PN\)\.w\_\{j\}\(q\)\\coloneqq\\frac\{1\}\{N\}a\_\{k\_\{j\}\}\(q\\mid P\_\{N\}\)\.Then,wj​\(q\)≥0w\_\{j\}\(q\)\\geq 0,∑j=1Nwj​\(q\)=1\\sum\_\{j=1\}^\{N\}w\_\{j\}\(q\)=1, and

Att⁡\(q∣PN\)=∑j=1Nwj​\(q\)​vj\.\\operatorname\{Att\}\(q\\mid P\_\{N\}\)=\\sum\_\{j=1\}^\{N\}w\_\{j\}\(q\)v\_\{j\}\.
Sinceakj​\(0∣PN\)=1a\_\{k\_\{j\}\}\(0\\mid P\_\{N\}\)=1for everyj∈\[N\]j\\in\[N\], centering gives

Att⁡\(0∣PN\)=1N​∑j=1Nvj=0\.\\operatorname\{Att\}\(0\\mid P\_\{N\}\)=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}v\_\{j\}=0\.
Fixi∈\[N\]i\\in\[N\]\. By sharpness,

∑j≠iwj​\(ki\)=1N​∑j≠iakj​\(ki∣PN\)≤N−3/2\.\\sum\_\{j\\neq i\}w\_\{j\}\(k\_\{i\}\)=\\frac\{1\}\{N\}\\sum\_\{j\\neq i\}a\_\{k\_\{j\}\}\(k\_\{i\}\\mid P\_\{N\}\)\\leq N^\{\-3/2\}\.Hence, using‖vj‖2=ρ0​V\\\|v\_\{j\}\\\|\_\{2\}=\\rho\_\{0\}Vfor allj∈\[N\]j\\in\[N\],

‖Att⁡\(ki∣PN\)−vi‖2\\displaystyle\\\|\\operatorname\{Att\}\(k\_\{i\}\\mid P\_\{N\}\)\-v\_\{i\}\\\|\_\{2\}=‖\(wi​\(ki\)−1\)​vi\+∑j≠iwj​\(ki\)​vj‖2\\displaystyle=\\left\\\|\(w\_\{i\}\(k\_\{i\}\)\-1\)v\_\{i\}\+\\sum\_\{j\\neq i\}w\_\{j\}\(k\_\{i\}\)v\_\{j\}\\right\\\|\_\{2\}≤\(1−wi​\(ki\)\)​‖vi‖2\+∑j≠iwj​\(ki\)​‖vj‖2\\displaystyle\\leq\(1\-w\_\{i\}\(k\_\{i\}\)\)\\\|v\_\{i\}\\\|\_\{2\}\+\\sum\_\{j\\neq i\}w\_\{j\}\(k\_\{i\}\)\\\|v\_\{j\}\\\|\_\{2\}=2​ρ0​V​∑j≠iwj​\(ki\)≤2​ρ0​V​N−3/2\.\\displaystyle=2\\rho\_\{0\}V\\sum\_\{j\\neq i\}w\_\{j\}\(k\_\{i\}\)\\leq 2\\rho\_\{0\}VN^\{\-3/2\}\.This proves the approximation bound\.

The norm bound follows immediately\. ForN≥100N\\geq 100,

‖Att⁡\(ki∣PN\)‖2≤‖vi\+\(Att⁡\(ki∣PN\)−vi\)‖2≤‖vi‖2\+2​ρ0​V​N−3/2=ρ0​V​\(1\+2​N−3/2\)≤249250​V\.\\\|\\operatorname\{Att\}\(k\_\{i\}\\mid P\_\{N\}\)\\\|\_\{2\}\\leq\\\|v\_\{i\}\+\(\\operatorname\{Att\}\(k\_\{i\}\\mid P\_\{N\}\)\-v\_\{i\}\)\\\|\_\{2\}\\leq\\\|v\_\{i\}\\\|\_\{2\}\+2\\rho\_\{0\}VN^\{\-3/2\}=\\rho\_\{0\}V\(1\+2N^\{\-3/2\}\)\\leq\\frac\{249\}\{250\}V\.Finally, ifi≠ji\\neq j, then the separation condition gives

‖vi−vj‖22=2​ρ02​V2−2​⟨vi,vj⟩≥2​\(1−γ0\)​ρ02​V2\.\\\|v\_\{i\}\-v\_\{j\}\\\|\_\{2\}^\{2\}=2\\rho\_\{0\}^\{2\}V^\{2\}\-2\\left\\langle v\_\{i\},v\_\{j\}\\right\\rangle\\geq 2\(1\-\\gamma\_\{0\}\)\\rho\_\{0\}^\{2\}V^\{2\}\.Therefore, again usingN≥100N\\geq 100,ρ0=99/100\\rho\_\{0\}=99/100, andγ0=9/10\\gamma\_\{0\}=9/10,

‖Att⁡\(ki∣PN\)−Att⁡\(kj∣PN\)‖2\\displaystyle\\\|\\operatorname\{Att\}\(k\_\{i\}\\mid P\_\{N\}\)\-\\operatorname\{Att\}\(k\_\{j\}\\mid P\_\{N\}\)\\\|\_\{2\}≥‖vi−vj‖2−‖Att⁡\(ki∣PN\)−vi‖2−‖Att⁡\(kj∣PN\)−vj‖2\\displaystyle\\geq\\\|v\_\{i\}\-v\_\{j\}\\\|\_\{2\}\-\\\|\\operatorname\{Att\}\(k\_\{i\}\\mid P\_\{N\}\)\-v\_\{i\}\\\|\_\{2\}\-\\\|\\operatorname\{Att\}\(k\_\{j\}\\mid P\_\{N\}\)\-v\_\{j\}\\\|\_\{2\}≥ρ0​V​2​\(1−γ0\)−4​ρ0​V​N−3/2≥V4\.\\displaystyle\\geq\\rho\_\{0\}V\\sqrt\{2\(1\-\\gamma\_\{0\}\)\}\-4\\rho\_\{0\}VN^\{\-3/2\}\\geq\\frac\{V\}\{4\}\.That proves the claim\. ∎

###### Proof of[Lemma˜B\.6](https://arxiv.org/html/2607.01520#A2.Thmtheorem6)\.

Write

PN=1N​∑i=1Nδ\(ki,vi\),νN=1N​∑i=1Nδki,P\_\{N\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\delta\_\{\(k\_\{i\},v\_\{i\}\)\},\\qquad\\nu\_\{N\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\delta\_\{k\_\{i\}\},and let

yi≔Att⁡\(ki∣PN\)\.y\_\{i\}\\coloneqq\\operatorname\{Att\}\(k\_\{i\}\\mid P\_\{N\}\)\.By[Lemma˜B\.5](https://arxiv.org/html/2607.01520#A2.Thmtheorem5),

‖yi−vi‖2≤2​ρ0​V​N−3/2\\\|y\_\{i\}\-v\_\{i\}\\\|\_\{2\}\\leq 2\\rho\_\{0\}VN^\{\-3/2\}for alli∈\[N\]i\\in\[N\]\.

FixP^∈𝒫N/2​\(𝒳\)\\hat\{P\}\\in\\mathcal\{P\}\_\{N/2\}\(\\mathcal\{X\}\), and write

P^=∑r=1Mβr​δ\(k^r,v^r\)\\hat\{P\}=\\sum\_\{r=1\}^\{M\}\\beta\_\{r\}\\delta\_\{\(\\hat\{k\}\_\{r\},\\hat\{v\}\_\{r\}\)\}forM≤N/2M\\leq N/2\. By assumption,‖v^r‖2≤V\\\|\\hat\{v\}\_\{r\}\\\|\_\{2\}\\leq Vfor allr∈\[M\]r\\in\[M\]\. For eachi∈\[N\]i\\in\[N\], set

zi≔Att⁡\(ki∣P^\)\.z\_\{i\}\\coloneqq\\operatorname\{Att\}\(k\_\{i\}\\mid\\hat\{P\}\)\.Sinceziz\_\{i\}is a convex combination of the valuesv^1,…,v^M\\hat\{v\}\_\{1\},\\ldots,\\hat\{v\}\_\{M\}, we may write

zi=∑r=1Mωi​r​v^r,z\_\{i\}=\\sum\_\{r=1\}^\{M\}\\omega\_\{ir\}\\hat\{v\}\_\{r\},withωi​r≥0\\omega\_\{ir\}\\geq 0and‖ωi‖1=1\\\|\\omega\_\{i\}\\\|\_\{1\}=1for alli∈\[N\]i\\in\[N\]\.

Let

ℐ≔\{i∈\[N\]∣‖zi−yi‖2≤V200\}\\mathcal\{I\}\\coloneqq\\left\\\{i\\in\[N\]\\mid\\\|z\_\{i\}\-y\_\{i\}\\\|\_\{2\}\\leq\\frac\{V\}\{200\}\\right\\\}be the well\-approximated indices\. We claim that\|ℐ\|≤M\|\\mathcal\{I\}\|\\leq M\. To that end, fixi∈ℐi\\in\\mathcal\{I\}\. Then,

⟨zi,vi⟩\\displaystyle\\left\\langle z\_\{i\},v\_\{i\}\\right\\rangle=⟨yi,vi⟩\+⟨zi−yi,vi⟩\\displaystyle=\\left\\langle y\_\{i\},v\_\{i\}\\right\\rangle\+\\left\\langle z\_\{i\}\-y\_\{i\},v\_\{i\}\\right\\rangle≥⟨yi,vi⟩−‖zi−yi‖2​‖vi‖2\\displaystyle\\geq\\left\\langle y\_\{i\},v\_\{i\}\\right\\rangle\-\\\|z\_\{i\}\-y\_\{i\}\\\|\_\{2\}\\\|v\_\{i\}\\\|\_\{2\}=⟨vi,vi⟩\+⟨yi−vi,vi⟩−‖zi−yi‖2​‖vi‖2\\displaystyle=\\left\\langle v\_\{i\},v\_\{i\}\\right\\rangle\+\\left\\langle y\_\{i\}\-v\_\{i\},v\_\{i\}\\right\\rangle\-\\\|z\_\{i\}\-y\_\{i\}\\\|\_\{2\}\\\|v\_\{i\}\\\|\_\{2\}≥‖vi‖22−‖yi−vi‖2​‖vi‖2−V200​‖vi‖2\\displaystyle\\geq\\\|v\_\{i\}\\\|\_\{2\}^\{2\}\-\\\|y\_\{i\}\-v\_\{i\}\\\|\_\{2\}\\\|v\_\{i\}\\\|\_\{2\}\-\\frac\{V\}\{200\}\\\|v\_\{i\}\\\|\_\{2\}≥\(ρ02−2​ρ02​N−3/2−ρ0200\)​V2≥97100​V2,\\displaystyle\\geq\\left\(\\rho\_\{0\}^\{2\}\-2\\rho\_\{0\}^\{2\}N^\{\-3/2\}\-\\frac\{\\rho\_\{0\}\}\{200\}\\right\)V^\{2\}\\geq\\frac\{97\}\{100\}V^\{2\},where the last step usesN≥100N\\geq 100andρ0=99/100\\rho\_\{0\}=99/100\. Sincezi=∑rωi​r​v^rz\_\{i\}=\\sum\_\{r\}\\omega\_\{ir\}\\hat\{v\}\_\{r\}, there existsr​\(i\)∈\[M\]r\(i\)\\in\[M\]such that

⟨v^r​\(i\),vi,≥⟩​97100​V2\.\\left\\langle\\hat\{v\}\_\{r\(i\)\},v\_\{i\},\\geq\\right\\rangle\\frac\{97\}\{100\}V^\{2\}\.
The mapi↦r​\(i\)i\\mapsto r\(i\)is injective\. Indeed, suppose thatr​\(i\)=r​\(j\)=rr\(i\)=r\(j\)=rfor distincti,j∈ℐi,j\\in\\mathcal\{I\}\. Set

u≔v^rV,si≔viρ0​V,andsj≔vjρ0​V\.u\\coloneqq\\frac\{\\hat\{v\}\_\{r\}\}\{V\},\\qquad s\_\{i\}\\coloneqq\\frac\{v\_\{i\}\}\{\\rho\_\{0\}V\},\\qquad\\text\{and\}\\qquad s\_\{j\}\\coloneqq\\frac\{v\_\{j\}\}\{\\rho\_\{0\}V\}\.Then,‖u‖2≤1\\\|u\\\|\_\{2\}\\leq 1,‖si‖2=‖sj‖2=1\\\|s\_\{i\}\\\|\_\{2\}=\\\|s\_\{j\}\\\|\_\{2\}=1, and by separation,

⟨si,sj⟩≤γ0\.\\left\\langle s\_\{i\},s\_\{j\}\\right\\rangle\\leq\\gamma\_\{0\}\.On the other hand,

⟨u,si\+sj,≥⟩​2⋅97100​ρ0=19499\.\\left\\langle u,s\_\{i\}\+s\_\{j\},\\geq\\right\\rangle\\frac\{2\\cdot 97\}\{100\\rho\_\{0\}\}=\\frac\{194\}\{99\}\.This contradicts Cauchy–Schwarz, since

⟨u,si\+sj,≤⟩​‖si\+sj‖2≤2\+2​γ0=195<19499\.\\left\\langle u,s\_\{i\}\+s\_\{j\},\\leq\\right\\rangle\\\|s\_\{i\}\+s\_\{j\}\\\|\_\{2\}\\leq\\sqrt\{2\+2\\gamma\_\{0\}\}=\\sqrt\{\\frac\{19\}\{5\}\}<\\frac\{194\}\{99\}\.Hence,\|ℐ\|≤M≤N/2\|\\mathcal\{I\}\|\\leq M\\leq N/2\.

Therefore, at leastN/2N/2indices satisfy‖zi−yi‖2\>V/200\\\|z\_\{i\}\-y\_\{i\}\\\|\_\{2\}\>V/200\. SinceνN\\nu\_\{N\}is uniform on the hard queries,

ℰPN,νN​\(P^\)\\displaystyle\\mathcal\{E\}\_\{P\_\{N\},\\nu\_\{N\}\}\\bigl\(\\hat\{P\}\\bigr\)=1N​∑i=1N‖yi−zi‖22\\displaystyle=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\\|y\_\{i\}\-z\_\{i\}\\\|\_\{2\}^\{2\}≥1N⋅N2⋅\(V200\)2=V280000\.\\displaystyle\\geq\\frac\{1\}\{N\}\\cdot\\frac\{N\}\{2\}\\cdot\\left\(\\frac\{V\}\{200\}\\right\)^\{2\}=\\frac\{V^\{2\}\}\{80000\}\.That proves the claim\. ∎

#### B\.2\.3Query\-aware lower bound

With the lookup instance established as hard to compress, we turn to the query\-aware minimax lower bound\. By itself, the lookup instance only gives the lower bound at the lowest redundancy scale, wheretailK/3⁡\(ΣP,ν\)\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\_\{P,\\nu\}\)is of orderV2​KV^\{2\}Kand everyKK\-atomic approximation incurs error of orderV2V^\{2\}\. To show thattailK/3⁡\(ΣP,ν\)\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\_\{P,\\nu\}\)faithfully governs the minimax risk throughout all scalesR∈\[0,V2​K\]R\\in\[0,V^\{2\}K\], we need the same obstruction at every intermediate such scale\. We achieve this by diluting the lookup query distribution with the easy query distributionδ0\\delta\_\{0\}\. The dummy query contributes only low\-rank covariance, while the hard queries contribute both the\(K/3\)\(K/3\)\-tail and the approximation error\. Thus, the mixturep​νN\+\(1−p\)​δ0p\\nu\_\{N\}\+\(1\-p\)\\delta\_\{0\}scales the relevant redundancy and the compression difficulty by the same factorpp, allowing us to tune both to the desired levelRR\.

###### Lemma B\.7\.

LetK≥50K\\geq 50and setN≔2​KN\\coloneqq 2K\. Suppose[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)holds, and let\(PN,νN\)\(P\_\{N\},\\nu\_\{N\}\)be the lookup instance from[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4)\. Forp∈\[0,1\]p\\in\[0,1\], define the diluted query distribution

νN,p≔p​νN\+\(1−p\)​δ0\.\\nu\_\{N,p\}\\coloneqq p\\,\\nu\_\{N\}\+\(1\-p\)\\,\\delta\_\{0\}\.Then,

tailK/3⁡\(ΣPN,νN,p\)≍p​V2​K\.\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\_\{P\_\{N\},\\nu\_\{N,p\}\}\)\\asymp p\\,V^\{2\}K\.

###### Lemma B\.8\.

LetK≥50K\\geq 50and setN≔2​KN\\coloneqq 2K\. Suppose[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)holds and let\(PN,νN\)\(P\_\{N\},\\nu\_\{N\}\)be the lookup instance from[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4)\. Then, for everyk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\), there existsP^k∈𝒫2​\(𝒳\)\\hat\{P\}\_\{k\}\\in\\mathcal\{P\}\_\{2\}\(\\mathcal\{X\}\)such that

Att⁡\(0∣P^k\)=Att⁡\(0∣PN\),andAtt⁡\(k∣P^k\)=Att⁡\(k∣PN\)\.\\operatorname\{Att\}\(0\\mid\\hat\{P\}\_\{k\}\)=\\operatorname\{Att\}\(0\\mid P\_\{N\}\),\\qquad\\text\{and\}\\qquad\\operatorname\{Att\}\(k\\mid\\hat\{P\}\_\{k\}\)=\\operatorname\{Att\}\(k\\mid P\_\{N\}\)\.Consequently, for everyp∈\[0,1\]p\\in\[0,1\]and everyk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\),

ℰPN,p​δk\+\(1−p\)​δ0​\(P^k\)=0\.\\mathcal\{E\}\_\{P\_\{N\},p\\,\\delta\_\{k\}\+\(1\-p\)\\,\\delta\_\{0\}\}\\bigl\(\\hat\{P\}\_\{k\}\\bigr\)=0\.

###### Proof of[Lemma˜B\.7](https://arxiv.org/html/2607.01520#A2.Thmtheorem7)\.

Write

PN=1N​∑i=1Nδ\(ki,vi\),andνN=1N​∑i=1Nδki,P\_\{N\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\delta\_\{\(k\_\{i\},v\_\{i\}\)\},\\qquad\\text\{and\}\\qquad\\nu\_\{N\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\delta\_\{k\_\{i\}\},and abbreviate

ai​j≔akj​\(ki∣PN\),andyi≔Att⁡\(ki∣PN\)\.a\_\{ij\}\\coloneqq a\_\{k\_\{j\}\}\(k\_\{i\}\\mid P\_\{N\}\),\\qquad\\text\{and\}\\qquad y\_\{i\}\\coloneqq\\operatorname\{Att\}\(k\_\{i\}\\mid P\_\{N\}\)\.Let

𝖧0≔\{u∈L2​\(PN\)∣∫𝒳u​d​PN=0\}\.\\mathsf\{H\}\_\{0\}\\coloneqq\\left\\\{u\\in L\_\{2\}\(P\_\{N\}\)\\mid\\int\_\{\\mathcal\{X\}\}u\\,\\mathop\{\}\\\!\\mathrm\{d\}P\_\{N\}=0\\right\\\}\.This space has dimensionN−1N\-1\. Define

Tρ:𝖧0→ℋρ,u↦∫XΓPN​\(z\)​u​\(z\)​d​PN​\(z\)T\_\{\\rho\}:\\mathsf\{H\}\_\{0\}\\to\\mathcal\{H\}\_\{\\rho\},\\qquad u\\mapsto\\int\_\{X\}\\Gamma\_\{P\_\{N\}\}\(z\)u\(z\)\\,\\mathop\{\}\\\!\\mathrm\{d\}P\_\{N\}\(z\)and let

Gρ≔Tρ∗​TρG\_\{\\rho\}\\coloneqq T\_\{\\rho\}^\{\*\}T\_\{\\rho\}be the corresponding Gram operator on𝖧0\\mathsf\{H\}\_\{0\}\. Since

∫XΓPN​d​PN=0,\\int\_\{X\}\\Gamma\_\{P\_\{N\}\}\\,\\mathop\{\}\\\!\\mathrm\{d\}P\_\{N\}=0,we have

Tρ​Tρ∗=ΣPN,ρ\.T\_\{\\rho\}T\_\{\\rho\}^\{\*\}=\\Sigma\_\{P\_\{N\},\\rho\}\.Hence,GρG\_\{\\rho\}andΣPN,ρ\\Sigma\_\{P\_\{N\},\\rho\}have the same nonzero eigenvalues\.

We first prove

tailK/3⁡\(GνN\)≍V2​K\.\\operatorname\{tail\}\_\{K/3\}\(G\_\{\\nu\_\{N\}\}\)\\asymp V^\{2\}K\.Foru∈𝖧0u\\in\\mathsf\{H\}\_\{0\}, writeui≔u​\(ki,vi\)u\_\{i\}\\coloneqq u\(k\_\{i\},v\_\{i\}\), so that

1N​∑i=1Nui=0,and‖u‖L2​\(PN\)2=1N​∑i=1Nui2\.\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}u\_\{i\}=0,\\qquad\\text\{and\}\\qquad\\\|u\\\|\_\{L\_\{2\}\(P\_\{N\}\)\}^\{2\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}u\_\{i\}^\{2\}\.The scalar component ofTνN​uT\_\{\\nu\_\{N\}\}uat querykik\_\{i\}is

\(TνN​u\)sc​\(ki\)≔VN​∑j=1N\(ai​j−1\)​uj\.\(T\_\{\\nu\_\{N\}\}u\)\_\{\\mathrm\{sc\}\}\(k\_\{i\}\)\\coloneqq\\frac\{V\}\{N\}\\sum\_\{j=1\}^\{N\}\(a\_\{ij\}\-1\)u\_\{j\}\.By sharpness,

si≔∑j≠iai​j≤N−1/2,andai​i=N−si\.s\_\{i\}\\coloneqq\\sum\_\{j\\neq i\}a\_\{ij\}\\leq N^\{\-1/2\},\\qquad\\text\{and\}\\qquad a\_\{ii\}=N\-s\_\{i\}\.Hence, foru∈𝖧0u\\in\\mathsf\{H\}\_\{0\},

\(TνN​u\)sc​\(ki\)\\displaystyle\(T\_\{\\nu\_\{N\}\}u\)\_\{\\mathrm\{sc\}\}\(k\_\{i\}\)=VN​∑j=1N\(ai​j−1\)​uj\\displaystyle=\\frac\{V\}\{N\}\\sum\_\{j=1\}^\{N\}\(a\_\{ij\}\-1\)u\_\{j\}=V​ui\+VN​\(−si​ui\+∑j≠iai​j​uj\)\.\\displaystyle=Vu\_\{i\}\+\\frac\{V\}\{N\}\\left\(\-s\_\{i\}u\_\{i\}\+\\sum\_\{j\\neq i\}a\_\{ij\}u\_\{j\}\\right\)\.If‖u‖L2​\(PN\)=1\\\|u\\\|\_\{L\_\{2\}\(P\_\{N\}\)\}=1, then\|uj\|≤N\|u\_\{j\}\|\\leq\\sqrt\{N\}for alljj, and therefore

\|\(TνN​u\)sc​\(ki\)−V​ui\|≤2​VN\.\\left\|\(T\_\{\\nu\_\{N\}\}u\)\_\{\\mathrm\{sc\}\}\(k\_\{i\}\)\-Vu\_\{i\}\\right\|\\leq\\frac\{2V\}\{N\}\.Consequently,

‖TνN​u‖ℋνN\\displaystyle\\\|T\_\{\\nu\_\{N\}\}u\\\|\_\{\\mathcal\{H\}\_\{\\nu\_\{N\}\}\}≥\(1N​∑i=1N\|\(TνN​u\)sc​\(ki\)\|2\)1/2\\displaystyle\\geq\\left\(\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\left\|\(T\_\{\\nu\_\{N\}\}u\)\_\{\\mathrm\{sc\}\}\(k\_\{i\}\)\\right\|^\{2\}\\right\)^\{1/2\}≥\(1N​∑i=1NV2​ui2\)1/2−\(1N​∑i=1N4​V2N2\)1/2\\displaystyle\\geq\\left\(\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}V^\{2\}u\_\{i\}^\{2\}\\right\)^\{1/2\}\-\\left\(\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\frac\{4V^\{2\}\}\{N^\{2\}\}\\right\)^\{1/2\}≥V−2​VN≥V2\.\\displaystyle\\geq V\-\\frac\{2V\}\{N\}\\geq\\frac\{V\}\{2\}\.Thus,

GνN⪰V24​I𝖧0\.G\_\{\\nu\_\{N\}\}\\succeq\\frac\{V^\{2\}\}\{4\}I\_\{\\mathsf\{H\}\_\{0\}\}\.Sincedim𝖧0=N−1=2​K−1\\dim\\mathsf\{H\}\_\{0\}=N\-1=2K\-1,

tailK/3⁡\(GνN\)≳V2​K\.\\operatorname\{tail\}\_\{K/3\}\(G\_\{\\nu\_\{N\}\}\)\\gtrsim V^\{2\}K\.
For the reverse bound, again usesi≤N−1/2s\_\{i\}\\leq N^\{\-1/2\}\. Then,

∑j=1Nai​j2=\(N−si\)2\+∑j≠iai​j2≤N2\+1\.\\sum\_\{j=1\}^\{N\}a\_\{ij\}^\{2\}=\(N\-s\_\{i\}\)^\{2\}\+\\sum\_\{j\\neq i\}a\_\{ij\}^\{2\}\\leq N^\{2\}\+1\.Since∑jai​j=N\\sum\_\{j\}a\_\{ij\}=N,

∑j=1N\(ai​j−1\)2=∑j=1Nai​j2−N≤N2\+1\.\\sum\_\{j=1\}^\{N\}\(a\_\{ij\}\-1\)^\{2\}=\\sum\_\{j=1\}^\{N\}a\_\{ij\}^\{2\}\-N\\leq N^\{2\}\+1\.By[Lemma˜B\.5](https://arxiv.org/html/2607.01520#A2.Thmtheorem5),‖yi‖2≤V\\\|y\_\{i\}\\\|\_\{2\}\\leq V, and by the boundary condition,‖vj‖2≤V\\\|v\_\{j\}\\\|\_\{2\}\\leq V\. Hence,

‖vj−yi‖2≤2​V\.\\\|v\_\{j\}\-y\_\{i\}\\\|\_\{2\}\\leq 2V\.Therefore, using the trace formula forΣPN,δki\\Sigma\_\{P\_\{N\},\\delta\_\{k\_\{i\}\}\},

tr⁡\(ΣPN,δki\)\\displaystyle\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\delta\_\{k\_\{i\}\}\}\)=1N​∑j=1N\[V2​\(ai​j−1\)2\+ai​j2​‖vj−yi‖22\]\\displaystyle=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}\\left\[V^\{2\}\(a\_\{ij\}\-1\)^\{2\}\+a\_\{ij\}^\{2\}\\\|v\_\{j\}\-y\_\{i\}\\\|\_\{2\}^\{2\}\\right\]≤1N​\[V2​\(N2\+1\)\+4​V2​\(N2\+1\)\]≲V2​N\.\\displaystyle\\leq\\frac\{1\}\{N\}\\left\[V^\{2\}\(N^\{2\}\+1\)\+4V^\{2\}\(N^\{2\}\+1\)\\right\]\\lesssim V^\{2\}N\.Averaging overiigives

tr⁡\(GνN\)=tr⁡\(ΣPN,νN\)≲V2​N≍V2​K,\\operatorname\{tr\}\(G\_\{\\nu\_\{N\}\}\)=\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\nu\_\{N\}\}\)\\lesssim V^\{2\}N\\asymp V^\{2\}K,and hence

tailK/3⁡\(GνN\)≲V2​K\.\\operatorname\{tail\}\_\{K/3\}\(G\_\{\\nu\_\{N\}\}\)\\lesssim V^\{2\}K\.Combining the two bounds gives

tailK/3⁡\(GνN\)≍V2​K\.\\operatorname\{tail\}\_\{K/3\}\(G\_\{\\nu\_\{N\}\}\)\\asymp V^\{2\}K\.
Set

GN≔GνN,G0≔Gδ0,andGp≔GνN,p\.G\_\{N\}\\coloneqq G\_\{\\nu\_\{N\}\},\\qquad G\_\{0\}\\coloneqq G\_\{\\delta\_\{0\}\},\\qquad\\text\{and\}\\qquad G\_\{p\}\\coloneqq G\_\{\\nu\_\{N,p\}\}\.For everyu∈𝖧0u\\in\\mathsf\{H\}\_\{0\},

⟨u,Gp​u⟩𝖧0\\displaystyle\\left\\langle u,G\_\{p\}u\\right\\rangle\_\{\\mathsf\{H\}\_\{0\}\}=‖TνN,p​u‖ℋνN,p2\\displaystyle=\\\|T\_\{\\nu\_\{N,p\}\}u\\\|\_\{\\mathcal\{H\}\_\{\\nu\_\{N,p\}\}\}^\{2\}=p​‖TνN​u‖ℋνN2\+\(1−p\)​‖Tδ0​u‖ℋδ02\\displaystyle=p\\\|T\_\{\\nu\_\{N\}\}u\\\|\_\{\\mathcal\{H\}\_\{\\nu\_\{N\}\}\}^\{2\}\+\(1\-p\)\\\|T\_\{\\delta\_\{0\}\}u\\\|\_\{\\mathcal\{H\}\_\{\\delta\_\{0\}\}\}^\{2\}=⟨u,\(p​GN\+\(1−p\)​G0\)​u⟩𝖧0\.\\displaystyle=\\left\\langle u,\(pG\_\{N\}\+\(1\-p\)G\_\{0\}\)u\\right\\rangle\_\{\\mathsf\{H\}\_\{0\}\}\.Hence,

Gp=p​GN\+\(1−p\)​G0G\_\{p\}=pG\_\{N\}\+\(1\-p\)G\_\{0\}as operators on𝖧0\\mathsf\{H\}\_\{0\}\.

At query0, the scalar response vanishes and the value response lies inspan⁡\{v1,…,vN\}\\operatorname\{span\}\\\{v\_\{1\},\\ldots,v\_\{N\}\\\}\. By[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4),

rank⁡\(G0\)≤dimspan⁡\{v1,…,vN\}≤K/6\.\\operatorname\{rank\}\(G\_\{0\}\)\\leq\\dim\\operatorname\{span\}\\\{v\_\{1\},\\ldots,v\_\{N\}\\\}\\leq K/6\.The lower bound follows fromGp⪰p​GNG\_\{p\}\\succeq pG\_\{N\}\. By eigenvalue monotonicity,

tailK/3⁡\(Gp\)≥p​tailK/3⁡\(GN\)≳p​V2​K\.\\operatorname\{tail\}\_\{K/3\}\(G\_\{p\}\)\\geq p\\,\\operatorname\{tail\}\_\{K/3\}\(G\_\{N\}\)\\gtrsim pV^\{2\}K\.
For the upper bound, letU⊂𝖧0U\\subset\\mathsf\{H\}\_\{0\}be any subspace of dimension at mostK/3K/3containingran⁡\(G0\)\\operatorname\{ran\}\(G\_\{0\}\)\. Then,

tr⁡\(\(I−ΠU\)​G0\)=0\.\\operatorname\{tr\}\\bigl\(\(I\-\\Pi\_\{U\}\)G\_\{0\}\\bigr\)=0\.Hence, by Ky Fan’s variational formula,

tailK/3⁡\(Gp\)\\displaystyle\\operatorname\{tail\}\_\{K/3\}\(G\_\{p\}\)≤tr⁡\(\(I−ΠU\)​Gp\)\\displaystyle\\leq\\operatorname\{tr\}\\bigl\(\(I\-\\Pi\_\{U\}\)G\_\{p\}\\bigr\)=p​tr⁡\(\(I−ΠU\)​GN\)\+\(1−p\)​tr⁡\(\(I−ΠU\)​G0\)\\displaystyle=p\\,\\operatorname\{tr\}\\bigl\(\(I\-\\Pi\_\{U\}\)G\_\{N\}\\bigr\)\+\(1\-p\)\\operatorname\{tr\}\\bigl\(\(I\-\\Pi\_\{U\}\)G\_\{0\}\\bigr\)=p​tr⁡\(\(I−ΠU\)​GN\)≤p​tr⁡\(GN\)≲p​V2​K\.\\displaystyle=p\\,\\operatorname\{tr\}\\bigl\(\(I\-\\Pi\_\{U\}\)G\_\{N\}\\bigr\)\\leq p\\,\\operatorname\{tr\}\(G\_\{N\}\)\\lesssim pV^\{2\}K\.Thus,

tailK/3⁡\(Gp\)≍p​V2​K\.\\operatorname\{tail\}\_\{K/3\}\(G\_\{p\}\)\\asymp pV^\{2\}K\.SinceGpG\_\{p\}andΣPN,νN,p\\Sigma\_\{P\_\{N\},\\nu\_\{N,p\}\}have the same nonzero eigenvalues,

tailK/3⁡\(ΣPN,νN,p\)≍p​V2​K\.\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\_\{P\_\{N\},\\nu\_\{N,p\}\}\)\\asymp pV^\{2\}K\.

###### Proof of[Lemma˜B\.8](https://arxiv.org/html/2607.01520#A2.Thmtheorem8)\.

Fixk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\), and set

y≔Att⁡\(k∣PN\)\.y\\coloneqq\\operatorname\{Att\}\(k\\mid P\_\{N\}\)\.By[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4),

‖k‖2=5​dk1/4​log⁡N\.\\\|k\\\|\_\{2\}=5d\_\{k\}^\{1/4\}\\sqrt\{\\log N\}\.SinceN=2​KN=2KandK≥50K\\geq 50,

‖k‖2≤8​dk1/4​log⁡K,\\\|k\\\|\_\{2\}\\leq 8d\_\{k\}^\{1/4\}\\sqrt\{\\log K\},sok,−k∈𝒦k,\-k\\in\\mathcal\{K\}by[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)\. Moreover,

‖k‖22dk=25​log⁡N\.\\frac\{\\\|k\\\|\_\{2\}^\{2\}\}\{\\sqrt\{d\_\{k\}\}\}=25\\log N\.By[Lemma˜B\.5](https://arxiv.org/html/2607.01520#A2.Thmtheorem5),

Att⁡\(0∣PN\)=0,and‖y‖2≤249250​V\.\\operatorname\{Att\}\(0\\mid P\_\{N\}\)=0,\\qquad\\text\{and\}\\qquad\\\|y\\\|\_\{2\}\\leq\\frac\{249\}\{250\}V\.SinceN≥100N\\geq 100,

tanh⁡\(25​log⁡N\)=N50−1N50\+1≥249250\.\\tanh\(25\\log N\)=\\frac\{N^\{50\}\-1\}\{N^\{50\}\+1\}\\geq\\frac\{249\}\{250\}\.Hence the vector

w≔ytanh⁡\(25​log⁡N\)w\\coloneqq\\frac\{y\}\{\\tanh\(25\\log N\)\}satisfies‖w‖2≤V\\\|w\\\|\_\{2\}\\leq V, and thereforew,−w∈𝒱w,\-w\\in\\mathcal\{V\}\.

Define

P^k≔12​δ\(k,w\)\+12​δ\(−k,−w\)\.\\hat\{P\}\_\{k\}\\coloneqq\\frac\{1\}\{2\}\\delta\_\{\(k,w\)\}\+\\frac\{1\}\{2\}\\delta\_\{\(\-k,\-w\)\}\.At query0, the two logits are equal, so

Att⁡\(0∣P^k\)=12​w\+12​\(−w\)=0=Att⁡\(0∣PN\)\.\\operatorname\{Att\}\(0\\mid\\hat\{P\}\_\{k\}\)=\\frac\{1\}\{2\}w\+\\frac\{1\}\{2\}\(\-w\)=0=\\operatorname\{Att\}\(0\\mid P\_\{N\}\)\.At querykk, the two logits are

⟨k,k⟩dk=25​log⁡N,and⟨k,−k⟩dk=−25​log⁡N\.\\frac\{\\langle k,k\\rangle\}\{\\sqrt\{d\_\{k\}\}\}=25\\log N,\\qquad\\text\{and\}\\qquad\\frac\{\\langle k,\-k\\rangle\}\{\\sqrt\{d\_\{k\}\}\}=\-25\\log N\.Thus

Att⁡\(k∣P^k\)=tanh⁡\(25​log⁡N\)​w=y=Att⁡\(k∣PN\)\.\\operatorname\{Att\}\(k\\mid\\hat\{P\}\_\{k\}\)=\\tanh\(25\\log N\)\\,w=y=\\operatorname\{Att\}\(k\\mid P\_\{N\}\)\.That proves the first claim\.

The final claim follows because the query distributionp​δk\+\(1−p\)​δ0p\\delta\_\{k\}\+\(1\-p\)\\delta\_\{0\}is supported on the two points wherePNP\_\{N\}andP^k\\hat\{P\}\_\{k\}have identical attention outputs, so

ℰPN,p​δk\+\(1−p\)​δ0​\(P^k\)\\displaystyle\\phantom\{=\}\\mathcal\{E\}\_\{P\_\{N\},p\\delta\_\{k\}\+\(1\-p\)\\delta\_\{0\}\}\\bigl\(\\hat\{P\}\_\{k\}\\bigr\)=p​‖Att⁡\(k∣PN\)−Att⁡\(k∣P^k\)‖22\+\(1−p\)​‖Att⁡\(0∣PN\)−Att⁡\(0∣P^k\)‖22=0\.\\displaystyle=p\\\|\\operatorname\{Att\}\(k\\mid P\_\{N\}\)\-\\operatorname\{Att\}\(k\\mid\\hat\{P\}\_\{k\}\)\\\|\_\{2\}^\{2\}\+\(1\-p\)\\\|\\operatorname\{Att\}\(0\\mid P\_\{N\}\)\-\\operatorname\{Att\}\(0\\mid\\hat\{P\}\_\{k\}\)\\\|\_\{2\}^\{2\}=0\.∎

###### Proof of[Section˜4\.3\.1](https://arxiv.org/html/2607.01520#S4.SS3.SSS1)\.

LetN≔2​KN\\coloneqq 2K, and let\(PN,νN\)\(P\_\{N\},\\nu\_\{N\}\)be the lookup instance from[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4)\. Set

p≔RV2​K,andνR≔p​νN\+\(1−p\)​δ0,p\\coloneqq\\frac\{R\}\{V^\{2\}K\},\\qquad\\text\{and\}\\qquad\\nu\_\{R\}\\coloneqq p\\,\\nu\_\{N\}\+\(1\-p\)\\,\\delta\_\{0\},and define

ℱ≔\{\(PN,νR\)\}\.\\mathcal\{F\}\\coloneqq\\\{\(P\_\{N\},\\nu\_\{R\}\)\\\}\.SinceR∈\[0,V2​K\]R\\in\[0,V^\{2\}K\], we havep∈\[0,1\]p\\in\[0,1\]\. Also, by[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4),\|supp⁡\(PN\)\|=N=2​K\|\\operatorname\{supp\}\(P\_\{N\}\)\|=N=2Kandℱ⊆𝒫2​K⋆​\(𝒳\)×𝒫​\(𝒬\)\\mathcal\{F\}\\subseteq\\mathcal\{P\}\_\{2K\}^\{\\star\}\(\\mathcal\{X\}\)\\times\\mathcal\{P\}\(\\mathcal\{Q\}\)\.

By[Lemma˜B\.7](https://arxiv.org/html/2607.01520#A2.Thmtheorem7),

tailK/3⁡\(ΣPN,νR\)≍p​V2​K=R\.\\operatorname\{tail\}\_\{K/3\}\(\\Sigma\_\{P\_\{N\},\\nu\_\{R\}\}\)\\asymp pV^\{2\}K=R\.
Now, fix any query\-awareKK\-atomic compressorAA, and letP^\\hat\{P\}be its random output on\(PN,νR\)\(P\_\{N\},\\nu\_\{R\}\)\. For every realization ofP^\\hat\{P\},

ℰPN,νR​\(P^\)=p​ℰPN,νN​\(P^\)\+\(1−p\)​ℰPN,δ0​\(P^\)≥p​ℰPN,νN​\(P^\)\.\\mathcal\{E\}\_\{P\_\{N\},\\nu\_\{R\}\}\\bigl\(\\hat\{P\}\\bigr\)=p\\,\\mathcal\{E\}\_\{P\_\{N\},\\nu\_\{N\}\}\\bigl\(\\hat\{P\}\\bigr\)\+\(1\-p\)\\,\\mathcal\{E\}\_\{P\_\{N\},\\delta\_\{0\}\}\\bigl\(\\hat\{P\}\\bigr\)\\geq p\\,\\mathcal\{E\}\_\{P\_\{N\},\\nu\_\{N\}\}\\bigl\(\\hat\{P\}\\bigr\)\.By[Lemma˜B\.6](https://arxiv.org/html/2607.01520#A2.Thmtheorem6),

ℰPN,νN​\(P^\)≳V2\\mathcal\{E\}\_\{P\_\{N\},\\nu\_\{N\}\}\\bigl\(\\hat\{P\}\\bigr\)\\gtrsim V^\{2\}almost surely\. Hence,

ℰPN,νR​\(P^\)≳p​V2=RK\\mathcal\{E\}\_\{P\_\{N\},\\nu\_\{R\}\}\\bigl\(\\hat\{P\}\\bigr\)\\gtrsim pV^\{2\}=\\frac\{R\}\{K\}almost surely\. Taking expectation over the randomness ofAA, and then taking the infimum over all query\-awareKK\-atomic compressors, gives

ℜ​\(𝒜Kaw,ℱ\)≳RK\.\\mathfrak\{R\}\\\!\\left\(\\mathcal\{A\}\_\{K\}^\{\\mathrm\{aw\}\},\\mathcal\{F\}\\right\)\\gtrsim\\frac\{R\}\{K\}\.

#### B\.2\.4Query\-agnostic lower bound

We now prove the query\-agnostic lower bound and simultaneously show a separation from the query\-aware regime\. We use the same hard lookup instance as before, but expose it through a family of query distributions of the formp​δk\+\(1−p\)​δ0p\\delta\_\{k\}\+\(1\-p\)\\delta\_\{0\}, one for each keyk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\)\. Each member of this family is easy for query\-aware compression: once the query law is known, the compressor only has to match the output at0and at one key\. At the same time, the family is hard for query\-agnostic compression because a query\-agnostic compressor must choose oneKK\-atomic summary that protects against all active keys\. Averaging over the active key then recovers the lookup query distributionνN\\nu\_\{N\}, on which any compression to half the context size necessarily incurs large error\.

As in the query\-aware regime, dilution by the0query controls the error and redundancy scale\. Each hard query contributes trace of orderV2​KV^\{2\}K, while the dummy query contributes trace of orderV2V^\{2\}\. Thus, the mixturep​δk\+\(1−p\)​δ0p\\delta\_\{k\}\+\(1\-p\)\\delta\_\{0\}tunestr⁡\(ΣP,ν\)\\operatorname\{tr\}\(\\Sigma\_\{P,\\nu\}\)to the target levelT∈\[V2,V2​K\]T\\in\[V^\{2\},V^\{2\}K\], while the averaged error retains the corresponding fraction of the lookup compression error\.

###### Lemma B\.9\.

LetK≥50K\\geq 50and setN≔2​KN\\coloneqq 2K\. Suppose[Figure˜1](https://arxiv.org/html/2607.01520#S4.F1)holds and let\(PN,νN\)\(P\_\{N\},\\nu\_\{N\}\)be the lookup instance from[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4)\. Then,

tr⁡\(ΣPN,δ0\)≍V2,\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\delta\_\{0\}\}\)\\asymp V^\{2\},and, for everyk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\),

tr⁡\(ΣPN,δk\)≍V2​K\.\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\delta\_\{k\}\}\)\\asymp V^\{2\}K\.Consequently, for everyp∈\[0,1\]p\\in\[0,1\]and everyk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\),

tr⁡\(ΣPN,p​δk\+\(1−p\)​δ0\)≍p​V2​K\+V2\.\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},p\\,\\delta\_\{k\}\+\(1\-p\)\\,\\delta\_\{0\}\}\)\\asymp pV^\{2\}K\+V^\{2\}\.

###### Proof of[Lemma˜B\.9](https://arxiv.org/html/2607.01520#A2.Thmtheorem9)\.

Write

PN=1N​∑i=1Nδ\(ki,vi\),ai​\(q\)≔aki​\(q∣PN\),andy​\(q\)≔Att⁡\(q∣PN\)\.P\_\{N\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\delta\_\{\(k\_\{i\},v\_\{i\}\)\},\\qquad a\_\{i\}\(q\)\\coloneqq a\_\{k\_\{i\}\}\(q\\mid P\_\{N\}\),\\qquad\\text\{and\}\\qquad y\(q\)\\coloneqq\\operatorname\{Att\}\(q\\mid P\_\{N\}\)\.By the definition,

tr⁡\(ΣPN,δq\)=1N​∑i=1N\[V2​\(ai​\(q\)−1\)2\+ai​\(q\)2​‖vi−y​\(q\)‖22\]\.\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\delta\_\{q\}\}\)=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\left\[V^\{2\}\(a\_\{i\}\(q\)\-1\)^\{2\}\+a\_\{i\}\(q\)^\{2\}\\\|v\_\{i\}\-y\(q\)\\\|\_\{2\}^\{2\}\\right\]\.
Atq=0q=0, we haveai​\(0\)=1a\_\{i\}\(0\)=1for allii, andAtt⁡\(0∣PN\)=0\\operatorname\{Att\}\(0\\mid P\_\{N\}\)=0by[Lemma˜B\.5](https://arxiv.org/html/2607.01520#A2.Thmtheorem5)\. Hence

tr⁡\(ΣPN,δ0\)=1N​∑i=1N‖vi‖22=ρ02​V2≍V2\.\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\delta\_\{0\}\}\)=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\\|v\_\{i\}\\\|\_\{2\}^\{2\}=\\rho\_\{0\}^\{2\}V^\{2\}\\asymp V^\{2\}\.
Now, fixk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\)\. By[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4), the keys are distinct, sok=kik=k\_\{i\}for a uniquei∈\[N\]i\\in\[N\]\. Since∑ℓ=1Naℓ​\(ki\)=N\\sum\_\{\\ell=1\}^\{N\}a\_\{\\ell\}\(k\_\{i\}\)=N, the sharpness condition gives

∑ℓ≠iaℓ​\(ki\)≤N−1/2,and henceai​\(ki\)≥N−N−1/2\.\\sum\_\{\\ell\\neq i\}a\_\{\\ell\}\(k\_\{i\}\)\\leq N^\{\-1/2\},\\qquad\\text\{and hence\}\\qquad a\_\{i\}\(k\_\{i\}\)\\geq N\-N^\{\-1/2\}\.Therefore, the scalar part alone gives

tr⁡\(ΣPN,δki\)≥V2N​\(ai​\(ki\)−1\)2≳V2​N\.\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\delta\_\{k\_\{i\}\}\}\)\\geq\\frac\{V^\{2\}\}\{N\}\(a\_\{i\}\(k\_\{i\}\)\-1\)^\{2\}\\gtrsim V^\{2\}N\.Conversely,

∑ℓ=1Naℓ​\(ki\)2=ai​\(ki\)2\+∑ℓ≠iaℓ​\(ki\)2≤N2\+\(∑ℓ≠iaℓ​\(ki\)\)2≤N2\+1\.\\sum\_\{\\ell=1\}^\{N\}a\_\{\\ell\}\(k\_\{i\}\)^\{2\}=a\_\{i\}\(k\_\{i\}\)^\{2\}\+\\sum\_\{\\ell\\neq i\}a\_\{\\ell\}\(k\_\{i\}\)^\{2\}\\leq N^\{2\}\+\\left\(\\sum\_\{\\ell\\neq i\}a\_\{\\ell\}\(k\_\{i\}\)\\right\)^\{2\}\\leq N^\{2\}\+1\.Also, by[Lemma˜B\.5](https://arxiv.org/html/2607.01520#A2.Thmtheorem5)and the boundary condition of[Definition˜B\.3](https://arxiv.org/html/2607.01520#A2.Thmtheorem3),

‖y​\(ki\)‖2≤Vand‖vℓ‖2≤V\\\|y\(k\_\{i\}\)\\\|\_\{2\}\\leq V\\qquad\\text\{and\}\\qquad\\\|v\_\{\\ell\}\\\|\_\{2\}\\leq Vfor allℓ∈\[N\]\\ell\\in\[N\], and hence‖vℓ−y​\(ki\)‖2≤2​V\\\|v\_\{\\ell\}\-y\(k\_\{i\}\)\\\|\_\{2\}\\leq 2V\. Thus

tr⁡\(ΣPN,δki\)\\displaystyle\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\delta\_\{k\_\{i\}\}\}\)≤V2N​∑ℓ=1N\(aℓ​\(ki\)−1\)2\+4​V2N​∑ℓ=1Naℓ​\(ki\)2\\displaystyle\\leq\\frac\{V^\{2\}\}\{N\}\\sum\_\{\\ell=1\}^\{N\}\(a\_\{\\ell\}\(k\_\{i\}\)\-1\)^\{2\}\+\\frac\{4V^\{2\}\}\{N\}\\sum\_\{\\ell=1\}^\{N\}a\_\{\\ell\}\(k\_\{i\}\)^\{2\}≲V2N​\(N\+∑ℓ=1Naℓ​\(ki\)2\)≲V2​N\.\\displaystyle\\lesssim\\frac\{V^\{2\}\}\{N\}\\left\(N\+\\sum\_\{\\ell=1\}^\{N\}a\_\{\\ell\}\(k\_\{i\}\)^\{2\}\\right\)\\lesssim V^\{2\}N\.SinceN=2​KN=2K, this proves

tr⁡\(ΣPN,δk\)≍V2​K\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\delta\_\{k\}\}\)\\asymp V^\{2\}Kfor allk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\)\.

Finally, the trace is affine in the query distribution, so

tr⁡\(ΣPN,p​δk\+\(1−p\)​δ0\)\\displaystyle\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},p\\delta\_\{k\}\+\(1\-p\)\\delta\_\{0\}\}\)=p​tr⁡\(ΣPN,δk\)\+\(1−p\)​tr⁡\(ΣPN,δ0\)\\displaystyle=p\\,\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\delta\_\{k\}\}\)\+\(1\-p\)\\,\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\delta\_\{0\}\}\)≍p​V2​K\+\(1−p\)​V2≍p​V2​K\+V2,\\displaystyle\\asymp pV^\{2\}K\+\(1\-p\)V^\{2\}\\asymp pV^\{2\}K\+V^\{2\},for allp∈\[0,1\]p\\in\[0,1\]\. That proves the claim\. ∎

###### Proof of[Section˜4\.3\.2](https://arxiv.org/html/2607.01520#S4.SS3.SSS2)\.

LetN≔2​KN\\coloneqq 2K, and let\(PN,νN\)\(P\_\{N\},\\nu\_\{N\}\)be the lookup instance from[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4)\. Set

p≔TV2​K\.p\\coloneqq\\frac\{T\}\{V^\{2\}K\}\.SinceT∈\[V2,V2​K\]T\\in\[V^\{2\},V^\{2\}K\], we havep∈\[1/K,1\]p\\in\[1/K,1\]\. For everyk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\), set

νk≔p​δk\+\(1−p\)​δ0,\\nu\_\{k\}\\coloneqq p\\,\\delta\_\{k\}\+\(1\-p\)\\,\\delta\_\{0\},and define

ℱ≔\{\(PN,νk\)∣k∈supp⁡\(νN\)\}\.\\mathcal\{F\}\\coloneqq\\\{\(P\_\{N\},\\nu\_\{k\}\)\\mid k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\)\\\}\.By[Lemma˜B\.4](https://arxiv.org/html/2607.01520#A2.Thmtheorem4),ℱ⊆𝒫2​K⋆​\(𝒳\)×𝒫​\(𝒬\)\\mathcal\{F\}\\subseteq\\mathcal\{P\}\_\{2K\}^\{\\star\}\(\\mathcal\{X\}\)\\times\\mathcal\{P\}\(\\mathcal\{Q\}\)\. By[Lemma˜B\.9](https://arxiv.org/html/2607.01520#A2.Thmtheorem9), for everyk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\),

tr⁡\(ΣPN,νk\)≍p​V2​K\+V2=T\+V2≍T,\\operatorname\{tr\}\(\\Sigma\_\{P\_\{N\},\\nu\_\{k\}\}\)\\asymp pV^\{2\}K\+V^\{2\}=T\+V^\{2\}\\asymp T,where the last comparison usesT≥V2T\\geq V^\{2\}\. Hence, every\(P,ν\)∈ℱ\(P,\\nu\)\\in\\mathcal\{F\}satisfies

tr⁡\(ΣP,ν\)≍T\.\\operatorname\{tr\}\(\\Sigma\_\{P,\\nu\}\)\\asymp T\.
We next prove the error lower bound\. Fix any query\-agnosticKK\-atomic compressorAA, and letP^\\hat\{P\}be its random output on the common contextPNP\_\{N\}\. For every realization ofP^\\hat\{P\},

1N​∑k∈supp⁡\(νN\)ℰPN,νk​\(P^\)\\displaystyle\\phantom\{=\}\\frac\{1\}\{N\}\\sum\_\{k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\)\}\\mathcal\{E\}\_\{P\_\{N\},\\nu\_\{k\}\}\\bigl\(\\hat\{P\}\\bigr\)=1N​∑k∈supp⁡\(νN\)\[p​‖Att⁡\(k∣PN\)−Att⁡\(k∣P^\)‖22\+\(1−p\)​‖Att⁡\(0∣PN\)−Att⁡\(0∣P^\)‖22\]\\displaystyle=\\frac\{1\}\{N\}\\sum\_\{k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\)\}\\left\[p\\\|\\operatorname\{Att\}\(k\\mid P\_\{N\}\)\-\\operatorname\{Att\}\(k\\mid\\hat\{P\}\)\\\|\_\{2\}^\{2\}\+\(1\-p\)\\\|\\operatorname\{Att\}\(0\\mid P\_\{N\}\)\-\\operatorname\{Att\}\(0\\mid\\hat\{P\}\)\\\|\_\{2\}^\{2\}\\right\]≥p​ℰPN,νN​\(P^\)≳p​V2=TK,\\displaystyle\\geq p\\,\\mathcal\{E\}\_\{P\_\{N\},\\nu\_\{N\}\}\\bigl\(\\hat\{P\}\\bigr\)\\gtrsim pV^\{2\}=\\frac\{T\}\{K\},where the last inequality uses[Lemma˜B\.6](https://arxiv.org/html/2607.01520#A2.Thmtheorem6)\. Taking expectation over the randomness ofAAgives

1N​∑k∈supp⁡\(νN\)𝔼⁡\[ℰPN,νk​\(P^\)\]≳TK\.\\frac\{1\}\{N\}\\sum\_\{k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\)\}\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\mathcal\{E\}\_\{P\_\{N\},\\nu\_\{k\}\}\\bigl\(\\hat\{P\}\\bigr\)\\right\]\\gtrsim\\frac\{T\}\{K\}\.Hence,

sup\(P,ν\)∈ℱ𝔼⁡\[ℰP,ν​\(A​\(P\)\)\]≳TK\.\\sup\_\{\(P,\\nu\)\\in\\mathcal\{F\}\}\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\mathcal\{E\}\_\{P,\\nu\}\\bigl\(A\(P\)\\bigr\)\\right\]\\gtrsim\\frac\{T\}\{K\}\.Taking the infimum over all query\-agnosticKK\-atomic compressors gives the claim\.

Finally, we prove that the same family is exactly approximable by query\-awareKK\-atomic compressors\. By[Lemma˜B\.8](https://arxiv.org/html/2607.01520#A2.Thmtheorem8), for everyk∈supp⁡\(νN\)k\\in\\operatorname\{supp\}\(\\nu\_\{N\}\), there existsP^k∈𝒫2​\(𝒳\)⊆𝒫K​\(𝒳\)\\hat\{P\}\_\{k\}\\in\\mathcal\{P\}\_\{2\}\(\\mathcal\{X\}\)\\subseteq\\mathcal\{P\}\_\{K\}\(\\mathcal\{X\}\)such that

ℰPN,νk​\(P^k\)=0\.\\mathcal\{E\}\_\{P\_\{N\},\\nu\_\{k\}\}\\bigl\(\\hat\{P\}\_\{k\}\\bigr\)=0\.Since the query\-aware compressor may depend onνk\\nu\_\{k\}, this gives

ℜ​\(𝒜Kaw,ℱ\)=0\.\\mathfrak\{R\}\\\!\\left\(\\mathcal\{A\}\_\{K\}^\{\\mathrm\{aw\}\},\\mathcal\{F\}\\right\)=0\.

## Appendix CAlgorithms

This appendix gives proofs and instantiations for the algorithmic framework of[Section˜5](https://arxiv.org/html/2607.01520#S5)\.[Algorithm˜1](https://arxiv.org/html/2607.01520#alg1)and[Algorithm˜2](https://arxiv.org/html/2607.01520#alg2)are parameterized by a local compression subroutine𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}\. Any choice for𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}compressing a context from2​K2KtoKKpoints yields an efficient and parallel KV compression algorithm for causally masked prefill via[Algorithm˜1](https://arxiv.org/html/2607.01520#alg1)and autoregressive decoding via[Algorithm˜2](https://arxiv.org/html/2607.01520#alg2)\. Recall that we use the following design criteria for𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}, under which we obtain guarantees on the compression quality\.

When𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}satisfies these criteria, the global merge\-reduce scheme matches the minimax\-optimal query\-agnostic trace risk, up to logarithmic degradation in time\. To obtain stronger guarantees in the query\-agnostic regime, additional structure is necessary\. Recall that we capture such structure with the following assumption\.

Under this assumption, there is a reference geometry induced by the feature mapΦ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}that is comparable, on balanced reweightings of the current context, to the query\-visible response geometry inℋν\\mathcal\{H\}\_\{\\nu\}\. Query\-agnostic compressors can then use the geometry ofℋ\\mathcal\{H\}as aν\\nu\-independent proxy forℋν\\mathcal\{H\}\_\{\\nu\}\.

##### Roadmap\.

This section is organized as follows\. In[Section˜C\.1](https://arxiv.org/html/2607.01520#A3.SS1), we discuss the choice ofΦ\\Phiand give a natural softmax\-attention instantiation for which[Section˜5](https://arxiv.org/html/2607.01520#S5)holds in stylized but informative settings\. In[Section˜C\.2](https://arxiv.org/html/2607.01520#A3.SS2), we show that[Section˜5\.2](https://arxiv.org/html/2607.01520#S5.SS2)can guide concrete algorithm design by constructing two local reducers satisfying the required criteria\.[Section˜C\.3](https://arxiv.org/html/2607.01520#A3.SS3)proves[Section˜5\.2](https://arxiv.org/html/2607.01520#S5.SS2), which lifts these local guarantees to the full prefill and decoding schemes, with a strengthened spectral bound under[Section˜5](https://arxiv.org/html/2607.01520#S5)\. Finally,[Section˜C\.4](https://arxiv.org/html/2607.01520#A3.SS4)proves[Section˜5\.1](https://arxiv.org/html/2607.01520#S5.SS1)on the efficiency of the global merge\-reduce scheme\.

##### Notation\.

We briefly recall the notation used throughout this section\. We work with a stream\(ki,vi\)i≥1\(k\_\{i\},v\_\{i\}\)\_\{i\\geq 1\}of key–value pairs\. For an index setI⊆ℕI\\subseteq\\mathbb\{N\}, we writePIP\_\{I\}for the context measure associated with\(ki,vi\)i∈I\(k\_\{i\},v\_\{i\}\)\_\{i\\in I\}, and abbreviatePt≔P\{1,…,t\}P\_\{t\}\\coloneqq P\_\{\\\{1,\\ldots,t\\\}\}\. We also use the unnormalized mass

w​\(I\)≔∑i∈Iexp⁡\(‖ki‖2/\(2​dk\)\),w\(I\)\\coloneqq\\sum\_\{i\\in I\}\\exp\(\\\|k\_\{i\}\\\|^\{2\}/\(2\\sqrt\{d\_\{k\}\}\)\),which comes from the Gaussian\-kernel form of softmax attention in[Equation˜2](https://arxiv.org/html/2607.01520#S3.E2)\. Finally, we fix a bounded feature mapΦ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}for this appendix and use the notationμPΦ\\mu\_\{P\}^\{\\Phi\}andΣPΦ\\Sigma\_\{P\}^\{\\Phi\}from[Section˜5\.2](https://arxiv.org/html/2607.01520#S5.SS2)for context measuresP∈𝒫fin​\(𝒳\)P\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)\.

### C\.1Choice of feature mapΦ\\Phi

The choice of feature mapΦ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}determines which structure a query\-agnostic compressor can exploit beyond random sampling\. While there is no single choice that results in a good proxy for all query distributions, a natural query\-agnostic choice is obtained from the two linear quantities that define attention before normalization: the unnormalized, value\-weighted numerator and the scalar normalizer\.

Concretely, we choose

Φ:𝒳→ℋ,\(k,v\)↦\(κ​\(⋅,k\)⊗v,V​κ​\(⋅,k\)\),\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\},\\qquad\(k,v\)\\mapsto\\bigl\(\\kappa\(\\cdot,k\)\\otimes v,\\;V\\,\\kappa\(\\cdot,k\)\\bigr\),\(9\)where

ℋ≔ℋκ⊗ℝdv⊕ℋκ\\mathcal\{H\}\\coloneqq\\mathcal\{H\}\_\{\\kappa\}\\otimes\\mathbb\{R\}^\{d\_\{v\}\}\\oplus\\mathcal\{H\}\_\{\\kappa\}is constructed from the feature spaceℋκ\\mathcal\{H\}\_\{\\kappa\}of the Gaussian kernelκ\\kappafrom[Section˜3](https://arxiv.org/html/2607.01520#S3)\. As for the response profiles in[Equation˜4](https://arxiv.org/html/2607.01520#S3.E4), the factorVVputs the normalizer component on the same scale as the value component\.

Intuitively,[Section˜5](https://arxiv.org/html/2607.01520#S5)then requires that the query distribution sees the same context directions as the reference features\. If a reweighting of the context substantially changes the numerator or normalizer of[Equation˜2](https://arxiv.org/html/2607.01520#S3.E2), then this change should also be visible in the attention responses under queriesq∼νq\\sim\\nu, and, conversely, a reweighting that is small in these quantities should not produce a large response\.

Formally, the response profileΓP\\Gamma\_\{P\}measures the query\-visible effect of a context reweighting after attention normalization and averaging overq∼νq\\sim\\nu\. For the reference features in[Equation˜9](https://arxiv.org/html/2607.01520#A3.E9),[Section˜5](https://arxiv.org/html/2607.01520#S5)requires that, on balanced reweightings ofsupp⁡\(P\)\\operatorname\{supp\}\(P\),

‖∫𝒳Φ​d​σ‖ℋ2≍‖∫𝒳ΓP​d​σ‖ℋν2\.\\left\\\|\\int\_\{\\mathcal\{X\}\}\\Phi\\,\\mathop\{\}\\\!\\mathrm\{d\}\\sigma\\right\\\|\_\{\\mathcal\{H\}\}^\{2\}\\asymp\\left\\\|\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\}\\,\\mathop\{\}\\\!\\mathrm\{d\}\\sigma\\right\\\|\_\{\\mathcal\{H\}\_\{\\nu\}\}^\{2\}\.Thus, large perturbations to the pre\-normalization numerator or normalizer are exactly the perturbations that produce large query\-visible changes in the attention response\. When this is the case, a query\-agnostic compressor can exploitΦ\\Phias a hint at theν\\nu\-dependent geometry, and beat the lower bound barrier of[Section˜4\.3\.2](https://arxiv.org/html/2607.01520#S4.SS3.SSS2)\.

However, this assumption need not always hold\. In particular, it fails on the hard instances of[Appendix˜B](https://arxiv.org/html/2607.01520#A2)used to prove lower bounds\. The following examples illustrate when this comparison is a reasonable model of the attention head and when it should not be expected to hold\.

###### Example C\.1\.

Suppose a contextPPconsists ofmmwell\-separated clusters of nearly interchangeable tokens, and the query distributionν\\nuassigns comparable mass to queries that attend to each cluster\. Then, for the reference features in[Equation˜9](https://arxiv.org/html/2607.01520#A3.E9),[Section˜5](https://arxiv.org/html/2607.01520#S5)holds up to the within\-cluster approximation error\.

###### Example C\.2\.

Suppose the keys insupp⁡\(P\)\\operatorname\{supp\}\(P\)lie on a low\-dimensional semantic manifold, values vary smoothly along this manifold, andν\\nuconsists of queries whose attention weights vary on the same scale\. Then, the reference geometry induced byΦ\\Phiis comparable to the response geometry ofΓP\\Gamma\_\{P\}\.

###### Example C\.3\.

Suppose a contextPPcontains two well\-separated groups of tokens, but queries fromν\\nuattend only to the first group\. Then, moving mass inside the second group can be large in the reference geometry induced byΦ\\Phiwhile nearly invisible in response geometry, so[Section˜5](https://arxiv.org/html/2607.01520#S5)fails\.

### C\.2Admissible reducers

We instantiate the abstract design criteria of[Section˜5\.2](https://arxiv.org/html/2607.01520#S5.SS2)by giving two concrete local reducers\. Both algorithms operate on a candidate contextQ∈𝒫2​K​\(𝒳\)Q\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\)through the reference feature mapΦ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}\.

The first reducer is a constrained rounding procedure derived directly from the sparse barycenter approximation theorem in[Appendix˜A](https://arxiv.org/html/2607.01520#A1)\. It provides the cleanest theoretical guarantee: for every target rankr≤\(K−3\)/3r\\leq\(K\-3\)/3, it is\(Φ,r,0\)\(\\Phi,r,0\)\-admissible\. Its drawback is that the constrained rounding step is inherently sequential in the protected dimensionrr, making it poorly suited to modern parallel hardware\.

To compensate for this, we also provide a more practical, clustering\-based reducer\. It maps efficiently to parallel hardware, but it is only\(Φ,r,τ\)\(\\Phi,r,\\tau\)\-admissible with a non\-zeroτ\\tauterm determined by the achieved clustering error\.

Both reducers require an orthonormal basis of the top\-rreigenspace of the centered covarianceΣQΦ\\Sigma\_\{Q\}^\{\\Phi\}\. For computational efficiency, practical implementations may replace this exact eigenspace by an approximate one, with the resulting loss absorbed into the tail condition of[Section˜5\.2](https://arxiv.org/html/2607.01520#S5.SS2)\. Finally, although both reducers are stated using the abstract feature mapΦ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}, they can be implemented using only inner products⟨Φ​\(z\),Φ​\(z′\)⟩ℋ\\langle\\Phi\(z\),\\Phi\(z^\{\\prime\}\)\\rangle\_\{\\mathcal\{H\}\}\. The framework therefore also covers infinite\-dimensional feature spaces\.

Algorithm 5RoundReduce1:context measure

Q=∑i=12​Kqi​δzi∈𝒫2​K​\(𝒳\)Q=\\sum\_\{i=1\}^\{2K\}q\_\{i\}\\,\\delta\_\{z\_\{i\}\}\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\)
2:feature map

Φ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}
3:integer

r≤\(K−3\)/3r\\leq\(K\-3\)/3
4:

Φ¯←∑i=12​Kqi​Φ​\(zi\)\\bar\{\\Phi\}\\leftarrow\\sum\_\{i=1\}^\{2K\}q\_\{i\}\\,\\Phi\(z\_\{i\}\)
5:

ΣQΦ←∑i=12​Kqi​\(Φ​\(zi\)−Φ¯\)⊗\(Φ​\(zi\)−Φ¯\)\\Sigma\_\{Q\}^\{\\Phi\}\\leftarrow\\sum\_\{i=1\}^\{2K\}q\_\{i\}\\,\(\\Phi\(z\_\{i\}\)\-\\bar\{\\Phi\}\)\\otimes\(\\Phi\(z\_\{i\}\)\-\\bar\{\\Phi\}\)
6:

U←Topr⁡\(ΣQΦ\)U\\leftarrow\\operatorname\{Top\}\_\{r\}\(\\Sigma\_\{Q\}^\{\\Phi\}\)⊳\\trianglerighttop\-rreigenspace ofΣQΦ\\Sigma\_\{Q\}^\{\\Phi\}

7:

μ←∑i=12​Kqi​δΦ​\(zi\)\\mu\\leftarrow\\sum\_\{i=1\}^\{2K\}q\_\{i\}\\,\\delta\_\{\\Phi\(z\_\{i\}\)\}⊳\\trianglerightpushforward

8:

μ^←ProtectedSparsify​\(μ,U,K−2​r−2\)=∑i=12​Kq^i​δΦ​\(zi\)\\hat\{\\mu\}\\leftarrow\\textsc\{ProtectedSparsify\}\(\\mu,U,K\-2r\-2\)=\\sum\_\{i=1\}^\{2K\}\\hat\{q\}\_\{i\}\\,\\delta\_\{\\Phi\(z\_\{i\}\)\}⊳\\triangleright[Algorithm˜4](https://arxiv.org/html/2607.01520#alg4)

9:return

Q^←∑i=12​Kq^i​δzi\\hat\{Q\}\\leftarrow\\sum\_\{i=1\}^\{2K\}\\hat\{q\}\_\{i\}\\,\\delta\_\{z\_\{i\}\}⊳\\trianglerightpullback

Algorithm 6ClusterReduce1:context measure

Q=∑i=12​Kqi​δzi∈𝒫2​K​\(𝒳\)Q=\\sum\_\{i=1\}^\{2K\}q\_\{i\}\\,\\delta\_\{z\_\{i\}\}\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\)
2:feature map

Φ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}
3:integer

r≥0r\\geq 0
4:

H←\{i:qi\>1/K\}H\\leftarrow\\\{i:q\_\{i\}\>1/K\\\},

L←\[2​K\]∖HL\\leftarrow\[2K\]\\setminus H⊳\\trianglerightfilter out heavy\-weight atoms

5:if

∑i∈Lqi=0\\sum\_\{i\\in L\}q\_\{i\}=0then

6:return

QQ
7:

m←K−\|H\|m\\leftarrow K\-\|H\|,

β←∑i∈Lqi\\beta\\leftarrow\\sum\_\{i\\in L\}q\_\{i\},

ρ←β/m\\rho\\leftarrow\\beta/m
8:

Φ¯←∑iqi​Φ​\(zi\)\\bar\{\\Phi\}\\leftarrow\\sum\_\{i\}q\_\{i\}\\,\\Phi\(z\_\{i\}\)
9:

ΣQΦ←∑iqi​\(Φ​\(zi\)−Φ¯\)⊗\(Φ​\(zi\)−Φ¯\)\\Sigma\_\{Q\}^\{\\Phi\}\\leftarrow\\sum\_\{i\}q\_\{i\}\\,\(\\Phi\(z\_\{i\}\)\-\\bar\{\\Phi\}\)\\otimes\(\\Phi\(z\_\{i\}\)\-\\bar\{\\Phi\}\)
10:

U←Topr⁡\(ΣQΦ\)U\\leftarrow\\operatorname\{Top\}\_\{r\}\(\\Sigma\_\{Q\}^\{\\Phi\}\)⊳\\trianglerighttop\-rreigenspace ofΣQΦ\\Sigma\_\{Q\}^\{\\Phi\}

11:for

i∈Li\\in Ldo⊳\\trianglerightcluster atoms projected ontoUU

12:

ui←ΠU​\(Φ​\(zi\)−Φ¯\)u\_\{i\}\\leftarrow\\Pi\_\{U\}\(\\Phi\(z\_\{i\}\)\-\\bar\{\\Phi\}\)
13:

pi←qi/ρp\_\{i\}\\leftarrow q\_\{i\}/\\rho
14:

X←ClusterSlots​\(\(pi,ui\)i∈L,m\)∈\[0,1\]m×\|L\|X\\leftarrow\\textsc\{ClusterSlots\}\(\(p\_\{i\},u\_\{i\}\)\_\{i\\in L\},m\)\\in\[0,1\]^\{m\\times\|L\|\}
15:for

a=1,…,ma=1,\\ldots,mdo⊳\\trianglerightsample one atom per cluster

16:sample

Ia∈LI\_\{a\}\\in Lwith

ℙ​\(Ia=i\)=Xa​i\\mathbb\{P\}\(I\_\{a\}=i\)=X\_\{ai\}
17:return

Q^←∑i∈Hqi​δzi\+∑a=1mρ​δzIa\.\\hat\{Q\}\\leftarrow\\sum\_\{i\\in H\}q\_\{i\}\\delta\_\{z\_\{i\}\}\+\\sum\_\{a=1\}^\{m\}\\rho\\,\\delta\_\{z\_\{I\_\{a\}\}\}\.

#### C\.2\.1Perfect admissibility via constrained rounding

[Algorithm˜5](https://arxiv.org/html/2607.01520#alg5)is a direct implementation of the Hilbert\-space sparsification argument in[Appendix˜A](https://arxiv.org/html/2607.01520#A1)\. Given a context measureQ∈𝒫2​K​\(𝒳\)Q\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\), the reducer preserves the featuresΦ\\Phiexactly in the leading eigenspace ofΣQΦ\\Sigma\_\{Q\}^\{\\Phi\}, and sparsifies the measure only in the orthogonal complement\. Because of that, the reducer is\(Φ,r,0\)\(\\Phi,r,0\)\-admissible\.

###### Theorem C\.4\.

FixK≥3K\\geq 3andr≤\(K−3\)/3r\\leq\(K\-3\)/3\. Letℋ\\mathcal\{H\}be a Hilbert space andΦ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}bounded\. Then,RoundReduce​\(⋅,Φ,r\)\\textsc\{RoundReduce\}\(\\cdot,\\Phi,r\)is\(Φ,r,0\)\(\\Phi,r,0\)\-admissible\.

###### Proof\.

Fix

Q=∑i=12​Kqi​δzi∈𝒫2​K​\(𝒳\),Q=\\sum\_\{i=1\}^\{2K\}q\_\{i\}\\,\\delta\_\{z\_\{i\}\}\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\),setm≔K−2​r−2m\\coloneqq K\-2r\-2, and use the notation of[Algorithm˜5](https://arxiv.org/html/2607.01520#alg5)\. Sincer≤\(K−3\)/3r\\leq\(K\-3\)/3, we havem≥K/3m\\geq K/3\.

RoundReduce​\(Q,Φ,r\)\\textsc\{RoundReduce\}\(Q,\\Phi,r\)appliesProtectedSparsifyto the pushforward

μ=∑i=1sqi​δΦ​\(zi\)\\mu=\\sum\_\{i=1\}^\{s\}q\_\{i\}\\delta\_\{\\Phi\(z\_\{i\}\)\}with protected subspace

U=Topr⁡\(ΣQΦ\)U=\\operatorname\{Top\}\_\{r\}\(\\Sigma\_\{Q\}^\{\\Phi\}\)and sparsification budgetmm\.

By[Proposition˜A\.3](https://arxiv.org/html/2607.01520#A1.Thmtheorem3),

\|supp⁡\(Q^\)\|≤m\+2​dim\(U\)\+2≤K\|\\operatorname\{supp\}\(\\hat\{Q\}\)\|\\leq m\+2\\dim\(U\)\+2\\leq Kalmost surely, andsupp⁡\(Q^\)⊆supp⁡\(Q\)\\operatorname\{supp\}\(\\hat\{Q\}\)\\subseteq\\operatorname\{supp\}\(Q\)\. Hence,

Q^∈𝒫K​\(supp⁡\(Q\)\)\.\\hat\{Q\}\\in\\mathcal\{P\}\_\{K\}\(\\operatorname\{supp\}\(Q\)\)\.[Proposition˜A\.3](https://arxiv.org/html/2607.01520#A1.Thmtheorem3)also gives𝔼⁡\[μ^\]=μ\\operatorname\{\\mathbb\{E\}\}\[\\hat\{\\mu\}\]=\\muand therefore𝔼⁡\[Q^\]=Q\\operatorname\{\\mathbb\{E\}\}\[\\hat\{Q\}\]=Q\. That proves unbiasedness\.

Let𝒢\\mathcal\{G\}be a Hilbert space and letΨ:𝒳→𝒢\\Psi:\\mathcal\{X\}\\to\\mathcal\{G\}be bounded\. Applying the variance bound in[Proposition˜A\.3](https://arxiv.org/html/2607.01520#A1.Thmtheorem3)withyi=Ψ​\(zi\)y\_\{i\}=\\Psi\(z\_\{i\}\)gives

𝔼⁡‖∫𝒳Ψ​d​\(Q^−Q\)‖𝒢2≤209​m​tr⁡\(ΣQΨ\)≤203​K​tr⁡\(ΣQΨ\)\.\\operatorname\{\\mathbb\{E\}\}\\Bigl\\\|\\int\_\{\\mathcal\{X\}\}\\Psi\\,d\(\\hat\{Q\}\-Q\)\\Bigr\\\|\_\{\\mathcal\{G\}\}^\{2\}\\leq\\frac\{20\}\{9m\}\\operatorname\{tr\}\(\\Sigma\_\{Q\}^\{\\Psi\}\)\\leq\\frac\{20\}\{3K\}\\operatorname\{tr\}\(\\Sigma\_\{Q\}^\{\\Psi\}\)\.
For the feature mapΦ\\Phi, let

ηΦ≔∫𝒳Φ​d​\(Q^−Q\)\.\\eta\_\{\\Phi\}\\coloneqq\\int\_\{\\mathcal\{X\}\}\\Phi\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{Q\}\-Q\)\.By the protected covariance bound in[Proposition˜A\.3](https://arxiv.org/html/2607.01520#A1.Thmtheorem3),

𝔼⁡\[ηΦ⊗ηΦ\]⪯209​m​\(I−ΠU\)​ΣQΦ​\(I−ΠU\)\.\\operatorname\{\\mathbb\{E\}\}\[\\eta\_\{\\Phi\}\\otimes\\eta\_\{\\Phi\}\]\\preceq\\frac\{20\}\{9m\}\(I\-\\Pi\_\{U\}\)\\Sigma\_\{Q\}^\{\\Phi\}\(I\-\\Pi\_\{U\}\)\.Taking traces and usingU=Topr⁡\(ΣQΦ\)U=\\operatorname\{Top\}\_\{r\}\(\\Sigma\_\{Q\}^\{\\Phi\}\), we obtain

𝔼⁡‖∫𝒳Φ​d​\(Q^−Q\)‖ℋ2≤209​m​tailr⁡\(ΣQΦ\)≤203​K​tailr⁡\(ΣQΦ\)\.\\operatorname\{\\mathbb\{E\}\}\\Bigl\\\|\\int\_\{\\mathcal\{X\}\}\\Phi\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{Q\}\-Q\)\\Bigr\\\|\_\{\\mathcal\{H\}\}^\{2\}\\leq\\frac\{20\}\{9m\}\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{Q\}^\{\\Phi\}\)\\leq\\frac\{20\}\{3K\}\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{Q\}^\{\\Phi\}\)\.
Thus, all design criteria hold for everyQ∈𝒫2​K​\(X\)Q\\in\\mathcal\{P\}\_\{2K\}\(X\), makingRoundReduce​\(⋅,Φ,r\)\\textsc\{RoundReduce\}\(\\cdot,\\Phi,r\)\(Φ,r,0\)\(\\Phi,r,0\)\-admissible\. ∎

#### C\.2\.2Practical admissibility via clustering

[Algorithm˜6](https://arxiv.org/html/2607.01520#alg6)provides an admissible reducer based on clustering\. On a context measureQQ, the algorithm protects the principal directions ofΣQΦ\\Sigma\_\{Q\}^\{\\Phi\}by projecting the features onto the leading eigenspace ofΣQΦ\\Sigma\_\{Q\}^\{\\Phi\}and clustering them in these protected coordinates\.

For the clustering step, any finite\-dimensional clustering subroutineClusterSlotscan be used, provided it produces a balanced latent\-slot assignment\. Concretely, using the notation of[Algorithm˜6](https://arxiv.org/html/2607.01520#alg6), the subroutine receives the weighted protected coordinates\(pi,ui\)i∈L\(p\_\{i\},u\_\{i\}\)\_\{i\\in L\}, wherepi=qi/ρp\_\{i\}=q\_\{i\}/\\rho, and returns a matrixX∈\[0,1\]m×\|L\|X\\in\[0,1\]^\{m\\times\|L\|\}\. RowaaofXXis the sampling distribution used by latent slotaa, while the column constraints ensure that each light atom receives the correct expected mass\. Formally,XXmust satisfy

∑i∈LXa​i=1and∑a=1mXa​i=pi\.\\sum\_\{i\\in L\}X\_\{ai\}=1\\qquad\\text\{and\}\\qquad\\sum\_\{a=1\}^\{m\}X\_\{ai\}=p\_\{i\}\.
For example, one might use weighted k\-means to group nearby protected coordinates and then sample representatives from those groups in proportion to their total mass, withXXrecording the resulting sampling probabilities\.

Naturally, the quality of the compressed summary depends on the quality of the clustering\. Using the notation of[Algorithm˜6](https://arxiv.org/html/2607.01520#alg6), define the clustering error of the call toClusterSlotsby

𝒞​\(Q\)≔1K​∑a=1m∑i∈LXa​i​‖ui−u¯a‖ℋ2,u¯a≔∑i∈LXa​i​ui\.\\mathcal\{C\}\(Q\)\\coloneqq\\frac\{1\}\{K\}\\sum\_\{a=1\}^\{m\}\\sum\_\{i\\in L\}X\_\{ai\}\\\|u\_\{i\}\-\\bar\{u\}\_\{a\}\\\|\_\{\\mathcal\{H\}\}^\{2\},\\qquad\\bar\{u\}\_\{a\}\\coloneqq\\sum\_\{i\\in L\}X\_\{ai\}u\_\{i\}\.If the light mass is zero, we set𝒞ClusterSlots​\(Q\)=0\\mathcal\{C\}\_\{\\textsc\{ClusterSlots\}\}\(Q\)=0\.

When this clustering error is small,ClusterReduceis admissible, as the following theorem shows\.

###### Theorem C\.5\.

FixK≥1K\\geq 1andr≥0r\\geq 0\. Letℋ\\mathcal\{H\}be a Hilbert space andΦ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}bounded\. Assume thatClusterSlotssatisfies𝒞​\(Q\)≤τ\\mathcal\{C\}\(Q\)\\leq\\taufor every inputQ∈𝒫2​K​\(𝒳\)Q\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\)\. Then,ClusterReduce​\(⋅,Φ,r\)\\textsc\{ClusterReduce\}\(\\cdot,\\Phi,r\)is\(Φ,r,τ\)\(\\Phi,r,\\tau\)\-admissible\.

###### Proof\.

Let

Q=∑i=12​Kqi​δzi∈𝒫2​K​\(𝒳\)\.Q=\\sum\_\{i=1\}^\{2K\}q\_\{i\}\\delta\_\{z\_\{i\}\}\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\)\.
Use the notation of[Algorithm˜6](https://arxiv.org/html/2607.01520#alg6)\. If the light mass is zero, thenQQmust already belong to𝒫K​\(𝒳\)\\mathcal\{P\}\_\{K\}\(\\mathcal\{X\}\)\. The algorithm then returnsQQexactly, so all claims are immediate\.

Assume therefore thatβ\>0\\beta\>0\. Since eachi∈Hi\\in Hsatisfiesqi\>1/Kq\_\{i\}\>1/K,

β=1−∑i∈Hqi<1−\|H\|K=K−\|H\|K=mK\.\\beta=1\-\\sum\_\{i\\in H\}q\_\{i\}<1\-\\frac\{\|H\|\}\{K\}=\\frac\{K\-\|H\|\}\{K\}=\\frac\{m\}\{K\}\.Thus,ρ=βm≤1K\\rho=\\frac\{\\beta\}\{m\}\\leq\\frac\{1\}\{K\}

The output is supported onsupp⁡\(Q\)\\operatorname\{supp\}\(Q\)and uses at most\|H\|\+m=K\|H\|\+m=Katoms, soQ^∈𝒫K​\(supp⁡\(Q\)\)\\hat\{Q\}\\in\\mathcal\{P\}\_\{K\}\(\\operatorname\{supp\}\(Q\)\)almost surely\. Moreover, the heavy atoms are kept exactly, and for everyi∈Li\\in L,

𝔼⁡\[mass assigned to​zi\]=ρ​∑a=1mXa​i=ρ​pi=qi\.\\operatorname\{\\mathbb\{E\}\}\[\\text\{mass assigned to \}z\_\{i\}\]=\\rho\\sum\_\{a=1\}^\{m\}X\_\{ai\}=\\rho p\_\{i\}=q\_\{i\}\.Hence𝔼⁡\[Q^\]=Q\\operatorname\{\\mathbb\{E\}\}\[\\hat\{Q\}\]=Q\. That proves unbiasedness\.

Let𝒢\\mathcal\{G\}be a Hilbert space and letΨ:𝒳→𝒢\\Psi:\\mathcal\{X\}\\to\\mathcal\{G\}be bounded\. Write

yi≔Ψ​\(zi\),y¯≔∫𝒳Ψ​d​Q\.y\_\{i\}\\coloneqq\\Psi\(z\_\{i\}\),\\qquad\\bar\{y\}\\coloneqq\\int\_\{\\mathcal\{X\}\}\\Psi\\mathop\{\}\\\!\\mathrm\{d\}Q\.The heavy atoms contribute no error, while the light slots give

∫𝒳Ψ​d​\(Q^−Q\)=ρ​∑a=1m\(yIa−∑i∈LXa​i​yi\)\.\\int\_\{\\mathcal\{X\}\}\\Psi\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{Q\}\-Q\)=\\rho\\sum\_\{a=1\}^\{m\}\\left\(y\_\{I\_\{a\}\}\-\\sum\_\{i\\in L\}X\_\{ai\}y\_\{i\}\\right\)\.The summands are independent and centered\. Therefore

𝔼⁡\[‖∫𝒳Ψ​d​\(Q^−Q\)‖𝒢2\]\\displaystyle\\operatorname\{\\mathbb\{E\}\}\\left\[\\left\\\|\\int\_\{\\mathcal\{X\}\}\\Psi\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{Q\}\-Q\)\\right\\\|\_\{\\mathcal\{G\}\}^\{2\}\\right\]=ρ2​∑a=1m𝔼⁡‖yIa−∑i∈LXa​i​yi‖𝒢2\\displaystyle=\\rho^\{2\}\\sum\_\{a=1\}^\{m\}\\operatorname\{\\mathbb\{E\}\}\\left\\\|y\_\{I\_\{a\}\}\-\\sum\_\{i\\in L\}X\_\{ai\}y\_\{i\}\\right\\\|\_\{\\mathcal\{G\}\}^\{2\}≤ρ2​∑a=1m∑i∈LXa​i​‖yi−y¯‖𝒢2\\displaystyle\\leq\\rho^\{2\}\\sum\_\{a=1\}^\{m\}\\sum\_\{i\\in L\}X\_\{ai\}\\\|y\_\{i\}\-\\bar\{y\}\\\|\_\{\\mathcal\{G\}\}^\{2\}=ρ2​∑i∈Lpi​‖yi−y¯‖𝒢2\\displaystyle=\\rho^\{2\}\\sum\_\{i\\in L\}p\_\{i\}\\\|y\_\{i\}\-\\bar\{y\}\\\|\_\{\\mathcal\{G\}\}^\{2\}=ρ​∑i∈Lqi​‖yi−y¯‖𝒢2\\displaystyle=\\rho\\sum\_\{i\\in L\}q\_\{i\}\\\|y\_\{i\}\-\\bar\{y\}\\\|\_\{\\mathcal\{G\}\}^\{2\}≤1K​∑i=12​Kqi​‖yi−y¯‖𝒢2\\displaystyle\\leq\\frac\{1\}\{K\}\\sum\_\{i=1\}^\{2K\}q\_\{i\}\\\|y\_\{i\}\-\\bar\{y\}\\\|\_\{\\mathcal\{G\}\}^\{2\}=1K​tr⁡\(ΣQΨ\)\.\\displaystyle=\\frac\{1\}\{K\}\\operatorname\{tr\}\(\\Sigma\_\{Q\}^\{\\Psi\}\)\.This proves the universal trace criterion\.

It remains to prove the distinguishedΦ\\Phi\-criterion\. Let

xi≔Φ​\(zi\),x¯≔∫𝒳Φ​d​Q,ξi≔xi−x¯,x\_\{i\}\\coloneqq\\Phi\(z\_\{i\}\),\\qquad\\bar\{x\}\\coloneqq\\int\_\{\\mathcal\{X\}\}\\Phi\\mathop\{\}\\\!\\mathrm\{d\}Q,\\qquad\\xi\_\{i\}\\coloneqq x\_\{i\}\-\\bar\{x\},and let

U≔Topr⁡\(ΣQΦ\)\.U\\coloneqq\\operatorname\{Top\}\_\{r\}\(\\Sigma\_\{Q\}^\{\\Phi\}\)\.Then,ui=ΠU​ξiu\_\{i\}=\\Pi\_\{U\}\\xi\_\{i\}for alli∈Li\\in L\. As above,

ηΦ≔∫𝒳Φ​d​\(Q^−Q\)=ρ​∑a=1m\(ξIa−∑i∈LXa​i​ξi\)\.\\eta\_\{\\Phi\}\\coloneqq\\int\_\{\\mathcal\{X\}\}\\Phi\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{Q\}\-Q\)=\\rho\\sum\_\{a=1\}^\{m\}\\left\(\\xi\_\{I\_\{a\}\}\-\\sum\_\{i\\in L\}X\_\{ai\}\\xi\_\{i\}\\right\)\.Decomposing orthogonally alongU⊕U⟂U\\oplus U^\{\\perp\}, the protected part is

𝔼⁡‖ΠU​ηΦ‖ℋ2=ρ2​∑a=1m∑i∈LXa​i​‖ui−u¯a‖ℋ2\.\\operatorname\{\\mathbb\{E\}\}\\\|\\Pi\_\{U\}\\eta\_\{\\Phi\}\\\|\_\{\\mathcal\{H\}\}^\{2\}=\\rho^\{2\}\\sum\_\{a=1\}^\{m\}\\sum\_\{i\\in L\}X\_\{ai\}\\\|u\_\{i\}\-\\bar\{u\}\_\{a\}\\\|\_\{\\mathcal\{H\}\}^\{2\}\.By the definition of𝒞​\(Q\)\\mathcal\{C\}\(Q\)andρ≤1/K\\rho\\leq 1/K,

𝔼⁡‖ΠU​ηΦ‖ℋ2=ρ2​K​𝒞​\(Q\)≤𝒞​\(Q\)K≤τK\.\\operatorname\{\\mathbb\{E\}\}\\\|\\Pi\_\{U\}\\eta\_\{\\Phi\}\\\|\_\{\\mathcal\{H\}\}^\{2\}=\\rho^\{2\}K\\,\\mathcal\{C\}\(Q\)\\leq\\frac\{\\mathcal\{C\}\(Q\)\}\{K\}\\leq\\frac\{\\tau\}\{K\}\.For the residual part, the same variance estimate used above gives

𝔼⁡‖\(I−ΠU\)​ηΦ‖ℋ2\\displaystyle\\operatorname\{\\mathbb\{E\}\}\\\|\(I\-\\Pi\_\{U\}\)\\eta\_\{\\Phi\}\\\|\_\{\\mathcal\{H\}\}^\{2\}≤ρ​∑i∈Lqi​‖\(I−ΠU\)​ξi‖ℋ2\\displaystyle\\leq\\rho\\sum\_\{i\\in L\}q\_\{i\}\\\|\(I\-\\Pi\_\{U\}\)\\xi\_\{i\}\\\|\_\{\\mathcal\{H\}\}^\{2\}≤1K​tr⁡\(\(I−ΠU\)​ΣQΦ\)\.\\displaystyle\\leq\\frac\{1\}\{K\}\\operatorname\{tr\}\\\!\\left\(\(I\-\\Pi\_\{U\}\)\\Sigma\_\{Q\}^\{\\Phi\}\\right\)\.SinceU=Topr⁡\(ΣQΦ\)U=\\operatorname\{Top\}\_\{r\}\(\\Sigma\_\{Q\}^\{\\Phi\}\),

tr⁡\(\(I−ΠU\)​ΣQΦ\)=tailr⁡\(ΣQΦ\)\.\\operatorname\{tr\}\\\!\\left\(\(I\-\\Pi\_\{U\}\)\\Sigma\_\{Q\}^\{\\Phi\}\\right\)=\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{Q\}^\{\\Phi\}\)\.Combining the protected and residual parts yields

𝔼⁡\[‖∫𝒳Φ​d​\(Q^−Q\)‖ℋ2\]≤1K​\(tailr⁡\(ΣQΦ\)\+τ\)\.\\operatorname\{\\mathbb\{E\}\}\\left\[\\left\\\|\\int\_\{\\mathcal\{X\}\}\\Phi\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{Q\}\-Q\)\\right\\\|\_\{\\mathcal\{H\}\}^\{2\}\\right\]\\leq\\frac\{1\}\{K\}\\bigl\(\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{Q\}^\{\\Phi\}\)\+\\tau\\bigr\)\.
Thus, all design criteria hold for everyQ∈𝒫2​K​\(X\)Q\\in\\mathcal\{P\}\_\{2K\}\(X\), makingClusterReduce​\(⋅,Φ,r\)\\textsc\{ClusterReduce\}\(\\cdot,\\Phi,r\)\(Φ,r,τ\)\(\\Phi,r,\\tau\)\-admissible\. ∎

### C\.3Global risk guarantees

This section proves the risk guarantee of[Section˜5\.2](https://arxiv.org/html/2607.01520#S5.SS2)for the chunked prefill and decoding algorithms\. Both schedules build summaries in a merge\-reduce tree: starting from token chunks ofKKtokens, disjoint summaries are merged, their weighted union is compressed back toKKatoms, and the unfinished current chunk is appended exactly\. We show that, for admissible local compressors𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}, the approximation error accumulates linearly with the depth of the merge\-reduce tree, which then transfers to the claimed risk guarantees\.

#### C\.3\.1Geometry transfer

𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}is designed in the fixed reference geometry induced byΦ:𝒳→ℋ\\Phi:\\mathcal\{X\}\\to\\mathcal\{H\}\. In contrast, the desired risk bound is expressed in the geometry induced byΓP:𝒳→ℋν\\Gamma\_\{P\}:\\mathcal\{X\}\\to\\mathcal\{H\}\_\{\\nu\}, which depends on both the exact contextPPand the query distributionν\\nu\. We record here the transfer guarantees from the covarianceΣPΦ\\Sigma^\{\\Phi\}\_\{P\}inℋ\\mathcal\{H\}to the covarianceΣP,ν\\Sigma\_\{P,\\nu\}inℋν\\mathcal\{H\}\_\{\\nu\}\. Concretely,[Lemma˜C\.6](https://arxiv.org/html/2607.01520#A3.Thmtheorem6)shows that, under[Section˜5](https://arxiv.org/html/2607.01520#S5), the spectral decay ofΣPΦ\\Sigma\_\{P\}^\{\\Phi\}is controlled by the spectral decay ofΣP,ν\\Sigma\_\{P,\\nu\}in the reference geometry ofΦ\\Phi\.

###### Lemma C\.6\.

Suppose[Section˜5](https://arxiv.org/html/2607.01520#S5)holds forP,νP,\\nuand the chosen feature mapΦ\\Phi\. Then,

tailr⁡\(ΣPΦ\)≲tailr⁡\(ΣP,ν\)\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{P\}^\{\\Phi\}\)\\lesssim\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{P,\\nu\}\)for everyr≥0r\\geq 0\.

###### Proof\.

Let

L02​\(P\)≔\{f∈L2​\(P\)∣∫𝒳f​d​P=0\}\\textstyle L\_\{0\}^\{2\}\(P\)\\coloneqq\\left\\\{f\\in L^\{2\}\(P\)\\mid\\int\_\{\\mathcal\{X\}\}f\\mathop\{\}\\\!\\mathrm\{d\}P=0\\right\\\}and define

TΦ​f≔∫𝒳f​\(Φ−μPΦ\)​d​P,TΓ​f≔∫𝒳f​ΓP​d​P\.\\textstyle T\_\{\\Phi\}f\\coloneqq\\int\_\{\\mathcal\{X\}\}f\(\\Phi\-\\mu\_\{P\}^\{\\Phi\}\)\\mathop\{\}\\\!\\mathrm\{d\}P,\\qquad T\_\{\\Gamma\}f\\coloneqq\\int\_\{\\mathcal\{X\}\}f\\Gamma\_\{P\}\\mathop\{\}\\\!\\mathrm\{d\}P\.Sincef​PfPis a signed, zero\-mass measure supported onsupp⁡\(P\)\\operatorname\{supp\}\(P\),[Section˜5](https://arxiv.org/html/2607.01520#S5)gives

‖TΦ​f‖ℋ2≲‖TΓ​f‖ℋν2\\\|T\_\{\\Phi\}f\\\|\_\{\\mathcal\{H\}\}^\{2\}\\lesssim\\\|T\_\{\\Gamma\}f\\\|\_\{\\mathcal\{H\}\_\{\\nu\}\}^\{2\}for allf∈L02​\(P\)f\\in L\_\{0\}^\{2\}\(P\)\. Equivalently,

TΦ∗​TΦ⪯C​TΓ∗​TΓ\.T\_\{\\Phi\}^\{\*\}T\_\{\\Phi\}\\preceq C\\,T\_\{\\Gamma\}^\{\*\}T\_\{\\Gamma\}\.for a universal constantC\>0C\>0\. Moreover,

TΦ​TΦ∗=ΣPΦandTΓ​TΓ∗=ΣP,ν,T\_\{\\Phi\}T\_\{\\Phi\}^\{\*\}=\\Sigma\_\{P\}^\{\\Phi\}\\qquad\\text\{and\}\\qquad T\_\{\\Gamma\}T\_\{\\Gamma\}^\{\*\}=\\Sigma\_\{P,\\nu\},where the second identity uses∫ΓP​𝑑P=0\\int\\Gamma\_\{P\}\\,dP=0\. SinceT​T∗TT^\{\*\}andT∗​TT^\{\*\}Thave the same eigenvalues,

tailr⁡\(ΣPΦ\)=tailr⁡\(TΦ∗​TΦ\)≲tailr⁡\(TΓ∗​TΓ\)=tailr⁡\(ΣP,ν\)\.\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{P\}^\{\\Phi\}\)=\\operatorname\{tail\}\_\{r\}\(T\_\{\\Phi\}^\{\*\}T\_\{\\Phi\}\)\\lesssim\\operatorname\{tail\}\_\{r\}\(T\_\{\\Gamma\}^\{\*\}T\_\{\\Gamma\}\)=\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{P,\\nu\}\)\.

#### C\.3\.2Risk propagation through merge\-reduce trees

We now turn the one\-step guarantees of𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}into guarantees for full causal prefixes\. To that end, we show that the design criteria in[Section˜5\.2](https://arxiv.org/html/2607.01520#S5.SS2)for𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}— unbiasedness of the summary and second\-moment control of its feature defect — propagate through the merge\-reduce recursion\. To express this invariant, we introduce the notion of a*valid summary*, which is a random context measure of a token block that satisfies the same criteria, with a parameter counting the number of reducer layers on its dependency path\. Each node of the merge\-reduce tree preserves validity, increasing this parameter by at most one\. The global risk bound in[Section˜5\.2](https://arxiv.org/html/2607.01520#S5.SS2)then follows by recognizing the prefill and decoding summaries as outputs of such trees, whose depth is logarithmic in the number of completed chunks\.

###### Definition C\.7\(Valid summary\)\.

Letd,r,τ≥0d,r,\\tau\\geq 0andA⊆ℕA\\subseteq\\mathbb\{N\}\. We say that a random context measureSAS\_\{A\}is*\(d,r,τ\)\(d,r,\\tau\)\-valid*forAAif the following hold:

1. \(i\)𝔼⁡\[SA\]=PA\\operatorname\{\\mathbb\{E\}\}\[S\_\{A\}\]=P\_\{A\}
2. \(ii\)𝔼⁡\[‖∫𝒳Ψ​d​\(SA−PA\)‖𝒢2\]≲dK​tr⁡\(ΣPAΨ\)\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\\|\\int\_\{\\mathcal\{X\}\}\\Psi\\mathop\{\}\\\!\\mathrm\{d\}\(S\_\{A\}\-P\_\{A\}\)\\\|\_\{\\mathcal\{G\}\}^\{2\}\\bigr\]\\lesssim\\frac\{d\}\{K\}\\operatorname\{tr\}\(\\Sigma\_\{P\_\{A\}\}^\{\\Psi\}\)for all Hilbert spaces𝒢\\mathcal\{G\}and boundedΨ:𝒳→𝒢\\Psi:\\mathcal\{X\}\\to\\mathcal\{G\}
3. \(iii\)𝔼⁡\[‖∫𝒳Φ​d​\(SA−PA\)‖ℋ2\]≲dK​\(tailr⁡\(ΣPAΦ\)\+τ\)\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\\|\\int\_\{\\mathcal\{X\}\}\\Phi\\mathop\{\}\\\!\\mathrm\{d\}\(S\_\{A\}\-P\_\{A\}\)\\\|\_\{\\mathcal\{H\}\}^\{2\}\\bigr\]\\lesssim\\frac\{d\}\{K\}\\bigl\(\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{P\_\{A\}\}^\{\\Phi\}\)\+\\tau\\bigr\)

Note that this resembles directly the design criteria of a\(Φ,r,τ\)\(\\Phi,r,\\tau\)\-admissible reducer, as defined in[Section˜5\.2](https://arxiv.org/html/2607.01520#S5.SS2)\. In particular, if𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}is\(Φ,r,τ\)\(\\Phi,r,\\tau\)\-admissible andPA∈𝒫2​K​\(𝒳\)P\_\{A\}\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\)is the context measure of the tokens inA⊆ℕA\\subseteq\\mathbb\{N\}, then𝖱𝖤𝖣𝖴𝖢𝖤⁡\(PA\)\\operatorname\{\\mathsf\{REDUCE\}\}\(P\_\{A\}\)is\(1,r,τ\)\(1,r,\\tau\)\-valid forAA\.

We briefly record the following fact that we use repeatedly later on\.

###### Lemma C\.8\.

LetP∈𝒫fin​\(𝒳\)P\\in\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)and letQQbe a random variable taking values in𝒫fin​\(𝒳\)\\mathcal\{P\}\_\{\\mathrm\{fin\}\}\(\\mathcal\{X\}\)\. If𝔼⁡\[Q\]=P\\operatorname\{\\mathbb\{E\}\}\[Q\]=P, then𝔼⁡\[ΣQΨ\]⪯ΣPΨ\\operatorname\{\\mathbb\{E\}\}\[\\Sigma^\{\\Psi\}\_\{Q\}\]\\preceq\\Sigma^\{\\Psi\}\_\{P\}for any Hilbert space𝒢\\mathcal\{G\}and boundedΨ:𝒳→𝒢\\Psi:\\mathcal\{X\}\\to\\mathcal\{G\}\.

###### Proof\.

Let𝒢\\mathcal\{G\}be a Hilbert space andΨ:𝒳→𝒢\\Psi:\\mathcal\{X\}\\to\\mathcal\{G\}be bounded\. Then,

𝔼⁡\[ΣQΨ\]\\displaystyle\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\Sigma\_\{Q\}^\{\\Psi\}\\right\]=𝔼⁡\[∫𝒳Ψ⊗Ψ​d​Q\]−𝔼⁡\[μQΨ⊗μQΨ\]\\displaystyle=\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\int\_\{\\mathcal\{X\}\}\\Psi\\otimes\\Psi\\mathop\{\}\\\!\\mathrm\{d\}Q\\right\]\-\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\mu\_\{Q\}^\{\\Psi\}\\otimes\\mu\_\{Q\}^\{\\Psi\}\\right\]=∫𝒳Ψ⊗Ψ​d​P−𝔼⁡\[μQΨ⊗μQΨ\]\\displaystyle=\\int\_\{\\mathcal\{X\}\}\\Psi\\otimes\\Psi\\mathop\{\}\\\!\\mathrm\{d\}P\-\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\mu\_\{Q\}^\{\\Psi\}\\otimes\\mu\_\{Q\}^\{\\Psi\}\\right\]=∫𝒳Ψ⊗Ψ​d​P−𝔼⁡\[\(μQΨ−𝔼⁡\[μQΨ\]\)⊗\(μQΨ−𝔼⁡\[μQΨ\]\)\]−𝔼⁡\[μQΨ\]⊗𝔼⁡\[μQΨ\]\\displaystyle=\\int\_\{\\mathcal\{X\}\}\\Psi\\otimes\\Psi\\mathop\{\}\\\!\\mathrm\{d\}P\-\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\left\(\\mu\_\{Q\}^\{\\Psi\}\-\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\mu\_\{Q\}^\{\\Psi\}\\right\]\\right\)\\otimes\\left\(\\mu\_\{Q\}^\{\\Psi\}\-\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\mu\_\{Q\}^\{\\Psi\}\\right\]\\right\)\\right\]\-\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\mu\_\{Q\}^\{\\Psi\}\\right\]\\otimes\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\mu\_\{Q\}^\{\\Psi\}\\right\]=∫𝒳Ψ⊗Ψ​d​P−𝔼⁡\[\(μQΨ−μPΨ\)⊗\(μQΨ−μPΨ\)\]−μPΨ⊗μPΨ\\displaystyle=\\int\_\{\\mathcal\{X\}\}\\Psi\\otimes\\Psi\\mathop\{\}\\\!\\mathrm\{d\}P\-\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\left\(\\mu\_\{Q\}^\{\\Psi\}\-\\mu\_\{P\}^\{\\Psi\}\\right\)\\otimes\\left\(\\mu\_\{Q\}^\{\\Psi\}\-\\mu\_\{P\}^\{\\Psi\}\\right\)\\right\]\-\\mu\_\{P\}^\{\\Psi\}\\otimes\\mu\_\{P\}^\{\\Psi\}=ΣPΨ−𝔼⁡\[\(μQΨ−μPΨ\)⊗\(μQΨ−μPΨ\)\]\\displaystyle=\\Sigma\_\{P\}^\{\\Psi\}\-\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\left\(\\mu\_\{Q\}^\{\\Psi\}\-\\mu\_\{P\}^\{\\Psi\}\\right\)\\otimes\\left\(\\mu\_\{Q\}^\{\\Psi\}\-\\mu\_\{P\}^\{\\Psi\}\\right\)\\right\]⪯ΣPΨ\.\\displaystyle\\preceq\\Sigma\_\{P\}^\{\\Psi\}\.∎

We now consider the operation at a single node in the merge\-reduce tree\. Given two summaries of two disjoint blocks of tokens, we form a compressed summary of their union by compressing their weighted combination using𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}\. When the two input summaries are\(d,r,τ\)\(d,r,\\tau\)\-valid, the resulting summary is\(d\+1,r,τ\)\(d\+1,r,\\tau\)\-valid, thereby propagating the guarantees up the merge\-reduce tree\.

###### Lemma C\.9\.

LetA,B⊆ℕA,B\\subseteq\\mathbb\{N\}be finite, nonempty, and disjoint\. LetSAS\_\{A\}be\(dA,r,τ\)\(d\_\{A\},r,\\tau\)\-valid forAA, and letSBS\_\{B\}be\(dB,r,τ\)\(d\_\{B\},r,\\tau\)\-valid forBB\. Assume that𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}is\(Φ,r,τ\)\(\\Phi,r,\\tau\)\-admissible and thatSAS\_\{A\}andSBS\_\{B\}are independent\. Define

Q≔w​\(A\)w​\(A\)\+w​\(B\)​SA\+w​\(B\)w​\(A\)\+w​\(B\)​SB\.Q\\coloneqq\\frac\{w\(A\)\}\{w\(A\)\+w\(B\)\}S\_\{A\}\+\\frac\{w\(B\)\}\{w\(A\)\+w\(B\)\}S\_\{B\}\.IfQ∈𝒫2​K​\(𝒳\)Q\\in\\mathcal\{P\}\_\{2K\}\(\\mathcal\{X\}\), then𝖱𝖤𝖣𝖴𝖢𝖤⁡\(Q\)\\operatorname\{\\mathsf\{REDUCE\}\}\(Q\)is\(1\+max⁡\{dA,dB\},r,τ\)\(1\+\\max\\\{d\_\{A\},d\_\{B\}\\\},r,\\tau\)\-valid forA∪BA\\cup B\.

###### Proof\.

LetSA∪B≔𝖱𝖤𝖣𝖴𝖢𝖤⁡\(Q\)S\_\{A\\cup B\}\\coloneqq\\operatorname\{\\mathsf\{REDUCE\}\}\(Q\), and write

P≔PA∪B=α​PA\+β​PBP\\coloneqq P\_\{A\\cup B\}=\\alpha P\_\{A\}\+\\beta P\_\{B\}forα≔w​\(A\)/\(w​\(A\)\+w​\(B\)\)\\alpha\\coloneqq w\(A\)/\(w\(A\)\+w\(B\)\)andβ≔w​\(B\)/\(w​\(A\)\+w​\(B\)\)\\beta\\coloneqq w\(B\)/\(w\(A\)\+w\(B\)\)\. Additionally, introduce the defects

σA≔SA−PA,σB≔SB−PB,σA∪B≔SA∪B−PA∪B,\\sigma\_\{A\}\\coloneqq S\_\{A\}\-P\_\{A\},\\qquad\\sigma\_\{B\}\\coloneqq S\_\{B\}\-P\_\{B\},\\qquad\\sigma\_\{A\\cup B\}\\coloneqq S\_\{A\\cup B\}\-P\_\{A\\cup B\},and

η≔SA∪B−Q\\eta\\coloneqq S\_\{A\\cup B\}\-Qwhich are all zero\-mass, signed measures on𝒳\\mathcal\{X\}\.

For a Hilbert space𝒢\\mathcal\{G\}and boundedΨ:𝒳→𝒢\\Psi:\\mathcal\{X\}\\to\\mathcal\{G\}, introduce the notation

Ψ​\[σ\]≔∫𝒳Ψ​d​σ\\Psi\[\\sigma\]\\coloneqq\\int\_\{\\mathcal\{X\}\}\\Psi\\mathop\{\}\\\!\\mathrm\{d\}\\sigmafor zero\-mass, signed measuresσ\\sigmaon𝒳\\mathcal\{X\}\.

We next prove orthogonality of the defects after taking expectation\. Condition on the already constructed summariesSA,SBS\_\{A\},S\_\{B\}\. ThenQQ,σA\\sigma\_\{A\}, andσB\\sigma\_\{B\}are fixed, and the only remaining randomness is the fresh call to𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}\. Admissibility of𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}gives𝔼⁡\[η∣SA,SB\]=0\\operatorname\{\\mathbb\{E\}\}\[\\eta\\mid S\_\{A\},S\_\{B\}\]=0\. Hence,

𝔼⁡\[⟨Ψ​\[η\],Ψ​\[σA\]⟩𝒢\]=𝔼⁡\[⟨𝔼⁡\[Ψ​\[η\]∣SA,SB\],Ψ​\[σA\]⟩𝒢\]=0,\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\langle\\Psi\[\\eta\],\\Psi\[\\sigma\_\{A\}\]\\rangle\_\{\\mathcal\{G\}\}\\bigr\]=\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\langle\\operatorname\{\\mathbb\{E\}\}\[\\Psi\[\\eta\]\\mid S\_\{A\},S\_\{B\}\],\\Psi\[\\sigma\_\{A\}\]\\rangle\_\{\\mathcal\{G\}\}\\bigr\]=0,and similarly for𝔼⁡\[⟨Ψ​\[η\],Ψ​\[σB\]⟩𝒢\]\\operatorname\{\\mathbb\{E\}\}\[\\langle\\Psi\[\\eta\],\\Psi\[\\sigma\_\{B\}\]\\rangle\_\{\\mathcal\{G\}\}\]\. Moreover,SAS\_\{A\}andSBS\_\{B\}are independent and validity ofSAS\_\{A\}andSBS\_\{B\}imply

𝔼⁡\[Ψ​\[σA\]\]=𝔼⁡\[Ψ​\[σB\]\]=0\.\\operatorname\{\\mathbb\{E\}\}\[\\Psi\[\\sigma\_\{A\}\]\]=\\operatorname\{\\mathbb\{E\}\}\[\\Psi\[\\sigma\_\{B\}\]\]=0\.Therefore,

𝔼⁡\[⟨Ψ​\[σA\],Ψ​\[σB\]⟩𝒢\]=⟨𝔼⁡\[Ψ​\[σA\]\],𝔼⁡\[Ψ​\[σB\]\]⟩𝒢=0\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\langle\\Psi\[\\sigma\_\{A\}\],\\Psi\[\\sigma\_\{B\}\]\\rangle\_\{\\mathcal\{G\}\}\\bigr\]=\\bigl\\langle\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\Psi\[\\sigma\_\{A\}\]\\bigr\],\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\Psi\[\\sigma\_\{B\}\]\\bigr\]\\bigr\\rangle\_\{\\mathcal\{G\}\}=0and thus expanding the square norm gives

𝔼⁡‖Ψ​\[σA∪B\]‖𝒢2=𝔼⁡‖Ψ​\[η\]‖𝒢2\+α2​𝔼⁡‖Ψ​\[σA\]‖𝒢2\+β2​𝔼⁡‖Ψ​\[σB\]‖𝒢2\.\\operatorname\{\\mathbb\{E\}\}\\bigl\\\|\\Psi\[\\sigma\_\{A\\cup B\}\]\\bigr\\\|\_\{\\mathcal\{G\}\}^\{2\}=\\operatorname\{\\mathbb\{E\}\}\\bigl\\\|\\Psi\[\\eta\]\\bigr\\\|\_\{\\mathcal\{G\}\}^\{2\}\+\\alpha^\{2\}\\,\\operatorname\{\\mathbb\{E\}\}\\bigl\\\|\\Psi\[\\sigma\_\{A\}\]\\bigr\\\|\_\{\\mathcal\{G\}\}^\{2\}\+\\beta^\{2\}\\,\\operatorname\{\\mathbb\{E\}\}\\bigl\\\|\\Psi\[\\sigma\_\{B\}\]\\bigr\\\|\_\{\\mathcal\{G\}\}^\{2\}\.\(9\)
We now prove the three validity conditions forSA∪BS\_\{A\\cup B\}\.

For unbiasedness,

𝔼⁡\[SA∪B\]=𝔼⁡\[𝔼⁡\[𝖱𝖤𝖣𝖴𝖢𝖤⁡\(Q\)∣SA,SB\]\]=𝔼⁡\[Q\]=α​PA\+β​PB=P,\\operatorname\{\\mathbb\{E\}\}\[S\_\{A\\cup B\}\]=\\operatorname\{\\mathbb\{E\}\}\[\\operatorname\{\\mathbb\{E\}\}\[\\operatorname\{\\mathsf\{REDUCE\}\}\(Q\)\\mid S\_\{A\},S\_\{B\}\]\]=\\operatorname\{\\mathbb\{E\}\}\[Q\]=\\alpha P\_\{A\}\+\\beta P\_\{B\}=P,which follows immediately from unbiasedness ofSAS\_\{A\}andSBS\_\{B\}\.

For trace control, we bound the three terms in[Equation˜9](https://arxiv.org/html/2607.01520#A3.E9a)\. Fix a Hilbert space𝒢\\mathcal\{G\}and a bounded mapΨ:𝒳→𝒢\\Psi:\\mathcal\{X\}\\to\\mathcal\{G\}\. Since𝔼⁡\[Q\]=P\\operatorname\{\\mathbb\{E\}\}\[Q\]=P,[Lemma˜C\.8](https://arxiv.org/html/2607.01520#A3.Thmtheorem8)gives𝔼⁡\[ΣQΨ\]⪯ΣPΨ\\operatorname\{\\mathbb\{E\}\}\[\\Sigma^\{\\Psi\}\_\{Q\}\]\\preceq\\Sigma^\{\\Psi\}\_\{P\}\. Conditioning onSA,SBS\_\{A\},S\_\{B\}, the inputQQis fixed and the remaining randomness is the fresh reducer randomness\. Thus, admissibility of𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}gives

𝔼⁡\[‖Ψ​\[η\]‖𝒢2\|SA,SB\]≲1K​tr⁡\(ΣQΨ\)\.\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\bigl\\\|\\Psi\[\\eta\]\\bigr\\\|\_\{\\mathcal\{G\}\}^\{2\}\\bigm\|S\_\{A\},S\_\{B\}\\bigr\]\\lesssim\\frac\{1\}\{K\}\\operatorname\{tr\}\\bigl\(\\Sigma\_\{Q\}^\{\\Psi\}\\bigr\)\.Taking expectations,

𝔼⁡‖Ψ​\[η\]‖𝒢2≲1K​𝔼⁡tr⁡\(ΣQΨ\)≤1K​tr⁡\(ΣPΨ\)\.\\operatorname\{\\mathbb\{E\}\}\\bigl\\\|\\Psi\[\\eta\]\\bigr\\\|\_\{\\mathcal\{G\}\}^\{2\}\\lesssim\\frac\{1\}\{K\}\\operatorname\{\\mathbb\{E\}\}\\operatorname\{tr\}\\bigl\(\\Sigma\_\{Q\}^\{\\Psi\}\\bigr\)\\leq\\frac\{1\}\{K\}\\operatorname\{tr\}\\bigl\(\\Sigma\_\{P\}^\{\\Psi\}\\bigr\)\.
By validity ofSAS\_\{A\}andSBS\_\{B\},

α2​𝔼⁡‖Ψ​\[σA\]‖𝒢2\+β2​𝔼⁡‖Ψ​\[σB\]‖𝒢2\\displaystyle\\alpha^\{2\}\\operatorname\{\\mathbb\{E\}\}\\bigl\\\|\\Psi\[\\sigma\_\{A\}\]\\bigr\\\|\_\{\\mathcal\{G\}\}^\{2\}\+\\beta^\{2\}\\operatorname\{\\mathbb\{E\}\}\\bigl\\\|\\Psi\[\\sigma\_\{B\}\]\\bigr\\\|\_\{\\mathcal\{G\}\}^\{2\}≲max⁡\{dA,dB\}K​\[α2​tr⁡\(ΣPAΨ\)\+β2​tr⁡\(ΣPBΨ\)\]\\displaystyle\\lesssim\\frac\{\\max\\\{d\_\{A\},d\_\{B\}\\\}\}\{K\}\\Bigl\[\\alpha^\{2\}\\operatorname\{tr\}\(\\Sigma^\{\\Psi\}\_\{P\_\{A\}\}\)\+\\beta^\{2\}\\operatorname\{tr\}\(\\Sigma^\{\\Psi\}\_\{P\_\{B\}\}\)\\Bigr\]≤max⁡\{dA,dB\}K​\[tr⁡\(α​ΣPAΨ\+β​ΣPBΨ\)\]\\displaystyle\\leq\\frac\{\\max\\\{d\_\{A\},d\_\{B\}\\\}\}\{K\}\\Bigl\[\\operatorname\{tr\}\\bigl\(\\alpha\\Sigma^\{\\Psi\}\_\{P\_\{A\}\}\+\\beta\\Sigma^\{\\Psi\}\_\{P\_\{B\}\}\\bigr\)\\Bigr\]≤max⁡\{dA,dB\}K​\[tr⁡\(ΣPΨ\)\],\\displaystyle\\leq\\frac\{\\max\\\{d\_\{A\},d\_\{B\}\\\}\}\{K\}\\Bigl\[\\operatorname\{tr\}\\bigl\(\\Sigma^\{\\Psi\}\_\{P\}\\bigr\)\\Bigr\],where we additionally useα2≤α\\alpha^\{2\}\\leq\\alpha,β2≤β\\beta^\{2\}\\leq\\beta, andα​ΣPAΨ\+β​ΣPBΨ⪯ΣPΨ\\alpha\\Sigma^\{\\Psi\}\_\{P\_\{A\}\}\+\\beta\\Sigma^\{\\Psi\}\_\{P\_\{B\}\}\\preceq\\Sigma^\{\\Psi\}\_\{P\}by[Lemma˜C\.8](https://arxiv.org/html/2607.01520#A3.Thmtheorem8)\. Combining this with[Equation˜9](https://arxiv.org/html/2607.01520#A3.E9a)and admissibility of𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}proves

𝔼⁡‖Ψ​\[σA∪B\]‖𝒢2≲\(1\+max⁡\{dA,dB\}K\)​tr⁡\(ΣPΨ\)\.\\operatorname\{\\mathbb\{E\}\}\\bigl\\\|\\Psi\[\\sigma\_\{A\\cup B\}\]\\bigr\\\|\_\{\\mathcal\{G\}\}^\{2\}\\lesssim\\left\(\\frac\{1\+\\max\\\{d\_\{A\},d\_\{B\}\\\}\}\{K\}\\right\)\\operatorname\{tr\}\\bigl\(\\Sigma^\{\\Psi\}\_\{P\}\\bigr\)\.SinceΨ\\Psiand𝒢\\mathcal\{G\}were arbitrary, that proves the trace condition\.

For tail control, condition onSA,SBS\_\{A\},S\_\{B\}\. Then,QQis fixed and the remaining randomness is the fresh reducer randomness\. Admissibility applied to this realized inputQQgives

𝔼⁡\[‖Φ​\[η\]‖ℋ2\|SA,SB\]≲1K​\(tailr⁡\(ΣQΦ\)\+τ\)\.\\operatorname\{\\mathbb\{E\}\}\\\!\\left\[\\\|\\Phi\[\\eta\]\\\|\_\{\\mathcal\{H\}\}^\{2\}\\bigm\|S\_\{A\},S\_\{B\}\\right\]\\lesssim\\frac\{1\}\{K\}\\left\(\\operatorname\{tail\}\_\{r\}\(\\Sigma^\{\\Phi\}\_\{Q\}\)\+\\tau\\right\)\.Taking expectations gives

𝔼⁡‖Φ​\[η\]‖ℋ2≲1K​\(𝔼⁡\[tailr⁡\(ΣQΦ\)\]\+τ\)\.\\operatorname\{\\mathbb\{E\}\}\\\|\\Phi\[\\eta\]\\\|\_\{\\mathcal\{H\}\}^\{2\}\\lesssim\\frac\{1\}\{K\}\\left\(\\operatorname\{\\mathbb\{E\}\}\[\\operatorname\{tail\}\_\{r\}\(\\Sigma^\{\\Phi\}\_\{Q\}\)\]\+\\tau\\right\)\.Since𝔼⁡\[Q\]=P\\operatorname\{\\mathbb\{E\}\}\[Q\]=P,[Lemma˜C\.8](https://arxiv.org/html/2607.01520#A3.Thmtheorem8)gives𝔼⁡\[ΣQΦ\]⪯ΣPΦ\\operatorname\{\\mathbb\{E\}\}\[\\Sigma^\{\\Phi\}\_\{Q\}\]\\preceq\\Sigma^\{\\Phi\}\_\{P\}\. By concavity and monotonicity oftailr\\operatorname\{tail\}\_\{r\},

𝔼⁡\[tailr⁡\(ΣQΦ\)\]≤tailr⁡\(𝔼⁡\[ΣQΦ\]\)≤tailr⁡\(ΣPΦ\)\.\\operatorname\{\\mathbb\{E\}\}\[\\operatorname\{tail\}\_\{r\}\(\\Sigma^\{\\Phi\}\_\{Q\}\)\]\\leq\\operatorname\{tail\}\_\{r\}\(\\operatorname\{\\mathbb\{E\}\}\[\\Sigma^\{\\Phi\}\_\{Q\}\]\)\\leq\\operatorname\{tail\}\_\{r\}\(\\Sigma^\{\\Phi\}\_\{P\}\)\.Hence

𝔼⁡‖Φ​\[η\]‖ℋ2≲1K​\(tailr⁡\(ΣPΦ\)\+τ\)\.\\operatorname\{\\mathbb\{E\}\}\\\|\\Phi\[\\eta\]\\\|\_\{\\mathcal\{H\}\}^\{2\}\\lesssim\\frac\{1\}\{K\}\\bigl\(\\operatorname\{tail\}\_\{r\}\(\\Sigma^\{\\Phi\}\_\{P\}\)\+\\tau\\bigr\)\.By validity ofSAS\_\{A\}andSBS\_\{B\},

α2​𝔼⁡‖Φ​\[σA\]‖ℋ2\+β2​𝔼⁡‖Φ​\[σB\]‖ℋ2\\displaystyle\\alpha^\{2\}\\operatorname\{\\mathbb\{E\}\}\\bigl\\\|\\Phi\[\\sigma\_\{A\}\]\\bigr\\\|\_\{\\mathcal\{H\}\}^\{2\}\+\\beta^\{2\}\\operatorname\{\\mathbb\{E\}\}\\bigl\\\|\\Phi\[\\sigma\_\{B\}\]\\bigr\\\|\_\{\\mathcal\{H\}\}^\{2\}≲max⁡\{dA,dB\}K​\[α2​\(tailr⁡\(ΣPAΦ\)\+τ\)\+β2​\(tailr⁡\(ΣPBΦ\)\+τ\)\]\\displaystyle\\lesssim\\frac\{\\max\\\{d\_\{A\},d\_\{B\}\\\}\}\{K\}\\Bigl\[\\alpha^\{2\}\(\\operatorname\{tail\}\_\{r\}\(\\Sigma^\{\\Phi\}\_\{P\_\{A\}\}\)\+\\tau\)\+\\beta^\{2\}\(\\operatorname\{tail\}\_\{r\}\(\\Sigma^\{\\Phi\}\_\{P\_\{B\}\}\)\+\\tau\)\\Bigr\]≤max⁡\{dA,dB\}K​\[α​tailr⁡\(ΣPAΦ\)\+β​tailr⁡\(ΣPBΦ\)\+τ\]\\displaystyle\\leq\\frac\{\\max\\\{d\_\{A\},d\_\{B\}\\\}\}\{K\}\\Bigl\[\\alpha\\,\\operatorname\{tail\}\_\{r\}\(\\Sigma^\{\\Phi\}\_\{P\_\{A\}\}\)\+\\beta\\,\\operatorname\{tail\}\_\{r\}\(\\Sigma^\{\\Phi\}\_\{P\_\{B\}\}\)\+\\tau\\Bigr\]≤max⁡\{dA,dB\}K​\[tailr⁡\(ΣPΦ\)\+τ\],\\displaystyle\\leq\\frac\{\\max\\\{d\_\{A\},d\_\{B\}\\\}\}\{K\}\\Bigl\[\\operatorname\{tail\}\_\{r\}\\bigl\(\\Sigma^\{\\Phi\}\_\{P\}\\bigr\)\+\\tau\\Bigr\],where we additionally useα2≤α\\alpha^\{2\}\\leq\\alpha,β2≤β\\beta^\{2\}\\leq\\beta,α​ΣPAΦ\+β​ΣPBΦ⪯ΣPΦ\\alpha\\Sigma^\{\\Phi\}\_\{P\_\{A\}\}\+\\beta\\Sigma^\{\\Phi\}\_\{P\_\{B\}\}\\preceq\\Sigma^\{\\Phi\}\_\{P\}by[Lemma˜C\.8](https://arxiv.org/html/2607.01520#A3.Thmtheorem8), and concavity oftailr\\operatorname\{tail\}\_\{r\}\. Plugging into[Equation˜9](https://arxiv.org/html/2607.01520#A3.E9a), we get

𝔼⁡‖Φ​\[σA∪B\]‖ℋ2≲1\+max⁡\{dA,dB\}K​\(tailr⁡\(ΣPΦ\)\+τ\)\.\\operatorname\{\\mathbb\{E\}\}\\bigl\\\|\\Phi\[\\sigma\_\{A\\cup B\}\]\\bigr\\\|\_\{\\mathcal\{H\}\}^\{2\}\\lesssim\\frac\{1\+\\max\\\{d\_\{A\},d\_\{B\}\\\}\}\{K\}\\bigl\(\\operatorname\{tail\}\_\{r\}\(\\Sigma^\{\\Phi\}\_\{P\}\)\+\\tau\\bigr\)\.That proves the tail condition\. Hence,SA∪BS\_\{A\\cup B\}is\(1\+max⁡\{dA,dB\},r,τ\)\(1\+\\max\\\{d\_\{A\},d\_\{B\}\\\},r,\\tau\)\-valid forA∪BA\\cup B\. ∎

Validity of a summary gives us the desired risk control, but only in terms of the fixed geometry ofℋ\\mathcal\{H\}\.[Proposition˜C\.10](https://arxiv.org/html/2607.01520#A3.Thmtheorem10)transfers this control to the attention error, which is measured in the geometry ofℋν\\mathcal\{H\}\_\{\\nu\}\.

###### Proposition C\.10\.

Fixν∈𝒫​\(𝒬\)\\nu\\in\\mathcal\{P\}\(\\mathcal\{Q\}\), and letP^t\\hat\{P\}\_\{t\}be a random context measure\. IfP^t\\hat\{P\}\_\{t\}is\(dt,r,τ\)\(d\_\{t\},r,\\tau\)\-valid for\{1,…,t\}\\\{1,\\ldots,t\\\}, then

𝔼⁡\[ℰPt,ν​\(P^t\)\]≲dtK​tr⁡\(ΣPt,ν\)\.\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\mathcal\{E\}\_\{P\_\{t\},\\nu\}\\bigl\(\\hat\{P\}\_\{t\}\\bigr\)\\bigr\]\\lesssim\\frac\{d\_\{t\}\}\{K\}\\operatorname\{tr\}\(\\Sigma\_\{P\_\{t\},\\nu\}\)\.If additionally\(Pt,ν\)\(P\_\{t\},\\nu\)satisfies[Section˜5](https://arxiv.org/html/2607.01520#S5), then

𝔼⁡\[ℰPt,ν​\(P^t\)\]≲dtK​\(tailr⁡\(ΣPt,ν\)\+τ\)\.\\operatorname\{\\mathbb\{E\}\}\\bigl\[\\mathcal\{E\}\_\{P\_\{t\},\\nu\}\\bigl\(\\hat\{P\}\_\{t\}\\bigr\)\\bigr\]\\lesssim\\frac\{d\_\{t\}\}\{K\}\\left\(\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{P\_\{t\},\\nu\}\)\+\\tau\\right\)\.

###### Proof\.

First, note that

ℰPt,ν​\(P^t\)​≲\(1\)​‖∫𝒳ΓPt​d​P^t‖ℋν2​=\(2\)​‖∫𝒳ΓPt​d​\(P^t−Pt\)‖ℋν2\\mathcal\{E\}\_\{P\_\{t\},\\nu\}\\bigl\(\\hat\{P\}\_\{t\}\\bigr\)\\overset\{\(1\)\}\{\\lesssim\}\\left\\\|\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\_\{t\}\}\\mathop\{\}\\\!\\mathrm\{d\}\\hat\{P\}\_\{t\}\\right\\\|\_\{\\mathcal\{H\}\_\{\\nu\}\}^\{2\}\\overset\{\(2\)\}\{=\}\\left\\\|\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\_\{t\}\}\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{P\}\_\{t\}\-P\_\{t\}\)\\right\\\|\_\{\\mathcal\{H\}\_\{\\nu\}\}^\{2\}\(10\)where\(1\)\(1\)uses[Lemma˜B\.1](https://arxiv.org/html/2607.01520#A2.Thmtheorem1), and\(2\)\(2\)uses∫𝒳ΓPt​d​Pt=0\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\_\{t\}\}\\mathop\{\}\\\!\\mathrm\{d\}P\_\{t\}=0\.

Using validity ofP^t\\hat\{P\}\_\{t\}applied toΓPt\\Gamma\_\{P\_\{t\}\}, it follows that

𝔼⁡ℰPt,ν​\(P^t\)≲dtK​tr⁡\(ΣPtΓPt\)=dtK​tr⁡\(ΣPt,ν\)\.\\operatorname\{\\mathbb\{E\}\}\\mathcal\{E\}\_\{P\_\{t\},\\nu\}\\bigl\(\\hat\{P\}\_\{t\}\\bigr\)\\lesssim\\frac\{d\_\{t\}\}\{K\}\\operatorname\{tr\}\(\\Sigma\_\{P\_\{t\}\}^\{\\Gamma\_\{P\_\{t\}\}\}\)=\\frac\{d\_\{t\}\}\{K\}\\operatorname\{tr\}\(\\Sigma\_\{P\_\{t\},\\nu\}\)\.That proves the first claim\.

Suppose now that[Section˜5](https://arxiv.org/html/2607.01520#S5)holds\. Then, using validity ofP^t\\hat\{P\}\_\{t\}and[Lemma˜C\.6](https://arxiv.org/html/2607.01520#A3.Thmtheorem6),

𝔼⁡ℰPt,ν​\(P^t\)​≲\([10](https://arxiv.org/html/2607.01520#A3.E10)\)​𝔼⁡‖∫𝒳ΓPt​d​\(P^t−Pt\)‖ℋν2≲dtK​\(tailr⁡\(ΣPtΦ\)\+τ\)≲dtK​\(tailr⁡\(ΣPt,ν\)\+τ\),\\operatorname\{\\mathbb\{E\}\}\\mathcal\{E\}\_\{P\_\{t\},\\nu\}\\bigl\(\\hat\{P\}\_\{t\}\\bigr\)\\overset\{\\eqref\{eq:algorithm\-risk:transfer\}\}\{\\lesssim\}\\operatorname\{\\mathbb\{E\}\}\\left\\\|\\int\_\{\\mathcal\{X\}\}\\Gamma\_\{P\_\{t\}\}\\mathop\{\}\\\!\\mathrm\{d\}\(\\hat\{P\}\_\{t\}\-P\_\{t\}\)\\right\\\|\_\{\\mathcal\{H\}\_\{\\nu\}\}^\{2\}\\lesssim\\frac\{d\_\{t\}\}\{K\}\\bigl\(\\operatorname\{tail\}\_\{r\}\\bigl\(\\Sigma\_\{P\_\{t\}\}^\{\\Phi\}\\bigr\)\+\\tau\\bigr\)\\lesssim\\frac\{d\_\{t\}\}\{K\}\\bigl\(\\operatorname\{tail\}\_\{r\}\(\\Sigma\_\{P\_\{t\},\\nu\}\)\+\\tau\\bigr\),which proves the second claim\. ∎

The final summaryP^t\\hat\{P\}\_\{t\}used as the compressed context for the token at positionttis a combination of a compressed summary produced by the merge\-reduce tree and an exact context measure of the tokens inside the same chunk as tokentt\. The next lemma shows that this is harmless for the validity of the resulting summary\.

###### Lemma C\.11\.

LetA,R⊆ℕA,R\\subseteq\\mathbb\{N\}be finite and disjoint andA≠∅A\\neq\\emptyset\. IfSAS\_\{A\}is\(d,r,τ\)\(d,r,\\tau\)\-valid forAA, then

SA∪R≔w​\(A\)w​\(A\)\+w​\(R\)​SA\+w​\(R\)w​\(A\)\+w​\(R\)​PRS\_\{A\\cup R\}\\coloneqq\\frac\{w\(A\)\}\{w\(A\)\+w\(R\)\}S\_\{A\}\+\\frac\{w\(R\)\}\{w\(A\)\+w\(R\)\}P\_\{R\}is\(d,r,τ\)\(d,r,\\tau\)\-valid forA∪RA\\cup R\.

###### Proof\.

Unbiasedness ofSA∪RS\_\{A\\cup R\}is immediate from unbiasedness ofSAS\_\{A\}andPRP\_\{R\}\. Setα≔w​\(A\)/\(w​\(A\)\+w​\(R\)\)\\alpha\\coloneqq w\(A\)/\(w\(A\)\+w\(R\)\)\.

Let𝒢\\mathcal\{G\}be a Hilbert space andΨ:𝒳→𝒢\\Psi:\\mathcal\{X\}\\to\\mathcal\{G\}bounded\. Then, the defect ofSA∪RS\_\{A\\cup R\}is

∫𝒳Ψ​d​\(SA∪R−PA∪R\)=α​\(∫𝒳Ψ​d​\(SA−PA\)\)\\int\_\{\\mathcal\{X\}\}\\Psi\\mathop\{\}\\\!\\mathrm\{d\}\(S\_\{A\\cup R\}\-P\_\{A\\cup R\}\)=\\alpha\\left\(\\int\_\{\\mathcal\{X\}\}\\Psi\\mathop\{\}\\\!\\mathrm\{d\}\(S\_\{A\}\-P\_\{A\}\)\\right\)Sinceα​ΣPAΨ⪯ΣPA∪RΨ\\alpha\\Sigma^\{\\Psi\}\_\{P\_\{A\}\}\\preceq\\Sigma^\{\\Psi\}\_\{P\_\{A\\cup R\}\}, the validity estimates forSAS\_\{A\}imply the validity estimates forSA∪RS\_\{A\\cup R\}\. ∎

It remains only to count the validity depth of the summaries produced by[Algorithms˜1](https://arxiv.org/html/2607.01520#alg1)and[2](https://arxiv.org/html/2607.01520#alg2)\. The prefill scan and the decoding buckets both construct summaries by repeated binary merges of disjoint completed chunks\. Along any dependency path there are only logarithmically many reducer calls, and the unfinished current chunk is appended exactly\. Thus, if𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}is\(Φ,r,τ\)\(\\Phi,r,\\tau\)\-admissible, each resulting prefix summaryP^t\\hat\{P\}\_\{t\}is\(d,r,τ\)\(d,r,\\tau\)\-valid, withddproportional to the depth of the tree, which is logarithmic in the number of tokens\. The claimed risk guarantees then follow immediately from[Proposition˜C\.10](https://arxiv.org/html/2607.01520#A3.Thmtheorem10)\.

###### Proof\.

We first show thatP^t\\hat\{P\}\_\{t\}as produced by[Algorithms˜1](https://arxiv.org/html/2607.01520#alg1)and[2](https://arxiv.org/html/2607.01520#alg2)is𝒪⁡\(log⁡\(2\+t/K\),r,τ\)\\operatorname\{\\mathcal\{O\}\}\(\\log\(2\+t/K\),r,\\tau\)\-valid for\{1,…,t\}\\\{1,\\ldots,t\\\}\. The risk bounds then follow immediately from[Proposition˜C\.10](https://arxiv.org/html/2607.01520#A3.Thmtheorem10)\.

LetI1,I2,…I\_\{1\},I\_\{2\},\\ldotsdenote the consecutive chunks of sizeKK, and letc​\(t\)c\(t\)be the chunk containingtt\. Write

At≔I1∪⋯∪Ic​\(t\)−1,Rt≔Ic​\(t\)∩\{1,…,t\}\.A\_\{t\}\\coloneqq I\_\{1\}\\cup\\cdots\\cup I\_\{c\(t\)\-1\},\\qquad R\_\{t\}\\coloneqq I\_\{c\(t\)\}\\cap\\\{1,\\ldots,t\\\}\.Thus,PtP\_\{t\}is the normalized mixture ofPAtP\_\{A\_\{t\}\}and the exact current\-chunk suffixPRtP\_\{R\_\{t\}\}, with the obvious convention whenAt=∅A\_\{t\}=\\emptyset\.

We claim that every summary stored by the merge\-reduce computation for a union ofmmcompleted chunks is\(O​\(log⁡\(2\+m\)\),r,τ\)\(O\(\\log\(2\+m\)\),r,\\tau\)\-valid for its underlying index set, and that summaries stored on disjoint index sets are independent\. This follows by induction on the binary merge tree\. A single chunk is represented exactly, hence is deterministic and\(0,r,τ\)\(0,r,\\tau\)\-valid\. Whenever two disjoint summaries with validityd1d\_\{1\}andd2d\_\{2\}are merged, the induction hypothesis gives their independence; the reducer call uses fresh randomness independent of both summaries\. Thus,[Lemma˜C\.9](https://arxiv.org/html/2607.01520#A3.Thmtheorem9)shows that the resulting summary is\(\(1\+max⁡\{d1,d2\}\),r,τ\)\(\(1\+\\max\\\{d\_\{1\},d\_\{2\}\\\}\),r,\\tau\)\-valid\. The resulting summary depends only on its two children and on the fresh randomness used in this reducer call, so it remains independent of all stored summaries supported on disjoint index sets\. Along any path in the merge tree formmchunks there are at most𝒪⁡\(log⁡\(2\+m\)\)\\operatorname\{\\mathcal\{O\}\}\(\\log\(2\+m\)\)merges\. Hence the summary of any completed\-chunk prefix produced by the prefill scan, and the completed\-prefix summary maintained by the decoding buckets, is\(𝒪⁡\(log⁡\(2\+m\)\),r,τ\)\(\\operatorname\{\\mathcal\{O\}\}\(\\log\(2\+m\)\),r,\\tau\)\-valid\.

Apply this to the completed chunks precedingtt\. IfAt≠∅A\_\{t\}\\neq\\emptyset, the algorithm has an\(𝒪⁡\(log⁡\(2\+\|At\|/K\)\),r,τ\)\(\\operatorname\{\\mathcal\{O\}\}\(\\log\(2\+\|A\_\{t\}\|/K\)\),r,\\tau\)\-valid summarySAtS\_\{A\_\{t\}\}forAtA\_\{t\}\. SinceRtR\_\{t\}is kept exact,

P^t=w​\(At\)w​\(At\)\+w​\(Rt\)​SAt\+w​\(Rt\)w​\(At\)\+w​\(Rt\)​PRt\.\\hat\{P\}\_\{t\}=\\frac\{w\(A\_\{t\}\)\}\{w\(A\_\{t\}\)\+w\(R\_\{t\}\)\}S\_\{A\_\{t\}\}\+\\frac\{w\(R\_\{t\}\)\}\{w\(A\_\{t\}\)\+w\(R\_\{t\}\)\}P\_\{R\_\{t\}\}\.By[Lemma˜C\.11](https://arxiv.org/html/2607.01520#A3.Thmtheorem11), and using\|At\|≤t\|A\_\{t\}\|\\leq t,P^t\\hat\{P\}\_\{t\}is\(𝒪⁡\(log⁡\(2\+t/K\)\),r,τ\)\(\\operatorname\{\\mathcal\{O\}\}\(\\log\(2\+t/K\)\),r,\\tau\)\-valid for\{1,…,t\}\\\{1,\\ldots,t\\\}\.

The claim then follows immediately from[Proposition˜C\.10](https://arxiv.org/html/2607.01520#A3.Thmtheorem10)\. ∎

The global guarantee therefore reduces entirely to the design of𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}\. Any concrete reducer satisfying the one\-step trace and tail conditions of[Section˜5\.2](https://arxiv.org/html/2607.01520#S5.SS2)inherits the same causal prefill and decoding guarantees through the merge\-reduce schedules of[Algorithms˜1](https://arxiv.org/html/2607.01520#alg1)and[2](https://arxiv.org/html/2607.01520#alg2), with only the logarithmic depth loss\.

### C\.4Computational cost of the global merge\-reduce scheme

We prove[Section˜5\.1](https://arxiv.org/html/2607.01520#S5.SS1)from the main text\.

###### Proof\.

LetM=n/KM=n/KandL=log2⁡ML=\\log\_\{2\}M\. We count calls toReduceand atom storage, treating eachKK\-atomic summary as an object of sizeKK\.

First, consider Algorithm[1](https://arxiv.org/html/2607.01520#alg1)\. In the upsweep stage, levelℓ\\ellcontainsM/2ℓM/2^\{\\ell\}independent merge\-reduce operations\. Hence, the number of calls toReducein the upsweep is

∑ℓ=1LM2ℓ=M−1\.\\sum\_\{\\ell=1\}^\{L\}\\frac\{M\}\{2^\{\\ell\}\}=M\-1\.The downsweep has the same count\. The total number of calls is thus2​M−2=𝒪⁡\(n/K\)2M\-2=\\operatorname\{\\mathcal\{O\}\}\(n/K\)\.

The parallel depth is also immediate from the tree structure\. All calls at a fixed level are independent, so each level contributes one parallel round ofReduce\. There areLLupsweep levels andLLdownsweep levels, giving depth𝒪⁡\(L\)=𝒪⁡\(log⁡\(n/K\)\)\\operatorname\{\\mathcal\{O\}\}\(L\)=\\operatorname\{\\mathcal\{O\}\}\(\\log\(n/K\)\)\.

For memory, the algorithm stores𝒪⁡\(M\)\\operatorname\{\\mathcal\{O\}\}\(M\)summaries across all levels of the scan tree, and each summary isKK\-atomic\. Thus the total number of stored atoms is𝒪⁡\(M​K\)=𝒪⁡\(n\)\\operatorname\{\\mathcal\{O\}\}\(MK\)=\\operatorname\{\\mathcal\{O\}\}\(n\)\.

Now consider Algorithm[2](https://arxiv.org/html/2607.01520#alg2)\. Aftersscompleted chunks have been incorporated, including the prompt summary, the nonempty buckets satisfy the standard binary\-counter invariant: bucketBℓB\_\{\\ell\}, when nonempty, stores aKK\-atomic summary of a block of2ℓ2^\{\\ell\}chunks, and there is at most one nonempty bucket at each level\. Therefore, the number of nonempty buckets is at most1\+⌊log2⁡s⌋1\+\\lfloor\\log\_\{2\}s\\rfloor\. For a prefix of lengtht=s​Kt=sK, the number of stored atoms is consequently

𝒪⁡\(K​log⁡s\)=𝒪⁡\(K​log⁡\(t/K\)\)\.\\operatorname\{\\mathcal\{O\}\}\(K\\log s\)=\\operatorname\{\\mathcal\{O\}\}\(K\\log\(t/K\)\)\.
It remains to count calls toReduce\. Each new chunk first creates a level\-zero bucket\. The while loop then performs the carry operations of a binary counter\. Whenever a bucket is already occupied, the twoKK\-atomic summaries are merged and reduced to form one summary at the next level\. Oversschunk insertions, levelℓ\\ellparticipates in at mostO​\(s/2ℓ\+1\)O\(s/2^\{\\ell\+1\}\)such carries\. Summing overℓ\\ellgives𝒪⁡\(s\)\\operatorname\{\\mathcal\{O\}\}\(s\)carry calls in total, hence𝒪⁡\(1\)\\operatorname\{\\mathcal\{O\}\}\(1\)amortized carry calls per chunk\.

After the carry step, the algorithm forms the current history summary by merging the nonempty buckets in decreasing order\. Since there are𝒪⁡\(log⁡s\)\\operatorname\{\\mathcal\{O\}\}\(\\log s\)nonempty buckets, this uses𝒪⁡\(log⁡s\)\\operatorname\{\\mathcal\{O\}\}\(\\log s\)calls toReduceper completed chunk\. Combining this with the𝒪⁡\(1\)\\operatorname\{\\mathcal\{O\}\}\(1\)amortized carry cost gives𝒪⁡\(log⁡s\)\\operatorname\{\\mathcal\{O\}\}\(\\log s\)calls per chunk\. Since each chunk containsKKtokens, the amortized number of calls per token is

𝒪⁡\(log⁡s/K\)=O​\(log⁡\(t/K\)/K\)\.\\operatorname\{\\mathcal\{O\}\}\(\\log s/K\)=O\(\\log\(t/K\)/K\)\.That proves the claim\. ∎

## Appendix DExperimental details

This appendix describes the implementation of our experiments in[Section˜6](https://arxiv.org/html/2607.01520#S6)\. Detailed instructions for reproducing our experiments are provided with the code in the supplementary material\.

##### Dataset and model\.

We evaluate on the “long” subset of LongBench\-v2\[[6](https://arxiv.org/html/2607.01520#bib.bib6)\]\. The dataset is available at[https://longbench2\.github\.io/](https://longbench2.github.io/)\. We use the official LongBench\-v2 evaluation protocol\. We run all experiments on Qwen3\-32B\[[25](https://arxiv.org/html/2607.01520#bib.bib25)\], usingint4quantization\. We extend the context window to131​k131\\mathrm\{k\}tokens using YaRN RoPE scaling\[[21](https://arxiv.org/html/2607.01520#bib.bib21)\]\. The model is available at[https://huggingface\.co/Qwen/Qwen3\-32B](https://huggingface.co/Qwen/Qwen3-32B)\. We disable chain\-of\-thought generation and use greedy decoding throughout\. LongBench\-v2 and Qwen3\-32B are released under the Apache\-2\.0 license\.

##### Computational resources\.

All experiments were conducted on a single NVIDIA H100 GPU with95,83095\{,\}830MiB VRAM\. The full set of runs took approximately 100 GPU hours, including failed runs\.

##### Implementation details\.

For the full\-attention baseline, we use the standard implementation of the Qwen3\-32B model\. For the compressed methods, we use the algorithms described in[Section˜5](https://arxiv.org/html/2607.01520#S5), with two additions\. First, each token may attend to the first88tokens, following the attention\-sink heuristic ofXiao et al\. \[[24](https://arxiv.org/html/2607.01520#bib.bib24)\]\. Second, each token may attend to theKKtokens immediately preceding its chunk boundary\. Thus, each token attends to at most3​K\+83K\+8tokens\. This is compatible with our theory and enjoys the same guarantees\. We useK=2048K=2048, corresponding to a compression rate of approximately95%95\\%relative to the131​k131\\mathrm\{k\}\-token context available to the full\-attention baseline\.

Many LongBench\-v2 contexts exceed even the extended context window of Qwen3\-32B\. For such examples, we follow the official truncation protocol ofBai et al\. \[[6](https://arxiv.org/html/2607.01520#bib.bib6)\]by removing the middle section of the prompt\.

##### Local reducer instantiations\.

We report results for two instantiations of𝖱𝖤𝖣𝖴𝖢𝖤\\operatorname\{\\mathsf\{REDUCE\}\}: random sampling and protected clustering\. All randomized methods are evaluated with a single run using a fixed random seed\.

For random sampling, we sample i\.i\.d\. from the context measure, i\.e\., proportionally to

exp⁡\(‖k‖22/\(2​dk\)\),\\textstyle\\exp\\\!\\left\(\\\|k\\\|\_\{2\}^\{2\}/\(2\\sqrt\{d\_\{k\}\}\)\\right\),\(11\)as described in[Section˜3](https://arxiv.org/html/2607.01520#S3)\.

Protected clustering corresponds to[Algorithm˜6](https://arxiv.org/html/2607.01520#alg6), using the static feature map described in[Section˜C\.1](https://arxiv.org/html/2607.01520#A3.SS1)with the value norm bound fixed toV2=50V^\{2\}=50\. We evaluate protected ranksr=64r=64andr=128r=128\. To compute the leading eigenspace, we use a randomized Nyström approximation with four\-fold oversampling\.

ForClusterSlotsin[Algorithm˜6](https://arxiv.org/html/2607.01520#alg6), we use a sliced equal\-mass clustering rule\. After the heavy atoms have been retained exactly as in[Algorithm˜6](https://arxiv.org/html/2607.01520#alg6), the light atomsi∈Li\\in Lhave normalized massespi=qi/ρp\_\{i\}=q\_\{i\}/\\rhosatisfying∑i∈Lpi=m\\sum\_\{i\\in L\}p\_\{i\}=m\. For each direction in a fixed bank of random directions in the protected coordinate space, we sort the light atoms by their one\-dimensional projection and place intervals of lengthpip\_\{i\}consecutively on\[0,m\]\[0,m\]\. Themmlatent slots are the unit intervals of this line, and the assignment matrixXXis given by the overlap of each atom interval with each unit slot\. This construction satisfies the row and column constraints required in[Algorithm˜6](https://arxiv.org/html/2607.01520#alg6)\. We choose the direction with the smallest within\-slot squared error in the protected coordinates and sample one representative from each slot by inverse\-transform sampling\.

For this method, we treat the initial context measure as uniform over the tokens rather than proportional to[Equation˜11](https://arxiv.org/html/2607.01520#A4.E11), as this significantly improved numerical stability in the computation of the leading eigenspace\.

##### Literature baselines\.

We compare performance against ScissorHands\[[20](https://arxiv.org/html/2607.01520#bib.bib20)\], SnapKV\[[19](https://arxiv.org/html/2607.01520#bib.bib19)\], and StreamingLLM\[[24](https://arxiv.org/html/2607.01520#bib.bib24)\]\. All three methods use a sliding window of the most recent40964096tokens\. The allocation of the remaining20562056token budget is dictated by the respective method\. StreamingLLM retains the initial88sink tokens and uses the rest of this budget to extend the recent\-token window\. SnapKV retains the top\-ranked non\-recent prompt tokens by attention score from the final prompt window, using average pooling with kernel size 5, as in the original paper\[[19](https://arxiv.org/html/2607.01520#bib.bib19)\]\. ScissorHands retains the top\-ranked non\-recent tokens by cumulative pivotal\-token counts\.