LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs

# One Cached State Suffices for Position-Independent Caching in Hybrid LLMs
Source: [https://arxiv.org/html/2608.11231](https://arxiv.org/html/2608.11231)
Yirui Liu1,∗Ruoling Qi2,1,∗Longwen Wang3,1Xuaner Wu1Jian Chen4 Yuxin Jin1Jiawei Shao1Xuelong Li1,† 1Institute of Artificial Intelligence, China Telecom \(TeleAI\) 2Shanghai Jiao Tong University3Xi’an Jiaotong University4University at Buffalo ∗Equal contribution\.†Corresponding author\. Yirui Liu:yiruiliu926@gmail\.comRuoling Qi:qiruoling760@sjtu\.edu\.cn Xuelong Li:xuelong\_li@ieee\.org

###### Abstract

LLM serving is increasingly accelerated by position\-independent caching \(PIC\)\. Existing PIC methods, however, are built for full\-attention models, where a token\-indexed KV cache underlies its core operations: matching reusable token chunks, concatenating their KV entries, and selectively recomputing a few tokens to restore cross\-chunk context\. Hybrid LLMs break these primitives—they replace most attention layers with linear recurrences that expose only a fixed\-size state, leaving no token\-indexed KV to concatenate or to locally repair\. This raises a natural question: can PIC benefit hybrid models, and what would it take? We present LinearKV, a training\-free hybrid\-PIC framework\. Its key insight is a*decoupled initialization*: each linear layer maps itsKKmatched local states to a single initial state, while full\-attention layers concatenate their KV as before\. LinearKV is therefore compatible with existing PIC methods, reusing their token selection and recomputation as\-is\. Under this framework, we find that a*single cached state*suffices as the linear layer’s initializer\. The algebraically principled alternative—composing allKKcached states into the exact full\-prefix state, as concurrent work HYPIC does—is unnecessary and, on some architectures, even harmful\. We compare the two across three hybrid models \(one Mamba\-2, two GDN\) and three PIC selectors \(CacheBlend, EPIC, ProphetKV\)\. On the two GDN models the two tie, both recovering most of full quality \(up to92%92\\%\); on the Mamba\-2 model, exact composition instead collapses under every selector—under EPIC, for instance, it recovers only46\.6%46\.6\\%of full quality, versus86\.8%86\.8\\%for a single cached block initializer\. A single state initializer is also cheaper, cutting time\-to\-first\-token to0\.46×0\.46\\timesfull prefill versus a further55–17%17\\%overhead for exact composition; results hold across LongBench QA and RULER at 8K–32K\.

## 1Introduction

Long\-context LLM applications—multi\-turn conversations, agentic workflows, long\-document understanding—incur a prefill cost that grows with prompt length, inflating time to first token \(TTFT\) and reducing throughput\. Context \(or prompt\) caching mitigates this by storing KV\-cache entries for previously processed tokens and reusing them across requests\. Standard implementations are prefix\-based\(Kwonet al\.,[2023](https://arxiv.org/html/2608.11231#bib.bib39); Zhenget al\.,[2024](https://arxiv.org/html/2608.11231#bib.bib40)\): a cached entry is reusable only when the same tokens follow the same prefix, limiting the hit rate\.

Position\-independent caching \(PIC\) relaxes this constraint, letting an independently cached chunk be reused at any position\. Reuse then becomes lossy—a chunk prefilled in isolation lacks the cross\-chunk attention a joint prefill would supply—so PIC methods selectively recompute a small subset of tokens to restore the cross\-chunk attention\(Yaoet al\.,[2025](https://arxiv.org/html/2608.11231#bib.bib5); Huet al\.,[2025](https://arxiv.org/html/2608.11231#bib.bib6); Yanget al\.,[2025b](https://arxiv.org/html/2608.11231#bib.bib21); Wanget al\.,[2026](https://arxiv.org/html/2608.11231#bib.bib7)\)\. Offline, each chunk is prefilled independently and its KV stored under an identifier; online, a request retrieves the matching chunks, concatenates their KV in context order, recomputes selected positions, and serves from the repaired cache\.

![Refer to caption](https://arxiv.org/html/2608.11231v1/x1.png)Figure 1:Why token\-KV PIC does not transfer directly to hybrid models\.\(a\)Full attention: every layer stores per\-token KV that concatenates across chunks\.\(b\)Hybrid: recurrent layers expose only a per\-chunk state, which does not\.The PIC methods above are designed for full\-attention models\. Hybrid architectures, however, have emerged as a promising long\-context backbone: they interleave a few full\-attention \(FA\) layers with a majority of recurrent layers such as Mamba\-2\(Dao and Gu,[2024](https://arxiv.org/html/2608.11231#bib.bib9)\)and Gated DeltaNet \(GDN\)\(Yanget al\.,[2025d](https://arxiv.org/html/2608.11231#bib.bib10)\)\. This raises a natural question: Can PIC benefit hybrid models, and what would it take? Answering it exposes two challenges\.\(C1\) Can the existing PIC machinery be reused, or must hybrid PIC be redesigned?The reuse step is where the mismatch surfaces \(Figure[1](https://arxiv.org/html/2608.11231#S1.F1)\): FA layers still expose a per\-token KV cache to concatenate, but each recurrent layer summarizes its whole chunk into a single fixed\-size state, leaving no per\-token entries to concatenate or selectively recompute\. Chunk matching still works by identifier, but PIC’s other core operations—concatenation and selective repair—assume a token\-indexed cache the recurrent path lacks, so whether PIC can work on this stateful representation at all is unclear\.\(C2\) Does hybrid PIC generalize across models?Hybrids vary in recurrence family \(Mamba\-2 vs\. GDN\) and recurrent\-to\-FA ratio, so whether one construction stays beneficial across these designs—or must be re\-tuned per architecture—is unknown\.

We present LinearKV, a training\-free PIC framework for hybrid models\. Its key insight is a*decoupled initialization*applied right after token\-chunk matching: each FA layer is initialized by concatenating its cached KV entries, exactly as in full\-attention PIC, while each linear layer is initialized by a function that maps theKKmatched local states to a single initial state\. As it leaves the FA path and the selector interface untouched, this decoupling lets LinearKV reuse existing PIC methods as a plug\-in for choosing which tokens to recompute \(C1\): LinearKV then applies that choice across both layer types—overwriting the selected FA KV while updating each linear state through the selected positions in context order\. Moreover, LinearKV shows that*one cached state suffices*as the linear\-layer initializer: keeping just a single matched chunk’s cached linear state is remarkably robust across architectures \(C2\)\. It matches the algebraically principled alternative—composing allKKcached states into the exact full\-prefix state, as concurrent HYPIC\(Liuet al\.,[2026b](https://arxiv.org/html/2608.11231#bib.bib11)\)does—on the two GDN models, while on Mamba\-2, exactly where that exact composition collapses, a single cached state initializer lifts quality from46\.6%46\.6\\%to86\.8%86\.8\\%of full\. A*random*single block works as well \(Appendix[A](https://arxiv.org/html/2608.11231#A1)\), so it is the single\-source construction, not the specific block, that matters\.

We evaluate LinearKV on three hybrid models—Granite \(Mamba\-2\), and OLMo and Qwen \(GDN\)—covering both recurrence families and different linear\-to\-FA ratios\. On each, CacheBlend, EPIC, and ProphetKV serve as three token\-recomputation selectors, and we compare the last\-block initializer against exact composition under identical selection and recomputation\. Across LongBench QA and RULER at 8K–32K, the same architecture\-dependent boundary holds—exact composition collapses on Mamba\-2 under every selector, while the last\-block initializer stays robust and matches it on the two GDN models—and LinearKV also reduces TTFT relative to full prefill in all evaluated settings\.

In summary, this paper makes three contributions:

- •We formalize position\-independent caching for hybrid LLMs as a decoupled\-initialization framework, identifying*linear\-state initialization*—mapping theKKcached states to one initial state—as the single hybrid\-specific operation, within which existing full\-attention PIC selectors are reused unchanged\.
- •We show that the natural choice—composing allKKcached states \(exact composition, as in concurrent HYPIC\)—is unnecessary and architecture\-fragile:*one cached state suffices*\. A single\-block initializer with ordered recomputation matches exact composition on GDN, lifts Mamba\-2 from46\.6%46\.6\\%to86\.8%86\.8\\%of full quality, and avoids its per\-request composition cost; a random single block does as well \(Appendix[A](https://arxiv.org/html/2608.11231#A1)\), isolating single\-source construction, not the specific block, as the operative factor\.
- •We evaluate LinearKV across three hybrid models and three PIC selectors on LongBench QA and RULER \(8K–32K\), where a training\-free last\-block initializer is robust across architectures and reduces TTFT relative to full prefill in all evaluated settings\.

## 2Preliminaries

Hybrid LLMs interleave a small number of FA layers for global token–token interaction with a majority of recurrent layers, such as Mamba\-2\(Dao and Gu,[2024](https://arxiv.org/html/2608.11231#bib.bib9)\)and Gated DeltaNet \(GDN\)\(Yanget al\.,[2025d](https://arxiv.org/html/2608.11231#bib.bib10)\), that summarize the processed prefix in fixed\-size state\.

An FA layer retains a token\-indexed KV cache\{\(Ki,Vi\)\}i=1N\\\{\(K\_\{i\},V\_\{i\}\)\\\}\_\{i=1\}^\{N\}\. A recurrent layer instead compresses the processed prefix into a fixed\-size stateSiℓ∈ℝdk×dvS\_\{i\}^\{\\ell\}\\in\\mathbb\{R\}^\{d\_\{k\}\\times d\_\{v\}\}, updated token by token through the general affine recurrence

Siℓ=Tiℓ​Si−1ℓ\+uiℓ,S\_\{i\}^\{\\ell\}=T\_\{i\}^\{\\ell\}S\_\{i\-1\}^\{\\ell\}\+u\_\{i\}^\{\\ell\},\(1\)whereTiℓT\_\{i\}^\{\\ell\}transports the previous state anduiℓu\_\{i\}^\{\\ell\}is a token\-dependent outer\-product update\. The two recurrence families instantiateTiℓT\_\{i\}^\{\\ell\}differently\. Suppressing the layer superscript for clarity,

Mamba\-2:Si=ai​Si−1\+ui,\\displaystyle S\_\{i\}=a\_\{i\}S\_\{i\-1\}\+u\_\{i\},Ti=ai​I,\\displaystyle T\_\{i\}=a\_\{i\}I,\(2\)GDN:Si=Ti​Si−1\+ui,\\displaystyle S\_\{i\}=T\_\{i\}S\_\{i\-1\}\+u\_\{i\},Ti=αi​\(I−βi​ki​ki⊤\),\\displaystyle T\_\{i\}=\\alpha\_\{i\}\(I\-\\beta\_\{i\}k\_\{i\}k\_\{i\}^\{\\top\}\),\(3\)whereai∈\(0,1\]a\_\{i\}\\in\(0,1\]is a per\-head scalar decay,αi=exp⁡\(gi\)\\alpha\_\{i\}=\\exp\(g\_\{i\}\)is a learned gate, andI−βi​ki​ki⊤I\-\\beta\_\{i\}k\_\{i\}k\_\{i\}^\{\\top\}is a token\-dependent rank\-1 correction\. Mamba\-2 therefore transports every state direction within a head using the same scalar, whereas GDN applies a direction\-dependent dense transition\. This difference later helps interpret the depth\-wise reuse error analyzed below\. After prefill, only the final state is required for subsequent decoding; the intermediate states do not form a persistent token\-indexed cache\.

## 3Method: LinearKV

We propose LinearKV, a training\-free PIC framework for hybrid LLMs such as Granite, OLMo, and Qwen\. Its key idea is a*decoupled linear\-state initialization*: each full\-attention \(FA\) layer is initialized by concatenating its cached KV entries, exactly as in full\-attention PIC, while each linear layer is initialized by a functionffthat maps theKKmatched local states to a single initial state\. Because this decoupling leaves the FA path and the selector interface untouched, LinearKV requires only that a PIC selector emit a set of token positions to recompute; any selector meeting that interface plugs in unchanged—LinearKV reuses its token selection and recomputation as\-is—and the design space collapses to the one remaining choice, the linear\-state initializerff\. We validate this with three selectors \(CacheBlend, EPIC, ProphetKV\)\. This section first gives an overview of LinearKV, then studies the choice offfin depth, showing that the algebraically exact composition fails on some architectures while a simple last\-block initialization is robust across models and selectors—recovering most of full quality on the GDN models and, on Mamba\-2, far exceeding exact composition\.

### 3\.1Overview

![Refer to caption](https://arxiv.org/html/2608.11231v1/x2.png)Figure 2:Overview of LinearKV\.\(1\)Each reusable token chunkCCis prefilled offline into one hybrid cache—per\-chunk FA KV together with, for every recurrent layer, an end stateSlocalS^\{\\mathrm\{local\}\}; online retrieval returns the matched chunksC1:KC\_\{1:K\}\.\(2\)*Decoupled initialization*: FA layers concatenate their cached KV along the token axis \(fixed\), while each recurrent layer folds theKKcached states into one initial stateSinit=f​\(S1local,…,SKlocal\)S\_\{\\mathrm\{init\}\}=f\(S^\{\\mathrm\{local\}\}\_\{1\},\\dots,S^\{\\mathrm\{local\}\}\_\{K\}\)\. Onlyffchanges—exact composition versus LinearKV’s last blockflast=SKlocalf\_\{\\mathrm\{last\}\}=S^\{\\mathrm\{local\}\}\_\{K\}—and everything downstream is shared\.\(3\)Any existing selectorΠ​\(q,C1:K,r\)\\Pi\(q,C\_\{1:K\},r\)outputs repair positions𝒰=\{i1<⋯<im\}\\mathcal\{U\}=\\\{i\_\{1\}<\\dots<i\_\{m\}\\\}; LinearKV replays𝒰\\mathcal\{U\}in original context order, advancing each recurrent state fromSinitS\_\{\\mathrm\{init\}\}while overwriting the FA KV at those positions\.\(4\)The query is served from the repaired context cache\.Figure[2](https://arxiv.org/html/2608.11231#S3.F2)summarizes the pipeline\.*Offline*, each reusable chunkCCis prefilled independently and cached as its per\-chunk FA KV together with each recurrent layer’s local stateSlocalS^\{\\mathrm\{local\}\}\(the last\-block initializer needs nothing more; exact composition additionally uses each chunk’s transition, obtained as described in*Linear\-State Initialization*below\)\.*Online*, given theKKmatched chunks, the FA path proceeds as in full\-attention PIC, while each recurrent layer forms its initial state through the initializerff:

Sinit=f​\(S1local,…,SKlocal\)\.S\_\{\\mathrm\{init\}\}=f\\\!\\left\(S^\{\\mathrm\{local\}\}\_\{1\},\\dots,S^\{\\mathrm\{local\}\}\_\{K\}\\right\)\.\(4\)LinearKV then adopts existing PIC method—CacheBlend, EPIC, or ProphetKV—to select a token position set𝒰=\{i1<⋯<im\}\\mathcal\{U\}=\\\{i\_\{1\}<\\dots<i\_\{m\}\\\}for recomputation, withm≈r​Nm\\approx rNunder budgetrr, and repairs the cache by processing*only*thesemmtokens in context order \(Algorithm[1](https://arxiv.org/html/2608.11231#alg1)\)\. The intervening unselected tokens are never re\-run: each FA layer recomputes the selected tokens against the assembled cache and overwrites their KV, while each recurrent layer advances its state fromSinitS\_\{\\mathrm\{init\}\}through the selected tokens alone \(Eq\.[1](https://arxiv.org/html/2608.11231#S2.E1)\), keeping the final state\. Dropping the unselected transitions makes the recomputation itself an approximation—one whose cost ism≈r​Nm\\approx rNtoken\-steps rather than a full prefill—after which the query is prefilled against the repaired cache and served\.

Algorithm 1LinearKV serving of one request0:matched chunks

C1:KC\_\{1:K\}\(cached FA KV \+ recurrent end states

SjlocalS^\{\\mathrm\{local\}\}\_\{j\}; exact composition also uses transitions

TCjT\_\{C\_\{j\}\}\); query

qq; selector

Π\\Pi; budget

rr
1:Init \(decoupled\):FA layers concatenate cached KV in context order; each recurrent layer sets

Sinit←f​\(S1local,…,SKlocal\)S\_\{\\mathrm\{init\}\}\{\\leftarrow\}f\(S^\{\\mathrm\{local\}\}\_\{1\},\\dots,S^\{\\mathrm\{local\}\}\_\{K\}\)\(

f=f\{=\}last block or exact\)

2:

𝒰←Π​\(q,C1:K,r\)=\{i1<⋯<im\}\\mathcal\{U\}\{\\leftarrow\}\\Pi\(q,C\_\{1:K\},r\)\{=\}\\\{i\_\{1\}\{<\}\\dots<i\_\{m\}\\\},

m≈r​Nm\{\\approx\}rN\{selector unchanged\}

3:Recompute:over the

mmselected tokens

𝒰\\mathcal\{U\}only, in context order, per layer—FA recomputes against the assembled cache and overwrites KV at

𝒰\\mathcal\{U\}; each recurrent layer advances

SSfrom

SinitS\_\{\\mathrm\{init\}\}through

𝒰\\mathcal\{U\}\(Eq\.[1](https://arxiv.org/html/2608.11231#S2.E1)\) and keeps the final

SS
4:prefill

qqagainst the repaired cache and decode

### 3\.2Linear\-State Initialization

The decoupled design of Eq\. \([4](https://arxiv.org/html/2608.11231#S3.E4)\) leaves one function unspecified: how each recurrent layer maps itsKKcached local states\{Sjlocal\}\\\{S^\{\\mathrm\{local\}\}\_\{j\}\\\}into a single initial stateSinitS\_\{\\mathrm\{init\}\}\. This subsection compares two choices, and finds—counterintuitively—that the algebraically principled one is the fragile one\.

#### Exact composition\.

The algebra of the recurrence points to a mathematically intuitive first choice\. Because the recurrence \(Eq\. \([1](https://arxiv.org/html/2608.11231#S2.E1)\)\) is linear in the state, running it through a whole chunk is itself an affine operator: the chunk takes whatever stateSinS\_\{\\mathrm\{in\}\}enters it, scales it by a cumulative transitionTCj=∏t∈CjTtT\_\{C\_\{j\}\}=\\prod\_\{t\\in C\_\{j\}\}T\_\{t\}\(the product of the per\-token transitions\), and adds its own contributionSjlocalS^\{\\mathrm\{local\}\}\_\{j\}—the state the chunk reaches from an empty start, which is exactly what we cache offline:

Sout=TCj​Sin\+Sjlocal\.S\_\{\\mathrm\{out\}\}=T\_\{C\_\{j\}\}\\,S\_\{\\mathrm\{in\}\}\+S^\{\\mathrm\{local\}\}\_\{j\}\.\(5\)To assemble theKKmatched chunks, we apply these operators back to back in context order—chunk 1’s output is chunk 2’s input, and so on\. This telescopes into a single closed form that composes the cached chunks with no approximation*of its own*:

Sinitexact=∑j=1K\(∏m\>jTCm\)​Sjlocal,S\_\{\\mathrm\{init\}\}^\{\\mathrm\{exact\}\}=\\sum\_\{j=1\}^\{K\}\\Big\(\\prod\_\{m\>j\}T\_\{C\_\{m\}\}\\Big\)\\,S^\{\\mathrm\{local\}\}\_\{j\},\(6\)each chunk’s contribution carried forward by the transitions of all later chunks\. This is the natural, algebraically principled choice, and the one concurrent HYPIC\(Liuet al\.,[2026b](https://arxiv.org/html/2608.11231#bib.bib11)\)adopts\. The two families differ only in what the transitionTCmT\_\{C\_\{m\}\}is: Mamba\-2’s per\-token transition is a scalar \(Eq\. \([2](https://arxiv.org/html/2608.11231#S2.E2)\)\), soTCmT\_\{C\_\{m\}\}is a single scalar decay, whereas GDN’s is a dense rank\-1 update \(Eq\. \([3](https://arxiv.org/html/2608.11231#S2.E3)\)\), soTCmT\_\{C\_\{m\}\}is a densedk×dkd\_\{k\}\\times d\_\{k\}matrix\.

#### Exactness is conditional\.

The exactness, though, is only relative to the chunk operators it composes—and each such operator, built offline from a chunk prefilled*in isolation*, is not the operator that chunk would have inside a full prefill\. Because an isolated chunk sees none of its predecessors, its operator is conditioned on inputs that omit all earlier context\. This is harmless at the first layer, where every chunk starts from the same empty history, but not above it: each layer reads the previous layer’s outputs, which in a full prefill already carry the earlier chunks’ context\. Composing exact algebra over these mis\-conditioned operators therefore need not reproduce the true state\. Concretely, leteje\_\{j\}be the gap between the composed state and the true full\-prefix state after chunkjj; it follows a simple recursion,

ej=TCj​ej−1\+δj,e\_\{j\}=T\_\{C\_\{j\}\}\\,e\_\{j\-1\}\+\\delta\_\{j\},\(7\)where the first term carries the accumulated error forward andδj\\delta\_\{j\}is the fresh mismatch injected because a chunk’s isolated operator differs from its full\-prefill operator\. Whether the accumulated error is retained or damped as it is carried forward is modulated by the transitionTCjT\_\{C\_\{j\}\}—a scalar decay can only retain and sum it, whereas a dense gated transition can also suppress or overwrite particular directions\. The injected termδj\\delta\_\{j\}itself is not set byTCjT\_\{C\_\{j\}\}: it also depends on the layer’s hidden inputs, its depth and placement relative to the FA layers, and normalization\. So the recurrence family shapes error*propagation*but is not the sole cause of the outcome; whether exact composition actually helps is therefore an empirical question, which we settle next\.

![Refer to caption](https://arxiv.org/html/2608.11231v1/x3.png)Figure 3:Per\-layer relative error of the constructed initial state vs\. the true full\-prefix state, at construction time \(before recomputation\), mean over five LongBench QA datasets\.
#### Diagnosis via relative error\.

We answer it by probing the constructed initial state directly, measuring its per\-layer relative error against the true full\-prefix stateS∗S^\{\*\}—i\.e\.∥Sinitℓ−S∗ℓ∥/∥S∗ℓ∥\\lVert S\_\{\\mathrm\{init\}\}^\{\\ell\}\-S^\{\*\\ell\}\\rVert/\\lVert S^\{\*\\ell\}\\rVertat each recurrent layerℓ\\ell, before any recomputation \(Figure[3](https://arxiv.org/html/2608.11231#S3.F3)\)\. At layer 0 the error is∼0\.01\{\\sim\}0\.01on every model—near\-exact, at thebf16/convolution\-boundary rounding floor rather than truly zero—confirming that composition is faithful where the operators are correct \(a numerical sanity check that the divergence at depth is model behavior, not an implementation artifact\)\. At depth the two recurrence families diverge\. On the GDN models \(OLMo, Qwen\) the error stays bounded \(<1<1\) at every layer\. On Granite \(Mamba\-2\) it compounds: one deep layer \(L32\) spikes to∼2×\{\\sim\}2\\timesthe state norm—stable across all five benchmarks \(range1\.71\.7–2\.32\.3\)—with several other layers above1\.01\.0\. This is exactly the split Eq\. \([7](https://arxiv.org/html/2608.11231#S3.E7)\) allows: Mamba\-2’s scalar decay retains and sums transported mismatch without a direction\-dependent correction, whereas GDN’s gate and dense rank\-1 update can suppress or overwrite particular state directions\. Neither the number nor the fraction of recurrent layers explains the split: Qwen has*more*recurrent layers than Granite in absolute terms \(4848GDN vs\.3636Mamba\-2\), and Granite has the*highest*recurrent fraction of the three \(90%90\\%vs\.75%75\\%\), yet Granite is the one that compounds—so within these models the governing factor tracks the recurrence dynamics, not how many layers are recurrent\. We observe this on one Mamba\-2 and two GDN hybrids; we therefore report it as an empirical boundary over the evaluated models, consistent with the error recursion of Eq\. \([7](https://arxiv.org/html/2608.11231#S3.E7)\), rather than a proven law for every hybrid, and Eq\. \([7](https://arxiv.org/html/2608.11231#S3.E7)\) makes the prediction testable as more hybrids of each family appear\.

#### Single\-block initialization\.

The diagnosis points to a fix\. The failure is not inaccurate algebra but the aggregation ofKKindependently conditioned deep operators; a construction that reads from a*single*cached block avoids it entirely\. It discardsK−1K\{\-\}1of the states and keeps just one as the initial state,Sinit=SblocalS\_\{\\mathrm\{init\}\}=S^\{\\mathrm\{local\}\}\_\{b\}for some blockbb; by default we take the last matched chunk \(b=Kb\{=\}K\)—the most recent context, already cached and needing no extra selection—and*which*block is kept turns out to barely matter, a random one doing as well \(see Appendix[A](https://arxiv.org/html/2608.11231#A1)\)\. Such a single cached state throws away information exact composition preserves and is provably not the full\-prefix state, so on its face it should be worse\. The relative\-error curves say otherwise \(Figure[3](https://arxiv.org/html/2608.11231#S3.F3), blue\): the single block tracks exact composition on the GDN models—exactness buys nothing there—and on Granite it stays near0\.30\.3exactly where exact composition explodes, with no spike at any depth\. The algebraically principled construction is the fragile one; a single cached state is the robust one\. Relative error, though, is only a construction\-time proxy—it scores the*starting point*recomputation is handed, not the served answer, and being norm\-normalized it can be inflated where the target state norm is small\. What ultimately matters is that this better starting point carries through to end\-to\-end quality once the selector’s recomputation runs\. That end\-to\-end quality is not norm\-sensitive, and it moves the same way: the next section shows the Granite collapse and the GDN tie reappear in served accuracy across all three models and all three selectors \(Tables[1](https://arxiv.org/html/2608.11231#S4.T1)and[2](https://arxiv.org/html/2608.11231#S4.T2)\), confirming the proxy is tracking a real effect rather than a normalization artifact\.

## 4Experiments

Table 1:LongBench results under a matched∼20%\{\\sim\}20\\%recompute budget \(r=0\.2r\{=\}0\.2\): QA token\-F1 per dataset and summarization ROUGE\-L per dataset, each with its Avg highlighted\.ex==exact composition,lb==LinearKV’s last\-block initial state \(shaded rows\)\. Summarization uses 512\-token generation and the LongBench\-officialrougeimplementation; naïve reuse already reaches7474–98%98\\%of full there, so the spread between methods is compressed\.‡Qwen anchors in the QA columns only are from an earlier torch build \(method\-independent\)\. RULER is in Table[2](https://arxiv.org/html/2608.11231#S4.T2)\.Table 2:RULER results \(string\-match recall\) under the same matched∼20%\{\\sim\}20\\%budget \(r=0\.2r\{=\}0\.2\): per subtask at 8K and at 32K, each with its Avg highlighted\. Row conventions follow Table[1](https://arxiv.org/html/2608.11231#S4.T1)\. Only four subtasks are run at 32K, so the two Avg columns are over different subtask sets and are not directly comparable\. Subtasks: common\-/frequent\-word extraction \(CWE, FWE\), single/multi\-key/multi\-query/multi\-value needle retrieval \(N\-S, N\-MK, N\-MQ, N\-MV\), SQuAD\-style QA, variable tracking \(VT\)\. Granite’s full\-recompute CWE cell \(0\.0030\.003, below its own naïve\-reuse score\) is a model\-level failure on that subtask rather than a reuse or scoring artifact\.### 4\.1Setup

#### Models\.

We evaluate three hybrid LLMs spanning both recurrence families, two scales, and different linear\-to\-FA ratios\.Granite\-4\.0\-H\-Tiny\(Granite Team, IBM,[2025](https://arxiv.org/html/2608.11231#bib.bib36)\)\(77B, MoE\) has*Mamba\-2*\(scalar\-decay\) linear layers—44FA and3636Mamba\-2 of4040layers \(1:91\{:\}9,90%90\\%recurrent\)\.OLMo\-Hybrid\-7B\-Instruct\(Merrillet al\.,[2026](https://arxiv.org/html/2608.11231#bib.bib37)\)\(77B\) andQwen3\.6\-27B\(Qwen Team,[2026](https://arxiv.org/html/2608.11231#bib.bib38)\)\(2727B\) use*Gated DeltaNet*\(GDN; dense transition\):88FA/2424GDN of3232layers and1616FA/4848GDN of6464layers, respectively \(both1:31\{:\}3,75%75\\%recurrent\)\.

#### Data and metrics\.

Following prior PIC work\(Yaoet al\.,[2025](https://arxiv.org/html/2608.11231#bib.bib5); Huet al\.,[2025](https://arxiv.org/html/2608.11231#bib.bib6); Wanget al\.,[2026](https://arxiv.org/html/2608.11231#bib.bib7)\), we use long\-context benchmarks whose prompts pair a large reusable context with a short query \(the retrieval\-augmented generation, RAG, setting\)\(Lewiset al\.,[2020](https://arxiv.org/html/2608.11231#bib.bib41)\): \(i\) five LongBench QA datasets\(Baiet al\.,[2024](https://arxiv.org/html/2608.11231#bib.bib16)\)—HotpotQA \(HQA\), 2WikiMQA \(2Wiki\), MuSiQue \(MSQ\), NarrativeQA \(NQA\), Qasper \(Qsp\); \(ii\) RULER\(Hsiehet al\.,[2024](https://arxiv.org/html/2608.11231#bib.bib17)\)at 8K–32K, with subtasks common\-/frequent\-word extraction \(CWE, FWE\), single/multi\-key/multi\-query/multi\-value needle retrieval \(N\-S, N\-MK, N\-MQ, N\-MV\), SQuAD\-style QA, and variable tracking \(VT\); and \(iii\) three LongBench summarization datasets \(QMSum, GovReport, MultiNews; 512\-token generation\)\. LongBench QA usesn=200n\{=\}200; RULER \(both 8K and 32K\) and LongBench summarization usen=100n\{=\}100\. Quality is token\-level F1 for QA \(averaged as Avg\-F1\), string\-match recall for RULER, and ROUGE\-L\(Lin,[2004](https://arxiv.org/html/2608.11231#bib.bib42)\)for summarization; we also report %\-of\-full==score//full\-recompute×100\\times 100\. Following ProphetKV, each context is split into fixed, contiguous 512\-token chunks, each prefilled and cached independently\. Decoding is greedy and every selector is deterministic, so runs are reproducible up to the recurrentfla/Mamba\-2 Triton kernels’ nondeterminism, which leaves a small run\-to\-run spread on Mamba\-2; we therefore read the two GDN initializers \(pairedex/lbgap≤0\.013\\leq 0\.013Avg\-F1 and sign\-changing\) as interchangeable, while the Granite gaps we highlight \(e\.g\.0\.1450\.145vs\.0\.2700\.270under EPIC\) are an order of magnitude larger\.

#### Methods\.

The selectorΠ\\Piis CacheBlend, EPIC, or ProphetKV; we pair exact composition and LinearKV under the same selected positions and recomputation\. Two anchors bound every table:*full recompute*\(r=1r\{=\}1, lossless upper bound\) and*naïve reuse*\(r=0r\{=\}0, the last\-block state with no recomputation, lower bound\)\. The*effective recompute ratio*isr=\|𝒰\|/Nr=\|\\mathcal\{U\}\|/N, the fraction of theNNcontext tokens the selector recomputes; since both initializers process the identical𝒰\\mathcal\{U\}through identical layers, any quality difference is attributable toSinitS\_\{\\mathrm\{init\}\}alone\. Main\-table runs use matchedr≈0\.2r\{\\approx\}0\.2; sweeps coverr∈\{0\.03,0\.05,0\.1,0\.2,0\.3,0\.4\}r\\in\\\{0\.03,0\.05,0\.1,0\.2,0\.3,0\.4\\\}\.

#### Environment\.

Experiments run on NVIDIA H800 GPUs \(80 GB\) using HuggingFace Transformers\(Wolfet al\.,[2020](https://arxiv.org/html/2608.11231#bib.bib43)\)withflash\-linear\-attentionkernels\(Yang and Zhang,[2024](https://arxiv.org/html/2608.11231#bib.bib44)\)inbfloat16; each model fits one device, so GPUs shard the workload across datasets\. Cached artifacts—per\-chunk FA KV and each recurrent layer’s end state \(plus, for exact composition, the per\-token key/gate stream folded online\)—are stored inbfloat16and loaded from host memory at serving time\. TTFT \(Table[3](https://arxiv.org/html/2608.11231#S4.T3)\) is measured on a single exclusive GPU\.

### 4\.2Main Results

Tables[1](https://arxiv.org/html/2608.11231#S4.T1)and[2](https://arxiv.org/html/2608.11231#S4.T2)evaluate LinearKV as a hybrid\-PIC framework: three existing PIC selectors—CacheBlend, EPIC, and ProphetKV—plug in unchanged, each run with two initializers under identical selected positions and recomputation—exact composition \(ex\) and LinearKV’s last\-block state \(lb\)—at a matched∼20%\{\\sim\}20\\%budget, with full recompute and naïve reuse as anchors\. All four benchmarks are reported per dataset or subtask: token\-F1 on LongBench QA and ROUGE\-L on LongBench summarization \(Table[1](https://arxiv.org/html/2608.11231#S4.T1)\), and string\-match recall on RULER at 8K and 32K \(Table[2](https://arxiv.org/html/2608.11231#S4.T2)\)\. Summarization and RULER\-32K confirm the split holds under generation and at longer context\.

Our first finding concerns the framework itself: with nothing more than the decoupled initialization, every existing selector transfers to hybrid models\. On the two GDN models the choice of linear\-state initializer is immaterial—the pairedex/lbgap is at most0\.0130\.013Avg\-F1 and even changes sign across selectors—yet both recover most of full quality, reaching92%92\\%under ProphetKV \(OLMo, RULER\-8K Avg; Table[2](https://arxiv.org/html/2608.11231#S4.T2)\)\.

The second finding is more surprising: the equivalence breaks on Mamba\-2, and in the counterintuitive direction the principled construction is the fragile one\. On Granite,excollapses under*every*selector whilelbstays robust \(Avg\-F10\.063/0\.145/0\.1250\.063/0\.145/0\.125vs\.0\.118/0\.270/0\.2050\.118/0\.270/0\.205for CacheBlend/EPIC/ProphetKV\)\. The gap is largest under EPIC, where changing*only*the initializer—not the selector or the recomputed positions—lifts quality from46\.6%46\.6\\%to86\.8%86\.8\\%of full\. RULER reproduces the split under a recall metric \(Table[2](https://arxiv.org/html/2608.11231#S4.T2); Granite Avg0\.030/0\.110/0\.3620\.030/0\.110/0\.362vs\.0\.375/0\.596/0\.6220\.375/0\.596/0\.622\), and it persists at4×4\\timescontext \(RULER\-32K Avg, Granite0\.043/0\.069/0\.0770\.043/0\.069/0\.077vs\.0\.419/0\.631/0\.6070\.419/0\.631/0\.607\) and under generation on summarization \(Table[1](https://arxiv.org/html/2608.11231#S4.T1), Summ Avg, Granite0\.146/0\.197/0\.1980\.146/0\.197/0\.198vs\.0\.245/0\.269/0\.2570\.245/0\.269/0\.257\)\.111One cell warrants explicit comment: Granite’s full\-recompute score on RULER CWE is0\.0030\.003,*below*its own naïve\-reuse score of0\.1920\.192\. This is a model\-level failure on that subtask, not a reuse artifact or a scoring bug: under the identical pipeline, prompt, and metric, OLMo and Qwen reach0\.8690\.869and0\.9980\.998, and Granite’s answers there are well formed but list words with no overlap with the reference set\. Excluding CWE, Granite’s RULER\-8K full\-recompute average is0\.8000\.800rather than0\.7000\.700, and the %\-of\-full figures we report for Granite on RULER shift by at most2\.32\.3pp; we retain CWE in the average rather than dropping an unfavorable subtask\.Because last\-block matches exact composition on GDN yet dominates it on Mamba\-2 under every selector, LinearKV adopts it as the default; the ablations below trace the collapse to the multi\-source construction, not the architecture\.

### 4\.3Efficiency: TTFT comparison

Table 3:TTFT across models and context lengths at a matched recomputation ratior=0\.2r\{=\}0\.2, with the initial\-state construction held side by side:lb= LinearKV andex= exact composition\. Each cell is TTFT in milliseconds \(median over44documents×\\times33repeats;p90p\_\{90\}within2%2\\%\)\.Having shown that the last\-block initial state matches or exceeds exact composition on quality, we turn to cost\. Table[3](https://arxiv.org/html/2608.11231#S4.T3)reports TTFT atr=0\.2r\{=\}0\.2\. Both initializers make reuse a clear win over full prefix recomputation: in every model, context, and selector, TTFT stays well below a full recompute—at 32K last\-block needs only0\.460\.46–0\.62×0\.62\\timesfull\-recomputation TTFT and exact composition≤0\.69×\{\\leq\}0\.69\\times, and the benefit already holds at 8K\. Hybrid PIC therefore pays off regardless of how the linear state is initialized\.

Between the two, last\-block is uniformly cheaper: it is faster than exact composition in*all2727measured pairs*, by55–17%17\\%: on GDN, exact composition must online\-fold a dense per\-chunk transitionTCjT\_\{C\_\{j\}\}, whereas last\-block reads a single state and folds nothing\. Last\-block thus wins on*both*axes—the better end\-to\-end quality of Table[1](https://arxiv.org/html/2608.11231#S4.T1)*and*the lower TTFT here—further confirming its advantage\. Efficiency is single\-request TTFT on an idle GPU; batched serving and host\-to\-device cache transfer are future work\.

### 4\.4Ablation Studies

Having shown that the last\-block initial state wins on both quality \(Table[1](https://arxiv.org/html/2608.11231#S4.T1)\) and latency \(Table[3](https://arxiv.org/html/2608.11231#S4.T3)\), we now examine exact composition’s failure: more recompute does not fix it, and it tracks the multi\-source construction rather than the architecture\.

![Refer to caption](https://arxiv.org/html/2608.11231v1/x4.png)Figure 4:Exact composition init vs\. lastblock init across recompute ratios \(EPIC selector; Granite, OLMo\)\.#### A bad initializer cannot be rescued by more recompute\.

We hold the selector, its positions, and the recomputation fixed, and vary*only*the linear\-state initializer—exact composition versus last\-block—across recompute budgetsr∈\{0\.03,0\.05,0\.1,0\.2,0\.3,0\.4\}r\\in\\\{0\.03,0\.05,0\.1,0\.2,0\.3,0\.4\\\}\(Figure[4](https://arxiv.org/html/2608.11231#S4.F4), EPIC selector\)\. On Granite \(Mamba\-2\), exact composition stays flat at4141–52%52\\%of full quality at*every*budget while last\-block reaches7676–89%89\\%; the3535–4040pp gap never closes, even atr=0\.4r\{=\}0\.4\. The initializer therefore sets a quality ceiling that added recompute cannot lift—a bad initial state is not bought back with more repair\. This also locates the failure: because last\-block, a*different*initializer, recovers most of full quality on the*same*model, exact’s collapse is a property of its multi\-source construction, not a Mamba\-2 reuse ceiling \(on OLMo the two coincide at every budget\)\. The same gap holds across selectors atr=0\.2r\{=\}0\.2: on Granite, exact composition loses0\.0550\.055–0\.1250\.125Avg\-F1 to the last\-block initial state under CacheBlend, EPIC, and ProphetKV—largest under EPIC, whose cell drops from0\.2700\.270to0\.1450\.145, i\.e\. from86\.8%86\.8\\%to46\.6%46\.6\\%of full quality—while on the two GDN models the paired gap stays within±0\.013\\pm 0\.013and changes sign\. This asymmetry matches the depth\-wise error diagnosis of Figure[3](https://arxiv.org/html/2608.11231#S3.F3): on Mamba\-2 the composition error compounds through depth, while on GDN it stays bounded\.

![Refer to caption](https://arxiv.org/html/2608.11231v1/x5.png)Figure 5:End\-to\-end comparison of LinearKV and HYPIC\. For fairness both use HYPIC’s seam selector; the only variable is the initializer—HYPIC’s exact composition vs\. LinearKV’s lastblock init\.
#### Comparison with a HYPIC\-style seam selector\.

Concurrent HYPIC pairs a*seam*selector—recomputing a fixed window around each chunk boundary—with exact composition\. We compare LinearKV against HYPIC’s initializer under this shared seam selector, holding the seam positions fixed and changing only the linear\-state initializer—HYPIC’s exact composition vs\. LinearKV’s last block \(Figure[5](https://arxiv.org/html/2608.11231#S4.F5)\); to keep the initializer the sole variable, neither side adds HYPIC’s remaining pipeline stages \(boundary\-token exclusion, per\-seam recomposition, causal\-convolution warm\-up\)\. The main\-table pattern survives intact\. At a matched∼20%\{\\sim\}20\\%budget \(Figure[5](https://arxiv.org/html/2608.11231#S4.F5)a\), replacing exact composition with the last\-block initializer lifts Granite from47\.6%47\.6\\%to67\.8%67\.8\\%of full recompute—a2020pp gain from the initializer alone—while OLMo and Qwen stay within22pp\. Sweeping the seam window on Granite \(Figure[5](https://arxiv.org/html/2608.11231#S4.F5)b\) shows the same separation at every budget, with no sign of closing\. LinearKV’s last\-block initializer therefore dominates HYPIC’s exact composition under the shared seam selector: it removes the Mamba\-2 fragility and costs nothing on GDN\.

## 5Related Work

#### Position\-independent caching for LLMs\.

PIC reuses independently prefilled chunks regardless of prefix, recomputing a few tokens to restore cross\-chunk context\. For full\-attention models, CacheBlend\(Yaoet al\.,[2025](https://arxiv.org/html/2608.11231#bib.bib5)\), EPIC\(Huet al\.,[2025](https://arxiv.org/html/2608.11231#bib.bib6)\), KVShare\(Yanget al\.,[2025b](https://arxiv.org/html/2608.11231#bib.bib21)\), ProphetKV\(Wanget al\.,[2026](https://arxiv.org/html/2608.11231#bib.bib7)\), KVLink\(Yanget al\.,[2025c](https://arxiv.org/html/2608.11231#bib.bib8)\), PromptCache\(Gimet al\.,[2024](https://arxiv.org/html/2608.11231#bib.bib18)\), CacheClip\(Yanget al\.,[2025a](https://arxiv.org/html/2608.11231#bib.bib30)\), Cache\-Craft\(Agarwalet al\.,[2025](https://arxiv.org/html/2608.11231#bib.bib29)\), CacheSlide\(Liuet al\.,[2026a](https://arxiv.org/html/2608.11231#bib.bib23)\), and MiniPIC\(Ordonez and Parnell,[2026](https://arxiv.org/html/2608.11231#bib.bib31)\)vary the chunking, token\-selection, repair, and engine\-integration policies, while MEPIC\(Wanget al\.,[2025](https://arxiv.org/html/2608.11231#bib.bib32)\)and C2KV\(Duet al\.,[2026](https://arxiv.org/html/2608.11231#bib.bib33)\)make the reused KV memory\-efficient and composable\. A parallel line adapts PIC to non\-token\-indexed representations: Irminsul\(Maet al\.,[2026](https://arxiv.org/html/2608.11231#bib.bib34)\)for multi\-head latent attention, COMB\(Zhaoet al\.,[2026](https://arxiv.org/html/2608.11231#bib.bib35)\)via a retrained encoder, and concurrent HYPIC\(Liuet al\.,[2026b](https://arxiv.org/html/2608.11231#bib.bib11)\)for hybrid LLMs via exact operator composition\. LinearKV joins this line for the recurrent\-state representation, training\-free: it keeps the compile/link workflow and existing selectors unchanged, and its contribution begins where the token\-indexed abstraction ends—how the same selected positions initialize and advance recurrent states to jointly repair a hybrid cache\.

#### SSM and hybrid\-state caching\.

Marconi\(Panet al\.,[2025](https://arxiv.org/html/2608.11231#bib.bib12)\), sparse prefix caching\(Shirokikh and Nikolenko,[2026](https://arxiv.org/html/2608.11231#bib.bib13)\), and compiler\-level SSD caching\(Santoni and Thapar,[2026](https://arxiv.org/html/2608.11231#bib.bib14)\)reuse recurrent states only at exact prefix checkpoints, not under lossy assembly of independently prefilled segments\. Recurrent\-state error has been studied from an error\-control view\(Chunget al\.,[2026](https://arxiv.org/html/2608.11231#bib.bib15)\); we identify independent\-prefill hidden\-input mismatch as a concrete source\.

#### AI\-infrastructure positioning\.

Cache reuse instantiates the computation–memory leg of the computation–bandwidth–memory trade\-off\(Fanet al\.,[2026](https://arxiv.org/html/2608.11231#bib.bib4)\); the same lens motivates generative, task\-oriented transmission when bandwidth binds\(Chenet al\.,[2026](https://arxiv.org/html/2608.11231#bib.bib3)\)\. Hybrid LLMs are natural device–edge targets\(Anet al\.,[2026](https://arxiv.org/html/2608.11231#bib.bib1)\), where a principled quality–recompute dial beats an all\-or\-nothing cache\.

## 6Conclusion

We studied cross\-request cache reuse for hybrid LLMs and showed that composing allKKcached states into the exact full\-prefix state is input\-conditional and architecture\-fragile—harmful on the evaluated Mamba\-2 model and unnecessary on two GDN models—because independently prefilled deep operators are built from context\-mismatched hidden inputs\. LinearKV instead keeps a single cached block as the initial state and turns any existing selector’s positions into an ordered recomputation stream that repairs recurrent states and FA KV jointly, raising Mamba\-2 quality from46\.6%46\.6\\%to86\.8%86\.8\\%of full under a matched budget while avoiding exact composition’s55–17%17\\%online overhead\. The result holds across LongBench QA and RULER at 8K–32K\.

## References

- S\. Agarwal, S\. Sundaresan, S\. Mitra, D\. Mahapatra, A\. Gupta, R\. Sharma, N\. J\. Kapu, T\. Yu, and S\. Saini \(2025\)Cache\-Craft: managing chunk\-caches for efficient retrieval\-augmented generation\.InProceedings of the ACM on Management of Data \(SIGMOD\),Note:arXiv:2502\.15734Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- H\. An, W\. Hu, S\. Huang, S\. Huang, R\. Li, Y\. Liang, J\. Shao, Y\. Song, Z\. Wang, C\. Yuan, C\. Zhang, H\. Zhang, W\. Zhuang, and X\. Li \(2026\)AI flow: perspectives, scenarios, and approaches\.Vicinagearth\.Note:arXiv:2506\.12479Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px3.p1.1)\.
- Y\. Bai, X\. Lv, J\. Zhang, H\. Lyu, J\. Tang, Z\. Huang, Z\. Du, X\. Liu, A\. Zeng, L\. Hou, Y\. Dong, J\. Tang, and J\. Li \(2024\)LongBench: a bilingual, multitask benchmark for long context understanding\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\),Note:arXiv:2308\.14508Cited by:[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px2.p1.8)\.
- Generative transmission: rethinking computation, bandwidth, and memory in communication\.External Links:2607\.17482Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px3.p1.1)\.
- J\. Chung, H\. Choi, and S\. J\. Kim \(2026\)Rethinking state tracking in recurrent models through error control dynamics\.External Links:2605\.07755Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px2.p1.1)\.
- T\. Dao and A\. Gu \(2024\)Transformers are SSMs: generalized models and efficient algorithms through structured state space duality\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),Note:arXiv:2405\.21060Cited by:[§1](https://arxiv.org/html/2608.11231#S1.p3.1),[§2](https://arxiv.org/html/2608.11231#S2.p1.1)\.
- C\. Du, J\. Chen, H\. Tang, K\. Liu, T\. Lan, L\. Qu, C\. Niu, S\. Liu, G\. Chen, and F\. Wu \(2026\)C2KV: compressed and composable KV cache reuse for efficient LLM inference\.Note:arXiv:2607\.17715Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- Y\. Fan, Q\. Weng, and X\. Li \(2026\)Computation\-bandwidth\-memory trade\-offs: a unified paradigm for AI infrastructure\.External Links:2601\.11577Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px3.p1.1)\.
- I\. Gim, G\. Chen, S\. Lee, N\. Sarda, A\. Khandelwal, and L\. Zhong \(2024\)Prompt cache: modular attention reuse for low\-latency inference\.InProceedings of Machine Learning and Systems \(MLSys\),Vol\.6\.Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- Granite Team, IBM \(2025\)Granite 4\.0 language models\.Note:[https://huggingface\.co/collections/ibm\-granite/granite\-40\-language\-models](https://huggingface.co/collections/ibm-granite/granite-40-language-models)IBM ResearchCited by:[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px1.p1.16)\.
- C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, Y\. Zhang, and B\. Ginsburg \(2024\)RULER: what’s the real context size of your long\-context language models?\.InConference on Language Modeling \(COLM\),Note:arXiv:2404\.06654Cited by:[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px2.p1.8)\.
- J\. Hu, W\. Huang, W\. Wang, H\. Wang, T\. Hu, Q\. Zhang, H\. Feng, X\. Chen, Y\. Shan, and T\. Xie \(2025\)EPIC: efficient position\-independent caching for serving large language models\.InProceedings of the 42nd International Conference on Machine Learning \(ICML\),Note:arXiv:2410\.15332Cited by:[§1](https://arxiv.org/html/2608.11231#S1.p2.1),[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px2.p1.8),[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with PagedAttention\.InProceedings of the 29th ACM Symposium on Operating Systems Principles \(SOSP\),Note:arXiv:2309\.06180Cited by:[§1](https://arxiv.org/html/2608.11231#S1.p1.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela \(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2005\.11401Cited by:[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px2.p1.8)\.
- C\. Lin \(2004\)ROUGE: a package for automatic evaluation of summaries\.InText Summarization Branches Out, ACL Workshop,Cited by:[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px2.p1.8)\.
- Y\. Liu, Y\. Gu, L\. Zhang, C\. Wu, G\. Xue, J\. Li, M\. Guo, J\. Hu, and J\. Meng \(2026a\)CacheSlide: unlocking cross position\-aware KV cache reuse for accelerating LLM serving\.InProceedings of the 24th USENIX Conference on File and Storage Technologies \(FAST\),pp\. 83–99\.Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- Y\. Liu, J\. Wu, Y\. Liu, J\. Hu, M\. Li, X\. Chen, and W\. Chen \(2026b\)HYPIC: accelerating hybrid\-attention LLM serving with position\-independent caching\.External Links:2607\.01299Cited by:[§1](https://arxiv.org/html/2608.11231#S1.p4.4),[§3\.2](https://arxiv.org/html/2608.11231#S3.SS2.SSS0.Px1.p1.8),[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- B\. Ma, J\. Eitzinger, and H\. Köstler \(2026\)Irminsul: MLA\-native position\-independent caching for agentic LLM serving\.Note:arXiv:2605\.05696Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- W\. Merrill, Y\. Li, T\. Romero, A\. Svete, C\. Costello, P\. Dasigi, D\. Groeneveld, D\. Heineman, B\. Kuehl, N\. Lambert, C\. Li, K\. Lo, S\. Malik, B\. Minixhofer, J\. Morrison, L\. Soldaini, P\. Walsh, N\. A\. Smith, H\. Hajishirzi, and A\. Sabharwal \(2026\)Olmo Hybrid: from theory to practice\.Note:[https://allenai\.org/papers/olmo\-hybrid](https://allenai.org/papers/olmo-hybrid)Allen Institute for AICited by:[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px1.p1.16)\.
- N\. Ordonez and T\. Parnell \(2026\)MiniPIC: flexible position\-independent caching in<100\{<\}100LOC\.Note:arXiv:2606\.13126Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- R\. Pan, Z\. Wang, Z\. Jia, C\. Karakus, L\. Zancato, T\. Dao, Y\. Wang, and R\. Netravali \(2025\)Marconi: prefix caching for the era of hybrid LLMs\.InProceedings of Machine Learning and Systems \(MLSys\),Note:arXiv:2411\.19379Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px2.p1.1)\.
- Qwen Team \(2026\)Qwen3\.6\.Note:[https://github\.com/QwenLM/Qwen3\.6](https://github.com/QwenLM/Qwen3.6)Alibaba GroupCited by:[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px1.p1.16)\.
- C\. Santoni and A\. Thapar \(2026\)Compiler\-first state space duality and portableO​\(1\)O\(1\)autoregressive caching for inference\.Transactions on Machine Learning Research\.Note:arXiv:2603\.09555Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px2.p1.1)\.
- M\. Shirokikh and S\. Nikolenko \(2026\)Sparse prefix caching for hybrid and recurrent LLM serving\.External Links:2605\.05219Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px2.p1.1)\.
- Q\. Wang, Z\. Yousefijamarani, M\. L\. Heisler,et al\.\(2025\)MEPIC: memory efficient position independent caching for LLM serving\.Note:arXiv:2512\.16822Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- S\. Wang, J\. Chen, Y\. Pan, H\. Huang, Y\. Hao, X\. Zou, W\. Xia, W\. Zhang, C\. Qiu, and P\. Wang \(2026\)ProphetKV: user\-query\-driven selective recomputation for efficient KV cache reuse in retrieval\-augmented generation\.External Links:2602\.02579Cited by:[§1](https://arxiv.org/html/2608.11231#S1.p2.1),[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px2.p1.8),[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- T\. Wolf, L\. Debut, V\. Sanh, J\. Chaumond, C\. Delangue, A\. Moi, P\. Cistac, T\. Rault, R\. Louf, M\. Funtowicz, J\. Davison, S\. Shleifer, P\. von Platen, C\. Ma, Y\. Jernite, J\. Plu, C\. Xu, T\. Le Scao, S\. Gugger, M\. Drame, Q\. Lhoest, and A\. M\. Rush \(2020\)Transformers: state\-of\-the\-art natural language processing\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\): System Demonstrations,Note:arXiv:1910\.03771Cited by:[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px4.p1.1)\.
- B\. Yang, Q\. Leng, J\. Zeng, and Z\. Wu \(2025a\)CacheClip: accelerating RAG with effective KV cache reuse\.Note:arXiv:2510\.10129Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- H\. Yang, R\. Zhang, M\. Huang, W\. Wang, Y\. Tang, Y\. Li, Y\. Liu, and D\. Zhang \(2025b\)KVShare: an LLM service system with efficient and effective multi\-tenant KV cache reuse\.External Links:2503\.16525Cited by:[§1](https://arxiv.org/html/2608.11231#S1.p2.1),[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- J\. Yang, B\. Hou, W\. Wei, Y\. Bao, and S\. Chang \(2025c\)KVLink: accelerating large language models via efficient KV cache reuse\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2502\.16002Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- S\. Yang, J\. Kautz, and A\. Hatamizadeh \(2025d\)Gated delta networks: improving Mamba2 with delta rule\.InThe Thirteenth International Conference on Learning Representations \(ICLR\),Note:arXiv:2412\.06464Cited by:[§1](https://arxiv.org/html/2608.11231#S1.p3.1),[§2](https://arxiv.org/html/2608.11231#S2.p1.1)\.
- S\. Yang and Y\. Zhang \(2024\)FLA: a triton\-based library for hardware\-efficient implementations of linear attention mechanism\.Note:[https://github\.com/fla\-org/flash\-linear\-attention](https://github.com/fla-org/flash-linear-attention)Cited by:[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px4.p1.1)\.
- J\. Yao, H\. Li, Y\. Liu, S\. Ray, Y\. Cheng, Q\. Zhang, K\. Du, S\. Lu, and J\. Jiang \(2025\)CacheBlend: fast large language model serving for RAG with cached knowledge fusion\.InProceedings of the Twentieth European Conference on Computer Systems \(EuroSys\),Note:arXiv:2405\.16444Cited by:[§1](https://arxiv.org/html/2608.11231#S1.p2.1),[§4\.1](https://arxiv.org/html/2608.11231#S4.SS1.SSS0.Px2.p1.8),[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- S\. Zhao, J\. Hu, J\. Zheng, and G\. Chen \(2026\)You need an encoder for native position\-independent caching\.Note:arXiv:2602\.01519Cited by:[§5](https://arxiv.org/html/2608.11231#S5.SS0.SSS0.Px1.p1.1)\.
- L\. Zheng, L\. Yin, Z\. Xie, C\. Sun, J\. Huang, C\. H\. Yu, S\. Cao, C\. Kozyrakis, I\. Stoica, J\. E\. Gonzalez, C\. Barrett, and Y\. Sheng \(2024\)SGLang: efficient execution of structured language model programs\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2312\.07104Cited by:[§1](https://arxiv.org/html/2608.11231#S1.p1.1)\.

## Appendix ASingle\-source ablation: last block vs\. random block

A natural worry is that our result hinges on the last block in particular—its recency, or its adjacency to the query—rather than on using a single source\. It does not\. We add a*random\-block*control that initializes each recurrent layer from one*randomly chosen*matched chunk’s cached state \(seeded; same selector, positions, and budget\), and compare all three initializers atr=0\.2r\{=\}0\.2\(Table[4](https://arxiv.org/html/2608.11231#A1.T4)\)\. On Granite \(Mamba\-2\) the random block tracks the last block—within0\.0140\.014Avg\-F1 at every selector—and both roughly double exact composition, so what matters is reading from a*single*cached source, not which source\. On the two GDN models all three initializers agree within0\.020\.02Avg\-F1, as expected where composition does not compound\.

Two further controls delimit the effect: naïve reuse \(Table[1](https://arxiv.org/html/2608.11231#S4.T1)\)—last\-block*without*recomputation—trails every recomputed single\-source initializer, so recomputation is necessary; and the three selectors, choosing different positions, all clear naïve reuse, so the selected positions matter too\. Last\-block is therefore not a special trick but the natural zero\-cost instance of single\-source initialization—the most recent chunk, already cached, needing no extra selection—which is why we adopt it as the default\.

Table 4:Single\-block init \(random vs\. last\-block\) compared with exact composition\. Granite \(Mamba\-2\), Avg\-F1 atr=0\.2r\{=\}0\.2\.