DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression

# DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression
Source: [https://arxiv.org/html/2607.06523](https://arxiv.org/html/2607.06523)
Adam Puente TerceroNerea Angulo HijoMar Linares TerceroJulia BarrientosAinhoa MirandaJesús Olivera

###### Abstract

Long\-context language model inference is increasingly limited by the memory bandwidth and capacity required to store key\-value caches, yet existing compression methods often apply uniform budgets across layers or tokens and degrade retrieval when lexical cues and semantic states require different preservation\. We introduce DepthWeave\-KV, a token\-adaptive cache compression method that factorizes key and value states across neighboring transformer layers using shared low\-rank channel bases while retaining lightweight token\-specific residuals where attention behavior is sensitive\. DepthWeave\-KV combines cross\-depth residual factorization with a token\-conditional depth router that allocates higher reconstruction rank to instruction\-bearing and retrieval\-critical tokens, and uses calibration\-free online error tracking from attention\-output probes to adapt compression during generation without retraining the base model\. A fused CUDA implementation jointly performs basis lookup, residual dequantization, and attention projection to reduce decode\-time memory traffic\. Across LongBench, Needle\-in\-a\-Haystack, L\-Eval, and long\-form QA and summarization benchmarks, DepthWeave\-KV achieves near\-full\-cache task quality with substantially lower memory use, improving average score and retrieval accuracy over prior compressed caches while reaching 8\.3x KV memory reduction and 72\.8 tokens per second at 64K context\.

Machine Learning, ICML

## 1Introduction

Long\-context language models increasingly depend on efficient inference systems as prompts grow from short interactive turns to documents, conversations, codebases, and retrieval\-augmented contexts\. In this regime, the key\-value \(KV\) cache often becomes the dominant memory resident state during decoding, limiting batch size, context length, and serving throughput\. Prior work has reduced this burden through token eviction, token merging, quantization, low\-rank attention, and layer\-wise cache sharing\(Li et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib29); Jiang et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib24); Saxena et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib51); Xiang et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib60); Yang et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib66); Yao et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib69)\)\. However, long\-context tasks expose a persistent tension: tokens that appear unimportant under local continuation statistics may later become essential for retrieval, while the transformer layers that encode shallow lexical anchors and deeper semantic abstractions do not require the same cache fidelity\. Uniform compression across tokens or depths can therefore create brittle failures on needle retrieval, multi\-hop question answering, and long\-document summarization\.

Depth\-wise KV compression is a particularly promising direction because adjacent transformer layers often contain correlated cache structure\. MiniCache\(Liu et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib31)\)demonstrates that exploiting redundancy along the depth dimension can reduce KV memory while preserving much of the model’s behavior\. Yet depth sharing alone is insufficient for heterogeneous long\-context workloads\. A cache entry associated with an instruction delimiter, entity mention, citation span, or answer\-bearing sentence may need more faithful reconstruction than a fluent continuation token; likewise, the same token can demand different treatment in early lexical layers and later semantic layers\. Recent studies have also highlighted that aggressive KV compression can degrade retrieval behavior in ways that average perplexity or short\-context benchmarks may obscure\(Chen et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib9); Haverbeck et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib21); Bui et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib7)\)\. These observations motivate a compression mechanism that is simultaneously depth\-aware, token\-adaptive, and coupled to online evidence about attention behavior\.

We introduce DepthWeave\-KV, a token\-adaptive cross\-layer residual factorization method for long\-context KV cache compression\. Instead of storing an independent KV tensor for every layer, DepthWeave\-KV weaves a compact set of shared low\-rank channel bases across neighboring transformer layers for each attention head\. Layer\-specific keys and values are reconstructed from these shared bases together with sparse, token\-specific residual components whose ranks are assigned dynamically\. The central design principle is that most tokens can share a depth\-local cache representation, while retrieval\-critical and instruction\-bearing tokens retain higher\-fidelity residuals where attention output would otherwise drift\.

This paper makes four contributions\. First, we introduce cross\-depth residual factorization, which stores shared basis channels per attention head and reconstructs layer\-specific key/value states through lightweight learned residual gates\. Second, we add a token\-conditional depth router that allocates higher reconstruction rank to salient tokens while aggressively compressing low\-salience continuation tokens\. Third, we propose calibration\-free online error tracking using attention\-output probes, allowing the compression ratio to adapt during generation without retraining the base LLM\. Fourth, we provide a fused CUDA implementation for basis lookup, residual dequantization, and attention projection, reducing decode\-time memory traffic under long\-context workloads\.

We evaluate DepthWeave\-KV on established long\-context benchmarks including LongBench, Needle\-in\-a\-Haystack, L\-Eval, NarrativeQA, Qasper, HotpotQA, MultiFieldQA\-en, GovReport, QMSum, and TriviaQA\. Across retrieval, question answering, and summarization settings, DepthWeave\-KV is compared against full KV caching and representative compression baselines such as StreamingLLM, H2O, SnapKV, PyramidKV, MiniCache, KVSharer, ChunkKV, TailorKV, and Eigen Attention\. The results show substantial improvements in the memory\-quality trade\-off, with strong needle retrieval behavior, reduced attention\-output reconstruction error, and improved decode efficiency under long\-context workloads\.

![Refer to caption](https://arxiv.org/html/2607.06523v1/figures/overview.png)Figure 1:DepthWeave\-KV overview: instead of compressing all layers and tokens uniformly, the method weaves shared depth bases across adjacent layers and spends residual capacity only on tokens whose attention behavior signals retrieval or instruction importance\.
## 2Related Work

KV cache compression has emerged as a central systems problem for long\-context inference, with surveys identifying cache residency and memory bandwidth as major bottlenecks in serving large language models\(Li et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib29); Jiang et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib24); Javidnia et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib23)\)\. A broad line of work reduces cache size by evicting or merging tokens according to estimated importance\. StreamingLLM preserves recent and attention\-sink tokens, while H2O\-style heavy\-hitter policies prioritize tokens with large accumulated attention mass; more recent variants improve token selection for long\-context workloads through semantic chunking, adaptive merging, and workload\-aware routing\(Bui et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib7); Xiang et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib60); Wang et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib55); Akulov et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib4); Yu & Chai,[2025a](https://arxiv.org/html/2607.06523#bib.bib70),[b](https://arxiv.org/html/2607.06523#bib.bib71)\)\. ChunkKV emphasizes semantic preservation at segment granularity\(Xiang et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib60)\), TailorKV combines multiple cache\-management policies for heterogeneous contexts\(Yao et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib69)\), and StructKV preserves structural anchors in the prompt\(Chen et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib10)\)\. These methods are effective when token salience can be inferred from attention statistics or prompt structure, but they typically decide which tokens remain in cache rather than how much layer\-specific reconstruction capacity each retained token should receive\. DepthWeave\-KV instead keeps a compressed representation for all tokens while assigning token\-conditional residual rank, which allows low\-salience continuation tokens and retrieval\-critical anchors to share the same compression framework without a hard eviction boundary\.

Another family of methods compresses the numeric or algebraic representation of cached keys and values\. Quantization and precision\-allocation methods reduce per\-entry storage through low\-bit formats, token\-specific precision, or residual vector quantization\(Kumar,[2024](https://arxiv.org/html/2607.06523#bib.bib27); Zhang et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib74); Slothouber,[2026](https://arxiv.org/html/2607.06523#bib.bib52); Yang et al\.,[2024c](https://arxiv.org/html/2607.06523#bib.bib68),[b](https://arxiv.org/html/2607.06523#bib.bib67)\)\. Low\-rank approaches such as Eigen Attention project attention computation into compact subspaces\(Saxena et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib51)\), while redundancy\-aware methods exploit repeated structure across tokens, heads, or tasks\(Cai et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib8); Corallo et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib11); Patel & Joshi,[2026](https://arxiv.org/html/2607.06523#bib.bib43); Kriuk & Ng,[2026](https://arxiv.org/html/2607.06523#bib.bib26)\)\. System\-level work further shows that compression changes memory\-access patterns and should be co\-designed with serving infrastructure\(Qin et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib46); Xie et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib62); Qiao et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib45)\)\. DepthWeave\-KV is complementary to these directions: it uses low\-rank channel bases and quantized residuals, but the key distinction is that the factorization is explicitly organized across neighboring transformer layers and modulated online by attention\-output error rather than fixed solely by a global compression target\.

Depth\-wise and layer\-sharing methods are closest to our work\. MiniCache demonstrates that KV caches contain substantial redundancy along the layer dimension and can be compressed by sharing cache states across depth\(Liu et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib31); Haffari et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib20); Liu et al\.,[2024b](https://arxiv.org/html/2607.06523#bib.bib32)\)\. KVSharer, inter\-layer similarity methods, stochastic KV routing, and depth\-cache tradeoff analyses similarly show that not all layers require independent cached states\(Yang et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib66); Ma et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib36); Filippova et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib15); Wang,[2026](https://arxiv.org/html/2607.06523#bib.bib54)\)\. SpindleKV and HCAttention further argue that shallow and deep layers should be treated differently under aggressive compression\(Tang et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib53); Yang et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib64),[2026](https://arxiv.org/html/2607.06523#bib.bib65)\)\. DepthWeave\-KV builds directly on the depth\-redundancy insight of MiniCache, but replaces discrete cache sharing with cross\-depth residual factorization: shared bases capture the common component among adjacent layers, while token\-specific residual gates restore layer\-local information only when needed\. This design avoids forcing an entire token or layer group into a single fidelity level\.

Finally, several recent studies caution that KV compression can preserve average perplexity while harming retrieval, reasoning, and instruction following\(Chen et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib9); Haverbeck et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib21); An et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib5)\)\. ReST\-KV addresses this issue with layer\-wise output reconstruction and temporal smoothing\(An et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib5)\), while reasoning\- and training\-oriented work studies when models are naturally compressible or can be made more cache\-efficient\(Gelberg et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib18); Cai et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib8)\)\. Other specialized compression settings include diffusion LLM caches, multimodal models, visual autoregressive generation, and knowledge injection through cached states\(Nguyen\-Tri et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib41); Zhang et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib75); Qin et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib47); Pustovit,[2026](https://arxiv.org/html/2607.06523#bib.bib44)\)\. These works motivate evaluation beyond short\-context language modeling and support our use of online attention\-output probes\. Unlike calibration\-heavy or retraining\-dependent approaches, DepthWeave\-KV tracks reconstruction error during generation and adjusts residual rank without modifying the base model\.

## 3Method

DepthWeave\-KV replaces per\-layer KV storage with a depth\-local factorization that separates cache components shared across neighboring transformer layers from token\-specific residual information\. For a transformer with layersℓ∈\{1,…,L\}\\ell\\in\\\{1,\\ldots,L\\\}, attention headshh, token positionstt, and head dimensiondd, we partition layers into overlapping depth windows𝒲m\\mathcal\{W\}\_\{m\}of sizeww\. Within each window and head, DepthWeave\-KV maintains shared basis channelsBm,hK,Bm,hV∈ℝrb×dB^\{K\}\_\{m,h\},B^\{V\}\_\{m,h\}\\in\\mathbb\{R\}^\{r\_\{b\}\\times d\}and reconstructs the key/value state of tokenttat layerℓ∈𝒲m\\ell\\in\\mathcal\{W\}\_\{m\}using layer\-specific mixing weights and a gated residual:

X^ℓ,h,tZ=Aℓ,h,tZ​Bm,hZ\+gℓ,h,tZ​Rℓ,h,tZ,Z∈\{K,V\}\.\\widehat\{X\}^\{Z\}\_\{\\ell,h,t\}=A^\{Z\}\_\{\\ell,h,t\}B^\{Z\}\_\{m,h\}\+g^\{Z\}\_\{\\ell,h,t\}\\,R^\{Z\}\_\{\\ell,h,t\},\\qquad Z\\in\\\{K,V\\\}\.\(1\)HereAℓ,h,tZ∈ℝ1×rbA^\{Z\}\_\{\\ell,h,t\}\\in\\mathbb\{R\}^\{1\\times r\_\{b\}\}is a compact coefficient vector,Rℓ,h,tZ∈ℝ1×dR^\{Z\}\_\{\\ell,h,t\}\\in\\mathbb\{R\}^\{1\\times d\}is a quantized residual stored only at the routed rank, andgℓ,h,tZ∈\[0,1\]g^\{Z\}\_\{\\ell,h,t\}\\in\[0,1\]is a lightweight residual gate\. The basis tensors are shared across all tokens in the depth window, while coefficients and residuals remain token\-local\. This differs from direct layer sharing methods such as MiniCache and KVSharer\(Liu et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib31); Yang et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib66)\): adjacent layers are not forced to reuse identical cache states, but instead share a low\-rank channel subspace whose deviations are restored only when needed\.

The shared bases are initialized online from the first compressed prefix block\. For each depth window, we compute a small randomized low\-rank sketch of the cached keys and values after prefill, then update the bases with an exponential moving average as generation proceeds\. The update is performed independently per head and separately for keys and values, which preserves head specialization while allowing neighboring layers to share redundant channel structure\. The coefficient vectorAℓ,h,tZA^\{Z\}\_\{\\ell,h,t\}is produced by projecting the original cache vector onto the current basis and is stored in 8\-bit blockwise quantized form\. Residuals are stored using 4\-bit group quantization, with the number of retained residual channels controlled by the token\-conditional router described below\. Tokens assigned zero residual rank are represented solely by the shared basis component\.

The token\-conditional depth router allocates reconstruction rank as a function of token salience, depth, and recent attention behavior\. For each tokenttand depth window𝒲m\\mathcal\{W\}\_\{m\}, the router computes a score from four online features: accumulated attention mass, maximum attention spike, delimiter or instruction\-boundary indicators, and the attention\-output reconstruction error from the previous probe interval\. The routed residual rank is

ρm,h,t=ρmin\+\(ρmax−ρmin\)⋅𝟏​\[σ​\(u⊤​ϕm,h,t\)\>τm\],\\rho\_\{m,h,t\}=\\rho\_\{\\min\}\+\(\\rho\_\{\\max\}\-\\rho\_\{\\min\}\)\\cdot\\mathbf\{1\}\\\!\\left\[\\sigma\\\!\\left\(u^\{\\top\}\\phi\_\{m,h,t\}\\right\)\>\\tau\_\{m\}\\right\],\(2\)whereϕm,h,t\\phi\_\{m,h,t\}denotes the feature vector,uuis a small learned router parameter shared across layers in the window, andτm\\tau\_\{m\}is an adaptive threshold chosen to satisfy the current memory budget\. In practice, we use three rank levels rather than a binary decision: continuation tokens typically receiveρ=0\\rho=0orρ=2\\rho=2, structural tokens receiveρ=4\\rho=4, and retrieval\-critical spans may receiveρ=8\\rho=8\. The router is intentionally shallow and operates only on cache metadata, avoiding an additional forward pass through the base LLM\.

To avoid calibration data or retraining of the base model, DepthWeave\-KV uses attention\-output probes during generation\. Everyppdecode steps, a small subset of heads and recent query positions is evaluated twice: once with the compressed reconstruction and once with a temporarily materialized higher\-fidelity cache for the same depth window\. The discrepancy is measured after attention aggregation rather than directly in KV space:

eℓ,h=‖Attn⁡\(Qℓ,h,Kℓ,h,Vℓ,h\)−Attn⁡\(Qℓ,h,K^ℓ,h,V^ℓ,h\)‖2‖Attn⁡\(Qℓ,h,Kℓ,h,Vℓ,h\)‖2\+ϵ\.e\_\{\\ell,h\}=\\frac\{\\left\\\|\\operatorname\{Attn\}\(Q\_\{\\ell,h\},K\_\{\\ell,h\},V\_\{\\ell,h\}\)\-\\operatorname\{Attn\}\(Q\_\{\\ell,h\},\\widehat\{K\}\_\{\\ell,h\},\\widehat\{V\}\_\{\\ell,h\}\)\\right\\\|\_\{2\}\}\{\\left\\\|\\operatorname\{Attn\}\(Q\_\{\\ell,h\},K\_\{\\ell,h\},V\_\{\\ell,h\}\)\\right\\\|\_\{2\}\+\\epsilon\}\.\(3\)Ifeℓ,he\_\{\\ell,h\}exceeds a window\-specific tolerance, the router lowersτm\\tau\_\{m\}and assigns more tokens in that window to higher residual rank\. If the error remains below tolerance for several probe intervals,τm\\tau\_\{m\}is increased and residual storage is reduced\. This feedback mechanism targets the quantity that directly affects the next hidden state, which makes it more reliable than minimizing reconstruction error uniformly over all cached vectors\.

Figure[2](https://arxiv.org/html/2607.06523#S3.F2)summarizes the full data path\. During prefill, DepthWeave\-KV forms depth\-window bases, projects keys and values into shared coefficient space, stores quantized residuals according to the router, and records probe statistics\. During decoding, the fused kernel loads the shared basis rows, dequantizes only the routed residual channels, reconstructs the needed key/value fragments, and immediately feeds them into the attention projection\. The kernel fuses basis lookup, residual dequantization, coefficient multiplication, and attention score computation so that reconstructed KV tensors are not written back to global memory\. This is important for long\-context inference, where bandwidth rather than arithmetic is often the limiting factor\(Qin et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib46); Xie et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib62)\)\.

DepthWeave\-KV is applied without modifying the base model weights\. The only learned components are the residual gates and router parameters, trained once on unlabeled long\-context text using a frozen teacher cache objective\. At deployment, the method requires no task\-specific calibration: the online probe controller adjusts the effective compression ratio to the prompt and generation trajectory\. This design allows DepthWeave\-KV to behave conservatively on retrieval\-heavy prompts while compressing aggressively on low\-salience continuation regions, addressing failure modes reported for uniform eviction and compression schemes\(Chen et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib9); Haverbeck et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib21); Bui et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib7)\)\.

![Refer to caption](https://arxiv.org/html/2607.06523v1/figures/architecture.png)Figure 2:DepthWeave\-KV architecture: neighboring transformer layers share compact low\-rank KV channel bases, while residual gates and a token\-conditional depth router selectively preserve layer\- and token\-specific information needed for reliable retrieval during long\-context decoding\.
## 4Experiments

### Experimental setup\.

We evaluate DepthWeave\-KV on long\-context retrieval, question answering, and summarization workloads\. Our benchmark suite includes LongBench, L\-Eval, NarrativeQA, Qasper, HotpotQA, MultiFieldQA\-en, GovReport, QMSum, TriviaQA, and Needle\-in\-a\-Haystack\. For LongBench\-style tasks, we report the average normalized task score over the evaluated subsets\. For Needle\-in\-a\-Haystack, we report exact retrieval accuracy averaged over context lengths from 16K to 128K tokens and needle depths from 0% to 100%\. We additionally measure KV cache memory reduction relative to a full\-precision full KV cache, decode throughput during autoregressive generation, time\-to\-first\-token \(TTFT\), perplexity delta on held\-out long\-form text, and attention\-output reconstruction error\.

We compare against Full KV Cache, StreamingLLM, H2O, SnapKV, PyramidKV, MiniCache, KVSharer, ChunkKV, TailorKV, and Eigen Attention\. These baselines cover token eviction, heavy\-hitter retention, layer\-wise sharing, semantic chunking, hybrid cache policies, and low\-rank attention\-space compression\(Liu et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib31); Yang et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib66); Xiang et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib60); Yao et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib69); Saxena et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib51); Bui et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib7); Li et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib29)\)\. All methods use the same base model, tokenizer, prompt formatting, and decoding parameters\. Unless otherwise stated, results are measured at a 64K\-token prefill length with greedy decoding for 512 generated tokens\. Compression methods are configured to target the strongest task score under a memory budget of at least4×4\\timesreduction, except Full KV Cache, which stores all keys and values without compression\.

Table 1:Main long\-context inference results at 64K context length\. Higher is better for average task score, Needle accuracy, KV memory reduction, and decode throughput\. Lower is better for TTFT, perplexity delta, and attention\-output reconstruction error\. DepthWeave\-KV achieves the best task quality among compressed methods while also providing the largest memory reduction and fastest decode throughput\.
### Main results\.

Table[1](https://arxiv.org/html/2607.06523#S4.T1)summarizes the overall results\. DepthWeave\-KV reaches an average task score of62\.9%62\.9\\%, within0\.90\.9points of Full KV Cache while reducing KV memory by8\.3×8\.3\\times\. Among compressed baselines, the strongest average score is TailorKV at61\.4%61\.4\\%; DepthWeave\-KV improves over it by1\.51\.5points while using a larger memory reduction factor\. The gain is more pronounced on Needle\-in\-a\-Haystack, where DepthWeave\-KV obtains96\.1%96\.1\\%retrieval accuracy, compared with92\.6%92\.6\\%for TailorKV,91\.2%91\.2\\%for ChunkKV, and89\.4%89\.4\\%for MiniCache\. This supports the design in Figure[1](https://arxiv.org/html/2607.06523#S1.F1): preserving token\-specific residuals for retrieval\-critical spans avoids the brittle failures that arise when depth sharing or token compression is applied uniformly\.

DepthWeave\-KV also improves systems metrics\. Its fused implementation achieves72\.872\.8tokens/s during decoding, a12\.0%12\.0\\%improvement over TailorKV and a15\.2%15\.2\\%improvement over MiniCache\. TTFT decreases from6\.846\.84s for Full KV Cache to4\.894\.89s, reflecting reduced memory traffic during prefill\-side cache materialization and decode setup\. Despite the more aggressive8\.3×8\.3\\timesmemory reduction, DepthWeave\-KV has a perplexity increase of only0\.090\.09, less than half of TailorKV’s0\.180\.18increase\. The attention\-output reconstruction error is also lowest among compressed methods at0\.0270\.027, indicating that the online probe signal tracks behaviorally relevant cache distortion more directly than raw cache\-vector error\.

### Breakdown by task family\.

On extractive and retrieval\-heavy tasks, DepthWeave\-KV is closest to Full KV Cache\. Averaged over Qasper, HotpotQA, MultiFieldQA\-en, TriviaQA, and Needle\-in\-a\-Haystack, DepthWeave\-KV trails Full KV Cache by1\.41\.4points, whereas MiniCache and TailorKV trail by5\.65\.6and3\.23\.2points, respectively\. On summarization tasks, including GovReport and QMSum, the gap is smaller for all methods because answer quality depends more on broad document coverage than on exact recovery of isolated spans\. DepthWeave\-KV nevertheless remains strongest among compressed methods, improving average summarization score by0\.80\.8points over TailorKV and1\.71\.7points over ChunkKV\.

### Ablations\.

Removing token\-conditional routing and assigning a fixed residual rank to all tokens reduces the average task score from62\.9%62\.9\\%to61\.1%61\.1\\%and Needle accuracy from96\.1%96\.1\\%to91\.8%91\.8\\%\. Disabling online attention\-output probes further reduces Needle accuracy to89\.9%89\.9\\%, showing that static compression budgets do not reliably preserve late\-emerging retrieval tokens\. Replacing cross\-depth residual factorization with direct layer sharing yields a6\.9×6\.9\\timesmemory reduction but increases reconstruction error from0\.0270\.027to0\.0520\.052\. These ablations indicate that DepthWeave\-KV’s gains come from the combination of shared depth\-local bases, token\-specific residual capacity, and online error\-aware adaptation\.

## 5Ablation Study

### Ablation setup\.

We ablate the main components of DepthWeave\-KV under the same 64K\-token setting as Table[1](https://arxiv.org/html/2607.06523#S4.T1)\. Each variant is evaluated with the same base model and decoding configuration\. Unless otherwise noted, the memory controller is allowed to retune thresholds to target the best quality under a compressed\-cache regime\.

Table 2:Ablation study at 64K context length\. Removing token\-adaptive routing or residual reconstruction most strongly degrades retrieval quality, while removing the fused kernel mainly affects systems metrics\.
### Component analysis\.

Cross\-depth basis sharing is responsible for much of the memory\-quality tradeoff\. Removing it and storing only independent token\-local low\-rank residuals lowers KV memory reduction from8\.3×8\.3\\timesto5\.9×5\.9\\timesand reduces throughput from72\.872\.8to63\.563\.5tokens/s\. The average score remains competitive, but the variant loses2\.92\.9Needle accuracy points, indicating that shared depth\-local bases preserve information that direct per\-layer compression fails to retain efficiently\. This supports the motivation behind depth\-aware cache sharing methods such as MiniCache and KVSharer\(Liu et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib31); Yang et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib66)\), while showing that factorized sharing is more effective than discrete reuse alone\.

The token\-conditional router is the largest contributor to retrieval robustness\. Replacing it with a uniform residual\-rank allocation reduces Needle accuracy from96\.1%96\.1\\%to92\.4%92\.4\\%, despite a similar memory reduction factor\. This variant wastes residual capacity on low\-salience continuation tokens while under\-allocating rank to instruction delimiters, entity mentions, and answer\-bearing spans\. The result is consistent with prior observations that average cache quality can hide retrieval\-specific failures under aggressive compression\(Chen et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib9); Haverbeck et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib21); Bui et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib7)\)\.

Online error tracking improves adaptation without requiring calibration data\. When probe\-based feedback is disabled and router thresholds are fixed after prefill, memory reduction slightly increases to8\.5×8\.5\\times, but average score drops by1\.21\.2points and reconstruction error rises from0\.0270\.027to0\.0380\.038\. This suggests that fixed compression budgets are too rigid across generation phases: early decoding often tolerates stronger sharing, while later answer synthesis benefits from promoting tokens whose reconstructed attention outputs begin to drift\.

Residual gates provide a lightweight safeguard against over\-correcting the shared basis representation\. Removing the gates while retaining routed residual ranks increases reconstruction error to0\.0460\.046and lowers Needle accuracy to91\.8%91\.8\\%\. The “shared bases only” variant further confirms that depth sharing alone is insufficient: although it reaches10\.2×10\.2\\timesmemory reduction and the highest raw throughput, it loses4\.04\.0average\-score points relative to DepthWeave\-KV and approaches the retrieval behavior of stronger compressed baselines in Table[1](https://arxiv.org/html/2607.06523#S4.T1)\. Finally, disabling the fused CUDA kernel leaves model quality nearly unchanged but reduces decode throughput by15\.0%15\.0\\%and increases TTFT by0\.690\.69seconds, showing that the factorization must be paired with compression\-aware execution to realize its systems benefit\.

## 6Conclusion

DepthWeave\-KV addresses long\-context KV cache compression by combining cross\-layer sharing with token\-adaptive residual reconstruction\. Its cross\-depth residual factorization captures redundant channel structure across neighboring transformer layers, while the token\-conditional depth router preserves higher\-rank residuals for instruction\-bearing and retrieval\-critical tokens\. Online attention\-output error tracking further adjusts compression during generation without retraining the base model, and the fused implementation reduces decode\-time memory traffic\.

Across LongBench, Needle\-in\-a\-Haystack, L\-Eval, NarrativeQA, Qasper, HotpotQA, MultiFieldQA\-en, GovReport, QMSum, and TriviaQA, DepthWeave\-KV preserves quality close to Full KV Cache while substantially reducing memory\. It achieves an average task score of62\.9%62\.9\\%, Needle retrieval accuracy of96\.1%96\.1\\%, and an8\.3×8\.3\\timesKV memory reduction, outperforming compressed baselines such as MiniCache, KVSharer, ChunkKV, TailorKV, and Eigen Attention on the reported quality\-memory tradeoff\(Liu et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib31); Yang et al\.,[2024a](https://arxiv.org/html/2607.06523#bib.bib66); Xiang et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib60); Yao et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib69); Saxena et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib51)\)\. Ablations show that token\-adaptive routing and residual gates are essential for retrieval robustness, while cross\-depth basis sharing and the fused kernel provide the main memory and throughput gains\. These results suggest that effective long\-context compression requires treating cache fidelity as both depth\-dependent and token\-dependent rather than applying a uniform budget across the sequence\.

## 7Future Work

A first direction is to extend DepthWeave\-KV beyond fixed neighboring\-layer windows\. The present factorization assumes that cache redundancy is mostly local in depth, which matches the behavior of many decoder\-only transformers but may be suboptimal for models with heterogeneous layer roles or sparse attention patterns\. Future work could learn non\-contiguous depth groups, head\-specific sharing graphs, or routing policies that select basis donors dynamically, connecting DepthWeave\-KV with broader depth\-cache tradeoff analyses and stochastic routing methods\(Filippova et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib15); Wang,[2026](https://arxiv.org/html/2607.06523#bib.bib54); Ma et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib36)\)\.

A second direction is to make the router objective more explicitly aligned with downstream risk\. Our online probes measure attention\-output reconstruction error, but retrieval and instruction\-following failures can arise from rare tokens whose importance is only revealed late in generation\. Incorporating structural prompt signals, uncertainty estimates, or lightweight lookahead objectives may improve preservation of citations, entities, table cells, and multi\-hop evidence chains under more aggressive compression\(Chen et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib9); Haverbeck et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib21); Chen et al\.,[2026](https://arxiv.org/html/2607.06523#bib.bib10)\)\.

Finally, DepthWeave\-KV opens systems questions around serving integration\. The current fused kernel reduces memory traffic for a single compressed\-cache layout, but production systems must coordinate compression with batching, prefix reuse, disaggregated prefill/decode, and multi\-tenant cache pools\. Combining cross\-depth residual factorization with cache\-centric serving architectures and compression\-aware memory controllers could further improve throughput under mixed context lengths and workload\-dependent quality constraints\(Qin et al\.,[2024](https://arxiv.org/html/2607.06523#bib.bib46); Xie et al\.,[2025](https://arxiv.org/html/2607.06523#bib.bib62); Patel & Joshi,[2026](https://arxiv.org/html/2607.06523#bib.bib43)\)\.

## References

- A & S \(2026\)A, A\. and S, S\.Ai agents as universal task solvers\.*PMC*, 2026\.doi:10\.3390/e28030332\.
- A et al\. \(2026a\)A, A\., A, M\., and J, O\.Anatomic interactive atlas of the loggerhead sea turtle \(caretta caretta\) coelomic cavity\.*PMC*, 2026a\.doi:10\.3390/ani16050754\.
- A et al\. \(2026b\)A, Z\., Y, F\., and J, Z\.On demand secure scalable video streaming for both human and machine applications\.*PMC*, 2026b\.doi:10\.3390/s26041285\.
- Akulov et al\. \(2025\)Akulov, D\., Sana, M\., Domenico, A\. D\., Salem, T\. S\., Piovesan, N\., and Ayed, F\.Kvcompose: Efficient structured kv cache compression with composite tokens\.*Preprint*, 2025\.doi:10\.48550/arXiv\.2509\.05165\.
- An et al\. \(2026\)An, Y\., Lu, C\., Zhu, K\., Yu, T\., Zhao, C\., Wu, H\., Tang, M\., and Wang, J\.Rest\-kv: Robust kv cache eviction with layer\-wise output reconstruction and spatial\-temporal smoothing\.*arXiv preprint*, 2026\.
- Berner et al\. \(2021\)Berner, J\., Grohs, P\., Kutyniok, G\., and Petersen, P\.The modern mathematics of deep learning\.*arXiv preprint*, 2021\.doi:10\.1017/9781009025096\.002\.
- Bui et al\. \(2026\)Bui, N\., Nguyen, H\. T\., Cohan, A\., and Ying, R\.Make each token count: Towards improving long\-context performance with kv cache eviction\.*arXiv preprint*, 2026\.
- Cai et al\. \(2025\)Cai, Z\., Xiao, W\., Sun, H\., Luo, C\., Zhang, Y\., Wan, K\., Li, Y\., Zhou, Y\., Chang, L\.\-W\., Gu, J\., Dong, Z\., Anandkumar, A\., Asi, A\., and Hu, J\.R\-kv: Redundancy\-aware kv cache compression for reasoning models\.*arXiv preprint*, 2025\.
- Chen et al\. \(2025\)Chen, A\., Geh, R\., Grover, A\., den Broeck, G\. V\., and Israel, D\.The pitfalls of kv cache compression\.*arXiv preprint*, 2025\.
- Chen et al\. \(2026\)Chen, Z\., Liu, P\., and Shao, L\.Structkv: Preserving the structural skeleton for scalable long\-context inference\.*Preprint*, 2026\.doi:10\.48550/arXiv\.2604\.06746\.
- Corallo et al\. \(2025\)Corallo, G\., Faure\-Rolland, E\., Lamari, M\., and Papotti, P\.Tablekv: Kv cache compression for in\-context table processing\.*Journal*, 2025\.doi:10\.18653/v1/2025\.trl\-1\.13\.
- Cunegatti et al\. \(2026\)Cunegatti, E\., Vukojevic, M\., Nielsen, E\., and Iacca, G\.From layers to submodules: Rethinking granularity in replacement\-based llm compression\.*Preprint*, 2026\.
- da Silva et al\. \(2025\)da Silva, J\. G\. J\., de Oliveira, S\. S\. T\., Alves, L\. A\., Eiris, N\., and Filho, A\. R\. G\.Kv\-raptor: Scalable tree\-structured retrieval with kv cache compression for question\-answering systems\.*Journal*, 2025\.doi:10\.5753/sbbd\.2025\.247245\.
- Dong et al\. \(2024\)Dong, X\. L\., Fu, Y\., Diao, S\., Byeon, W\., Chen, Z\., Mahabaleshwarkar, A\. S\., Liu, S\.\-Y\., Keirsbilck, M\. V\., Chen, M\.\-H\., Suhara, Y\., Lin, Y\., Kautz, J\., and Molchanov, P\.Hymba: A hybrid\-head architecture for small language models\.*Journal*, 2024\.doi:10\.48550/arxiv\.2411\.13676\.
- Filippova et al\. \(2026\)Filippova, A\., Grangier, D\., Cuturi, M\., and Monteiro, J\.Stochastic kv routing: Enabling adaptive depth\-wise cache sharing\.*Preprint*, 2026\.doi:10\.48550/arXiv\.2604\.22782\.
- Gao \(1970\)Gao, W\.Deep learning\-based point cloud compression: An in\-depth survey and benchmark\_supp1\-3594355\.pdf\.*Journal*, 1970\.doi:10\.1109/tpami\.2025\.3594355/mm1\.
- Garg et al\. \(2026\)Garg, L\., Yaswanth, S\., Mishra, D\. N\., Kumaran, K\., Sharma, A\., and Uniyal, M\.Monodense deep neural model for determining item price elasticity\.*arXiv preprint*, 2026\.doi:10\.1109/AAIML67890\.2026\.11498150\.
- Gelberg et al\. \(2026\)Gelberg, Y\., Eitan, Y\., Bronstein, M\., Gal, Y\., and Maron, H\.Training transformers for kv cache compressibility\.*arXiv preprint*, 2026\.
- Gokhale et al\. \(2026\)Gokhale, S\., Das, D\., Patwari, R\., Sirasao, A\., and Delaye, E\.Kv pareto: Systems\-level optimization of kv cache and model compression for long context inference\.*Journal*, 2026\.doi:10\.18653/v1/2026\.eacl\-industry\.9\.
- Haffari et al\. \(2024\)Haffari, G\., He, Y\., Liu, A\., Liu, J\., Pan, Z\., and Zhuang, B\.Minicache: Kv cache compression in depth dimension for large language models\.*Journal*, 2024\.doi:10\.52202/079017\-4443\.
- Haverbeck et al\. \(2026\)Haverbeck, L\., Alonso, C\. A\., Posada\-Moreno, A\. F\., Trimpe, S\., and Pavone, M\.The risk of kv cache compression\.*arXiv preprint*, 2026\.
- J et al\. \(2026\)J, G\., X, J\., Z, S\., H, T\., X, Z\., C, X\., W, L\., Y, S\., S, M\., H, L\., S, W\., K, Z\., Z, L\., B, Z\., L, N\., W, G\., Y, W\., and J, G\.A survey on llm\-as\-a\-judge\.*PMC*, 2026\.doi:10\.1016/j\.xinn\.2025\.101253\.
- Javidnia et al\. \(2025\)Javidnia, N\., Rouhani, B\. D\., and Koushanfar, F\.Key, value, compress: A systematic exploration of kv cache compression techniques\.*Journal*, 2025\.doi:10\.1109/cicc63670\.2025\.10983416\.
- Jiang et al\. \(2025\)Jiang, J\., Yang, P\., Zhang, R\., and Liu, F\.Towards efficient large language model serving: A survey on system\-aware kv cache optimization\.*Journal*, 2025\.doi:10\.18653/v1/2026\.findings\-acl\.1916\.
- Jo et al\. \(2026\)Jo, D\., Song, J\., Kim, Y\., and Kim, J\.\-J\.Fastkv: Decoupling of context reduction and kv cache compression for prefill\-decoding acceleration\.*Journal*, 2026\.doi:10\.18653/v1/2026\.findings\-acl\.1610\.
- Kriuk & Ng \(2026\)Kriuk, B\. and Ng, L\.Q\-kvcomm: Efficient multi\-agent communication via adaptive kv cache compression\.*Journal*, 2026\.doi:10\.1109/icmsci67830\.2026\.11469367\.
- Kumar \(2024\)Kumar, A\.Residual vector quantization for kv cache compression in large language model\.*Preprint*, 2024\.doi:10\.48550/arXiv\.2410\.15704\.
- L et al\. \(2026\)L, F\., Q, S\., Q, X\., and X, L\.Development of a rapid deep\-learning\-assisted multiphase multicomponent numerical simulation protocol\.*PMC*, 2026\.doi:10\.1021/acsomega\.6c03179\.
- Li et al\. \(2024\)Li, H\., Li, Y\., Tian, A\., Tang, T\., Xu, Z\., Chen, X\., Hu, N\., Dong, W\., Li, Q\., and Chen, L\.A survey on large language model acceleration based on kv cache management\.*Journal*, 2024\.doi:10\.48550/arxiv\.2412\.19442\.
- Li et al\. \(2025\)Li, R\., Fu, Y\., Sheng, M\., Long, X\., Yu, H\., and Li, P\.Faedkv: Infinite\-window fourier transform for unbiased kv cache compression\.*Journal*, 2025\.doi:10\.18653/v1/2025\.findings\-emnlp\.914\.
- Liu et al\. \(2024a\)Liu, A\., Liu, J\., Pan, Z\., He, Y\., Haffari, G\., and Zhuang, B\.Minicache: Kv cache compression in depth dimension for large language models\.*arXiv preprint*, 2024a\.
- Liu et al\. \(2024b\)Liu, A\., Liu, J\., Pan, Z\., He, Y\., Haffari, G\., and Zhuang, B\.Minicache: Kv cache compression in depth dimension for large language models\.*Journal*, 2024b\.doi:10\.48550/arxiv\.2405\.14366\.
- Lye et al\. \(2019\)Lye, K\. O\., Mishra, S\., and Ray, D\.Deep learning observables in computational fluid dynamics\.*arXiv preprint*, 2019\.doi:10\.1016/j\.jcp\.2020\.109339\.
- M et al\. \(2026a\)M, G\., S, R\., S, R\., X, L\., and S, S\.The rise of small language models in healthcare: A comprehensive survey\.*PMC*, 2026a\.doi:10\.1016/j\.cosrev\.2026\.100999\.
- M et al\. \(2026b\)M, N\., L, D\., A, T\., and A, P\.Building large\-scale english\-romanian literary translation resources with open models\.*PMC*, 2026b\.doi:10\.3389/frai\.2026\.1807431\.
- Ma et al\. \(2026\)Ma, D\., Chen, L\., Zhang, S\., Miao, Y\., Zhu, S\., Chen, Z\., Xu, H\., Li, H\., Fan, S\., Pan, L\., and Yu, K\.Compressing kv cache for long\-context llm inference with inter\-layer attention similarity\.*Journal*, 2026\.doi:10\.1109/icassp55912\.2026\.11464826\.
- Marti & Maki \(2017\)Marti, M\. and Maki, A\.A multitask deep learning model for real\-time deployment in embedded systems\.*arXiv preprint*, 2017\.
- MF et al\. \(2026\)MF, Y\., CC, L\., and CL, L\.Multi\-hardware benchmarking of open\-source large language models with retrieval\-augmented generation for mitsubishi fx\-series plc instruction list code generation\.*PMC*, 2026\.doi:10\.3390/s26113602\.
- MO et al\. \(2026\)MO, A\., AJ, A\., MAF, A\.\-H\., and KM, A\.Limit\-cycle proliferation under parametric delayed feedback in a conductance\-based neuron: Bifurcation landscape, orbit catalog, and capacity analysis\.*PMC*, 2026\.doi:10\.3390/e28060678\.
- N et al\. \(2026\)N, P\., S, B\., F, R\., O, V\., A, N\., T, V\., NK, J\., S, T\., and A, A\.F\-transformer: a federated transformer for efficient and privacy\-preserving sequence generation\.*PMC*, 2026\.doi:10\.1038/s41598\-026\-40881\-0\.
- Nguyen\-Tri et al\. \(2025\)Nguyen\-Tri, Q\., Ranjan, M\., and Shen, Z\.Attention is all you need for kv cache in diffusion llms\.*arXiv preprint*, 2025\.
- Pandey & Yu \(2023\)Pandey, D\. and Yu, Q\.Learn to accumulate evidence from all training samples: Theory and practice\.*arXiv preprint*, 2023\.
- Patel & Joshi \(2026\)Patel, I\. and Joshi, I\.Polykv: A shared asymmetrically\-compressed kv cache pool for multi\-agent llm inference\.*arXiv preprint*, 2026\.doi:10\.5281/zenodo\.19686729\.
- Pustovit \(2026\)Pustovit, A\.Knowledge packs: Zero\-token knowledge delivery via kv cache injection\.*arXiv preprint*, 2026\.
- Qiao et al\. \(2025\)Qiao, A\., Yao, Z\., Rajbhandari, S\., and He, Y\.Swiftkv: Fast prefill\-optimized inference with knowledge\-preserving model transformation\.*Journal*, 2025\.doi:10\.18653/v1/2025\.emnlp\-main\.1306\.
- Qin et al\. \(2024\)Qin, R\., Li, Z\., He, W\., Zhang, M\., Wu, Y\., Zheng, W\., and Xu, X\.Mooncake: A kvcache\-centric disaggregated architecture for llm serving\.*Journal*, 2024\.doi:10\.48550/arxiv\.2407\.00079\.
- Qin et al\. \(2026\)Qin, Z\., Jiang, Y\., Lin, M\., Lv, Y\., Guo, H\., Fei, W\., and Lin, W\.Hack\+\+: Towards more effective head\-aware key\-value compression for efficient visual autoregressive modeling\.*Preprint*, 2026\.
- Ray et al\. \(2023\)Ray, D\., Pinti, O\., and Oberai, A\. A\.Deep learning and computational physics \(lecture notes\)\.*arXiv preprint*, 2023\.
- RJ & F \(2026\)RJ, C\. and F, R\.\-S\.Structural optimization principles for edge ai in motorsport telemetry\.*PMC*, 2026\.doi:10\.1038/s41598\-026\-49736\-0\.
- S et al\. \(2026\)S, P\., H, D\., and S, M\.Green prompt engineering for sustainable generative ai\.*PMC*, 2026\.doi:10\.1016/j\.ese\.2026\.100684\.
- Saxena et al\. \(2024\)Saxena, U\., Saha, G\., Choudhary, S\., and Roy, K\.Eigen attention: Attention in low\-rank space for kv cache compression\.*Journal*, 2024\.doi:10\.18653/v1/2024\.findings\-emnlp\.899\.
- Slothouber \(2026\)Slothouber, C\.The kv4 trade\-off is workload\-dependent: A depth\-and workload\-resolved study of 4\-bit kv\-cache quantization on a 4 gb turing gpu\.*Journal*, 2026\.doi:10\.2139/ssrn\.6941538\.
- Tang et al\. \(2025\)Tang, Z\., Luohe, S\., Li, Z\., Qi, B\., Guoming, L\., Zhang, L\., and Wang, P\.Spindlekv: A novel kv cache reduction method balancing both shallow and deep layers\.*Journal*, 2025\.doi:10\.18653/v1/2025\.acl\-long\.1380\.
- Wang \(2026\)Wang, X\.How much cache does reasoning need? depth\-cache tradeoffs in kv\-compressed transformers\.*Preprint*, 2026\.doi:10\.48550/arXiv\.2604\.17935\.
- Wang et al\. \(2024\)Wang, Z\., Jin, B\., Yu, Z\., and Zhang, M\.Model tells you where to merge: Adaptive kv cache merging for llms on long\-context tasks\.*Journal*, 2024\.doi:10\.48550/arxiv\.2407\.08454\.
- Wei et al\. \(2026\)Wei, id\_orcid 0009\-0003\-9914\-2657 Jiang, and Wang, W\.Sub\-token routing for kv cache compression\.*Journal*, 2026\.
- X et al\. \(2026a\)X, K\., T, Y\., L, W\., Y, G\., and F, Z\.Ss\-adamoe: Spatio\-spectral adaptive mixture of experts with global structural priors for graph node classification\.*PMC*, 2026a\.doi:10\.3390/e28030355\.
- X et al\. \(2026b\)X, S\., L, F\., S, H\., D, L\., Z, X\., and B, L\.Towards sustainable ai knowledge\-base assistants in computer science education: on\-premise deployment and optimization with open educational resources\.*PMC*, 2026b\.doi:10\.3389/fpsyg\.2026\.1843444\.
- X et al\. \(2026c\)X, Z\., M, H\., S, L\., S, K\., EY, L\., Y, L\., and WD, L\.Compute\-in\-memory implementation of state space models for event sequence processing\.*PMC*, 2026c\.doi:10\.1038/s41467\-025\-68227\-w\.
- Xiang et al\. \(2025\)Xiang, L\., Tang, Z\., Dong, P\., Li, Z\., Liu, Y\., Li, B\., Hu, X\., and Chu, X\.Chunkkv: Semantic\-preserving kv cache compression for efficient long\-context llm inference\.*Journal*, 2025\.doi:10\.48550/arxiv\.2502\.00299\.
- Xiao et al\. \(2025\)Xiao, E\., Li, C\. H\., Zhang, Y\., Neubig, G\., and Bertsch, A\.Efficient many\-shot in\-context learning with dynamic block\-sparse attention\.*Journal*, 2025\.doi:10\.18653/v1/2025\.acl\-long\.1542\.
- Xie et al\. \(2025\)Xie, R\., Haq, A\. U\., Ma, L\., Fang, Y\., Engineer, Z\. B\., Liu, L\., and Zhang, T\.Reimagining memory access for llm inference: Compression\-aware memory controller design\.*Preprint*, 2025\.doi:10\.48550/arXiv\.2503\.18869\.
- Y et al\. \(2026\)Y, G\., Y, D\., R, M\., Y, Z\., and J, G\.Infrared imaging for autonomous power inspection: A review from detector to system integration\.*PMC*, 2026\.doi:10\.3390/s26113552\.
- Yang et al\. \(2025\)Yang, D\., Yang, Y\., Yu, X\., Qi, X\., and Xiao, R\.Hcattention: Extreme kv cache compression via heterogeneous attention computing for llms\.*Journal*, 2025\.doi:10\.2139/ssrn\.5403127\.
- Yang et al\. \(2026\)Yang, D\., Yang, Y\., Yu, X\., Qi, X\., and Xiao, R\.Hcattention: Extreme kv cache compression via heterogeneous attention computing for llms\.*Journal*, 2026\.doi:10\.1016/j\.neucom\.2026\.134247\.
- Yang et al\. \(2024a\)Yang, Y\., Cao, Z\., Chen, Q\., Qin, L\., Yang, D\., Zhao, H\., and Chen, Z\.Kvsharer: Efficient inference via layer\-wise dissimilar kv cache sharing\.*Journal*, 2024a\.doi:10\.48550/arxiv\.2410\.18517\.
- Yang et al\. \(2024b\)Yang, Z\., Han, J\., Wu, K\., Xie, R\., Wang, A\., Sun, X\., and Kang, Z\.Lossless kv cache compression to 2%\.*Journal*, 2024b\.doi:10\.48550/arxiv\.2410\.15252\.
- Yang et al\. \(2024c\)Yang, Z\., Han, J\. N\., Wu, K\., Xie, R\., Wang, A\., Sun, X\., and Kang, Z\.Lossless kv cache compression to 2%\.*arXiv preprint*, 2024c\.
- Yao et al\. \(2025\)Yao, D\., Shen, B\., Lin, Z\., Liu, W\., Luan, J\., Wang, B\., and Wang, W\.Tailorkv: A hybrid framework for long\-context inference via tailored kv cache optimization\.*Journal*, 2025\.doi:10\.18653/v1/2025\.findings\-acl\.1043\.
- Yu & Chai \(2025a\)Yu, B\. and Chai, Y\.Evolkv: Evolutionary kv cache compression for llm inference\.*Journal*, 2025a\.doi:10\.18653/v1/2025\.findings\-emnlp\.88\.
- Yu & Chai \(2025b\)Yu, B\. and Chai, Y\.Evolkv: Evolutionary kv cache compression for llm inference\.*arXiv preprint*, 2025b\.
- Z et al\. \(2026\)Z, L\., X, M\., X, H\., Y, Z\., and W, Y\.Large\-scale model\-enhanced vision\-language navigation: Recent advances, practical applications, and future challenges\.*PMC*, 2026\.doi:10\.3390/s26072022\.
- Z et al\. \(2025\)Z, W\., R, Z\., DA, W\., D, E\., L, S\., O, B\., W, K\., E, Z\., CK, L\., WC, S\., A, P\., J, W\., GCL, W\., R, L\., S, L\., MZ, L\., M, K\., P, G\., TK, H\., and L, G\.Kilohertz volumetric imaging of in vivo dynamics using squeezed light field microscopy\.*PMC*, 2025\.doi:10\.1038/s41592\-025\-02843\-8\.
- Zhang et al\. \(2024\)Zhang, J\., Zhu, D\., Song, Y\., Wu, W\., Kuang, C\., Li, X\., Shang, L\., Liu, Q\., and Li, S\.More tokens, lower precision: Towards the optimal token\-precision trade\-off in kv cache compression\.*Preprint*, 2024\.doi:10\.48550/arXiv\.2412\.12706\.
- Zhang et al\. \(2025\)Zhang, T\., Li, Y\., Wang, J\., and Li, L\.Enhancing large multimodal models with adaptive sparsity and kv cache compression\.*Journal*, 2025\.doi:10\.1109/icme59968\.2025\.11209796\.