KV Cache Compression Through the Lens of Transform Coding
Summary
The paper proposes Attention-Aware Transform Coding (AATC) for compressing KV caches in large language models, achieving near-lossless accuracy at around 5.8x compression by minimizing distortion through attention mechanisms.
View Cached Full Text
Cached at: 08/17/26, 10:22 AM
# KV Cache Compression Through the Lens of Transform Coding Source: [https://arxiv.org/abs/2608.14191](https://arxiv.org/abs/2608.14191) [View PDF](https://arxiv.org/pdf/2608.14191) > Abstract:The key\-value \(KV\) cache stores information from past tokens and is a major memory bottleneck in long\-context inference\. Existing quantization methods address this bottleneck by representing the KV cache uniformly with lower\-precision data types and designing quantization schemes to minimize reconstruction error in the cache itself, without accounting for how that error propagates through attention mechanisms\. We prove that, under a white\-noise quantization model, the expected attention\-aware distortion decomposes into additive key and value contributions that factor across tokens and channels\. Building on transform coding and reverse water\-filling, which are classical tools from signal processing and rate\-distortion theory, we introduce Attention\-Aware Transform Coding \(AATC\), which allocates bits over a calibration set to minimize attention\-aware distortion\. On Llama\-3\.1\-8B\-Instruct and Qwen\-2\.5\-7B\-Instruct, evaluated across LongBench, RULER, GSM8K, MMLU\-Pro, and MATH\-500, our method achieves near\-lossless accuracy at approximately $5\.8\\times$ compression, whereas each baseline degrades in at least some settings\. ## Submission history From: Hannah Sophie Laus \[[view email](https://arxiv.org/show-email/57f59d7d/2608.14191)\] **\[v1\]**Fri, 14 Aug 2026 11:08:01 UTC \(129 KB\)
Similar Articles
KV Cache Compression 900000x Beyond TurboQuant and Per-Vector Shannon Limit
A new paper proposes sequential KV cache compression using probabilistic language tries and predictive delta coding, achieving theoretical compression ratios of ~914,000× beyond TurboQuant by exploiting the sequential structure of language model tokens rather than treating vectors independently.
Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms
This paper proposes NOVA-KV, a transform-coding approach to KV cache quantization that uses attention-preserving transforms to allocate bits where queries actually attend, improving long-context retrieval accuracy at low bit rates compared to prior methods.
Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers
Introduces Looped Latent Attention (LLA), a post-training codec that compresses KV cache in looped transformers by exploiting low-rank structure across recurrence steps, achieving significant compression ratios while maintaining performance.
The risk of KV cache compression
This paper theoretically characterizes the minimax risk of KV cache compression in transformers, providing design principles for accurate compression under causal masking, and instantiates them in a practical algorithm with promising results on LongBench.
Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches
Codec-Gauge learns small orthogonal channel transforms (gauges) for Transformer KV caches to improve compression fidelity at fixed bit rates, achieving significant KL divergence reduction across multiple models and backends.