Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Summary
Grouped Value Attention reduces transformer KV cache size by storing grouped values and reconstructing keys via a learned linear map, achieving near-GQA accuracy with a smaller persistent cache.
View Cached Full Text
Cached at: 09/15/26, 10:41 AM
Paper page - Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Source: https://huggingface.co/papers/2609.13285
Abstract
Grouped Value Attention reduces transformer KV cache size by storing grouped values and reconstructing keys via a learned linear map, achieving near-GQA accuracy with a smaller persistent cache.
TheKV cacheis a primary bottleneck for Transformer decoding: its memory footprint and cache-read traffic grow with sequence length.Grouped-query attention(GQA) reduces this cost by sharing key-value heads, but still stores both a key and a value at every step. We introduceGrouped Value Attention(GVA), which stores grouped values and reconstructscontent keyswith a learned linear map. At inference, the map can be absorbed into the query, eliminating the need to materializecontent keysin the intended decode path. A small shareddecoupled RoPEchannel retains positional information through a separately cachedpositional key. For the configurations studied, this representation reduces persistent cache scalars by approximately 45-47% relative to matched GQA. At the 350M-parameter scale with 30B FineWeb-Edu tokens, the 16-dimensional positional variant reaches 44.18 average accuracy across five tasks, compared with 44.36 for GQA and 43.88 for MLA. These results demonstrate near-GQA benchmark accuracy with a more compact cache representation. To translate this compact representation into fasterautoregressive inference, we have developed customdecoding kernelsand are currently evaluating their end-to-end inference performance with an open-source release planned soon.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2609\.13285
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.13285 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.13285 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.13285 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
@che_shr_cat: 1/ We have spent years optimizing KV cache via head-sharing (GQA/MQA), but we ignored a fundamental assumption: why do …
This thread challenges the fundamental assumption that Transformers require separate Q, K, and V projections, proposing that merging them can yield massive memory savings for KV cache.
ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression
ResKV proposes a KV cache compression method that divides a fixed budget into an exact main cache and a compact residual cache to reconstruct omitted attention contributions, improving performance on LongBench and RULER across multiple backbones.
KV Cache Compression Through the Lens of Transform Coding
The paper proposes Attention-Aware Transform Coding (AATC) for compressing KV caches in large language models, achieving near-lossless accuracy at around 5.8x compression by minimizing distortion through attention mechanisms.
Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms
This paper proposes NOVA-KV, a transform-coding approach to KV cache quantization that uses attention-preserving transforms to allocate bits where queries actually attend, improving long-context retrieval accuracy at low bit rates compared to prior methods.
CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference
CompressKV proposes a semantic-retrieval-guided KV-cache compression method for GQA-based LLMs, identifying Semantic Retrieval Heads to retain critical tokens. It achieves over 97% full-cache performance using only 3% of the KV cache on LongBench tasks.