Tag
The paper proposes Attention-Aware Transform Coding (AATC) for compressing KV caches in large language models, achieving near-lossless accuracy at around 5.8x compression by minimizing distortion through attention mechanisms.
This paper proposes NOVA-KV, a transform-coding approach to KV cache quantization that uses attention-preserving transforms to allocate bits where queries actually attend, improving long-context retrieval accuracy at low bit rates compared to prior methods.