REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression

arXiv cs.CL Papers
kv-cache-compression attention-behavior long-context retrieval-reasoning noise-reduction eviction-method

Summary

REAL introduces an Attention Behavior Matrix to analyze attention head behaviors in both success and failure cases, enabling more effective KV cache eviction. It achieves comparable accuracy to the strongest baseline while requiring 32x less space.

arXiv:2508.15806v2 Announce Type: replace Abstract: The growing sequence length of large language models poses significant challenges for key-value (KV) caches. Existing state-of-the-art cache eviction methods primarily analyze the inference behavior of attention heads in successful retrieval-reasoning cases, often overlooking diverse behaviors in failure cases, such as bias and distraction. This oversight limits the potential to leverage heterogeneous head behaviors for improved eviction performance. Inspired by the confusion matrix, we introduce an Attention Behavior Matrix to comprehensively analyze attention head behaviors in both success and failure scenarios. By maximizing the signal-to-noise ratio -- strengthening valid reasoning pathways in success cases while inhibiting noise from bias and distraction in failure cases -- we propose REtrieval-reAsoning and Logic-constructed (REAL) KV cache eviction, the first method to leverage multi-behavior analysis. Comprehensive evaluations show that REAL achieves remarkable performance across various models and benchmarks; notably, on LongBench v2, it achieves comparable accuracy to the strongest baseline, HeadKV-R2, while requiring 32x less space (Figure 1). By offering a novel perspective on behavior analysis, we pave the way for a shift from success-only to comprehensive, failure-aware methods in long-context modeling. Our code is available at https://github.com/yonseicasl/REAL.
Original Article
View Cached Full Text

Cached at: 07/13/26, 08:00 AM 

# REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression
Source: [https://arxiv.org/abs/2508.15806](https://arxiv.org/abs/2508.15806)
[View PDF](https://arxiv.org/pdf/2508.15806)

> Abstract:The growing sequence length of large language models poses significant challenges for key\-value \(KV\) caches\. Existing state\-of\-the\-art cache eviction methods primarily analyze the inference behavior of attention heads in successful retrieval\-reasoning cases, often overlooking diverse behaviors in failure cases, such as bias and distraction\. This oversight limits the potential to leverage heterogeneous head behaviors for improved eviction performance\. Inspired by the confusion matrix, we introduce an Attention Behavior Matrix to comprehensively analyze attention head behaviors in both success and failure scenarios\. By maximizing the signal\-to\-noise ratio \-\- strengthening valid reasoning pathways in success cases while inhibiting noise from bias and distraction in failure cases \-\- we propose REtrieval\-reAsoning and Logic\-constructed \(REAL\) KV cache eviction, the first method to leverage multi\-behavior analysis\. Comprehensive evaluations show that REAL achieves remarkable performance across various models and benchmarks; notably, on LongBench v2, it achieves comparable accuracy to the strongest baseline, HeadKV\-R2, while requiring 32x less space \(Figure 1\)\. By offering a novel perspective on behavior analysis, we pave the way for a shift from success\-only to comprehensive, failure\-aware methods in long\-context modeling\. Our code is available at[this https URL](https://github.com/yonseicasl/REAL)\.

## Submission history

From: Mengjie Li \[[view email](https://arxiv.org/show-email/5e3e306f/2508.15806)\] **[\[v1\]](https://arxiv.org/abs/2508.15806v1)**Thu, 14 Aug 2025 14:08:58 UTC \(992 KB\) **\[v2\]**Fri, 10 Jul 2026 13:38:08 UTC \(1,079 KB\)

Similar Articles

Information-Aware KV Cache Compression for Long Reasoning

arXiv cs.CL

This paper proposes InfoKV, an entropy-aware KV cache compression framework that combines token-level predictive uncertainty with attention scores to improve long-context reasoning efficiency. Experiments show it outperforms existing attention-based methods on Llama-3.1, Llama-3.2, and DeepSeek-R1.

Thought-Aware KV Cache Compaction for Reasoning via Adaptive Attention Matching

arXiv cs.CL

This paper introduces Thought-Aware Attention Matching (TAM), a method for compressing KV caches during long chain-of-thought reasoning by segmenting reasoning traces, adaptively allocating compression budgets, and protecting pivotal tokens. Experiments on AIME 2024 and MATH-500 with Qwen3-4B show significant memory reduction while maintaining accuracy.