cache-compaction

This empirical study examines practical online KV cache compaction for LLM agents, comparing token eviction and attention matching methods under different proxy query sources. It finds that delaying compaction to use future agent queries recovers performance, and token eviction preserves accuracy while reducing KV cache by 80%.