eviction

The paper introduces PAGE, a partition-aware gated KV-cache eviction method that uses a scalar metric to predict input classes and apply eviction only when safe, reducing accuracy degradation in large language models.

This paper introduces QEvict, a KV-cache management scheme for LLMs that uses recoverable quantized eviction to handle attention drift during long-context decoding, improving memory efficiency while preserving important historical context.

This paper proposes a KV cache eviction strategy that scores tokens by counter-causal surprise, removing past tokens that are well-predicted from future context. The method is training-free, in-distribution, and achieves competitive performance with a fast single-layer approximation.

This paper investigates the memory contract of sparse event-KV serving, showing that evicted source events can still influence answers via cached rows, and that deliberately phrased events can enable donor-aligned recovery without naming the value.

Introduces Nexus Sampling, a training-free KV-cache eviction method using weighted reservoir sampling instead of deterministic top-k, improving long-context LLM inference under fixed memory budgets, matching dense attention performance at 80% eviction.

VaSE is a training-free method for KV cache eviction that protects large-magnitude value states and introduces stochasticity to improve reasoning model accuracy under compression, outperforming existing methods.

Introduces triattention v3, a new attention mechanism that enables safe eviction without recall loss for long-context inference, demonstrated on a hybrid mamba+attention model up to 256k tokens.