Tag
KVBoost is a chunk-level key-value cache reuse system for efficient large language model inference that achieves high cache hit rates and significant speedup in time-to-first-token without quality loss, using dual-hash keying and deviation-guided recomputation.
Introduces MaSRead, a content-addressed reading mechanism for replicated latent stores where agents share KV cache fragments, enabling later queries to reliably retrieve cached reasoning via opaque keyed tag sets and hard attention masks.
This paper introduces WorldTrace, a training-free memory framework for long-horizon video world models that keeps compressed cache addressable, plus LoopBench, a benchmark for episodic recall after long detours. It improves temporal consistency by +15.5% and episodic recall by +19.5% on LoopBench.
KVpop introduces a learned KV cache eviction policy supervised by future-attention targets, achieving high compression rates (e.g., 98% performance at 75% compression) on Qwen3 models while maintaining quality.
This paper proposes InfoKV, an entropy-aware KV cache compression framework that combines token-level predictive uncertainty with attention scores to improve long-context reasoning efficiency. Experiments show it outperforms existing attention-based methods on Llama-3.1, Llama-3.2, and DeepSeek-R1.
The author expresses surprise at how effective key-value cache quantization (q4_0) remains even with large context windows, citing accurate retrieval from a 100k context.
Introduces KV-Compression Aware Training (KV-CAT), a method that encourages transformers to learn compressible key-value caches during training, improving memory efficiency for long-context tasks without sacrificing performance.
Introduces Self-Pruned Key-Value Attention (SP-KV), a mechanism that learns to predict future utility of key-value pairs to dynamically prune the KV cache, reducing memory usage and decoding speed by 3-10x with minimal performance degradation. The model and utility predictor are trained end-to-end using next-token prediction.