Tag
AbridgeKV trains models to compress completed blocks of generated chain-of-thought text into the KV entries of a few learned pause tokens, achieving a 3.5× smaller cache and 3.2× faster decoding on a saturated H100 with no accuracy loss at a quarter of the full KV budget.
Telperion 发布了 Qwen3.8-27B 的潜在注意力（MLA 与 GLA-g2）变体，将 KV cache 压缩 4 倍，在消费级硬件上实现接近 4 倍的最大上下文或服务器端 4 倍并发，并在 SWE-bench Verified 上取得 +6.58% 的性能提升，同时减少约 20% 的推理 token 消耗。
论文提出 Latent Compaction，通过测试时训练将 KV 缓存压缩为紧凑的文档特定表示，在冻结模型权重的前提下实现 32 倍压缩下 96.7% 的 RULER 准确率，显著优于 KVZip+ 等基线，并可跨模型规模与系列迁移。
提出 Kronecker Product Attention (KPA)，用 Kronecker 积紧凑矩阵因子表示键值激活，实现最高 10× 的 KV 缓存压缩，同时在多个模型规模和基准上匹配或超越 TPA、MQA、GQA 和 MLA 等方法。
LlamAmpere 新版本发布，面向 12GB Ampere 显卡优化，可在 RTX 3060/3080/3090 上运行 200K+ 上下文的 Qwen3.8 27B 量化模型，并引入 Staged + Journaled KVaRN 变体将 KV 缓存压缩的 KLD 降低约 40%，性能达到 BF16 的约 85%。
作者介绍 ALHR（Adaptive Learnable Hierarchical Routing），一种基于静态二叉树与可学习函数的稀疏注意力系统，在 MQAR 测试中每查询仅读取约 30 个键（相比稠密注意力的 512 个），实现约 35.3 倍 KV 压缩，同时保持接近的准确率，并将推理复杂度降至 NlogN、显存线性扩展。
本文档研究了基于SVD的KV缓存压缩中"仅编码器修复"的参数高效微调方法，发现当为每个实验组单独调整学习率后，仅训练编码器（冻结解码器）以3倍更少的可训练参数和3倍更少的优化器内存即可达到与全量微调相同的精度，同时揭示了共享学习率在比较不同参数量微调方案时的常见实验缺陷。
DeepSeek-V4.1-Flash is a multimodal Mixture-of-Experts model with 552B backbone parameters and 1M-token context that achieves major KV cache compression gains via CSA2 and FP4 caching, cutting HBM footprint to ~890 bytes per token while boosting agentic performance.
This article argues the AI race narrative has shifted: Western labs now quietly adopt open Chinese advances like DeepSeek's KV cache compression (437x smaller than V1), which has driven inference and cached-token pricing at Anthropic and OpenAI sharply down.
This paper identifies 'phase sensitivity' in models using chunked KV-cache compression, showing that retrieval accuracy oscillates in cycles matching the compression stride and can flip answers or shift long-context accuracy by up to 40 points at 128K tokens, an effect confirmed via controlled 0.6B pretraining experiments to stem from the compression itself.
StepKV is a step-aware KV cache compression method for LLM agents that retains reasoning steps to maintain accuracy under low memory budgets, addressing reasoning continuity disruption in multi-step inference.
A tweet sharing three interesting papers from Hugging Face Daily Paper on advancements in KV cache compression for local LLMs and agent harness optimization.
Introduces DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts model with 552B parameters, featuring advanced KV cache compression techniques to reduce deployment costs and improve efficiency for long-context agent workloads.
SGD-KV is a framework that uses summarization to guide KV cache compression in large language models, reducing memory usage by up to 75% for contexts up to 1M tokens while achieving state-of-the-art performance on long-context benchmarks.
HeadWiseKV is a training-free framework that compresses KV caches in hybrid long-context language models, reducing GPU memory usage and extending context lengths while maintaining quality.
SemKV introduces a mixed-precision quantization framework for KV cache guided by an empirically measured quality cliff, achieving significant storage reduction without detectable quality loss in long-context LLM inference.
HyQuant improves low-bit LLM attention quantization by preserving critical vertical-line tokens and local windows in high precision while quantizing the rest, maintaining accuracy with low overhead.
PuzzleKV is a training-free method for compressing key-value cache in large language models using page-wise low-rank decomposition, achieving over 96% performance with approximately 60% storage.
This paper compares tensor parallelism and KV-cache compression techniques for memory-bound LLM serving, finding that compression is generally cheaper and discusses decision rules based on model size relative to device memory.
Minima-KV presents a retention-preserving KV cache compression method using mixed-format paged attention to reduce memory footprint in long-context LLM serving, with evaluated performance on benchmarks.