DKV: Open-source KV-cache compression framework for local LLM inference (CLI + technical report)
Summary
DKV is an open-source framework for compressing KV-cache during local LLM inference, providing a CLI and a technical report.
Similar Articles
CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference
CompressKV proposes a semantic-retrieval-guided KV-cache compression method for GQA-based LLMs, identifying Semantic Retrieval Heads to retain critical tokens. It achieves over 97% full-cache performance using only 3% of the KV cache on LongBench tasks.
MosaicKV: Serving Long-Context LLM with Dynamic Two-D KV Cache Compression
MosaicKV introduces dynamic two-dimensional KV cache compression for long-context LLM serving, achieving up to 16x attention speedup and 3x memory reduction with minimal accuracy loss.
PolyKV: Heterogeneous Retention and Allocation for KV Cache Compression
PolyKV is a layer-wise KV cache compression framework that assigns heterogeneous eviction policies and non-uniform budgets per layer, significantly improving over uniform baselines on LongBench with LLaMA-3.1-8B and Qwen3-8B.
TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inference
TTKV introduces a temporal-tiered KV cache that mimics human memory to cut 128K-context LLM inference latency by 76% and double throughput while reducing cross-tier traffic 5.94×.
HARD-KV: Head-Adaptive Regularization for Decoding-time KV Compression
Hard-KV introduces a Cascade Cache hierarchy and Logits Calibration mechanism to resolve the static-dynamic mismatch in head-adaptive KV cache compression, achieving up to 2x throughput improvement in long-context LLM inference.