Tag
This paper introduces a diversity-aware, layer-wise scoring method for KV cache eviction in large language models, incorporating attention dispersion and redundancy to improve performance on LongBench datasets.
SemKV introduces a mixed-precision quantization framework for KV cache guided by an empirically measured quality cliff, achieving significant storage reduction without detectable quality loss in long-context LLM inference.
PuzzleKV is a training-free method for compressing key-value cache in large language models using page-wise low-rank decomposition, achieving over 96% performance with approximately 60% storage.