Tag
SemKV introduces a mixed-precision quantization framework for KV cache guided by an empirically measured quality cliff, achieving significant storage reduction without detectable quality loss in long-context LLM inference.