@anirudhbv_ce: Introducing SpectralQuant.. here to save your KV cache :)
Summary
SpectralQuant is a new KV cache quantization technique achieving 5.95× compression on Mistral 7B with only 7.5% perplexity overhead, significantly outperforming TurboQuant while requiring only 15 seconds of calibration per model.
View Cached Full Text
Cached at: 05/20/26, 04:26 AM
Introducing SpectralQuant.. here to save your KV cache :)
Ashwin Gopinath (@ashwingop): @sentra_app just killed @GoogleResearch’s TurboQuant.
SpectralQuant — 5.95× KV cache compression on Mistral 7B at +7.5% perplexity overhead. TurboQuant at the same compression: +22%.
3× less degradation. 15-second calibration. One per-model, then drop-in for any HuggingFace
Similar Articles
Here are my KV cache quantization benchmarks: TurboQuant is overrated but saved by TCQ, q5 deserves more attention, and symmetric q8 might be a waste of VRAM
A detailed benchmark comparing KV cache quantization methods (TurboQuant, TCQ, q4, q5, q8) using PPL and KLD metrics on Qwen 3.6 27B, finding that TCQ improves low-bit quantization, asymmetric KV beats symmetric at same size, and q8 is often overkill. Includes analysis and data in linked article.
Statistical Inference and Quality Measures of KV Cache Quantisations Inspired by TurboQuant
This paper analyzes KV cache quantization schemes inspired by TurboQuant, using statistical inference and a new 6D error framework to evaluate quality measures like KL divergence and geometric error.
Ablation, Statistical Inference, and Validation for KV-Cache Compression
This paper presents a systematic comparative study of KV-cache compression schemes (TurboQuant and SpectralQuant), introduces a statistical validation methodology, and offers regime-specific recommendations for efficient transformer inference.
KVarN: Native vLLM backend for KV-cache quantization by Huawei
Huawei CSL releases KVarN, a native vLLM attention backend for KV-cache quantization that delivers 3-5x more KV-cache capacity and up to ~1.3x the throughput of FP16, with no calibration required. It claims up to ~2.4x the throughput of TurboQuant while maintaining FP16-level accuracy on models like Qwen3-32B.
RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
This paper introduces RateQuant, a method for optimal mixed-precision KV cache quantization that uses rate-distortion theory to address distortion model mismatch. It significantly reduces perplexity compared to existing methods like KIVI and QuaRot with minimal calibration overhead.