Quantization of Linear-Attention (Qwen & Kimi)
Summary
腾讯研究人员在 Hugging Face 上发布了 STEPQuant，一个针对 Delta-rule 循环状态的空间-时间后训练量化框架，用 6-bit 状态即可匹配 FP32 精度，并将服务内存最多削减 68.7%。
View Cached Full Text
Cached at: 10/09/26, 06:31 AM
Tencent researchers just released STEPQuant on Hugging Face
A spatial-temporal post-training quantization framework for Delta-rule recurrent states, matching FP32 accuracy with 6-bit states while cutting total serving memory by up to 68.7% https://t.co/ilVY2w5tVm
Similar Articles
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
The paper proposes STEPQuant, a spatial-temporal post-training quantization framework for delta-rule recurrent states in linear attention models, achieving near-FP32 accuracy at 6-bit precision, over 5x state compression, and up to 68.7% serving memory reduction when integrated into SGLang.
QuantWM: Temporally Consistent 2-Bit KV Cache Quantization for Video World Models
QuantWM 提出一种免训练的 2-bit KV cache 量化框架，通过量化敏感性感知聚类（QSAC）与主子空间注意力补偿（PSAC）来稳定视频世界模型的注意力 logits，缓解时间闪烁与视觉退化，并在多个基准上实现最高 6.20 倍 KV cache 内存压缩。
SoloQ: Calibration-Free Quantization for Diffusion Language Models
SoloQ提出一种免校准的量化框架，将权重和激活映射到归一化旋转基底，实现数据无关的4比特量化，在扩散语言模型上保持精度并最多降低2.61倍峰值内存、加速2.24倍推理。
Right On Q: Recovering Reasoning under Quantization via Self-Generated Trajectories
论文提出 OnQ 流水线，通过 on-policy 蒸馏与候选级 Dynamic KL 恢复低比特量化 LLM 的推理能力，在 Qwen3 系列 2/3 比特量化上取得最优九基准平均成绩，2 比特下 Qwen3-8B 平均提升 10.26 分。
My Qwen3.8-27B task-aware quant reaches 99% of BF16 reasoning performance at 15% of the size.
A new task-aware quantization method called TAK achieves near-BF16 reasoning performance at significantly reduced model size, outperforming Unsloth on various AI models.