Tag
作者在 ARM (Neoverse-N1) 和 AVX2 x86 CPU 上对比了多个低比特（1-bit/三值/2-bit）GGUF 模型运行时的正确性与速度：bitnet.cpp main 在 CPU 上仍输出错误结果（需 #588/#642 补丁），Bonsai 1.7B 在所有能加载的运行时中均正确，而 ik_llama.cpp 配合 -rtr 通常最快。附带可复现脚本与 Kaggle/Colab 验证笔记本。
腾讯混元团队成员公开了 Hy4 Preview（770B 参数 MoE）的极致量化方案，通过 STQ1_0 编码与混合精度分配将约 1.5TB 权重压缩至 214 GiB，并在 llama.cpp 中实现接近 IQ1_M 的推理速度。
The paper introduces HeadGuard, a composable method that protects a small subset of KV heads in high precision while applying low-bit quantization to the rest of a VLM's image KV cache, substantially recovering accuracy lost by aggressive 2-bit quantization across multiple models and benchmarks.
This research report evaluates post-training ternarization of the Qwen3-4B model, achieving a 1.641-bit effective weight representation with substantial storage compression, while noting a performance trade-off and unresolved deployment acceleration issues.