Tag
ThinkBit 是一个严格标量的两比特量化框架，通过分层比特分配、Gumbel-softmax 搜索码字、知识蒸馏重校准和量化感知训练来保留推理能力，在 Qwen3-1.7B 和 14B 模型上以远小于 bfloat16 的体积取得接近原模型的 MATH-500 成绩。
QUASAR 释放了针对 Qwen3.5-4B、Gemma 4 E4B/12B 和 Muse Glimmer 30B 的 4-bit QAT 检查点（NVFP4 与 Q4_0 GGUF 格式），通过量化感知蒸馏在 KL 散度和下游基准上超越 Google 与 NVIDIA 的官方 4-bit 版本。代码、论文与项目页现已公开。
Q-PACE 提出一种基于二阶敏感度模型的动态精度分配方法，用于量化感知训练（QAT），在预训练和监督微调实验中（模型规模达4B参数）相比现有混合精度训练方案取得了更优效果，并在更低内存预算下达到可比拟的损失。研究还发现量化敏感度可通过深度和层类型高度预测，从而支持低成本的低频重校准。
OnlineQAT 采用两阶段框架：先通过块级 QAT 得到低位初始化，再在学生自生成的轨迹上用冻结的全精度教师进行反向 KL 的在线蒸馏，从而缓解固定补全训练与部署时学生自采样前缀之间的分布错配。在 Qwen3-1.7B 上，OnlineQAT 在 W3A16（57.28）和 W2A16（32.52）设置中取得对比方法中的最佳平均分，较 ReasoningQAT 分别提升 2.90 和 0.44 分。
The post questions why quantization-aware training is not a standard practice for open-weight AI models and asks about potential barriers like compute costs or performance impacts.
This research paper analyzes capability-stratified degradation in ternary quantized language models, showing that while factual knowledge deteriorates significantly, commonsense reasoning and downstream task adaptability are retained, making the models viable for efficient edge deployment.
QUASAR is a quantization-aware training method that uses loss-aware reconstruction to lower the loss floor, improving low-bit model performance in large language models with significant accuracy gains at 2-4 bits.
Commentary on Google's Gemma 4 technical report, noting aggressive optimization via Quantization-Aware Training and Multi-Token Prediction for faster inference.
Introduces Variable Bit-width Quantization (VBQ), a training-time method where each group of 64 weights learns its own bit-width (1,2,4,8) via Gumbel-Softmax relaxation. VBQ discovers a heterogeneous allocation that yields a 'bigger-but-smaller' regime, e.g., a 131M parameter model at 1.82 mean bits beats a 55M FP16 model while using less storage, and a 1.46B model matches a 593M FP16 with ~3.7x less storage.
Release of Gemma4-12B-QAT Uncensored Balanced, a fine-tuned uncensored model with a multi-token-prediction draft head for ~60% faster speculative decoding, optimized for llama.cpp and offering vision support.
Google released Gemma 4 models with quantization-aware training (QAT) at Q4_0 precision on Hugging Face, offering efficient variants from 5B to 33B parameters.
New QAT Gemma 4 checkpoints offer similar performance with ~4x less memory, enabling a 1GB memory footprint for Gemma 4 E2B via a new mobile quantization format.
Google releases Gemma 4 models optimized with Quantization-Aware Training (QAT) to improve efficiency for mobile and laptop deployment, reducing memory footprint to 1GB for the E2B model while preserving quality.
Google DeepMind releases Gemma 4 models optimized with Quantization-Aware Training (QAT) in multiple formats including GGUF, enabling high quality with reduced memory requirements.
This paper systematically studies HiF8 W8A8 quantization-aware training for OpenPangu-Embedded-1B, identifying and addressing failure modes such as amax saturation and catastrophic forgetting, achieving near-lossless performance with a 64-step max-algorithm DTS strategy and a 500-step BF16 warmup.