post-training-quantization

提出 Message-Passing Quantization (MPQ)，将逐行 PTQ 视为离散码本上的后验推断，通过在块间保留软后验信息来改进 GPTQ 等硬决策求解器，在 Llama、Qwen、Mistral 模型上提升了 2/3/4-bit 量化的困惑度和零样本精度。

This paper studies learned post-training quantization calibration for 4-bit text embedders (e5-base-v2, BGE-M3, EmbeddingGemma-300m), finding that calibration outcome ranges from gains to catastrophic collapse depending on the 'dose' of update, and proposes a validation-gated abstention rule that never selects harmful checkpoints while retaining most beneficial ones.

论文指出权重量化中降低重建损失并不总能提升模型性能，甚至可能因输入激活分布偏移而劣化，并提出分布鲁棒量化（DRQ）后处理优化框架，通过在约束激活分布集上最小化最坏情况重建损失来改进 AWQ、GPTQ、ParoQuant 等六种 PTQ 方法的下游表现，且不增加推理开销。

腾讯研究人员在 Hugging Face 上发布了 STEPQuant，一个针对 Delta-rule 循环状态的空间-时间后训练量化框架，用 6-bit 状态即可匹配 FP32 精度，并将服务内存最多削减 68.7%。

A study investigating post-training quantization (PTQ) on pre-trained AI weather emulators (DLWP and FourCastNet) to accelerate autoregressive inference and enable deployment on edge hardware, finding that quantized configurations retain qualitatively meaningful short-range forecasts.

JARQ proposes a plug-in joint refinement for group-wise post-training quantizers that alternates joint least-squares scale fitting with bounded Babai proposals to move many codes together. It lowers perplexity in 90 of 96 comparisons across Llama-2/3 and Qwen models with RTN, GPTQ, OmniQuant, and AWQ hosts, cutting three-bit RTN perplexity by up to 36% at under a minute per 7B block.

ShamAN-Q 提出一种亚1比特后训练量化方法，通过用 Shampoo 优化器启发的稠密曲率度量（Kronecker 拟合经验 Fisher 信息矩阵）替代 NanoQuant 的对角重构几何，在 1 bpw 下显著降低 Qwen3-Base 的 WikiText-2 围惑度并保持零样本准确率。

ThinQuant proposes a data-efficient gradient-free rotation learning method for weight and activation quantization of LLMs, using convex-hull-based calibration data selection and an ADMM-based Stiefel manifold optimizer that scales to Llama-3.1-405B on a single H200 GPU with improved perplexity and drastically reduced calibration time.

This paper presents G^2PTQ, a unified post-training quantization framework for large language models that uses generalized gradient compensation to improve alignment with full-precision models, outperforming state-of-the-art baselines.

This paper presents a decomposition framework for quantization error in large language models, separating it into activation-guided weight compensation and orthogonal residual, and derives practical guidelines for improving W4A4 quantization through techniques like Hadamard rotation and sign selection.

This paper revisits Kashin-decomposition-based weight quantization for large language models and proposes an improved algorithm using structured orthogonal transforms, reducing computational cost and ensuring numerical stability compared to methods like OPTQ and QuIP.

This research report evaluates post-training ternarization of the Qwen3-4B model, achieving a 1.641-bit effective weight representation with substantial storage compression, while noting a performance trade-off and unresolved deployment acceleration issues.

QTEA is a sub-2-bit post-training quantization framework for large language models that achieves ternary weight compression with improved accuracy and hardware efficiency through sparse residual salient weights and column-wise optimization.

The paper proposes REAL-Q, a novel end-to-end aligned post-training quantization method for large language models that uses dynamic gradient descent to mitigate quantization errors, achieving significant improvements in KL divergence reduction over state-of-the-art methods.

OCGQuant proposes an outlier-companion grouping method for NVFP4 quantization to reduce activation block errors, enhancing LLM inference efficiency. Experiments on Llama3 and Qwen3 demonstrate its superior performance over existing post-training quantization techniques.

HyGenQ is a post-training quantization framework for hybrid iterative generative models that addresses challenges like excessive outliers and amplified anomalies, enabling 8-bit precision quantization while maintaining generation quality.

ReRound is a post-training quantization method that uses a conditional diffusion model to guide rounding of near-midpoint weights, improving small LLM accuracy at 3-bit and 4-bit quantization without inference overhead.

ReQuant introduces a backpropagation-free, fixed-grid discrete refinement stage for post-training quantization (PTQ) that iteratively improves initial quantized models while preserving the quantized format, showing consistent gains across various LLMs and bit-widths.

Introduces Recurrent Residual Quantization (RRQ), a post-training quantization framework that enables multiple effective precisions from a single LLM checkpoint via additive residual refinement, improving flexibility and construction speed.

ExTernD introduces an expanded-rank ternary decomposition for post-training LLM quantization, enabling accuracy approaching bf16 by using a factored representation with free inner rank. It matches Q4_K accuracy at 5.2-5.5 effective bits per weight on models like Gemma-4 and Qwen3.5.