post-training-quantization

This research report evaluates post-training ternarization of the Qwen3-4B model, achieving a 1.641-bit effective weight representation with substantial storage compression, while noting a performance trade-off and unresolved deployment acceleration issues.

QTEA is a sub-2-bit post-training quantization framework for large language models that achieves ternary weight compression with improved accuracy and hardware efficiency through sparse residual salient weights and column-wise optimization.

The paper proposes REAL-Q, a novel end-to-end aligned post-training quantization method for large language models that uses dynamic gradient descent to mitigate quantization errors, achieving significant improvements in KL divergence reduction over state-of-the-art methods.

OCGQuant proposes an outlier-companion grouping method for NVFP4 quantization to reduce activation block errors, enhancing LLM inference efficiency. Experiments on Llama3 and Qwen3 demonstrate its superior performance over existing post-training quantization techniques.

HyGenQ is a post-training quantization framework for hybrid iterative generative models that addresses challenges like excessive outliers and amplified anomalies, enabling 8-bit precision quantization while maintaining generation quality.

ReRound is a post-training quantization method that uses a conditional diffusion model to guide rounding of near-midpoint weights, improving small LLM accuracy at 3-bit and 4-bit quantization without inference overhead.

ReQuant introduces a backpropagation-free, fixed-grid discrete refinement stage for post-training quantization (PTQ) that iteratively improves initial quantized models while preserving the quantized format, showing consistent gains across various LLMs and bit-widths.

Introduces Recurrent Residual Quantization (RRQ), a post-training quantization framework that enables multiple effective precisions from a single LLM checkpoint via additive residual refinement, improving flexibility and construction speed.

ExTernD introduces an expanded-rank ternary decomposition for post-training LLM quantization, enabling accuracy approaching bf16 by using a factored representation with free inner rank. It matches Q4_K accuracy at 5.2-5.5 effective bits per weight on models like Gemma-4 and Qwen3.5.

KronQ is a post-training quantization framework that incorporates gradient covariance using a Kronecker-factored Hessian approximation, enabling bidirectional incoherence processing and improved sensitivity metrics for mixed-precision allocation. It achieves low perplexity even at 2-bit weight-only quantization on large models like LLaMA-3-70B.

OrbitQuant introduces a data-agnostic quantization method for diffusion transformers that eliminates the need for recalibration across timesteps and modalities, achieving state-of-the-art post-training quantization at low-bit settings for models like FLUX.1 and CogVideoX.

CAT-Q introduces a post-training ternary quantization method for LLMs that uses learnable modulation and softened ternarization, achieving superior performance over BitNet 1.58-bit while using only 512 calibration samples and scaling to 235B parameters.

A novel end-to-end framework for LLM compression that jointly optimizes structural pruning and mixed-precision quantization, achieving significant perplexity reductions and speedups over state-of-the-art methods, especially at ultra-low bit precisions.

ScaleSweep proposes a new block scale initialization method for NVFP4 post-training quantization of LLMs, achieving improved accuracy by sweeping over feasible block scale candidates. Experiments on Llama and Qwen models show it preserves over 93% of full-precision performance under aggressive quantization.

This paper proposes FAIR-Calib, a two-stage post-training quantization framework for diffusion large language models that addresses the instability of token commitments during iterative refinement. It achieves state-of-the-art results on LLaDA and Dream models under low-bit quantization.

The paper proposes VSRAQ, a post-training quantization method for Mixture-of-Experts models that preserves expert-selection behavior by aligning routing-relevant logits and expert ordering, reducing quantization-induced degradation without inference overhead.

This paper introduces Qift, a fixed no-zero two-bit weight quantization level set designed for Hadamard-rotated LLMs, achieving improved W2A4/KV4 inference by leveraging the near-zero-centered Gaussian-like distribution of rotated weights. Experiments on LLaMA-2-7B and LLaMA-3.1-8B show consistent perplexity gains over standard W2 quantization.

Introduces QAM-W, a joint 2D codebook quantization method for LLM weights using Hadamard rotation and activation-aware scaling, achieving near BF16 perplexity at 5–6 bits per weight and matching SmoothQuant W8A8 quality with 32% fewer weight bits.

This paper presents Tail-Aware HiFloat4, a W4A4 post-training quantization method for the Wan2.2 text-to-video diffusion model, which uses activation-tail-aware percentile calibration to mitigate outlier effects while preserving HiFloat4 arithmetic.

InfoQuant introduces a train-free method, Peak Suppression Orthogonal Transformation (PSOT), to reshape activation distributions for low-bit LLM quantization, preserving 97% floating-point accuracy under W4A4KV4 and outperforming prior PTQ methods.