SoloQ: Calibration-Free Quantization for Diffusion Language Models

Hugging Face Daily Papers Papers
quantization diffusion-language-models calibration-free nvfp4 inference-efficiency model-compression

Summary

SoloQ提出一种免校准的量化框架，将权重和激活映射到归一化旋转基底，实现数据无关的4比特量化，在扩散语言模型上保持精度并最多降低2.61倍峰值内存、加速2.24倍推理。

Diffusion large language models dLLMs) have emerged as a promising alternative to autoregressive language models through bidirectional diffusion-based token generation. However, their growing model sizes and high inference costs make efficient deployment challenging: full-sequence denoising repeatedly invokes compute-intensive forward passes, while block-diffusion models additionally introduce a memory-intensive KV-cache. Low-bit weight-activation quantization is therefore attractive, yet existing dLLM post-training quantization methods rely on calibration data despite activation distributions shifting across masking states and denoising steps. We present SoloQ, a calibration-free quantization framework that maps weights and activations into a normalized rotated basis with a predictable marginal distribution, enabling data-independent quantization. SoloQ combines a structured K-RPBH rotation with a lightweight rescaling correction for calibration-free quantization. Its predictable post-rotation distribution supports both distribution-matched codebooks and hardware-native NVFP4. For block-diffusion models, SoloQ further applies commit-time KV-cache quantization to compress persistent states without perturbing the actively denoised block. Across full-sequence dLLMs (LLaDA and Dream) and block-diffusion dLLMs(Fast-dLLM v2 and Nemotron-Labs-Diffusion), SoloQ retains accuracy under 4-bit quantization and outperforms calibration-based baselines on knowledge- and reasoning-intensive benchmarks. With NVFP4, SoloQ reduces peak memory by up to 2.61X and accelerates end-to-end inference by up to 2.24X.
Original Article
View Cached Full Text

Cached at: 10/12/26, 02:37 AM

Paper page - SoloQ: Calibration-Free Quantization for Diffusion Language Models

Source: https://huggingface.co/papers/2610.07121

Abstract

DiffusionlargelanguagemodelsdLLMs)haveemergedasapromisingalternativetoautoregressivelanguagemodelsthroughbidirectionaldiffusion-basedtokengeneration.However,theirgrowingmodelsizesandhighinferencecostsmakeefficientdeploymentchallenging:full-sequencedenoisingrepeatedlyinvokescompute-intensiveforwardpasses,whileblock-diffusionmodelsadditionallyintroduceamemory-intensiveKV-cache.Low-bitweight-activationquantizationisthereforeattractive,yetexistingdLLMpost-trainingquantizationmethodsrelyoncalibrationdatadespiteactivationdistributionsshiftingacrossmaskingstatesanddenoisingsteps.WepresentSoloQ,acalibration-freequantizationframeworkthatmapsweightsandactivationsintoanormalizedrotatedbasiswithapredictablemarginaldistribution,enablingdata-independentquantization.SoloQcombinesastructuredK-RPBHrotationwithalightweightrescalingcorrectionforcalibration-freequantization.Itspredictablepost-rotationdistributionsupportsbothdistribution-matchedcodebooksandhardware-nativeNVFP4.Forblock-diffusionmodels,SoloQfurtherappliescommit-timeKV-cachequantizationtocompresspersistentstateswithoutperturbingtheactivelydenoisedblock.Acrossfull-sequencedLLMs(LLaDAandDream)andblock-diffusiondLLMs(Fast-dLLMv2andNemotron-Labs-Diffusion),SoloQretainsaccuracyunder4-bitquantizationandoutperformscalibration-basedbaselinesonknowledge-andreasoning-intensivebenchmarks.WithNVFP4,SoloQreducespeakmemorybyupto2.61Xandacceleratesend-to-endinferencebyupto2.24X.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2610\.07121

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.07121 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.07121 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.07121 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

REAL-Q: E2E LLM Quantization via Dynamic Gradient Descent

arXiv cs.LG

The paper proposes REAL-Q, a novel end-to-end aligned post-training quantization method for large language models that uses dynamic gradient descent to mitigate quantization errors, achieving significant improvements in KL divergence reduction over state-of-the-art methods.

Structured Transforms for Low-Overhead Quantization of Language Models

arXiv cs.CL

This paper revisits Kashin-decomposition-based weight quantization for large language models and proposes an improved algorithm using structured orthogonal transforms, reducing computational cost and ensuring numerical stability compared to methods like OPTQ and QuIP.

Quantization of Linear-Attention (Qwen & Kimi)

Reddit r/LocalLLaMA

腾讯研究人员在 Hugging Face 上发布了 STEPQuant，一个针对 Delta-rule 循环状态的空间-时间后训练量化框架，用 6-bit 状态即可匹配 FP32 精度，并将服务内存最多削减 68.7%。