Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor

# Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor
Source: [https://arxiv.org/html/2605.20402](https://arxiv.org/html/2605.20402)
Shi\-Liang \(Bruce\) Wu Xiao\-Can \(Bruce\) Li11footnotemark:1Zheng Shen Huawei Canada

###### Abstract

MXFP4 arithmetic can dramatically accelerate reinforcement learning \(RL\) post\-training of large language models \(LLMs\), yet the quantization error introduces severe accuracy degradation\. Existing work treats the quantization error as a monolithic noise term, missing the distinct mechanisms upon interpreting how quantization error damages training\. We prove an exact three\-way decomposition of quantization error and show how each component dominates a distinct RL training pathway\. Our theoretical and empirical analysis decomposes the MXFP4 quantization error into three additive components:*scale bias*from power\-of\-two rounding,*deadzone truncation*from zeroing small values, and*grid noise*from rounding to the nearest 4\-bit grid\. Each component dominates a distinct RL failure mode: scale bias accumulates multiplicatively through the backward pass, affecting gradient accuracy; deadzone truncation degrades rollout quality; and grid noise raises the policy’s entropy\. We combine corrections that are RL failure mode\-targeted but not component\-exclusive: Macro\-block scaling to reduce scale bias, Outlier Fallback recovers deadzone entries, but also partially reduces scale bias induced error, and Adaptive Quantization Noise \(AQN\) for controlling the policy entropy\. On Qwen2\.5\-3B dense and Qwen3\-30B\-A3B\-Base mixture\-of\-experts model, the targeted corrections recover BF16 accuracy to within 0\.7 pp and 3\.0 pp respectively\.

## 1Introduction

RL post\-training turns base LLMs into capable reasoners\(Shao et al\.,[2024](https://arxiv.org/html/2605.20402#bib.bib34);Yu et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib42)\), but the compute cost is steep: rollout generation alone can dominate the training budget\. The MXFP4 format\(Rouhani et al\.,[2023](https://arxiv.org/html/2605.20402#bib.bib33)\)has 4\-bit elements with a shared E8M0 scale per block of 32, offering up to4×4\\timesthroughput and4×4\\timesmemory reduction, with native support on NVIDIA Blackwell, AMD MI350, and other accelerators\. However, simply replacing BF16 with MXFP4 during RL post\-training produces a large accuracy gap\. To restore the accuracy, quantization\-aware trainingLiu et al\.\([2024](https://arxiv.org/html/2605.20402#bib.bib26)\)and post\-training quantizationLin et al\.\([2024](https://arxiv.org/html/2605.20402#bib.bib25)\);Frantar et al\.\([2023](https://arxiv.org/html/2605.20402#bib.bib17)\)techniques are employed to mitigate this degradation\. These works treat quantization error as a single noise source\.

Different from how previous works view the quantization error, our novel perspective decomposes the MXFP4 quantization error into three additive error components induced by: \(1\)Scale bias: the E8M0 scale overshoots the ideal scale \(E8M∞\\infty\) by∼44%\{\\sim\}44\\%on average, an bias that accumulates multiplicatively across layers in backward pass, affecting the gradient accuracy\. This error can be reduced by improving scale precision\. \(2\)Deadzone truncation: absolute values below0\.52÷6=1/24\\frac\{0\.5\}\{2\}\\div 6=1/24of the block absolute maximum are zeroed, pruning 9% of weights and degrading rollout quality\. The weight values zeroed by the deadzone are recoverable by a known technique called Outlier Fallback \(OF\)Zhang et al\.\([2025](https://arxiv.org/html/2605.20402#bib.bib43)\)\. \(3\)Grid noise: the coarse E2M1 grid adds zero\-mean rounding noise that raises the policy’s effective temperature, increasing exploration\. This component is*invariant to scale precision*, as it only depends on weights and the E2M1 grid, not the block scale\. Therefore, once the precision format is given, this error is irreducible\.

Contributions\.

1. 1\.We provide a structural decomposition of MXFP4 quantization error into three additive components: scale bias \(from power\-of\-two rounding\), deadzone truncation \(from zeroing small values\), and grid noise \(from rounding to the 4\-bit grid\)\. We demonstrate that the grid component is invariant to scale precision, establishing an irreducible error floor even with optimal scaling\.
2. 2\.Adominance analysismapping each error component to a specific RL failure mode: scale bias causes inaccurate gradient, deadzone causes rollout degradation in forward pass, and grid noise increases the policy entropy\.
3. 3\.Rather than treating quantization error as monolithic noise, we introduce specific corrections for each failure mode: Macro\-block scaling to mitigate scale bias, Outlier Fallback to address deadzone truncation, and Adaptive Quantization Noise to control policy entropy\.
4. 4\.We validate our targeted corrections on both dense \(Qwen2\.5\-3B\) and mixture\-of\-experts \(Qwen3\-30B\-A3B\-Base\) models during RL post\-training\. Our methods recover BF16 accuracy to within 0\.7 percentage points and 3\.0 percentage points respectively\.

## 2Related work

Post\-training quantization \(PTQ\)\.Early LLM quantization focused on integer formats: LLM\.int8\(\)\(Dettmers et al\.,[2022](https://arxiv.org/html/2605.20402#bib.bib11)\)and ZeroQuant\(Yao et al\.,[2022](https://arxiv.org/html/2605.20402#bib.bib41)\)demonstrated INT8 inference with round\-to\-nearest over weight groups\. GPTQ\(Frantar et al\.,[2023](https://arxiv.org/html/2605.20402#bib.bib17)\)pushed to INT4 via second\-order weight adjustments, and AWQ\(Lin et al\.,[2024](https://arxiv.org/html/2605.20402#bib.bib25)\)introduced activation\-aware scaling\. SqueezeLLM\(Kim et al\.,[2024](https://arxiv.org/html/2605.20402#bib.bib23)\)and SpQR\(Dettmers et al\.,[2024](https://arxiv.org/html/2605.20402#bib.bib13)\)handle outliers with mixed dense\-sparse representations\. These methods assume uniform integer grids and relatively large group sizes; MXFP4’s non\-uniform E2M1 grid and small block size \(32\) change the quantization dynamics fundamentally, as our decomposition reveals\.

Rotation\-based and extreme compression\.QuIP\(Chee et al\.,[2023](https://arxiv.org/html/2605.20402#bib.bib4)\), QuIP\#\(Tseng et al\.,[2024a](https://arxiv.org/html/2605.20402#bib.bib35)\), and QTIP\(Tseng et al\.,[2024b](https://arxiv.org/html/2605.20402#bib.bib36)\)use incoherence processing \(random rotations, Hadamard transforms, trellis codes\) to push compression to 2 bits with theoretical guarantees\. SmoothQuant\(Xiao et al\.,[2023](https://arxiv.org/html/2605.20402#bib.bib39)\)redistributes activation outliers to weights for INT8 quantization, while QuaRot\(Ashkboos et al\.,[2024](https://arxiv.org/html/2605.20402#bib.bib1)\)and SpinQuant\(Liu et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib27)\)apply rotation matrices to eliminate outliers entirely\. More recent rotation variants include grouped sequency\-arranged rotations\(Choi et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib8)\), pairwise rotations for reasoning\-LLM inference\(Liang et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib24)\), and MR\-GPTQ\(Egiazarian et al\.,[2026](https://arxiv.org/html/2605.20402#bib.bib14)\)which fuses micro\-rotation into GPTQ\-style PTQ specifically for MXFP4/NVFP4 with an accompanying CUTLASS\-based kernel library \(QuTLASS\)\. These techniques operate*upstream*of quantization by reshaping the weight/activation distribution\. In the language of our decomposition \(Section[3](https://arxiv.org/html/2605.20402#S3)\), they primarily reduce grid and deadzone error by making the input more amenable to the quantizer\. Our corrections instead operate*during or after*quantization and are complementary\.

Quantization\-aware training \(QAT\)\.LLM\-QAT\(Liu et al\.,[2024](https://arxiv.org/html/2605.20402#bib.bib26)\)performs data\-free QAT by using the pre\-trained model to generate training data\. QLoRA\(Dettmers et al\.,[2023](https://arxiv.org/html/2605.20402#bib.bib12)\)combines NF4 weight quantization with LoRA adapters for memory\-efficient fine\-tuning\. QeRL\(Huang et al\.,[2026](https://arxiv.org/html/2605.20402#bib.bib22)\)applies LoRA\-based QAT specifically to RL post\-training, demonstrating that quantization noise can enhance exploration\. Our work differs from QeRL in two ways: we use full\-parameter updates \(no LoRA\), providing cleaner causal evidence, and we ground the exploration benefit in our error decomposition\. Specifically, the grid noise component acts as a natural entropy regularizer\.

Low\-precision FP4/FP8 training\.A growing line of work targets training \(not just inference\) in 4–8 bit floating\-point formats\. On the FP8 side,Micikevicius et al\.\([2022](https://arxiv.org/html/2605.20402#bib.bib28)\)introduced the E4M3/E5M2 formats andFishman et al\.\([2025](https://arxiv.org/html/2605.20402#bib.bib15)\);Hernández\-Cano et al\.\([2025](https://arxiv.org/html/2605.20402#bib.bib21)\)demonstrated trillion\-token\-scale FP8 LLM pre\-training, identifying activation\-outlier instability as the central obstacle\. On the FP4 side, beyond the original MXFP4 spec\(Rouhani et al\.,[2023](https://arxiv.org/html/2605.20402#bib.bib33)\)and the MXFP4\-RL pre\-training ofTseng et al\.\([2025](https://arxiv.org/html/2605.20402#bib.bib37)\), recent work targets native FP4 training \(Quartet\(Castro et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib3)\), FP4 All The Way\(Chmiel et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib7)\)\), NVFP4 with finer scale precision\(NVIDIA,[2025](https://arxiv.org/html/2605.20402#bib.bib30);Panferov et al\.,[2026](https://arxiv.org/html/2605.20402#bib.bib31);Cook et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib10)\), and MoE\-specific FP4 training on Hopper\(Zhang et al\.,[2026](https://arxiv.org/html/2605.20402#bib.bib44)\)\. Outlier\-Safe Pre\-Training\(Park et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib32)\)attacks the same problem from the pre\-training side by preventing outliers from forming in the first place\.Hao et al\.\([2025](https://arxiv.org/html/2605.20402#bib.bib20)\)provide a recent survey covering both fixed\-point and floating\-point low\-precision training\. Our contribution is orthogonal to these training\-time format choices: we target RL post\-training of an existing BF16 checkpoint and explain how the static MXFP4 error structure maps onto specific RL failure modes\.

Table 1:Empirical error decomposition across model scales\. All three models show the same structure: scale and grid are anti\-correlated \(cos≈−0\.66\\cos\\approx\-0\.66\); deadzone is exactly orthogonal to both; the grid component is∼71%\{\\sim\}71\\%of total MSE and invariant to model scale\.![Refer to caption](https://arxiv.org/html/2605.20402v1/x1.png)Figure 1:Pairwise error component cosine similarities across 18,624 weight tensors \(Qwen3\-30B\-A3B\-Base\)\. DZ is exactly orthogonal to both scale and grid \(point masses at 0\); scale and grid are anti\-correlated withcos≈−0\.66\\cos\\approx\-0\.66and minimal variance\. This visualizes the structural relationships summarized in Table[1](https://arxiv.org/html/2605.20402#S2.T1)\.![Refer to caption](https://arxiv.org/html/2605.20402v1/x2.png)\(a\)Scale mantissa precision sweep\.
![Refer to caption](https://arxiv.org/html/2605.20402v1/x3.png)\(b\)Per\-layer\-type error decomposition \(Qwen3\-30B\-A3B\-Base\)\.

Figure 2:Error decomposition analysis\. \(a\) Improving scale precision drives total error to the irreducible floor‖𝒆grid‖2\+‖𝒆DZ‖2\\left\\\|\\bm\{e\}^\{\\mathrm\{grid\}\}\\right\\\|^\{2\}\+\\left\\\|\\bm\{e\}^\{\\mathrm\{DZ\}\}\\right\\\|^\{2\}without affecting grid error\. \(b\) All layer types share nearly same decomposition:‖𝒆scale‖2≈1\.72​‖𝒆‖2\\left\\\|\\bm\{e\}^\{\\mathrm\{scale\}\}\\right\\\|^\{2\}\\approx 1\.72\\left\\\|\\bm\{e\}\\right\\\|^\{2\},‖𝒆grid‖2≈0\.71​‖𝒆‖2\\left\\\|\\bm\{e\}^\{\\mathrm\{grid\}\}\\right\\\|^\{2\}\\approx 0\.71\\left\\\|\\bm\{e\}\\right\\\|^\{2\},‖𝒆DZ‖2≈0\.02​‖𝒆‖2\\left\\\|\\bm\{e\}^\{\\mathrm\{DZ\}\}\\right\\\|^\{2\}\\approx 0\.02\\left\\\|\\bm\{e\}\\right\\\|^\{2\}\.
## 3MXFP4 quantization error decomposition

### 3\.1Preliminaries and decomposition

The MXFP4 format \(OCP MX standard\) quantizes a tensor𝒙∈ℝn\\bm\{x\}\\in\\mathbb\{R\}^\{n\}block\-wise with block sizeB=32B\{=\}32\. Each block𝒙b=\(xb,1,…,xb,B\)∈ℝB\\bm\{x\}\_\{b\}=\(x\_\{b,1\},\\ldots,x\_\{b,B\}\)\\in\\mathbb\{R\}^\{B\}is quantized in two steps:\(i\)a shared E8M0 scalesb=2⌈log2⁡\(maxi⁡\|xb,i\|/qmax\)⌉s\_\{b\}=2^\{\\lceil\\log\_\{2\}\(\\max\_\{i\}\|x\_\{b,i\}\|/q\_\{\\max\}\)\\rceil\}and\(ii\)per\-element rounding to the E2M1 grid𝒢=\{0,±0\.5,±1,±1\.5,±2,±3,±4,±6\}\\mathcal\{G\}=\\\{0,\\pm 0\.5,\\pm 1,\\pm 1\.5,\\pm 2,\\pm 3,\\pm 4,\\pm 6\\\}\(qmax=6q\_\{\\max\}\{=\}6\), giving adequantizedvalueQ​\(xb,i\)=sb⋅arg​ming∈𝒢⁡\|xb,i/sb−g\|Q\(x\_\{b,i\}\)=s\_\{b\}\\cdot\\operatorname\*\{arg\\,min\}\_\{g\\in\\mathcal\{G\}\}\|x\_\{b,i\}/s\_\{b\}\-g\|\.

We introduce the*ideal\-scale quantizer*Q∗Q^\{\*\}that uses the unquantized scalesb∗=maxi⁡\|xb,i\|/qmaxs\_\{b\}^\{\*\}=\\max\_\{i\}\|x\_\{b,i\}\|/q\_\{\\max\}instead of the E8M0sbs\_\{b\}\. This separates scale quantization from element\-level quantization\. The*deadzone*𝒟b\\mathcal\{D\}\_\{b\}is the set of elements satisfying\|xb,i\|<mb/24\|x\_\{b,i\}\|<m\_\{b\}/24\(wheremb=maxj⁡\|xb,j\|m\_\{b\}=\\max\_\{j\}\|x\_\{b,j\}\|\), which are rounded to zero; empirically 9% of values per block\.

###### Definition 3\.1\(Error components\)\.

The total quantization erroreb,i=Q​\(xb,i\)−xb,ie\_\{b,i\}=Q\(x\_\{b,i\}\)\-x\_\{b,i\}decomposes as

eb,i=Q​\(xb,i\)−Q∗​\(xb,i\)⏟eb,iscale\+\[Q∗​\(xb,i\)−xb,i\]​1𝒟b​\(i\)⏟eb,iDZ\+\[Q∗​\(xb,i\)−xb,i\]​1𝒟bc​\(i\)⏟eb,igrid,e\_\{b,i\}=\\underbrace\{Q\(x\_\{b,i\}\)\-Q^\{\*\}\(x\_\{b,i\}\)\}\_\{e\_\{b,i\}^\{\\mathrm\{scale\}\}\}\+\\underbrace\{\[Q^\{\*\}\(x\_\{b,i\}\)\-x\_\{b,i\}\]\\,\\mathbf\{1\}\_\{\\mathcal\{D\}\_\{b\}\}\(i\)\}\_\{e\_\{b,i\}^\{\\mathrm\{DZ\}\}\}\+\\underbrace\{\[Q^\{\*\}\(x\_\{b,i\}\)\-x\_\{b,i\}\]\\,\\mathbf\{1\}\_\{\\mathcal\{D\}\_\{b\}^\{c\}\}\(i\)\}\_\{e\_\{b,i\}^\{\\mathrm\{grid\}\}\}\\,,\(1\)where𝟏𝒟b​\(i\)=𝟏​\[\|xb,i/sb∗\|<qmin/2\]\\mathbf\{1\}\_\{\\mathcal\{D\}\_\{b\}\}\(i\)=\\mathbf\{1\}\[\|x\_\{b,i\}/s\_\{b\}^\{\*\}\|<q\_\{\\min\}/2\]is the deadzone indicator \(qmin=0\.5q\_\{\\min\}\{=\}0\.5\)\. Since𝟏𝒟b\+𝟏𝒟bc=1\\mathbf\{1\}\_\{\\mathcal\{D\}\_\{b\}\}\+\\mathbf\{1\}\_\{\\mathcal\{D\}\_\{b\}^\{c\}\}=1, the decomposition is exact:𝒆=𝒆scale\+𝒆DZ\+𝒆grid\\bm\{e\}=\\bm\{e\}^\{\\mathrm\{scale\}\}\+\\bm\{e\}^\{\\mathrm\{DZ\}\}\+\\bm\{e\}^\{\\mathrm\{grid\}\}\.

### 3\.2Interactions among three error components

###### Lemma 3\.2\(Exact orthogonality: DZ⟂\\perpScale and DZ⟂\\perpGrid\)\.

⟨𝒆DZ,𝒆scale⟩=⟨𝒆DZ,𝒆grid⟩=0\\langle\\bm\{e\}^\{\\mathrm\{DZ\}\},\\,\\bm\{e\}^\{\\mathrm\{scale\}\}\\rangle=\\langle\\bm\{e\}^\{\\mathrm\{DZ\}\},\\,\\bm\{e\}^\{\\mathrm\{grid\}\}\\rangle=0\(pointwise, no assumptions needed\)\.

###### Proof\.

𝒆DZ\\bm\{e\}^\{\\mathrm\{DZ\}\}and𝒆grid\\bm\{e\}^\{\\mathrm\{grid\}\}have disjoint support by construction \(𝟏𝒟b⋅𝟏𝒟bc=0\\mathbf\{1\}\_\{\\mathcal\{D\}\_\{b\}\}\\cdot\\mathbf\{1\}\_\{\\mathcal\{D\}\_\{b\}^\{c\}\}=0\)\. For𝒆DZ⟂𝒆scale\\bm\{e\}^\{\\mathrm\{DZ\}\}\\perp\\bm\{e\}^\{\\mathrm\{scale\}\}: on deadzone elements,\|xi/sb∗\|<qmin/2\|x\_\{i\}/s\_\{b\}^\{\*\}\|<q\_\{\\min\}/2, soQ∗​\(xi\)=0Q^\{\*\}\(x\_\{i\}\)=0\. Sincesb≥sb∗s\_\{b\}\\geq s\_\{b\}^\{\*\}\(ceiling rounding\),\|xi/sb\|≤\|xi/sb∗\|<qmin/2\|x\_\{i\}/s\_\{b\}\|\\leq\|x\_\{i\}/s\_\{b\}^\{\*\}\|<q\_\{\\min\}/2, soQ​\(xi\)=0Q\(x\_\{i\}\)=0as well\. Thereforeeiscale=Q​\(xi\)−Q∗​\(xi\)=0e\_\{i\}^\{\\mathrm\{scale\}\}=Q\(x\_\{i\}\)\-Q^\{\*\}\(x\_\{i\}\)=0on every deadzone element\. ∎

Expanding‖𝒆‖2=‖𝒆scale\+𝒆DZ\+𝒆grid‖2\\left\\\|\\bm\{e\}\\right\\\|^\{2\}=\\left\\\|\\bm\{e\}^\{\\mathrm\{scale\}\}\+\\bm\{e\}^\{\\mathrm\{DZ\}\}\+\\bm\{e\}^\{\\mathrm\{grid\}\}\\right\\\|^\{2\}and applying Lemma[3\.2](https://arxiv.org/html/2605.20402#S3.Thmtheorem2)to eliminate two of three cross terms yields the MSE identity:

‖𝒆‖2=‖𝒆scale‖2\+‖𝒆DZ‖2\+‖𝒆grid‖2\+2​⟨𝒆scale,𝒆grid⟩,\\boxed\{\\left\\\|\\bm\{e\}\\right\\\|^\{2\}=\\left\\\|\\bm\{e\}^\{\\mathrm\{scale\}\}\\right\\\|^\{2\}\+\\left\\\|\\bm\{e\}^\{\\mathrm\{DZ\}\}\\right\\\|^\{2\}\+\\left\\\|\\bm\{e\}^\{\\mathrm\{grid\}\}\\right\\\|^\{2\}\+2\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{grid\}\}\\rangle\}\\,,\(2\)with exactly one surviving cross term\.

### 3\.3Grid noise is invariant to scale precision

The grid erroreigride\_\{i\}^\{\\mathrm\{grid\}\}on non\-deadzone elements depends only on the weight valuesxix\_\{i\}and the ideal scalesb∗s\_\{b\}^\{\*\},*not on the actual E8M0 scalesbs\_\{b\}*\. Changing the scale precision \(e\.g\., from E8M0 to E8M8 via MBS\) modifies𝒆scale\\bm\{e\}^\{\\mathrm\{scale\}\}but leaves𝒆grid\\bm\{e\}^\{\\mathrm\{grid\}\}completely unchanged\. As scale precision improves:

‖𝒆scale‖→0,⟨𝒆scale,𝒆grid⟩→0,‖𝒆total‖2→‖𝒆grid‖2\+‖𝒆DZ‖2⏟irreducible floor\.\\left\\\|\\bm\{e\}^\{\\mathrm\{scale\}\}\\right\\\|\\to 0,\\quad\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{grid\}\}\\rangle\\to 0,\\quad\\left\\\|\\bm\{e\}^\{\\mathrm\{total\}\}\\right\\\|^\{2\}\\to\\underbrace\{\\left\\\|\\bm\{e\}^\{\\mathrm\{grid\}\}\\right\\\|^\{2\}\+\\left\\\|\\bm\{e\}^\{\\mathrm\{DZ\}\}\\right\\\|^\{2\}\}\_\{\\text\{irreducible floor\}\}\\,\.\(3\)This floor is the structural limit of MXFP4: to go below it, the E2M1 grid itself must change\. Figure[2\(a\)](https://arxiv.org/html/2605.20402#S2.F2.sf1)visualizes this convergence\.

## 4Impact on RL training

Each error component affects all three RL training pathways \(policy gradient accuracy, rollout quality, and exploration–exploitation balance\), but the mathematical structure of each creates adominanteffect on one pathway\. We analyze these below; the full3×33\\times 3analysis is in Appendix[D](https://arxiv.org/html/2605.20402#A4)\.

![Refer to caption](https://arxiv.org/html/2605.20402v1/x4.png)Figure 3:Scale bias from E8M0 scale rounding \(Qwen3\-30B\-A3B\-Base,L=48L\{=\}48\)\. \(a\) Per\-block rounding residualδb\\delta\_\{b\}is approximately Uniform\(0,1\) with mean 0\.546\. \(b\) Cumulative scale bias∑lδl\\sum\_\{l\}\\delta\_\{l\}across layers matches the theoretical standard deviation with empiricalσ=1\.97≈σtheory=2\.0\\sigma=1\.97\\approx\\sigma\_\{\\mathrm\{theory\}\}=2\.0, with empirical mean near 2\.0 due to imperfect Uniform distribution, validating the gradient inaccuracy in Section[4\.1](https://arxiv.org/html/2605.20402#S4.SS1)\.### 4\.1Scale bias: dominant effect on policy gradients

Scale bias is*multiplicative*: each layer’s output is scaled byγb=sb/sb∗\\gamma\_\{b\}=s\_\{b\}/s\_\{b\}^\{\*\}\. In the forward pass, LayerNorm resets activation magnitudes at every layer, preventing accumulation, so the per\-layer forward\-pass error isO​\(γb−1\)≈O​\(0\.44\)O\(\\gamma\_\{b\}\{\-\}1\)\\approx O\(0\.44\), calculation shown in text after Eq\. \([4](https://arxiv.org/html/2605.20402#S4.E4)\) \. In the backward pass, however, the straight\-through estimator \(STE\) chain rule multiplies scale factors across allLLlayers without any normalization\. Let∇^\\hat\{\\nabla\}denote the gradient computed under quantization and∇true\\nabla\_\{\\mathrm\{true\}\}the full\-precision gradient\. Their log magnitude ratio accumulates as:

log⁡‖∇^‖‖∇true‖≈∑l=1Lδb​\(l\),δb∼Uniform​\[0,1\),\\log\\frac\{\\left\\\|\\hat\{\\nabla\}\\right\\\|\}\{\\left\\\|\\nabla\_\{\\mathrm\{true\}\}\\right\\\|\}\\approx\\sum\_\{l=1\}^\{L\}\\delta\_\{b\(l\)\}\\,,\\quad\\delta\_\{b\}\\sim\\mathrm\{Uniform\}\[0,\\,1\)\\,,\(4\)whereδb=⌈log2⁡sb∗⌉−log2⁡sb∗\\delta\_\{b\}=\\lceil\\log\_\{2\}s\_\{b\}^\{\*\}\\rceil\-\\log\_\{2\}s\_\{b\}^\{\*\}is the E8M0 ceiling\-rounding error in log\-space, approximately uniform by a Benford’s\-law argument \(Appendix[A](https://arxiv.org/html/2605.20402#A1), Step 1\)\. Under ceiling rounding,γb=2δb\\gamma\_\{b\}=2^\{\\delta\_\{b\}\}withδb∼Uniform​\[0,1\)\\delta\_\{b\}\\sim\\mathrm\{Uniform\}\[0,1\), givingmean​\(γ−1\)=1/ln2−1≈0\.44\\mathrm\{mean\}\(\\gamma\-1\)=1/\\mathrm\{ln\}2\-1\\approx 0\.44andRMSE​\(γ−1\)≈0\.57\\mathrm\{RMSE\}\(\\gamma\-1\)\\approx 0\.57\. By the CLT, the centred sum∑l\(δb​\(l\)−12\)∼𝒩​\(0,L/12\)\\sum\_\{l\}\(\\delta\_\{b\(l\)\}\-\\tfrac\{1\}\{2\}\)\\sim\\mathcal\{N\}\(0,L/12\)\. For Qwen2\.5\-3B withL=36L\{=\}36layers,std=3≈1\.73\\mathrm\{std\}=\\sqrt\{3\}\\approx 1\.73, so the gradient magnitude ratio‖∇^‖/‖∇true‖\\left\\\|\\hat\{\\nabla\}\\right\\\|/\\left\\\|\\nabla\_\{\\mathrm\{true\}\}\\right\\\|ranges from∼0\.18×\{\\sim\}0\.18\\timesto∼5\.6×\{\\sim\}5\.6\\timeswithin one standard deviation\. Thisexponential amplification in the backward pass, absent in the forward pass, is what makes scale bias primarily a gradient problem\. For the 48\-layer MoE model \(Qwen3\-30B\-A3B\-Base\),stdtheory=48/12=2\.0\\mathrm\{std\}\_\{\\mathrm\{theory\}\}=\\sqrt\{48/12\}=2\.0; we measurestdemp=1\.97\\mathrm\{std\}\_\{\\mathrm\{emp\}\}=1\.97\(Figure[3](https://arxiv.org/html/2605.20402#S4.F3)\)\. MBS corrects this by adding an 8\-bit mantissa to the block scale, reducingVar⁡\(γ\)\\operatorname\{Var\}\(\\gamma\)by∼\(256\)−2\{\\sim\}\(256\)^\{\-2\}\.

*Why not the other pathways?*Scale bias does perturb rollout logits and exploration atO​\(γ−1\)≈49%O\(\\gamma\{\-\}1\)\\approx 49\\%per layer, but LayerNorm re\-normalizes inputs between layers, preventing multiplicative compounding in the forward pass\.

### 4\.2Deadzone: dominant effect on rollout distribution

Deadzone truncation is*pruning*: it affects RL training by deterministically setting weights to zeros in the forward pass, changing the rollout distribution\. On Qwen3\-30B\-A3B\-Base model,9\.0%9\.0\\%of elements fall in this ideal deadzone and contribute2\.2%2\.2\\%of‖etotal‖2\\\|e\_\{\\rm total\}\\\|^\{2\}\.

*Why not gradients?*The STE backward passignores the deadzone: it passes gradients through as if the full\-precision value were present\. So deadzone damage is fully visible in the forward pass \(degraded rollouts\) but largely invisible in the backward pass \(intact gradients\)\. This asymmetry makes deadzone primarily a rollout quality problem\.

*Why not exploration?*Deadzone issystematic and contractive\(eiDZ=−xie\_\{i\}^\{\\mathrm\{DZ\}\}=\-x\_\{i\}, always toward zero\), not stochastic\. It removes capacity rather than adding randomness, producing blander outputs \(lower reward mean\) rather than more diverse ones\. In our dense experiments, OF recovers deadzone values \(DZ rate: 9%→\\to2%\), yielding\+\+17\.5 pp\.

### 4\.3Grid noise: dominant effect on exploration

Grid noise is the only component that isdense and approximately zero\-mean\. A zero\-mean perturbation to logits does not shift the mode of the output distribution; it*widens*it, acting approximately as temperature scaling\. Under additive noise𝜼∼𝒩​\(𝟎,ση2​𝑰\)\\bm\{\\eta\}\\sim\\mathcal\{N\}\(\\bm\{0\},\\sigma\_\{\\eta\}^\{2\}\\bm\{I\}\)on logits, the effective temperature is

Teff≈1\+2​ση2/Var⁡\(Δ​ℓ\)\>1,T\_\{\\mathrm\{eff\}\}\\approx\\sqrt\{1\+2\\sigma\_\{\\eta\}^\{2\}/\\operatorname\{Var\}\(\\Delta\\ell\)\}\>1\\,,\(5\)whereση2∝∑lσδ,l2​\(grid\)\\sigma\_\{\\eta\}^\{2\}\\propto\\sum\_\{l\}\\sigma\_\{\\delta,l\}^\{2\}\(\\mathrm\{grid\}\)accumulates over layers\. The noise level is*constant*throughout training, providing no mechanism for annealing\. Grid noise adds variance but no systematic bias to gradients \(far milder than the5\.6×5\.6\\timesscale\-induced fluctuation\), and its effect on reward quality averages out over full response sequences\. Detailed explanation for Eq\. \([5](https://arxiv.org/html/2605.20402#S4.E5)\) can be found in Appendix[D\.4](https://arxiv.org/html/2605.20402#A4.SS4)\.

## 5Method

Guided by the decomposition in Section[3](https://arxiv.org/html/2605.20402#S3), we adopt two error corrections and one training recipe\.MBS\(Chhugani et al\.,[2026](https://arxiv.org/html/2605.20402#bib.bib6)\)andOF\(Zhang et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib43)\)are static error corrections: MBS reduces the scale component, OF recovers the deadzone component, together driving total error toward the irreducible grid floor\.AQN\(Huang et al\.,[2026](https://arxiv.org/html/2605.20402#bib.bib22)\)does not reduce any static error; it is an RL training recipe that helps the model learn effectively on this floor\. We repurpose MBS and OF \(originally designed for inference\) for RL training with quantize\-dequantize \(QDQ\) emulation, and adapt AQN with modifications for full\-parameter training\.

QDQ emulation\.To ensure compatibility, experiments are performed using simulated quantization/dequantization in PyTorch, the standard methodology for MXFP4 training research\(Tseng et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib37);Chen et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib5)\), as native W4A4 tensor cores are not yet widely accessible\. QDQ is numerically faithful: the quantization error is format\-identical in both emulation and hardware\-native execution \(same E2M1 grid, same E8M0 block scale, FP32 accumulation\)\. End\-to\-end latency on native accelerators is complementary future work that does not affect the accuracy results here\.

### 5\.1Macro Block Scaling \(MBS\): reducing scale bias toward the grid floor

Recall from Section[3\.1](https://arxiv.org/html/2605.20402#S3.SS1)that the E8M0 shared exponentsb=2es\_\{b\}=2^\{e\}is a pure power of two, introducing a scale ratioγb=sb/sb∗\\gamma\_\{b\}=s\_\{b\}/s\_\{b\}^\{\*\}with∼54%\{\\sim\}54\\%error\. The root cause is that E8M0 has*zero mantissa bits*: it can only represent scales at powers of two, wasting up to half the dynamic range\.

MBS\(Chhugani et al\.,[2026](https://arxiv.org/html/2605.20402#bib.bib6)\)compensates for this by computing an 8\-bit mantissa correction at a coarser*macro\-block*granularity\. Consider a macro\-block ofBM=128B\_\{M\}=128elements, spanning four contiguous MXFP4 blocks ofB=32B=32\. The macro\-block scale is factored as

sMBS=2eM⋅\(1\+mMBS\),s^\{\\mathrm\{MBS\}\}=2^\{e\_\{M\}\}\\cdot\(1\+m\_\{\\mathrm\{MBS\}\}\)\\,,\(6\)whereeMe\_\{M\}is the macro\-block exponent andmMBS∈\[0,1\)m\_\{\\mathrm\{MBS\}\}\\in\[0,1\)is an 8\-bit mantissa \(E0M8: 0 exponent bits, 8 mantissa bits, complementary to the E8M0 block scale; 256 levels\)\. Since the per\-block E8M0 scales already capture the exponent, MBS only needs to store the mantissa correctionmMBSm\_\{\\mathrm\{MBS\}\}, a single byte per 128 elements, adding negligible memory overhead \(<0\.1<0\.1bits/element\)\.

Prescale–quantize–postscale\.In the QDQ emulation used during training, MBS wraps the standard MXFP4 quantization:

x^i=11\+mMBS⋅Q​\(\(1\+mMBS\)⋅xi\),\\hat\{x\}\_\{i\}=\\frac\{1\}\{1\+m\_\{\\mathrm\{MBS\}\}\}\\cdot Q\\\!\\bigl\(\(1\+m\_\{\\mathrm\{MBS\}\}\)\\cdot x\_\{i\}\\bigr\)\\,,\(7\)whereQ​\(⋅\)Q\(\\cdot\)is the standard MXFP4 block quantizer \(Definition[3\.1](https://arxiv.org/html/2605.20402#S3.Thmtheorem1)\)\. The prescale\(1\+mMBS\)\(1\+m\_\{\\mathrm\{MBS\}\}\)shifts the input distribution so that the power\-of\-two block scalesbs\_\{b\}more closely matches the ideal scalesb∗s\_\{b\}^\{\*\}; the postscale restores the original magnitude\. In hardware GEMM, the prescale and postscale fuse into the tile\-level epilogue at negligible cost\(Chhugani et al\.,[2026](https://arxiv.org/html/2605.20402#bib.bib6)\)\. The same adaptive\-block\-scaling idea has concurrently been proposed for NVFP4\(Cook et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib10)\), demonstrating that scale\-precision correction is broadly useful across MX\-family formats\.

Effect on error budget\.With MBS, the effective scale ratio becomesγbMBS=sb/\[sb∗⋅\(1\+mMBS\)\]\\gamma\_\{b\}^\{\\mathrm\{MBS\}\}=s\_\{b\}/\[s\_\{b\}^\{\*\}\\cdot\(1\+m\_\{\\mathrm\{MBS\}\}\)\]\. The 8\-bit mantissa reduces the scale quantization step from 1 \(pure power\-of\-two\) to1/2561/256, shrinkingVar⁡\(γ\)\\operatorname\{Var\}\(\\gamma\)by a factor of∼\(256\)−2≈1\.5×10−5\{\\sim\}\(256\)^\{\-2\}\\approx 1\.5\\times 10^\{\-5\}\. By the grid\-invariance property \(Section[3\.3](https://arxiv.org/html/2605.20402#S3.SS3)\),𝒆grid\\bm\{e\}^\{\\mathrm\{grid\}\}is completely unaffected\. As‖𝒆scale‖→0\\left\\\|\\bm\{e\}^\{\\mathrm\{scale\}\}\\right\\\|\\to 0, the cross term⟨𝒆scale,𝒆grid⟩→0\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{grid\}\}\\rangle\\to 0as well, and the total MSE converges to the irreducible floor‖𝒆grid‖2\+‖𝒆DZ‖2\\left\\\|\\bm\{e\}^\{\\mathrm\{grid\}\}\\right\\\|^\{2\}\+\\left\\\|\\bm\{e\}^\{\\mathrm\{DZ\}\}\\right\\\|^\{2\}\.

### 5\.2Outlier Fallback \(OF\): recovering deadzone values

The deadzone truncates all values with\|xi\|<mb/24\|x\_\{i\}\|<m\_\{b\}/24to zero, destroying 9% of values per block \(Section[4\.2](https://arxiv.org/html/2605.20402#S4.SS2)\)\. Unlike scale bias, which can be corrected by adjusting the scale factor, deadzone is an*information loss*: once a value is mapped to zero, no rescaling can recover it\. OF addresses this through a two\-pass residual quantization, adapting the block\-level fallback technique ofZhang et al\.\([2025](https://arxiv.org/html/2605.20402#bib.bib43)\)from INT8 to MXFP4\.

Two\-pass QDQ\.LetQ​\(⋅\)Q\(\\cdot\)denote the standard MXFP4 block quantizer\. OF replaces the single\-passQ​\(𝒙\)Q\(\\bm\{x\}\)with:

𝒙^1\\displaystyle\\hat\{\\bm\{x\}\}\_\{1\}=Q​\(𝒙\),\\displaystyle=Q\(\\bm\{x\}\)\\,,\(8\)𝒙^2\\displaystyle\\hat\{\\bm\{x\}\}\_\{2\}=Q​\(𝒙−𝒙^1\),\\displaystyle=Q\(\\bm\{x\}\-\\hat\{\\bm\{x\}\}\_\{1\}\)\\,,\(9\)𝒙^OF\\displaystyle\\hat\{\\bm\{x\}\}\_\{\\mathrm\{OF\}\}=𝒙^1\+𝒙^2\.\\displaystyle=\\hat\{\\bm\{x\}\}\_\{1\}\+\\hat\{\\bm\{x\}\}\_\{2\}\\,\.\(10\)In Pass 1, outliers are quantized accurately \(they set the block scale\), but small values fall into the deadzone\. In Pass 2, the residual𝒙−𝒙^1\\bm\{x\}\-\\hat\{\\bm\{x\}\}\_\{1\}has much smaller dynamic range, so the block scale is set by the previously dead values, which can now be represented on the E2M1 grid\.

### 5\.3Complementary RL training recipe: Adaptive Quantization Noise

After MBS and OF reduce the scale and deadzone components, the residual error is dominated by𝒆grid\\bm\{e\}^\{\\mathrm\{grid\}\}, the irreducible floor of the MXFP4 format\. AQN\(Huang et al\.,[2026](https://arxiv.org/html/2605.20402#bib.bib22)\)does not reduce this static error; it is a complementary RL training technique that helps the model learn effectively on the grid noise floor\. The idea of*controlled*parameter noise to aid exploration has classical roots in NoisyNet\(Fortunato et al\.,[2018](https://arxiv.org/html/2605.20402#bib.bib16)\)and gradient\-noise injection\(Neelakantan et al\.,[2015](https://arxiv.org/html/2605.20402#bib.bib29)\); AQN specialises this lineage to the structured grid\-noise floor of MXFP4\. Before each rollout, AQN perturbs weights with Gaussian noiseϵ∼𝒩​\(𝟎,σ2​𝑰\)\\bm\{\\epsilon\}\\sim\\mathcal\{N\}\(\\bm\{0\},\\sigma^\{2\}\\bm\{I\}\)whose magnitude decays exponentially fromσstart\\sigma\_\{\\mathrm\{start\}\}toσend\\sigma\_\{\\mathrm\{end\}\}overKKstages, preventing the premature entropy collapse caused by constant grid noise \(Section[4\.3](https://arxiv.org/html/2605.20402#S4.SS3)\)\. We useσstart=1%\\sigma\_\{\\mathrm\{start\}\}\{=\}1\\%,σend=0\.1%\\sigma\_\{\\mathrm\{end\}\}\{=\}0\.1\\%,K=10K\{=\}10, applying noise to both LayerNorm and linear layer weights\.

![Refer to caption](https://arxiv.org/html/2605.20402v1/x5.png)Figure 4:Ablation results on GSM8K\. \(a\) MoE: corrections stack incrementally; AQN\+MBS\+OF reaches−\-3\.0 pp\. \(b\) Dense: OF is critical \(\+\+17\.5 pp alone\); the full stack recovers to−\-0\.7 pp\. Dashed line: BF16 baseline\.

## 6Experiments

### 6\.1Setup

Models and task\.We evaluate on two architectures: Qwen2\.5\-3B \(dense, 36 layers\) and Qwen3\-30B\-A3B\-Base \(MoE, 48 layers, 3B active parameters\)\. The task is GSM8K\(Cobbe et al\.,[2021](https://arxiv.org/html/2605.20402#bib.bib9)\)math reasoning with verifiable rewards\.

Training\.All runs use GRPO\(Shao et al\.,[2024](https://arxiv.org/html/2605.20402#bib.bib34)\)with batch size 64 and max response length 1024; the scaffolding builds on verl\-VeRL alongside related RL\-LLM frameworks\(Yu et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib42);Fu et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib18)\)\. For MoE: 8×\\timesH100 GPUs, Megatron parallelism, 2 samples per prompt\. For dense: 2×\\timesH100 GPUs, FSDP2, 4 samples per prompt\. Full hyperparameter list in Appendix[G](https://arxiv.org/html/2605.20402#A7)\.

Quantization\.MXFP4 W4A4 via QDQ emulation: both weights and activations quantized to E2M1 elements with E8M0 block scale \(block size 32\)\. Truncated Importance Sampling \(TIS\)Yao et al\.\([2025](https://arxiv.org/html/2605.20402#bib.bib40)\)clips the importance ratioρi\\rho\_\{i\}from above atρmax\\rho\_\{\\max\}\(i\.e\.,min⁡\(ρi,ρmax\)\\min\(\\rho\_\{i\},\\rho\_\{\\max\}\)\): we use TIS=2\{\}=2for MoE and TIS=5\{\}=5for dense\. The larger threshold for dense is necessary because quantization\-induced policy divergence concentrates importance ratios near the clipping boundary at TIS=2\{\}=2, effectively zeroing out the learning signal \(MBS and OF both cause training collapse at TIS=2\{\}=2on dense\)\.

![Refer to caption](https://arxiv.org/html/2605.20402v1/x6.png)Figure 5:Training dynamics \(MoE, GSM8K\)\. \(a\) AQN sustains policy entropy, preventing premature convergence\. \(b\) AQN\+MBS maintains higher gradient norms, indicating continued learning\. BF16 baseline shown for reference\.
### 6\.2Main results

Table 2:MoE results \(Qwen3\-30B\-A3B\-Base, GSM8K, W4A4\)\. BF16 baseline: 94\.09%\.Table 3:Dense results \(Qwen2\.5\-3B, GSM8K, MXFP4 W4A4\)\. BF16 baseline: 82\.0%\.Tables[2](https://arxiv.org/html/2605.20402#S6.T2)–[3](https://arxiv.org/html/2605.20402#S6.T3)present the main results\. On the MoE model, AQN\+MBS\+OF closes the gap from−\-5\.3 pp to−\-3\.0 pp \(91\.1%\)\. On dense, the naïve baseline suffers−\-21\.9 pp, but MBS\+AQN\+OF recovers to within 0\.7 pp of BF16 \(81\.3% vs\. 82\.0%\)\. Figure[4](https://arxiv.org/html/2605.20402#S5.F4)visualizes these results\.

Correction composition\.The three corrections compose near\-additively on MoE: MBS alone\+\+1\.3 pp, OF alone\+\+1\.5 pp, AQN alone\+\+0\.4 pp, versus\+\+2\.3 pp combined\. The slight subadditivity is consistent with partial overlap between MBS and OF on outlier\-heavy blocks\. On dense, the picture is strikingly different: OF alone \(\+\+17\.5 pp\) dwarfs MBS \(\+\+15\.1 pp\), and without OF, MBS recovers only 15 pp of the 21\.9 pp gap\. This asymmetry is predicted by the decomposition: dense models lack expert routing to compensate for deadzone pruning, making OF the critical correction\.

Dense vs\. MoE dichotomy\.OF provides\+\+17\.5 pp on dense but only\+\+1\.5 pp on MoE, because expert routing provides natural redundancy against deadzone pruning\. Dense models have no such mechanism, making OF essential\.

AQNσ\\sigmasensitivity\.On dense MXFP4 with MBS\+OF,σstart=0\.1%\\sigma\_\{\\mathrm\{start\}\}=0\.1\\%is too weak \(71\.2%\),2%2\\%is slightly too strong \(80\.1%\), and1%1\\%is optimal \(81\.3%\)\. The optimalσstart\\sigma\_\{\\mathrm\{start\}\}should be comparable to the per\-element grid noise magnitude, which for MXFP4 is on the order of1%1\\%of the weight scale\.

Training dynamics\.Figure[5](https://arxiv.org/html/2605.20402#S6.F5)shows that the MXFP4 baseline’s entropy drops to 0\.35 by step 50 \(premature convergence\), while AQN\+MBS maintains 0\.61\. Gradient norms show the same pattern: baseline decays to 0\.16 vs\. 0\.24 for AQN\+MBS\.

## 7Conclusion

We have presented a three\-way decomposition of MXFP4 quantization error into scale bias, deadzone truncation, and grid noise: three additive components with distinct statistical signatures and distinct effects on RL training dynamics\. The decomposition rests on two formal results: \(i\) exact orthogonality of the deadzone component to both others \(Lemma[3\.2](https://arxiv.org/html/2605.20402#S3.Thmtheorem2)\), which follows from the ceiling\-rounding propertysb≥sb∗s\_\{b\}\\geq s\_\{b\}^\{\*\}; and \(ii\) invariance of grid noise to scale precision \(Section[3\.3](https://arxiv.org/html/2605.20402#S3.SS3)\), which establishes an irreducible error floor intrinsic to the E2M1 grid\. These properties are not model\-dependent: the ratios‖𝐞scale‖2/‖𝐞‖2≈1\.72\\\|\\mathbf\{e\}^\{\\mathrm\{scale\}\}\\\|^\{2\}/\\\|\\mathbf\{e\}\\\|^\{2\}\\approx 1\.72andcos⁡\(𝐞scale,𝐞grid\)≈−0\.66\\cos\(\\mathbf\{e\}^\{\\mathrm\{scale\}\},\\mathbf\{e\}^\{\\mathrm\{grid\}\}\)\\approx\-0\.66are consistent across all 18,876 weight tensors measured in Qwen2\.5\-3B and Qwen3\-30B\-A3B\-Base, suggesting they are properties of the MXFP4 format itself rather than of tested models\.

The corrections are mechanism\-targeted but not component\-exclusive: MBS reduces scale bias \(driving total MSE toward the irreducible floor\), OF recovers deadzone entries \(addressing what MBS cannot\), but also partially reduces scale bias induced error, and AQN prevents premature entropy collapse on the residual grid noise floor\. On Qwen2\.5\-3B \(dense\), the full combination recovers BF16 accuracy to within 0\.7 pp \(81\.3% vs\. 82\.0%\); on Qwen3\-30B\-A3B\-Base \(MoE\), the gap narrows to 3\.0 pp \(91\.1% vs\. 94\.1%\)\.

Limitations\.All experiments use GSM8K; generalization to other reasoning, coding, and safety benchmarks needs validation\. We use QDQ emulation rather than native MXFP4 hardware; throughput measurements on Blackwell or MI350 accelerators are future work\. The AQN noise schedule \(σstart\\sigma\_\{\\mathrm\{start\}\}\) requires per\-model tuning and we do not yet provide an adaptive rule\. Most configurations are single\-seed\.

Future work\.Combining MBS/OF with upstream techniques \(stochastic rounding, Hadamard transforms\(Tseng et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib37)\)\) could further reduce the grid\+deadzone floor\. Extending the analysis to activation quantization, where deadzone rates are higher, would complete the W4A4 picture\.

## References

- Ashkboos et al\. \[2024\]Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L\. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman\.QuaRot: Outlier\-free 4\-bit inference in rotated LLMs\.In*Advances in Neural Information Processing Systems*, volume 37, 2024\.
- Bennett \[1948\]W\. R\. Bennett\.Spectra of quantized signals\.*Bell System Technical Journal*, 27\(3\):446–472, 1948\.
- Castro et al\. \[2025\]Roberto L\. Castro, Andrei Panferov, Soroush Tabesh, Oliver Sieberling, Jiale Chen, Mahdi Nikdan, Saleh Ashkboos, and Dan Alistarh\.Quartet: Native FP4 training can be optimal for large language models\.*arXiv preprint arXiv:2505\.14669*, 2025\.
- Chee et al\. \[2023\]Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher De Sa\.QuIP: 2\-bit quantization of large language models with guarantees\.In*Advances in Neural Information Processing Systems*, volume 36, 2023\.
- Chen et al\. \[2025\]Yuxiang Chen, Haocheng Xi, Jun Zhu, and Jianfei Chen\.Oscillation\-reduced MXFP4 training for vision transformers\.In*International Conference on Machine Learning \(ICML\)*, 2025\.
- Chhugani et al\. \[2026\]Jatin Chhugani, Geonhwa Jeong, Bor\-Yiing Su, Yunjie Pan, Hanmei Yang, Aayush Ankit, Jiecao Yu, Summer Deng, Yunqing Chen, Nadathur Satish, and Changkyu Kim\.Unveiling the potential of quantization with MXFP4: Strategies for quantization error reduction\.*arXiv preprint arXiv:2603\.08713*, 2026\.
- Chmiel et al\. \[2025\]Brian Chmiel, Maxim Fishman, Ron Banner, and Daniel Soudry\.FP4 all the way: Fully quantized training of LLMs\.*arXiv preprint arXiv:2505\.19115*, 2025\.
- Choi et al\. \[2025\]Euntae Choi, Sumin Song, Woosang Lim, and Sungjoo Yoo\.Grouped sequency\-arranged rotation: Optimizing rotation transformation for quantization for free\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics: Student Research Workshop*, pages 165–172, 2025\.doi:10\.18653/v1/2025\.acl\-srw\.10\.
- Cobbe et al\. \[2021\]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman\.Training verifiers to solve math word problems\.*arXiv preprint arXiv:2110\.14168*, 2021\.
- Cook et al\. \[2025\]Jack Cook, Junxian Guo, Guangxuan Xiao, Yujun Lin, and Song Han\.Four over six: More accurate NVFP4 quantization with adaptive block scaling\.*arXiv preprint arXiv:2512\.02010*, 2025\.
- Dettmers et al\. \[2022\]Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer\.LLM\.int8\(\): 8\-bit Matrix Multiplication for Transformers at Scale\.In*Advances in Neural Information Processing Systems*, volume 35, 2022\.
- Dettmers et al\. \[2023\]Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer\.QLoRA: Efficient finetuning of quantized LLMs\.In*Advances in Neural Information Processing Systems*, volume 36, 2023\.
- Dettmers et al\. \[2024\]Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh\.SpQR: A sparse\-quantized representation for near\-lossless LLM weight compression\.In*International Conference on Learning Representations*, 2024\.
- Egiazarian et al\. \[2026\]Vage Egiazarian, Roberto L\. Castro, Denis Kuznedelev, Andrei Panferov, Eldar Kurtic, Shubhra Pandit, Alexandre Noll Marques, Mark Kurtz, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh\.Bridging the gap between promise and performance for microscaling FP4 quantization\.In*International Conference on Learning Representations*, 2026\.Introduces MR\-GPTQ \(micro\-rotated GPTQ\) and the QuTLASS GPU kernel library; arXiv:2509\.23202\.
- Fishman et al\. \[2025\]Maxim Fishman, Brian Chmiel, Ron Banner, and Daniel Soudry\.Scaling FP8 training to trillion\-token LLMs\.In*International Conference on Learning Representations*, 2025\.URL[https://openreview\.net/forum?id=E1EHO0imOb](https://openreview.net/forum?id=E1EHO0imOb)\.
- Fortunato et al\. \[2018\]Meire Fortunato, Mohammad Gheshlaghi Azar, Bilal Piot, Jacob Menick, Matteo Hessel, Ian Osband, Alex Graves, Volodymyr Mnih, Rémi Munos, Demis Hassabis, Olivier Pietquin, Charles Blundell, and Shane Legg\.Noisy networks for exploration\.In*International Conference on Learning Representations*, 2018\.URL[https://openreview\.net/forum?id=rywHCPkAW](https://openreview.net/forum?id=rywHCPkAW)\.
- Frantar et al\. \[2023\]Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh\.GPTQ: Accurate post\-training quantization for generative pre\-trained transformers\.In*International Conference on Learning Representations*, 2023\.
- Fu et al\. \[2025\]Wei Fu, Jiaxuan Gao, Xujie Shen, Chen Zhu, Zhiyu Mei, Chuyi He, Shusheng Xu, Guo Wei, Jun Mei, Jiashu Wang, Tongkai Yang, Binhang Yuan, and Yi Wu\.AReaL: A large\-scale asynchronous reinforcement learning system for language reasoning\.*arXiv preprint arXiv:2505\.24298*, 2025\.
- Gupta et al\. \[2015\]Suyog Gupta, Ankur Agrawal, Kailash Gopalakrishnan, and Pritish Narayanan\.Deep learning with limited numerical precision\.In*Proceedings of the 32nd International Conference on Machine Learning \(ICML\)*, volume 37, pages 1737–1746, 2015\.URL[http://proceedings\.mlr\.press/v37/gupta15\.html](http://proceedings.mlr.press/v37/gupta15.html)\.
- Hao et al\. \[2025\]Zhiwei Hao, Jianyuan Guo, Li Shen, Yong Luo, Han Hu, Guoxia Wang, Dianhai Yu, Yonggang Wen, and Dacheng Tao\.Low\-precision training of large language models: Methods, challenges, and opportunities\.*arXiv preprint arXiv:2505\.01043*, 2025\.
- Hernández\-Cano et al\. \[2025\]Alejandro Hernández\-Cano, Dhia Garbaya, Imanol Schlag, and Martin Jaggi\.Towards fully FP8 GEMM LLM training at scale\.*arXiv preprint arXiv:2505\.20524*, 2025\.
- Huang et al\. \[2026\]Wei Huang, Yi Ge, Shuai Yang, Yicheng Xiao, Huizi Mao, Yujun Lin, Hanrong Ye, Sifei Liu, Ka Chun Cheung, Hongxu Yin, Yao Lu, Xiaojuan Qi, Song Han, and Yukang Chen\.QeRL: Beyond efficiency—quantization\-enhanced reinforcement learning for LLMs\.In*International Conference on Learning Representations*, 2026\.
- Kim et al\. \[2024\]Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W\. Mahoney, and Kurt Keutzer\.SqueezeLLM: Dense\-and\-sparse quantization\.In*Proceedings of the 41st International Conference on Machine Learning*, 2024\.
- Liang et al\. \[2025\]Yesheng Liang, Haisheng Chen, Song Han, and Zhijian Liu\.ParoQuant: Pairwise rotation quantization for efficient reasoning LLM inference\.*arXiv preprint arXiv:2511\.10645*, 2025\.
- Lin et al\. \[2024\]Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei\-Ming Chen, Wei\-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han\.AWQ: Activation\-aware weight quantization for on\-device LLM compression and acceleration\.In*Proceedings of Machine Learning and Systems*, volume 6, 2024\.
- Liu et al\. \[2024\]Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra\.Llm\-qat: Data\-free quantization aware training for large language models\.In*Findings of the Association for Computational Linguistics: ACL 2024*, pages 467–484, 2024\.
- Liu et al\. \[2025\]Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort\.SpinQuant: LLM quantization with learned rotations\.In*International Conference on Learning Representations*, 2025\.
- Micikevicius et al\. \[2022\]Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu, Naveen Mellempudi, Stuart F\. Oberman, Mohammad Shoeybi, Michael Y\. Siu, and Hao Wu\.FP8 formats for deep learning\.*arXiv preprint arXiv:2209\.05433*, 2022\.
- Neelakantan et al\. \[2015\]Arvind Neelakantan, Luke Vilnis, Quoc V\. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, and James Martens\.Adding gradient noise improves learning for very deep networks\.*arXiv preprint arXiv:1511\.06807*, 2015\.
- NVIDIA \[2025\]NVIDIA\.Pretraining large language models with NVFP4\.*arXiv preprint arXiv:2509\.25149*, 2025\.
- Panferov et al\. \[2026\]Andrei Panferov, Erik Schultheis, Soroush Tabesh, and Dan Alistarh\.Quartet II: Accurate LLM pre\-training in NVFP4 by improved unbiased gradient estimation\.*arXiv preprint arXiv:2601\.22813*, 2026\.
- Park et al\. \[2025\]Jungwoo Park, Taewhoo Lee, Chanwoong Yoon, Hyeon Hwang, and Jaewoo Kang\.Outlier\-safe pre\-training for robust 4\-bit quantization of large language models\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(ACL\)*, pages 12582–12600, 2025\.
- Rouhani et al\. \[2023\]Bita Darvish Rouhani, Ritchie Zhao, Ankit More, Mathew Hall, Alireza Khodamoradi, Summer Deng, Dhruv Choudhary, Marius Cornea, Eric Dellinger, Kristof Denolf, et al\.OCP microscaling formats \(MX\) specification\.*Open Compute Project*, 2023\.
- Shao et al\. \[2024\]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y\. K\. Li, Y\. Wu, and Daya Guo\.DeepSeekMath: Pushing the limits of mathematical reasoning in open language models\.*arXiv preprint arXiv:2402\.03300*, 2024\.
- Tseng et al\. \[2024a\]Albert Tseng, Jerry Chee, Qingyao Sun, Volodymyr Kuleshov, and Christopher De Sa\.QuIP\#: Even better LLM quantization with hadamard incoherence and lattice codebooks\.In*Proceedings of the 41st International Conference on Machine Learning*, 2024a\.
- Tseng et al\. \[2024b\]Albert Tseng, Qingyao Sun, David Hou, and Christopher De Sa\.QTIP: Quantization with trellises and incoherence processing\.In*Advances in Neural Information Processing Systems*, volume 37, 2024b\.
- Tseng et al\. \[2025\]Albert Tseng, Tao Yu, and Youngsuk Park\.Training llms with mxfp4, 2025\.URL[https://arxiv\.org/abs/2502\.20586](https://arxiv.org/abs/2502.20586)\.
- Widrow and Kollár \[2008\]Bernard Widrow and István Kollár\.*Quantization Noise: Roundoff Error in Digital Computation, Signal Processing, Control, and Communications*\.Cambridge University Press, 2008\.ISBN 978\-0\-521\-88671\-0\.
- Xiao et al\. \[2023\]Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han\.SmoothQuant: Accurate and efficient post\-training quantization for large language models\.In*Proceedings of the 40th International Conference on Machine Learning*, 2023\.
- Yao et al\. \[2025\]Feng Yao, Liyuan Liu, Dinghuai Zhang, Chengyu Dong, Jingbo Shang, and Jianfeng Gao\.Your efficient rl framework secretly brings you off\-policy rl training, August 2025\.URL[https://fengyao\.notion\.site/off\-policy\-rl](https://fengyao.notion.site/off-policy-rl)\.
- Yao et al\. \[2022\]Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He\.ZeroQuant: Efficient and affordable post\-training quantization for large\-scale transformers\.In*Advances in Neural Information Processing Systems*, volume 35, pages 27168–27183, 2022\.
- Yu et al\. \[2025\]Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, et al\.DAPO: An open\-source LLM reinforcement learning system at scale\.*arXiv preprint arXiv:2503\.14476*, 2025\.
- Zhang et al\. \[2025\]Pengle Zhang, Jia Wei, Jintao Zhang, Jun Zhu, and Jianfei Chen\.Accurate INT8 training through dynamic block\-level fallback\.*arXiv preprint arXiv:2503\.08040*, 2025\.
- Zhang et al\. \[2026\]Wuyue Zhang, Chongdong Huang, Chunbo You, Cheng Gu, Fengjuan Wang, and Mou Sun\.Practical FP4 training for large\-scale MoE models on hopper GPUs\.*arXiv preprint arXiv:2603\.02731*, 2026\.

## Appendix AAnalysis of the cross term⟨𝒆scale,𝒆grid⟩\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{grid\}\}\\rangle

Note on scope\.The main body establishes the MSE identity \(Eq\.[2](https://arxiv.org/html/2605.20402#S3.E2)\) and reports the empirical anti\-correlationcos⁡\(𝒆scale,𝒆grid\)≈−0\.66\\cos\(\\bm\{e\}^\{\\mathrm\{scale\}\},\\bm\{e\}^\{\\mathrm\{grid\}\}\)\\approx\-0\.66\(Table[1](https://arxiv.org/html/2605.20402#S2.T1)\)\. This appendix analyzes the cross term under the following idealized assumption\. Under this assumption the cross term is𝒪​\(B−1/2\)\\mathcal\{O\}\(B^\{\-1/2\}\), but in practice, the ceiling\-based scale selection \(γb≥1\\gamma\_\{b\}\\geq 1always\) creates a structural anti\-correlation absent from the idealized model\. The analysis below characterizes the idealized setting for completeness; the empirical measurements in Table[1](https://arxiv.org/html/2605.20402#S2.T1)and Appendix[F](https://arxiv.org/html/2605.20402#A6)characterize the real setting\.

###### Assumption A\.1\.

Within each block the elements\(xb,1,…,xb,B\)\(x\_\{b,1\},\\ldots,x\_\{b,B\}\)are i\.i\.d\. from a continuous, symmetric distribution𝒫\\mathcal\{P\}with finite fourth moment\. The quantitylog2⁡sb∗\\log\_\{2\}s\_\{b\}^\{\*\}is non\-lattice\.

Define the scale ratioγb=sb/sb∗=2δb\\gamma\_\{b\}=s\_\{b\}/s\_\{b\}^\{\*\}=2^\{\\delta\_\{b\}\}, normalized elementsub,i=qmax⋅xb,i/mbu\_\{b,i\}=q\_\{\\max\}\\cdot x\_\{b,i\}/m\_\{b\}, and letr​\(⋅\)r\(\\cdot\)denoteroundE2M1\\mathrm\{round\}\_\{\\mathrm\{E2M1\}\}\.

Step 1: Characterizeγb\\gamma\_\{b\}\.Under Assumption[A\.1](https://arxiv.org/html/2605.20402#A1.Thmtheorem1),δb=⌈log2⁡sb∗⌉−log2⁡sb∗\\delta\_\{b\}=\\lceil\\log\_\{2\}s\_\{b\}^\{\*\}\\rceil\-\\log\_\{2\}s\_\{b\}^\{\*\}depends only on the fractional part oflog2⁡mb\\log\_\{2\}m\_\{b\}\. For ceiling rounding,δb∈\[0,1\)\\delta\_\{b\}\\in\[0,1\)\. By the non\-lattice condition,δb\\delta\_\{b\}is approximatelyUniform​\[0,1\)\\mathrm\{Uniform\}\[0,1\), giving

γb∼2U,U∼Uniform​\[0,1\),𝔼​\[γb\]=1ln⁡2≈1\.44,RMSE​\(γb−1\)≈0\.57\.\\gamma\_\{b\}\\sim 2^\{U\},\\;U\\sim\\mathrm\{Uniform\}\[0,\\,1\)\\,,\\quad\\mathbb\{E\}\[\\gamma\_\{b\}\]=\\frac\{1\}\{\\ln 2\}\\approx 1\.44\\,,\\quad\\mathrm\{RMSE\}\(\\gamma\_\{b\}\{\-\}1\)\\approx 0\.57\\,\.\(11\)Empirically, we measureRMSE​\(γb−1\)=0\.57\\mathrm\{RMSE\}\(\\gamma\_\{b\}\{\-\}1\)=0\.57on Qwen3\-30B\-A3B\-Base, consistent with the ceiling\-rounding model\.

Step 2: Conditional independence structure\.Write the cross term for blockbb:

⟨𝒆bscale,𝒆belem⟩=∑i=1Beb,iscale⋅eb,ielem,\\langle\\bm\{e\}\_\{b\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}\_\{b\}^\{\\mathrm\{elem\}\}\\rangle=\\sum\_\{i=1\}^\{B\}e\_\{b,i\}^\{\\mathrm\{scale\}\}\\cdot e\_\{b,i\}^\{\\mathrm\{elem\}\}\\,,\(12\)whereeelem=eDZ\+egride^\{\\mathrm\{elem\}\}=e^\{\\mathrm\{DZ\}\}\+e^\{\\mathrm\{grid\}\}\. Conditioning onmbm\_\{b\}, the scale ratioγb\\gamma\_\{b\}becomes deterministic:

eb,iscale\\displaystyle e\_\{b,i\}^\{\\mathrm\{scale\}\}=sb⋅r​\(xb,i/sb\)−sb∗⋅r​\(xb,i/sb∗\)=h​\(γb,ub,i\),\\displaystyle=s\_\{b\}\\cdot r\(x\_\{b,i\}/s\_\{b\}\)\-s\_\{b\}^\{\*\}\\cdot r\(x\_\{b,i\}/s\_\{b\}^\{\*\}\)=h\(\\gamma\_\{b\},\\,u\_\{b,i\}\)\\,,\(13\)eb,ielem\\displaystyle e\_\{b,i\}^\{\\mathrm\{elem\}\}=sb∗​\[r​\(ub,i\)−ub,i\]=sb∗​g​\(ub,i\),\\displaystyle=s\_\{b\}^\{\*\}\[r\(u\_\{b,i\}\)\-u\_\{b,i\}\]=s\_\{b\}^\{\*\}\\,g\(u\_\{b,i\}\)\\,,\(14\)withh​\(γb,u\)=0h\(\\gamma\_\{b\},u\)=0whenγb=1\\gamma\_\{b\}=1\.

Step 3: Conditional distribution of non\-max elements\.Since theBBelements are i\.i\.d\. from𝒫\\mathcal\{P\}, conditioning on the block maximummb=maxj⁡\|xb,j\|m\_\{b\}=\\max\_\{j\}\|x\_\{b,j\}\|only tells us that each non\-max element satisfies\|xb,i\|≤mb\|x\_\{b,i\}\|\\leq m\_\{b\}\. By Bayes’ rule, the conditional distribution of each non\-max element is simply𝒫\\mathcal\{P\}truncated to\[−mb,mb\]\[\-m\_\{b\},m\_\{b\}\]:

P​\(xb,i∈A∣mb\)=P​\(xb,i∈A∩\[−mb,mb\]\)F​\(mb\)−F​\(−mb\),P\(x\_\{b,i\}\\in A\\mid m\_\{b\}\)=\\frac\{P\(x\_\{b,i\}\\in A\\cap\[\-m\_\{b\},m\_\{b\}\]\)\}\{F\(m\_\{b\}\)\-F\(\-m\_\{b\}\)\}\\,,\(15\)and theB−1B\{\-\}1non\-max elements remain mutually independent \(conditioning on the max restricts their range but does not introduce dependence\)\. For symmetric𝒫\\mathcal\{P\}with smooth densityff, the conditional expectation of any function of the normalized elementub,i=qmax⋅xb,i/mbu\_\{b,i\}=q\_\{\\max\}\\cdot x\_\{b,i\}/m\_\{b\}is:

𝔼​\[g​\(ub,i\)∣mb\]=∫−qmaxqmaxg​\(u\)​f​\(u⋅mb/qmax\)F​\(mb\)−F​\(−mb\)⋅mbqmax​𝑑u\.\\mathbb\{E\}\[g\(u\_\{b,i\}\)\\mid m\_\{b\}\]=\\int\_\{\-q\_\{\\max\}\}^\{q\_\{\\max\}\}g\(u\)\\,\\frac\{f\(u\\cdot m\_\{b\}/q\_\{\\max\}\)\}\{F\(m\_\{b\}\)\-F\(\-m\_\{b\}\)\}\\cdot\\frac\{m\_\{b\}\}\{q\_\{\\max\}\}\\,du\\,\.\(16\)
Forγb\\gamma\_\{b\}near 1, Taylor expansion givesh​\(γb,u\)≈\(γb−1\)⋅ϕ​\(u\)h\(\\gamma\_\{b\},u\)\\approx\(\\gamma\_\{b\}\-1\)\\cdot\\phi\(u\), so:

𝔼​\[h​\(γb,ub,i\)⋅g​\(ub,i\)∣mb\]≈\(γb−1\)​𝔼​\[ϕ​\(ub,i\)​g​\(ub,i\)∣mb\]\.\\mathbb\{E\}\[h\(\\gamma\_\{b\},u\_\{b,i\}\)\\cdot g\(u\_\{b,i\}\)\\mid m\_\{b\}\]\\approx\(\\gamma\_\{b\}\-1\)\\,\\mathbb\{E\}\[\\phi\(u\_\{b,i\}\)\\,g\(u\_\{b,i\}\)\\mid m\_\{b\}\]\\,\.\(17\)
Step 4: CLT averaging\.Taking expectation overmbm\_\{b\}:

𝔼​\[eb,iscale⋅eb,ielem\]=𝔼mb​\[\(γb−1\)​ψ​\(mb\)\],\\mathbb\{E\}\[e\_\{b,i\}^\{\\mathrm\{scale\}\}\\cdot e\_\{b,i\}^\{\\mathrm\{elem\}\}\]=\\mathbb\{E\}\_\{m\_\{b\}\}\[\(\\gamma\_\{b\}\-1\)\\,\\psi\(m\_\{b\}\)\]\\,,\(18\)whereψ​\(mb\)=𝔼​\[ϕ​\(u\)​g​\(u\)∣mb\]\\psi\(m\_\{b\}\)=\\mathbb\{E\}\[\\phi\(u\)\\,g\(u\)\\mid m\_\{b\}\]\. The block\-summed cross term hasBBconditionally i\.i\.d\. terms, so by the CLT:

1B​∑i=1Beb,iscale⋅eb,ielem=\(γb−1\)​ψ​\(mb\)\+𝒪P​\(B−1/2\)\.\\frac\{1\}\{B\}\\sum\_\{i=1\}^\{B\}e\_\{b,i\}^\{\\mathrm\{scale\}\}\\cdot e\_\{b,i\}^\{\\mathrm\{elem\}\}=\(\\gamma\_\{b\}\-1\)\\,\\psi\(m\_\{b\}\)\+\\mathcal\{O\}\_\{P\}\(B^\{\-1/2\}\)\\,\.\(19\)
The leading term𝔼​\[\(γ−1\)​ψ\]\\mathbb\{E\}\[\(\\gamma\{\-\}1\)\\psi\]is small:ψ\\psiinvolves the productϕ​\(u\)⋅g​\(u\)\\phi\(u\)\\cdot g\(u\), which has alternating sign across E2M1 grid intervals\. Under the smoothness offf, these cancel to𝒪​\(Var⁡\(γ\)/B\)\\mathcal\{O\}\(\\operatorname\{Var\}\(\\gamma\)/\\sqrt\{B\}\)\. Summing overNNblocks and normalizing by𝔼​\[‖𝒆‖2\]=Θ​\(N​B⋅Var⁡\(γ\)\)\\mathbb\{E\}\[\\left\\\|\\bm\{e\}\\right\\\|^\{2\}\]=\\Theta\(NB\\cdot\\operatorname\{Var\}\(\\gamma\)\)yields the𝒪​\(B−1/2\)\\mathcal\{O\}\(B^\{\-1/2\}\)relative bound\.■\\blacksquare

## Appendix BIndividual bounds for Scale⟂\\perpDZ and Scale⟂\\perpGrid

This appendix illustrates the orthogonality⟨𝒆scale,𝒆DZ⟩=0\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{DZ\}\}\\rangle=0\(Lemma[3\.2](https://arxiv.org/html/2605.20402#S3.Thmtheorem2)\) and the anti\-correlation⟨𝒆scale,𝒆grid⟩<0\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{grid\}\}\\rangle<0\(Remark[3\.3](https://arxiv.org/html/2605.20402#S3.Thmtheorem3)\) with a worked numerical example\.

Because𝒆DZ\\bm\{e\}^\{\\mathrm\{DZ\}\}and𝒆grid\\bm\{e\}^\{\\mathrm\{grid\}\}have disjoint support \(Lemma[3\.2](https://arxiv.org/html/2605.20402#S3.Thmtheorem2)\), the single surviving cross term partitions element\-wise:

⟨𝒆scale,𝒆DZ⟩\\displaystyle\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{DZ\}\}\\rangle=∑b∑i∈𝒟beb,iscale⋅eb,iDZ,\\displaystyle=\\sum\_\{b\}\\sum\_\{i\\in\\mathcal\{D\}\_\{b\}\}e\_\{b,i\}^\{\\mathrm\{scale\}\}\\cdot e\_\{b,i\}^\{\\mathrm\{DZ\}\}\\,,\(20\)⟨𝒆scale,𝒆grid⟩\\displaystyle\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{grid\}\}\\rangle=∑b∑i∉𝒟beb,iscale⋅eb,igrid\.\\displaystyle=\\sum\_\{b\}\\sum\_\{i\\notin\\mathcal\{D\}\_\{b\}\}e\_\{b,i\}^\{\\mathrm\{scale\}\}\\cdot e\_\{b,i\}^\{\\mathrm\{grid\}\}\\,\.\(21\)These are sums overdisjoint index setswithin each block\. Two consequences follow\.

#### Why⟨𝒆scale,𝒆DZ⟩=0\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{DZ\}\}\\rangle=0exactly\.

For ceiling\-based exponent selection \(sb=2⌈log2⁡sb∗⌉s\_\{b\}=2^\{\\lceil\\log\_\{2\}s\_\{b\}^\{\*\}\\rceil\}, the standard convention to avoid overflow\),γb≥1\\gamma\_\{b\}\\geq 1, so the deadzone undersbs\_\{b\}is*wider*than undersb∗s\_\{b\}^\{\*\}\. Every element in𝒟b\\mathcal\{D\}\_\{b\}maps to zero under both scales, givingeiscale=Q​\(xi\)−Q∗​\(xi\)=0e\_\{i\}^\{\\mathrm\{scale\}\}=Q\(x\_\{i\}\)\-Q^\{\*\}\(x\_\{i\}\)=0for alli∈𝒟bi\\in\\mathcal\{D\}\_\{b\}\. The only surviving cross term in the MSE identity \(Eq\.[2](https://arxiv.org/html/2605.20402#S3.E2)\) is therefore⟨𝒆scale,𝒆grid⟩\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{grid\}\}\\rangle, which acts exclusively on non\-deadzone elements\.

#### Numerical example\.

Consider a block𝒙=\[0\.03,0\.1,0\.3,0\.5,0\.9,1\.5,2\.0,4\.0\]\\bm\{x\}=\[0\.03,\\;0\.1,\\;0\.3,\\;0\.5,\\;0\.9,\\;1\.5,\\;2\.0,\\;4\.0\]withqmax=6q\_\{\\max\}\{=\}6\. The ideal scale iss∗=4\.0/6=0\.667s^\{\*\}=4\.0/6=0\.667; the E8M0 scale iss=20=1\.0s=2^\{0\}=1\.0;γ=s/s∗=1\.5\\gamma=s/s^\{\*\}=1\.5\. The deadzone threshold iss∗×0\.25=0\.167s^\{\*\}\\times 0\.25=0\.167, placingx1=0\.03x\_\{1\}\{=\}0\.03andx2=0\.1x\_\{2\}\{=\}0\.1in𝒟b\\mathcal\{D\}\_\{b\}\.

Sinceγ≥1\\gamma\\geq 1, both deadzone elements haveeiscale=0e\_\{i\}^\{\\mathrm\{scale\}\}=0\(bold\), giving⟨𝒆scale,𝒆DZ⟩=0\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{DZ\}\}\\rangle=0exactly\. The only nonzero cross term is⟨𝒆scale,𝒆grid⟩=\(\+0\.167\)​\(\+0\.033\)\+\(−0\.167\)​\(\+0\.167\)\+\(\+0\.167\)​\(−0\.167\)=−0\.050\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{grid\}\}\\rangle=\(\+0\.167\)\(\+0\.033\)\+\(\-0\.167\)\(\+0\.167\)\+\(\+0\.167\)\(\-0\.167\)=\-0\.050, which has mixed\-sign terms that partially cancel within themselves\. There is no opposing⟨𝒆scale,𝒆DZ⟩\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{DZ\}\}\\rangleto mask this value\.

The numerical example also illustrates the anti\-correlation: ati=3i=3,escale=\+0\.167e^\{\\mathrm\{scale\}\}=\+0\.167andegrid=\+0\.033e^\{\\mathrm\{grid\}\}=\+0\.033\(same sign, weak\); ati=4i=4,escale=−0\.167e^\{\\mathrm\{scale\}\}=\-0\.167andegrid=\+0\.167e^\{\\mathrm\{grid\}\}=\+0\.167\(opposite sign, strong\); ati=6i=6,escale=\+0\.167e^\{\\mathrm\{scale\}\}=\+0\.167andegrid=−0\.167e^\{\\mathrm\{grid\}\}=\-0\.167\(opposite sign\)\. The net inner product is negative \(−0\.050\-0\.050\), consistent with thecos≈−0\.66\\cos\\approx\-0\.66observed at scale across all weight tensors \(Table[1](https://arxiv.org/html/2605.20402#S2.T1)\)\.

## Appendix CGEMM output error propagation

For a linear layer𝒚=𝑾​𝒙\\bm\{y\}=\\bm\{W\}\\bm\{x\}with quantized weightsQ​\(𝑾\)=𝑾\+𝑬Q\(\\bm\{W\}\)=\\bm\{W\}\+\\bm\{E\}:

𝒚^=Q​\(𝑾\)​𝒙=𝒚\+𝑬​𝒙,𝑬=𝑬scale\+𝑬DZ\+𝑬grid\.\\hat\{\\bm\{y\}\}=Q\(\\bm\{W\}\)\\bm\{x\}=\\bm\{y\}\+\\bm\{E\}\\bm\{x\}\\,,\\quad\\bm\{E\}=\\bm\{E\}^\{\\mathrm\{scale\}\}\+\\bm\{E\}^\{\\mathrm\{DZ\}\}\+\\bm\{E\}^\{\\mathrm\{grid\}\}\\,\.\(22\)
###### Corollary C\.1\(GEMM\-level decomposition\)\.

If𝐄\\bm\{E\}and𝐱\\bm\{x\}are independent, then𝔼​\[‖𝐄​𝐱‖2\]=tr⁡\(𝔼​\[𝐄⊤​𝐄\]​𝚺x\)\\mathbb\{E\}\[\\left\\\|\\bm\{E\}\\bm\{x\}\\right\\\|^\{2\}\]=\\operatorname\{tr\}\(\\mathbb\{E\}\[\\bm\{E\}^\{\\top\}\\bm\{E\}\]\\,\\bm\{\\Sigma\}\_\{x\}\)\. By Eq\.[2](https://arxiv.org/html/2605.20402#S3.E2)and Lemma[3\.2](https://arxiv.org/html/2605.20402#S3.Thmtheorem2),𝔼​\[𝐄⊤​𝐄\]\\mathbb\{E\}\[\\bm\{E\}^\{\\top\}\\bm\{E\}\]decomposes with one cross term \(scale×\\timesgrid\) and exact orthogonality of the DZ component:

𝔼​\[‖𝜹‖2\]≈tr⁡\(𝔼​\[\(𝑬scale\)⊤​𝑬scale\]​𝚺x\)⏟σδ2​\(scale\)\+tr⁡\(𝔼​\[\(𝑬DZ\)⊤​𝑬DZ\]​𝚺x\)⏟σδ2​\(DZ\)\+tr⁡\(𝔼​\[\(𝑬grid\)⊤​𝑬grid\]​𝚺x\)⏟σδ2​\(grid\)\.\\mathbb\{E\}\[\\left\\\|\\bm\{\\delta\}\\right\\\|^\{2\}\]\\approx\\underbrace\{\\operatorname\{tr\}\\\!\\bigl\(\\mathbb\{E\}\[\(\\bm\{E\}^\{\\mathrm\{scale\}\}\)^\{\\top\}\\bm\{E\}^\{\\mathrm\{scale\}\}\]\\,\\bm\{\\Sigma\}\_\{x\}\\bigr\)\}\_\{\\sigma\_\{\\delta\}^\{2\}\(\\mathrm\{scale\}\)\}\+\\underbrace\{\\operatorname\{tr\}\\\!\\bigl\(\\mathbb\{E\}\[\(\\bm\{E\}^\{\\mathrm\{DZ\}\}\)^\{\\top\}\\bm\{E\}^\{\\mathrm\{DZ\}\}\]\\,\\bm\{\\Sigma\}\_\{x\}\\bigr\)\}\_\{\\sigma\_\{\\delta\}^\{2\}\(\\mathrm\{DZ\}\)\}\+\\underbrace\{\\operatorname\{tr\}\\\!\\bigl\(\\mathbb\{E\}\[\(\\bm\{E\}^\{\\mathrm\{grid\}\}\)^\{\\top\}\\bm\{E\}^\{\\mathrm\{grid\}\}\]\\,\\bm\{\\Sigma\}\_\{x\}\\bigr\)\}\_\{\\sigma\_\{\\delta\}^\{2\}\(\\mathrm\{grid\}\)\}\\,\.\(23\)

The approximation drops the scale×\\timesgrid cross term2​tr⁡\(𝔼​\[\(𝑬scale\)⊤​𝑬grid\]​𝚺x\)2\\operatorname\{tr\}\(\\mathbb\{E\}\[\(\\bm\{E\}^\{\\mathrm\{scale\}\}\)^\{\\top\}\\bm\{E\}^\{\\mathrm\{grid\}\}\]\\,\\bm\{\\Sigma\}\_\{x\}\); under MBS \(‖𝑬scale‖→0\\left\\\|\\bm\{E\}^\{\\mathrm\{scale\}\}\\right\\\|\\to 0\) this term vanishes and the equality becomes exact\.

For W4A4 \(both weights and activations quantized\), the activation error𝒆A\\bm\{e\}\_\{A\}adds an independent term‖𝑾​𝒆A‖2\\left\\\|\\bm\{W\}\\bm\{e\}\_\{A\}\\right\\\|^\{2\}\(the cross term𝑬W​𝒆A\\bm\{E\}\_\{W\}\\bm\{e\}\_\{A\}is second\-order\)\. The same three\-way decomposition applies to𝒆A\\bm\{e\}\_\{A\}\.

## Appendix DDetailed RL pathway analysis

This appendix provides \(i\) the full3×33\\times 3analysis of how each error component interacts with each RL pathway, and \(ii\) extended derivations for each dominant effect\.

### D\.1Full3×33\\times 3interaction analysis

#### Scale bias on reward signal \(secondary\)\.

Scale error perturbs each layer’s output by a multiplicative factorγb\\gamma\_\{b\}, shifting rollout logits\. However, LayerNorm resets activation magnitudes at each layer boundary, limiting the per\-layer forward\-pass distortion toO​\(γb−1\)≈O​\(0\.5\)O\(\\gamma\_\{b\}\{\-\}1\)\\approx O\(0\.5\)\. This is significant but does not compound across layers the way it does in the backward pass\.

#### Scale bias on exploration \(secondary\)\.

The multiplicative logit shift also changes the effective entropy, but by the sameO​\(γb−1\)O\(\\gamma\_\{b\}\{\-\}1\)per\-layer amount\. Since it has nonzero mean \(𝔼​\[γ−1\]≈0\.44\\mathbb\{E\}\[\\gamma\{\-\}1\]\\approx 0\.44under ceiling rounding\), the effect is more bias\-like than noise\-like, contributing to mode shift rather than distribution widening\.

#### Deadzone on gradients \(secondary\)\.

The STE backward pass treatsQQas identity: gradients flow through deadzone elements as if they retained their original values\. The deadzone therefore has minimal direct impact on gradient computation, though the degraded forwardpass indirectly changes the rollout distribution\.

#### Deadzone on exploration \(secondary\)\.

Deadzone is contractive \(eiDZ=−xie\_\{i\}^\{\\mathrm\{DZ\}\}=\-x\_\{i\}, always toward zero\), not stochastic\. It reduces the model’s representational capacity without adding randomness\. The entropy may increase slightly \(a less capable model produces more uniform predictions\), but this is a side effect of capacity loss, not a controlled exploration mechanism\.

#### Grid noise on reward signal \(secondary\)\.

Grid noise adds zero\-mean variance to each token’s logit vector\. Over a full response sequence of hundreds of tokens, these i\.i\.d\. perturbations average out in their effect on the overall response quality\. The reward is determined by the complete sequence, not individual tokens, so the impact on reward signal quality is attenuated byO​\(1/T\)O\(1/\\sqrt\{T\}\)whereTTis the sequence length\.

#### Grid noise on gradients \(secondary\)\.

Zero\-mean forward\-pass noise produces zero\-mean gradient noise\. This adds variance to the gradient estimate but no systematic bias\. In contrast, scale error introduces a multiplicative bias that grows exponentially with depth\. The gradient variance from grid noise is comparable to the inherent stochasticity of mini\-batch sampling and does not qualitatively change training dynamics\.

### D\.2Dominant effect 1: deadzone and reward signal degradation

#### Effective rank reduction\.

Definereff​\(𝑾\)=‖𝑾‖∗2/‖𝑾‖F2r\_\{\\mathrm\{eff\}\}\(\\bm\{W\}\)=\\left\\\|\\bm\{W\}\\right\\\|\_\{\*\}^\{2\}/\\left\\\|\\bm\{W\}\\right\\\|\_\{F\}^\{2\}\. The deadzone applies a soft\-thresholding operator𝒮t​\(𝑾\)\\mathcal\{S\}\_\{t\}\(\\bm\{W\}\)that zeroes elements below thresholdt=mb/24t=m\_\{b\}/24\. Since small singular values are disproportionately affected by element\-wise thresholding \(by the Eckart–Young theorem\):

reff​\(𝒮t​\(𝑾\)\)<reff​\(𝑾\)\.r\_\{\\mathrm\{eff\}\}\(\\mathcal\{S\}\_\{t\}\(\\bm\{W\}\)\)<r\_\{\\mathrm\{eff\}\}\(\\bm\{W\}\)\\,\.\(24\)

#### Advantage variance increase\.

Lower effective rank leads to rollout responses with less nuanced reasoning\. The reward signalrir\_\{i\}has lower mean and higher variance, making the group\-normalized advantageA^i=\(ri−r¯\)/σr\\hat\{A\}\_\{i\}=\(r\_\{i\}\-\\bar\{r\}\)/\\sigma\_\{r\}noisier:

Var⁡\(A^iQ\)\>Var⁡\(A^i\)\.\\operatorname\{Var\}\(\\hat\{A\}\_\{i\}^\{Q\}\)\>\\operatorname\{Var\}\(\\hat\{A\}\_\{i\}\)\\,\.\(25\)

#### Why OF helps\.

OF’s two\-pass residual QDQ recovers deadzone values: Pass 1 quantizes normally \(outliers are accurate, small values enter the deadzone\); Pass 2 quantizes the residual \(whose dynamic range is small enough for previously dead values to be represented\)\. The DZ rate drops from 9% to 2%, directly restoring effective rank and rollout quality\. In our dense experiments, OF alone provides\+\+17\.5 pp recovery \(60\.1%→\\to77\.6%\)\.

#### Dense vs\. MoE asymmetry\.

In MoE architectures, expert routing acts as a natural error\-correcting code: even if one expert’s small weights are pruned, other experts can compensate\. Dense models have no such redundancy, so deadzone damage is fully expressed\. This explains why OF is critical for dense \(\+\+17\.5 pp\) but only modestly helpful for MoE \(\+\+1\.5 pp\)\. Figure[6](https://arxiv.org/html/2605.20402#A4.F6)shows the training curves\.

![Refer to caption](https://arxiv.org/html/2605.20402v1/x7.png)Figure 6:OF sensitivity by architecture\. \(a\) Dense: OF provides\+\+17\.5 pp\. \(b\) MoE: OF provides only\+\+1\.5 pp\. MoE expert routing compensates for deadzone pruning\.

### D\.3Pathway 2 details: policy gradient bias

#### Gradient scale derivation\.

For a single linear layer with quantized weight, the forward pass output is𝒚^=γb⋅𝑾b​𝒙\+𝒆~b​𝒙\\hat\{\\bm\{y\}\}=\\gamma\_\{b\}\\cdot\\bm\{W\}\_\{b\}\\bm\{x\}\+\\tilde\{\\bm\{e\}\}\_\{b\}\\bm\{x\}, where𝒆~b\\tilde\{\\bm\{e\}\}\_\{b\}contains element\-level errors\. The STE backward pass treatsQQas identity, computing∂ℒ/∂𝑾\|STE=\(∂ℒ/∂𝒚^\)⋅𝒙⊤\\partial\\mathcal\{L\}/\\partial\\bm\{W\}\|\_\{\\mathrm\{STE\}\}=\(\\partial\\mathcal\{L\}/\\partial\\hat\{\\bm\{y\}\}\)\\cdot\\bm\{x\}^\{\\top\}at a point shifted byγb\\gamma\_\{b\}\. Through the chain rule acrossLLlayers:

∇^θ​ℒ=∏l=1Lγb​\(l\)⋅∇θℒ\|true\+higher\-order\.\\hat\{\\nabla\}\_\{\\theta\}\\mathcal\{L\}=\\prod\_\{l=1\}^\{L\}\\gamma\_\{b\(l\)\}\\cdot\\nabla\_\{\\theta\}\\mathcal\{L\}\\big\|\_\{\\mathrm\{true\}\}\+\\text\{higher\-order\}\\,\.\(26\)
Sinceδb∼Uniform​\[0,1\)\\delta\_\{b\}\\sim\\mathrm\{Uniform\}\[0,1\)approximately i\.i\.d\., by the CLT:

∑l=1Lδb​\(l\)∼𝒩​\(0,L/12\)\.\\sum\_\{l=1\}^\{L\}\\delta\_\{b\(l\)\}\\sim\\mathcal\{N\}\(0,\\,L/12\)\\,\.\(27\)For Qwen2\.5\-3B withL=36L\{=\}36,std=36/12=3≈1\.73\\mathrm\{std\}=\\sqrt\{36/12\}=\\sqrt\{3\}\\approx 1\.73, so the gradient magnitude ratio ranges from∼0\.18×\{\\sim\}0\.18\\timesto∼5\.6×\{\\sim\}5\.6\\timeswithin one standard deviation\.

#### Why MBS helps\.

MBS replacessb=2es\_\{b\}=2^\{e\}withsbMBS=2e⋅\(1\+c/256\)s\_\{b\}^\{\\mathrm\{MBS\}\}=2^\{e\}\\cdot\(1\+c/256\), whereccis an 8\-bit mantissa correction at macro\-block granularity \(128 elements\)\. This reducesVar⁡\(γ\)\\operatorname\{Var\}\(\\gamma\)by∼\(256\)−2\{\\sim\}\(256\)^\{\-2\}, effectively eliminating scale\-induced gradient bias\.

#### Connection to TIS\.

TIS \(Section[6\.1](https://arxiv.org/html/2605.20402#S6.SS1)\) clips the importance ratioρi\\rho\_\{i\}toρmax\\rho\_\{\\max\}\. In our experiments, TIS=5\{\}=5is critical for dense models: with TIS=2\{\}=2, the quantized rollout policy diverges too far from the training policy, causing importance ratios to concentrate near the clipping boundary and effectively zeroing out the learning signal\. The larger TIS=5\{\}=5threshold accommodates the additional policy divergence introduced by quantization\. This is consistent with scale bias being a gradient\-pathway problem: quantization\-induced logit perturbations increaseρi\\rho\_\{i\}variance, and a tighter TIS clip amplifies the resulting gradient bias\.

### D\.4Pathway 3 details: exploration–exploitation perturbation

#### Grid noise distribution\.

The grid error for non\-deadzone elements has variance depending on the local E2M1 step size:

eb,igrid∼Uniform​\[−Δ​\(ub,i\)/2,Δ​\(ub,i\)/2\],e\_\{b,i\}^\{\\mathrm\{grid\}\}\\sim\\mathrm\{Uniform\}\[\-\\Delta\(u\_\{b,i\}\)/2,\\;\\Delta\(u\_\{b,i\}\)/2\]\\,,\(28\)whereΔ=0\.5\\Delta=0\.5for\|u\|∈\[0,2\]\|u\|\\in\[0,2\],Δ=1\\Delta=1for\|u\|∈\[2,4\]\|u\|\\in\[2,4\], andΔ=2\\Delta=2for\|u\|∈\[4,6\]\|u\|\\in\[4,6\]\.

#### Effective temperature\.

This noise propagates to logits asℓQ=ℓ\+𝜼\\bm\{\\ell\}^\{Q\}=\\bm\{\\ell\}\+\\bm\{\\eta\},𝜼∼𝒩​\(𝟎,ση2​𝑰\)\\bm\{\\eta\}\\sim\\mathcal\{N\}\(\\bm\{0\},\\sigma\_\{\\eta\}^\{2\}\\bm\{I\}\)approximately by CLT, whereση2∝∑lσδ,l2​\(grid\)\\sigma\_\{\\eta\}^\{2\}\\propto\\sum\_\{l\}\\sigma\_\{\\delta,l\}^\{2\}\(\\mathrm\{grid\}\)accumulates over layers\. Matching the signal\-to\-noise ratio of logit differences under temperature scaling versus additive noise \(derivation below\) givesTeff≈1\+2​ση2/Var⁡\(Δ​ℓ\)\>1T\_\{\\mathrm\{eff\}\}\\approx\\sqrt\{1\+2\\sigma\_\{\\eta\}^\{2\}/\\operatorname\{Var\}\(\\Delta\\ell\)\}\>1, directly increasing policy entropy:

ℋ​\[πQ​\(𝜽\)\]\>ℋ​\[π𝜽\]\.\\mathcal\{H\}\[\\pi\_\{Q\(\\bm\{\\theta\}\)\}\]\>\\mathcal\{H\}\[\\pi\_\{\\bm\{\\theta\}\}\]\.\(29\)This matches the experimental observation: baseline entropy drops to 0\.35 by step 50, while AQN\+MBS entropy stays at 0\.61\.

#### Derivation ofTeffT\_\{\\mathrm\{eff\}\}\(Eq\.[5](https://arxiv.org/html/2605.20402#S4.E5)\)\.

The square\-root form arises from matching the softmax\-induced action distribution under additive logit noise to the same distribution under deterministic temperature scaling\. Three observations drive the derivation\.

*\(i\) Logit differences are all that matter\.*The softmax policy depends only on pairwise differences: for actionsa,ba,b,

logπ​\(a\|s\)π​\(b\|s\)=ℓa−ℓb=:Δℓa​b\.\\log\\frac\{\\pi\(a\\,\|\\,s\)\}\{\\pi\(b\\,\|\\,s\)\}\\;=\\;\\ell\_\{a\}\-\\ell\_\{b\}\\;=:\\;\\Delta\\ell\_\{ab\}\.\(30\)Temperature scaling atTTshrinks these differences uniformly toΔ​ℓa​b/T\\Delta\\ell\_\{ab\}/T\.

*\(ii\) Per\-pair noise has doubled variance\.*Withηa,ηb\\eta\_\{a\},\\eta\_\{b\}i\.i\.d\.∼𝒩​\(0,ση2\)\\sim\\mathcal\{N\}\(0,\\sigma\_\{\\eta\}^\{2\}\), the pairwise difference is

Δ​ηa​b:=ηa−ηb∼𝒩​\(0,2​ση2\),\\Delta\\eta\_\{ab\}\\;:=\\;\\eta\_\{a\}\-\\eta\_\{b\}\\;\\sim\\;\\mathcal\{N\}\(0,\\,2\\sigma\_\{\\eta\}^\{2\}\),\(31\)which is the source of the factor 2 inside the square root of Eq\.[5](https://arxiv.org/html/2605.20402#S4.E5)\.

*\(iii\) Probit–logit Gaussian\-marginalization identity\.*Using the classical approximationσ​\(z\)≈Φ​\(z/κ\)\\sigma\(z\)\\approx\\Phi\(z/\\kappa\)withκ=8/π\\kappa=\\sqrt\{8/\\pi\}, and the fact that the convolution of a probit with a Gaussian is another probit with summed variances,

𝔼Δ​ηa​b​\[σ​\(Δ​ℓa​b\+Δ​ηa​b\)\]≈σ​\(Δ​ℓa​b1\+Var⁡\(Δ​ηa​b\)/κ2\),\\mathbb\{E\}\_\{\\Delta\\eta\_\{ab\}\}\\\!\\bigl\[\\sigma\(\\Delta\\ell\_\{ab\}\+\\Delta\\eta\_\{ab\}\)\\bigr\]\\;\\approx\\;\\sigma\\\!\\left\(\\frac\{\\Delta\\ell\_\{ab\}\}\{\\sqrt\{1\+\\operatorname\{Var\}\(\\Delta\\eta\_\{ab\}\)/\\kappa^\{2\}\}\}\\right\)\\,,\(32\)whereσ​\(⋅\)\\sigma\(\\cdot\)is the logistic function andΦ​\(⋅\)\\Phi\(\\cdot\)the standard\-normal CDF\. The right\-hand side is exactly a temperature\-scaled deterministic softmax atT=1\+2​ση2/κ2T=\\sqrt\{1\+2\\sigma\_\{\\eta\}^\{2\}/\\kappa^\{2\}\}, recovering the square\-root form\.

*Fromκ2\\kappa^\{2\}toVar⁡\(Δ​ℓ\)\\operatorname\{Var\}\(\\Delta\\ell\)\.*Eq\.[5](https://arxiv.org/html/2605.20402#S4.E5)substitutes the empirical logit\-difference varianceVar⁡\(Δ​ℓ\)\\operatorname\{Var\}\(\\Delta\\ell\)for the constantκ2\\kappa^\{2\}\. This is a signal\-to\-noise argument:κ\\kappais the characteristic logit scale over which the sigmoid transitions, whileVar\(Δℓ\)1/2\\operatorname\{Var\}\(\\Delta\\ell\)^\{1/2\}is the policy’s*own*characteristic logit scale at the current training step\. The substitution makesTeffT\_\{\\mathrm\{eff\}\}adaptive to the current policy sharpness: \(a\) when the policy is confident,Var⁡\(Δ​ℓ\)≫ση2\\operatorname\{Var\}\(\\Delta\\ell\)\\gg\\sigma\_\{\\eta\}^\{2\}, soTeff→1T\_\{\\mathrm\{eff\}\}\\to 1and noise is negligible; \(b\) when the policy is flat,Var⁡\(Δ​ℓ\)≪ση2\\operatorname\{Var\}\(\\Delta\\ell\)\\ll\\sigma\_\{\\eta\}^\{2\}, soTeff→2​ση2/Var⁡\(Δ​ℓ\)T\_\{\\mathrm\{eff\}\}\\to\\sqrt\{2\\sigma\_\{\\eta\}^\{2\}/\\operatorname\{Var\}\(\\Delta\\ell\)\}and noise dominates\. The “1\+1\{\+\}” clampsTeff≥1T\_\{\\mathrm\{eff\}\}\\geq 1: zero\-mean grid noise can only*widen*, never sharpen, the policy\. In the noise\-free limitση→0\\sigma\_\{\\eta\}\\to 0,Teff=1T\_\{\\mathrm\{eff\}\}=1recovers the unperturbed policy\.

#### AQN as controlled exploration\.

AQN injects scheduled noiseσAQN​\(t\):σstart→σend\\sigma\_\{\\mathrm\{AQN\}\}\(t\):\\sigma\_\{\\mathrm\{start\}\}\\to\\sigma\_\{\\mathrm\{end\}\}, giving total noiseσtotal​\(t\)=σgrid2\+σAQN2​\(t\)\\sigma\_\{\\mathrm\{total\}\}\(t\)=\\sqrt\{\\sigma\_\{\\mathrm\{grid\}\}^\{2\}\+\\sigma\_\{\\mathrm\{AQN\}\}^\{2\}\(t\)\}\. Early in training,σAQN≫σgrid\\sigma\_\{\\mathrm\{AQN\}\}\\gg\\sigma\_\{\\mathrm\{grid\}\}provides controlled broad exploration; late,σAQN→0\\sigma\_\{\\mathrm\{AQN\}\}\\to 0allows exploitation\.

#### AQN\+MBS synergy\.

Without MBS, the noise floor includes scale error \(∼54%\{\\sim\}54\\%RMSE in the scale ratio, block\-correlated\)\. This structured noise contaminates AQN’s controlled exploration\. After MBS reducesσscale\\sigma\_\{\\mathrm\{scale\}\}to∼0\.1%\{\\sim\}0\.1\\%, only the i\.i\.d\.\-likeσgrid\\sigma\_\{\\mathrm\{grid\}\}remains, which AQN can effectively “train through”\. This explains the superadditive result: MBS alone\+\+0\.9 pp, AQN alone\+\+0\.4 pp, combined\+\+2\.0 pp\. Figures[7](https://arxiv.org/html/2605.20402#A4.F7)and[3](https://arxiv.org/html/2605.20402#S4.F3)provide supporting evidence\.

![Refer to caption](https://arxiv.org/html/2605.20402v1/x8.png)Figure 7:AQNσstart\\sigma\_\{\\mathrm\{start\}\}sensitivity \(Dense, MBS\+OF\)\.σ=1%\\sigma=1\\%is optimal;2%2\\%overshoots and degrades below no\-AQN baseline\.

## Appendix EComplementarity with upstream techniques

Our two error corrections \(MBS, OF\) operate during quantization, while AQN operates on the training dynamics\. An alternative strategy is to reshape the input distribution*before*quantization so that the format’s limitations bite less\. Stochastic rounding \(SR\), originally introduced for low\-precision neural training byGupta et al\.\([2015](https://arxiv.org/html/2605.20402#bib.bib19)\), and random Hadamard transforms \(RHT\), as used byTseng et al\.\([2025](https://arxiv.org/html/2605.20402#bib.bib37)\)for MXFP4 pre\-training, both follow this upstream approach\.

In terms of our decomposition, SR primarily targets𝒆grid\\bm\{e\}^\{\\mathrm\{grid\}\}: it makes the rounding error exactly zero\-mean at every point \(not just on average; classical quantization\-noise theory\(Bennett,[1948](https://arxiv.org/html/2605.20402#bib.bib2);Widrow and Kollár,[2008](https://arxiv.org/html/2605.20402#bib.bib38)\)already gives the additive\-uniform model that SR enforces unconditionally\), eliminating the local bias within each quantization bin\. SR also softens the deadzone \(a value below the threshold has a nonzero probability of rounding to the nearest nonzero grid point\) but does not eliminate it\. RHT primarily targets𝒆DZ\\bm\{e\}^\{\\mathrm\{DZ\}\}: by spreading outliers across elements via a random orthogonal rotation, it equalizes magnitudes within each block, reducing the fraction of values that fall below the deadzone thresholdmb/24m\_\{b\}/24\.

Crucially, neither SR nor RHT addresses𝒆scale\\bm\{e\}^\{\\mathrm\{scale\}\}directly: the E8M0 power\-of\-two scale remains after any input rotation\. MBS is the only technique in our toolkit \(or theirs\) that corrects this component\. The two families of techniques are thereforecomplementary: SR/RHT reshape the distribution to reduce deadzone and grid error at the source, while MBS/OF/AQN correct whatever error remains after quantization\. Exploring their combination is a promising direction for future work\.

## Appendix FEmpirical validation of decomposition structure

We measure the pairwise cosine similarities and the cross term on real weight tensors to validate the MSE identity \(Eq\.[2](https://arxiv.org/html/2605.20402#S3.E2)\) and the anti\-correlation \(Remark[3\.3](https://arxiv.org/html/2605.20402#S3.Thmtheorem3)\)\. For each linear layer’s weight tensor, we compute the three error components \(Definition[3\.1](https://arxiv.org/html/2605.20402#S3.Thmtheorem1)\) and report:

1. 1\.Identity verification: confirm that‖𝒆scale‖2\+‖𝒆DZ‖2\+‖𝒆grid‖2\+2​⟨𝒆scale,𝒆grid⟩=‖𝒆‖2\\left\\\|\\bm\{e\}^\{\\mathrm\{scale\}\}\\right\\\|^\{2\}\+\\left\\\|\\bm\{e\}^\{\\mathrm\{DZ\}\}\\right\\\|^\{2\}\+\\left\\\|\\bm\{e\}^\{\\mathrm\{grid\}\}\\right\\\|^\{2\}\+2\\langle\\bm\{e\}^\{\\mathrm\{scale\}\},\\,\\bm\{e\}^\{\\mathrm\{grid\}\}\\rangle=\\left\\\|\\bm\{e\}\\right\\\|^\{2\}holds exactly \(up to floating\-point precision\) for each tensor\.
2. 2\.Exact orthogonality: verifycos⁡\(𝒆scale,𝒆DZ\)=0\\cos\(\\bm\{e\}^\{\\mathrm\{scale\}\},\\bm\{e\}^\{\\mathrm\{DZ\}\}\)=0andcos⁡\(𝒆DZ,𝒆grid\)=0\\cos\(\\bm\{e\}^\{\\mathrm\{DZ\}\},\\bm\{e\}^\{\\mathrm\{grid\}\}\)=0exactly, confirming Lemma[3\.2](https://arxiv.org/html/2605.20402#S3.Thmtheorem2)\.
3. 3\.Anti\-correlation: reportcos⁡\(𝒆scale,𝒆grid\)\\cos\(\\bm\{e\}^\{\\mathrm\{scale\}\},\\bm\{e\}^\{\\mathrm\{grid\}\}\)per layer; expected≈−0\.66\\approx\-0\.66consistently \(Remark[3\.3](https://arxiv.org/html/2605.20402#S3.Thmtheorem3)\)\.

The main\-body Table[1](https://arxiv.org/html/2605.20402#S2.T1)reports aggregated results across two model scales, and the accompanying main\-body Figure[1](https://arxiv.org/html/2605.20402#S2.F1)visualizes the distribution of pairwise cosine similarities across all weight tensors, confirming:cos⁡\(𝒆scale,𝒆DZ\)=0\\cos\(\\bm\{e\}^\{\\mathrm\{scale\}\},\\bm\{e\}^\{\\mathrm\{DZ\}\}\)=0andcos⁡\(𝒆DZ,𝒆grid\)=0\\cos\(\\bm\{e\}^\{\\mathrm\{DZ\}\},\\bm\{e\}^\{\\mathrm\{grid\}\}\)=0exactly \(Lemma[3\.2](https://arxiv.org/html/2605.20402#S3.Thmtheorem2)\), whilecos⁡\(𝒆scale,𝒆grid\)≈−0\.66\\cos\(\\bm\{e\}^\{\\mathrm\{scale\}\},\\bm\{e\}^\{\\mathrm\{grid\}\}\)\\approx\-0\.66with remarkably tight spread\.

## Appendix GTraining hyperparameters \(MoE\)

We list every hyperparameter explicitly set in our MoE training scripttrain\_moe\_w4a4\_aqn\_rht\_sr\.sh\. All other knobs use the verl\-VeRL defaults\.

### GRPO

- •Advantage estimator:grpo\.
- •rollout\.n\(rollouts per prompt\):22\.
- •train\_batch\_size\(prompts\):6464\.
- •ppo\_mini\_batch\_size\(prompts\):3232\.
- •ppo\_micro\_batch\_size\_per\_gpu:11\(training and inference\)\.
- •use\_dynamic\_bsz:True;ppo\_max\_token\_len\_per\_gpu=2048=2048\.
- •max\_prompt\_length:10241024;max\_response\_length:10241024\.
- •KL:algorithm\.use\_kl\_in\_reward=False,algorithm\.kl\_ctrl\.kl\_coef=0\.0;actor\.use\_kl\_loss=False\.
- •PPO clip \(token\-level, asymmetric\):ϵlow=0\.2\\epsilon\_\{\\rm low\}\{=\}0\.2,ϵhigh=0\.28\\epsilon\_\{\\rm high\}\{=\}0\.28\(DAPO\-style clip\-higher\), dual\-clipc=10\.0c\{=\}10\.0;loss\_type=ppo\_clip\.
- •Token\-level Truncated Importance Sampling \(TIS\)\(Yao et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib40)\):rollout\_is=token,rollout\_is\_threshold=2\.0,tis\_strategy=fixed,bypass\_mode=false\.
- •loss\_agg\_mode=token\-mean;entropy\_coeff=0\.
- •Reward: rule\-based GSM8K grader \(verl built\-in,reward\_model\.enable=Falsesince rewards are computed from the dataset’s ground\-truth field\)\.
- •Optimizer:lr=1e\-6,lr\_warmup\_steps=10,lr\_decay\_style=constant,weight\_decay=0\.1,clip\_grad=1\.0\.
- •trainer\.total\_epochs=1;trainer\.val\_before\_train=True;trainer\.test\_freq=10;trainer\.save\_freq=300\.
- •Single seed per configuration; statistical significance limitation acknowledged in Section[7](https://arxiv.org/html/2605.20402#S7)\.

### Rollout

- •Engine: vLLM \(rollout\.name=vllm\);enforce\_eager=True;free\_cache\_engine=True;enable\_chunked\_prefill=True;max\_num\_batched\_tokens=8192\.
- •tensor\_model\_parallel\_size=8for inference\.
- •gpu\_memory\_utilization=0\.75\.
- •Training rollout:temperature=1\.0,top\_p=1\.0,top\_k=\-1\.
- •Validation rollout \(val\_kwargs\):temperature=1\.0,top\_p=1\.0,top\_k=\-1,do\_sample=True,n=1\.
- •calculate\_log\_probs=True;enable\_rollout\_routing\_replay=True\(MoE router cached from rollout to actor\)\.

### Quantization \(MXFP4 W4A4\)

- •Format: MXFP4 \(E2M1 grid \+ E8M0 power\-of\-two block scale\)\(Rouhani et al\.,[2023](https://arxiv.org/html/2605.20402#bib.bib33)\); W4A4 — both weight and activation tensors entering each matmul are quantized; gradients remain BF16\.
- •Block size:3232along the input axis of each Linear\.
- •Quantizer path: native CUDA kernel \(actor\.force\_py=False\); inference uses the same kernel viarollout\.prequant\_format=mxfp4\.

### Corrections

#### Macro Block Scaling \(MBS\)

\(Chhugani et al\.,[2026](https://arxiv.org/html/2605.20402#bib.bib6)\)\. Macro\-block size128128\(=4=4contiguous MXFP4 sub\-blocks of3232\);88\-bit mantissa correction \(E0M8,256256levels\) per macro\-block; applied identically to weights on both OF passes when MBS\+\+OF is enabled\.

#### Outlier Fallback \(OF\)

\(Zhang et al\.,[2025](https://arxiv.org/html/2605.20402#bib.bib43)\)\. Two\-pass residual MXFP4 QDQ withα=1\.0\\alpha=1\.0:q1=MXFP4​\(𝒘\)q\_\{1\}=\\mathrm\{MXFP4\}\(\\bm\{w\}\),q2=MXFP4​\(𝒘−q1\)q\_\{2\}=\\mathrm\{MXFP4\}\(\\bm\{w\}\-q\_\{1\}\), outputq1\+q2q\_\{1\}\+q\_\{2\}\.

#### Adaptive Quantization Noise \(AQN\)

Huang et al\.\([2026](https://arxiv.org/html/2605.20402#bib.bib22)\)\. Per\-rollout Gaussian additive noise injection on weights with exponentially decaying magnitude:

- •trainer\.aqn\.enabled=True\.
- •sigma\_start=0\.01\.
- •sigma\_end=0\.001\.
- •num\_stages=10\.
- •target\_patterns=\[’layernorm’\]\.
- •target\_multipliers\.post\_attention\_layernorm=1\.414\(i\.e\.2\\sqrt\{2\}, compensating for the magnitude doubling at the residual\-stream branch\)\.

### System / parallelism

- •Model:Qwen3\-30B\-A3B\-Base\.
- •Backend: Megatron\-Core \(via verl\-VeRL,config\-name=ppo\_megatron\_trainer\); customlayers\.py\(the ACS replacement\) provides MXFP4 \+ RHT \+ SR for attention matmuls;te\_grouped\_quant\_patch\.pyprovides MXFP4 for grouped MoE GEMMs\.
- •Parallelism \(training\):TP=1,PP=1,VPP=null,CP=1,EP=8,ETP=1\.
- •Megatron MoE knobs:moe\_grouped\_gemm=True,moe\_token\_dispatcher\_type=flex,moe\_router\_dtype=fp32,moe\_enable\_deepep=True,moe\_permute\_fusion=False\.
- •Memory:param\_offload=True,grad\_offload=True,optimizer\_offload=True\(optimizer\_offload\_fraction=1\.0\);use\_precision\_aware\_optimizer=True; gradient checkpointing enabled\.
- •Checkpointing:use\_mbridge=True,use\_dist\_checkpointing=False\.
- •Hardware:11node×\\times88GPUs \(trainer\.n\_gpus\_per\_node=8,trainer\.nnodes=1\)\.
- •Router replay:router\_replay\.mode=R3\(replay rollout routing decisions during the actor update\)\.
- •LoRA: disabled \(lora\.rank=0; full\-parameter training\)\.

### Data

- •Training: GSM8Ktrain\.parquet\(7,4737\{,\}473problems\)\.
- •Validation: GSM8Ktest\.parquet\(1,3191\{,\}319problems\)\.
- •Preprocessing: parquet viaverl\.data\_preprocess\.gsm8k; prompt keyprompt; truncation modeerror\(longer\-than\-budget prompts abort the job rather than silently truncate\)\.

## NeurIPS Paper Checklist

1. 1\.Claims
2. Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
3. Answer:\[Yes\]
4. Justification: The abstract claims a structural decomposition of MXFP4 error, a dominance analysis of RL damage pathways, and composition of corrections\. All are supported by theoretical results and experiments\.
5. Guidelines: - •The answer\[N/A\]means that the abstract and introduction do not include the claims made in the paper\. - •The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations\. A\[No\]or\[N/A\]answer to this question will not be perceived well by the reviewers\. - •The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings\. - •It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper\.
6. 2\.Limitations
7. Question: Does the paper discuss the limitations of the work performed by the authors?
8. Answer:\[Yes\]
9. Justification: Section 7 includes a dedicated Limitations paragraph discussing: single benchmark \(GSM8K\), QDQ emulation \(not native hardware\), the i\.i\.d\. assumption in Assumption 1, AQN hyperparameter sensitivity, and single\-seed results\.
10. Guidelines: - •The answer\[N/A\]means that the paper has no limitation while the answer\[No\]means that the paper has limitations, but those are not discussed in the paper\. - •The authors are encouraged to create a separate “Limitations” section in their paper\. - •The paper should point out any strong assumptions and how robust the results are to violations of these assumptions \(e\.g\., independence assumptions, noiseless settings, model well\-specification, asymptotic approximations only holding locally\)\. The authors should reflect on how these assumptions might be violated in practice and what the implications would be\. - •The authors should reflect on the scope of the claims made, e\.g\., if the approach was only tested on a few datasets or with a few runs\. In general, empirical results often depend on implicit assumptions, which should be articulated\. - •The authors should reflect on the factors that influence the performance of the approach\. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting\. Or a speech\-to\-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon\. - •The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size\. - •If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness\. - •While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper\. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community\. Reviewers will be specifically instructed to not penalize honesty concerning limitations\.
11. 3\.Theory assumptions and proofs
12. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete \(and correct\) proof?
13. Answer:\[Yes\]
14. Justification: Assumption 1 is stated before Theorem 1\. Lemma 1 has a complete proof in the main text\. Lemma 2 has a proof sketch in the main text and a full proof in Appendix A\. A remark on the practical validity of Assumption 1 is included\. All results are numbered and cross\-referenced\.
15. Guidelines: - •The answer\[N/A\]means that the paper does not include theoretical results\. - •All the theorems, formulas, and proofs in the paper should be numbered and cross\-referenced\. - •All assumptions should be clearly stated or referenced in the statement of any theorems\. - •The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition\. - •Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material\. - •Theorems and Lemmas that the proof relies upon should be properly referenced\.
16. 4\.Experimental result reproducibility
17. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper \(regardless of whether the code and data are provided or not\)?
18. Answer:\[Yes\]
19. Justification: Section 6\.1 specifies models \(Qwen2\.5\-3B, Qwen3\-30B\-A3B\), task \(GSM8K\), training algorithm \(GRPO\), batch size, sampling parameters, quantization format \(MXFP4 W4A4 with block size 32\), TIS thresholds, and all correction hyperparameters \(MBS macro\-block size, AQN sigma schedule\)\.
20. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •If the paper includes experiments, a\[No\]answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not\. - •If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable\. - •Depending on the contribution, reproducibility can be accomplished in various ways\. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model\. In general\. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model \(e\.g\., in the case of a large language model\), releasing of a model checkpoint, or other means that are appropriate to the research performed\. - •While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution\. For example 1. \(a\)If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm\. 2. \(b\)If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully\. 3. \(c\)If the contribution is a new model \(e\.g\., a large language model\), then there should either be a way to access this model for reproducing the results or a way to reproduce the model \(e\.g\., with an open\-source dataset or instructions for how to construct the dataset\)\. 4. \(d\)We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility\. In the case of closed\-source models, it may be that access to the model is limited in some way \(e\.g\., to registered users\), but it should be possible for other researchers to have some path to reproducing or verifying the results\.
21. 5\.Open access to data and code
22. Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
23. Answer:\[No\]
24. Justification: Code will be released upon acceptance\. GSM8K is publicly available\. The QDQ emulation and correction implementations are described in sufficient detail \(Section 5\) for independent reimplementation\.
25. Guidelines: - •The answer\[N/A\]means that paper does not include experiments requiring code\. - • - •While we encourage the release of code and data, we understand that this might not be possible, so\[No\]is an acceptable answer\. Papers cannot be rejected simply for not including code, unless this is central to the contribution \(e\.g\., for a new open\-source benchmark\)\. - •The instructions should contain the exact command and environment needed to run to reproduce the results\. See the NeurIPS code and data submission guidelines \([https://neurips\.cc/public/guides/CodeSubmissionPolicy](https://neurips.cc/public/guides/CodeSubmissionPolicy)\) for more details\. - •The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc\. - •The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines\. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why\. - •At submission time, to preserve anonymity, the authors should release anonymized versions \(if applicable\)\. - •Providing as much information as possible in supplemental material \(appended to the paper\) is recommended, but including URLs to data and code is permitted\.
26. 6\.Experimental setting/details
27. Question: Does the paper specify all the training and test details \(e\.g\., data splits, hyperparameters, how they were chosen, type of optimizer\) necessary to understand the results?
28. Answer:\[Yes\]
29. Justification: Section 6\.1 provides model architectures, hardware, training algorithm \(GRPO\), batch size, number of samples per prompt, max response length, quantization details, and TIS thresholds\. Sections 5\.1–5\.3 detail all correction hyperparameters\.
30. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them\. - •The full details can be provided either with the code, in appendix, or as supplemental material\.
31. 7\.Experiment statistical significance
32. Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
33. Answer:\[No\]
34. Justification: Most configurations are single\-seed due to computational cost \(each MoE run requires 8 GPUs for∼\{\\sim\}5 hours\)\. This is acknowledged as a limitation in Section 7\. We report peak accuracy and per\-step timing for all runs\.
35. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The authors should answer\[Yes\]if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper\. - •The factors of variability that the error bars are capturing should be clearly stated \(for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions\)\. - •The method for calculating the error bars should be explained \(closed form formula, call to a library function, bootstrap, etc\.\) - •The assumptions made should be given \(e\.g\., Normally distributed errors\)\. - •It should be clear whether the error bar is the standard deviation or the standard error of the mean\. - •It is OK to report 1\-sigma error bars, but one should state it\. The authors should preferably report a 2\-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified\. - •For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range \(e\.g\., negative error rates\)\. - •If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text\.
36. 8\.Experiments compute resources
37. Question: For each experiment, does the paper provide sufficient information on the computer resources \(type of compute workers, memory, time of execution\) needed to reproduce the experiments?
38. Answer:\[Yes\]
39. Justification: Section 6\.1 specifies GPU count \(2 for dense, 8 for MoE\) and parallelism strategy \(FSDP2 and Megatron\)\. Tables 3–4 report per\-step wall\-clock time for every configuration\.
40. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage\. - •The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute\. - •The paper should disclose whether the full research project required more compute than the experiments reported in the paper \(e\.g\., preliminary or failed experiments that didn’t make it into the paper\)\.
41. 9\.Code of ethics
43. Answer:\[Yes\]
44. Justification: We have reviewed the NeurIPS Code of Ethics\. Our work analyzes quantization error in LLM training and does not raise ethical concerns beyond those inherent to LLM research generally\.
45. Guidelines: - •The answer\[N/A\]means that the authors have not reviewed the NeurIPS Code of Ethics\. - •If the authors answer\[No\], they should explain the special circumstances that require a deviation from the Code of Ethics\. - •The authors should make sure to preserve anonymity \(e\.g\., if there is a special consideration due to laws or regulations in their jurisdiction\)\.
46. 10\.Broader impacts
47. Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
48. Answer:\[Yes\]
49. Justification: Reducing the compute cost of RL post\-training broadens access to LLM alignment techniques\. We do not identify direct negative societal impacts: our work improves training efficiency of existing models rather than introducing new capabilities\.
50. Guidelines: - •The answer\[N/A\]means that there is no societal impact of the work performed\. - •If the authors answer\[N/A\]or\[No\], they should explain why their work has no societal impact or why the paper does not address societal impact\. - •Examples of negative societal impacts include potential malicious or unintended uses \(e\.g\., disinformation, generating fake profiles, surveillance\), fairness considerations \(e\.g\., deployment of technologies that could make decisions that unfairly impact specific groups\), privacy considerations, and security considerations\. - •The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments\. However, if there is a direct path to any negative applications, the authors should point it out\. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation\. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster\. - •The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from \(intentional or unintentional\) misuse of the technology\. - •If there are negative societal impacts, the authors could also discuss possible mitigation strategies \(e\.g\., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML\)\.
51. 11\.Safeguards
52. Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse \(e\.g\., pre\-trained language models, image generators, or scraped datasets\)?
53. Answer:\[N/A\]
54. Justification: We do not release pre\-trained models or datasets\. Our contribution is a theoretical analysis and training methodology\.
55. Guidelines: - •The answer\[N/A\]means that the paper poses no such risks\. - •Released models that have a high risk for misuse or dual\-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters\. - •Datasets that have been scraped from the Internet could pose safety risks\. The authors should describe how they avoided releasing unsafe images\. - •We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort\.
56. 12\.Licenses for existing assets
57. Question: Are the creators or original owners of assets \(e\.g\., code, data, models\), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
58. Answer:\[Yes\]
59. Justification: Qwen models \(Apache 2\.0\) and GSM8K \(MIT license\) are cited\. MBS, OF, and AQN are attributed to their original papers with citations\.
60. Guidelines: - •The answer\[N/A\]means that the paper does not use existing assets\. - •The authors should cite the original paper that produced the code package or dataset\. - •The authors should state which version of the asset is used and, if possible, include a URL\. - •The name of the license \(e\.g\., CC\-BY 4\.0\) should be included for each asset\. - •For scraped data from a particular source \(e\.g\., website\), the copyright and terms of service of that source should be provided\. - •If assets are released, the license, copyright information, and terms of use in the package should be provided\. For popular datasets,[paperswithcode\.com/datasets](https://arxiv.org/html/2605.20402v1/paperswithcode.com/datasets)has curated licenses for some datasets\. Their licensing guide can help determine the license of a dataset\. - •For existing datasets that are re\-packaged, both the original license and the license of the derived asset \(if it has changed\) should be provided\. - •If this information is not available online, the authors are encouraged to reach out to the asset’s creators\.
61. 13\.New assets
62. Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
63. Answer:\[N/A\]
64. Justification: No new datasets or models are released in this paper\.
65. Guidelines: - •The answer\[N/A\]means that the paper does not release new assets\. - •Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates\. This includes details about training, license, limitations, etc\. - •The paper should discuss whether and how consent was obtained from people whose asset is used\. - •At submission time, remember to anonymize your assets \(if applicable\)\. You can either create an anonymized URL or include an anonymized zip file\.
66. 14\.Crowdsourcing and research with human subjects
67. Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation \(if any\)?
68. Answer:\[N/A\]
69. Justification: This work does not involve crowdsourcing or human subjects\.
70. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper\. - •According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector\.
71. 15\.Institutional review board \(IRB\) approvals or equivalent for research with human subjects
72. Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board \(IRB\) approvals \(or an equivalent approval/review based on the requirements of your country or institution\) were obtained?
73. Answer:\[N/A\]
74. Justification: This work does not involve human subjects research\.
75. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Depending on the country in which research is conducted, IRB approval \(or equivalent\) may be required for any human subjects research\. If you obtained IRB approval, you should clearly state this in the paper\. - •We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution\. - •For initial submissions, do not include any information that would break anonymity \(if applicable\), such as the institution conducting the review\.
76. 16\.Declaration of LLM usage
77. Question: Does the paper describe the usage of LLMs if it is an important, original, or non\-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does*not*impact the core methodology, scientific rigor, or originality of the research, declaration is not required\.
78. Answer:\[N/A\]
79. Justification: LLMs are the subject of study \(we train them with RL\), not a methodological component for conducting the research\. No LLMs were used to generate experimental results or derive theoretical claims\.
80. Guidelines: - •The answer\[N/A\]means that the core method development in this research does not involve LLMs as any important, original, or non\-standard components\. - •Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described\.