GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
Summary
Proposes GEMQ, a global expert-level mixed-precision quantization method for MoE LLMs that uses linear programming and router fine-tuning to reduce memory and accelerate inference with minimal accuracy degradation.
View Cached Full Text
Cached at: 05/25/26, 08:59 AM
# GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs Source: [https://arxiv.org/abs/2605.23078](https://arxiv.org/abs/2605.23078) [View PDF](https://arxiv.org/pdf/2605.23078) > Abstract:Mixture\-of\-Experts Large Language Models \(MoE\-LLMs\) achieve strong performance but incur substantial memory overhead due to massive expert parameters\. Mixed\-precision quantization mitigates this cost by allocating expert\-wise bit\-widths based on their importance, approaching the accuracy\-memory Pareto frontier and enabling extreme low\-bit quantization\. However, existing methods rely on layer\-wise importance estimation and overlook router shifts induced by quantization, resulting in suboptimal allocation and routing\. In this work, we propose Global Expert\-level Mixed\-precision Quantization \(GEMQ\) to overcome these limitations via \(1\) a global linear\-programming formulation that captures model\-wide expert importance based on quantization error analysis, and \(2\) efficient router fine\-tuning to adapt routing to quantized experts\. These components are integrated into a progressive quantization framework that iteratively refines importance estimation and allocation\. Experiments demonstrate that GEMQ significantly reduces memory and accelerates inference with minimal accuracy degradation\. Source code is available at[this https URL](https://github.com/jndeng/GEMQ)\. ## Submission history From: Jianing Deng \[[view email](https://arxiv.org/show-email/49ac9fb1/2605.23078)\] **\[v1\]**Thu, 21 May 2026 22:23:38 UTC \(653 KB\)
Similar Articles
MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs
This paper introduces MODE, a modality-decomposed expert-level mixed-precision quantization framework for MoE multimodal LLMs that addresses biases in expert importance estimation by decomposing selection frequency by modality and filtering redundant vision tokens, achieving minimal performance loss under aggressive quantization.
BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization
BitsMoE introduces a spectral-energy-guided bit allocation framework for quantizing Mixture-of-Experts LLMs, achieving substantial accuracy improvements and speedups under ultra-low-bit quantization.
MixQuant: Adaptive Mixed-Precision Quantization for Large Language Models
MixQuant proposes an adaptive mixed-precision quantization framework for LLMs that handles variable memory budgets by marginalizing layer distortion over random upstream configurations, outperforming existing methods across multiple models and budgets.
Colla-Q: Toward Collaborative Experts in MoE Quantization via Minimax Precision Balancing
This paper presents Colla-Q, a bit-allocation framework for quantizing Mixture-of-Experts models using activation entropy to balance expert performance, improving overall model efficiency and reducing calibration dependence.
GEM: Geometric Entropy Mixing for Optimal LLM Data Curation
GEM reformulates LLM data curation as a variational problem on the hypersphere, using geometric entropy mixing and a minorize-maximize algorithm to discover balanced semantic clusters, achieving state-of-the-art improvements in data mixing strategies by up to 1.2% average downstream accuracy.