[Paper] Statistically-Lossless Quantization of Large Language Models
Summary
This paper presents a statistically-lossless quantization method for large language models, aiming to reduce model size without information loss.
Similar Articles
Structured Transforms for Low-Overhead Quantization of Language Models
This paper revisits Kashin-decomposition-based weight quantization for large language models and proposes an improved algorithm using structured orthogonal transforms, reducing computational cost and ensuring numerical stability compared to methods like OPTQ and QuIP.
Quantization-Robust Unlearning through the Lens of Retain-Forget Loss Landscapes Interaction
This paper proposes a quantization-robust unlearning framework for large language models, using loss landscape analysis to ensure effective forgetting while maintaining model utility after compression.
Jacobian-guided Noise Injection for Quantization Robustness in Large Language Models
This paper proposes Jacobian-guided noise injection, a training strategy that improves quantization robustness in large language models by injecting noise into pre-attention logits with variance derived from the Jacobian norm, leading to significant performance gains in low-bit quantization settings.
MixQuant: Adaptive Mixed-Precision Quantization for Large Language Models
MixQuant proposes an adaptive mixed-precision quantization framework for LLMs that handles variable memory budgets by marginalizing layer distortion over random upstream configurations, outperforming existing methods across multiple models and budgets.
Compressing What Matters: Neuron Importance Meets Data-Aware Low Rank Approximation for Language Model Compression
This paper proposes a method for compressing large language models by combining neuron importance and data-aware low rank approximation, along with an efficient dynamic compression rate allocation algorithm, achieving performance on par with or better than previous state-of-the-art.