[Paper] Statistically-Lossless Quantization of Large Language Models
Summary
This paper presents a statistically-lossless quantization method for large language models, aiming to reduce model size without information loss.
Similar Articles
Compressing What Matters: Neuron Importance Meets Data-Aware Low Rank Approximation for Language Model Compression
This paper proposes a method for compressing large language models by combining neuron importance and data-aware low rank approximation, along with an efficient dynamic compression rate allocation algorithm, achieving performance on par with or better than previous state-of-the-art.
How Quantization Changes Interpretable Features: A Sparse Autoencoder Analysis of Language Models
This paper investigates whether interpretable features identified by sparse autoencoders in full-precision language models remain faithful after quantization, finding systematic degradation that behavioral metrics like perplexity can miss.
Tequila: Trapping-free Ternary Quantization for Large Language Models
This paper introduces Tequila, a trapping-free quantization method for Large Language Models that improves ternary quantization accuracy and inference speed by repurposing deadzone-trapped weights as dynamic biases.
Theory-optimal Quantization Based on Flatness
Introduces Flatness metric and Bidirectional Diagonal Quantization (BDQ) for post-training quantization of large language models, achieving near-lossless 4-bit weight and activation quantization and substantial improvements at extreme low-bit settings.
Fitting Is Not Enough: Smoothness in Extremely Quantized LLMs
This paper investigates smoothness degradation in extremely quantized Large Language Models, arguing that preserving smoothness is crucial for maintaining performance beyond numerical accuracy.