weight-compression

QTEA is a sub-2-bit post-training quantization framework for large language models that achieves ternary weight compression with improved accuracy and hardware efficiency through sparse residual salient weights and column-wise optimization.

This paper introduces Qift, a fixed no-zero two-bit weight quantization level set designed for Hadamard-rotated LLMs, achieving improved W2A4/KV4 inference by leveraging the near-zero-centered Gaussian-like distribution of rotated weights. Experiments on LLaMA-2-7B and LLaMA-3.1-8B show consistent perplexity gains over standard W2 quantization.