few-bit-integers

This paper proposes signed symmetric quantization for few-bit integers, which assigns the extra representable value to the dominant outlier tail, improving quantization error and perplexity in LLMs with no extra inference cost compared to standard symmetric quantization.