Tequila: Trapping-free Ternary Quantization for Large Language Models

Papers with Code Trending Papers
quantization large-language-models ternary-quantization edge-deployment optimization open-source

Summary

This paper introduces Tequila, a trapping-free quantization method for Large Language Models that improves ternary quantization accuracy and inference speed by repurposing deadzone-trapped weights as dynamic biases.

Quantization techniques are essential for the deployment of Large Language Models (LLMs) on edge devices. However, prevailing methods often rely on mixed-precision multiplication that lacks efficient hardware support, making it not feasible. Ternary weight quantization addresses this by constraining weights to {-1, 0, 1}, replacing expensive multiplications with hardware-efficient additions. However, such aggressive compression leads to significant accuracy degradation, even after costly quantization-aware training with massive data. We identify the core issue as deadzone trapping: a large number of weights are trapped at the deadzone boundary. This occurs because these weights receive only noisy, uninformative gradients, preventing stable escape from the deadzone and severely impeding model capacity and optimization. To address this issue, we propose Tequila, a trapping-free quantization optimization method that reactivates deadzone-trapped weights by repurposing them as dynamic biases. This allows the repurposed weights to provide a continuous signal in the forward pass and, critically, receive direct, meaningful gradient signals during backpropagation, thereby enhancing model capacity and optimization with nearly zero inference overhead. Extensive evaluations demonstrate that Tequila outperforms state-of-the-art (SOTA) ternary quantization methods across five benchmarks. Specifically, on the ARC benchmark, it achieves >4% accuracy gain over the SOTA baseline, nearly matching full-precision performance (within <1% gap) with a 3.0x inference speedup. Consequently, Tequila offers a highly practical and efficient implementation for the deployment of advanced LLMs in resource-constrained environments. The code is available at https://github.com/Tencent/AngelSlim.
Original Article
View Cached Full Text

Cached at: 05/08/26, 08:55 AM

Paper page - Tequila: Trapping-free Ternary Quantization for Large Language Models

Source: https://huggingface.co/papers/2509.23809 Published on Sep 28, 2025

Abstract

Tequila, a trapping-free quantization method, enhances ternary quantization of Large Language Models by repurposing deadzone-trapped weights as dynamic biases, improving accuracy and inference speed.

Quantization techniquesare essential for the deployment of Large Language Models (LLMs) on edge devices. However, prevailing methods often rely onmixed-precision multiplicationthat lacks efficient hardware support, making it not feasible.Ternary weight quantizationaddresses this by constraining weights to {-1, 0, 1}, replacing expensive multiplications with hardware-efficient additions. However, such aggressive compression leads to significant accuracy degradation, even after costlyquantization-aware trainingwith massive data. We identify the core issue asdeadzone trapping: a large number of weights are trapped at the deadzone boundary. This occurs because these weights receive only noisy, uninformative gradients, preventing stable escape from the deadzone and severely impeding model capacity and optimization. To address this issue, we propose Tequila, a trapping-free quantization optimization method that reactivates deadzone-trapped weights by repurposing them asdynamic biases. This allows the repurposed weights to provide a continuous signal in the forward pass and, critically, receive direct, meaningful gradient signals duringbackpropagation, thereby enhancing model capacity and optimization with nearly zeroinference overhead. Extensive evaluations demonstrate that Tequila outperforms state-of-the-art (SOTA) ternary quantization methods across five benchmarks. Specifically, on the ARC benchmark, it achieves >4% accuracy gain over the SOTA baseline, nearly matchingfull-precision performance(within <1% gap) with a 3.0x inference speedup. Consequently, Tequila offers a highly practical and efficient implementation for the deployment of advanced LLMs in resource-constrained environments. The code is available at https://github.com/Tencent/AngelSlim.

View arXiv pageView PDFGitHub1.03kautoAdd to collection

Get this paper in your agent:

hf papers read 2509\.23809

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper31

#### AngelSlim/Qwen3-a3B_eagle3 #### AngelSlim/Qwen3-32B_eagle3 UpdatedJan 13 • 2.19k • 7 #### AngelSlim/Qwen3-1.7B_eagle3 UpdatedJan 13 • 44.5k • 5 #### AngelSlim/Fun-CosyVoice3-0.5B-2512_eagle3 24.4M• UpdatedJan 13 • 10 • 5 Browse 31 models citing this paper## Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2509.23809 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2509.23809 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs

arXiv cs.CL

CAT-Q introduces a post-training ternary quantization method for LLMs that uses learnable modulation and softened ternarization, achieving superior performance over BitNet 1.58-bit while using only 512 calibration samples and scaling to 235B parameters.

Theory-optimal Quantization Based on Flatness

arXiv cs.LG

Introduces Flatness metric and Bidirectional Diagonal Quantization (BDQ) for post-training quantization of large language models, achieving near-lossless 4-bit weight and activation quantization and substantial improvements at extreme low-bit settings.