Tag
Introduces Tritium, an open-source Rust/CUDA engine for ternary (1.58-bit) quantization, serving, and training of LLMs on consumer GPUs. It claims faster inference than llama.cpp for ternary models and introduces a new quantization method called SALT.
Prism ML released Ternary-Bonsai-27B, a ternary-quantized version of Qwen3.6-27B that retains 95% of FP16 intelligence at a ~7.2 GB footprint, enabling full 27B-class reasoning on laptops and single GPUs with speeds up to 26 tok/s on Apple M5 Pro.
Clark Labs released Clark Air Sana 1.6B, a ternary-quantized version of the Sana 1.6B text-to-image transformer that is 8.6× smaller than FP16 while maintaining near-FP16 quality, enabling efficient deployment.
CAT-Q introduces a post-training ternary quantization method for LLMs that uses learnable modulation and softened ternarization, achieving superior performance over BitNet 1.58-bit while using only 512 calibration samples and scaling to 235B parameters.
OpenBMB releases BitCPM4-CANN, a collection of natively trained 1.58-bit ternary quantized LLMs (0.5B to 8B) optimized for Ascend NPUs via CANN, achieving 6× memory reduction at inference and minimal training overhead.
This paper introduces Tequila, a trapping-free quantization method for Large Language Models that improves ternary quantization accuracy and inference speed by repurposing deadzone-trapped weights as dynamic biases.