Is ternary (1.58-bit) LLMs making a come back?

Reddit r/LocalLLaMA 08/16/26, 09:36 AM News

ternary-llm ai-models efficiency small-labs coding-performance moe-architecture open-source

Summary

Recent ternary 1.58-bit LLM releases from small labs demonstrate speed and medical specialization but struggle with long-horizon tasks, with optimism for future models to compete with larger architectures like Qwen.

I'm just thinking, ever since microsoft announced bitnet, this sub (and myself) has been hoping for massive ternary models. In the last month alone, prismML dropped 27B ternary (though I've read community experience suggested it sometimes didn't hold up to it's benchmarks), Deepgrove dropped their ternary maple-20b-a1b which from my experience works really well and clocks like 100 tok/s on an iphone, and Doses AI dropped pestle-27b-ternary medical specialised which beats medgemma-27b nearly across the board. The common problem across all of them is long-horizon agentic coding/work, but i really think that's because all of these are new small labs that haven't prioritised RL-maxxing yet - they have indicated this is their next step though. I'm hopeful, and it seems like we could be very close to a massive ternary model MoE that's actually competitive with qwen3.8 at coding and agentic work. Or have most folks lost faith in ternary architecture?

Original Article

Similar Articles

ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level

arXiv cs.LG

ExTernD introduces an expanded-rank ternary decomposition for post-training LLM quantization, enabling accuracy approaching bf16 by using a factored representation with free inner rank. It matches Q4_K accuracy at 5.2-5.5 effective bits per weight on models like Gemma-4 and Qwen3.5.

Open Source Ternary LLM Engine in Rust/CUDA for Quantization, Serving, and Training of models on consumer GPUs, called Tritium (Apache 2.0)

Reddit r/LocalLLaMA

Introduces Tritium, an open-source Rust/CUDA engine for ternary (1.58-bit) quantization, serving, and training of LLMs on consumer GPUs. It claims faster inference than llama.cpp for ternary models and introduces a new quantization method called SALT.

CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs

arXiv cs.CL

CAT-Q introduces a post-training ternary quantization method for LLMs that uses learnable modulation and softened ternarization, achieving superior performance over BitNet 1.58-bit while using only 512 calibration samples and scaling to 235B parameters.

prism-ml/Ternary-Bonsai-27B-mlx-2bit

Hugging Face Models Trending

Prism ML releases Ternary-Bonsai-27B-mlx-2bit, a ternary-quantized 27B-parameter language model that achieves ~95% of FP16 performance while fitting in ~7.2 GB, enabling full reasoning on laptops.

How to pack ternary numbers in 8-bit bytes

Hacker News Top

A blog post describing an efficient method to pack ternary numbers into 8-bit bytes using SIMD-friendly unpacking, achieving 1.6 bits per trit, with applications in LLM weight quantization like BitNet b1.58.