ternary

A new ternary quantized version of Qwen3.6 27B, called Bonsai 27B, allows running the model on 12GB GPUs with 10x less memory and 95% of original performance, making it accessible for local deployment.

User tests ternary quantized Qwen3.6 27B on an RTX 3090, achieving 60 tk/s with two slots and 100k KV cache using 21GB VRAM, with good quality and stable tool calls.

Ternary Bonsai 27B, a large language model, is demonstrated running locally on an NVIDIA RTX 5090 GPU, requiring under 6GB of memory and enabling end-to-end agentic workflows on consumer hardware.

BitCPM is a new open-source model from ModelBest, Tsinghua, and OpenBMB that uses ternary weights (-1,0,1) to run full-sized AI models on phones.

PrismML releases Bonsai 27B, a vision-language model with agentic tool calling and long context, along with 1-bit and ternary variants. The demo repository allows running these models locally on various hardware.