For those with 12GB GPUs, you can now run QWEN 3.6 27B wth little loss via the new Ternary version.

Reddit r/ArtificialInteligence Models
quantization ternary local-llm gpu open-source qwen bonsai

Summary

A new ternary quantized version of Qwen3.6 27B, called Bonsai 27B, allows running the model on 12GB GPUs with 10x less memory and 95% of original performance, making it accessible for local deployment.

The new Bonsai 27B Model from PrismML is Qwen3.6 27B, a beloved workhorse for many, updated into something you can run on local computer. 10x less memory and a much more modest file size while still benchmarking 95% of the original FP16 model. Of course, no model is perfect, but if you have been wanting to run Qwen3.6 27B and dont have the computer or headroom, you finally can. The Ternary format is much smarter, but takes a bit more compute. The Binary model is enough to fit into a phone form factor. Imagine 27B, even remotely, in your pocket. That is now a reality. Not my video, but here is a setup tutorial
Original Article

Similar Articles

Prism-ML Bonsai Qwen 3.6 27B

Reddit r/LocalLLaMA

Prism ML released Ternary-Bonsai-27B, a ternary-quantized version of Qwen3.6-27B that retains 95% of FP16 intelligence at a ~7.2 GB footprint, enabling full 27B-class reasoning on laptops and single GPUs with speeds up to 26 tok/s on Apple M5 Pro.

Running Qwen3.6 35b a3b on 8gb vram and 32gb ram ~190k context

Reddit r/LocalLLaMA

The author shares a high-performance local inference configuration for running Qwen3.6 35B A3B on limited hardware (8GB VRAM, 32GB RAM) using a modified llama.cpp with TurboQuant support, achieving ~37-51 tok/sec with ~190k context.

Ternary Qwen3.6 27B Tested on 3090!

Reddit r/LocalLLaMA

User tests ternary quantized Qwen3.6 27B on an RTX 3090, achieving 60 tk/s with two slots and 100k KV cache using 21GB VRAM, with good quality and stable tool calls.

Qwen 3.6 27B is a BEAST

Reddit r/LocalLLaMA

A developer reports that the new 27B Qwen 3.6 model runs excellently on a 24GB VRAM laptop, passing all PySpark/Python data-transformation benchmarks and eliminating the need for cloud subscriptions.