I ran Ternary-Bonsai-27B (2-bit) and Bonsai-27B (1-bit) on Terminal-Bench 2.0, in 8GB VRAM
Summary
A user tested quantized 1-bit and 2-bit versions of the 27B-parameter Bonsai model on Terminal-Bench 2.0, achieving results within 8GB VRAM.
Similar Articles
Ternary Bonsai 2 (27B) just released on Hugging Face. At <6GB in size, it can even run locally in-browser on WebGPU.
Ternary Bonsai 2 is a 27B parameter model derived from Qwen3.8-27B that uses ternary weights to achieve a size under 6GB while retaining 98.2% of its intelligence, enabling it to run in-browser on WebGPU.
@sudoingX: every day someone asks how i'm running bonsai 27b on hardware that shouldn't handle it. so here's the whole thing in on…
A detailed guide on running the 27B Bonsai model on hardware with only 8GB VRAM using a 1-bit quantized version and the PrismML fork of llama.cpp, including exact server commands and configuration.
Using the Bonsai 27b 1b quant locally - regularly.
A user shares their positive experience using the 1-bit quantized version of Bonsai 27b locally on a 16GB MacBook Air for casual chat, tutoring in Go, and analyzing personal notes, praising its intelligence and small footprint.
Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone
PrismML announces Bonsai 27B, a 1-bit and ternary quantized version of Qwen3.6 27B that runs on phones and laptops, retaining 90-95% of baseline performance with a 3.9GB footprint, enabling agentic and multimodal on-device AI.
@HuggingModels: Meet Ternary-Bonsai-2-27B: a 27B parameter model squeezed into 2-bit ternary format. Runs on llama.cpp with CUDA and Me…
Ternary-Bonsai-2-27B is a 27B parameter AI model compressed into 2-bit ternary format, running on llama.cpp with CUDA and Metal support, facilitating lightweight on-device AI deployment.