Tag
A modified GGUF version of Bonsai-27b with refusals removed has been released on Hugging Face.
PrismML's Bonsai-27B model achieves 99.2% on the Math 500 dataset, outperforming DeepSeek-R1 and Kimi K2.
Updates on Bonsai-27B and Ternary-Bonsai-27B models, detailing upstream merge status in llama.cpp across CPU, Metal, CUDA, Vulkan backends, and discussing model limitations and roadmap.
@MaziyarPanahi runs GLM-5.2 and Bonsai 27B locally on a Mac Studio using llama.cpp to process a 3-year patient chart, catching a dangerous drug interaction that was previously flagged but overlooked. The models operate entirely on-device under Apache-2.0, with Bonsai answering queries in ~2s and @PrismML claiming a 1-bit build fits an iPhone.
User tests PrismML's new Bonsai 27B model on Nvidia DGX Spark, reporting benchmark speeds and issues with llama.cpp build, while PrismML announces the model as the first 27B-class model to run on a phone.
PrismML's Bonsai 27B model runs on the Jetson Orin Nano 8GB with 4.31 tokens/s and 27 t/s prompt processing, using 6.2GB RAM and about 25W power. It indicates surprisingly usable edge AI performance.
Bonsai 27B achieves 93% size reduction via 1-bit quantization while retaining 90% intelligence, enabling local browser inference with custom WebGPU kernels.