Prism-ML's Bonsai-27B Benchmarks
Summary
Prism-ML published benchmarks for their Bonsai-27B model.
Similar Articles
prism-ml/Bonsai-27B-gguf
Prism ML releases Bonsai-27B-gguf, a 27-billion parameter language model with binary (1.125-bit) weights, achieving a ~14x size reduction while retaining ~90% of FP16 reasoning performance. It runs on consumer hardware with high throughput.
I benchmarked PrismML's 1-bit Bonsai-8B against IBM's Granite on CPU tool calling. The 1-bit model won, but only with grammar-constrained decoding
An independent benchmark of PrismML's 1-bit Bonsai-8B against IBM's Granite and other models on CPU tool calling shows that with grammar-constrained decoding, Bonsai-8B achieves a 92% pass rate, outperforming larger models, but fails without constraints. Granite is the best raw model at 72%.
Prism-ML Bonsai Qwen 3.6 27B
Prism ML released Ternary-Bonsai-27B, a ternary-quantized version of Qwen3.6-27B that retains 95% of FP16 intelligence at a ~7.2 GB footprint, enabling full 27B-class reasoning on laptops and single GPUs with speeds up to 26 tok/s on Apple M5 Pro.
@AgentSparko: I tested the PrismML Bonsai 27B on the DGX Spark but I think I messed something up with llama.cpp build because the spe…
User tests PrismML's new Bonsai 27B model on Nvidia DGX Spark, reporting benchmark speeds and issues with llama.cpp build, while PrismML announces the model as the first 27B-class model to run on a phone.
PrismML Bonsai 27B is surprisingly usable on the Jetson Orin Nano 8GB
PrismML's Bonsai 27B model runs on the Jetson Orin Nano 8GB with 4.31 tokens/s and 27 t/s prompt processing, using 6.2GB RAM and about 25W power. It indicates surprisingly usable edge AI performance.