PrismML Bonsai 27B is surprisingly usable on the Jetson Orin Nano 8GB
Summary
PrismML's Bonsai 27B model runs on the Jetson Orin Nano 8GB with 4.31 tokens/s and 27 t/s prompt processing, using 6.2GB RAM and about 25W power. It indicates surprisingly usable edge AI performance.
Similar Articles
prism-ml/Bonsai-27B-gguf
Prism ML releases Bonsai-27B-gguf, a 27-billion parameter language model with binary (1.125-bit) weights, achieving a ~14x size reduction while retaining ~90% of FP16 reasoning performance. It runs on consumer hardware with high throughput.
Prism-ML Bonsai Qwen 3.6 27B
Prism ML released Ternary-Bonsai-27B, a ternary-quantized version of Qwen3.6-27B that retains 95% of FP16 intelligence at a ~7.2 GB footprint, enabling full 27B-class reasoning on laptops and single GPUs with speeds up to 26 tok/s on Apple M5 Pro.
Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone
PrismML announces Bonsai 27B, a 1-bit and ternary quantized version of Qwen3.6 27B that runs on phones and laptops, retaining 90-95% of baseline performance with a 3.9GB footprint, enabling agentic and multimodal on-device AI.
@AgentSparko: I tested the PrismML Bonsai 27B on the DGX Spark but I think I messed something up with llama.cpp build because the spe…
User tests PrismML's new Bonsai 27B model on Nvidia DGX Spark, reporting benchmark speeds and issues with llama.cpp build, while PrismML announces the model as the first 27B-class model to run on a phone.
@PrismML: Today, we’re announcing Bonsai 27B: the first 27B-class model to run on a phone. Bonsai 27B is the new multimodal flags…
PrismML announces Bonsai 27B, the first 27-billion parameter model able to run on a phone, with ternary and 1-bit variants (5.9 GB and 3.9 GB respectively) that enable multi-step reasoning and agentic workflows on local devices, all open-sourced under Apache 2.0.