I tested Qwen3.8 27B IQ3_XXS (10.18GiB) vs Bonsai Ternary PQ2 (6.42GiB)

Reddit r/LocalLLaMA News
ai-models quantization benchmarking performance vram inference-speed llama-cpp

Summary

The author compared the performance of Qwen3.8 27B IQ3_XXS and Bonsai Ternary PQ2 on limited VRAM, finding that Qwen is faster and uses fewer tokens, while Bonsai has a smaller file size but longer generation times.

I did a small test of the new hyped quantisation of Qwen3.8 vs the biggest quant which fits into my limited 16GB VRAM with decent context. The results are interesting. Of course, the smaller file gives worse results. However they are not that far off. Unfortunately, this comes at the expense of even more tokens beeing used by the Bonsai model and thus much longer generation times. Visually I prefer the IQ3_XXS results, but see for yourself. The test is by no means scientific - just few UI generation tasks for direct comparison on the same hardware. Also, I ran llama.cpp with MTP while the Bonsai model doesn't seem to have MTP which makes it even slower. Metric Qwen IQ3 Bonsai PQ2 Tasks completed 4/4 4/4 Fixed assertions 20/20 20/20 Agent wall time 8:00 24:09 Output tokens 27,197 84,176 Weighted decode 83.59 tok/s 64.91 tok/s Speculative acceptance 65.22% MTP 39.76% modified N-gram Compactions 0 0 Length stops 0 1 Across the complete suite, Qwen finished 3.02× faster and used 3.10× fewer output tokens. Results: https://danmoreng.github.io/qwen3-8-27b-iq3-xxs-vs-bonsai/ Repo: https://github.com/Danmoreng/qwen3-8-27b-iq3-xxs-vs-bonsai
Original Article

Similar Articles

Prism-ML Bonsai Qwen 3.6 27B

Reddit r/LocalLLaMA

Prism ML released Ternary-Bonsai-27B, a ternary-quantized version of Qwen3.6-27B that retains 95% of FP16 intelligence at a ~7.2 GB footprint, enabling full 27B-class reasoning on laptops and single GPUs with speeds up to 26 tok/s on Apple M5 Pro.

My Qwen 3.8 27B tests on limited VRAM (16-20GB)

Reddit r/LocalLLaMA

This article tests various quantized versions of the Qwen 3.8 27B AI model on limited VRAM setups, comparing their performance on tasks like animation generation, app development, and word generation.

RTX 5090 Bonsai 2 27B vs Gemma 4 12B vs Qwen 3.5 9B Japanese voxel pagoda

Reddit r/LocalLLaMA

This article compares the performance of Bonsai 2 27B, Gemma 4 12B, and Qwen 3.5 9B models on generating a Japanese voxel pagoda using an RTX 5090, concluding that Bonsai offers superior intelligence and detail for its memory footprint, benefiting the local AI community.