Tag
Discussion on the feasibility of 1-bit models like Bonsai 8b and 27b, which achieve small file sizes while remaining functional, questioning their primary use cases and future.
An independent benchmark of PrismML's 1-bit Bonsai-8B against IBM's Granite and other models on CPU tool calling shows that with grammar-constrained decoding, Bonsai-8B achieves a 92% pass rate, outperforming larger models, but fails without constraints. Granite is the best raw model at 72%.