Tag
PrismML announces Bonsai 27B, a binary-quantized version of Qwen3.6 27B that runs on a phone using only 1.125 bits per weight, claiming 89.5% intelligence retention. The model is being independently tested by @sudoingX to verify performance.
Mixedbread Search introduces asymmetric quantization for late interaction retrieval, achieving near-lossless quality with 97% storage reduction by storing document vectors as binary signs while keeping query vectors at higher precision.
NanoQuant is a flexible binary quantization method that compresses dense transformers to sub-1-bit per weight. This repository provides a PyTorch implementation, still a work in progress, capable of quantizing models like Qwen3-0.6B and Qwen3-4B.