A new task-aware quantization method called TAK achieves near-BF16 reasoning performance at significantly reduced model size, outperforming Unsloth on various AI models.
TL;DR My TAK quant of Qwen 3.8 27b scored 82.81% on reasoning, comparted with 77.34% for the byte matched Unsloth UD IQ2_S and 83.59% for BF16. Over the last few months, I've been exploring task aware quantization. I've now turned that work into a clean, repeatable pipeline under the reasoning domain. Coding is my next goal. For a while I used Qlab a much more broad measurement heavy system that helped me to determine what to test and where to investigate. It was great for exploration, but it accumulated a ton of gates and operational overhead. Once I found a reliable pipeline, I specialized it and retired the older application. The new system is called TAK: Task Aware Knapsack. I'm using TAK as both the application and models it produces. At a high level, TAK is a blend of TASA and TAQ. It starts with an imatrix built from a task specific corpus. Then we find the model cliff at is smallest size before complete collapse. It then combines those measurements with tensor level allocation promoting and demoting tensors within a byte specific budget. The result is a purpose built quantization rather than a general purpose recovery. Unsloth is included as the industry standard reference. Not as a claim that the methods are equivalent. There is no pruning, fine-tuning, model merging or anything else. This is purely an Imatrix + damage allocation process. These are all tested on a held out dataset. These are my current winners: https://huggingface.co/ByteOtter Qwen3.8-27B: 82.81% vs 77.34% Unsloth, +5.47 points Qwen3.5-4B: 73.44% vs 61.72% Unsloth, , +11.72 points Gemma 4 E4B: 69.53% vs 55.47% Unsloth, , +14.06 points Gemma 3 4B QAT: 54.69% vs 35.16% Unsloth, , +19.53 points Across these runs, TAK has beaten matched Unsloth Dynamic 1.0, 2.0 and now 3.0 comparators on the target reasoning benchmark. The method has worked across Gemma 3, Gemma 4, Qwen3.5 and Qwen3.8 covering both dense, QAT and MoE architectures. Taken together these results give me strong evidence that task aware precision allocation works well for reasoning. Im excited to expand the pipeline to other domains like coding and math. Charts were provided by ChatGPT on my data. You can follow the work u/byteotter on X https://x.com/byteotter or support it on Buy Me a Coffee. https://buymeacoffee.com/byteotter
Unsloth AI released 1-bit quantized versions of the Qwen3.8-27B AI model that run on 8GB RAM, retaining approximately 77% accuracy compared to BF16 precision.
ByteOtter replicates tensor-level allocation on Qwen 3.5 4B, achieving a 16.67% relative improvement in reasoning performance with only a 0.412% increase in model size, marking the first cross-family application outside Gemma.
The author shares a quantization recipe for Qwen3.6 27B that makes the model use significantly fewer thinking tokens while still producing correct answers, leading to faster inference on math benchmarks.