ByteShape released ShapeLearn GGUFs for Qwen 3.8 27B, achieving high accuracy on benchmarks, and discussed the role of KLD in quantization, with a related paper accepted at EMNLP 2026.
Hey r/LocalLLaMA, We’ve released our full ShapeLearn GGUFs for Qwen 3.8 27B. Blog / Download models TL;DR 3.84 bpw (GPU-5) reaches 99.63% of BF16’s aggregate score of 8 benchmarks, being the most accurate quant we’ve evaluated; 3.23 bpw (GPU-4) reaches 98.72%. These average BF16-normalized scores across instruct and thinking benchmarks. All five new models sit on the measured quality/speed-bpw frontier across six GPUs. In this model’s case, lower BPW translates directly to TPS. Comparisons include Unsloth v3, ISTA-DASLab, AtomicChat and Bartowski (not Bartowski’s newest release). Congrats to the team at ISTA for also landing a frontier model. DFlash2 delivered 1.34-2.10× baseline throughput; MTP delivered 1.28-1.66×, with temperature sampling rather than greedy decoding. Lite held up very well. As we expected. We released ShapeLearn-Lite quants a couple of days after Qwen arrived: less optimization, targeted sanity checks, full benchmarking after release. Then Unsloth v3 arrived with lower KLD at several comparable sizes. Lite looked overtaken, until the task results came in. Three of six Lite models made the quality/speed frontier against twelve Unsloth v3 models in our RTX Pro 6000 comparison. Pretty good for an impatient release. Full ShapeLearn now pushes that frontier further. Which brings us to KLD. Unsloth Dynamic V3’s UD-IQ3_S had ~20% lower KLD than our similarly sized smallest Lite model, but scored 95.55% versus Lite’s 97.33% of BF16’s aggregate benchmark score. Closer token distributions did not mean better task performance. KLD is useful to avoid a quant that has fallen over the edge, but it isn’t a quantization leaderboard. That distinction is the subject of our paper on KLD and quantization fidelity, recently accepted for publication to the EMNLP 2026 Industry Track. We also released blog post version of the paper a few weeks back. We benchmarked this release on RTX 6000 Pro Blackwell, RTX 5090, RTX 4090, RTX 3090, RTX 4080 and RTX 5060 Ti. The benchmarks we used to measure quality are: GSM8K for math, IFEval for instruction following, MMLU for general knowledge, LiveCodeBench V6 for coding, Multi-IF for multi-turn and multilingual instruction following, ACEBench for tool use and agentic tasks (both thinking and instruct), Multiple HumanEval for coding (thinking) and BFCL V4 for tool calling and agentic tasks (thinking). If you want to dive deeper or choose the best model for your use case, the blog has the complete results across all tested GPUs, along with the methodology, model sizes, and full legend.
This article benchmarks various Qwen3.6-27B quantizations (Q8 to Q2) using KLD and Same Top P metrics, comparing providers like Unsloth and mradermacher, and offers recommendations for quality-size trade-offs.
The article evaluates how different quantizations of the Qwen3.6-27B model affect output quality using KL divergence and top-1 token accuracy, as well as visual examples like SVG drawings.
A detailed benchmark comparing 16 quantizations of Qwen3.6 27B across GGUF, NVFP4, AWQ, AutoRound, and FP8 formats, measuring KL divergence from the unquantized reference. Weight-only GGUF quants generally offer the best quality-size tradeoffs, while vLLM quants vary substantially.
A detailed benchmark comparing ByteShape and Unsloth quantizations of Qwen3.6-35B-A3B on tool calling performance, KV cache quantization effects, and long context degradation using llama.cpp and tool-eval-bench.
A community researcher shares a custom quantization recipe for Qwen3.6-27B that produces a smaller 30GB Q8 GGUF by keeping high-outlier sublayers in BF16, achieving better KLD and top-p metrics than Unsloth's 33GB Q8_K_XL variant.