Qwen 27B 3.8 quants: How low can you go?
Summary
A user shares their positive experience with low quantizations of Qwen 27B 3.8 on a Mac mini M4, using Unsloth's Q3 XXS quant, and asks for others' experiences with sub-Q3 quants.
Similar Articles
Qwen3.6-27B Quantization Benchmark
This article benchmarks various Qwen3.6-27B quantizations (Q8 to Q2) using KLD and Same Top P metrics, comparing providers like Unsloth and mradermacher, and offers recommendations for quality-size trade-offs.
Qwen 3.8 27B Released! Please Share Your Experience
Qwen 3.8 27B has been released, and the author invites users to share their experience, including which frontier model it resembles and which quantization they used.
Qwen 3.8 27b is strong even at Q3_xxs
The user finds Qwen 3.8 27b in Q3 quantization highly effective for coding tasks with fast inference speeds, outperforming previous models, despite minor issues in general conversations.
Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses
Benchmarking shows that 4-bit quantized Qwen3.8 27B retains performance on benchmarks like Terminal-Bench 2.1 while fitting on 24GB GPUs, but 1-bit quantization causes severe degradation.
Qwen 3.6 27B 30GB Same top p: 98.358 ± 0.033 % vs UD Q8 K XL 33GB Same top p: 97.426 ± 0.041 %
A community researcher shares a custom quantization recipe for Qwen3.6-27B that produces a smaller 30GB Q8 GGUF by keeping high-outlier sublayers in BF16, achieving better KLD and top-p metrics than Unsloth's 33GB Q8_K_XL variant.