We quantized Qwen 3.8 27B and compared the quants on an RTX 6000

Reddit r/LocalLLaMA Models
model-quantization ai-benchmark qwen-3 gguf hardware-benchmark open-source inference-optimization

Summary

The team quantized Qwen 3.8 27B into various GGUF formats and benchmarked them on an RTX 6000, finding similar performance across quants with AD-Q6_K recommended for safety.

Me and my team made Atomic Dynamic GGUF quants for Qwen 3.8 27B, so we wanted to see the difference between them by giving each quant the same voxel island creation task First of all we were surprised at how well Qwen 3.8 27B handled the 3D scenes in general, though part of that is probably because all the scenes were voxels quant size top-1 vs BF16 mean KLD decode, RTX PRO 6000 AD-Q4_K_M 17.1 GB 95.6% 0.0113 67 tok/s AD-Q5_K_M 20.2 GB 97.3% 0.0042 57 tok/s AD-Q6_K 25.0 GB 98.7% 0.0011 49 tok/s Q8_0 28.9 GB 98.9% 0.0006 50 tok/s We think that each quant handled the scenes in a pretty similar way, the difference isn't that drastic, to the point that sometimes we preferred the Q4 output overall, though for the safest pick we recommend AD-Q6_K We ran the test inside atomic.chat and watched the output right there, the quants are available to download directly inside the app or on huggingface ( https://huggingface.co/collections/AtomicChat/qwen-38-27b ) (any feedback is appreciated, we're trying to make the product and models as good for you guys as possible)
Original Article

Similar Articles

I compared GGUF quants of Qwen3.6 27B to NVFP4, AWQ, AutoRound, and FP8

Reddit r/LocalLLaMA

A detailed benchmark comparing 16 quantizations of Qwen3.6 27B across GGUF, NVFP4, AWQ, AutoRound, and FP8 formats, measuring KL divergence from the unquantized reference. Weight-only GGUF quants generally offer the best quality-size tradeoffs, while vLLM quants vary substantially.

Qwen3.6 27B Pure Quant: 40 tok/s on 16 GB VRAM

Reddit r/LocalLLaMA

A quantized version of Qwen3.6 27B using a pure Q4_K_M method fits entirely in 16 GB VRAM, achieving up to 40 tok/s token generation speed with MTP, and significantly reducing model size compared to other GGUF variants.

Ternary Qwen3.6 27B Tested on 3090!

Reddit r/LocalLLaMA

User tests ternary quantized Qwen3.6 27B on an RTX 3090, achieving 60 tk/s with two slots and 100k KV cache using 21GB VRAM, with good quality and stable tool calls.