Qwen3.6-27B KLDs - INTs and NVFPs
Summary
Reddit post compares quantized Qwen3.6-27B variants (INT4, NVFP4, BF16-INT4) showing trade-offs between memory size and accuracy for different use-cases.
Similar Articles
Qwen 3.6 27B - VLLM Performance Benchmark Results (BF16, FP8, NVFP4)
A detailed benchmark of Qwen 3.6 27B using VLLM across BF16, FP8, and NVFP4 quantizations, showing NVFP4 fastest for token generation but FP8 best for prompt processing, with practical advice on choosing the right quantization for coding tasks.
unsloth/Qwen3.6-27B-NVFP4 vs. Intel/Qwen3.6-27B-int4-AutoRound vs. nvidia/Qwen3.6-27B-NVFP4 -- which one to choose?
The article compares three quantized variants of Qwen3.6-27B (NVFP4 from Unsloth and Nvidia, int4-AutoRound from Intel) and requests benchmarks and hallucination data from the community.
Qwen3.6-27B Quantization Benchmark
This article benchmarks various Qwen3.6-27B quantizations (Q8 to Q2) using KLD and Same Top P metrics, comparing providers like Unsloth and mradermacher, and offers recommendations for quality-size trade-offs.
UPDATE: Qwen-27B-IQ4_KS and Qwen-27B-IQ_KS_KT for ik_llama.cpp, especially for NVIDIA with 16GB VRAM
New GGUF quantizations of Qwen3.6-27B optimized for 16GB VRAM NVIDIA GPUs, including an experimental Trellis variant, with perplexity benchmarks.
I compared GGUF quants of Qwen3.6 27B to NVFP4, AWQ, AutoRound, and FP8
A detailed benchmark comparing 16 quantizations of Qwen3.6 27B across GGUF, NVFP4, AWQ, AutoRound, and FP8 formats, measuring KL divergence from the unquantized reference. Weight-only GGUF quants generally offer the best quality-size tradeoffs, while vLLM quants vary substantially.