I tested Qwen3.8 27B KLD for 21 Models at 9 different KV quants that fit on 16GB VRAM (189 benchmarks).
Summary
A community benchmark evaluated 21 Qwen3.8 27B model/KV quantization combinations (189 benchmarks) that fit in 16GB VRAM, finding q5_1/q5_1 is usually optimal and that upping model quant at q4_0/q4_0 beats lower quants at full f16 KV cache.
Similar Articles
Here are my KV cache quantization benchmarks: TurboQuant is overrated but saved by TCQ, q5 deserves more attention, and symmetric q8 might be a waste of VRAM
A detailed benchmark comparing KV cache quantization methods (TurboQuant, TCQ, q4, q5, q8) using PPL and KLD metrics on Qwen 3.6 27B, finding that TCQ improves low-bit quantization, asymmetric KV beats symmetric at same size, and q8 is often overkill. Includes analysis and data in linked article.
Qwen3.8-27b q8 KV cache does seem to actually hurt model performance
The article discusses how on-the-fly KV cache quantization can reduce long-context model performance due to compounding errors, based on experiments with Qwen3.8-27B.
My Qwen 3.8 27B tests on limited VRAM (16-20GB)
This article tests various quantized versions of the Qwen 3.8 27B AI model on limited VRAM setups, comparing their performance on tasks like animation generation, app development, and word generation.
yall are sleeping on qwen 3.8 27b q2 + q2 dflash + q5 kv
A user shares their experience running a quantized Qwen 3.8 27B model using QAT Q2 and Q5 KV, achieving high performance on a 12GB GPU with up to 200K token context, surpassing models like Sonnet 4.6.
Qwen 3.8 27B KV f16 vs q8_0 are not equivalents
A user shares test results indicating that KV cache types f16 and q8_0 are not equivalent for the Qwen 3.8 27B model, with f16 showing better detail and consistency, and provides configuration details for AMD ROCm hardware.