Qwen3.6-27B speculative decoding gets better on heavier quants
Summary
Qwen3.6-27B model shows improved speculative decoding performance when using heavier quantization levels, enhancing inference efficiency.
Similar Articles
Qwen 3.8 27b is strong even at Q3_xxs
The user finds Qwen 3.8 27b in Q3 quantization highly effective for coding tasks with fast inference speeds, outperforming previous models, despite minor issues in general conversations.
Qwen3.8-27B-int4-AutoRound (18GB) - with working MTP spec decode
This article presents a quantized version of the Qwen3.8-27B model using INT4 AutoRound quantization with working MTP for speculative decoding, achieving significant inference speedups on GPUs.
Qwen 27B
A user reports that Qwen 27B at q6kxl quantization with multi-token prediction achieves 50-90 token/s decode and 1500-2200 token/s pre-fill on a 4090+3090 system using LCPP, noting it is reliably coherent and fast for various coding tasks.
Need a second pair of eyes, this Qwen3.6 27B quant recipe consistently thinks less and is correct
The author shares a quantization recipe for Qwen3.6 27B that makes the model use significantly fewer thinking tokens while still producing correct answers, leading to faster inference on math benchmarks.
Qwen/Qwen3.6-35B-A3B-FP8
Alibaba releases Qwen3.6-35B-A3B-FP8, an open-weight quantized variant of Qwen3.6 with 35B parameters and 3B activated via MoE, featuring improved agentic coding capabilities and thinking preservation for iterative development.