Reddit user demonstrates llamacpp speculative decoding boosting Qwen-3.6-27B token speed from 13.6 to 136.75 t/s, sharing exact commands and hardware setup.
First a little explanation about what is happening in the pictures. I did a small experiment with the aim of determining how much improvement using speculative decoding brings to the speed of the new Qwen (TL;DR big!). 1. image shows my simple prompt at the beginning of the session. 2. image shows time and token generation speed (13.60 t/s) for making the first version of the program. Also it shows my prompt asking for a new feature. 3. image shows time and token generation speed for a second version of the program (25.53 t/s - you can notice an improvement). Also on the image you can see there was a bug. I presented to Qwen the screenshot with browser console opened. Qwen correctly spotted what kind of bug it is and fixed it. 4. image shows time and token generation speed for a fixed version of the program (68.35 t/s - big improvement). Also image shows my prompt for making a small change in the program. 5. image shows time and token generation speed for final version of the program after small change (136.75 t/s !!!) Last image shows finished beautiful aquarium. Aesthetics and functionality is another level compared with the older models of similar size and many much bigger ones. So speed goes 13.60 > 25.53 > 68.35 > 136.75 t/s during session. Every time Qwen delivered full code. Similar kind of workflow I use very often. And all this thanks to one simple line in llama-server command '`--spec-type ngram-mod --spec-ngram-size-n 24 --draft-min 12 --draft-max 48`'. I am not sure this is the best setting but it works well for me. I will play with it more. My llama-swap command: ${llama-server} -m ${models}/Qwen3.6-27B/Qwen3.6-27B-Q8_0.gguf --mmproj ${models}/Qwen3.6-27B/mmproj-BF16Qwen3.6-27B.gguf --no-mmproj-offload --spec-type ngram-mod --spec-ngram-size-n 24 --draft-min 12 --draft-max 48 --ctx-size 128000 --temp 1.0 --top-p 0.95 --top-k 20 --presence_penalty 1.5 --chat-template-kwargs '{"preserve_thinking": true}' My linux PC has 40GB VRAM (rtx3090 and rtx4060ti) and 128GB DDR5 RAM. Big thanks to all smart people who contribute to llamacpp, to this Reddit community and to the Qwen crew. Free lunch, try it out... Edit: I forgot to mention some changes in llama.cpp from two days ago. So try to update.
A user reports that Qwen 27B at q6kxl quantization with multi-token prediction achieves 50-90 token/s decode and 1500-2200 token/s pre-fill on a 4090+3090 system using LCPP, noting it is reliably coherent and fast for various coding tasks.
A detailed benchmark comparing speculative decoding engines for Qwen 3.6 27B on a single RTX 3090, showing ik_llama achieving ~100 tokens per second in code generation. Results include decode TPS, TTFT, VRAM usage, and context degradation across 5 engine variants.
A user shares their llama.cpp server configuration for running Qwen 3.6 27B on an RTX 5090, achieving 80-100 t/s, and asks the community for alternative settings and tips.
A user shares a configuration for achieving over 80 tokens per second with Qwen3.6 35B A3B on a 12GB VRAM GPU using llama.cpp and Multi-Token Prediction (MTP). The post includes benchmark results and specific command-line parameters to optimize performance.
Comprehensive benchmarks of llama.cpp's speculative decoding methods on Qwen 3.6 27B show n-gram stacking on DFlash achieves up to 6x speedup on iterative coding tasks, with ngram-mod providing most of the gain and zero VRAM cost.