@nullfoundry: hey everyone. i'd like to share my new recipe for dflash ( merged yesterday on oficial llama.cpp ) llama-server -hf uns…
Summary
Sharing a new recipe for dflash speculative decoding in llama.cpp, achieving ~70 TPS on a single RTX 3090 using Qwen3.6-27B GGUF with a draft model.
View Cached Full Text
Cached at: 06/30/26, 07:37 AM
hey everyone.
i’d like to share my new recipe for dflash ( merged yesterday on oficial llama.cpp )
llama-server -hf unsloth/Qwen3.6-27B-GGUF:Q4_K_M –host 0.0.0.0 –port ${PORT} –threads 8 –threads-batch 8 –ctx-size 120000 –predict 16384 –batch-size 2048 –ubatch-size 1024 –gpu-layers all –flash-attn on –cache-type-k q8_0 –cache-type-v q8_0 –no-mmap –temp 0.6 –top-k 20 –top-p 0.95 –min-p 0.0 –repeat-penalty 1.0 –presence-penalty 0.0 –parallel 1 –metrics –jinja –reasoning off –reasoning-format auto –reasoning-budget 2048 -ctxcp 32 -fitt 1024 –cache-ram 16384 –chat-template-kwargs “{ "preserve_thinking": false}” –checkpoint-min-step 512 –reasoning-budget-message “Okay, I have thought enough. I will now provide the final answer” –cache-prompt –no-mmproj –kv-unified –spec-type draft-dflash -md “C:\Users\bagcn.cache\huggingface\hub\qwen3.6-27b-dflash-IQ4_XS.gguf” -ngld 99
~70TPS - 1x RTX 3090
Huuuuuge improvement!!!
Similar Articles
I tested freshly merged DFlash in llama.cpp on Qwen 3.6 27B Local AI win. 4.44x faster at 36K context. Here are my findings RTX 6000 PRO.
A user benchmarks the newly merged DFlash speculative decoding method in llama.cpp on Qwen 3.6 27B, achieving up to 4.44x speedup at 36K context compared to baseline, with detailed leaderboard and quality tests.
I tested all llama.cpp's speculative decoding methods on Qwen 3.6 27B: MTP ~2.7x, DFlash ~3.7x, n-gram stack ~6x on real coding. Local AI win. My findings on RTX 6000 PRO.
Comprehensive benchmarks of llama.cpp's speculative decoding methods on Qwen 3.6 27B show n-gram stacking on DFlash achieves up to 6x speedup on iterative coding tasks, with ngram-mod providing most of the gain and zero VRAM cost.
@pupposandro: https://x.com/pupposandro/status/2054241934164492328
The article announces support for DFlash and PFlash speculative decoding in llama.cpp for AMD Strix Halo iGPUs, demonstrating significant speedups in inference performance using ROCm.
@pupposandro: 2.5x faster than llama.cpp on Strix Halo. We just shipped DFlash + PFlash for the AMD Ryzen AI MAX+ 395 iGPU (gfx1151, …
A new toolset (DFlash + PFlash) achieves 2.5x faster inference than llama.cpp on AMD Ryzen AI MAX+ 395 iGPU, demonstrating significant speedups for Qwen3.6-27B with 128 GiB unified memory.
@pupposandro: PFlash now run @poolsideai's Laguna-XS.2 (33B-A3B MoE) on a single RTX 3090. - 111 tok/s decode @ short ctx - 128K TTFT…
PFlash now supports running @poolsideai's Laguna-XS.2 (33B-A3B MoE) on a single RTX 3090, achieving 111 tok/s decode and 5.4x faster prefill than llama.cpp, with NIAH passes up to 131K context.