@nullfoundry: hey everyone. i'd like to share my new recipe for dflash ( merged yesterday on oficial llama.cpp ) llama-server -hf uns…
Summary
Sharing a new recipe for dflash speculative decoding in llama.cpp, achieving ~70 TPS on a single RTX 3090 using Qwen3.6-27B GGUF with a draft model.
View Cached Full Text
Cached at: 06/30/26, 07:37 AM
hey everyone.
i’d like to share my new recipe for dflash ( merged yesterday on oficial llama.cpp )
llama-server -hf unsloth/Qwen3.6-27B-GGUF:Q4_K_M –host 0.0.0.0 –port ${PORT} –threads 8 –threads-batch 8 –ctx-size 120000 –predict 16384 –batch-size 2048 –ubatch-size 1024 –gpu-layers all –flash-attn on –cache-type-k q8_0 –cache-type-v q8_0 –no-mmap –temp 0.6 –top-k 20 –top-p 0.95 –min-p 0.0 –repeat-penalty 1.0 –presence-penalty 0.0 –parallel 1 –metrics –jinja –reasoning off –reasoning-format auto –reasoning-budget 2048 -ctxcp 32 -fitt 1024 –cache-ram 16384 –chat-template-kwargs “{ "preserve_thinking": false}” –checkpoint-min-step 512 –reasoning-budget-message “Okay, I have thought enough. I will now provide the final answer” –cache-prompt –no-mmproj –kv-unified –spec-type draft-dflash -md “C:\Users\bagcn.cache\huggingface\hub\qwen3.6-27b-dflash-IQ4_XS.gguf” -ngld 99
~70TPS - 1x RTX 3090
Huuuuuge improvement!!!
Similar Articles
I tested freshly merged DFlash in llama.cpp on Qwen 3.6 27B Local AI win. 4.44x faster at 36K context. Here are my findings RTX 6000 PRO.
A user benchmarks the newly merged DFlash speculative decoding method in llama.cpp on Qwen 3.6 27B, achieving up to 4.44x speedup at 36K context compared to baseline, with detailed leaderboard and quality tests.
I tested all llama.cpp's speculative decoding methods on Qwen 3.6 27B: MTP ~2.7x, DFlash ~3.7x, n-gram stack ~6x on real coding. Local AI win. My findings on RTX 6000 PRO.
Comprehensive benchmarks of llama.cpp's speculative decoding methods on Qwen 3.6 27B show n-gram stacking on DFlash achieves up to 6x speedup on iterative coding tasks, with ngram-mod providing most of the gain and zero VRAM cost.
Dflash 2 speculative decoding by SamuelOliveirads · Pull Request #2345 · ikawrakow/ik_llama.cpp
This pull request introduces Dflash 2 speculative decoding and other performance improvements to ik_llama.cpp, a fork of llama.cpp focused on enhancing CPU inference and quantization techniques for LLMs.
@ggerganov: llama.cpp v0.7.0, 128GB Mac/Spark: llama serve -hf ggml-org/Qwen3.8-Flash-Next-GGUF:IQ4_NL --spec-type draft-mtp --lazy…
llama.cpp v0.7.0 released with support for Qwen3.8-Flash-Next GGUF models via MTP-based speculative decoding on 128GB Mac/Spark setups, showcasing efficient local LLM serving.
@pupposandro: https://x.com/pupposandro/status/2054241934164492328
The article announces support for DFlash and PFlash speculative decoding in llama.cpp for AMD Strix Halo iGPUs, demonstrating significant speedups in inference performance using ROCm.