@nullfoundry: hey everyone. i'd like to share my new recipe for dflash ( merged yesterday on oficial llama.cpp ) llama-server -hf uns…
Summary
Sharing a new recipe for dflash speculative decoding in llama.cpp, achieving ~70 TPS on a single RTX 3090 using Qwen3.6-27B GGUF with a draft model.
View Cached Full Text
Cached at: 06/30/26, 07:37 AM
hey everyone.
i’d like to share my new recipe for dflash ( merged yesterday on oficial llama.cpp )
llama-server -hf unsloth/Qwen3.6-27B-GGUF:Q4_K_M –host 0.0.0.0 –port ${PORT} –threads 8 –threads-batch 8 –ctx-size 120000 –predict 16384 –batch-size 2048 –ubatch-size 1024 –gpu-layers all –flash-attn on –cache-type-k q8_0 –cache-type-v q8_0 –no-mmap –temp 0.6 –top-k 20 –top-p 0.95 –min-p 0.0 –repeat-penalty 1.0 –presence-penalty 0.0 –parallel 1 –metrics –jinja –reasoning off –reasoning-format auto –reasoning-budget 2048 -ctxcp 32 -fitt 1024 –cache-ram 16384 –chat-template-kwargs “{ "preserve_thinking": false}” –checkpoint-min-step 512 –reasoning-budget-message “Okay, I have thought enough. I will now provide the final answer” –cache-prompt –no-mmproj –kv-unified –spec-type draft-dflash -md “C:\Users\bagcn.cache\huggingface\hub\qwen3.6-27b-dflash-IQ4_XS.gguf” -ngld 99
~70TPS - 1x RTX 3090
Huuuuuge improvement!!!
Similar Articles
I tested freshly merged DFlash in llama.cpp on Qwen 3.6 27B Local AI win. 4.44x faster at 36K context. Here are my findings RTX 6000 PRO.
A user benchmarks the newly merged DFlash speculative decoding method in llama.cpp on Qwen 3.6 27B, achieving up to 4.44x speedup at 36K context compared to baseline, with detailed leaderboard and quality tests.
@pupposandro: https://x.com/pupposandro/status/2054241934164492328
The article announces support for DFlash and PFlash speculative decoding in llama.cpp for AMD Strix Halo iGPUs, demonstrating significant speedups in inference performance using ROCm.
@pupposandro: 2.5x faster than llama.cpp on Strix Halo. We just shipped DFlash + PFlash for the AMD Ryzen AI MAX+ 395 iGPU (gfx1151, …
A new toolset (DFlash + PFlash) achieves 2.5x faster inference than llama.cpp on AMD Ryzen AI MAX+ 395 iGPU, demonstrating significant speedups for Qwen3.6-27B with 128 GiB unified memory.
@pupposandro: PFlash now run @poolsideai's Laguna-XS.2 (33B-A3B MoE) on a single RTX 3090. - 111 tok/s decode @ short ctx - 128K TTFT…
PFlash now supports running @poolsideai's Laguna-XS.2 (33B-A3B MoE) on a single RTX 3090, achieving 111 tok/s decode and 5.4x faster prefill than llama.cpp, with NIAH passes up to 131K context.
BeeLlama v0.2.0 – major DFlash update. Single RTX 3090: Qwen 3.6 27B up to 164 tps (4.40x), Gemma 4 31B up to 177.8 tps (4.93x). Prompt processing speed near baseline.
BeeLlama v0.2.0 introduces major DFlash speculative decoding improvements, achieving up to 4.93x speedup on single RTX 3090 for Gemma 4 31B and 4.40x for Qwen 3.6 27B, with prompt processing near baseline.