dflash

Sharing a new recipe for dflash speculative decoding in llama.cpp, achieving ~70 TPS on a single RTX 3090 using Qwen3.6-27B GGUF with a draft model.

Ornith 35B shows 30-40% token generation speedup when paired with Qwen3.6 35B DFlash speculative model in llama-server, achieving 80% acceptance rate on mixed code and text, though prompt processing suffers.

Progress update on DSpark: training of DFlash backbone and markov head is complete, enabling use on 27B. Next is training the confidence head for adaptive drafting, expected 8-14% speed improvement over DFlash.

Charles Frye announces the co-release with Z Lab of six new DFlash speculators for Alibaba Qwen 3.x models, achieving over 1k output tokens per second for Qwen 3.5 122B-A10B on a B200.

Modal and Z Lab release six new DFlash speculative decoding draft models for Qwen 3.x, achieving over 1000 tokens per second on a B200 and arguing that speculative decoding is the most impactful inference optimization.

Z Lab, SGLang, and Modal release DFlash, a new speculative decoding model for Qwen 3.5 397B-A17B that uses block diffusion and KV injection to achieve over 4x throughput improvement over baseline and 1.5x over native MTP.

New research on DFlash and Spec V2 speculative decoding methods achieves >4.3X baseline throughput for LLM inference, released as the default speculative decoding engine in SGLang.

Xiaomi has released MiMo V2.5 with DFlash and Persistent kernel, achieving 1000-3000 tps. The DFlash model is now available and open-source release is promised soon.

Benchmarks of DFlash speculative decoding combined with KV cache compression on RTX 5090 show up to 3.26x speedup on Qwen3.6-27B with minimal perplexity degradation, with q4_0/turbo4 providing the best balance.

A GitHub repository called club-3090 provides recipes and configs for serving large language models locally on RTX 3090 GPUs, with support for multiple engines and quantization methods like Dflash and TurboQuant, including newly unlocked Q5 quants.

BeeLlama v0.2.0 introduces major DFlash speculative decoding improvements, achieving up to 4.93x speedup on single RTX 3090 for Gemma 4 31B and 4.40x for Qwen 3.6 27B, with prompt processing near baseline.

User seeking to optimize token generation speed on dual 3090s using DFlash and MTP speculative decoding techniques with llama.cpp and beellama, sharing their configuration and commands.

A token timing simulator widget was added to the LLM Engineer's Almanac, demonstrating the DFlash technique achieving ~1k TPS, to help users viscerally understand benchmark performance numbers.

Initial DFlash implementation by Zai_org is integrated into ZML AI, with plans to include it in zml/llmd.

dflash-mlx v0.1.6 is released with major agentic improvements, including adaptive verification, custom kernels, prefix cache improvements, and broader compatibility with agentic coding tools like OpenCode, aider, and Continue.

The article discusses the potential compatibility of DFlash and PFlash multi-model speedup methods with Heretic, a tool used for model decensoring, while highlighting the performance benefits on models like Qwen3.6 and Gemma 4.

Kimi K2.6 paired with DFlash inference system achieves 508 tokens/s on 8×AMD MI300X, a 5.6× throughput jump from 90 tokens/s baseline with zero quality loss.

Z-lab releases DFlash for Qwen3.6-35B-A3B, a model fine-tuning/compression technique, with training complete and weights now available on GitHub and HuggingFace.