@simplifyinAI: Researchers just made LLMs 8.5x faster with zero accuracy loss. It's called DFlash. It replaces the slow autoregressive…
Summary
Researchers introduced DFlash, a method that replaces autoregressive drafters with block diffusion models to achieve 8.5x faster LLM inference with zero accuracy loss.
Similar Articles
@_avichawla: Researchers found a way to make LLMs 8.5x faster! (without compromising accuracy) Speculative decoding is quite an effe…
Researchers introduced DFlash, a technique using block diffusion models for speculative decoding that accelerates LLM inference by up to 8.5x without accuracy loss. It is already integrated with major frameworks like vLLM and SGLang.
Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
Flash-dLLM is a training-free inference acceleration framework for diffusion LLMs that uses IO-aware KV caching and parallel decoding to achieve significant speedups and memory efficiency improvements.
@DivyanshT91162: Autoregressive LLMs might already be getting replaced Someone built dLLM — an open-source library that can turn ANY aut…
dLLM is an open-source library that converts any autoregressive LLM into a diffusion LLM, enabling parallel decoding and faster text generation.
@zhijianliu_: DFlash is now running in a production inference stack. More draft models coming soon. https://github.com/z-lab/dflash
DFlash is a lightweight block diffusion model for speculative decoding, now running in production with support for various LLMs like Qwen and Gemma.
@LiorOnAI: You now convert any LLM into a faster one without retraining from scratch. NVIDIA just did this to their 30B model. Her…
NVIDIA proposes a method to convert any LLM into a faster one by splitting it into two copies: one frozen for context, the other trained to generate multiple tokens in parallel, achieving 2.4x speedup with ~99% quality retention using only 8% of training data.