Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs

Hugging Face Daily Papers Papers
diffusion-models large-language-models kv-caching parallel-decoding inference-acceleration io-aware memory-efficiency

Summary

Flash-dLLM is a training-free inference acceleration framework for diffusion LLMs that uses IO-aware KV caching and parallel decoding to achieve significant speedups and memory efficiency improvements.

Diffusion Large Language Models (dLLMs) have recently emerged as a promising alternative to autoregressive LLMs by enabling non-autoregressive text generation. However, their practical deployment remains limited by inefficient inference, largely due to the absence of effective Key-Value (KV) caching and scalable parallel decoding mechanisms. Existing acceleration methods typically study KV caching and parallel decoding in isolation, overlooking the I/O bottlenecks that arise when cache reuse and parallel token verification are jointly applied. In this work, we introduce Flash-dLLM, a training-free inference acceleration framework for fast and memory-efficient dLLMs. Flash-dLLM first identifies GPU memory I/O as a dominant bottleneck in KV-cache-enabled dLLM inference and addresses it with an I/O-aware fused KV-cache kernel that reduces redundant memory movement. Building on this optimized cache mechanism, Flash-dLLM further proposes an efficient KV-cache-driven draft-and-verify decoding strategy, where the dLLM itself serves as both drafter and verifier without requiring an auxiliary model. This unified design enables faster decoding while preserving generation quality and improving scalability to longer sequences and larger batch size. Extensive experiments on mathematical reasoning and code-generation benchmarks demonstrate that Flash-dLLM consistently outperforms existing state-of-the-art dLLM acceleration methods in both inference speed and memory efficiency. In particular, it achieves 5.1times and 11.0times speedups over prior strongest baseline Elastic-Cache on GSM8K and HumanEval, respectively.
Original Article
View Cached Full Text

Cached at: 09/23/26, 03:32 AM

Paper page - Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs

Source: https://huggingface.co/papers/2609.26796

Abstract

DiffusionLargeLanguageModels(dLLMs)haverecentlyemergedasapromisingalternativetoautoregressiveLLMsbyenablingnon-autoregressivetextgeneration.However,theirpracticaldeploymentremainslimitedbyinefficientinference,largelyduetotheabsenceofeffectiveKey-Value(KV)cachingandscalableparalleldecodingmechanisms.ExistingaccelerationmethodstypicallystudyKVcachingandparalleldecodinginisolation,overlookingtheI/Obottlenecksthatarisewhencachereuseandparalleltokenverificationarejointlyapplied.Inthiswork,weintroduceFlash-dLLM,atraining-freeinferenceaccelerationframeworkforfastandmemory-efficientdLLMs.Flash-dLLMfirstidentifiesGPUmemoryI/OasadominantbottleneckinKV-cache-enableddLLMinferenceandaddressesitwithanI/O-awarefusedKV-cachekernelthatreducesredundantmemorymovement.Buildingonthisoptimizedcachemechanism,Flash-dLLMfurtherproposesanefficientKV-cache-drivendraft-and-verifydecodingstrategy,wherethedLLMitselfservesasbothdrafterandverifierwithoutrequiringanauxiliarymodel.Thisunifieddesignenablesfasterdecodingwhilepreservinggenerationqualityandimprovingscalabilitytolongersequencesandlargerbatchsize.Extensiveexperimentsonmathematicalreasoningandcode-generationbenchmarksdemonstratethatFlash-dLLMconsistentlyoutperformsexistingstate-of-the-artdLLMaccelerationmethodsinbothinferencespeedandmemoryefficiency.Inparticular,itachieves5.1timesand11.0timesspeedupsoverpriorstrongestbaselineElastic-CacheonGSM8KandHumanEval,respectively.

View arXiv pageView PDFGitHub2Add to collection

Get this paper in your agent:

hf papers read 2609\.26796

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.26796 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.26796 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.26796 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles