MMFlash: Faster Speculative Decoding for Vision-Language Models
Summary
MMFlash improves multimodal speculative decoding by adaptively upweighting visually dependent tokens and using sparse attention for long video inputs, achieving up to 3.58× lossless decoding speedup and outperforming EAGLE-3 and DFlash.
View Cached Full Text
Cached at: 11/16/26, 08:48 AM
Paper page - MMFlash: Faster Speculative Decoding for Vision-Language Models
Source: https://huggingface.co/papers/2611.18482
Abstract
MMFlash improves multimodal speculative decoding by adaptively upweighting visually dependent tokens and using sparse attention for long video, yielding up to 3.58× lossless speedup.
Speculative decodingaccelerates generation by letting alightweight drafterpropose tokens forparallel verificationby atarget model.Diffusion drafterssuch asDFlashpropose an entiretoken blockin asingle forward pass, but mainly focus ontext-only tasks, leavingmultimodal draftingunderexplored. To study this setting, we first introduce a metric oftoken-level visual dependencebased on theKL divergencebetweennext-token distributionswith and withoutvisual input. We find thatacceptance ratesdecrease from 46% in the lowestvisual-dependence quartileto 22% in the highest, with this disparity persisting aftermultimodal training. Motivated by this observation, we proposeMMFlash, a simple and effective framework thatadaptively upweightsvisually dependent tokensthat the drafter predicts poorly. Forlong video inputs,MMFlashfurther employssparse attentionto reduceredundant attention computationand improvedrafting quality. Across variousmultimodal benchmarks,MMFlashachieveslossless decoding speedupsof up to 3.58times, outperformingEAGLE-3andDFlashby up to 195.4% and 17.1%, respectively.
View arXiv pageView PDFProject pageGitHub13Add to collection
Get this paper in your agent:
hf papers read 2611\.18482
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2611.18482 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2611.18482 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2611.18482 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding
Introduces Faster Flash Decoding (FFD), a training-free hardware-algorithm co-design framework that accelerates long-context decoding in LLMs by exploiting attention sparsity, achieving up to 11.6x speedup and scaling to 256K context length.
@_avichawla: Researchers found a way to make LLMs 8.5x faster! (without compromising accuracy) Speculative decoding is quite an effe…
Researchers introduced DFlash, a technique using block diffusion models for speculative decoding that accelerates LLM inference by up to 8.5x without accuracy loss. It is already integrated with major frameworks like vLLM and SGLang.
Vision Is Not Overhead: One-Pass Block Drafting for Lossless Speculative Decoding in Vision-Language Models
The paper introduces GLANCE, a one-pass block drafting method for lossless speculative decoding in vision-language models, achieving up to 2.93x faster generation without changing output.
VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs
VideoMM introduces an adaptive macro-micro inference framework that reduces visual token overhead in video MLLMs, achieving a 6.13× speedup and 7.4% accuracy gain for efficient long-form video understanding.
Accelerating vision-language models with LFM2.5-VL-DSpark
Today, we release an experimental DSpark draft model for our vision-language model (VLM) LFM2.5-VL-3B, which adds a speculative decoding path for faster inference with minimal memory cost.