MMFlash: Faster Speculative Decoding for Vision-Language Models

Hugging Face Daily Papers Papers

Summary

MMFlash improves multimodal speculative decoding by adaptively upweighting visually dependent tokens and using sparse attention for long video inputs, achieving up to 3.58× lossless decoding speedup and outperforming EAGLE-3 and DFlash.

Speculative decoding accelerates generation by letting a lightweight drafter propose tokens for parallel verification by a target model. Diffusion drafters such as DFlash propose an entire token block in a single forward pass, but mainly focus on text-only tasks, leaving multimodal drafting underexplored. To study this setting, we first introduce a metric of token-level visual dependence based on the KL divergence between next-token distributions with and without visual input. We find that acceptance rates decrease from 46% in the lowest visual-dependence quartile to 22% in the highest, with this disparity persisting after multimodal training. Motivated by this observation, we propose MMFlash, a simple and effective framework that adaptively upweights visually dependent tokens that the drafter predicts poorly. For long video inputs, MMFlash further employs sparse attention to reduce redundant attention computation and improve drafting quality. Across various multimodal benchmarks, MMFlash achieves lossless decoding speedups of up to 3.58times, outperforming EAGLE-3 and DFlash by up to 195.4% and 17.1%, respectively.
Original Article
View Cached Full Text

Cached at: 11/16/26, 08:48 AM

Paper page - MMFlash: Faster Speculative Decoding for Vision-Language Models

Source: https://huggingface.co/papers/2611.18482

Abstract

MMFlash improves multimodal speculative decoding by adaptively upweighting visually dependent tokens and using sparse attention for long video, yielding up to 3.58× lossless speedup.

Speculative decodingaccelerates generation by letting alightweight drafterpropose tokens forparallel verificationby atarget model.Diffusion drafterssuch asDFlashpropose an entiretoken blockin asingle forward pass, but mainly focus ontext-only tasks, leavingmultimodal draftingunderexplored. To study this setting, we first introduce a metric oftoken-level visual dependencebased on theKL divergencebetweennext-token distributionswith and withoutvisual input. We find thatacceptance ratesdecrease from 46% in the lowestvisual-dependence quartileto 22% in the highest, with this disparity persisting aftermultimodal training. Motivated by this observation, we proposeMMFlash, a simple and effective framework thatadaptively upweightsvisually dependent tokensthat the drafter predicts poorly. Forlong video inputs,MMFlashfurther employssparse attentionto reduceredundant attention computationand improvedrafting quality. Across variousmultimodal benchmarks,MMFlashachieveslossless decoding speedupsof up to 3.58times, outperformingEAGLE-3andDFlashby up to 195.4% and 17.1%, respectively.

View arXiv pageView PDFProject pageGitHub13Add to collection

Get this paper in your agent:

hf papers read 2611\.18482

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2611.18482 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2611.18482 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2611.18482 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles