Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification

Hugging Face Daily Papers Papers
video-generation attention-sparsification inference-acceleration diffusion-transformers training-free online-softmax

Summary

Sol-Attn introduces a training-free method to sparsify attention for video generation inference, achieving over 2x speedup by dynamically selecting key-value blocks during online softmax with minimal quality loss.

Diffusion transformers are essential for high-fidelity video generation, but long token sequences make attention a dominant inference bottleneck. Training-free dynamic sparse attention alleviates this bottleneck by computing only selected key-value blocks, yet existing methods struggle to sparsify attention both efficiently and accurately for two reasons: (1) Rigid, unpredictable, and costly routing: selecting a fixed fraction of top-ranked blocks by proxy score imposes fixed budgets, whereas retaining blocks to reach a target cumulative proxy probability mass yields dynamic but potentially imbalanced budgets; both incur non-negligible overhead from computing and materializing proxy scores. (2) Lossy keep-or-drop sparsification: unselected blocks are discarded entirely, degrading accuracy under aggressive sparsity. These limitations motivate cheaper dynamic-budget routing while limiting accuracy degradation. In this paper, we introduce training-free Sol-Attn (Sparsifying online attention), which unifies dynamic routing, sparse computation, and approximation correction in a single online-softmax pass, achieving a better accuracy-efficiency trade-off in sparse attention. The core of Sol-Attn is on-the-fly block thresholding with proxy-score reuse, which selects critical blocks by comparing block proxy scores against a threshold during online softmax. This design enables dynamic yet controllable block budgets without materializing the proxy map, while directly reusing the proxy scores of unselected blocks to approximate their contribution. Experiments across image and video generation tasks show that Sol-Attn advances the quality-efficiency frontier of training-free sparse attention, delivering 2.1 times and 2.3 times end-to-end speedups for video generation and editing, respectively, while preserving visual quality.
Original Article
View Cached Full Text

Cached at: 07/28/26, 10:24 AM

Paper page - Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification

Source: https://huggingface.co/papers/2607.24027 Published on Jul 27

·

Submitted byhttps://huggingface.co/Owen777

Owenon Jul 28

Abstract

Diffusiontransformersareessentialforhigh-fidelityvideogeneration,butlongtokensequencesmakeattentionadominantinferencebottleneck.Training-freedynamicsparseattentionalleviatesthisbottleneckbycomputingonlyselectedkey-valueblocks,yetexistingmethodsstruggletosparsifyattentionbothefficientlyandaccuratelyfortworeasons:(1)Rigid,unpredictable,andcostlyrouting:selectingafixedfractionoftop-rankedblocksbyproxyscoreimposesfixedbudgets,whereasretainingblockstoreachatargetcumulativeproxyprobabilitymassyieldsdynamicbutpotentiallyimbalancedbudgets;bothincurnon-negligibleoverheadfromcomputingandmaterializingproxyscores.(2)Lossykeep-or-dropsparsification:unselectedblocksarediscardedentirely,degradingaccuracyunderaggressivesparsity.Theselimitationsmotivatecheaperdynamic-budgetroutingwhilelimitingaccuracydegradation.Inthispaper,weintroducetraining-freeSol-Attn(Sparsifyingonlineattention),whichunifiesdynamicrouting,sparsecomputation,andapproximationcorrectioninasingleonline-softmaxpass,achievingabetteraccuracy-efficiencytrade-offinsparseattention.ThecoreofSol-Attnison-the-flyblockthresholdingwithproxy-scorereuse,whichselectscriticalblocksbycomparingblockproxyscoresagainstathresholdduringonlinesoftmax.Thisdesignenablesdynamicyetcontrollableblockbudgetswithoutmaterializingtheproxymap,whiledirectlyreusingtheproxyscoresofunselectedblockstoapproximatetheircontribution.ExperimentsacrossimageandvideogenerationtasksshowthatSol-Attnadvancesthequality-efficiencyfrontieroftraining-freesparseattention,delivering2.1timesand2.3timesend-to-endspeedupsforvideogenerationandediting,respectively,whilepreservingvisualquality.

View arXiv pageView PDFProject pageAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.24027 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.24027 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.24027 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

FVAttn: Adaptive Sparse Attention with Runtime Load Balancing for Video Generation

Hugging Face Daily Papers

FVAttn is a training-free sparse attention system that uses runtime load balancing to improve distributed execution efficiency of adaptive sparse attention under multi-GPU sequence parallelism for video generation, achieving up to 4.41x attention speedup and 2.11x inference speedup over FlashAttention on step-distilled Wan2.2 I2V while maintaining competitive video quality.