SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation

Hugging Face Daily Papers Papers
video-generation diffusion-transformer linear-attention attention-residuals hybrid-attention efficient-video-generation

Summary

SANA-Video 2.0 introduces a hybrid linear-softmax attention mechanism for video diffusion transformers, achieving high-quality video generation up to 720p on a single GPU with significantly reduced latency compared to full-softmax models, while maintaining competitive VBench scores.

We introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality video up to 720p on a single GPU, SANA-Video 2.0 matches full-softmax video DiTs in quality while retaining the favorable long-sequence scaling of linear attention. To avoid quadratic attention throughout, Hybrid Linear-Softmax Attention combines gated linear attention for O(N)-dominated mixing with periodic gated-softmax anchors at a 3:1 ratio, restoring the full-rank token interactions that pure linear attention lacks. To propagate these refreshed representations across depth, Block Attention Residuals (AttnRes) route completed block summaries into later linear layers, enabling anchor-feature reuse and boosting deep-layer effective rank by ~12%. Through from-scratch training, SANA-Video 2.0 learns the complete hybrid directly rather than linearizing pretrained models, with reduced-resolution proxy studies establishing 25% softmax as the optimal quality-efficiency trade-off. With 40-step sampling, SANA-Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, remaining competitive with far larger softmax video DiTs at a fraction of the latency. Its compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline at 720p/60s, a gap that expands with video duration. Furthermore, full-stack Sol-Engine optimization (kernel fusion, caching, and sparse attention) accelerates this hardware-friendly backbone by a further 3.58x, bringing the 5B pipeline to 13.06s at 720p/5s and making it 120x faster than Wan 2.2-A14B on one H100. Overall, our hybrid design recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.
Original Article
View Cached Full Text

Cached at: 07/24/26, 01:01 PM

Paper page - SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation

Source: https://huggingface.co/papers/2607.21553 Published on Jul 23

·

Submitted byhttps://huggingface.co/Lawrence-cj

Chenon Jul 24

Authors:

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

WeintroduceSANA-Video2.0,ahybridvideodiffusiontransformerinstantiatedat5Band14Bscalesunderaunifiedarchitecture.Designedtogeneratehigh-qualityvideoupto720ponasingleGPU,SANA-Video2.0matchesfull-softmaxvideoDiTsinqualitywhileretainingthefavorablelong-sequencescalingoflinearattention.Toavoidquadraticattentionthroughout,HybridLinear-SoftmaxAttentioncombinesgatedlinearattentionforO(N)-dominatedmixingwithperiodicgated-softmaxanchorsata3:1ratio,restoringthefull-ranktokeninteractionsthatpurelinearattentionlacks.Topropagatetheserefreshedrepresentationsacrossdepth,BlockAttentionResiduals(AttnRes)routecompletedblocksummariesintolaterlinearlayers,enablinganchor-featurereuseandboostingdeep-layereffectiverankby~12%.Throughfrom-scratchtraining,SANA-Video2.0learnsthecompletehybriddirectlyratherthanlinearizingpretrainedmodels,withreduced-resolutionproxystudiesestablishing25%softmaxastheoptimalquality-efficiencytrade-off.With40-stepsampling,SANA-Video2.0achievesaVBenchscoreof84.30in13.2sat480ponasingleH100,remainingcompetitivewithfarlargersoftmaxvideoDiTsatafractionofthelatency.ItscompiledDiTforwardpassis3.2xfasterthanamatchedfull-softmaxbaselineat720p/60s,agapthatexpandswithvideoduration.Furthermore,full-stackSol-Engineoptimization(kernelfusion,caching,andsparseattention)acceleratesthishardware-friendlybackbonebyafurther3.58x,bringingthe5Bpipelineto13.06sat720p/5sandmakingit120xfasterthanWan2.2-A14BononeH100.Overall,ourhybriddesignrecoverssoftmax-levelexpressivenessatsubstantiallyreducedcost,unlockingscalablelong,highresolutionvideogeneration.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2607\.21553

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.21553 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.21553 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.21553 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Nvidia's New Long-Form Video Generation (12 minute read)

TLDR AI

NVIDIA Research introduces SANA-Video 2.0, a hybrid video diffusion transformer that generates high-quality 720p video on a single GPU, achieving up to 120× speedup over Wan 2.2-14B via hybrid linear-softmax attention and block attention residuals.

Efficient-Large-Model/SANA-WM_bidirectional

Hugging Face Models Trending

SANA-WM is an efficient 2.6B-parameter open-source world model for minute-scale video generation with precise camera control. It uses a hybrid linear diffusion transformer and a two-stage pipeline to produce 720p videos from images and text prompts.

FVAttn: Adaptive Sparse Attention with Runtime Load Balancing for Video Generation

Hugging Face Daily Papers

FVAttn is a training-free sparse attention system that uses runtime load balancing to improve distributed execution efficiency of adaptive sparse attention under multi-GPU sequence parallelism for video generation, achieving up to 4.41x attention speedup and 2.11x inference speedup over FlashAttention on step-distilled Wan2.2 I2V while maintaining competitive video quality.