@songhan_mit: Speed-of-light block sparse attention :
Summary
Sol-Engine weekly update announces integration of Sol Attention, a training-free sparse attention method for video diffusion, with full paper coming next week.
View Cached Full Text
Cached at: 07/21/26, 04:47 PM
Speed-of-light block sparse attention :
Enze Xie (@xieenze_jr): 🚀 Sol-Engine weekly update is here!
This week we released some new things:
• Integrated Sol Attention — our team’s new training-free sparse attention method for video diffusion (full paper coming next week 🚀). It’s already available in the engine as an early easter egg.
•
Similar Articles
Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification
Sol-Attn introduces a training-free method to sparsify attention for video generation inference, achieving over 2x speedup by dynamically selecting key-value blocks during online softmax with minimal quality loss.
Faster Video Diffusion with Trainable Sparse Attention
This paper introduces Trainable Sparse Attention (VSA), a hardware-efficient sparse attention mechanism that reduces computational costs in video diffusion transformers without compromising performance, enabling more efficient scaling and faster generation.
LVSA: Training-Free Sparse Attention for Long Video Diffusion
LVSA introduces a training-free sparse attention mechanism for video diffusion models, reducing compute up to 3.17x while enabling generation beyond training horizons without quality loss.
FreeSolo: Linear-time Sparse Attention with Zero-Shot Length Extrapolation
FreeSolo introduces a native sparse attention mechanism combining short-range sliding window and long-range keypoint-based branches, achieving linear-time complexity and improved zero-shot length extrapolation to 256k+ tokens without costly long-context fine-tuning.
Sparsity Should Be ALLOTted by Output: Training-Free Allocation of Block-Sparse Attention for Video Diffusion Transformers
提出 ALLOT：一种训练自由的块稀疏注意力分配方法，通过细胞探测与轨迹校准在延迟预算内优化视频扩散 Transformer 各层-步的稀疏度，在 HunyuanVideo 上实现 11.8 dB PSNR 提升。