Tag
SAS introduces a gated sparse attention mechanism that optimizes context ranking end-to-end with language modeling loss, improving performance in reasoning and long-context tasks under tight attention budgets.
Sol-Attn introduces a training-free method to sparsify attention for video generation inference, achieving over 2x speedup by dynamically selecting key-value blocks during online softmax with minimal quality loss.