DoSA: Dots Sparse Attention

Hugging Face Daily Papers Papers
sparse-attention inference-optimization llm-efficiency top-k-selection gpu-acceleration token-selection

Summary

论文提出 DoSA（Dots Sparse Attention），一种无损稀疏注意力方法，通过阈值证书使 indexer 打分计算变为条件式执行，同时保持精确 token 选择；GPU 实现较 SGLang 加速最多 2.74 倍，并在 GLM-5.2 和 DeepSeek-V3.2 上实现最高 1.94 倍的首 token 时间加速。

Sparse attention visits few keys, yet finding them can still require scoring every key with every indexer head. We introduce DoSA (Dots Sparse Attention), a lossless method for making this computation conditional while preserving exact token selection. Its central observation is that Top-K selection requires sufficient evidence to reject a candidate, not necessarily its full score. A seed set supplies a lower bound on the winning threshold. Nonnegative head projections and joint residual bounds upper-bound positive contributions, while activation-informed aggregation bounds the magnitude subtracted by negative heads. The certificate directs full scoring toward candidates that remain eligible; exact-score filtering further reduces writeback and selection work. We establish the certificate and its exact-selection conditions, and analyze how geometry and selection margins govern its effectiveness. Our GPU implementation accelerates the complete Indexer by up to 2.74 times over SGLang and 1.69 times over LiteTopK across English, Chinese, and code workloads. End-to-end evaluation on GLM-5.2 and DeepSeek-V3.2 achieves up to 1.94 times and 1.50 times time-to-first-token speedups over the two baselines, respectively.
Original Article
View Cached Full Text

Cached at: 10/14/26, 02:49 AM

Paper page - DoSA: Dots Sparse Attention

Source: https://huggingface.co/papers/2610.13990

Abstract

Sparseattentionvisitsfewkeys,yetfindingthemcanstillrequirescoringeverykeywitheveryindexerhead.WeintroduceDoSA(DotsSparseAttention),alosslessmethodformakingthiscomputationconditionalwhilepreservingexacttokenselection.ItscentralobservationisthatTop-Kselectionrequiressufficientevidencetorejectacandidate,notnecessarilyitsfullscore.Aseedsetsuppliesalowerboundonthewinningthreshold.Nonnegativeheadprojectionsandjointresidualboundsupper-boundpositivecontributions,whileactivation-informedaggregationboundsthemagnitudesubtractedbynegativeheads.Thecertificatedirectsfullscoringtowardcandidatesthatremaineligible;exact-scorefilteringfurtherreduceswritebackandselectionwork.Weestablishthecertificateanditsexact-selectionconditions,andanalyzehowgeometryandselectionmarginsgovernitseffectiveness.OurGPUimplementationacceleratesthecompleteIndexerbyupto2.74timesoverSGLangand1.69timesoverLiteTopKacrossEnglish,Chinese,andcodeworkloads.End-to-endevaluationonGLM-5.2andDeepSeek-V3.2achievesupto1.94timesand1.50timestime-to-first-tokenspeedupsoverthetwobaselines,respectively.

View arXiv pageView PDFGitHub5Add to collection

Get this paper in your agent:

hf papers read 2610\.13990

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.13990 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.13990 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.13990 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Simplified Sparse Attention via Gist Tokens

Hugging Face Daily Papers

This paper introduces Simplified Sparse Attention (SSA), a method that uses gist tokens during continued pretraining to enable efficient chunk selection at inference without architectural changes, achieving high compression ratios and outperforming baselines on long-context tasks like LongBench and retrieval-augmented generation.

SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference

arXiv cs.CL

SparDA proposes a decoupled sparse attention architecture that adds a lightweight 'Forecast' projection to predict future KV cache needs, enabling lookahead prefetching from CPU to GPU and reducing selection overhead. On 8B sparse-pretrained models, it achieves up to 1.25× prefill and 1.7× decode speedup, with up to 5.3× higher decode throughput over non-offload baselines.