FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM Serving
Summary
FlashPrefill V2 improves long-context LLM serving through mean-corrected sparse attention and optimized GPU operators, delivering substantial speedups over FlashAttention-2 and dense baselines.
View Cached Full Text
Cached at: 08/21/26, 08:09 AM
Paper page - FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM Serving
Source: https://huggingface.co/papers/2608.19758
Abstract
FlashPrefill V2 improves long-context serving via mean-corrected sparse attention, optimized GPU operators, and framework integration, achieving large speedups over dense baselines.
Long-context modelingis a pivotal capability for Large Language Models, yet the quadratic complexity of attention remains a critical bottleneck, particularly during the compute-intensive prefilling phase. Our previous work, FlashPrefill, mitigates this cost through instantaneous pattern discovery and max-based dynamic thresholding; however, it remains an algorithmic prototype that is still distant from production deployment. In this paper, we presentFlashPrefill V2, which evolves FlashPrefill from a prototype toward practical long-context serving along three dimensions. First, we introduce amean correctionterm that effectively suppresses the approximation error, keeping performance degradation manageable even at extreme sparsity levels. Second, we redesign thesparse attentionoperator withPackGQAmemory access,warp specialization, andpingpong pipelining, fully aligning with the latestFlashAttention-3/4implementations and supportingFP8 inferenceto meet practical quantization requirements. Third,FlashPrefill V2natively supportspaged KV cacheandcontinuous batching, allowing integration as an attention backend in modern inference frameworks such as SGLang. Extensive evaluations on NVIDIA H20 GPUs---among the most widely deployed inference accelerators---demonstrate thatFlashPrefill V2delivers up to 47.26x and 27.19x speedups over FlashAttention-2 at 128K context length under FP8 and BF16 precision, respectively, and, in FP8, still achieves a 30.49x speedup against an FA3/4-aligned dense baseline.
View arXiv pageView PDFGitHub1Add to collection
Get this paper in your agent:
hf papers read 2608\.19758
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.19758 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.19758 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.19758 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
UniPrefill: Universal Long-Context Prefill Acceleration via Block-wise Dynamic Sparsification
UniPrefill is a new prefill acceleration framework proposed in a research paper that enables block-wise dynamic sparsification for universal long-context processing in LLMs. It integrates with vLLM to achieve up to 2.1x speedup in Time-To-First-Token across various model architectures.
Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding
Introduces Faster Flash Decoding (FFD), a training-free hardware-algorithm co-design framework that accelerates long-context decoding in LLMs by exploiting attention sparsity, achieving up to 11.6x speedup and scaling to 256K context length.
FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention
Proposes Lookahead Sparse Attention with a Neural Memory Indexer on DeepSeek-V4, reducing GPU memory usage to ~13.5% of full-context baseline while maintaining or slightly improving accuracy.
CompactAttention: Accelerating Chunked Prefill with Block-Union KV Selection
CompactAttention introduces Block-Union KV Selection to accelerate chunked prefill for long-context LLMs, achieving up to 2.72x attention speedup on LLaMA-3.1-8B at 128K context while maintaining accuracy close to dense attention.
SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference
SparDA proposes a decoupled sparse attention architecture that adds a lightweight 'Forecast' projection to predict future KV cache needs, enabling lookahead prefetching from CPU to GPU and reducing selection overhead. On 8B sparse-pretrained models, it achieves up to 1.25× prefill and 1.7× decode speedup, with up to 5.3× higher decode throughput over non-offload baselines.