@ModelScope2022: 1M-token context with only ~3B parameters active per token. Meituan’s LongCat-Flash-Lite-Sparse brings sparse attention…
Summary
Meituan released LongCat-Flash-Lite-Sparse, a sparse-attention model supporting 1M-token context with only ~3B active parameters per token, achieving strong SWE-Bench scores under an MIT license.
View Cached Full Text
Cached at: 08/03/26, 01:43 PM
1M-token context with only ~3B parameters active per token. Meituan’s LongCat-Flash-Lite-Sparse brings sparse attention to long-context agents. https://modelscope.ai/models/meituan-longcat/LongCat-Flash-Lite-Sparse…
The sparse variant scores 68.2 on SWE-Bench Verified and 59.33 on SWE-Bench Multilingual, up from 54.4 and 38.1 for its dense predecessor. LongCat Sparse Attention combines streaming-aware, cross-layer, and hierarchical indexing to reduce long-context computation while retaining model quality. 69B total parameters. MIT License.
Similar Articles
@eliebakouch: the new sparse attention method introduced with this model is basically a combination of components from existing ones.…
Meituan introduces LongCat-2.0, a 1.6T parameter MoE model with 48B active parameters and 1M context length, featuring a new LongCat Sparse Attention (LSA) method that combines components from existing sparse attention techniques.
@Meituan_LongCat: Introducing LongCat-2.0 1.6T parameters · MoE with ~48B active · 1M context The full model behind Owl Alpha on @OpenRou…
Meituan introduces LongCat-2.0, a 1.6T parameter MoE model with ~48B active parameters and 1M context, featuring novel architectures like LongCat Sparse Attention and Zero-Compute Experts, achieving strong benchmark scores on coding and reasoning tasks.
Meituan launches LongCat-2.0 1.6T parameter model on APIs (2 minute read)
Meituan launched LongCat-2.0, a 1.6 trillion-parameter Mixture-of-Experts model with a 1 million-token context window, available via API for agentic coding, tool use, and complex workflows.
MiniMax teases upcoming M3 model with new sparse attention mechanism and 15.6X long-context response speed boost (12 minute read)
MiniMax has released a detailed technical report on its M2 series and teased the upcoming M3 model, which uses a novel sparse attention mechanism to achieve up to 15.6× faster decoding at million-token contexts.
MiniMax Sparse Attention
MiniMax Sparse Attention introduces a blockwise sparse attention mechanism that achieves significant speedups for ultra-long-context LLMs, reducing per-token attention compute by 28.4x at 1M context with wall-clock speedups of 14.2x for prefill and 7.6x for decoding on H800 GPUs. The method is accompanied by an open-source inference kernel and a publicly released multimodal model.