@no_stp_on_snek: meanwhile the rest of us just plug away. im trying to solve sparse attention for mlx-swift-lm right now. making good pr…
Summary
Developer reports progress implementing sparse attention for mlx-swift-lm, achieving only +4% overhead vs dense attention on M5 Max.
View Cached Full Text
Cached at: 05/16/26, 07:15 AM
@AlexJonesax meanwhile the rest of us just plug away. im trying to solve sparse attention for mlx-swift-lm right now. making good progress. closed sparse attention to +4% overhead vs dense on m5 max
Similar Articles
@rohanpaul_ai: Quite incredible, MiniMax Sparse Attention cuts attention compute by 28.4X at 1M tokens, with 14.2X faster prefill and …
MiniMax Sparse Attention (MSA) achieves up to 28.4x reduction in attention compute at 1M tokens by adding a routing branch that selectively chooses key-value blocks for attention, enabling 14.2x faster prefill and 7.6x faster decoding on H800 GPUs while matching full attention benchmark performance.
@no_stp_on_snek: anyone still talking about mlx-swift-lm? said i was taking the day off... cleaned the chicken coop, got a workout in, f…
The author describes implementing TurboQuant KV-cache compression into Apple's mlx-swift-lm, achieving 2.7x compression with quality on par with 8-bit, and 3-4x decode speed improvements via a fused Metal kernel.
MiniMax Sparse Attention
MiniMax Sparse Attention introduces a blockwise sparse attention mechanism that achieves significant speedups for ultra-long-context LLMs, reducing per-token attention compute by 28.4x at 1M context with wall-clock speedups of 14.2x for prefill and 7.6x for decoding on H800 GPUs. The method is accompanied by an open-source inference kernel and a publicly released multimodal model.
Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps
RTPurbo converts full-attention LLMs into sparse models with only a few hundred training steps, achieving near-lossless accuracy and up to 9.36x prefill and 2.01x decode speedups.
@no_stp_on_snek: vllm-swift 0.6.3 + longctx 0.3.2 are out. highlights: - triattentionv3 + longctx rescue path hits 256k niah on apple si…
vllm-swift 0.6.3 and longctx 0.3.2 releases bring triattentionv3 with 256k context on Apple Silicon, Gemma 4 MTP drafter support, Hermes tool calling with auto-recovery, and a longctx-svc daemon for scaling to 12M-token corpora.