@no_stp_on_snek: first receipts: triattention v3 evicts safely with longctx. ✓HIT every rung 32k → 256k on qwen3.5-2b-4bit (hybrid mamba…
Summary
Introduces triattention v3, a new attention mechanism that enables safe eviction without recall loss for long-context inference, demonstrated on a hybrid mamba+attention model up to 256k tokens.
Similar Articles
@no_stp_on_snek: vllm-swift 0.6.3 + longctx 0.3.2 are out. highlights: - triattentionv3 + longctx rescue path hits 256k niah on apple si…
vllm-swift 0.6.3 and longctx 0.3.2 releases bring triattentionv3 with 256k context on Apple Silicon, Gemma 4 MTP drafter support, Hermes tool calling with auto-recovery, and a longctx-svc daemon for scaling to 12M-token corpora.
Epiphany-Aware KV Cache Eviction Without the Attention Matrix
This paper introduces EpiKV, a KV cache eviction method that scores token importance via changes in internal representations (epiphany score) instead of attention weights, avoiding the need to materialize the attention matrix. It achieves competitive performance on reasoning benchmarks while enabling up to 16× longer context lengths.
@no_stp_on_snek: small update from the long-context experiments: I got MRCR v2 running out to 1M on a single MI300X droplet with an open…
The author reports successful experiments running MRCR v2 with 1M context length on a single MI300X using Qwen2.5-32B and FAISS, achieving competitive scores at low cost.
Subquadratic AI introduces SubQ-1.1-Small, a new model using Smart Sparse Attention
Subquadratic AI introduces SubQ-1.1-Small, a model leveraging Smart Sparse Attention to achieve near-perfect long-context retrieval up to 12M tokens with up to 1,000x attention compute reduction. It balances long-context optimization with strong general reasoning, outperforming baselines on benchmarks like NIAH and RULER.
@omarsar0: NEW paper worth reading. (bookmark it) The basic idea is to pair a compressive recurrent state with a small exact memor…
HOLA (Hippocampal Linear Attention) augments linear attention with a bounded exact KV cache inspired by hippocampal memory, improving long-range recall and perplexity without sacrificing efficiency. At 340M parameters, it outperforms full-attention Transformers on Wikitext and achieves robust needle recall up to 32k tokens.