FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention

Hugging Face Daily Papers Papers
lookahead-sparse-attention kv-cache long-context memory-efficiency deepseek-v4 gpu-memory attention-mechanism

Summary

Proposes Lookahead Sparse Attention with a Neural Memory Indexer on DeepSeek-V4, reducing GPU memory usage to ~13.5% of full-context baseline while maintaining or slightly improving accuracy.

Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose Lookahead Sparse Attention (LSA), a novel inference paradigm powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query-critical KV chunks in the GPU memory. Crucially, we instantiate this architecture via a backbone-free decoupled training strategy. By formulating the indexer as a standard dual-encoder architecture, we train it independently using standard retrieval training frameworks without ever loading the massive backbone model into GPU memory. We demonstrate that this "less is more" paradigm significantly maximizes serving efficiency while acting as an effective attention denoiser in tasks that rely on long-term global memory. Across primary long-context evaluation suites (e.g., LongBench-v2, LongMemEval, and RULER), FM-DS-V4 compresses the average physical KV cache footprint down to merely 13.5% of the full-context baseline, while consistently preserving or slightly elevating downstream accuracy (+0.6% absolute margin on average). Crucially, at extreme 500K scales, FlashMemory suppresses the physical KV cache overhead by over 90% without destabilizing the backbone's core reasoning capacities.
Original Article
View Cached Full Text

Cached at: 06/09/26, 08:42 AM

Paper page - FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention

Source: https://huggingface.co/papers/2606.09079 Authors:

,

,

,

,

,

,

,

,

,

,

,

Abstract

Lookahead Sparse Attention with Neural Memory Indexer reduces GPU memory usage for long-context LLM inference while maintaining accuracy through proactive KV cache management and decoupled training.

ConventionalLLMskeep the fullKV cacheloaded duringdecoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we proposeLookahead Sparse Attention(LSA), a novel inference paradigm powered by aNeural Memory Indexerbuilt upon the DeepSeek-V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query-critical KV chunks in the GPU memory. Crucially, we instantiate this architecture via a backbone-free decoupled training strategy. By formulating the indexer as a standarddual-encoder architecture, we train it independently using standard retrieval training frameworks without ever loading the massive backbone model into GPU memory. We demonstrate that this “less is more” paradigm significantly maximizes serving efficiency while acting as an effectiveattention denoiserin tasks that rely on long-term global memory. Across primarylong-context evaluationsuites (e.g., LongBench-v2, LongMemEval, and RULER), FM-DS-V4 compresses the average physicalKV cachefootprint down to merely 13.5% of the full-context baseline, while consistently preserving or slightly elevating downstream accuracy (+0.6% absolute margin on average). Crucially, at extreme 500K scales,FlashMemorysuppresses the physicalKV cacheoverhead by over 90% without destabilizing the backbone’s core reasoning capacities.

View arXiv pageView PDFGitHub1Add to collection

Get this paper in your agent:

hf papers read 2606\.09079

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### libertywing/FlashMemory-Deepseek-V4 Updatedabout 5 hours ago • 6

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.09079 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.09079 in a Space README.md to link it from this page.

Collections including this paper2

Similar Articles

FlashMemory DeepSeek-V4 Retriever (GitHub Repo)

TLDR AI

Introduces FlashMemory DeepSeek-V4 Retriever, a lightweight model that sparsifies DeepSeek-V4's CSA KV-cache by predicting which chunks will be attended to next, keeping only ~10-15% on-device while matching full-attention performance.

@karminski3: Magic! DeepSeekV4 context memory compressed to 1/10! Everyone knows DeepSeekV4 supports 1M context and is heavily optimized. To actually use 1M context, VRAM usage is only about 10GB (compared to DeepSeek-V3.2 which needs about…

X AI KOLs Following

FlashMemory-DeepSeek-V4 proposes a novel inference paradigm called Lookahead Sparse Attention (LSA), which uses a neural memory indexer to actively predict future context needs, compressing physical KV cache usage to 13.5% of full context baseline while improving average accuracy by 0.6%. This method adopts a decoupled training strategy that allows independent training of the indexer without loading the base model, significantly reducing training cost.

DeepSeek-V4-Flash 284B on 5.3GB of memory

Reddit r/LocalLLaMA

A developer showcases Mference, a new inference engine that runs MoE models like DeepSeek-V4-Flash on just ~5.3GB of memory by streaming experts from SSD, with a native Mac app and OpenAI-compatible server.