@ClementDelangue: Totally agree it deserves more upvotes! Here is a link to it if you and others want to upvote it (I don’t think you upv…
Summary
DeepSeek-V4.1-Flash is a multimodal Mixture-of-Experts model with 552B backbone parameters and 1M-token context that achieves major KV cache compression gains via CSA2 and FP4 caching, cutting HBM footprint to ~890 bytes per token while boosting agentic performance.
View Cached Full Text
Cached at: 10/03/26, 09:03 PM
Totally agree it deserves more upvotes! Here is a link to it if you and others want to upvote it (I don’t think you upvoted it): https://huggingface.co/papers/2609.19969…. Also hf papers is work in progress, any idea how we can make it better?
Paper page - DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
Source: https://huggingface.co/papers/2609.19969 Published on Sep 17
#1 Paper of the day Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Thewidespreadadoptionoflong-horizonagentshasmademodelworkloadsincreasinglyinput-heavy.Althoughpriorworkhassubstantiallyreducedthecostoflong-contextcomputation,prefillremainscomputationallyexpensive,andlargeKVcachescontinuetostrainHBMandSSDcapacityanddata-transferbandwidth.Together,thesecompute,storage,andbandwidthdemandsconstitutetheprimarybottlenecktofurtherloweringdeploymentcosts.Toaddressthischallenge,weintroduceDeepSeek-V4.1-Flash,amultimodalMixture-of-Experts(MoE)modelwith552Bbackboneparametersandsupportforcontextsofuptoonemilliontokens.WithitsCausalEncoder-Decoder(CED)architecture,themodelactivates16Bparameterspertokenduringdecodebutonly8Bparametersduringprefill,substantiallyimprovingcostefficiencyforagenticworkloads.TopushthelimitsofKVcachecompression,DeepSeek-V4.1-Flashcombinescross-layerKVcachereuseinCompressedSparseAttention2(CSA2)withFP4KVcaching.ThesedesignsreduceitsglobalKVcachefootprint(alwaysinHBM)to890bytespertoken,roughly1/4ofthecorrespondingfootprintofDeepSeek-V4-Flash.Further,throughadedicateddeploymentoptimizationknownasSWABoundedReplay,DeepSeek-V4.1-FlashreducesitspersistentKVcachefootprint(alwaysonSSDorinhostmemory)toroughly1/8ofthatofDeepSeek-V4-Flash.DespiteitsmuchsmallerKVcachefootprint,themodeldeliverssubstantiallybetterperformancethanthebaseline.Inaddition,westreamlinetheDeepSeek-V4architectureandintroduceseveralefficientarchitecturalextensions.WepretrainDeepSeek-V4.1-Flashonamultimodalcorpuscomprising45Ttokensandconductcomprehensivepost-training,yieldingstrongperformanceacrossdiversetext-basedandmultimodalagenticscenarios.Modelcheckpointsareavailableathttps://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash.
View arXiv pageView PDFAdd to collection
Models citing this paper2
#### enver/rootformer-v13-sovereign-arabic Translation• 0.4B• Updated6 days ago • 49
#### enver/rootformer-v17-deepseek-flash Text Generation• 0.5B• Updated5 days ago • 625
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.19969 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.19969 in a Space README.md to link it from this page.
Collections including this paper13
Similar Articles
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
Introduces DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts model with 552B parameters, featuring advanced KV cache compression techniques to reduce deployment costs and improve efficiency for long-context agent workloads.
deepseek-ai/DeepSeek-V4-Flash-DSpark
DeepSeek releases V4 series of Mixture-of-Experts language models (Pro 1.6T/49B activated, Flash 284B/13B activated) supporting one-million-token context with hybrid attention and speculative decoding, claiming best open-source model performance.
@modal: DeepSeek-V4-Flash has 284B total parameters with 13B active per token. Combined with a hybrid compressed attention mech…
DeepSeek-V4-Flash is a 284B-parameter MoE model with 13B active parameters per token, featuring a hybrid compressed attention mechanism that reduces KV cache needs for 1M-token contexts. It can be served with SGLang on Modal for fast decoding on a single B300.
@nrehiew_: For the visual learners
A tweet describes a large mixture-of-experts model with 975B total parameters (41B active) trained on 45T tokens of multimodal data, featuring 6 routed experts and 2 shared experts, with comparisons to DeepSeek-V3.
deepseek-ai/DeepSeek-V4-Pro
DeepSeek releases V4-Pro and V4-Flash, Mixture-of-Experts models supporting million-token context with hybrid attention and Muon optimizer.