DeepSeek V4.1 Flash: Large-Scale Inference Deployment (15 minute read)
Summary
DeepSeek V4.1 Flash 的大规模推理部署文章，介绍 DeepInfer 结合分布式推理、缓存、负载均衡与投机解码提升效率，DeepSeek 单集群日处理 155 万亿 token，每 GPU 小时收入 38 美元，并开源了大部分推理框架。
Similar Articles
DeepSeek-V4.1-Flash-1017 at 41 tok/s on one 3090
Delta Inference Engine 是一个 Windows 原生推理引擎，可在单张消费级 NVIDIA GPU 上以 2-bit 量化运行 5520 亿参数的 DeepSeek-V4.1-Flash-1017，RTX 3090 上生成速度达 41 tok/s，并提供 OpenAI 兼容 API。
@jakevin7: Were it not for DeepSeek, the ages would be an endless night. DeepSeek has fully open-sourced DeepInfer, its internal inference framework used to run production services. This is the high-performance inference engine they use to power the DeepSeek-V4.1-Flash model, focused on high throughput and low latency.
DeepSeek has fully open-sourced DeepInfer, its internal production-grade inference framework used to power the DeepSeek-V4.1-Flash model, emphasizing high throughput and low latency. They also open-sourced the deepseek-recipe component to ensure community deployments behave consistently with the official API.
DeepInfer: High-performance inference framework for DeepSeek models
DeepSeek open-sources DeepInfer, the high-performance inference framework powering its large-scale production services, featuring optimized kernels, MoE expert parallelism, speculative decoding, batch-invariant inference, and support for NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.
@scaling01: DeepSeek just made their inference ~5x cheaper at 50 TPS
DeepSeek has reduced inference costs by approximately 5x while maintaining 50 tokens per second throughput.
DeepSeek-V4-Flash 284B on 5.3GB of memory
A developer showcases Mference, a new inference engine that runs MoE models like DeepSeek-V4-Flash on just ~5.3GB of memory by streaming experts from SSD, with a native Mac app and OpenAI-compatible server.