@Necokeine: If you're not in the MaaS business, you might not be aware — DeepSeek's open-source DeepInfer is actually a framework that's been very deeply customized for DeepSeek 4.1F. For example, DeepInfer's task scheduling doesn't consider VRAM capacity at all. Why? Because DS 4.1F's KV cache is extremely optimized…

X AI KOLs Following News
deepseek inference-optimization kv-cache mlops kimi model-serving open-source

Summary

The discussion points out that DeepSeek's open-source DeepInfer is an inference framework deeply customized for DeepSeek 4.1F. Its KV cache is extremely optimized at just 890 bytes per token, meaning task scheduling never needs to account for VRAM capacity. In contrast, the Kimi series models have KV cache overhead 15–30x higher, resulting in poor inference performance and user experience.

If you're not in the MaaS business, you might not be aware — DeepSeek's open-source DeepInfer is actually a framework that has been very deeply customized for DeepSeek 4.1F. For example, DeepInfer's task scheduling doesn't take VRAM capacity into account at all. Why? Because DS 4.1F's KV cache is extremely optimized — each token only requires 890 bytes, which means even for a full 1M-token context, you'd only need 890MB, less than 1GB of VRAM. This is virtually impossible with other vendors; even DS 4F requires four times that amount. It's precisely because the KV cache is this small that DeepInfer can completely ignore VRAM and simply limit batch size. Setting aside the massive model size of Kimi K3.1 and all the complications that come with its huge number of activated parameters — back in the K3 era, each token required 27kB under bf16, roughly 30x the overhead, and even with FP8 it was still 15x. With such a huge difference in KV cache footprint, combined with Kimi's long-context focus… running inference work is extremely difficult.
Original Article
View Cached Full Text

Cached at: 10/28/26, 05:38 PM

For those not working in MaaS, they might not be aware — but DeepSeek’s open-source DeepInfer is actually deeply customized specifically for DeepSeek 4.1F. For example, DeepInfer’s task scheduling completely ignores VRAM capacity. Why? Because DS 4.1F’s KV cache is extremely optimized — it only requires 890 bytes per token. That means even a full 1M-token context only needs 890MB, less than 1GB of VRAM. That’s virtually impossible with other models — even DS 4F needs four times that amount. Thanks to such a small KV cache, DeepInfer can completely disregard VRAM constraints and simply limit batch size.

Setting aside the enormous model size and the various headaches caused by Kimi K3.1’s massive number of activated parameters — back in the K3 era, each token required 27kB under bf16, roughly 30x the overhead, and even with FP8 it was still 15x. Combined with Kimi’s emphasis on long-context capabilities, this huge KV cache difference makes inference work extremely difficult to implement.

Yunchen Cyan (@CyanCloud77): Can’t Kimi even copy DeepSeek’s inference framework?

Kimi’s problem is that their infra is terrible — meaning inference performance is terrible.

This causes subscription users’ quotas to deplete at lightning speed, yet they keep making all sorts of marketing noise.

It’s clearly a very strong model 😑😑😑

Similar Articles

@jakevin7: Were it not for DeepSeek, the ages would be an endless night. DeepSeek has fully open-sourced DeepInfer, its internal inference framework used to run production services. This is the high-performance inference engine they use to power the DeepSeek-V4.1-Flash model, focused on high throughput and low latency.

X AI KOLs Following

DeepSeek has fully open-sourced DeepInfer, its internal production-grade inference framework used to power the DeepSeek-V4.1-Flash model, emphasizing high throughput and low latency. They also open-sourced the deepseek-recipe component to ensure community deployments behave consistently with the official API.

DeepInfer: High-performance inference framework for DeepSeek models

Hacker News Top

DeepSeek open-sources DeepInfer, the high-performance inference framework powering its large-scale production services, featuring optimized kernels, MoE expert parallelism, speculative decoding, batch-invariant inference, and support for NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.

@karminski3: Magic! DeepSeekV4 context memory compressed to 1/10! Everyone knows DeepSeekV4 supports 1M context and is heavily optimized. To actually use 1M context, VRAM usage is only about 10GB (compared to DeepSeek-V3.2 which needs about…

X AI KOLs Following

FlashMemory-DeepSeek-V4 proposes a novel inference paradigm called Lookahead Sparse Attention (LSA), which uses a neural memory indexer to actively predict future context needs, compressing physical KV cache usage to 13.5% of full context baseline while improving average accuracy by 0.6%. This method adopts a decoupled training strategy that allows independent training of the indexer without loading the base model, significantly reducing training cost.