Deepseek v4 flash - 100-150 faster t/s in prefill/pp.
Summary
This post shares fixes to improve DeepSeek v4 Flash prefill/PP speed: downgrading CUDA from 13.3 to 13.1 or using a custom fork, achieving up to 1.3K prompt processing tokens/s.
Similar Articles
Deepseek V4 Flash ~105 t/s on two Nvidia 4090d 48G (ada) in vLLM
Technical post detailing how to run DeepSeek V4 Flash on two Nvidia 4090d GPUs using custom Triton kernels and vLLM, achieving ~105 tokens/second with 262k context.
@superalesha: I sped up deepseek v4 flash by 29x on my 4x3090s !!! No, its not joke. 15 -> 443 t/s. a 23k prompt used to take 25 mins…
A user achieved a 29x speedup for DeepSeek V4 Flash inference on 4x RTX 3090 GPUs by optimizing llama.cpp, reducing a 23k prompt from 25 minutes to 53 seconds.
DeepSeek V4 Flash (98GB) on 1x 4060ti + CPU got 300% faster this week [ 2->7t/s]
DeepSeek V4 Flash (98GB) now runs up to 7 tokens per second on a single RTX 4060 Ti with CPU offloading, a 3x speed improvement over the previous week's 2 t/s.
@malikwas1f: The four-line fix took DeepSeek-V4-Flash prefill from 127 → 312 t/s @10K and 91 → 283 @40K, with a passed recall check …
A four-line fix reportedly boosts DeepSeek-V4-Flash prefill speed from 127 to 312 t/s at 10K context and 91 to 283 t/s at 40K, with decode unchanged. The patch is shared via the club-3090 project for serving LLMs on RTX 3090s.
Deepseek V4 Flash running on RTX 5090 MoE
User shares optimization benchmarks for DeepSeek-V4-Flash (Q2_K) running on an RTX 5090 using a fork of llama.cpp, achieving 21.3 tokens/s generation and 1 million context size.