@YRSM_Simon: Amazing!
Summary
shi3z shares optimization experience for running DeepSeek v4.1 Flash on an A100 GPU without FP4 support, boosting inference speed from 33 tok/s to 673 tok/s, surpassing the official API speed.
View Cached Full Text
Cached at: 09/14/26, 01:16 AM
Impressive
shi3z (@shi3z): Wrote over 10k characters | The story of getting DeepSeek v4.1 Flash, which I wanted to run, to work on an A100 that doesn’t support FP4, boosting it from 33tok/s to 673tok/s, and before I knew it, it was faster than the official API | shi3z @shi3z
Similar Articles
Someone made DSV4.1 run faster than official API on A100(s)
Someone has optimized DeepSeek V4.1 to run faster than the official API on NVIDIA A100 GPUs, despite A100 not supporting FP4 precision, and shared the implementation on GitHub.
Deepseek V4 Flash running on RTX 5090 MoE
User shares optimization benchmarks for DeepSeek-V4-Flash (Q2_K) running on an RTX 5090 using a fork of llama.cpp, achieving 21.3 tokens/s generation and 1 million context size.
DeepSeek-V4-Flash W4A16+FP8 with MTP self-speculation: 85 tok/s @ 524k on 2× RTX PRO 6000 Max-Q
The article details a customized quantized version of DeepSeek-V4-Flash with MTP self-speculation enabled, achieving significant speedups on dual RTX PRO 6000 Max-Q GPUs using a patched vLLM setup.
@superalesha: I sped up deepseek v4 flash by 29x on my 4x3090s !!! No, its not joke. 15 -> 443 t/s. a 23k prompt used to take 25 mins…
A user achieved a 29x speedup for DeepSeek V4 Flash inference on 4x RTX 3090 GPUs by optimizing llama.cpp, reducing a 23k prompt from 25 minutes to 53 seconds.
DeepSeek v4 Flash on 4090 + DDR5, my experience
A user shares their experience running the DeepSeek v4 Flash model with a 24GB GPU and DDR5 RAM, including performance numbers and tips for optimization.