@Tono_Ken3: Oh man, I did it! It went off—DeepSeek-V4-Flash-FP8 8 parallel aggregate 400TPS!! Local LLM revolution yesssssss lol
Summary
Achieved 400 tokens per second with DeepSeek-V4-Flash-FP8 using 8 parallel aggregates on local hardware, marking a significant milestone for local LLM inference.
Similar Articles
Deepseek V4 Flash ~105 t/s on two Nvidia 4090d 48G (ada) in vLLM
Technical post detailing how to run DeepSeek V4 Flash on two Nvidia 4090d GPUs using custom Triton kernels and vLLM, achieving ~105 tokens/second with 262k context.
Here are some tips on hitting nearly 200 tok/s for DeepSeek v4 Flash on Hopper
This blog post provides tips and benchmarks for achieving nearly 200 tokens per second inference on DeepSeek V4 Flash using vLLM on a dual GH200 workstation, highlighting the use of a quantized checkpoint from Canada-Quant and tensor parallelism optimizations.
@Tono_Ken3: On GPU, driving Qwen3.6-27b (100 TPS) and on CPU, Hy3-299B (25 TPS) simultaneously The dawn of a new era in local LLM i…
Demonstration of running Qwen3.6-27b on GPU at 100 TPS and Hy3-299B on CPU at 25 TPS simultaneously, marking a milestone in local LLM inference.
Running DeepSeek-V4 locally with 4x legacy RTX 2080 Ti ($2k budget setup). Custom Turing kernels, W8A8 quantization, and 255 prefill tok/s!
A developer successfully runs DeepSeek-V4-Flash (284B total, 13B active) locally on four RTX 2080 Ti GPUs with a $2,500 budget, achieving 255 prefill tokens/s using custom Turing CUDA kernels, W8A8 quantization, and heterogeneous inference. The implementation is open-sourced.
DeepSeek V4 Flash (98GB) on 1x 4060ti + CPU got 300% faster this week [ 2->7t/s]
DeepSeek V4 Flash (98GB) now runs up to 7 tokens per second on a single RTX 4060 Ti with CPU offloading, a 3x speed improvement over the previous week's 2 t/s.