DeepSeek-V4-Flash-Vision-Exp (285B MoE) on 10-12x RTX 3090 — spec decoding, vision

Reddit r/LocalLLaMA Models
deepseek-v4 moe-model spec-decoding vision-ai rtx-3090 vllm consumer-gpu

Summary

Running the DeepSeek-V4-Flash-Vision-Exp 285B MoE model on 10-12x RTX 3090 GPUs achieves over 60-120 tok/s decode speeds with vision and tool support, fully documented for reproducibility.

Running the full deepseek-ai/DeepSeek-V4-Flash-Vision-Exp on consumer Ampere — 10-12x RTX 3090, SM86-compatible vLLM build. 285B MoE, FP4 experts + FP8 attention, 157 GB weights. Highlights: - **60+ tok/s** decode, DSpark spec (k=3) on 10 GPUs (TP2xPP5), at a 240 W cap - **120+ tok/s** on 12 GPUs (TP4xPP3) - **Vision + spec + tool calls all working** - **1M context** (no offload) / **4M** (RAM offload) - ~3,500 tok/s long-context prefill Fully documented + reproducible: - Pre-built image: `docker pull ghcr.io/ciprianveg/3090-vllm:dsv4-flash-vision-sm86` - Repo: https://github.com/ciprianveg/3090-vllm (build guide, start scripts, runtime patches) The patches cover the DSpark propose-gate (spec + vision fix), scheduler mm x spec row-crossing, grammar-bitmask validation, vision ViT OOM fix, FlashInfer workspace-lane keying, and more.
Original Article

Similar Articles

Deepseek V4 Flash running on RTX 5090 MoE

Reddit r/LocalLLaMA

User shares optimization benchmarks for DeepSeek-V4-Flash (Q2_K) running on an RTX 5090 using a fork of llama.cpp, achieving 21.3 tokens/s generation and 1 million context size.