Building a 4x R9700 setup for a 10 person startup
Summary
A user shares a 4x AMD Radeon R9700 AI Pro workstation build (128GB total VRAM, Threadripper 9970x) serving Qwen 3.8 Next Flash/27B and DSV4 Flash for a 10-person startup, reporting strong aggregate prefill/decode throughput across 16 concurrent sessions.
Similar Articles
4x R9700 builds with $127 used EPYCs: $10.7k all-in per machine. I built three (384GB VRAM)
A builder shares three self-hosted 4x Radeon AI PRO R9700 inference servers (~$10.7k each, 384GB VRAM total) running Qwen3.8-27B via vLLM-Radiance/ROCm, with detailed throughput, latency, and long-context benchmarks after hosted models began refusing firearm-industry research tasks.
4x 6000 Pro for the company
A user shares how they serve 40 employees (10 developers) with a fully local AI stack powered by a 4U server with 4x RTX 6000 Blackwell MaxQ GPUs (384GB VRAM), running DeepSeek V4 Flash via vLLM, Qwen 3.8 27B and K2-Horizon 7B via llama.cpp, plus n8n automation, OpenWebUI, Whisper/Vexa AI Teams transcription, and ComfyUI for marketing.
Qwen3.8-Flash-Next turns 4xR9700 into a local AI powerhouse! 120 t/s TG and 12k t/s PP single request with optimized vLLM
The article reports that the Qwen3.8-Flash-Next model achieves 120 tokens/second generation speed and 12k tokens/second prefill on a system with 4x AMD R9700 GPUs using optimized vLLM and a custom Docker image.
4xR9700, 2xMi210 or 4x4080S 32G
The user is comparing GPU options like R9700, Mi210, and 4080S to achieve 128GB VRAM for running multiple AI models in parallel, considering factors like cost, performance, and compatibility.
$2800 rig with 8x Radeon Pro V620 (256 GB VRAM) + custom vLLM fork = Qwen3.8-Flash-Next at 60 to 100 t/s decode and 3000+ t/s prefill
一位 Reddit 用户用 8 张 Radeon Pro V620（256GB VRAM）和自定义 vLLM fork，让 Qwen3.8-Flash-Next 达到 60-100 t/s 解码速度和 3000+ t/s 预填充，比 llama.cpp 快约 800%。