Optimized performance for running the Qwen3.8 27b NVFP4 model on a single AMD Radeon R9700 GPU, achieving up to 153 tokens per second in decode and 3,619 tokens per second in prefill with improved concurrency.
People kept commenting and asking about single AMD 1xR9700 cards in the comments and discord. Well, I finally had time to do some optimizations for 1xR9700 owners and performance has doubled across the board. You can see the results in BetterBench above if you like visuals or below if you're more into text. These results were measured running Unsloth's Qwen3.8 27b NVFP4. Decode ┌───────────────┬───────────────┬──────────────────┐ │ category │ update p50 ms │ decode t/s (med) │ ├───────────────┼───────────────┼──────────────────┤ │ chat │ 42.3 │ 67.1 │ ├───────────────┼───────────────┼──────────────────┤ │ code │ 42.5 │ 120.5 │ ├───────────────┼───────────────┼──────────────────┤ │ file_edit │ 42.5 │ 138.0 │ ├───────────────┼───────────────┼──────────────────┤ │ json │ 42.4 │ 153.1 │ ├───────────────┼───────────────┼──────────────────┤ │ math │ 42.5 │ 140.0 │ ├───────────────┼───────────────┼──────────────────┤ │ prose │ 42.3 │ 69.2 │ ├───────────────┼───────────────┼──────────────────┤ │ reasoning │ 34.3 │ 123.9 │ ├───────────────┼───────────────┼──────────────────┤ │ summarization │ 34.2 │ 141.7 │ └───────────────┴───────────────┴──────────────────┘ Prefill ┌───────────────┬───────────────┐ │ prefill depth │ pp tok/s │ ├───────────────┼───────────────| │ 2000 │ 3552 │ ├───────────────┼───────────────| │ 8000 │ 3536 │ ├───────────────┼───────────────| │ 16000 │ 3619 │ ├───────────────┼───────────────| │ 32000 │ 3437 │ ├───────────────┼───────────────| │ 64000 │ 3192 │ ├───────────────┼───────────────| Concurrency ┌───────────────┬───────────────┐ │ level │ tok/s │ ├───────────────┼───────────────| │ 1 │ 120 │ ├───────────────┼───────────────| │ 2 │ 215 │ ├───────────────┼───────────────| │ 4 │ 322 │ ├───────────────┼───────────────| │ 8 │ 471 │ ├───────────────┼───────────────| Links (Both repo's updated as some users wanted Github) https://codeberg.org/ggz14/radiance-vllm-mxfp4 https://github.com/GGZ14/vllm-mxfp4 https://x.com/bkuyper I hope you single R9700 card owners enjoy this release!
The article reports that the Qwen3.8-Flash-Next model achieves 120 tokens/second generation speed and 12k tokens/second prefill on a system with 4x AMD R9700 GPUs using optimized vLLM and a custom Docker image.
A user demonstrates running the Qwen 27b AI model quantized to Q3_K_M on two RX 580 GPUs, achieving 7.39 tokens per second using old DDR3 hardware for under $100.
The article details an experiment achieving 50 tokens per second inference with Qwen3.8-27B at 256K context on a 24GB GPU using Multi-Token Prediction and custom optimizations.
User reports running Qwen 3.6 35B A3B-Q8_0 gguf on a Radeon 7600 with llama.cpp and ROCm, achieving 21 tokens per second after VRAM overclocking, with a note about a display-related performance bug.
A user reports achieving 125 tokens per second running Qwen3.6 q4xl on two RTX 4060 Ti GPUs, highlighting excellent performance per dollar and wondering if further optimization can reach 150 tok/s.