I’m the guy who posted the 4× R9700 builds with $127 used EPYCs (https://www.reddit.com/r/LocalLLaMA/comments/1x67qeu/4x_r9700_builds_with_127_used_epycs_107k_allin/). After seeing the Nanbeige benchmark results here, I spent the past few days testing it against Qwen on the work these machines actually do. Jobs > TPS My business, USA Arms, maintains information about firearms-related businesses and I use local AI to maintain the majority of the information... services, hours, pricing, facilities, etc. Each configuration ran the same 2,484 cases across 25 field level goldens as an auditor with accept, reject, or abstain as options. (ambiguity goldens) Correct Jobs/hour Measurement Qwen 3.8 27B Nanbeige 4.3 33B Processed jobs/hour, thinking off 1,855 1,281 Verified correct jobs/hour, thinking off 1,174 798 Accuracy, thinking off 77.4% 76.2% Accuracy, thinking on 86.6% 82.4% The jobs/hour numbers use 2x 2-GPU engines. Token speed Separate MT-Bench speed sweep, measured on one two-GPU engine (these aren't 4 card numbers): Concurrent streams Qwen output tok/s Nanbeige output tok/s 1 38.4 30.7 4 135.4 112.7 8 246.0 214.0 Auditor The video goes over it, but Qwen actually does a better job auditing itself with thinking mode on than Nanbeige with thinking on auditing the same jobs. In total: Qwen was about 25% faster in the single-stream token test, but delivered about 45% more jobs/hour with thinking off than Nanbeige. Curious whether people testing coding or creative writing are seeing a different result with Nanbeige... once again, hyped model doesn't dethrone the king. Full results with more details on this short video: https://youtu.be/kmmN49ecyvc (I tried to upload direct to reddit but it had issues)
A user reports that the Qwen 3.8 27B model achieves 50-60 tokens per second on dual 5060 TI cards, showing unexpected speed improvements over previous versions like Qwen 3.6.
User reports Qwen 3.5 122B significantly outperforms Qwen 3.6 35B on multi-step tasks despite benchmark claims, questioning if quantization or setup issues are to blame.
A user benchmarks three Qwen models (Qwen3.5-27B dense, Qwen3.5-122B-A10B MoE, Qwen3.6-35B-A3B MoE) on 4x RTX 3090 GPUs under real agentic workloads, finding that MoE models consistently underperform the dense 27B at following strict global rules despite speed advantages, with the Qwen3.6-35B leading in generation throughput.
NVIDIA showcases the high-throughput performance of serving the Qwen3-8B 2.4T parameter model on GB300 NVL72 hardware, achieving over 4k tokens per second per GPU.