Benchmarks of Gemma 4 and Qwen 3.6 MoE models on AMD 6800H iGPU using llama.cpp Vulkan backend show that MoE models and lower-bit quantization (Q4_0) provide the best performance, while Q8_0 is too slow for large models.
I’ve been doing some benchmarking on my mini-PC setup (AMD Ryzen 7 6800H) to see how it handles the new Gemma 4 and Qwen 3.6 MoE models using the llama.cpp Kubuntu with Vulkan backend. Since this is an APU, I’m relying entirely on Shared System Memory (UMA) and the Radeon 680M iGPU. No dedicated VRAM here—just the iGPU pulling from the system RAM. I've varied allocation of memory from 1gb to 16gb and doesn't affect inference. I ran several quantization types (NVFP4, Q4_K, Q8_0) to see where the "sweet spot" is for performance vs. intelligence. Here are the results: Benchmark Results (t/s) Sorted by Model Size model size params pp512 (t/s) tg128 (t/s) gpt-oss 20B Q6_K 11.20 GiB 20.91 B 353.87 16.85 gemma4 26B.A4B Q4_0 13.26 GiB 25.23 B 312.67 18.35 gemma4 26B.A4B MXFP4 MoE 15.40 GiB 25.23 B 261.32 11.93 gemma4 26B.A4B Q4_K - Medium 15.77 GiB 25.23 B 258.16 11.92 gemma4 26B.A4B NVFP4 16.45 GiB 25.23 B 152.35 7.53 gemma4 31B Q8_0 16.74 GiB 30.70 B 30.26 2.30 qwen35moe 35B.A3B NVFP4 19.07 GiB 35.51 B 153.75 15.05 Key Observations: 1. The "MoE" Advantage (Qwen 3.6 35B): Even though the Qwen 3.6 35B MoE model is the largest in terms of total parameters (35.51B), it significantly outperformed the 31B Q8_0 model in decoding speed (15.05 t/s vs 2.30 t/s). This proves that for iGPU/APU users, Mixture of Experts (MoE) is the way to go. You get the "knowledge" of a large model with the inference speed of a much smaller one. 2. Quantization & Memory Bandwidth: The performance drop-off from Q4_0 to Q8_0 is massive. While Q4_0 provides a very usable ~18 t/s, the Q8_0 on the 31B model drops to a crawl (2.3 t/s). Since we are limited by system memory bandwidth (DDR5/LPDDR5), keeping the model footprint smaller is crucial. 3. NVFP4/MXFP4 Performance: The newer FP4 formats are interesting. The gemma4 26B NVFP4 was actually slower than the standard Q4_0 in my tests. It seems there's still some optimization work needed for these specific formats on the Vulkan/AMD driver stack to truly see the efficiency gains. 4. Summary for APU Users: If you are running on a 6800H or similar: Avoid Q8 quantization for anything over 20B params. Target MoE models to get high-parameter intelligence with usable speeds. Stick to 4-bit (Q4_K or Q4_0) to keep your decoding speed above that "reading speed" threshold. System Specs: CPU/iGPU: AMD Ryzen 7 6800H (Radeon 680M) Backend: llama.cpp (Vulkan) OS: Linux/Kubuntu 26.04 https://preview.redd.it/j0mbrwo5ptfh1.png?width=720&format=png&auto=webp&s=ef6e600ba8d84604a91c08df639f98dd63066c81
A developer demonstrates running MoE models like Qwen 3.6 35B-A3B and Gemma 4 26B-A4B at 24+ tok/s on an old GTX 1080 (8GB VRAM) with 128k context using llama.cpp with MoE offloading and TurboQuant KV cache quantization, revealing optimization tricks for Gemma's MTP speculative decoding.
This article benchmarks and compares the performance of Qwen3.8-27B, Qwen3.6-27B, and Gemma 4 31B on a 24GB GPU, recommending Qwen3.8-27B as the best default for most users due to superior coding and reasoning capabilities.
A user compares Qwen3.6 35B-A3B and Gemma 4 26B-A4B-IT running locally on a 16GB VRAM GPU via LM Studio, finding Qwen3.6 produces more detailed outputs while both run at comparable speeds. The post is an informal community comparison using quantized models.
A hands-on benchmark of four local LLMs—Qwen3.6-27B, Qwen3.6-35B, Qwen3.5-27B and Gemma 4—on a 20k-token architecture-writing task shows Qwen3.6-27B delivering the best overall balance of clarity, completeness and usefulness on an RTX 5090.
Personal benchmark shows Qwen3.5-27B Dense and Gemma4-31B Dense fix 100 % of 37 test failures, outperforming Gemma4-26B MoE even at 8-bit quantization, while using fewer tokens and less wall-clock time.