QFN llama.cpp Any juice left to squeeze?
Summary
User asks for optimal llama.cpp configuration advice for running a heavily quantized Qwen 3.8 Flash Next model on 2x RTX 5060 Ti GPUs, sharing their current setup and performance benchmarks.
Similar Articles
Qwen 3.8 Flash Next-GSQ-RCO-IQ2_XS at ~21 tok/s on just an RTX 3060 12GB + 16GB DDR4 RAM(No gate pruning, 100% bit-exact)
A developer showcases a custom `--moe-direct-io` feature for llama.cpp that achieves 10–15x speedup (20–24 tok/s) running the 125B MoE Qwen3.5-Flash Next (IQ2_XS, 68GB) on just an RTX 3060 12GB with 16GB RAM, while keeping output bit-exact to stock.
Running Qwen 3.8 next on 16vram+32ram - A useful/fun post for the gpu poors
A Reddit user shares how they successfully ran the Qwen 3.8 Next MoE model on a system with 16GB VRAM and 32GB RAM using aggressive quantization and specific llama.cpp settings, achieving usable performance for large models on limited hardware.
TQwen 3.8 flash next ud1s on 6gb vram and 16 gb system ram
A user shares their experience running the Qwen 3.8 flash next model on a system with 6GB VRAM and 16GB RAM using llama.cpp, achieving 6-7 tokens per second with 1-bit quantization, and asks for recommendations on quantization variants.
Qwen3.8-Flash-Next (UD-IQ4_XS) on 2x RTX 3060 + 7800X3D, from initial 36 tps prefill to 400 tps and other benchmarks (-sm tensor trap) + VRAM/RAM usage
The article benchmarks llama.cpp and ik_llama.cpp for running the Qwen3.8-Flash-Next model on dual RTX 3060 hardware, showing that optimizing tensor splitting and ubatch settings can dramatically improve prefill speed from 36 t/s to 400 t/s, with comparisons of RAM usage.
Best config for Qwen3.6 27b / llama.cpp / opencode
Community thread sharing optimized llama.cpp launch commands for running the 27B Qwen3.6 GGUF model with long 100K-512K context on multi-GPU setups.