Qwen3.8-Flash-Next-GSQ-RCO (IQ3_S): ~20-30 tok/sec decode & 300-90k tok/sec prefill on 12GB VRAM + 32GB RAM + NVME
Summary
用户在仅 12GB 显存 + 32GB 内存 + NVMe 的消费级硬件上，通过自定义 Strata 分支运行 IQ3_S 量化版 Qwen3.8-Flash-Next-GSQ-RCO-Abliterated，实现 20-30 tok/s 解码与最高约 90k tok/s 的 131k 上下文预填性能。
Similar Articles
Strata on a power limited 5090 and 96GB of DDR5-6400 is cranking out 150-200 tok/s decode and 5-6k prefill! Qwen3.8-Flash-Next at IQ3_S, CTX at 128k tokens (8-bit).
Strata runs Qwen3.8-Flash-Next at IQ3_S quantization on a power-limited RTX 5090 with 96GB of DDR5-6400, achieving 150-200 tok/s decode and 5-6k prefill with a 128k-token context (8-bit).
Qwen3.8-Flash-Next (125B) at 12-15 tok/s on a 2021 32GB M1 Max
MoEspresso 3, an inference engine, achieves 12-15 decode tokens per second for the Qwen3.8-Flash-Next model on a 2021 M1 Max with 32GB RAM, using techniques like Cache-Prior to optimize performance and outperforming some hosted solutions in benchmarks.
176 t/s decode and 6,450 t/s prefill with Q4 Qwen-3.8 Flash-Next on 2xR9700 + 64GB DDR5 using Deadcode's Radiance engine
A user benchmarks Qwen-3.8 Flash-Next in Q4 quantization running on Deadcode's Radiance inference engine, achieving 176 t/s decode and 6,450 t/s prefill on a custom PC with 2xR9700 GPUs and 64GB DDR5, using the BetterBench throughput tool.
Qwen3.8-Flash-Next NVFP4 2xDGX Spark config: 50t/s decode, 2,900t/s prefill
A user shares their optimized configuration for deploying the Qwen3.8-Flash-Next model on dual DGX Spark hardware, achieving up to 50t/s decode and 2,900t/s prefill speeds with technical patches and setup details.
Qwen 3.8 Flash Next-GSQ-RCO-IQ2_XS at ~21 tok/s on just an RTX 3060 12GB + 16GB DDR4 RAM(No gate pruning, 100% bit-exact)
A developer showcases a custom `--moe-direct-io` feature for llama.cpp that achieves 10–15x speedup (20–24 tok/s) running the 125B MoE Qwen3.5-Flash Next (IQ2_XS, 68GB) on just an RTX 3060 12GB with 16GB RAM, while keeping output bit-exact to stock.