Strata with Qwen3.8 Flash Next UD-Q4_K_XL
Summary
A user reports strong single-request decode speeds running Qwen3.8-Flash-Next in Unsloth's UD-Q4_K_XL quant on Strata with an RTX PRO 6000, showing roughly 2x faster throughput than vLLM on the same GPU across prose, counting, coding, and reasoning tasks.
Similar Articles
Strata on a power limited 5090 and 96GB of DDR5-6400 is cranking out 150-200 tok/s decode and 5-6k prefill! Qwen3.8-Flash-Next at IQ3_S, CTX at 128k tokens (8-bit).
Strata runs Qwen3.8-Flash-Next at IQ3_S quantization on a power-limited RTX 5090 with 96GB of DDR5-6400, achieving 150-200 tok/s decode and 5-6k prefill with a 128k-token context (8-bit).
Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s
Strata 是一个免费开源的推理引擎，可在 RTX 4090 等消费级显卡（12GB+ 显存）本地运行 Qwen3.8-Flash-Next（125B）模型，写入速度约 100 tokens/s，且支持 Windows/Linux 和 NVIDIA/AMD 显卡。
Qwen3.8-Flash-Next (UD-IQ4_XS) on 2x RTX 3060 + 7800X3D, from initial 36 tps prefill to 400 tps and other benchmarks (-sm tensor trap) + VRAM/RAM usage
The article benchmarks llama.cpp and ik_llama.cpp for running the Qwen3.8-Flash-Next model on dual RTX 3060 hardware, showing that optimizing tensor splitting and ubatch settings can dramatically improve prefill speed from 36 t/s to 400 t/s, with comparisons of RAM usage.
Qwen3.8-Flash-Next on 2x3090 + DDR4: 17 → 25-29 t/s decode with the expert cache PR
User benchmarks and details a GPU-resident expert cache PR in llama.cpp that boosts decode speed for Qwen3.8-Flash-Next on a dual RTX 3090 system from 17 to 25-29 tokens per second.
I don't get the qwen 3.8 flash next hype
A community member reports that Qwen 3.8 Flash Next (UD-Q4_K_XL on 4x RTX 3090s) underperforms the Qwen 3.8 27B Q8 in a Tetris-building coding task, producing bugs and failing to deliver a playable .exe in Strata despite faster generation speeds, challenging the hype around the smaller model.