Strata - Qwen 3.8 Flash Next - boosting decoding speed Tip
Summary
A user shares benchmarks showing that raising the --adapt-every parameter from 1 to 6 in Strata boosts token generation speed by 9-55% across multiple Qwen 3.8 Flash Next prompts, with the largest gains on long-context code descriptions.
Similar Articles
Strata with Qwen3.8 Flash Next UD-Q4_K_XL
A user reports strong single-request decode speeds running Qwen3.8-Flash-Next in Unsloth's UD-Q4_K_XL quant on Strata with an RTX PRO 6000, showing roughly 2x faster throughput than vLLM on the same GPU across prose, counting, coding, and reasoning tasks.
Strata on a power limited 5090 and 96GB of DDR5-6400 is cranking out 150-200 tok/s decode and 5-6k prefill! Qwen3.8-Flash-Next at IQ3_S, CTX at 128k tokens (8-bit).
Strata runs Qwen3.8-Flash-Next at IQ3_S quantization on a power-limited RTX 5090 with 96GB of DDR5-6400, achieving 150-200 tok/s decode and 5-6k prefill with a 128k-token context (8-bit).
Qwen3.8-Flash-Next at 1M context on Strix Halo: 38 tok/s decode, 18 min prefill (halogen 0.12.0)
Halogen version 0.12.0 fixes performance degradation at high context depths, showing improved decode and prefill speeds for Qwen3.8-Flash-Next at 1 million tokens of context on AMD Ryzen AI Max+ hardware.
@Oluwaphilemon1: Qwen3.8-Flash-Next was doing around 25 tok/s. Now it’s over 50 tok/s. Qwen3.6-35B Q8 went from roughly 40 tok/s to 100 …
A cafe-llama.cpp update unlocks Qwen's Multi-Token Prediction potential for speculative decoding, roughly doubling decoding speed (~25 to 50+ tok/s on Qwen3.8-Flash-Next, ~40 to 100 tok/s on Qwen3.6-35B Q8) with just the --spec-draft-n-max 4 flag.
Qwen3.8 flash next ISTA-DASLab GGUF 50t/s TG and 1500t/s PP with 12GB VRAM and 64GB RAM Laptop on 'Strata' engine
The article highlights the Strata inference engine, which significantly outperforms llama.cpp for running Qwen3.8 models on a laptop with 12GB VRAM and 64GB RAM, achieving up to 50 tokens per second for text generation and 1500 tokens per second for prompt processing.