Qwen3.8-Next streaming - 150tps prefill, 3.6 tps decode on M5 Air
Summary
A user tested the Qwen3.8-Next model on an Apple M5 Air with 3-bit quantization, achieving 150 tokens per second prefill and 3.6 tps decode, outperforming a dense 27b model in some metrics.
Similar Articles
Qwen3.8-Flash-Next optimised for Macs
The article details custom optimizations for running the Qwen3.8-Flash-Next AI model on Mac M1 Max hardware, including SSD streaming, custom quantizations, and a sparse attention mechanism to improve performance.
Qwen 27B
A user reports that Qwen 27B at q6kxl quantization with multi-token prediction achieves 50-90 token/s decode and 1500-2200 token/s pre-fill on a 4090+3090 system using LCPP, noting it is reliably coherent and fast for various coding tasks.
Qwen3.8-Flash-Next NVFP4 2xDGX Spark config: 50t/s decode, 2,900t/s prefill
A user shares their optimized configuration for deploying the Qwen3.8-Flash-Next model on dual DGX Spark hardware, achieving up to 50t/s decode and 2,900t/s prefill speeds with technical patches and setup details.
Qwen3.6 27B on a 5090, 6.4k sample tok/s distribution after tuning MTP/cache settings
Running Qwen3.6 27B on an RTX 5090, achieving 6.4k tokens per second after tuning MTP and cache settings, demonstrating optimization techniques for inference.
Wow! Qwen 3.6:35b-a3b on a 3090... pretty amazing.
A user shares impressive results running a quantized Qwen 3.6:35b-a3b model on a used RTX 3090, achieving 160 tokens per second output after fitting the model into VRAM, and demonstrates vision capabilities with a 75-second video processing time.