@nicekate8888: For the past twenty days, I've been obsessing over one thing — how to make Qwen3.6-27B run fast and well on my Mac. I started with Unsloth Q5, got 18 tok/s, and the fan was roaring. Then I switched to MLX 6bit + DFlash, hitting 22 tok/s, still not fast enough. Eventually I found MTPLX 4bit: 43 tok/s with good quality.
Summary
The user shares their experience optimizing Qwen3.6-27B inference speed on a Mac using different quantization methods (Unsloth Q5, MLX 6bit + DFlash, MTPLX 4bit), ultimately reaching 43 tok/s.
Similar Articles
@linexjlin: K2.6 built a Zig LLM inference engine from scratch on Mac in 12h, pushing Qwen 3.5 0.8B from 15 tok/s to 193.1 tok/s
Developer wrote a Zig-based LLM inference engine from zero on macOS in 12 hours, boosting Qwen 3.5 0.8B throughput from 15 to 193 tokens per second.
Running 95.5 GiB Qwen3.8-Flash-Next at 41–52 tok/s on a 64GB Mac (1.76x faster than llama.cpp): Slipstream release, 130k context scaling, + Swift variant
作者发布 Slipstream——一款针对 Apple Silicon 的 C++ Metal 推理引擎,支持原生 SSD 专家流式加载与投机草稿解码,可在 64GB Mac 上以 41–52 tok/s 运行 95.5 GiB 的 Qwen3.8-Flash-Next 模型,比 llama.cpp 快 1.76 倍,且 130k 长上下文下速度不衰减。
@Oluwaphilemon1: Qwen3.8-Flash-Next was doing around 25 tok/s. Now it’s over 50 tok/s. Qwen3.6-35B Q8 went from roughly 40 tok/s to 100 …
A cafe-llama.cpp update unlocks Qwen's Multi-Token Prediction potential for speculative decoding, roughly doubling decoding speed (~25 to 50+ tok/s on Qwen3.8-Flash-Next, ~40 to 100 tok/s on Qwen3.6-35B Q8) with just the --spec-draft-n-max 4 flag.
Qwen3.8-Flash-Next (95.5 GiB) on a 64GB Mac at ~27 tok/s, checkpoint + fork
The author optimized the Qwen3.8-Flash-Next model to run on a 64GB Mac using expert streaming and other techniques, achieving ~27 tok/s by publishing a checkpoint and a llama.cpp fork.
I spent 3 weeks testing local Qwen3.8 on the new low-latency SGLang/vLLM recipes: DFlash2 2.8x. Builds: RadixArk + Inferact 27B NVFP4, 27B BF16, orcarouter 27B Uncensored, Flash-Next NVFP4
作者在单张 RTX PRO 6000 上对三款 Qwen3.8 27B 检查点进行 10 项统一测试,Flash-Next 赢下多数任务,而 DFlash2 投机解码将 27B 的推理速度从 75 提升到 210 tok/s(2.8 倍),并对比了 SGLang 与 vLLM 在不同量化导出下的性能差异。