@no_stp_on_snek: busy evening in custom rust kernel and swift dispatch land. 3 of the 9 PRs are genuine new or changed metal kernels. th…
Summary
在自定义Rust内核和Swift调度逻辑的优化下，Qwen3.6-35b-A3B模型的预填速度在2k提示下从255 tok/s提升到1058 tok/s，实现了约4倍的加速，解码和困惑度未受影响。
View Cached Full Text
Cached at: 07/07/26, 05:25 AM
busy evening in custom rust kernel and swift dispatch land.
3 of the 9 PRs are genuine new or changed metal kernels. the other 6 are swift-side dispatch logic that either wires those kernels up or restructures how the existing ones get called
net effect: qwen3.6-35b-a3b prefill went 255 to 1058 tok/s at a 2k prompt. about 4x. decode and perplexity didn’t move.
still lots to go
Similar Articles
@nicekate8888: For the past twenty days, I've been obsessing over one thing — how to make Qwen3.6-27B run fast and well on my Mac. I started with Unsloth Q5, got 18 tok/s, and the fan was roaring. Then I switched to MLX 6bit + DFlash, hitting 22 tok/s, still not fast enough. Eventually I found MTPLX 4bit: 43 tok/s with good quality.
The user shares their experience optimizing Qwen3.6-27B inference speed on a Mac using different quantization methods (Unsloth Q5, MLX 6bit + DFlash, MTPLX 4bit), ultimately reaching 43 tok/s.
@no_stp_on_snek: anyone still talking about mlx-swift-lm? said i was taking the day off... cleaned the chicken coop, got a workout in, f…
The author describes implementing TurboQuant KV-cache compression into Apple's mlx-swift-lm, achieving 2.7x compression with quality on par with 8-bit, and 3-4x decode speed improvements via a fused Metal kernel.
@seclink: Just hit 134 tok/s with Qwen 3.5-27B Dense and 73 tok/s with the new Qwen 3.6-27B on a single RTX 3090. The 2026 open-source scene is moving at lightspeed…
A single RTX 3090 pushes 134 tok/s on the fresh 27B Qwen 3.5 Dense and 73 tok/s on Qwen 3.6-27B via fused kernels plus speculative decoding, with GGUF drops the same evening.
@bstnxbt: DFlash v0.1.4 : custom Metal verify kernels for quantized Qwen3 hybrid models, plus significant peak memory reduction a…
DFlash v0.1.4 releases custom Metal verify kernels for quantized Qwen3 hybrid models with significant peak memory reduction and 2.2x throughput improvements at long context on M5 Max GPUs.
Qwen3.6:35b UD Q4_K_M 80 tok/s on Nvidia P40
A user shares achieving 80 tok/s on a Qwen3.6 35B model with Q4_K_M quantization and 100k context on a single Nvidia P40 using TheTom's TurboQuant fork of llama.cpp, highlighting various optimizations.