@ashxhart: Been giving my M3 Ultra Studio some love and tinkering with MLX. Started the afternoon at 42 tok/s. Now: 73 tok/s, and …
Summary
An author improved MLX performance on an M3 Ultra Studio to achieve 73 tok/s for a 4-bit Qwen3.8-Flash-Next model, which shows intelligence scores matching GPT-5.6 Sol, highlighting the growing potential of local AI.
View Cached Full Text
Cached at: 09/09/26, 03:48 AM
Been giving my M3 Ultra Studio some love and tinkering with MLX.
Started the afternoon at 42 tok/s. Now: 73 tok/s, and there’s still more on the table.
Running Qwen3.8-Flash-Next @ 4-bit locally.
Now for the crazy bit: Artificial Analysis Intelligence Index v4.3:
GPT-5.6 Sol 39 (Medium) GPT-5.6 Sol 42 (High) Qwen3.8-Flash-Next 42 GPT-5.6 Sol 44 (XHigh) GPT-5.6 Sol 47 (Max) Grok 4.5 39 (High) Claude Sonnet 5 38 (Max)
*Qwen score currently estimated by Artificial Analysis.
A 4-bit open-weights model running at 73 tok/s on a Mac under my desk… with an estimated intelligence score matching GPT-5.6 Sol at High reasoning.
Local AI is getting ridiculous. 🔥 Plus, this model does not say no to a bit of red teaming.
Once I am happy with it, I will do a pr for @jundotkim’s oMLX :)
Similar Articles
@volatilemarkts: Same Mac. Same model. Same words. My 512 GB M3 Ultra ran GLM-5.3-Flash on MLX last week: 27 tok/s, first token in 0.46 …
TensorFold 0.6.2 is an Apache 2.0 open-source LLM inference engine claiming over 2x faster token throughput than MLX on Apple Silicon (61 vs 27 tok/s on an M3 Ultra) via multi-token-prediction speculative decoding, with parallel concurrent requests and OpenAI-compatible API.
@ivanfioravanti: Video of tensorfold running Qwen3.8-Flash-Next-MLX-oQ8-MTP high on M3 Ultra ~80 t/s! It's a great experience, less erro…
A demo video shows the Qwen3.8-Flash-Next-MLX-oQ8-MTP model running on an Apple M3 Ultra via tensorfold at ~80 tokens/s, with the author recommending the highest possible quantization for better quality and fewer errors.
oMLX vs Rapid-MLX vs Splash vs MTPLX on M3 Max 36 GB: 110 tok/s on Qwen3.6-35B-A3B, ~32 tok/s on Qwen3.8-27B
A hands-on benchmark comparing four local LLM inference engines (oMLX 0.7.0, Rapid-MLX 0.15.5, Splash 1.2.1, MTPLX 2.12.2) on a 36 GB M3 Max MacBook Pro, finding Rapid-MLX fastest on Qwen3.6-35B-A3B (~110 tok/s) and Splash fastest on dense Qwen3.8-27B (~31.7 tok/s via speculative decoding).
Swift-1.5-Qwen3.8-27b-oQ8e-mtp on Apple M5 Max — 34.8 tok/s — llm-bench.io
The article benchmarks the Swift-1.5-Qwen3.8-27b-oQ8e-mtp AI model on Apple M5 Max, achieving 34.8 tokens per second, with reproduction steps for the run.
@Prince_Canuma: My home compute for MLX and research: • M3 Ultra — 512GB (sponsored by community + @wai_protocol) • RTX PRO 6000 — 96GB…
A researcher shares their home compute setup for MLX and AI research, featuring M3 Ultra with 512GB, RTX PRO 6000 with 96GB, and M3 Max with 96GB for model porting and stress testing.