@Youssofal_: MTPLX V0.3 Is Out!: - I realised M1 & M2 macs do not support BF16 and were emulating it leading to significantly decrea…
Summary
MTPLX v0.3 is released, a native runtime for Apple Silicon that uses Multi-Token Prediction (MTP) to double decode speed while maintaining distributional accuracy via Leviathan-Chen acceptance.
View Cached Full Text
Cached at: 05/10/26, 12:27 PM
MTPLX V0.3 Is Out!: - I realised M1 & M2 macs do not support BF16 and were emulating it leading to significantly decreased TPS. M1/M2 Macs now route to FP16 MTP heads. - Introduced “Optimized Quality” an 8 bit varient of the 27B model. Slower but better. - Thanks to @Daniel_Farinax implemented a 2049 -> 4096 prefill chunk change and serving changes which improve TTFT and the expereince in opencode, hermes & PI Try now: http://mtplx.com
MTPLX — Twice as fast on MLX
Source: https://mtplx.com/ Native MTP · MLX · v0.1.0-preview.1
MTPLXMTPLXMTPLXMTPLXMTPLX
Run local LLMs twice as fast.
Native MTP speculative decoding for Apple Silicon. Over 2× the decode speed at default model temperatures, using the model’s built-in MTP heads. No external drafter.
Install
One install. One wizard.
Wizard handles model, mode, and surface (browser or terminal) on first run. After that, one keypress.
# Install via Homebrew
brew install youssofal/mtplx/mtplx
# Start chatting — wizard picks model, mode, and surface
mtplx start
Same prompt. Same temperature.
Twice as fast. Still exact.
Most fast-decode tools cheat at temperature by matching greedy argmaxes — that silently breaks the target distribution. MTPLX accepts via the Leviathan–Chen probability ratio with residual(p − q)+correction. Verified bit-exact against single-token AR.
Speedup2.24×
Without MTPbaseline TPS
With MTPLXnative MTP
Qwen3.6-27B · MacBook Pro M5 Max · MLXtarget temp 0.6 / top_p 0.95 / top_k 20
What MTPLX is
A native-MTP runtime, not a wrapper.
01 — Native MTP
Single checkpoint.
The drafter is the target’s own MTP heads. No second model in memory. No distillation. No external drafter to maintain.
02 — Exact at T
Leviathan–Chen, not argmax.
Probability-ratio acceptance with residual(p − q)+correction. Verifiedmax_diff = 0.0against reference single-token AR.
03 — MLX-native
Built for Apple Silicon.
MLX source fork plus custom Metal kernels tuned for the verify hot path. Real OpenAI- and Anthropic-compatible serving stack on top.
How the cycle runs
One forward. K verified tokens.
Per cycle, the MTP head drafts K tokens, the target verifies all K in one batched forward, and the math decides — per position, exactly. A bonus token falls out for free when all K accept.
01 — Draft
MTP head proposes
K tokens drafted from the target’s own built-in MTP heads, with proposal probabilitiesq.
02 — Verify
Batched target forward
Target evaluates all K positions in one forward via GraphBank-compiled verify shapes.
03 — Accept
Probability ratio
Per-position acceptance via Leviathan–Chen rejection sampling. fp32 ratio path because BF16 underflows.
04 — Repair
Residual correction
On rejection, sample a clean replacement from(p − q)+. Rejected drafts never enter committed history.
05 — Commit
+ bonus token
Committed-history KV writeback. Bonus token at K+1 falls out for free when every position accepted.
What sits on top of MLX
Custom Metal kernels we own.
Clients
Browser chat · Open WebUI · Claude Code · Cline · Continue · openai-python · anthropic-python
Serving API
/v1/chat/completions · /v1/messages · /v1/models · /health · /metrics— OpenAI- and Anthropic-compatible, streaming SSE
Engine
Engine sessions · SessionBank warm-prefix exact-state reuse ·logits_max_abs_diff = 0.0across turns
MTPLX runtime
Native-MTP speculative cycle · committed-history KV contract verified vs vLLM CUDA reference at cosine > 0.9998 through D5
Custom Metal
linear-gdn-from-conv-tapefused GDN verify kernel ·verify_qmvsmall-M qmv · GraphBank compiled verify shapes · draft-only 4/3-bit LM head
MLX source fork
mlx-mtplx-0.31.2-qmm· small-MqmvretunedBN16 · 4-simdgroup · unroll_count(4)for verify shapes M=3..6
vs vLLM CUDA
Higher acceptance at every depth.
MTPLX D4 acceptance on Qwen3.6-27B is higher per position than vLLM’s CUDA MTP-5, on the same prompts.
MTPLX · D4 · Apple Silicon
D197.62%
D295.24%
D388.10%
D475.61%
vLLM · MTP-5 · CUDA
P192.70%
P277.00%
P363.00%
P450.90%
P543.00%
Similar Articles
MTPLX V1: The Swift App For Running & Creating MLX MTP Models (2x TPS Qwen 3.6 27B)
MTPLX V1 is a native Mac app that bundles the MTP speculative decoding engine for MLX models, offering features like model conversion via Forge, built-in chat, benchmarking, and support for smaller models. It achieves over 2x speedup with mathematical exactness.
@Youssofal_: Thank you to Kate for the comprehensive review of MTPLX. She’s tested multiple different MLX runtimes and concluded MTP…
nicekate tested multiple MLX runtimes on a Mac for running Qwen3.6-27B and concluded that MTPLX is the fastest, achieving 43 tok/s at 4bit quantization.
@Youssofal_: 72+ TPS on Qwen 3.6 27B on a Macbook pro M5 max. MTPLX V2 out now! The fastest way to run models on MLX.
MTPLX V2 is released, claiming 72+ tokens per second on Qwen 3.6 27B running on a Macbook Pro M5 Max via MLX.
@nash_su: Mac inference speed doubled. MTPLX is an integrated solution combining MLX and MTP, specifically optimized for model inference on Apple Silicon. By using models with a custom MTP head, it can deliver doubled inference speed. I tested it with Qwen3.6-27…
MTPLX is an integrated solution combining MLX and MTP, specifically optimized for model inference speed on Apple Silicon. Tests show that Qwen3.6-27B achieves double the inference speed of LM Studio, and it also integrates fan management.
@dealignai: Qwen3.6-27b and 35b MXFP4 MXFP8 CRACK is out now with MTP. Enjoy uncensored speediness! 35b mxfp4: https://huggingface.…
DealignAI releases CRACK-abliterated and MXFP4/MXFP8 quantized versions of Qwen3.6-27B and 35B models, preserving MTP for faster speculative decoding on Apple Silicon.