@julien_c: I've seen some confusion online on how to run llama.cpp with MTP (Multi-token prediction) in the simplest way possible.…
Summary
Julien C explains how to run llama.cpp with Multi-token prediction (MTP) for ~2x generation speed, using either the Dense 27B or MoE 35B model, with instructions for installation and configuration.
View Cached Full Text
Cached at: 05/19/26, 02:48 PM
I’ve seen some confusion online on how to run llama.cpp with MTP (Multi-token prediction) in the simplest way possible.
ICYMI, MTP is a new flavor of speculative decoding built-in to the model itself, that ~2x your tokens per sec for most use cases.
2x generation speed = Truly a game changer.
How to run it?
brew upgrade llama.cpp
or you might need to install from source until build 9200 is in your package manager:
brew install llama.cpp –HEAD
Then pick either the Dense 27B or the 35B A3B MoE.
Personally I tend to stick to the Dense model where I achieve ~30 tok/sec on my machine. The MoE is of course way faster at an impressive ~100 tok/sec on my machine. Truly rapid.
In both cases you probably want 48GB or better 64GB RAM or VRAM, though 36GB might work with more strongly-quantized versions.
Dense:
llama-server -hf ggml-org/Qwen3.6-27B-MTP-GGUF –spec-type draft-mtp –spec-draft-n-max 2
MoE:
llama-server -hf ggml-org/Qwen3.6-35B-A3B-MTP-GGUF –spec-type draft-mtp –spec-draft-n-max 3
Enjoy!
Similar Articles
Multi-Token Prediction (MTP) for LLaMA.cpp - Gemma 4 speedup by 40%
A new implementation of Multi-Token Prediction (MTP) in llama.cpp achieves a 40% speedup for Gemma 4 models, tested on a MacBook Pro M5Max. The post provides links to quantized GGUF models and the patched source code.
That's a good news...
Multi-token prediction (MTP) has been approved for integration into llama.cpp, indicating an upcoming update to the local LLM inference tool.
MTP+GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 - llama.cpp
A user benchmarks token generation speed on llama.cpp with the GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 flag, comparing performance with and without MTP (Multi-Token Prediction). Results show a significant speedup from 49 tok/s to 64 tok/s when MTP is enabled on an RTX5090 with a Qwen3.6-27B model.
llama + spec: MTP Support by am17an · Pull Request #22673 · ggml-org/llama.cpp
Pull request adding Multi-Token Prediction (MTP) support to llama.cpp, enabling speculative decoding for faster inference.
llama.cpp MTP speculative simplified for July 2026 big wins on dense models, underwhelming on MoE
An analysis of native MTP speculative decoding in llama.cpp shows significant speedups (1.4x-2.2x) for dense models like Qwen3.6-27B, but underwhelming results on MoE architectures, where gains are minimal due to already low per-step overhead.