@ivanfioravanti: llamacpp is gonna get MTP support soon!

That's a good news...

Reddit r/LocalLLaMA

Multi-token prediction (MTP) has been approved for integration into llama.cpp, indicating an upcoming update to the local LLM inference tool.

MTP support merged into llama.cpp

Reddit r/LocalLLaMA

The pull request adding MTP (Multi-Token Prediction) support to llama.cpp has been merged into the master branch.

llama + spec: MTP Support by am17an · Pull Request #22673 · ggml-org/llama.cpp

Reddit r/LocalLLaMA

Pull request adding Multi-Token Prediction (MTP) support to llama.cpp, enabling speculative decoding for faster inference.

@no_stp_on_snek: Tested out MTP for the first time on my llamacpp fork last night with turbo4 sym. GX10 hardware. using MoE model: llmfa…

X AI KOLs Following

Tested Multi-Token Prediction on a llamacpp fork with a Qwen-based MoE model, achieving +0.41% PPL improvement over fp16 baseline.

b9180 llama.ccp MTP landed

Reddit r/LocalLLaMA

llama.cpp version b9180 has been released, featuring Multi-Token Prediction (MTP). The release is marked by successful builds and developer relief.