MTP support merged into llama.cpp
Summary
The pull request adding MTP (Multi-Token Prediction) support to llama.cpp has been merged into the master branch.
PR [22673](https://github.com/ggml-org/llama.cpp/pull/22673) has been merged into master! 🎉
Similar Articles
llama + spec: MTP Support by am17an · Pull Request #22673 · ggml-org/llama.cpp
Pull request adding Multi-Token Prediction (MTP) support to llama.cpp, enabling speculative decoding for faster inference.
b9180 llama.ccp MTP landed
llama.cpp version b9180 has been released, featuring Multi-Token Prediction (MTP). The release is marked by successful builds and developer relief.
That's a good news...
Multi-token prediction (MTP) has been approved for integration into llama.cpp, indicating an upcoming update to the local LLM inference tool.
Remove padding and multiple D2D copies for MTP by gaugarg-nv · Pull Request #24086 · ggml-org/llama.cpp
A pull request for llama.cpp that removes padding and multiple device-to-device copies for Multi-Token Prediction (MTP), improving performance on GPU.
StepFun 3.5 MTP by pwilkin · Pull Request #23274 · ggml-org/llama.cpp
Pull request adding support for StepFun 3.5 MTP model in llama.cpp.