Tag
JetBrains released Mellum2.1-12B-A2.5B-Thinking-MTP, a 12B MoE (2.5B active) thinking coding model with a multi-token prediction head for speculative decoding in vLLM. The update focuses mainly on reinforcement learning post-training in real repository environments to improve agentic software engineering tasks.
@NVIDIARTXSpark shares three techniques to accelerate Mixture-of-Experts (MoE) inference on the NVIDIA DGX Spark, covering optimized kernels, FP8 quantization, and multi-token prediction to boost tokens per second.
Soft-MTP 提出一种 KL 梯度恢复技巧，去除多token预测训练中的噪声残差梯度，在无额外开销下改进困惑度、投机解码与推理能力。
The paper presents Lafa, an 8B-parameter blockwise continuous flow language model trained with under 30B tokens that matches or surpasses autoregressive quality while achieving 2.54x inference throughput via self-speculative decoding and blockwise parallel decoding.
ECHO integrates multi-horizon prediction directly into the backbone via self-recursion and Dynamic Depth Sampling, improving pretraining efficiency and enabling the model to serve as its own inference drafter without auxiliary MTP modules.
提醒用户尝试 llama.cpp 中新合并的概率式 MTP（多 token 预测）功能，在散文生成场景下解码速度提升约 14%，最优 draft-n-max / draft-p-min 参数与贪心采样基本一致。
MTP 投机解码的草稿 LM head 原本每步都要读取 1.27 GB 的 BF16 权重，现改为 Q4_0 量化草稿头（验证与正常解码仍用 BF16），实现约 10.6% 的 token/s 提升，且 greedy 输出 md5 完全一致。相关改动已在 ROCmFPX 的 PR #49 中提交。
发布者声称该模型基于 Qwen3.8 27B，使用 390 万行代码数据进行专项微训，融合高效思考机制压缩推理长度，并经由 Opus5.5、GPT6Astra、Grok4.7、DSV4Pro 蒸馏对齐，配备 K3 kernel，再通过 SFT、RLOO 强化学习与 MTP 多词元预测三重训练，同时集成 DFlash2 注意力加速。
HyperZip proposes an efficient LLM-based lossless data compression framework that uses diffusion-based LLMs with Multi-Token Prediction and a hypernetwork to adapt models to target data without fine-tuning, achieving a superior compression-rate/throughput trade-off over autoregressive baselines.
The author optimized DeepSeek V4.1 Flash for Apple M3 Ultra, achieving up to 40 t/s decode speed with DSpark speculative decoding while maintaining byte-identical accuracy to the upstream model.
Benchmark tests on Ling-3.0-flash show that higher acceptance length in multi-token prediction decreases prose throughput, with n=1 being the most efficient setting for the evaluated workloads.
GLM-5.3, an AI model with unchanged architecture from GLM-5.2, has achieved significant improvements through training and is now visualizable in HF Viewer, showcasing complex features like sparse attention and MoE routing.
A quantized GGUF release of the Qwen3.8-27B model with reduced refusal behavior, retaining multi-token prediction and offering various quantization options for use with llama.cpp.
A user reports that quantising the f16 MTP draft model to Q4_K for Gemma 4 31b gives roughly 10% faster decode (65 to 72 TPS) on dual 3090s compared to the default Q4_0, while Q2_K performs worse.
This paper introduces AdaMTP, an adaptive training paradigm for multi-token prediction that dynamically aligns prediction horizons with sequence predictability using entropy-based segmentation, consistently outperforming standard MTP on math, code, and general benchmarks across three LLM backbones.
AngelSpec introduces a unified training and inference framework for speculative decoding that jointly optimizes autoregressive multi-token prediction and block-parallel diffusion drafters to handle heterogeneous real-world workloads. Experiments on the Hy3 model series show up to 2.4x speedup over autoregressive decoding and 11.8% higher throughput than DFlash.
A guide on optimizing MTP (Multi-Token Prediction) performance by tuning n_max parameter, with benchmark results for various models like Gemma-31b and Qwen on P100 and V100 GPUs.
A detailed benchmark of Unsloth's Qwen3.6-27B NVFP4 model on RTX 5090 GPUs, showing MTP (multi-token prediction) gives large speedups for single requests at short context but becomes detrimental under batch concurrency or long contexts.
Commentary on Google's Gemma 4 technical report, noting aggressive optimization via Quantization-Aware Training and Multi-Token Prediction for faster inference.
A user reports that Qwen 27B at q6kxl quantization with multi-token prediction achieves 50-90 token/s decode and 1500-2200 token/s pre-fill on a 4090+3090 system using LCPP, noting it is reliably coherent and fast for various coding tasks.