Benchmark tests on Ling-3.0-flash show that higher acceptance length in multi-token prediction decreases prose throughput, with n=1 being the most efficient setting for the evaluated workloads.
The missing control is visible in sudoingX’s Ling-3.0-flash benchmark graphics. The earlier table leaves Ling’s no-speculation baseline as “not measured.” The later code/prose graphic fills it in: about 23 tok/s without the drafter, against 40.9 on code and 38.7 on prose with MTP n=1, at short prompts. That makes the tuning claim easier to inspect. The later graphic compares short code and prose workloads with and without drafting. A separate correction in the repository isolates CUDA graphs: the earlier “two flags” result changed graphs and multi-token prediction (MTP) together, so it could not tell you which change produced the gain. The corrected August 22 measurements on one 128GB DGX Spark, using the official INT4 checkpoint and the vendor’s vLLM fork, were: Configuration Reported tok/s, short coding task Eager execution, MTP off 20.8 CUDA graphs, MTP off 22.9 CUDA graphs, MTP n=1 40.9 That is roughly 10% more throughput from graphs relative to the eager baseline, followed by roughly 79% more from MTP relative to the graph baseline. Those percentages have different denominators. Then comes the setting that is tempting to turn up. Here n is num_speculative_tokens: how many draft tokens are proposed per step. vLLM’s documented acceptance-length metric includes a bonus token per verification step, so an average above 1 with n=1 is possible. The creator reports the metric from a vendor fork; its exact historical counting implementation is not supplied. MTP setting n=1 n=2 n=3 Mean acceptance length 1.87 2.39 2.77 Prose, 512-token output, tok/s 38.7 34.8 33.6 Prose, 2,048-token output, tok/s 37.3 33.6 31.6 Code throughput stayed approximately flat within the reported run-to-run variation. Prose got slower as the mean accepted draft length increased. Acceptance length is not an acceptance percentage, and it is not the optimization objective. sudoingX describes the configuration in the deployment thread; the pinned benchmark notes contain both tables. These are the author’s measurements, without an independent rerun here. The sweep describes streaming and server-side token counts but does not fully specify its timing denominator, so the numbers should stay labeled as reported throughput. For this checkpoint and these workloads, n=1 was the useful setting. The transferable experiment is to isolate the no-MTP baseline, then compare draft settings on the output types you actually generate.
A user benchmarked MTP (Multi-Token Prediction) on Gemma 4 with mlx-vlm on M4 Max Studio, finding it excellent for code generation (1.53x faster, 66% acceptance) but detrimental for JSON output (50% slower, only 8% acceptance) and neutral for long-form prose, suggesting MTP benefits vanish when acceptance drops below 50%.
AntLing open-sources Ling-3.0-flash-dspark, a DSpark draft model for Ling-3.0-flash, which achieves 1,120 tokens per second with low latency on NVIDIA Blackwell GPUs.
The article questions whether the Ling-2.6-1T model's size is justified by quality, local serving feasibility, or long context stability, describing it as an open-source MoE model with 1T total params and up to 1M native context.
This post describes two configuration flags that increase the official Ling-3.0-flash INT4 inference speed from 20.8 to 38.7 tok/s on a single DGX Spark, while warning about the need for a specific vLLM fork and noting tradeoffs with long-context performance.
Ling-2.6-flash is a 104B-total/7.4B-active sparse instruct model optimized for token efficiency, aiming to cut costs and boost throughput on agent tasks.