AntLing released a dspark draft model for Ling-3.0-flash

Reddit r/LocalLLaMA Models
open-source ai-model language-model nvidia-blackwell performance-benchmark

Summary

AntLing open-sources Ling-3.0-flash-dspark, a DSpark draft model for Ling-3.0-flash, which achieves 1,120 tokens per second with low latency on NVIDIA Blackwell GPUs.

Today we are open sourcing Ling-3.0-flash-dspark, a DSpark draft model built specifically for Ling-3.0-flash. On 4 NVIDIA Blackwell GPUs at batch 1, it delivered 1,120 tok/s, 0.78 ms mean TPOT, and an accept length of 9.95 across 1,000 requests. 🧵 https://t.co/7sDr7wy0rO
Original Article
View Cached Full Text

Cached at: 08/22/26, 09:14 AM

Today we are open sourcing Ling-3.0-flash-dspark, a DSpark draft model built specifically for Ling-3.0-flash. On 4 NVIDIA Blackwell GPUs at batch 1, it delivered 1,120 tok/s, 0.78 ms mean TPOT, and an accept length of 9.95 across 1,000 requests.

Ling-3.0-flash has 42 layers: 35 KDA linear attention layers and 7 MLA full attention layers. Its MoE contains 512 routed experts and one shared expert, with top 8 plus 1 active per token. This hybrid design keeps attention cost low, even at 8K context.

Batch 1 exposes every microsecond. There is no larger batch to absorb launch cost or fill pipeline gaps. On the NEXTN path, our systems work raised mean output throughput from 288 to 606 tok/s and reduced mean TPOT from 3.33 to 1.53 ms.

At batch 1, mean TPOT is roughly step time divided by accept length. We worked on both. SGLang shortened each target step. DSpark uses a parallel draft backbone, a lightweight Markov head for local dependency, and a confidence head for verification scheduling.

We adapted the public DSpark recipe to Ling with distribution aligned data, architecture ablations, and an acceptance aware loss. SplitServe Trainer divided one 8 GPU node between draft training and SGLang target inference, keeping long context online training local.

The first DSpark trace was 47% idle: 4.99 ms of idle in a 10.62 ms step. FlashInfer plan() made blocking device to host reads for metadata the CPU already knew. Building plans from host data removed the sync and let the CPU queue draft and verify before the GPU needed them.

Other gains came from removing per step sequence length synchronization, capturing metadata glue graphs, adding PDL chains across MoE, router, and KDA, fusing SwiGLU into the up GEMM, retuning KDA chain verify, and running the router and lm_head in bf16.

Same command and machine, 1,000 requests: Tuned NEXTN: 606 tok/s, 1.53 ms mean TPOT, 3.25 accept length DSpark: 1,120 tok/s, 0.78 ms mean TPOT, 9.95 accept length DSpark cut mean TPOT by 1.9×. The 9.95 figure is workload specific.

This configuration is best suited to interactive agents, coding assistants, and reasoning workloads where single user latency matters. The headline run used static DSpark verification, greedy decoding, 8,192 input tokens, 1,024 output tokens, and synthetic random data.

The checkpoint and serving recipe are now available.

Thanks to the @sgl_project and @lmsysorg communities for their support. Full engineering details: https://lmsys.org/blog/2026-08-21-ling3-flash-spec-decode-blackwell…

Hugging Face: https://huggingface.co/inclusionAI/Ling-3.0-flash-dspark…

ModelScope: https://modelscope.cn/models/inclusionAI/Ling-3.0-flash-dspark…

Join the Ant Ling Discord: https://discord.com/invite/GNaQc8WC5T…

Similar Articles

Ling-3.0-flash MXFP4 released and running locally on one DGX Spark.

Reddit r/LocalLLaMA

Ling-3.0-flash MXFP4, a quantized model, has been released and runs locally on a single DGX Spark, achieving ~80 tok/s decoding and 2,500-3,500 tok/s long-input prefilling, enabling private on-device inference for coding, agents, and offline batch jobs.

ling 3.0 flash/tiny base models

Reddit r/LocalLLaMA

InclusionAI has open-sourced the Ling-3.0 series, featuring highly efficient language models with sparse MoE architecture and hybrid linear attention, providing checkpoints at various training stages to support research and innovation.