DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation

Hugging Face Daily Papers Papers
speculative-decoding large-language-models inference-acceleration semi-autoregressive confidence-scheduled verification throughput

Summary

DSpark is a speculative decoding framework that combines semi-autoregressive draft generation with confidence-scheduled verification to accelerate LLM inference and improve throughput in high-concurrency settings.

Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent parallel drafters efficiently propose long token sequences in a single forward pass, they suffer from rapid acceptance decay due to a lack of inter-token dependencies. Furthermore, indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks, severely degrading throughput in high-concurrency serving systems. We introduce DSpark, a speculative decoding framework that unifies high-throughput parallel generation with adaptive, load-aware verification. To maintain draft quality, DSpark utilizes a semi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay. To optimize system efficiency, DSpark employs confidence-scheduled verification, dynamically tailoring the verification length for each request based on estimated prefix survival probabilities and engine-specific throughput profiles. On offline benchmarks across diverse domains, DSpark substantially improves the accepted length over state-of-the-art autoregressive and parallel drafters. When deployed within the DeepSeek-V4 serving system under live user traffic, DSpark successfully mitigates verification waste. Compared to the established production baseline (MTP-1), DSpark accelerates per-user generation speeds by 60 to 85 percent at matched throughput levels. More importantly, by preventing severe throughput degradation under strict interactivity constraints, it enables performance tiers that were previously unattainable, shifting the Pareto frontier of our serving system.
Original Article
View Cached Full Text

Cached at: 07/08/26, 06:48 AM

Paper page - DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation

Source: https://huggingface.co/papers/2607.05147 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

DSpark enhances LLM inference speed by combining parallel draft generation with adaptive verification that reduces waste and improves throughput in high-concurrency settings.

Speculative decodingaccelerates Large Language Model (LLM) inference by decoupling draft generation from targetverification. While recentparallel draftersefficiently propose long token sequences in a single forward pass, they suffer from rapidacceptance decaydue to a lack ofinter-token dependencies. Furthermore, indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks, severely degradingthroughputin high-concurrency serving systems. We introduce DSpark, aspeculative decodingframework that unifies high-throughputparallel generation with adaptive, load-awareverification. To maintain draft quality, DSpark utilizes asemi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay. To optimize system efficiency, DSpark employsconfidence-scheduled verification, dynamically tailoring theverificationlength for each request based on estimatedprefix survival probabilitiesand engine-specificthroughputprofiles. On offline benchmarks across diverse domains, DSpark substantially improves the accepted length over state-of-the-art autoregressive andparallel drafters. When deployed within the DeepSeek-V4 serving system under live user traffic, DSpark successfully mitigatesverificationwaste. Compared to the established production baseline (MTP-1), DSpark accelerates per-user generation speeds by 60 to 85 percent at matchedthroughputlevels. More importantly, by preventing severethroughputdegradation under strict interactivity constraints, it enables performance tiers that were previously unattainable, shifting thePareto frontierof our serving system.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2607\.05147

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.05147 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.05147 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.05147 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

What is Speculative Decoding? (trending on paperswithco.de) [R]

Reddit r/MachineLearning

Speculative decoding is an inference optimization technique that uses a fast draft model to propose future tokens verified in parallel by a larger model, improving LLM generation speed. The article highlights its trending status on Papers with Code and a recent SGLang blog post about state-of-the-art latencies using DFlash models.