Factorization-Guided Training and Inference for Diffusion-Drafted Speculative Decoding

Hugging Face Daily Papers Papers
speculative-decoding diffusion-models llm-inference inference-acceleration training-strategy throughput-optimization

Summary

DScout 通过将扩散草稿器的训练与推理分解为前缀接受度和延迟权衡，提升了推测解码的吞吐量，在 Qwen3-4B 和 Qwen3-8B 上分别实现 3.84 倍和 4.14 倍的平均加速。

Speculative decoding accelerates large language model inference through parallel verification of tokens proposed by a drafter. Diffusion drafters further parallelize proposal generation, but greater proposal capacity or higher mean accepted tokens (MAT) need not yield higher throughput. Because verification accepts only a consecutive prefix, early rejections can render later correct predictions ineffective, limiting MAT gains, while even higher MAT may not improve throughput if per-round latency increases. We introduce DScout, which uses two complementary factorizations to guide diffusion drafter training and inference for higher throughput. For training, DScout factorizes position-wise loss into inclusion, weighting, and prediction loss. This guides the design and combination of mixed-horizon training, intra-block unmasking, and acceptance-weighted loss to align supervision with prefix acceptance and improve MAT. For inference, DScout factorizes throughput as the ratio of MAT to per-round latency, guiding target-conditioning depth, drafter depth, and inference block size selection to identify a high-throughput configuration for each workload. Across six benchmarks, DScout achieves 3.84times and 4.14times mean speedups over autoregressive decoding on Qwen3-4B and Qwen3-8B, respectively, improving mean speedup over the state-of-the-art baseline by 9.1% and 7.1%. Across models and concurrency settings, our inference configuration policy provides up to 9.75% additional mean speedup over DScout's fixed configuration.
Original Article
View Cached Full Text

Cached at: 10/20/26, 07:35 AM

Paper page - Factorization-Guided Training and Inference for Diffusion-Drafted Speculative Decoding

Source: https://huggingface.co/papers/2610.23935

Abstract

DScout improves speculative decoding throughput by aligning diffusion drafter training and inference with prefix acceptance and latency trade-offs.

Speculative decodingaccelerates large language model inference through parallel verification of tokens proposed by a drafter.Diffusion draftersfurther parallelize proposal generation, but greater proposal capacity or highermean accepted tokens(MAT) need not yield higher throughput. Because verification accepts only a consecutive prefix, early rejections can render later correct predictions ineffective, limiting MAT gains, while even higher MAT may not improve throughput if per-round latency increases. We introduceDScout, which uses two complementary factorizations to guide diffusion drafter training and inference for higher throughput. For training,DScoutfactorizes position-wise loss into inclusion, weighting, and prediction loss. This guides the design and combination ofmixed-horizon training,intra-block unmasking, andacceptance-weighted lossto align supervision with prefix acceptance and improve MAT. For inference,DScoutfactorizes throughput as the ratio of MAT to per-round latency, guidingtarget-conditioning depth, drafter depth, andinference block sizeselection to identify a high-throughput configuration for each workload. Across six benchmarks,DScoutachieves 3.84times and 4.14times mean speedups over autoregressive decoding on Qwen3-4B and Qwen3-8B, respectively, improving mean speedup over the state-of-the-art baseline by 9.1% and 7.1%. Across models and concurrency settings, our inference configuration policy provides up to 9.75% additional mean speedup overDScout’s fixed configuration.

View arXiv pageView PDFAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.23935 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.23935 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.23935 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

DEdit: Iterative Draft Editing for Speculative Decoding

arXiv cs.CL

DEdit introduces a diffusion-based drafter for speculative decoding that iteratively edits drafts via token-to-token predictions, using a ProposalMix training scheme to repair errors while preserving correct tokens. It achieves macro-average speedups of 5.72× and 5.97× on Qwen3-4B and Qwen3-8B across seven benchmarks.

z-lab/Qwen3.6-35B-A3B-DFlash

Hugging Face Models Trending

z-lab releases DFlash, a speculative decoding drafter that uses a lightweight block-diffusion model to draft 15–16 tokens in parallel, yielding up to 2.9× speedup for Qwen3.6-35B-A3B inference.

What is Speculative Decoding? (70 minute read)

TLDR AI

一篇70分钟阅读量的深度技术长文，从第一性原理推导投机解码（speculative decoding），构建涵盖EAGLE-3、多token预测（MTP）以及块扩散drafters（DFlash、DSpark）的统一框架，并讨论drafter训练中最小化KL散度的问题。