Tag
提出DSpine，一种通过网络深度注入相邻因果条件的并行推测解码drafter，在SGLang中实现高效并行执行，较DFlash在Qwen3-8B上平均接受长度提升27.8%、吞吐量提升23.3%。
DFlash 2 improves speculative decoding by predicting tokens in parallel, achieving over 20% more output per verification pass with minimal latency, and is integrated into major inference engines like SGLang and vLLM.
Proposes Progressive Tree Drafting (PTD), a training-free, model-agnostic speculative decoding method that uses progressive tree structures and stepwise pruning to enable parallel generation of multiple draft paths, achieving up to 2x speedup on various benchmarks.
JetSpec is a speculative decoding framework that combines efficient forward drafting with causal conditioning to improve LLM inference speed and acceptance rates, achieving up to 9.64x speedup on MATH-500 and 4.58x on conversational workloads.
Domino is a speculative decoding framework that decouples causal dependency modeling from autoregressive drafting, using a parallel backbone and lightweight causal refinement head to achieve up to 5.49× end-to-end speedup on Qwen3 models.
This paper introduces D-PACE, a dynamic position-aware cross-entropy loss for training speculative decoding drafters that adaptively weights positions to improve acceptance length and inference speed, achieving consistent wall-clock speedups across benchmarks with minimal overhead.
This paper introduces PARD-2, a dual-mode speculative decoding framework that uses target-aligned parallel draft models to accelerate LLM inference, achieving up to 6.94x lossless acceleration on Llama 3.1-8B.