Tag
incoai发布了DFlash 2草稿模型（Qwen3.8-Flash-Next-DFlash2），这是一种用于推测解码的块扩散式草稿器，可在单次前向中预测整个token块，并通过SGLang实现无损解码加速，在接受长度上优于内置MTP。
DLite提出一种无历史上下文的投机解码框架，通过复用目标模型的隐状态进行局部预测并省去历史KV缓存维护，在agentic工作负载下将接受长度提升至2-6，较SOTA方法DSpark提升最高70%。
Today, we release an experimental DSpark draft model for our vision-language model (VLM) LFM2.5-VL-3B, which adds a speculative decoding path for faster inference with minimal memory cost.
A draft model for Qwen 3.8 27B is highly effective on 16 GB GPUs, achieving around 60 tokens per second on an RX 9070 XT and offering better VRAM efficiency than built-in MTP.
DFlash 2 is a block-diffusion draft model for speculative decoding that improves inference speed for the Qwen3.8-27B language model, offering higher acceptance length and throughput in benchmarks.
Introduces DFlash 2, a block-diffusion drafter for speculative decoding with the Qwen3.8-27B model, demonstrating improved acceptance length and throughput in benchmarks.
DeepSeek open-sourced DeepSpec, a full-stack codebase for training and evaluating draft models for speculative decoding, enabling 60-85% faster generation. It includes data preparation, training, and evaluation scripts with support for multiple draft model algorithms (DSpark, DFlash, Eagle3).
A GGUF conversion of MiniMax M3's EAGLE draft model for llama.cpp is now available, enabling speculative decoding speedups on compatible hardware.
Speculative decoding is an inference optimization technique that uses a fast draft model to propose future tokens verified in parallel by a larger model, improving LLM generation speed. The article highlights its trending status on Papers with Code and a recent SGLang blog post about state-of-the-art latencies using DFlash models.
Discussion of different flavors of speculative decoding and an attempt to produce a Qwen-3.6-27b EAGLE-3 drafter for the community.
Proposes PPOW, a reinforcement learning framework for optimizing draft models in speculative decoding using window-level objectives and adaptive windowing, achieving significant speedups across multiple benchmarks.
SlimSpec introduces a low-rank parameterization for drafter LM-heads to accelerate speculative decoding in LLMs, achieving 4-5x speedup while maintaining full vocabulary support.