Tag
Verification-Aware Training (VAT) improves draft models for speculative decoding by simulating sequential verification during training and adapting loss weights to acceptance patterns, leading to enhanced acceptance length and inference speedup.
LiquidAI releases draft models for speculative decoding to accelerate their LFM2.5 models, achieving up to 2× faster inference on H100 and Apple silicon without quality degradation.
The article presents benchmark results for DeepSeek V4 Flash 0731 on Strix Halo hardware, showing performance with different draft models and n_max settings, concluding that n_max=3 offers the best speed balance.
This paper compares three strategies to improve speculative decoding efficiency for non-English languages, finding that task-specific distillation improves acceptance rates but generalizes poorly, while n-gram draft models offer consistent speed-ups despite lower acceptance rates.
Draft-OPD introduces on-policy distillation with target-assisted rollouts and error replay to overcome the offline-to-inference mismatch in training draft models for speculative decoding, achieving over 5x lossless acceleration and improving upon EAGLE-3 and DFlash by 23% and 13% respectively.
ConFu introduces a novel speculative decoding framework that enables draft models to anticipate future generation directions through contemplate tokens and soft prompts, achieving 8-20% improvements in token acceptance rates and generation speed over EAGLE-3 across multiple LLM models.