Tag
AdaptiveSpec is a training-free per-step speculative decoding method that adaptively adjusts token verification and draft tree shape to enhance LLM inference throughput, improving performance by up to 56% while maintaining high accuracy across benchmarks.
This paper analyzes lossy verification schemes in speculative decoding, categorizing them into truncation-based and collaborative verification, and identifies pitfalls and principles for maintaining generation quality.