Tag
Osprey introduces a target-agnostic pre-training method for drafters in speculative decoding, improving efficiency by bootstrapping from off-the-shelf models and adapting with minimal target-specific work, achieving significant acceptance rate improvements across multiple LLMs.
On Friday, we released six new state-of-the-art drafters for accelerated inference, along with a blog post on speculative decoding and a roofline model tool to estimate speedups.