Trees from Marginals: Autoregressive drafting with factorized priors
Summary
This paper introduces Weaver, a lightweight autoregressive adapter that constructs proposal trees from factorized drafters' top-K marginals, achieving a 4.37x speedup over autoregressive decoding and outperforming the DFlash baseline by 24.7%.
View Cached Full Text
Cached at: 07/09/26, 07:43 AM
# Trees from Marginals: Autoregressive drafting with factorized priors Source: [https://arxiv.org/abs/2607.06763](https://arxiv.org/abs/2607.06763) [View PDF](https://arxiv.org/pdf/2607.06763) > Abstract:Speculative decoding greatly increases the interactivity of autoregressive language models by trading off computation for extra tokens generated in a single forward pass\. Factorized draft models are especially efficient because they predict future\-token marginals in parallel, but their independence assumption causes acceptance rates to degrade sharply as the speculative budget grows\. We analyze this limitation and introduce Weaver, a lightweight autoregressive adapter that constructs proposal trees from the top\-K marginals of a factorized drafter\. Weaver restores conditional dependencies between proposed tokens while avoiding a full\-vocabulary projection\. To support fast verification for models with Gated Delta Net layers, we derive a rollback\-free tree\-verification algorithm and implement optimized CUDA kernels in SGLang\. By combining these model and systems contributions we achieve a 4\.37\-fold speedup over autoregressive decoding, and outperform a highly optimized DFlash baseline by 24\.7%\. ## Submission history From: Roman Knyazhitskiy \[[view email](https://arxiv.org/show-email/ece9bcdf/2607.06763)\] **\[v1\]**Tue, 7 Jul 2026 19:48:36 UTC \(956 KB\)
Similar Articles
GRAFT: Adaptive DLM-Based Draft Tree Construction with Target-Distilled Edge Scoring
GRAFT introduces a draft-tree construction framework for diffusion language model-based speculative decoding, optimizing edge selection and budget allocation to achieve 2.13×–6.36× speedup over autoregressive decoding with low overhead.
Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting
Proposes Progressive Tree Drafting (PTD), a training-free, model-agnostic speculative decoding method that uses progressive tree structures and stepwise pruning to enable parallel generation of multiple draft paths, achieving up to 2x speedup on various benchmarks.
PRESTO: Prefix-Aligned Tree Drafting for Diffusion Speculative Decoding
PRESTO introduces a prefix-aligned tree drafting framework for diffusion speculative decoding, achieving up to 1.5x speedup on dedicated diffusion drafters and 1.12x on self-speculative diffusion LLMs.
DominoTree: Conditional Tree-Structured Drafting with Domino for Speculative Decoding
DominoTree introduces a training-free best-first draft tree for speculative decoding that uses conditional (non-factorized) correction from Domino to achieve up to 6.6x speedup over autoregressive decoding and the highest mean accept length across evaluated methods on Qwen3 models.
Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding
Graft is a training-free framework that enhances speculative decoding by combining pruning and retrieval to improve acceptance rates and inference speed, achieving up to 5.41x speedup on short-context benchmarks and up to 21.8% improvement over EAGLE-3 on Qwen3-235B.