ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models

Hugging Face Daily Papers Papers
speculative-decoding linear-attention llm-inference inference-speedup recurrent-states efficient-inference

Summary

ReaDy introduces a speculative decoding method that uses hybrid linear attention recurrent states as dynamic context for drafting, achieving a 3.16x decoding speedup over autoregressive generation with Qwen3.8-27B on an NVIDIA B200 while improving throughput and acceptance length over DSpark across seven benchmarks.

Speculative decoding accelerates autoregressive generation by drafting multiple tokens per step, yet feature-based drafters rely on target hidden features whose contextual emphasis may not align with the needs of subsequent draft positions. Hybrid linear attention (HLA) models maintain compact recurrent states that accumulate historical associations, offering an efficient source of complementary context for speculative drafting. We introduce ReaDy, the first speculative decoding method, to the best of our knowledge, to leverage HLA recurrent states for position-specific context retrieval during drafting. ReaDy equips each draft position with a learned query to retrieve supplementary context directly from the target's committed recurrent states. A lightweight state reader integrates this context into draft representations through residual updates, while low-rank state reads and fused operators keep additional computation and I/O overhead low. Building on DSpark's parallel backbone and sequential prediction head, ReaDy employs a lightweight two-stage adaptation strategy, with reader capacity, state sources, and residual placement examined through ablation studies. Across seven benchmarks spanning mathematics, code generation, and dialogue, ReaDy achieves an average 3.16times decoding speedup over autoregressive decoding with Qwen3.8-27B on a single NVIDIA B200 GPU. It outperforms DSpark on every benchmark, improving throughput by 7.97% and acceptance length by 8.50% on average. Our code is available at https://github.com/StartLuxLabs/ReaDy.
Original Article
View Cached Full Text

Cached at: 10/19/26, 03:26 AM

Paper page - ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models

Source: https://huggingface.co/papers/2610.20503

Abstract

ReaDy accelerates autoregressive generation by using hybrid linear attention recurrent states to retrieve position-specific context for speculative drafting, improving speed and acceptance rates.

Speculative decodingacceleratesautoregressive generationby drafting multiple tokens per step, yetfeature-based draftersrely on target hidden features whose contextual emphasis may not align with the needs of subsequent draft positions.Hybrid linear attention(HLA) models maintain compactrecurrent statesthat accumulate historical associations, offering an efficient source of complementary context for speculative drafting. We introduceReaDy, the firstspeculative decodingmethod, to the best of our knowledge, to leverage HLArecurrent statesfor position-specific context retrieval during drafting.ReaDyequips each draft position with a learned query to retrieve supplementary context directly from the target’s committedrecurrent states. A lightweightstate readerintegrates this context into draft representations through residual updates, whilelow-rank state readsand fused operators keep additional computation and I/O overhead low. Building onDSpark’s parallel backbone and sequential prediction head,ReaDyemploys a lightweight two-stage adaptation strategy, with reader capacity, state sources, and residual placement examined through ablation studies. Across seven benchmarks spanning mathematics, code generation, and dialogue,ReaDyachieves an average 3.16times decoding speedup over autoregressive decoding with Qwen3.8-27B on a single NVIDIA B200 GPU. It outperformsDSparkon every benchmark, improving throughput by 7.97% and acceptance length by 8.50% on average. Our code is available at https://github.com/StartLuxLabs/ReaDy.

View arXiv pageView PDFGitHubAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.20503 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.20503 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.20503 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Attention Drift: What Autoregressive Speculative Decoding Models Learn

Reddit r/LocalLLaMA

This paper identifies 'attention drift' in autoregressive speculative decoding models, where drafters' attention shifts from the prompt to their own generated tokens. The authors propose architectural changes, such as post-norm and RMSNorm, which improve acceptance rates and robustness across various benchmarks.

DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

arXiv cs.CL

DeLS-Spec decouples long- and short-context modeling in speculative decoding by adding a lightweight local head to DFlash, achieving consistent speedups without full retraining. It requires only standard next-token prediction training for the local head and improves acceptance length on Qwen3 benchmarks.

Dynamic Linear Attention

arXiv cs.CL

This paper proposes DLA, a dynamic memory modeling framework for multi-state linear attention that adaptively merges states based on token information variation and maintains a fixed-size state cache, enabling better long-context representation without the quadratic complexity of standard attention.