Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding
Summary
This paper proposes a principled framework for selecting decoder hidden-state layers in speculative decoding, using matrix-entropy analysis for informativeness and centered kernel alignment (CKA) for diversity, outperforming fixed heuristics in EAGLE3-style speculative decoding.
View Cached Full Text
Cached at: 10/12/26, 06:50 AM
# Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding Source: [https://arxiv.org/abs/2610.12939](https://arxiv.org/abs/2610.12939) [View PDF](https://arxiv.org/pdf/2610.12939) > Abstract:Decoder hidden states from different layers of large language models \(LLMs\) often encode complementary information\. However, existing speculative decoding methods typically rely on a small set of target\-model layers chosen using fixed, position\-based heuristics\. Selecting an optimal subset of layers is inherently a combinatorial problem, making exhaustive evaluation infeasible even for moderately deep models\. In this work, we present a principled framework for multi\-layer hidden\-state selection for speculative decoding based on two complementary criteria: informativeness, measured through matrix\-entropy analysis, and diversity, quantified using centered kernel alignment \(CKA\)\. We investigate layer\-wise Shannon entropy, aggregate entropy under hidden\-state concatenation, and pairwise CKA dynamics across depth, revealing distinct geometric and informational compartments within decoder representations\. Motivated by these findings, we propose entropy and CKA\-based layer selection algorithms that favor hidden\-state sets that are both highly informative and minimally redundant\. As part of this framework, we introduce token\-entropy filtering, which removes hidden states associated with low predictive\-entropy tokens, thereby improving the quality of layer\-selection statistics\. In EAGLE3\-style speculative decoding, our selected sets outperform fixed baseline choices, including the default $\[2, N//2, N\-3\]$ rule, while providing a clearer explanation of why some defaults can remain competitive\. Our results provide a practical recipe for multi\-layer hidden\-state selection for speculative decoding, insights into the geometry and information in LLM decoder states, and a foundation for extending similar analysis to related applications\. ## Submission history From: Raghavv Goel Mr\. \[[view email](https://arxiv.org/show-email/41e008c5/2610.12939)\] **\[v1\]**Fri, 9 Oct 2026 03:38:26 UTC \(1,550 KB\)
Similar Articles
Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding
该论文提出一个基于信息量(矩阵熵)与多样性(CKA)的多层隐藏态选择框架,用于改进EAGLE3风格的推测解码,超越了固定的基于位置的层选择启发式方法。
DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions
DecodeShare proposes a method to identify a low-dimensional subspace consistently shared across tasks in LLM decode-time hidden states and shows that disturbing this subspace degrades decision performance more than random or prefill-derived subspaces, with implications for activation steering.
A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decoding
This paper introduces SparseSpec-L, a training-free self-speculative decoding framework for long-context LLM inference that uses a dynamically sparsified and recallable KV cache along with an entropy-based controller for speculation length, achieving up to 2.79x speedup over autoregressive decoding.
The Economics of Speculative Decoding
A technical analysis of how mixture-of-experts and compressed attention change the cost-benefit of speculative decoding for LLM inference, explaining when speculated tokens become less free.
What is Speculative Decoding? (70 minute read)
一篇70分钟阅读量的深度技术长文,从第一性原理推导投机解码(speculative decoding),构建涵盖EAGLE-3、多token预测(MTP)以及块扩散drafters(DFlash、DSpark)的统一框架,并讨论drafter训练中最小化KL散度的问题。