Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding

arXiv cs.CL Papers

Summary

This paper proposes a principled framework for selecting decoder hidden-state layers in speculative decoding, using matrix-entropy analysis for informativeness and centered kernel alignment (CKA) for diversity, outperforming fixed heuristics in EAGLE3-style speculative decoding.

arXiv:2610.12939v1 Announce Type: new Abstract: Decoder hidden states from different layers of large language models (LLMs) often encode complementary information. However, existing speculative decoding methods typically rely on a small set of target-model layers chosen using fixed, position-based heuristics. Selecting an optimal subset of layers is inherently a combinatorial problem, making exhaustive evaluation infeasible even for moderately deep models. In this work, we present a principled framework for multi-layer hidden-state selection for speculative decoding based on two complementary criteria: informativeness, measured through matrix-entropy analysis, and diversity, quantified using centered kernel alignment (CKA). We investigate layer-wise Shannon entropy, aggregate entropy under hidden-state concatenation, and pairwise CKA dynamics across depth, revealing distinct geometric and informational compartments within decoder representations. Motivated by these findings, we propose entropy and CKA-based layer selection algorithms that favor hidden-state sets that are both highly informative and minimally redundant. As part of this framework, we introduce token-entropy filtering, which removes hidden states associated with low predictive-entropy tokens, thereby improving the quality of layer-selection statistics. In EAGLE3-style speculative decoding, our selected sets outperform fixed baseline choices, including the default $[2, N//2, N-3]$ rule, while providing a clearer explanation of why some defaults can remain competitive. Our results provide a practical recipe for multi-layer hidden-state selection for speculative decoding, insights into the geometry and information in LLM decoder states, and a foundation for extending similar analysis to related applications.
Original Article
View Cached Full Text

Cached at: 10/12/26, 06:50 AM

# Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding
Source: [https://arxiv.org/abs/2610.12939](https://arxiv.org/abs/2610.12939)
[View PDF](https://arxiv.org/pdf/2610.12939)

> Abstract:Decoder hidden states from different layers of large language models \(LLMs\) often encode complementary information\. However, existing speculative decoding methods typically rely on a small set of target\-model layers chosen using fixed, position\-based heuristics\. Selecting an optimal subset of layers is inherently a combinatorial problem, making exhaustive evaluation infeasible even for moderately deep models\. In this work, we present a principled framework for multi\-layer hidden\-state selection for speculative decoding based on two complementary criteria: informativeness, measured through matrix\-entropy analysis, and diversity, quantified using centered kernel alignment \(CKA\)\. We investigate layer\-wise Shannon entropy, aggregate entropy under hidden\-state concatenation, and pairwise CKA dynamics across depth, revealing distinct geometric and informational compartments within decoder representations\. Motivated by these findings, we propose entropy and CKA\-based layer selection algorithms that favor hidden\-state sets that are both highly informative and minimally redundant\. As part of this framework, we introduce token\-entropy filtering, which removes hidden states associated with low predictive\-entropy tokens, thereby improving the quality of layer\-selection statistics\. In EAGLE3\-style speculative decoding, our selected sets outperform fixed baseline choices, including the default $\[2, N//2, N\-3\]$ rule, while providing a clearer explanation of why some defaults can remain competitive\. Our results provide a practical recipe for multi\-layer hidden\-state selection for speculative decoding, insights into the geometry and information in LLM decoder states, and a foundation for extending similar analysis to related applications\.

## Submission history

From: Raghavv Goel Mr\. \[[view email](https://arxiv.org/show-email/41e008c5/2610.12939)\] **\[v1\]**Fri, 9 Oct 2026 03:38:26 UTC \(1,550 KB\)

Similar Articles

DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions

arXiv cs.AI

DecodeShare proposes a method to identify a low-dimensional subspace consistently shared across tasks in LLM decode-time hidden states and shows that disturbing this subspace degrades decision performance more than random or prefill-derived subspaces, with implications for activation steering.

A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decoding

arXiv cs.CL

This paper introduces SparseSpec-L, a training-free self-speculative decoding framework for long-context LLM inference that uses a dynamically sparsified and recallable KV cache along with an entropy-based controller for speculation length, achieving up to 2.79x speedup over autoregressive decoding.

The Economics of Speculative Decoding

Hacker News Top

A technical analysis of how mixture-of-experts and compressed attention change the cost-benefit of speculative decoding for LLM inference, explaining when speculated tokens become less free.

What is Speculative Decoding? (70 minute read)

TLDR AI

一篇70分钟阅读量的深度技术长文,从第一性原理推导投机解码(speculative decoding),构建涵盖EAGLE-3、多token预测(MTP)以及块扩散drafters(DFlash、DSpark)的统一框架,并讨论drafter训练中最小化KL散度的问题。