Depth-Entropy Guided Sampling for Training-Free LLM Reasoning

# Depth-Entropy Guided Sampling for Training-Free LLM Reasoning
Source: [https://arxiv.org/html/2607.09693](https://arxiv.org/html/2607.09693)
Zibin Meng1Peng Xie1Kani Chen1

\{zmengal, pxieaf\}@connect\.ust\.hkmakchen@ust\.hk 1The Hong Kong University of Science and Technology

###### Abstract

Reinforcement learning \(RL\) has become the dominant paradigm for improving the reasoning capabilities of large language models, but it requires expensive training, curated data, and reward signals\. Recent work shows that sampling from sharpened base\-model distributions at test time recovers much of the RL gain, yet existing methods rely solely on output\-layer likelihoods and ignore the transformer’s internal forward\-pass dynamics\. We introduceDepth\-Entropy Guided Sampling \(DEGS\), a training\-free, test\-time method that exploits*layer\-wise entropy collapse*as an intrinsic quality signal\. We observe that stronger reasoners—including RL\-posttrained variants—exhibit a distinctive “late collapse”: logit\-lens–decoded entropy stays elevated until deeper layers before converging\. We define a per\-sequence*collapse depth*D​\(𝐱\)D\(\\mathbf\{x\}\)and a joint objectiveπ​\(𝐱\)∝p​\(𝐱\)α​exp⁡\(β​D​\(𝐱\)\)\\pi\(\\mathbf\{x\}\)\\propto p\(\\mathbf\{x\}\)^\{\\alpha\}\\exp\\\!\\bigl\(\\beta\\,D\(\\mathbf\{x\}\)\\bigr\)that combines sequence likelihood with this depth\-entropy structure, instantiated inside an MCMC power\-sampling framework \(DEGS\-MCMC\)\. Across three open\-weight models and four reasoning benchmarks, this near\-chance per\-candidate signal compounds over the sampling trajectory into state\-of\-the\-art training\-free accuracy, with gains largest out of domain and on the harder splits—exactly where likelihood alone falls short—at single\-digit\-percent wall\-clock overhead\. DEGS narrowly trails an in\-house GRPO reference on the math splits GRPO was trained for, yet surpasses it out of domain on GPQA for all three models, without any training, reward model, or labeled data\.

## 1Introduction

Reinforcement learning \(RL\) with verifiable rewards—e\.g\., Group Relative Policy Optimization \(GRPO\)\(Shao et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib32)\)—is the dominant paradigm for improving the reasoning capabilities of large language models \(LLMs\), posttraining frontier models to sizeable gains on mathematical, scientific, and coding benchmarks\(Guo et al\.,[2025](https://arxiv.org/html/2607.09693#bib.bib11)\)\. Yet these gains are costly: RL posttraining requires curated data, extensive tuning, and—most critically—a reliable reward signal, which is unavailable in many domains of practical interest\(Prabhudesai et al\.,[2025](https://arxiv.org/html/2607.09693#bib.bib26)\)\. A growing body of evidence further suggests they may not reflect fundamentally new capabilities: RL\-posttrained models concentrate probability mass on traces that*already*have high likelihood under the base model\(He et al\.,[2025](https://arxiv.org/html/2607.09693#bib.bib12); Yue et al\.,[2025](https://arxiv.org/html/2607.09693#bib.bib43)\), and base models can even outperform them in the multi\-shot regime due to degraded diversity\(Song et al\.,[2025](https://arxiv.org/html/2607.09693#bib.bib35)\)\. This*distribution sharpening*hypothesis\(Shao et al\.,[2025](https://arxiv.org/html/2607.09693#bib.bib33)\)casts RL mainly as a filter, redistributing pass@kkcapability into single\-shot performance rather than teaching genuinely novel reasoning\.

This has inspired*training\-free*methods that sharpen the base distribution at inference time, part of a broader move toward scaling test\-time computation\(Snell et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib34); Welleck et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib40); Brown et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib3)\)\.Karan and Du \([2025](https://arxiv.org/html/2607.09693#bib.bib17)\)formalize this through the*power distribution*p​\(𝐱\)αp\(\\mathbf\{x\}\)^\{\\alpha\}\(α\>1\\alpha\>1\), targeted by Metropolis–Hastings \(MH\) sampling, to match GRPO without training, data, or reward models; Scalable Power Sampling\(Ji et al\.,[2026](https://arxiv.org/html/2607.09693#bib.bib15)\)and Power\-SMC\(Azizi et al\.,[2026](https://arxiv.org/html/2607.09693#bib.bib1)\)improve efficiency via token\-level approximation and a batch\-parallel SMC scheme\. Together they show base models are far more capable at single\-shot reasoning than standard sampling reveals\.

Despite their success, all existing training\-free methods rely exclusively on the*output\-layer likelihood*p​\(𝐱\)p\(\\mathbf\{x\}\), ignoring the structured intermediate representations of the forward pass\(Nostalgebraist,[2020](https://arxiv.org/html/2607.09693#bib.bib24); Belrose et al\.,[2023](https://arxiv.org/html/2607.09693#bib.bib2); Wendler et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib41)\)\. This is a missed opportunity:Wendler et al\. \([2024](https://arxiv.org/html/2607.09693#bib.bib41)\)identify three forward\-pass phases \(input processing, a middle “concept space,” and output mapping\), andTan et al\. \([2025](https://arxiv.org/html/2607.09693#bib.bib36)\)find that entropy evolves from high\-entropy exploration in early layers to deterministic refinement at the top\.

We contribute an empirical observation bridging these lines: comparing*layer\-wise entropy trajectories*of base models against their RL\-posttrained counterparts, stronger reasoners consistently exhibitlater entropy collapse\(Figure[1](https://arxiv.org/html/2607.09693#S4.F1)\)—base models lock in predictions at shallow layers, while RL\-posttrained models sustain elevated internal uncertainty until much deeper\. This extended “deliberation phase” is associated with downstream accuracy \(a small but consistent effect, quantified in Section[5](https://arxiv.org/html/2607.09693#S5)\) and offers a new lens on what RL posttraining achieves internally\. This motivates a simple idea:*if late entropy collapse is characteristic of strong reasoning, we can use it as a training\-free quality signal at test time*\. We formalize it through a scalar*collapse depth*D​\(𝐱\)D\(\\mathbf\{x\}\)and construct a joint objectiveπ​\(𝐱\)∝p​\(𝐱\)α⋅exp⁡\(β​D​\(𝐱\)\)\\pi\(\\mathbf\{x\}\)\\propto p\(\\mathbf\{x\}\)^\{\\alpha\}\\cdot\\exp\(\\beta\\,D\(\\mathbf\{x\}\)\)that augments the power distribution with this signal\. The resulting method,Depth\-Entropy Guided Sampling \(DEGS\), integrates into the MH power\-sampling framework \(Algorithm[1](https://arxiv.org/html/2607.09693#alg1)\)—without parameter updates, reward models, or labeled data\.

We keep two linked claims distinct: the*motivation*\(Figure[1](https://arxiv.org/html/2607.09693#S4.F1)\) is a cross\-stage contrast—posttrained reasoners collapse later than their base models—while the*operative*claim our reweighting of base\-model samples exploits is internal to a single base model: among its candidates, the correct ones collapse later than the incorrect ones, which we verify directly in Section[5](https://arxiv.org/html/2607.09693#S5)\(Figure[2](https://arxiv.org/html/2607.09693#S5.F2)\)\.

Our contributions are as follows:

1. 1\.We identifylate entropy collapseas an empirical signature of strong reasoning models and show that the collapse depthD​\(𝐱\)D\(\\mathbf\{x\}\)is a statistically significant predictor of solution correctness across models and benchmarks \(Section[3](https://arxiv.org/html/2607.09693#S3)\)\.
2. 2\.We proposeDEGS, built on a joint likelihood–depth objectiveπ​\(𝐱\)∝p​\(𝐱\)α​exp⁡\(β​D​\(𝐱\)\)\\pi\(\\mathbf\{x\}\)\\propto p\(\\mathbf\{x\}\)^\{\\alpha\}\\exp\(\\beta D\(\\mathbf\{x\}\)\)\(Section[4](https://arxiv.org/html/2607.09693#S4)\), in two decoders: a final\-layer reranker,*Best\-of\-NN\-Entropy*, which beats log\-likelihood Best\-of\-NNin all twelve cells by modest margins, and our main methodDEGS\-MCMC, which exploits the full depth profile to improve over likelihood\-only power sampling in all twelve cells by substantially larger margins \(median≈4\\approx 4points; Table[1](https://arxiv.org/html/2607.09693#S5.T1)\)\.
3. 3\.We characterizehow a weak per\-candidate signal becomes a reliable decoding gain: collapse depth is only a near\-chance discriminator in isolation \(AUC≈0\.6\\approx 0\.6; Section[3](https://arxiv.org/html/2607.09693#S3)\), yet by Proposition[8](https://arxiv.org/html/2607.09693#Thmproposition8)it can only reorder near\-equal\-likelihood candidates, so compounding this tiebreak over the MCMC trajectory yields a gain positive in all twelve cells and largest on the harder, out\-of\-domain splits where likelihood is weakest\.

To our knowledge, DEGS is the first method to use the*depth\-wise evolution of internal uncertainty*—rather than output\-layer signals alone—as a test\-time selection criterion for reasoning, indicating that intermediate representations carry quality signals not reducible to sequence\-level likelihood\.

## 2Related Work

#### RL posttraining and distribution sharpening\.

RL with verifiable rewards \(GRPO\)\(Shao et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib32); Guo et al\.,[2025](https://arxiv.org/html/2607.09693#bib.bib11)\), building on RL from human feedback\(Ouyang et al\.,[2022](https://arxiv.org/html/2607.09693#bib.bib25)\)and policy\-gradient methods such as PPO\(Schulman et al\.,[2017](https://arxiv.org/html/2607.09693#bib.bib30)\), is now the dominant approach for improving LLM reasoning, but growing evidence attributes its gains primarily to*distribution sharpening*rather than new capabilities\(He et al\.,[2025](https://arxiv.org/html/2607.09693#bib.bib12); Yue et al\.,[2025](https://arxiv.org/html/2607.09693#bib.bib43); Song et al\.,[2025](https://arxiv.org/html/2607.09693#bib.bib35)\), motivating the training\-free power\-sampling line above\(Karan and Du,[2025](https://arxiv.org/html/2607.09693#bib.bib17); Ji et al\.,[2026](https://arxiv.org/html/2607.09693#bib.bib15); Azizi et al\.,[2026](https://arxiv.org/html/2607.09693#bib.bib1)\); verifier\-based reranking, process/outcome reward models, self\-consistency, and pure repeated sampling\(Cobbe et al\.,[2021](https://arxiv.org/html/2607.09693#bib.bib6); Uesato et al\.,[2022](https://arxiv.org/html/2607.09693#bib.bib37); Lightman et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib21); Wang et al\.,[2023](https://arxiv.org/html/2607.09693#bib.bib38); Brown et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib3)\)are simpler alternatives\. Using a quality signal*inside*MH connects to QuEST\(Faria et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib8)\), SMC steering of language\-model programs\(Lew et al\.,[2023](https://arxiv.org/html/2607.09693#bib.bib20)\), and twist\-guided sequential Monte Carlo\(Zhao et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib45)\), but those steer the chain with an*external*, task\-specific estimator \(a learned MT metric, a trained twist\), whereas DEGS reads its signal off the base model’s own intermediate representations—no auxiliary model, reward, or training\. All these methods rely on*output\-layer*signals; DEGS complements them with*internal*depth\-entropy dynamics\.

#### Internal representations and logit lens\.

The logit lens\(Nostalgebraist,[2020](https://arxiv.org/html/2607.09693#bib.bib24)\), tuned lens\(Belrose et al\.,[2023](https://arxiv.org/html/2607.09693#bib.bib2)\), and unifying frameworks such as Patchscopes\(Ghandeharioun et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib10)\)decode intermediate hidden states into token distributions, revealing evolving internal beliefs;Wendler et al\. \([2024](https://arxiv.org/html/2607.09693#bib.bib41)\)show these carry information orthogonal to output tokens, andTan et al\. \([2025](https://arxiv.org/html/2607.09693#bib.bib36)\)find early layers maintain high entropy while top layers converge\. Closest to our use of depth is DoLa\(Chuang et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib5)\), which contrasts logit\-lens projections of a later and an earlier layer to reduce hallucination; unlike DoLa’s per\-token, two\-layer contrast, DEGS leaves the token distribution unchanged and instead aggregates the full depth profile of entropy collapse into a sequence\-level reweighting signal, used as a*test\-time selection signal*rather than for interpretation or RL optimization\.

#### Entropy signals for reasoning\.

Prabhudesai et al\. \([2025](https://arxiv.org/html/2607.09693#bib.bib26)\)maximize base\-model confidence via RL,Zhao et al\. \([2025](https://arxiv.org/html/2607.09693#bib.bib44)\)use self\-entropy as an RL reward, andCui et al\. \([2025](https://arxiv.org/html/2607.09693#bib.bib7)\)analyze*output\-policy*entropy collapse during RL training; all operate on output\-layer entropy within a training framework, whereas DEGS exploits entropy*across the depth dimension*entirely at*test time*\.

#### Confidence, calibration, and adaptive depth\.

Models’*own*uncertainty is also a usable signal: large models are often well calibrated and can partly predict their own correctness\(Kadavath et al\.,[2022](https://arxiv.org/html/2607.09693#bib.bib16)\), and output\-distribution measures such as semantic entropy flag unreliable generations\(Farquhar et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib9)\)\. Closest in spirit to our per\-token collapse depth is*confidence\-based early exiting*\(Schuster et al\.,[2022](https://arxiv.org/html/2607.09693#bib.bib31)\), which halts computation at a shallow layer once a per\-token confidence criterion is met; DEGS inverts this efficiency framing—rather than exiting early to save compute, it*reads how deep*the network defers commitment as a test\-time quality signal, leaving the forward pass intact\.

#### Notation\.

We consider an autoregressive LLMp​\(𝐱\)=∏tp​\(xt∣x<t\)p\(\\mathbf\{x\}\)=\\prod\_\{t\}p\(x\_\{t\}\\mid x\_\{<t\}\)withLLtransformer layers and hidden states𝐡l,t\\mathbf\{h\}\_\{l,t\}\. The*logit lens*\(Nostalgebraist,[2020](https://arxiv.org/html/2607.09693#bib.bib24)\)decodes layerllvia the unembeddingUU,pl​\(v∣x≤t\)=softmax​\(U​𝐡l,t\)vp\_\{l\}\(v\\mid x\_\{\\leq t\}\)=\\mathrm\{softmax\}\(U\\mathbf\{h\}\_\{l,t\}\)\_\{v\}, with layer\-wise entropyHl​\(t\)=−∑vpl​\(v\)​log⁡pl​\(v\)H\_\{l\}\(t\)=\-\\sum\_\{v\}p\_\{l\}\(v\)\\log p\_\{l\}\(v\); the*power distribution*\(Karan and Du,[2025](https://arxiv.org/html/2607.09693#bib.bib17)\)pα​\(𝐱\)∝p​\(𝐱\)αp^\{\\alpha\}\(\\mathbf\{x\}\)\\propto p\(\\mathbf\{x\}\)^\{\\alpha\}\(α≥1\\alpha\\geq 1\) is targeted by MH sampling\. Full definitions are in Appendix[B](https://arxiv.org/html/2607.09693#A2)\.

## 3Motivating Observation: Late Entropy Collapse in Strong Reasoners

Comparing layer\-wise entropy trajectories\{Hl​\(t\)\}l=1L\\\{H\_\{l\}\(t\)\\\}\_\{l=1\}^\{L\}across training stages \(Figure[1](https://arxiv.org/html/2607.09693#S4.F1)\),base modelscollapse early \(entropy drops to near zero at shallow layers\), whileRL\-posttrained modelssustain elevated entropy significantly deeper before converging—consistent with the three\-phase structure ofWendler et al\. \([2024](https://arxiv.org/html/2607.09693#bib.bib41)\)\. In Figures[1](https://arxiv.org/html/2607.09693#S4.F1)\(a\)–\(b\), base Qwen2\.5\-Math\-7B drops below 2\.5 nats by layer 10 whereas its RL\-posttrained variant stays above 5 nats until layer 15–20; the same pattern holds for DeepSeek\-Math\-7B \(Figures[1](https://arxiv.org/html/2607.09693#S4.F1)\(c\)–\(d\)\), and panel \(e\) confirms higher mean collapse depth for posttrained models across all four settings\. Combined with the finding that base models already contain high\-quality reasoning traces\(Karan and Du,[2025](https://arxiv.org/html/2607.09693#bib.bib17)\), this motivates our hypothesis:*among base\-model candidates, those whose entropy collapses later are more likely to be correct*\.

This is the cross\-stage*motivation*; the base\-internal claim our method uses is verified in Section[5](https://arxiv.org/html/2607.09693#S5)\(Figure[2](https://arxiv.org/html/2607.09693#S5.F2)\)\. Appendix[A](https://arxiv.org/html/2607.09693#A1)\(Figure[3](https://arxiv.org/html/2607.09693#A1.F3)\) extends the contrast to all twelve overlays, including GSM8K\(Cobbe et al\.,[2021](https://arxiv.org/html/2607.09693#bib.bib6)\)and Qwen2\.5\-Math\-72B \(motivation only\)\.

## 4Method: Depth\-Entropy Guided Sampling

We formalize the observation into a test\-time algorithm: we define*collapse depth*, construct the DEGS objective, and present a lightweight reranking variant alongside the MCMC variant we ship\.

### 4\.1Collapse Depth

\(a\) Qwen2\.5\-Math\-7B on MATH500
![Refer to caption](https://arxiv.org/html/2607.09693v1/x1.png)\(b\) Qwen2\.5\-Math\-7B on DeepMind500
![Refer to caption](https://arxiv.org/html/2607.09693v1/x2.png)\(c\) DeepSeek\-Math\-7B on MATH500
![Refer to caption](https://arxiv.org/html/2607.09693v1/x3.png)\(d\) DeepSeek\-Math\-7B on DeepMind500
![Refer to caption](https://arxiv.org/html/2607.09693v1/x4.png)
\(e\) Summary statistics of collapse depth ![Refer to caption](https://arxiv.org/html/2607.09693v1/x5.png)

Figure 1:Late entropy collapse in stronger reasoners\. Panels \(a\)–\(d\) plot layer\-wise entropy trajectories for representative base and posttrained models on mathematical benchmarks; panel \(e\) summarizes the effect via the mean collapse*layer*dt¯\\overline\{d\_\{t\}\}\(average over tokens of the shallowest probed layer reaching the threshold, in absolute layer index—not the normalizedD​\(𝐱\)∈\(0,1\]D\(\\mathbf\{x\}\)\\in\(0,1\]of Definition[2](https://arxiv.org/html/2607.09693#Thmdefinition2)\)\. Stronger reasoners collapse later in depth—equivalently, a largerD​\(𝐱\)D\(\\mathbf\{x\}\)—suggesting that internal uncertainty dynamics may provide a useful test\-time signal\.#### Per\-token collapse depth\.

Consider a generated sequence𝐱=\(x0,…,xT\)\\mathbf\{x\}=\(x\_\{0\},\\ldots,x\_\{T\}\)from a base modelpp\. To reduce cost, we compute entropy on a*layer subset*

ℒsub=\{l1,…,lK\}⊂\{1,…,L\},\\mathcal\{L\}\_\{\\mathrm\{sub\}\}=\\\{l\_\{1\},\\ldots,l\_\{K\}\\\}\\subset\\\{1,\\ldots,L\\\},\(1\)withK≪LK\\ll L; by default we use evenly spacedℒsub=\{4,8,12,16,20,24,28\}\\mathcal\{L\}\_\{\\mathrm\{sub\}\}=\\\{4,8,12,16,20,24,28\\\}for a 28\-layer model \(K=7K=7\)\.

###### Definition 1\(Per\-token collapse depth\)\.

Given an entropy thresholdτ\>0\\tau\>0, the*collapse depth*of token positionttis

dt=min⁡\{l∈ℒsub\|Hl​\(t\)≤τ\},d\_\{t\}=\\min\\bigl\\\{l\\in\\mathcal\{L\}\_\{\\mathrm\{sub\}\}\\;\\big\|\\;H\_\{l\}\(t\)\\leq\\tau\\bigr\\\},\(2\)with the conventiondt=Ld\_\{t\}=Lif no layer inℒsub\\mathcal\{L\}\_\{\\mathrm\{sub\}\}satisfies the threshold\.

Intuitivelydtd\_\{t\}measures how deep the network processes before becoming “confident” about tokentt: smalldtd\_\{t\}indicates \(possibly premature\) early commitment, largedtd\_\{t\}sustained deliberation\.

#### Sequence\-level collapse depth\.

###### Definition 2\(Sequence\-level collapse depth\)\.

The*normalized collapse depth*of𝐱=\(x0,…,xT\)\\mathbf\{x\}=\(x\_\{0\},\\ldots,x\_\{T\}\)is

D​\(𝐱\)=1T\+1​∑t=0TdtL\.D\(\\mathbf\{x\}\)=\\frac\{1\}\{T\+1\}\\sum\_\{t=0\}^\{T\}\\frac\{d\_\{t\}\}\{L\}\.\(3\)

By constructionD​\(𝐱\)∈\(0,1\]D\(\\mathbf\{x\}\)\\in\(0,1\]; larger values indicate internal uncertainty persisting to deeper layers on average\. Our central hypothesis \(validated in Section[5](https://arxiv.org/html/2607.09693#S5)\) is that*correct solutions tend to have largerD​\(𝐱\)D\(\\mathbf\{x\}\)*\. The thresholdτ\\tau\(default0\.250\.25nats\) controls sensitivity; DEGS is robust across a range of values \(Table[8](https://arxiv.org/html/2607.09693#A5.T8)\)\.

### 4\.2The DEGS Objective

Standard power sampling\(Karan and Du,[2025](https://arxiv.org/html/2607.09693#bib.bib17)\)targetsp​\(𝐱\)αp\(\\mathbf\{x\}\)^\{\\alpha\}using only the output\-layer likelihood\. We augment it with the depth\-entropy structure captured byD​\(𝐱\)D\(\\mathbf\{x\}\)\.

###### Definition 3\(DEGS target distribution\)\.

The DEGS target distribution over sequences is

π​\(𝐱\)∝p​\(𝐱\)α⋅exp⁡\(β⋅D​\(𝐱\)\),\\pi\(\\mathbf\{x\}\)\\propto p\(\\mathbf\{x\}\)^\{\\alpha\}\\cdot\\exp\\\!\\bigl\(\\beta\\cdot D\(\\mathbf\{x\}\)\\bigr\),\(4\)whereα≥1\\alpha\\geq 1is the likelihood exponent andβ≥0\\beta\\geq 0the depth weight\.

Thelikelihood termp​\(𝐱\)αp\(\\mathbf\{x\}\)^\{\\alpha\}favors globally coherent sequences \(as in power sampling\); thedepth\-entropy termexp⁡\(β​D​\(𝐱\)\)\\exp\(\\beta\\,D\(\\mathbf\{x\}\)\)favors sequences whose internal processing resembles strong reasoners’—elevated uncertainty across layers before committing\. It reduces to power sampling atβ=0\\beta=0and depth\-only scoring atα=0\\alpha=0\(not expected to perform well alone\)\. Equivalently, the log\-score for ranking is

S​\(𝐱\)=α⋅log⁡p​\(𝐱\)\+β⋅D​\(𝐱\)\.S\(\\mathbf\{x\}\)=\\alpha\\cdot\\log p\(\\mathbf\{x\}\)\+\\beta\\cdot D\(\\mathbf\{x\}\)\.\(5\)Defaults:α=4\\alpha\{=\}4,β=5\\beta\{=\}5,τ=0\.25\\tau\{=\}0\.25nats,ℒsub=\{4,8,…,28\}\\mathcal\{L\}\_\{\\mathrm\{sub\}\}\{=\}\\\{4,8,\\ldots,28\\\},N=16N\{=\}16candidates,TMCMC=10T\_\{\\mathrm\{MCMC\}\}\{=\}10MH steps; full ranges in Table[8](https://arxiv.org/html/2607.09693#A5.T8)\.

### 4\.3Reranking Variants

Both variants use one scoring signal but apply it differently: reranking applies the depth tiebreak once, whereas DEGS\-MCMC compounds it across the sampling trajectory\.Best\-of\-NN\-Entropyranks a fixed pool ofNNcandidates by final\-layer entropy alone; it beats log\-likelihood Best\-of\-NNin all twelve cells \(Table[1](https://arxiv.org/html/2607.09693#S5.T1)\), but by modest margins and well below the MCMC family\.DEGS rerankinginstead uses the full depth scoreS​\(𝐱\)=α​log⁡p​\(𝐱\)\+β​D​\(𝐱\)S\(\\mathbf\{x\}\)=\\alpha\\log p\(\\mathbf\{x\}\)\+\\beta D\(\\mathbf\{x\}\)\(Algorithm[2](https://arxiv.org/html/2607.09693#alg2), Appendix[C](https://arxiv.org/html/2607.09693#A3)\), computed from one teacher\-forced forward that caches hidden states atℒsub\\mathcal\{L\}\_\{\\mathrm\{sub\}\}\. By Proposition[8](https://arxiv.org/html/2607.09693#Thmproposition8)the depth term only reorders near\-equal\-likelihood candidates, so a single argmax overN=16N\{=\}16extracts little from it; we therefore keep DEGS reranking as a controlled probe of*where*in depth the signal acts \(Table[7](https://arxiv.org/html/2607.09693#A5.T7)\) and ship DEGS\-MCMC, which compounds the same tiebreak over the trajectory at a few percent overhead relative to Best\-of\-NN\(Table[6](https://arxiv.org/html/2607.09693#A5.T6)\)\.

### 4\.4DEGS\-MCMC: Integration with Power Sampling

For a more sample\-efficient exploration of equation[4](https://arxiv.org/html/2607.09693#S4.E4), we integrate DEGS into the MCMC power\-sampling framework ofKaran and Du \([2025](https://arxiv.org/html/2607.09693#bib.bib17)\)\.

#### Acceptance ratio\.

In standard power sampling the target isp​\(𝐱\)αp\(\\mathbf\{x\}\)^\{\\alpha\}with MH acceptance ratioAstd​\(𝐱′,𝐱\)=min⁡\(1,p​\(𝐱′\)α​q​\(𝐱∣𝐱′\)p​\(𝐱\)α​q​\(𝐱′∣𝐱\)\)A\_\{\\mathrm\{std\}\}\(\\mathbf\{x\}^\{\\prime\},\\mathbf\{x\}\)=\\min\\bigl\(1,\\frac\{p\(\\mathbf\{x\}^\{\\prime\}\)^\{\\alpha\}\\,q\(\\mathbf\{x\}\\mid\\mathbf\{x\}^\{\\prime\}\)\}\{p\(\\mathbf\{x\}\)^\{\\alpha\}\\,q\(\\mathbf\{x\}^\{\\prime\}\\mid\\mathbf\{x\}\)\}\\bigr\), whereqqis the proposal \(random\-index resampling\)\. Replacing the target withπ​\(𝐱\)\\pi\(\\mathbf\{x\}\)from equation[4](https://arxiv.org/html/2607.09693#S4.E4)yields

ADEGS​\(𝐱′,𝐱\)=min⁡\(1,p​\(𝐱′\)α​exp⁡\(β​D​\(𝐱′\)\)​q​\(𝐱∣𝐱′\)p​\(𝐱\)α​exp⁡\(β​D​\(𝐱\)\)​q​\(𝐱′∣𝐱\)\)\.A\_\{\\mathrm\{DEGS\}\}\(\\mathbf\{x\}^\{\\prime\},\\mathbf\{x\}\)=\\min\\\!\\left\(1,\\;\\frac\{p\(\\mathbf\{x\}^\{\\prime\}\)^\{\\alpha\}\\,\\exp\\\!\\bigl\(\\beta\\,D\(\\mathbf\{x\}^\{\\prime\}\)\\bigr\)\\,q\(\\mathbf\{x\}\\mid\\mathbf\{x\}^\{\\prime\}\)\}\{p\(\\mathbf\{x\}\)^\{\\alpha\}\\,\\exp\\\!\\bigl\(\\beta\\,D\(\\mathbf\{x\}\)\\bigr\)\\,q\(\\mathbf\{x\}^\{\\prime\}\\mid\\mathbf\{x\}\)\}\\right\)\.\(6\)All other components—proposal, block structure, chain selection—remain identical to Algorithm 1 ofKaran and Du \([2025](https://arxiv.org/html/2607.09693#bib.bib17)\); the only addition is an entropy\-probing pass to computeD​\(⋅\)D\(\\cdot\)\.

#### Algorithmic structure\.

FollowingKaran and Du \([2025](https://arxiv.org/html/2607.09693#bib.bib17)\), we divide the sequence into blocks of sizeBBwith intermediate targetsπk​\(x0:k​B\)∝p​\(x0:k​B\)α​exp⁡\(β​D​\(x0:k​B\)\)\\pi\_\{k\}\(x\_\{0:kB\}\)\\propto p\(x\_\{0:kB\}\)^\{\\alpha\}\\exp\\\!\\bigl\(\\beta D\(x\_\{0:kB\}\)\\bigr\), extending each prefix with the proposal model and then runningNMCMCN\_\{\\mathrm\{MCMC\}\}MH steps with equation[6](https://arxiv.org/html/2607.09693#S4.E6)\(Algorithm[1](https://arxiv.org/html/2607.09693#alg1)\)\.

Algorithm 1DEGS\-MCMC \(Entropy\-Guided Power Sampling\)0:Base model

pp; proposal model

ppropp\_\{\\mathrm\{prop\}\}; prompt;

α,β,τ\\alpha,\\beta,\\tau; block size

BB; max length

TT; MCMC steps

NMCMCN\_\{\\mathrm\{MCMC\}\}; layer subset

ℒsub\\mathcal\{L\}\_\{\\mathrm\{sub\}\}
0:Sequence

𝐱0:T∼π\\mathbf\{x\}\_\{0:T\}\\sim\\pi
1:for

k=0k=0to

⌈T/B⌉−1\\lceil T/B\\rceil\-1do

2:Initialize:Extend prefix

𝐱0:k​B\\mathbf\{x\}\_\{0:kB\}by sampling

xt∼pprop\(⋅∣x<t\)x\_\{t\}\\sim p\_\{\\mathrm\{prop\}\}\(\\cdot\\mid x\_\{<t\}\)for

t=k​B\+1,…,\(k\+1\)​Bt=kB\{\+\}1,\\ldots,\(k\{\+\}1\)B
3:Set current state

𝐱←𝐱0:\(k\+1\)​B\\mathbf\{x\}\\leftarrow\\mathbf\{x\}\_\{0:\(k\+1\)B\}
4:for

n=1n=1to

NMCMCN\_\{\\mathrm\{MCMC\}\}do

5:Sample index

m∼Uniform​\{1,…,\(k\+1\)​B\}m\\sim\\mathrm\{Uniform\}\\\{1,\\ldots,\(k\{\+\}1\)B\\\}
6:Construct proposal

𝐱′\\mathbf\{x\}^\{\\prime\}: keep prefix

x0:m−1x\_\{0:m\-1\}, resample

xt′∼pprop\(⋅∣x<t′\)x^\{\\prime\}\_\{t\}\\sim p\_\{\\mathrm\{prop\}\}\(\\cdot\\mid x^\{\\prime\}\_\{<t\}\)for

t=m,…,\(k\+1\)​Bt=m,\\ldots,\(k\{\+\}1\)B
7:Compute

log⁡p​\(𝐱\)\\log p\(\\mathbf\{x\}\),

log⁡p​\(𝐱′\)\\log p\(\\mathbf\{x\}^\{\\prime\}\),

D​\(𝐱\)D\(\\mathbf\{x\}\),

D​\(𝐱′\)D\(\\mathbf\{x\}^\{\\prime\}\),

q​\(𝐱∣𝐱′\)q\(\\mathbf\{x\}\\mid\\mathbf\{x\}^\{\\prime\}\),

q​\(𝐱′∣𝐱\)q\(\\mathbf\{x\}^\{\\prime\}\\mid\\mathbf\{x\}\)
8:

A←min⁡\(1,p​\(𝐱′\)α​exp⁡\(β​D​\(𝐱′\)\)​q​\(𝐱∣𝐱′\)p​\(𝐱\)α​exp⁡\(β​D​\(𝐱\)\)​q​\(𝐱′∣𝐱\)\)A\\leftarrow\\min\\\!\\Big\(1,\\;\\frac\{p\(\\mathbf\{x\}^\{\\prime\}\)^\{\\alpha\}\\exp\(\\beta\\,D\(\\mathbf\{x\}^\{\\prime\}\)\)\\,q\(\\mathbf\{x\}\\mid\\mathbf\{x\}^\{\\prime\}\)\}\{p\(\\mathbf\{x\}\)^\{\\alpha\}\\exp\(\\beta\\,D\(\\mathbf\{x\}\)\)\\,q\(\\mathbf\{x\}^\{\\prime\}\\mid\\mathbf\{x\}\)\}\\Big\)
9:if

Uniform​\(0,1\)≤A\\mathrm\{Uniform\}\(0,1\)\\leq Athen

10:

𝐱←𝐱′\\mathbf\{x\}\\leftarrow\\mathbf\{x\}^\{\\prime\}\{accept proposal\}

11:endif

12:endfor

13:Fix prefix

𝐱0:\(k\+1\)​B←𝐱\\mathbf\{x\}\_\{0:\(k\+1\)B\}\\leftarrow\\mathbf\{x\}
14:endfor

15:return

𝐱0:T\\mathbf\{x\}\_\{0:T\}

#### Convergence and cost\.

The chain converges toπ\\piasNMCMC→∞N\_\{\\mathrm\{MCMC\}\}\\to\\inftyby standard MCMC theory\(Neal,[1993](https://arxiv.org/html/2607.09693#bib.bib23)\)—π​\(𝐱\)\\pi\(\\mathbf\{x\}\)has nonzero mass whereverp​\(𝐱\)\>0p\(\\mathbf\{x\}\)\>0and the random\-index resampling proposal is irreducible and aperiodic\(Karan and Du,[2025](https://arxiv.org/html/2607.09693#bib.bib17)\)\(full statement and proof in Appendix[F\.2](https://arxiv.org/html/2607.09693#A6.SS2)\)\. Cost is dominated by theNMCMC​T24​B\\frac\{N\_\{\\mathrm\{MCMC\}\}\\,T^\{2\}\}\{4B\}generated tokens of power sampling, plusO​\(NMCMC⋅K\)O\(N\_\{\\mathrm\{MCMC\}\}\\cdot K\)entropy computations at the probed layers \(Table[6](https://arxiv.org/html/2607.09693#A5.T6)\); design choices and defaults are in Appendices[D](https://arxiv.org/html/2607.09693#A4)–[E](https://arxiv.org/html/2607.09693#A5)\.

## 5Experiments

#### Setup\.

We evaluate DEGS on mathematical reasoning \(MATH500\(Hendrycks et al\.,[2021](https://arxiv.org/html/2607.09693#bib.bib13); Lightman et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib21)\)and DeepMind500, the latter a held\-out subset we construct from the DeepMind mathematics dataset\(Saxton et al\.,[2019](https://arxiv.org/html/2607.09693#bib.bib29)\)\), code generation \(HumanEval\(Chen et al\.,[2021](https://arxiv.org/html/2607.09693#bib.bib4)\)\), and scientific reasoning \(GPQA\-Diamond\(Rein et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib28)\)\) with three open\-weight base models—Qwen2\.5\-7B\(Qwen,[2025](https://arxiv.org/html/2607.09693#bib.bib27)\), Qwen2\.5\-Math\-7B\(Yang et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib42)\), and DeepSeek\-Math\-7B\(Shao et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib32)\)\(full benchmark, grading, and training details in Appendix[E](https://arxiv.org/html/2607.09693#A5)\)\. The zero\-shot chain\-of\-thought prompts\(Wei et al\.,[2022](https://arxiv.org/html/2607.09693#bib.bib39); Kojima et al\.,[2022](https://arxiv.org/html/2607.09693#bib.bib18)\), answer parsers, and graders are held*identical*across all decoding methods, so accuracy differences reflect the decoder rather than the harness\. The*GRPO \(MATH\)*rows are not transcribed from prior work: we train one GRPO reference per base model under a standard RLVR recipe\(Lambert et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib19)\)\(DeepScaleR prompts\(Luo et al\.,[2025](https://arxiv.org/html/2607.09693#bib.bib22)\), a ground\-truth math verifier, group size1616\) and evaluate it under the same protocol, so GRPO functions as a fair, in\-domain reference trained under matched conditions\. Because DEGS adds an entropy\-probing pass, all methods are compared at matched test\-time compute, normalized to forward\-equivalent units \(Table[6](https://arxiv.org/html/2607.09693#A5.T6), Section[5\.3](https://arxiv.org/html/2607.09693#S5.SS3)\)\.

#### Baselines\.

We compare against: \(1\) base decoding, \(2\) low\-temperature sampling, \(3\) Best\-of\-NNwith log\-likelihood, \(4\) MCMC power sampling\(Karan and Du,[2025](https://arxiv.org/html/2607.09693#bib.bib17)\), \(5\) Scalable Power Sampling\(Ji et al\.,[2026](https://arxiv.org/html/2607.09693#bib.bib15)\), \(6\) Power\-SMC\(Azizi et al\.,[2026](https://arxiv.org/html/2607.09693#bib.bib1)\), and \(7\) GRPO\-posttrained models\(Shao et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib32)\)\. Our methods are*Best\-of\-NN\-Entropy*\(final\-layer\-entropy reranking\) and*DEGS\-MCMC*\.

### 5\.1Does Collapse Depth Alone Predict Correctness?

Qwen2\.5\-Math\-7B on MATH500![Refer to caption](https://arxiv.org/html/2607.09693v1/x6.png)\(a\)![Refer to caption](https://arxiv.org/html/2607.09693v1/x7.png)\(b\)![Refer to caption](https://arxiv.org/html/2607.09693v1/x8.png)\(c\)Qwen2\.5\-Math\-7B on DeepMind500![Refer to caption](https://arxiv.org/html/2607.09693v1/x9.png)\(d\)![Refer to caption](https://arxiv.org/html/2607.09693v1/x10.png)\(e\)![Refer to caption](https://arxiv.org/html/2607.09693v1/x11.png)\(f\)D​\(𝐱\)D\(\\mathbf\{x\}\)for correct vs\. incorrectAccuracy byD​\(𝐱\)D\(\\mathbf\{x\}\)bucketPredictive metrics forD​\(𝐱\)D\(\\mathbf\{x\}\)Figure 2:Collapse depthD​\(𝐱\)D\(\\mathbf\{x\}\)as a predictor of correctness, on two representative settings: Qwen2\.5\-Math\-7B on MATH500 \(a–c\) and DeepMind500 \(d–f\)\. Each row shows the distribution ofD​\(𝐱\)D\(\\mathbf\{x\}\)for correct vs\. incorrect candidates, the empirical accuracy of candidates byD​\(𝐱\)D\(\\mathbf\{x\}\)quantile bucket, and the predictive statistics \(AUC, Spearmanρ\\rho, mean gapΔ​D¯\\Delta\\bar\{D\}, pairwise ranking accuracy\)\. In both settings, later collapse is positively associated with correctness\.Before usingD​\(𝐱\)D\(\\mathbf\{x\}\)as a scoring signal, we verify it correlates with correctness: for each problem we sampleN=16N\{=\}16candidates \(T=0\.8T\{=\}0\.8, top\-p=0\.9p\{=\}0\.9\), judge correctness, and computeD​\(𝐱\)D\(\\mathbf\{x\}\)\. On Qwen2\.5\-Math\-7B / MATH500 \(Figure[2](https://arxiv.org/html/2607.09693#S5.F2)a–c\), correct candidates have significantly higher mean collapse depth than incorrect ones \(p<0\.001p<0\.001\), and accuracy climbs monotonically acrossD​\(𝐱\)D\(\\mathbf\{x\}\)buckets, from0\.170\.17in the lowest to0\.410\.41in the highest—a more than twofold spread that recurs, more sharply, on the harder DeepMind500 split \(d–f, top bucket≈2\.6×\\approx 2\.6\\timesthe bottom\)\. Taken on its own, however, collapse depth is a*weak per\-candidate*discriminator: near\-chance in isolation \(AUC≈0\.6\\approx 0\.6\), with a low Spearmanρ≈0\.15\\rho\\approx 0\.15and pairwise\-ranking accuracy barely above one\-half \(c,f\)\. The significance therefore reflects the size of then=8000n\{=\}8000candidate pool rather than a large effect\. This is precisely the regime DEGS targets: by Proposition[8](https://arxiv.org/html/2607.09693#Thmproposition8)a moderateβ\\betashifts each log\-score by at mostβ\\betanats, so the depth term acts only as a tiebreaker among near\-equal\-likelihood candidates—where being right slightly more often than chance suffices once compounded over the MCMC trajectory \(Section[5\.2](https://arxiv.org/html/2607.09693#S5.SS2)\)\.

### 5\.2Main Results

#### DEGS is the strongest training\-free decoder in1111of1212cells\.

DEGS\-MCMC attains the best training\-free accuracy on all four benchmarks for both Qwen backbones and on three of four for DeepSeek\-Math\-7B\. On Qwen2\.5\-Math\-7B it improves over MCMC power sampling\(Karan and Du,[2025](https://arxiv.org/html/2607.09693#bib.bib17)\)on every benchmark and beats the strongest prior training\-free method, Power\-SMC, on all four—though the margin narrows to0\.50\.5–22points on the math splits, where the depth term refines rather than replaces the likelihood signal\. Every power\-sampling method far outstrips naive decoding \(on Qwen2\.5\-7B / DeepMind500,0\.250→0\.5730\.250\\\!\\to\\\!0\.573\)\. The lone exception is DeepSeek\-Math\-7B / GPQA, where low\-temperature decoding is anomalously strong \(0\.4300\.430, beating every power\-sampling variant including DEGS\)—tellingly the setting where the late\-collapse signal is weakest \(Figure[1](https://arxiv.org/html/2607.09693#S4.F1)\(c–e\)\)\. This is a boundary of the method: when the depth signal is uninformative, the likelihood\-driven power distribution governs\.

#### DEGS narrows the gap to GRPO on math and surpasses it out of domain\.

GRPO is posttrained on MATH and stays strongest in\-domain, edging DEGS by1\.81\.8points on MATH500 and a comparable margin on DeepMind500; even so, with no training or reward signal, DEGS closes most of the3\.73\.7\- and5\.85\.8\-point gaps plain power sampling leaves on those splits\. Out of domain the picture reverses: DEGS exceeds GRPO on GPQA for all three models and on HumanEval for both Qwen models—reaching0\.7690\.769on Qwen2\.5\-7B HumanEval, well clear of GRPO—with GRPO keeping only a slight edge on DeepSeek\-Math\-7B HumanEval\. This supports our framing: GRPO redistributes capability within the distribution it was optimized on, whereas DEGS reweights base\-model samples and preserves cross\-domain coverage\. The per\-cell pattern tracks the late\-collapse signal: the two Qwen backbones, with the largest base\-versus\-posttrained collapse\-depth separation \(Figure[1](https://arxiv.org/html/2607.09693#S4.F1)\), show the clearest gains, while DeepSeek\-Math\-7B—where that separation is smallest—shows almost none\.

Table 1:Main benchmark results across models, tasks, and decoding methods, comparing standard decoding baselines, power\-sampling baselines, our entropy\-guided variants, and GRPO references\. For DeepSeek\-Math\-7B, the Base / Low\-temperature rows reflect the un\-tuned base checkpoint under our standardized decoding protocol; with no instruction tuning the model frequently fails to emit a parsable boxed answer\.
#### Takeaway and statistical reliability\.

The depth\-entropy term never lowers accuracy relative to the likelihood\-only power sampling it extends: it improves on that baseline in all twelve cells and is the best training\-free decoder in eleven, concentrating its gains exactly where likelihood is weakest—out\-of\-domain GPQA and the harder DeepMind500 split—rather than acting as a new capability\. The comparison is*within\-framework*: toggling onlyβ\\beta\(55vs\.0\) holds decoder, harness, and budget fixed, so the uniformly positive per\-cell deltas of Table[1](https://arxiv.org/html/2607.09693#S5.T1)\(median≈4\\approx 4points\) are not confounded by the implementation differences a head\-to\-head with an external baseline would introduce\. We claim only their*uniform direction*, not any single magnitude: with≈1\.9\\approx 1\.9points of binomial error on a500500\-item split we report no per\-cell variance, and although the twelve\-cell direction passes a one\-sided sign test \(p=2−12≈2\.4×10−4p=2^\{\-12\}\\approx 2\.4\\times 10^\{\-4\}\), we read this as corroborating theβ\\beta\-toggle rather than as a standalone test, since the cells share models and benchmarks and are not independent\.

### 5\.3Accuracy versus Compute Budget

#### Matched\-budget protocol\.

Because DEGS adds an entropy\-probing pass, we normalize all methods to forward\-equivalent units \(Table[6](https://arxiv.org/html/2607.09693#A5.T6)\): each reranking method is chargedNNgeneration passes plus probing, each MCMC\-family methodB⋅TMCMC=160B\\cdot T\_\{\\mathrm\{MCMC\}\}=160proposal forwards, and layer\-subset probingK/LK/Lof a forward per probed candidate\.

#### The depth signal is nearly free\.

The probe is teacher\-forced and touches onlyKKofLLlayers, so it costs a fraction of a full forward pass rather than a whole one\. On Qwen2\.5\-Math\-7B / MATH500 it adds just\+3\.3%\+3\.3\\%per\-problem wall\-clock to DEGS reranking and\+8\.8%\+8\.8\\%to DEGS\-MCMC over their likelihood\-only counterparts \(Table[6](https://arxiv.org/html/2607.09693#A5.T6)\)\. The accuracy gains of Table[1](https://arxiv.org/html/2607.09693#S5.T1)therefore come at single\-digit\-percent overhead—far below the near\-doubling that a naive count of one extra forward per candidate would imply\.

#### The signal is localizable in depth\.

Sweepingℒsub\\mathcal\{L\}\_\{\\mathrm\{sub\}\}\(full sweep in Table[7](https://arxiv.org/html/2607.09693#A5.T7), Appendix[E](https://arxiv.org/html/2607.09693#A5)\) separates two effects that “probe fewer layers” conflates—how*many*layers enter the grid versus*where*they sit\. Count is almost free: thinning the default seven\-layer grid down to five leaves MATH500 and GPQA accuracy unchanged, and probing all2828layers buys only\+0\.4\+0\.4point on MATH500 at1\.05×1\.05\\timescost\. Placement, by contrast, is decisive: holding the count at four and sliding the window toward the input lowers accuracy monotonically—76\.6→72\.876\.6\\\!\\to\\\!72\.8on MATH500, with the same monotone decline on GPQA—until an input\-only probe falls below even theβ=0\\beta\{=\}0baselines \(Table[2](https://arxiv.org/html/2607.09693#S5.T2)\)\. Because this ordering recurs across both out\-of\-domain GPQA and in\-domain MATH500, it reflects*where*entropy collapse carries discriminative content rather than a benchmark quirk, and the probe can safely be restricted to those layers at no accuracy penalty\.

### 5\.4Ablation Studies

Table 2:Core ablation of DEGS components on Qwen2\.5\-Math\-7B, isolating likelihood\-based scoring, depth\-based entropy scoring, and alternative entropy signals under a matched\-compute protocol \(N=16N=16, MCMC steps=10=10, block sizeB=16B=16\)\.#### Both terms are necessary, and the signal lives in the depth profile, not the final layer\.

DEGS\-MCMC improves over likelihood\-only power sampling \(β=0\\beta\{=\}0\) on all three benchmarks—up to\+4\.5\+4\.5on DeepMind500 \(Table[2](https://arxiv.org/html/2607.09693#S5.T2)\)—and over depth\-only scoring by a wide margin; neither term alone reproduces the combined result\. The gain is not merely an entropy effect: final\-layer\-entropy reranking lands barely above Best\-of\-NNon MATH500 yet trails DEGS\-MCMC by up to1414points on DeepMind500, so*when*a sequence’s predictions stabilize is more informative than its final\-layer entropy alone \(Appendix[D](https://arxiv.org/html/2607.09693#A4)\)—the distinction that motivates collapse depth\. Added to either selection mode, the depth term yields a positive marginal gain over its matched likelihood\-only baseline on all three benchmarks, largest in reranking on the harder DeepMind500 split, as Proposition[8](https://arxiv.org/html/2607.09693#Thmproposition8)predicts; but applying the tiebreak once \(reranking\) versus at each of∼160\\sim\\\!160accept/reject steps \(DEGS\-MCMC\) is what separates them—DEGS\-MCMC dominates reranking on all three benchmarks, and reranking serves only as a controlled probe of*where*the signal acts \(Table[7](https://arxiv.org/html/2607.09693#A5.T7)\)\.

## 6Discussion and Conclusion

We introduced DEGS, a training\-free method that turns layer\-wise entropy collapse depth into a test\-time quality signal: stronger reasoners collapse later, which we operationalize as a pseudo\-rewardD​\(𝐱\)D\(\\mathbf\{x\}\)tilting the power objective\. DEGS does not merely favor uncertain sequences—the ablation \(Table[2](https://arxiv.org/html/2607.09693#S5.T2)\) confirms both terms are necessary\.

#### Limitations and future work\.

Our evaluation centers on mathematical reasoning, with preliminary coding and scientific\-QA results; generalization to open\-ended or factual tasks remains open\. DEGS also requires hidden\-state access \(open\-weight models only\), and the logit lens is an imperfect probe\(Wendler et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib41)\)\. Promising extensions include richer entropy\-curve features via learned scorers, token\-selective collapse depth, cross\-architecture validation, and integration with efficient samplers such as Power\-SMC\(Azizi et al\.,[2026](https://arxiv.org/html/2607.09693#bib.bib1)\)\.

## Reproducibility Statement

All experiments use publicly available open\-weight base models \(Qwen2\.5\-7B, Qwen2\.5\-Math\-7B, DeepSeek\-Math\-7B\) and public benchmarks \(MATH500, DeepMind500, HumanEval, GPQA\-Diamond\)\. The collapse\-depth definition \(Definitions[1](https://arxiv.org/html/2607.09693#Thmdefinition1)–[2](https://arxiv.org/html/2607.09693#Thmdefinition2)\), the DEGS objective equation[4](https://arxiv.org/html/2607.09693#S4.E4), and both decoding variants are specified in full in Section[4](https://arxiv.org/html/2607.09693#S4)and Algorithms[1](https://arxiv.org/html/2607.09693#alg1)–[2](https://arxiv.org/html/2607.09693#alg2)\. Appendix[E](https://arxiv.org/html/2607.09693#A5)reports the complete harness—per\-backbone layer counts and probe grids, the prompt/parser/grader for each benchmark, decoding knobs, the entropy\-probing procedure, the GRPO training recipe, and the matched\-budget protocol—together with default hyperparameters and search ranges \(Table[8](https://arxiv.org/html/2607.09693#A5.T8)\) and the per\-step MCMC protocol \(Table[4](https://arxiv.org/html/2607.09693#A1.T4)\); every prompt, parser, and grader is shared verbatim across all decoding methods\. Formal statements and complete proofs of all theoretical claims are in Appendix[F](https://arxiv.org/html/2607.09693#A6)\.

## Ethics Statement

This work studies training\-free decoding for reasoning on standard public benchmarks using open\-weight models, and involves no new data collection or human subjects\. It inherits the general risks of LLM reasoning systems \(e\.g\., confidently stated incorrect answers\) but, to our knowledge, introduces no additional ethical concerns; because DEGS requires hidden\-state access, it applies only to open\-weight models\.

## References

- Azizi et al\. \(2026\)Seyedarmin Azizi, et al\.Power\-SMC: Low\-latency sequence\-level power sampling for training\-free LLM reasoning\.*arXiv preprint arXiv:2602\.10273*, 2026\.
- Belrose et al\. \(2023\)Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt\.Eliciting latent predictions from transformers with the tuned lens\.*arXiv preprint arXiv:2303\.08112*, 2023\.
- Brown et al\. \(2024\)Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V\. Le, Christopher Ré, and Azalia Mirhoseini\.Large language monkeys: Scaling inference compute with repeated sampling\.*arXiv preprint arXiv:2407\.21787*, 2024\.
- Chen et al\. \(2021\)Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al\.Evaluating large language models trained on code\.*arXiv preprint arXiv:2107\.03374*, 2021\.
- Chuang et al\. \(2024\)Yung\-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R\. Glass, and Pengcheng He\.DoLa: Decoding by contrasting layers improves factuality in large language models\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.
- Cobbe et al\. \(2021\)Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman\.Training verifiers to solve math word problems\.*arXiv preprint arXiv:2110\.14168*, 2021\.
- Cui et al\. \(2025\)Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, Zhiyuan Liu, Hao Peng, Lei Bai, Wanli Ouyang, Yu Cheng, Bowen Zhou, and Ning Ding\.The entropy mechanism of reinforcement learning for reasoning language models\.*arXiv preprint arXiv:2505\.22617*, 2025\.
- Faria et al\. \(2024\)Gonçalo R\. A\. Faria, Sweta Agrawal, Antonio Farinhas, Ricardo Rei, José G\. C\. de Souza, and André F\. T\. Martins\.QuEST: Quality\-aware Metropolis–Hastings sampling for machine translation\.In*NeurIPS*, 2024\.
- Farquhar et al\. \(2024\)Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal\.Detecting hallucinations in large language models using semantic entropy\.*Nature*, 630\(8017\):625–630, 2024\.
- Ghandeharioun et al\. \(2024\)Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva\.Patchscopes: A unifying framework for inspecting hidden representations of language models\.In*International Conference on Machine Learning \(ICML\)*, 2024\.
- Guo et al\. \(2025\)Daya Guo, Dejian Yang, Haowei Zhang, et al\.DeepSeek\-R1: Incentivizing reasoning capability in LLMs via reinforcement learning\.*arXiv preprint arXiv:2501\.12948*, 2025\.
- He et al\. \(2025\)Andre He, Daniel Fried, and Sean Welleck\.Rewarding the unlikely: Lifting GRPO beyond distribution sharpening\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, pp\. 25559–25571, 2025\.
- Hendrycks et al\. \(2021\)Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt\.Measuring mathematical problem solving with the MATH dataset\.In*Advances in Neural Information Processing Systems \(NeurIPS\) Datasets and Benchmarks Track*, 2021\.
- Holtzman et al\. \(2020\)Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi\.The curious case of neural text degeneration\.In*International Conference on Learning Representations \(ICLR\)*, 2020\.
- Ji et al\. \(2026\)Xiaotong Ji, Rasul Tutunov, Matthieu Zimmer, and Haitham Bou Ammar\.Scalable power sampling: Unlocking efficient, training\-free reasoning for LLMs via distribution sharpening\.*arXiv preprint arXiv:2601\.21590*, 2026\.
- Kadavath et al\. \(2022\)Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield\-Dodds, Nova DasSarma, Eli Tran\-Johnson, Scott Johnston, Sheer El\-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan\.Language models \(mostly\) know what they know\.*arXiv preprint arXiv:2207\.05221*, 2022\.
- Karan and Du \(2025\)Aayush Karan and Yilun Du\.Reasoning with sampling: Your base model is smarter than you think\.*arXiv preprint arXiv:2510\.14901*, 2025\.
- Kojima et al\. \(2022\)Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa\.Large language models are zero\-shot reasoners\.In*NeurIPS*, 2022\.
- Lambert et al\. \(2024\)Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V\. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al\.Tulu 3: Pushing frontiers in open language model post\-training\.*arXiv preprint arXiv:2411\.15124*, 2024\.
- Lew et al\. \(2023\)Alexander K\. Lew, Tan Zhi\-Xuan, Gabriel Grand, and Vikash K\. Mansinghka\.Sequential Monte Carlo steering of large language models using probabilistic programs\.*arXiv preprint arXiv:2306\.03081*, 2023\.
- Lightman et al\. \(2024\)Hunter Lightman, Vineet Kosaraju, Yura Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe\.Let’s verify step by step\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.
- Luo et al\. \(2025\)Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y\. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica\.DeepScaleR: Surpassing O1\-Preview with a 1\.5B model by scaling RL, 2025\.Notion Blog\.[https://github\.com/agentica\-project/rllm](https://github.com/agentica-project/rllm)\.
- Neal \(1993\)Radford M\. Neal\.Probabilistic inference using Markov chain Monte Carlo methods\.Technical report, University of Toronto, 1993\.
- Nostalgebraist \(2020\)Nostalgebraist\.Interpreting GPT: The logit lens\.*LessWrong*, 2020\.
- Ouyang et al\. \(2022\)Long Ouyang, Jeffrey Wu, Xu Jiang, et al\.Training language models to follow instructions with human feedback\.In*NeurIPS*, volume 35, pages 27730–27744, 2022\.
- Prabhudesai et al\. \(2025\)Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak\.Maximizing confidence alone improves reasoning\.*arXiv preprint arXiv:2505\.22660*, 2025\.
- Qwen \(2025\)Qwen: An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, et al\.Qwen2\.5 technical report\.*arXiv preprint arXiv:2412\.15115*, 2025\.
- Rein et al\. \(2024\)David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R\. Bowman\.GPQA: A graduate\-level google\-proof Q&A benchmark\.In*Conference on Language Modeling \(COLM\)*, 2024\.
- Saxton et al\. \(2019\)David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli\.Analysing mathematical reasoning abilities of neural models\.In*International Conference on Learning Representations \(ICLR\)*, 2019\.
- Schulman et al\. \(2017\)John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov\.Proximal policy optimization algorithms\.*arXiv preprint arXiv:1707\.06347*, 2017\.
- Schuster et al\. \(2022\)Tal Schuster, Adam Fisch, Jai Gupta, Mostafa Dehghani, Dara Bahri, Vinh Q\. Tran, Yi Tay, and Donald Metzler\.Confident adaptive language modeling\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2022\.
- Shao et al\. \(2024\)Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y\. K\. Li, Y\. Wu, and Daya Guo\.DeepSeekMath: Pushing the limits of mathematical reasoning in open language models\.*arXiv preprint arXiv:2402\.03300*, 2024\.
- Shao et al\. \(2025\)Rulin Shao, Shuyue Stella Li, Rui Xin, et al\.Spurious rewards: Rethinking training signals in RLVR\.*arXiv preprint arXiv:2506\.10947*, 2025\.
- Snell et al\. \(2024\)Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar\.Scaling LLM test\-time compute optimally can be more effective than scaling model parameters\.*arXiv preprint arXiv:2408\.03314*, 2024\.
- Song et al\. \(2025\)Yuda Song, Julia Kempe, and Remi Munos\.Outcome\-based exploration for LLM reasoning\.*arXiv preprint arXiv:2509\.06941*, 2025\.
- Tan et al\. \(2025\)Yuqiao Tan, Minzheng Wang, Shizhu He, Huanxuan Liao, Chengfeng Zhao, Qiunan Lu, Tian Liang, Jun Zhao, and Kang Liu\.Bottom\-up policy optimization: Your language model policy secretly contains internal policies\.*arXiv preprint arXiv:2512\.19673*, 2025\.
- Uesato et al\. \(2022\)Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins\.Solving math word problems with process\- and outcome\-based feedback\.*arXiv preprint arXiv:2211\.14275*, 2022\.
- Wang et al\. \(2023\)Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou\.Self\-consistency improves chain of thought reasoning in language models\.In*ICLR*, 2023\.
- Wei et al\. \(2022\)Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H\. Chi, Quoc V\. Le, and Denny Zhou\.Chain\-of\-thought prompting elicits reasoning in large language models\.In*NeurIPS*, 2022\.
- Welleck et al\. \(2024\)Sean Welleck, Amanda Bertsch, Matthew Finlayson, Hailey Schoelkopf, Alex Xie, Graham Neubig, Ilia Kulikov, and Zaid Harchaoui\.From decoding to meta\-generation: Inference\-time algorithms for large language models\.*Transactions on Machine Learning Research \(TMLR\)*, 2024\.
- Wendler et al\. \(2024\)Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West\.Do Llamas work in English? On the latent language of multilingual transformers\.In*Proceedings of ACL*, pages 15366–15394, 2024\.
- Yang et al\. \(2024\)An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang\.Qwen2\.5\-Math technical report: Toward mathematical expert model via self\-improvement\.*arXiv preprint arXiv:2409\.12122*, 2024\.
- Yue et al\. \(2025\)Yang Yue, Zhiqi Chen, Rui Lu, et al\.Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2025\.
- Zhao et al\. \(2025\)Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song\.Learning to reason without external rewards\.*arXiv preprint arXiv:2505\.19590*, 2025\.
- Zhao et al\. \(2024\)Stephen Zhao, Rob Brekelmans, Alireza Makhzani, and Roger Grosse\.Probabilistic inference in language models via twisted sequential Monte Carlo\.*arXiv preprint arXiv:2404\.17546*, 2024\.

## Appendix AAdditional Figures and Tables

![Refer to caption](https://arxiv.org/html/2607.09693v1/x12.png)\(a\)Qwen2\.5\-Math\-7BMATH500![Refer to caption](https://arxiv.org/html/2607.09693v1/x13.png)\(b\)Qwen2\.5\-Math\-7BDeepMind500![Refer to caption](https://arxiv.org/html/2607.09693v1/x14.png)\(c\)Qwen2\.5\-Math\-7BGSM8K![Refer to caption](https://arxiv.org/html/2607.09693v1/x15.png)\(d\)Qwen2\.5\-Math\-7BGPQA![Refer to caption](https://arxiv.org/html/2607.09693v1/x16.png)\(e\)DeepSeek\-Math\-7BMATH500![Refer to caption](https://arxiv.org/html/2607.09693v1/x17.png)\(f\)DeepSeek\-Math\-7BDeepMind500![Refer to caption](https://arxiv.org/html/2607.09693v1/x18.png)\(g\)DeepSeek\-Math\-7BGSM8K![Refer to caption](https://arxiv.org/html/2607.09693v1/x19.png)\(h\)DeepSeek\-Math\-7BGPQA![Refer to caption](https://arxiv.org/html/2607.09693v1/x20.png)\(i\)Qwen2\.5\-Math\-72BMATH500![Refer to caption](https://arxiv.org/html/2607.09693v1/x21.png)\(j\)Qwen2\.5\-Math\-72BDeepMind500![Refer to caption](https://arxiv.org/html/2607.09693v1/x22.png)\(k\)Qwen2\.5\-7BMATH500![Refer to caption](https://arxiv.org/html/2607.09693v1/x23.png)\(l\)Qwen2\.5\-7BDeepMind500Figure 3:Complete entropy\-versus\-layer visualizations across all twelve \(model, benchmark\) pairs\. Each panel overlays the layer\-wise token entropyHlH\_\{l\}in nats for the posttrained variant against its base counterpart, measured on the subset where the posttrained model is correct and the base model is wrong; this isolates the depths at which posttraining adds discriminative content\. This appendix figure complements Figure[1](https://arxiv.org/html/2607.09693#S4.F1)by expanding coverage and reducing the possibility that the main\-paper examples are selective\.Table 3:Full benchmark results with all reported settings\. Same data as Table[1](https://arxiv.org/html/2607.09693#S5.T1), with the decoding budget reported asN=candidatesN\{=\}\\text\{candidates\},M=MCMC stepsM\{=\}\\text\{MCMC steps\},T=temperatureT\{=\}\\text\{temperature\}\. Power\-SMC is included as an additional training\-free decoding baseline\(Azizi et al\.,[2026](https://arxiv.org/html/2607.09693#bib.bib1)\)\.Table 4:DEGS\-MCMC protocol details: the practical setup of the entropy\-guided MCMC variant, including proposal construction, chain length, acceptance computation, and stopping rules\.
## Appendix BPreliminaries \(Full\)

#### Autoregressive language models\.

Let𝒳\\mathcal\{X\}denote a finite vocabulary\. An autoregressive LLM definesp​\(𝐱\)=∏t=0Tp​\(xt∣x<t\)p\(\\mathbf\{x\}\)=\\prod\_\{t=0\}^\{T\}p\(x\_\{t\}\\mid x\_\{<t\}\)\. In a transformer withLLlayers, hidden states𝐡l,t∈ℝd\\mathbf\{h\}\_\{l,t\}\\in\\mathbb\{R\}^\{d\}are updated via𝐡l,t=𝐡l−1,t\+fl​\(𝐡l−1,1,…,𝐡l−1,t\)\\mathbf\{h\}\_\{l,t\}=\\mathbf\{h\}\_\{l\-1,t\}\+f\_\{l\}\(\\mathbf\{h\}\_\{l\-1,1\},\\ldots,\\mathbf\{h\}\_\{l\-1,t\}\), whereflf\_\{l\}is thell\-th transformer block\. After layerLL, the unembedding matrixU∈ℝ\|𝒳\|×dU\\in\\mathbb\{R\}^\{\|\\mathcal\{X\}\|\\times d\}produces next\-token logits\.

#### Logit lens\.

The logit lens\(Nostalgebraist,[2020](https://arxiv.org/html/2607.09693#bib.bib24)\)appliesUUat intermediate layers:pl​\(v∣x≤t\)=softmax​\(U​𝐡l,t\)vp\_\{l\}\(v\\mid x\_\{\\leq t\}\)=\\mathrm\{softmax\}\(U\\mathbf\{h\}\_\{l,t\}\)\_\{v\}\. The layer\-wise entropy isHl​\(t\)=−∑vpl​\(v∣x≤t\)​log⁡pl​\(v∣x≤t\)H\_\{l\}\(t\)=\-\\sum\_\{v\}p\_\{l\}\(v\\mid x\_\{\\leq t\}\)\\log p\_\{l\}\(v\\mid x\_\{\\leq t\}\)\. Prior work shows entropy is high in early layers and drops sharply in later layers\(Wendler et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib41)\)\.

#### Power distributions\.

The power distributionpα​\(𝐱\)∝p​\(𝐱\)αp^\{\\alpha\}\(\\mathbf\{x\}\)\\propto p\(\\mathbf\{x\}\)^\{\\alpha\}\(α≥1\\alpha\\geq 1\) sharpens the base distribution\.Karan and Du \([2025](https://arxiv.org/html/2607.09693#bib.bib17)\)show thatpαp^\{\\alpha\}differs from low\-temperature sampling and propose MH sampling to target it\.

## Appendix CDEGS Reranking Algorithm

Algorithm[2](https://arxiv.org/html/2607.09693#alg2)gives the depth\-based reranking procedure referenced in Section[4\.3](https://arxiv.org/html/2607.09693#S4.SS3)\. Given a problem prompt, sampleNNcandidate solutions\{𝐱\(i\)\}i=1N\\\{\\mathbf\{x\}^\{\(i\)\}\\\}\_\{i=1\}^\{N\}from the base modelppusing moderate randomness \(temperature∈\[0\.7,1\.0\]\\in\[0\.7,1\.0\], top\-p=0\.95p=0\.95\), recording the per\-token log\-probabilities\. For each candidate, compute the log\-likelihood from the recorded log\-probs and the collapse depthD​\(𝐱\(i\)\)D\(\\mathbf\{x\}^\{\(i\)\}\)from one additional teacher\-forced forward pass \(logit\-lens entropy at layersℒsub\\mathcal\{L\}\_\{\\mathrm\{sub\}\}\), then the combined scoreS\(i\)=α​log⁡p​\(𝐱\(i\)\)\+β​D​\(𝐱\(i\)\)S^\{\(i\)\}=\\alpha\\log p\(\\mathbf\{x\}^\{\(i\)\}\)\+\\beta D\(\\mathbf\{x\}^\{\(i\)\}\), and return𝐱∗=arg​maxi⁡S\(i\)\\mathbf\{x\}^\{\*\}=\\operatorname\*\{arg\\,max\}\_\{i\}S^\{\(i\)\}\. As discussed in Section[4\.3](https://arxiv.org/html/2607.09693#S4.SS3), atN=16N\{=\}16this is not a competitive standalone decoder \(Proposition[8](https://arxiv.org/html/2607.09693#Thmproposition8)\); we retain it as a controlled probe of*where*in depth the collapse signal carries discriminative content \(Table[7](https://arxiv.org/html/2607.09693#A5.T7)\)\.

Algorithm 2DEGS Reranking \(Best\-of\-NN\)0:Base model

pp; prompt; candidate budget

NN; layer subset

ℒsub\\mathcal\{L\}\_\{\\mathrm\{sub\}\}; threshold

τ\\tau; weights

α,β\\alpha,\\beta
0:Selected solution

𝐱∗\\mathbf\{x\}^\{\*\}
1:Sample

NNcandidates

\{𝐱\(i\)\}i=1N∼p\(⋅∣prompt\)\\\{\\mathbf\{x\}^\{\(i\)\}\\\}\_\{i=1\}^\{N\}\\sim p\(\\cdot\\mid\\text\{prompt\}\); record log\-probs

2:for

i=1,…,Ni=1,\\ldots,Ndo

3:

log⁡p​\(𝐱\(i\)\)←∑tlog⁡p​\(xt\(i\)∣x<t\(i\)\)\\log p\(\\mathbf\{x\}^\{\(i\)\}\)\\leftarrow\\sum\_\{t\}\\log p\(x\_\{t\}^\{\(i\)\}\\mid x\_\{<t\}^\{\(i\)\}\)\{from sampling\}

4:Run forward pass on prompt

⊕𝐱\(i\)\\oplus\\;\\mathbf\{x\}^\{\(i\)\}with hidden states at layers

ℒsub\\mathcal\{L\}\_\{\\mathrm\{sub\}\}
5:foreach layer

l∈ℒsubl\\in\\mathcal\{L\}\_\{\\mathrm\{sub\}\}and position

ttdo

6:

Hl​\(t\)←−∑vpl​\(v∣x≤t\(i\)\)​log⁡pl​\(v∣x≤t\(i\)\)H\_\{l\}\(t\)\\leftarrow\-\\sum\_\{v\}p\_\{l\}\(v\\mid x\_\{\\leq t\}^\{\(i\)\}\)\\log p\_\{l\}\(v\\mid x\_\{\\leq t\}^\{\(i\)\}\)\{logit\-lens entropy\}

7:endfor

8:foreach position

ttdo

9:

dt←min⁡\{l∈ℒsub:Hl​\(t\)≤τ\}d\_\{t\}\\leftarrow\\min\\\{l\\in\\mathcal\{L\}\_\{\\mathrm\{sub\}\}:H\_\{l\}\(t\)\\leq\\tau\\\}\(default

LLif none\)

10:endfor

11:

D​\(𝐱\(i\)\)←1T\(i\)\+1​∑tdt/LD\(\\mathbf\{x\}^\{\(i\)\}\)\\leftarrow\\frac\{1\}\{T^\{\(i\)\}\+1\}\\sum\_\{t\}d\_\{t\}/L
12:

S\(i\)←α⋅log⁡p​\(𝐱\(i\)\)\+β⋅D​\(𝐱\(i\)\)S^\{\(i\)\}\\leftarrow\\alpha\\cdot\\log p\(\\mathbf\{x\}^\{\(i\)\}\)\+\\beta\\cdot D\(\\mathbf\{x\}^\{\(i\)\}\)
13:endfor

14:return

𝐱∗←arg​maxi⁡S\(i\)\\mathbf\{x\}^\{\*\}\\leftarrow\\operatorname\*\{arg\\,max\}\_\{i\}S^\{\(i\)\}

## Appendix DDesign Choices and Practical Considerations

#### Relationship to existing signals\.

DEGS differs from final\-layer entropy reranking in that it captures the*depth profile*of entropy evolution, not merely its endpoint\. A candidate may have low final\-layer entropy yet collapse early, reflecting superficial rather than deep reasoning\. Table[2](https://arxiv.org/html/2607.09693#S5.T2)isolates the contribution of each component\.

## Appendix EImplementation Details

This appendix records the decoding, probing, training, and evaluation configuration behind Section[5](https://arxiv.org/html/2607.09693#S5)\. The tables in this section are referenced from the main text; the prose below specifies what the tables do not, so that every reported number is reproducible from a single fixed harness\.

#### Base models\.

All experiments use three open\-weight base checkpoints: Qwen2\.5\-7B and Qwen2\.5\-Math\-7B, each withL=28L\{=\}28transformer layers, and DeepSeek\-Math\-7B withL=30L\{=\}30layers\. Because collapse depth is read off a fixed grid of layer indices, the default probeℒsub\\mathcal\{L\}\_\{\\mathrm\{sub\}\}is rescaled to each backbone’s depth—\{4,8,12,16,20,24,28\}\\\{4,8,12,16,20,24,28\\\}for the two2828\-layer Qwen models and\{4,9,14,18,22,26,30\}\\\{4,9,14,18,22,26,30\\\}for the3030\-layer DeepSeek model—so that the seven probed layers occupy comparable*relative*depthsl/Ll/Lacross architectures rather than a fixed absolute index\.

#### Benchmarks, prompts, and grading\.

Table[5](https://arxiv.org/html/2607.09693#A5.T5)gives the output format and answer\-processing pipeline for each benchmark; the governing principle is that the harness is held fixed while only the decoder varies\. MATH500 and DeepMind500 share an identical zero\-shot chain\-of\-thought prompt, parser, and grader, with the final answer read from the last\\boxed\{\}span and scored by a sympy\-based symbolic/numeric matcher; the two splits therefore differ only in problem difficulty, which is what licenses reading the larger DEGS gains on DeepMind500 \(Section[5\.2](https://arxiv.org/html/2607.09693#S5.SS2)\) as a difficulty effect rather than a harness artifact\. GPQA \(Diamond\) is posed as a four\-way multiple\-choice task whose option order is shuffled once and held fixed across methods—so every decoder is scored against the same gold position—with the predicted letter read from the boxed answer, matched exactly against the gold letter, and completions capped at30723072tokens\. HumanEval is generated by direct completion from the function signature, without chain\-of\-thought, and graded for functional correctness \(Pass@1\) by the official test harness\. Every prompt template, parser, and grader is shared verbatim across all decoding methods and all models, so that accuracy differences between rows of Table[1](https://arxiv.org/html/2607.09693#S5.T1)isolate the decoder\.

#### Decoding configuration\.

Each method samples from the same base model under a shared set of decoding knobs\. We disable nucleus\(Holtzman et al\.,[2020](https://arxiv.org/html/2607.09693#bib.bib14)\)and top\-kktruncation throughout \(top\-p=1\.0p\{=\}1\.0\): the power\-sampling family targets the untruncated base distribution, and matching this across baselines keeps the comparison on equal footing\. The likelihood exponent is realized as an inverse sampling temperature \(α=1/T\\alpha\{=\}1/T\), so the defaultα=4\\alpha\{=\}4corresponds toT=0\.25T\{=\}0\.25on MATH500, DeepMind500, and GPQA, whereas HumanEval is decoded atT=0\.5T\{=\}0\.5\(α=2\\alpha\{=\}2\) to retain the diversity program synthesis requires\. The reranking methods \(Best\-of\-NN, Best\-of\-NN\-Entropy, and DEGS reranking\) drawN=16N\{=\}16candidates and return one; the MCMC family \(power sampling and DEGS\-MCMC\) runsTMCMC=10T\_\{\\mathrm\{MCMC\}\}\{=\}10Metropolis–Hastings steps in each ofB=16B\{=\}16blocks of192192tokens, i\.e\.160160proposals per sample\. All DEGS variants use depth weightβ=5\\beta\{=\}5and their likelihood\-only counterpartsβ=0\\beta\{=\}0, so that any DEGS\-versus\-baseline difference is attributable solely to the depth\-entropy term\. Defaults and search ranges are collected in Table[8](https://arxiv.org/html/2607.09693#A5.T8)\.

#### Collapse\-depth probing\.

Scoring a completion withD​\(𝐱\)D\(\\mathbf\{x\}\)adds one teacher\-forced forward pass that caches hidden states at the layers inℒsub\\mathcal\{L\}\_\{\\mathrm\{sub\}\}\. At each probed layer the model’s final RMSNorm is applied before the logit lens—matching the normalization used at the output head—and exact token entropy is computed over the full vocabulary in chunks of81928192entries to bound peak memory\. The per\-token collapse depthdtd\_\{t\}is the shallowest probed layer whose logit\-lens entropy is at or belowτ=0\.25\\tau\{=\}0\.25nats \(dt=Ld\_\{t\}\{=\}Lif none qualifies\), andD​\(𝐱\)D\(\\mathbf\{x\}\)averagesdt/Ld\_\{t\}/Lover all generated tokens, excluding EOS and scoring the full completion span\. In DEGS\-MCMC the current state’sD​\(𝐱\)D\(\\mathbf\{x\}\)is cached and refreshed only when a block is appended or a proposal is accepted, whereas every proposal recomputesD​\(𝐱′\)D\(\\mathbf\{x\}^\{\\prime\}\)from scratch; the complete per\-step protocol, including the acceptance ratio and stopping rule, is given in Table[4](https://arxiv.org/html/2607.09693#A1.T4)\.

#### Predictive\-validity sampling\.

The analysis of whetherD​\(𝐱\)D\(\\mathbf\{x\}\)predicts correctness \(Figure[2](https://arxiv.org/html/2607.09693#S5.F2)\) deliberately uses a higher\-entropy regime than the decoding runs—N=16N\{=\}16candidates per problem atT=0\.8T\{=\}0\.8, top\-p=0\.9p\{=\}0\.9—so that the candidate pool spans a wide range ofD​\(𝐱\)D\(\\mathbf\{x\}\)and the correlation is not estimated on a near\-degenerate set of near\-identical samples\.

#### GRPO reference\.

FollowingKaran and Du \([2025](https://arxiv.org/html/2607.09693#bib.bib17)\), our GRPO references are produced rather than quoted: we posttrain each base model with GRPO\(Shao et al\.,[2024](https://arxiv.org/html/2607.09693#bib.bib32)\)and evaluate the resulting policy through the same harness as every other method\. The training setup mirrors theirs—we adopt the GRPO implementation ofShao et al\. \([2025](https://arxiv.org/html/2607.09693#bib.bib33)\)with its default hyperparameters, optimize against a ground\-truth math verifier on the MATH training split, and use a group size of1616rollouts per prompt\. For the two Qwen backbones this configuration coincides withKaran and Du \([2025](https://arxiv.org/html/2607.09693#bib.bib17)\); DeepSeek\-Math\-7B is outside their study, so its GRPO reference is trained by us under the same recipe\. Decoding is greedy throughout, so that GRPO functions as an in\-domain reference obtained under matched conditions\.

#### Sharding and compute\.

Generation is distributed over a heterogeneous GPU cluster of RTX 5880 and RTX 3090 nodes, one worker per GPU; the shards—five for MATH500 and DeepMind500, six for GPQA, and four for HumanEval—are a throughput split over the problem set only\. Hardware is split by experiment: the main accuracy results are measured on the RTX 5880 nodes, whereas the compute\-efficiency sweep behind Table[7](https://arxiv.org/html/2607.09693#A5.T7)and Table[6](https://arxiv.org/html/2607.09693#A5.T6)runs on a single RTX 3090—a difference that moves wall\-clock alone and not accuracy, since the decoder and harness are identical across nodes\. Test\-time compute is normalized to forward\-equivalent units as defined in Table[6](https://arxiv.org/html/2607.09693#A5.T6), under which each reranking and MCMC method is matched against its same\-family baseline; the wall\-clock figures in that table are per\-problem means on Qwen2\.5\-Math\-7B / MATH500, and the corresponding per\-problem mean on GPQA under the MCMC setting is approximately0\.630\.63ks\.

Table 5:Benchmarks, metrics, and evaluation setup, with output format and answer\-processing details for each benchmark\.Table 6:Compute accounting and matched\-budget protocol: how test\-time compute is normalized across methods\. All MCMC\-family methods useB=16B\{=\}16blocks withTMCMC=10T\_\{\\mathrm\{MCMC\}\}\{=\}10MH steps per block \(160160total proposals\), and all layer\-subset probing is charged asK/LK/Lof a full forward pass per probed candidate \(defaultK=7K\{=\}7,L∈\{28,30\}L\{\\in\}\\\{28,30\\\}\)\. The wall\-clock column reports per\-problem mean elapsed seconds on Qwen2\.5\-Math\-7B / MATH500 measured during the compute\-efficiency sweep reported in Table[1](https://arxiv.org/html/2607.09693#S5.T1)\.Table 7:Accuracy–overhead trade\-off for the layer subsetℒsub\\mathcal\{L\}\_\{\\mathrm\{sub\}\}: representative layer\-subset choices for entropy probing with their measured wall\-clock cost \(per\-problem mean, averaged over both \(model, benchmark\) evaluation settings used in Table[6](https://arxiv.org/html/2607.09693#A5.T6)\) and downstream accuracy\. The DeepMind500 column is reported on DeepSeek\-Math\-7B; see Table[6](https://arxiv.org/html/2607.09693#A5.T6)for the matched\-budget protocol\.Table 8:Full DEGS hyperparameters and search ranges\. The likelihood exponentα\\alphais implemented as1/temperature1/\\mathrm\{temperature\}inside the power\-sampling objective\.

## Appendix FProofs and Derivations

This appendix collects formal statements and proofs for the claims used in the main text\. Throughout,ppis the base model,D​\(𝐱\)∈\(0,1\]D\(\\mathbf\{x\}\)\\in\(0,1\]is the normalized collapse depth \(Definition[2](https://arxiv.org/html/2607.09693#Thmdefinition2)\), andπ​\(𝐱\)∝p​\(𝐱\)α​exp⁡\(β​D​\(𝐱\)\)\\pi\(\\mathbf\{x\}\)\\propto p\(\\mathbf\{x\}\)^\{\\alpha\}\\exp\(\\beta D\(\\mathbf\{x\}\)\)is the DEGS target \(Definition[3](https://arxiv.org/html/2607.09693#Thmdefinition3)\) withα≥1\\alpha\\geq 1,β≥0\\beta\\geq 0\.

### F\.1Normalizability of the DEGS target

###### Proposition 1\(Normalizability\)\.

Let the support𝒮=\{𝐱:p​\(𝐱\)\>0\}\\mathcal\{S\}=\\\{\\mathbf\{x\}:p\(\\mathbf\{x\}\)\>0\\\}be finite \(e\.g\. sequences of bounded length over a finite vocabulary\)\. ThenZ=∑𝐱∈𝒮p​\(𝐱\)α​exp⁡\(β​D​\(𝐱\)\)Z=\\sum\_\{\\mathbf\{x\}\\in\\mathcal\{S\}\}p\(\\mathbf\{x\}\)^\{\\alpha\}\\exp\(\\beta D\(\\mathbf\{x\}\)\)satisfies0<Z<∞0<Z<\\infty, soπ​\(𝐱\)=p​\(𝐱\)α​exp⁡\(β​D​\(𝐱\)\)/Z\\pi\(\\mathbf\{x\}\)=p\(\\mathbf\{x\}\)^\{\\alpha\}\\exp\(\\beta D\(\\mathbf\{x\}\)\)/Zis a well\-defined probability distribution\.

###### Proof\.

Each term is nonnegative, and positive for at least one𝐱\\mathbf\{x\}\(any𝐱∈𝒮\\mathbf\{x\}\\in\\mathcal\{S\}\), soZ\>0Z\>0\. SinceD​\(𝐱\)≤1D\(\\mathbf\{x\}\)\\leq 1, each term is bounded byp​\(𝐱\)α​eβp\(\\mathbf\{x\}\)^\{\\alpha\}e^\{\\beta\}; summing over the finite set𝒮\\mathcal\{S\}givesZ≤eβ​∑𝐱∈𝒮p​\(𝐱\)α≤eβ​∑𝐱∈𝒮p​\(𝐱\)=eβ<∞Z\\leq e^\{\\beta\}\\sum\_\{\\mathbf\{x\}\\in\\mathcal\{S\}\}p\(\\mathbf\{x\}\)^\{\\alpha\}\\leq e^\{\\beta\}\\sum\_\{\\mathbf\{x\}\\in\\mathcal\{S\}\}p\(\\mathbf\{x\}\)=e^\{\\beta\}<\\infty, usingα≥1\\alpha\\geq 1andp​\(𝐱\)≤1p\(\\mathbf\{x\}\)\\leq 1\. Hence0<Z<∞0<Z<\\infty\. ∎

### F\.2Convergence of DEGS\-MCMC

###### Theorem 1\(Convergence to the DEGS target\)\.

Consider the Metropolis–Hastings chain with targetπ\\piand the random\-index resampling proposalqqofKaran and Du \([2025](https://arxiv.org/html/2607.09693#bib.bib17)\), using the acceptance ratio equation[6](https://arxiv.org/html/2607.09693#S4.E6)\. Ifqqis irreducible and aperiodic on𝒮\\mathcal\{S\}, then the chain hasπ\\pias its unique stationary distribution and the marginal law of thenn\-th state converges toπ\\piin total variation asn→∞n\\to\\infty\.

###### Proof\.

The acceptance ratio equation[6](https://arxiv.org/html/2607.09693#S4.E6)is exactly the Metropolis–Hastings ratio for targetπ\\pi, because the unknown normalizerZZcancels:

π​\(𝐱′\)​q​\(𝐱∣𝐱′\)π​\(𝐱\)​q​\(𝐱′∣𝐱\)=p​\(𝐱′\)α​exp⁡\(β​D​\(𝐱′\)\)​q​\(𝐱∣𝐱′\)p​\(𝐱\)α​exp⁡\(β​D​\(𝐱\)\)​q​\(𝐱′∣𝐱\)\.\\frac\{\\pi\(\\mathbf\{x\}^\{\\prime\}\)\\,q\(\\mathbf\{x\}\\mid\\mathbf\{x\}^\{\\prime\}\)\}\{\\pi\(\\mathbf\{x\}\)\\,q\(\\mathbf\{x\}^\{\\prime\}\\mid\\mathbf\{x\}\)\}=\\frac\{p\(\\mathbf\{x\}^\{\\prime\}\)^\{\\alpha\}\\exp\(\\beta D\(\\mathbf\{x\}^\{\\prime\}\)\)\\,q\(\\mathbf\{x\}\\mid\\mathbf\{x\}^\{\\prime\}\)\}\{p\(\\mathbf\{x\}\)^\{\\alpha\}\\exp\(\\beta D\(\\mathbf\{x\}\)\)\\,q\(\\mathbf\{x\}^\{\\prime\}\\mid\\mathbf\{x\}\)\}\.By Proposition[1](https://arxiv.org/html/2607.09693#Thmproposition1),π\\piis a well\-defined distribution, and standard Metropolis–Hastings theory\(Neal,[1993](https://arxiv.org/html/2607.09693#bib.bib23)\)guarantees that the chain satisfies detailed balance with respect toπ\\pi\. Detailed balance makesπ\\pistationary; irreducibility and aperiodicity of the proposal \(inherited by the accepted chain, since the acceptance probability is positive whereverπ\\piis\) make the stationary distribution unique and yield convergence in total variation by the ergodic theorem for Markov chains\. ∎

### F\.3Power vs\. temperature sampling

###### Proposition 2\(Power sampling is not temperature sampling\)\.

Letp​\(𝐱\)=∏t=1Tp​\(xt∣x<t\)p\(\\mathbf\{x\}\)=\\prod\_\{t=1\}^\{T\}p\(x\_\{t\}\\mid x\_\{<t\}\)be autoregressive\. The sequence\-level power distributionpα​\(𝐱\)∝p​\(𝐱\)αp^\{\\alpha\}\(\\mathbf\{x\}\)\\propto p\(\\mathbf\{x\}\)^\{\\alpha\}is in general*not*equal to the autoregressive distribution obtained by temperature sampling at temperature1/α1/\\alpha, i\.e\. the modelr​\(𝐱\)=∏tr​\(xt∣x<t\)r\(\\mathbf\{x\}\)=\\prod\_\{t\}r\(x\_\{t\}\\mid x\_\{<t\}\)withr​\(xt∣x<t\)∝p​\(xt∣x<t\)αr\(x\_\{t\}\\mid x\_\{<t\}\)\\propto p\(x\_\{t\}\\mid x\_\{<t\}\)^\{\\alpha\}, except in degenerate cases\.

###### Proof\.

Temperature sampling renormalizes*per step*:r​\(xt∣x<t\)=p​\(xt∣x<t\)α/Zt​\(x<t\)r\(x\_\{t\}\\mid x\_\{<t\}\)=p\(x\_\{t\}\\mid x\_\{<t\}\)^\{\\alpha\}/Z\_\{t\}\(x\_\{<t\}\)withZt​\(x<t\)=∑vp​\(v∣x<t\)αZ\_\{t\}\(x\_\{<t\}\)=\\sum\_\{v\}p\(v\\mid x\_\{<t\}\)^\{\\alpha\}\. Hence

r​\(𝐱\)=∏t=1Tp​\(xt∣x<t\)αZt​\(x<t\)=p​\(𝐱\)α∏t=1TZt​\(x<t\)\.r\(\\mathbf\{x\}\)=\\prod\_\{t=1\}^\{T\}\\frac\{p\(x\_\{t\}\\mid x\_\{<t\}\)^\{\\alpha\}\}\{Z\_\{t\}\(x\_\{<t\}\)\}=\\frac\{p\(\\mathbf\{x\}\)^\{\\alpha\}\}\{\\prod\_\{t=1\}^\{T\}Z\_\{t\}\(x\_\{<t\}\)\}\.The sequence\-level power distribution instead uses a single global normalizer,pα​\(𝐱\)=p​\(𝐱\)α/Zp^\{\\alpha\}\(\\mathbf\{x\}\)=p\(\\mathbf\{x\}\)^\{\\alpha\}/ZwithZ=∑𝐱p​\(𝐱\)αZ=\\sum\_\{\\mathbf\{x\}\}p\(\\mathbf\{x\}\)^\{\\alpha\}\. These coincide only if∏tZt​\(x<t\)\\prod\_\{t\}Z\_\{t\}\(x\_\{<t\}\)is the same constantZZfor every𝐱\\mathbf\{x\}in the support, which fails whenever the per\-step normalizersZt​\(x<t\)Z\_\{t\}\(x\_\{<t\}\)depend on the prefixx<tx\_\{<t\}—the generic case for a non\-uniform model\. Thus the two distributions differ in general\. ∎

### F\.4Why collapse depth concentrates on hard tokens

###### Proposition 3\(Concentrated futures collapse early\)\.

Fix a token positionttand consider its logit\-lens distributionspl\(⋅∣x≤t\)p\_\{l\}\(\\cdot\\mid x\_\{\\leq t\}\)across layersll\. Suppose at some layerl0l\_\{0\}the distribution is already concentrated,maxv⁡pl0​\(v∣x≤t\)≥1−ϵ\\max\_\{v\}p\_\{l\_\{0\}\}\(v\\mid x\_\{\\leq t\}\)\\geq 1\-\\epsilonfor smallϵ∈\(0,1\)\\epsilon\\in\(0,1\)\. Then the logit\-lens entropy atl0l\_\{0\}satisfiesHl0​\(t\)≤Hb​\(ϵ\)\+ϵ​log⁡\(\|𝒳\|−1\)H\_\{l\_\{0\}\}\(t\)\\leq H\_\{b\}\(\\epsilon\)\+\\epsilon\\log\(\|\\mathcal\{X\}\|\-1\), whereHbH\_\{b\}is the binary entropy function\. In particular, for smallϵ\\epsilonthe entropy is below a modest thresholdτ\\tau, sodt≤l0d\_\{t\}\\leq l\_\{0\}\.

###### Proof\.

This is Fano’s inequality applied to the logit\-lens distribution\. Letv⋆=arg​maxv⁡pl0​\(v∣x≤t\)v^\{\\star\}=\\operatorname\*\{arg\\,max\}\_\{v\}p\_\{l\_\{0\}\}\(v\\mid x\_\{\\leq t\}\)with mass1−δ1\-\\deltafor someδ≤ϵ\\delta\\leq\\epsilon\. Writing the entropy as that of a mixture betweenv⋆v^\{\\star\}and the remaining mass,

Hl0\(t\)=Hb\(δ\)\+δ⋅H\(pl0\(⋅∣x≤t,v≠v⋆\)\)≤Hb\(δ\)\+δlog\(\|𝒳\|−1\),H\_\{l\_\{0\}\}\(t\)=H\_\{b\}\(\\delta\)\+\\delta\\cdot H\\\!\\big\(p\_\{l\_\{0\}\}\(\\cdot\\mid x\_\{\\leq t\},\\,v\\neq v^\{\\star\}\)\\big\)\\leq H\_\{b\}\(\\delta\)\+\\delta\\log\(\|\\mathcal\{X\}\|\-1\),since the conditional distribution over the other\|𝒳\|−1\|\\mathcal\{X\}\|\-1tokens has entropy at mostlog⁡\(\|𝒳\|−1\)\\log\(\|\\mathcal\{X\}\|\-1\)\. BothHb​\(δ\)H\_\{b\}\(\\delta\)andδ​log⁡\(\|𝒳\|−1\)\\delta\\log\(\|\\mathcal\{X\}\|\-1\)are nondecreasing inδ\\deltaon\[0,1/2\]\[0,1/2\], so the bound holds withδ\\deltareplaced byϵ\\epsilon\. Forϵ\\epsilonsmall enough thatHb​\(ϵ\)\+ϵ​log⁡\(\|𝒳\|−1\)≤τH\_\{b\}\(\\epsilon\)\+\\epsilon\\log\(\|\\mathcal\{X\}\|\-1\)\\leq\\tau, we getHl0​\(t\)≤τH\_\{l\_\{0\}\}\(t\)\\leq\\tau, hencedt=min⁡\{l∈ℒsub:Hl​\(t\)≤τ\}≤l0d\_\{t\}=\\min\\\{l\\in\\mathcal\{L\}\_\{\\mathrm\{sub\}\}:H\_\{l\}\(t\)\\leq\\tau\\\}\\leq l\_\{0\}\. ∎

### F\.5Reranking approximates the DEGS target

###### Proposition 4\(Self\-normalized importance sampling\)\.

Let\{𝐱\(i\)\}i=1N\\\{\\mathbf\{x\}^\{\(i\)\}\\\}\_\{i=1\}^\{N\}be drawn i\.i\.d\. from a proposalggwithg​\(𝐱\)\>0g\(\\mathbf\{x\}\)\>0on𝒮\\mathcal\{S\}\. Define importance weightswi∝π​\(𝐱\(i\)\)/g​\(𝐱\(i\)\)w\_\{i\}\\propto\\pi\(\\mathbf\{x\}^\{\(i\)\}\)/g\(\\mathbf\{x\}^\{\(i\)\}\), normalized to sum to one\. Then for any bounded test functionhh, the self\-normalized estimator∑iwi​h​\(𝐱\(i\)\)\\sum\_\{i\}w\_\{i\}h\(\\mathbf\{x\}^\{\(i\)\}\)converges almost surely to𝔼π​\[h\]\\mathbb\{E\}\_\{\\pi\}\[h\]asN→∞N\\to\\infty\. Choosingg=pg=pgives unnormalized weightswi∝p​\(𝐱\(i\)\)α−1​exp⁡\(β​D​\(𝐱\(i\)\)\)w\_\{i\}\\propto p\(\\mathbf\{x\}^\{\(i\)\}\)^\{\\alpha\-1\}\\exp\(\\beta D\(\\mathbf\{x\}^\{\(i\)\}\)\)\.

###### Proof\.

Self\-normalized importance sampling is consistent wheneversupp​\(π\)⊆supp​\(g\)\\mathrm\{supp\}\(\\pi\)\\subseteq\\mathrm\{supp\}\(g\)\(Neal,[1993](https://arxiv.org/html/2607.09693#bib.bib23)\)\. The unnormalized weight isπ​\(𝐱\)/g​\(𝐱\)∝p​\(𝐱\)α​exp⁡\(β​D​\(𝐱\)\)/p​\(𝐱\)=p​\(𝐱\)α−1​exp⁡\(β​D​\(𝐱\)\)\\pi\(\\mathbf\{x\}\)/g\(\\mathbf\{x\}\)\\propto p\(\\mathbf\{x\}\)^\{\\alpha\}\\exp\(\\beta D\(\\mathbf\{x\}\)\)/p\(\\mathbf\{x\}\)=p\(\\mathbf\{x\}\)^\{\\alpha\-1\}\\exp\(\\beta D\(\\mathbf\{x\}\)\)wheng=pg=p, and the global constantZZcancels under normalization\. Almost\-sure convergence follows from the strong law of large numbers applied to numerator and denominator separately, both of which have finite expectation sincehhis bounded and the weights are integrable on the finite support𝒮\\mathcal\{S\}\. DEGS reranking returns the maximizerarg​maxi⁡\[α​log⁡p​\(𝐱\(i\)\)\+β​D​\(𝐱\(i\)\)\]\\operatorname\*\{arg\\,max\}\_\{i\}\[\\alpha\\log p\(\\mathbf\{x\}^\{\(i\)\}\)\+\\beta D\(\\mathbf\{x\}^\{\(i\)\}\)\], the mode of the reweighted set; the same weights underlie the full self\-normalized estimator\. ∎

### F\.6Cost accounting

###### Proposition 5\(Probing overhead\)\.

Let a full forward pass over a length\-TTsequence costCfwdC\_\{\\mathrm\{fwd\}\}\(dominated byLLtransformer layers\)\. ComputingD​\(𝐱\)D\(\\mathbf\{x\}\)by caching hidden states atK=\|ℒsub\|K=\|\\mathcal\{L\}\_\{\\mathrm\{sub\}\}\|layers and applying the logit lens adds entropy computations at thoseKKlayers only\. The marginal cost of probing isCprobe=K⋅clensC\_\{\\mathrm\{probe\}\}=K\\cdot c\_\{\\mathrm\{lens\}\}, whereclensc\_\{\\mathrm\{lens\}\}is the per\-layer logit\-lens\-plus\-entropy cost, independent ofLL; relative to a generation forward pass that already computes allLLlayers, probing reuses the same hidden states and adds only the lens/entropy arithmetic\.

###### Proof\.

The forward pass computes hidden states𝐡l,t\\mathbf\{h\}\_\{l,t\}at all layers regardless of probing\. Probing reads the cached states at theKKlayers inℒsub\\mathcal\{L\}\_\{\\mathrm\{sub\}\}and, for each, applies the unembedding and computes entropy, costingclensc\_\{\\mathrm\{lens\}\}per layer \(anO​\(\|𝒳\|​d\)O\(\|\\mathcal\{X\}\|d\)matrix\-vector product plus anO​\(\|𝒳\|\)O\(\|\\mathcal\{X\}\|\)entropy sum, optionally chunked over the vocabulary\)\. Summing over theKKprobed layers givesCprobe=K​clensC\_\{\\mathrm\{probe\}\}=K\\,c\_\{\\mathrm\{lens\}\}, with no dependence on the total depthLLbeyondK≤LK\\leq L\. The teacher\-forced scoring pass is a single additional forward, so the total scoring cost isCfwd\+CprobeC\_\{\\mathrm\{fwd\}\}\+C\_\{\\mathrm\{probe\}\}\. ∎

###### Proposition 6\(Token budget of DEGS\-MCMC\)\.

With block sizeBB, sequence lengthTT, andNMCMCN\_\{\\mathrm\{MCMC\}\}Metropolis–Hastings steps per block, the expected number of generated tokens isΘ​\(NMCMC​T2/B\)\\Theta\(N\_\{\\mathrm\{MCMC\}\}T^\{2\}/B\)up to the per\-step resampling fraction, matching the accounting ofKaran and Du \([2025](https://arxiv.org/html/2607.09693#bib.bib17)\)\.

###### Proof\.

There areT/BT/Bblocks\. Within blockkk, the state has lengthΘ​\(k​B\)\\Theta\(kB\), and a random\-index proposal resamples on average half the suffix from the chosen cut point, i\.e\.Θ​\(k​B\)\\Theta\(kB\)tokens per accepted/attempted proposal\. RunningNMCMCN\_\{\\mathrm\{MCMC\}\}steps per block and summing overk=1,…,T/Bk=1,\\dots,T/Bgives∑k=1T/BNMCMC​Θ​\(k​B\)=NMCMC​Θ​\(B⋅\(T/B\)22\)=Θ​\(NMCMC​T2/B\)\\sum\_\{k=1\}^\{T/B\}N\_\{\\mathrm\{MCMC\}\}\\,\\Theta\(kB\)=N\_\{\\mathrm\{MCMC\}\}\\,\\Theta\\\!\\big\(B\\cdot\\tfrac\{\(T/B\)^\{2\}\}\{2\}\\big\)=\\Theta\(N\_\{\\mathrm\{MCMC\}\}T^\{2\}/B\)\. This reproduces the quadratic\-in\-TTcost of block\-wise power sampling\. ∎

### F\.7Entropy decomposition for the depth signal

###### Definition 4\(Layer\-conditional entropy contribution\)\.

For tokentt, define the inter\-layer change in logit\-lens belief by the KL divergenceΔl\(t\)=KL\(pl\(⋅∣x≤t\)∥pl−1\(⋅∣x≤t\)\)\\Delta\_\{l\}\(t\)=\\mathrm\{KL\}\\\!\\big\(p\_\{l\}\(\\cdot\\mid x\_\{\\leq t\}\)\\,\\\|\\,p\_\{l\-1\}\(\\cdot\\mid x\_\{\\leq t\}\)\\big\)between consecutive layersl−1l\-1andll\.

###### Proposition 7\(Collapse depth tracks belief stabilization\)\.

If the logit\-lens beliefs stabilize after layerl⋆l^\{\\star\}in the sense thatΔl​\(t\)≤η\\Delta\_\{l\}\(t\)\\leq\\etafor alll\>l⋆l\>l^\{\\star\}and some smallη\\eta, and the entropyHl​\(t\)H\_\{l\}\(t\)is non\-increasing forl\>l⋆l\>l^\{\\star\}, then the per\-token collapse depth satisfiesdt≤l⋆d\_\{t\}\\leq l^\{\\star\}wheneverHl⋆​\(t\)≤τH\_\{l^\{\\star\}\}\(t\)\\leq\\tau\. ConsequentlyD​\(𝐱\)D\(\\mathbf\{x\}\)is governed by the depth at which beliefs stabilize, not by post\-stabilization fluctuations\.

###### Proof\.

By definitiondt=min⁡\{l∈ℒsub:Hl​\(t\)≤τ\}d\_\{t\}=\\min\\\{l\\in\\mathcal\{L\}\_\{\\mathrm\{sub\}\}:H\_\{l\}\(t\)\\leq\\tau\\\}\. IfHl⋆​\(t\)≤τH\_\{l^\{\\star\}\}\(t\)\\leq\\tauthen the minimizing layer is at mostl⋆l^\{\\star\}, sodt≤l⋆d\_\{t\}\\leq l^\{\\star\}\. The stabilization conditionΔl​\(t\)≤η\\Delta\_\{l\}\(t\)\\leq\\etaforl\>l⋆l\>l^\{\\star\}ensures the beliefs \(and hence the entropy\) change negligibly beyondl⋆l^\{\\star\}, so the threshold crossing that definesdtd\_\{t\}is determined at or beforel⋆l^\{\\star\}; later near\-constant layers cannot move the crossing earlier or later\. Averagingdt/Ld\_\{t\}/Lover tokens,D​\(𝐱\)D\(\\mathbf\{x\}\)depends on the stabilization depths\{lt⋆\}\\\{l^\{\\star\}\_\{t\}\\\}rather than on fluctuations in deeper layers\. ∎

### F\.8Bounded perturbation of the likelihood ranking

###### Proposition 8\(Depth term is a bounded reranking perturbation\)\.

Fix two candidate sequences𝐱,𝐱′\\mathbf\{x\},\\mathbf\{x\}^\{\\prime\}\. Under the DEGS scoreS​\(𝐱\)=α​log⁡p​\(𝐱\)\+β​D​\(𝐱\)S\(\\mathbf\{x\}\)=\\alpha\\log p\(\\mathbf\{x\}\)\+\\beta D\(\\mathbf\{x\}\)withD​\(𝐱\)∈\(0,1\]D\(\\mathbf\{x\}\)\\in\(0,1\], the depth term can change the score gap by at mostβ\\betain absolute value:\|\[S​\(𝐱′\)−S​\(𝐱\)\]−α​\[log⁡p​\(𝐱′\)−log⁡p​\(𝐱\)\]\|=β​\|D​\(𝐱′\)−D​\(𝐱\)\|≤β\\big\|\[S\(\\mathbf\{x\}^\{\\prime\}\)\-S\(\\mathbf\{x\}\)\]\-\\alpha\[\\log p\(\\mathbf\{x\}^\{\\prime\}\)\-\\log p\(\\mathbf\{x\}\)\]\\big\|=\\beta\\,\|D\(\\mathbf\{x\}^\{\\prime\}\)\-D\(\\mathbf\{x\}\)\|\\leq\\beta\. Hence ifα​\|log⁡p​\(𝐱′\)−log⁡p​\(𝐱\)\|\>β\\alpha\|\\log p\(\\mathbf\{x\}^\{\\prime\}\)\-\\log p\(\\mathbf\{x\}\)\|\>\\beta, the DEGS ranking of𝐱,𝐱′\\mathbf\{x\},\\mathbf\{x\}^\{\\prime\}agrees with the pure\-likelihood ranking; the depth signal can only reorder candidates whoseα\\alpha\-scaled log\-likelihoods lie withinβ\\betaof each other\.

###### Proof\.

By definitionS​\(𝐱′\)−S​\(𝐱\)=α​\[log⁡p​\(𝐱′\)−log⁡p​\(𝐱\)\]\+β​\[D​\(𝐱′\)−D​\(𝐱\)\]S\(\\mathbf\{x\}^\{\\prime\}\)\-S\(\\mathbf\{x\}\)=\\alpha\[\\log p\(\\mathbf\{x\}^\{\\prime\}\)\-\\log p\(\\mathbf\{x\}\)\]\+\\beta\[D\(\\mathbf\{x\}^\{\\prime\}\)\-D\(\\mathbf\{x\}\)\], so the deviation from the pure\-likelihood gap is exactlyβ​\[D​\(𝐱′\)−D​\(𝐱\)\]\\beta\[D\(\\mathbf\{x\}^\{\\prime\}\)\-D\(\\mathbf\{x\}\)\]\. SinceD​\(𝐱\),D​\(𝐱′\)∈\(0,1\]D\(\\mathbf\{x\}\),D\(\\mathbf\{x\}^\{\\prime\}\)\\in\(0,1\], their difference lies in\(−1,1\)\(\-1,1\), giving the boundβ​\|D​\(𝐱′\)−D​\(𝐱\)\|≤β\\beta\|D\(\\mathbf\{x\}^\{\\prime\}\)\-D\(\\mathbf\{x\}\)\|\\leq\\beta\. Ifα​\[log⁡p​\(𝐱′\)−log⁡p​\(𝐱\)\]\>β\\alpha\[\\log p\(\\mathbf\{x\}^\{\\prime\}\)\-\\log p\(\\mathbf\{x\}\)\]\>\\beta, thenS​\(𝐱′\)−S​\(𝐱\)≥α​\[log⁡p​\(𝐱′\)−log⁡p​\(𝐱\)\]−β\>0S\(\\mathbf\{x\}^\{\\prime\}\)\-S\(\\mathbf\{x\}\)\\geq\\alpha\[\\log p\(\\mathbf\{x\}^\{\\prime\}\)\-\\log p\(\\mathbf\{x\}\)\]\-\\beta\>0, preserving the likelihood order; symmetrically for the reverse inequality\. Thus reordering requiresα​\|log⁡p​\(𝐱′\)−log⁡p​\(𝐱\)\|≤β\\alpha\|\\log p\(\\mathbf\{x\}^\{\\prime\}\)\-\\log p\(\\mathbf\{x\}\)\|\\leq\\beta\. ∎