Sampling via Decision-Flow: Training-Free Extraction of Improved Latent Reasoning Paths in Large Language Models

# Sampling via Decision-Flow: Training-Free Extraction of Improved Latent Reasoning Paths in Large Language Models
Source: [https://arxiv.org/html/2609.12317](https://arxiv.org/html/2609.12317)
Zhendong Mi,Shaoyi Huang†

###### Abstract

A central question in LLM reasoning is whether reinforcement learning \(RL\) instills genuinely new capabilities or merely reshapes how existing knowledge is expressed during inference\. Building on the distribution\-sharpening hypothesis, which holds that RL reallocates probability mass toward high\-reward trajectories already latent in base models, we ask: can we unlock those latent paths without costly RL fine\-tuning? We presentDecision\-Flow Sampling \(DF\-Sample\), a training\-free, data\-free inference\-time framework that constructs a hierarchical reasoning tree, scores terminal nodes for quality, and back\-propagates utilities to inform each intermediate branching decision\. Unlike conventional sampling strategies that make purely local step\-wise choices, DF\-Sample performs explicit global trajectory evaluation before committing to a path, recovering high\-quality but low\-probability reasoning chains that standard decoding overlooks\. On GPQA, DF\-Sample achieves 45\.6% accuracy, surpassing power sampling \(38\.9%\) and GRPO \(39\.9%\), showing that a training\-free method can outperform a trained one\. Across three models and four benchmarks, DF\-Sample consistently outperforms baselines, indicating substantial latent reasoning potential in pretrained base models\.

## 1Introduction

Reinforcement learning has become the dominant post\-training paradigm for improving LLM reasoning[Guo et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib4);[Hu et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib5)\. By optimizing with outcome\- or process\-level rewards, RL\-based post\-training has pushed state\-of\-the\-art performance on challenging benchmarks in mathematics, code, and science[Hendrycks et al\. \(2021\)](https://arxiv.org/html/2609.12317#bib.bib12);[Li et al\. \(2022\)](https://arxiv.org/html/2609.12317#bib.bib13);[Rein et al\. \(2024\)](https://arxiv.org/html/2609.12317#bib.bib28)\. Yet how RL improves reasoning remains actively debated[He et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib30);[Karan and Du \(2025\)](https://arxiv.org/html/2609.12317#bib.bib31)\.

TheDistribution Sharpeninghypothesis[Shao et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib25);[Zhang et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib27);[Yue et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib26)provides one compelling answer: RL does not introduce genuinely new competencies, but rather reallocates probability mass toward high\-reward trajectories that were already latent in the base model\. While this effectively improves single\-attempt metrics such as Pass@1, it simultaneously reduces output diversity by concentrating mass on high\-confidence paths and suppressing alternative strategies\. This reduction in diversity comes at a cost\. Several recent works have implicated it in a form of model collapse[He et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib30);[Hao et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib18);[Song et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib17), in which the model’s reasoning coverage contracts even as peak performance improves[Liang et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib24)\. The model becomes more confident but less exploratory\.

Figure 1:DF\-Sample vs\. baselines on Qwen2\.5\-MATH\-7B using MATH500, HumanEval, GPQA, AlpacaEval 2\.0 datasets\. DF\-Sample achieves the best performance on different datasets\. Full results are as shown in Table[1](https://arxiv.org/html/2609.12317#S4.T1)\.This raises two natural questions\.First, if high\-quality reasoning paths are already latent in the base model[Karan and Du \(2025\)](https://arxiv.org/html/2609.12317#bib.bib31), can we unlock them without costly RL fine\-tuning[Dragoi et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib16);[Vafaii et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib14)? We argue yes\. The key insight is that the problem is not one of missing knowledge but of misallocated probability, and misallocation is a sampling problem, not a training problem\.Second, given that a correct reasoning path already exists, how do we effectively find it? Existing approaches inspired by probabilistic sampling techniques such as MCMC[Karan and Du \(2025\)](https://arxiv.org/html/2609.12317#bib.bib31);[Wu et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib15)correct reasoning locally, one step at a time\. But local optimality at each step does not guarantee global optimality for the full trajectory\. A locally plausible transition can still steer reasoning toward a globally wrong conclusion\.

To address both questions, we proposeDF\-Sample: a training\-free framework that selects high\-quality reasoning chains during inference by performing explicit global trajectory evaluation\. Rather than making greedy or locally\-guided decisions at each step, DF\-Sample constructs a hierarchical reasoning tree, evaluates terminal nodes for quality, propagates those quality signals backward through the tree, and samples a final path according to a posterior that integrates generation priors with the propagated utilities\. Our contributions are as follows:

- •Decision\-Flow Sampling \(DF\-Sample\)\. A training\-free inference\-time framework that extracts high\-quality but low\-probability reasoning paths from base language models, improving reasoning accuracy without any parameter updates\.
- •Global trajectory evaluation via terminal\-node utility and backward propagation\.We introduce a terminal energy function that scores complete reasoning chains and a backward propagation mechanism that carries those global signals to each intermediate decision, mitigating the myopic bias of standard sampling\.
- •Extensive empirical validation\.Across three model families \(Qwen2\.5\-Math\-7B, Qwen2\.5\-7B, Phi\-3\.5\-mini\-instruct\) and four datasets \(MATH500, HumanEval, GPQA\-Diamond, AlpacaEval 2\.0\), DF\-Sample consistently outperforms base models, competitive sampling baselines, and RL\-trained counterparts\. On MATH500 with Qwen2\.5\-Math\-7B, DF\-Sample achieves 81\.8% accuracy, surpassing GRPO \(78\.5%\) and power sampling \(74\.8%\) as shown in Figure[1](https://arxiv.org/html/2609.12317#S1.F1)\.

## 2Motivation

Before introducing the method, we present two empirical observations about base model reasoning behavior which motivate our work\.

Correct reasoning paths are latent, not absent\.Building on recent evidence[Karan and Du \(2025\)](https://arxiv.org/html/2609.12317#bib.bib31);[Hao et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib18), we observe that base models often already possess correct reasoning paths and they simply assign those paths lower probability than incorrect alternatives\. Consider the equation*x2=5​xx^\{2\}=5x*\(Figure[2](https://arxiv.org/html/2609.12317#S2.F2)\)\. A correct solution factors both sides:*x2−5​x=0⇒x⁡\(x−5\)=0x^\{2\}\-5x=0\\Rightarrow x\(x\-5\)=0\.*, yielding both solutions\. However, the model may instead divide both sides byxx, implicitly assumingx≠0x\\neq 0and missing the solutionx=0x=0\. The division path receives a higher probability because it matches common patterns and requires fewer steps, not because it is more correct\.

This is not an isolated failure mode\. It reflects a systematic mismatch between generation probability and reasoning quality in base models\. The implication for our method is direct: we do not need to train the model to produce better reasoning; we need a sampling procedure that looks past local probability and identifies globally superior paths\.

Figure 2:Upper path \(incorrect, high confidence\) vs\. lower path \(correct, low confidence\) for quadratic equation solving problem\.Local sampling optimality does not guarantee global reasoning quality\.Given that correct reasoning paths are latent in the model’s distribution, the question becomes how to find them\. A natural candidate is MCMC\-style sampling[Karan and Du \(2025\)](https://arxiv.org/html/2609.12317#bib.bib31)for correcting reasoning through local proposal\-and\-accept steps\. However, multi\-step CoT reasoning violates a key assumption that makes local corrections sufficient: the quality of a single reasoning step is not a reliable signal of the quality of the complete trajectory\.

Figure[3](https://arxiv.org/html/2609.12317#S2.F3)illustrates this with the task of determining*whetherf⁡\(x\)=x2f\(x\)=x^\{2\}is injective onℝ\\mathbb\{R\}*\. Path A exploits symmetry, noting thatf⁡\(x\)=f⁡\(−x\)f\(x\)=f\(\-x\)forx≠0x\\neq 0, and correctly concludesffis not injective\. Path B introduces a derivative argument \(f′​\(x\)=2​xf^\{\\prime\}\(x\)=2x,f′​\(x\)≠0f^\{\\prime\}\(x\)\\neq 0forx≠0x\\neq 0, therefore monotonic, therefore injective\), which is locally plausible at each step but reaches an incorrect conclusion\. A sampling procedure that accepts Path B’s steps because they are individually reasonable will systematically fail on problems where globally\-correct reasoning is locally counterintuitive\.

This motivates an approach that evaluates reasoning paths holistically, constructing a space of complete paths and selecting among them based on path\-level quality rather than step\-level probability\.

![Refer to caption](https://arxiv.org/html/2609.12317v1/fig3.png)Figure 3:Two reasoning paths for the same problem\. Path B is locally plausible at each step but reaches an incorrect conclusion; Path A reasons correctly\.
## 3Preliminary

Let𝒴\\mathcal\{Y\}denote a finite discrete token vocabulary, and consider sequences consisting of tokens in𝒴\\mathcal\{Y\}\. A sequence of lengthT∈ℤ≥0T\\in\\mathbb\{Z\}\_\{\\geq 0\}is denoted by𝐲0:T=\(y0,y1,…,yT\)\\mathbf\{y\}\_\{0:T\}=\(y\_\{0\},y\_\{1\},\\dots,y\_\{T\}\), where eachyt∈𝒴y\_\{t\}\\in\\mathcal\{Y\}\. We use𝐲\\mathbf\{y\}to represent the full sequence\. For any time steptt, we define the prefix \(historical context\) as𝐲<t=\(y0,…,yt−1\)\\mathbf\{y\}\_\{<t\}=\(y\_\{0\},\\dots,y\_\{t\-1\}\), and the suffix \(generation of future tokens\) as𝐲\>t=\(yt\+1,…,yT\)\\mathbf\{y\}\_\{\>t\}=\(y\_\{t\+1\},\\dots,y\_\{T\}\)\.

Under this formulation, a large language model \(LLM\) induces a probability distributionpθp\_\{\\theta\}over the sequence space\. Autoregressive language models parameterize this distribution through conditional factorspθ​\(yt∣𝐲<t\)p\_\{\\theta\}\(y\_\{t\}\\mid\\mathbf\{y\}\_\{<t\}\)\. By the chain rule of probability, the joint likelihood of a sequence can be factorized as

pθ\(𝐲0:T\)=∏t=0Tpθ\(yt∣𝐲<t\)\.p\_\{\\theta\}\(\\mathbf\{y\}\_\{0:T\}\)=\\prod\_\{t=0\}^\{T\}p\_\{\\theta\}\(y\_\{t\}\\mid\\mathbf\{y\}\_\{<t\}\)\.\(1\)
Consequently, sampling a complete sequence frompθp\_\{\\theta\}corresponds to sequentially sampling tokens according to the conditional distributions above\.

## 4Proposed Method

![Refer to caption](https://arxiv.org/html/2609.12317v1/main.png)Figure 4:An illustrative toy example of Decision\-Flow sampling with branching factorKK, showing how global reasoning strategies are selected by combining utilityUUwith prior probabilities\.DF\-Sample operates in four sequential stages, each building on the output of the previous\. Specifically,Stage I:Hierarchical Reasoning Tree Construction\.A hierarchical reasoning tree is constructed which systematically expands candidate reasoning steps\.Stage II:Terminal Node Energy Evaluation\.We assign each final node an utility score based on the quality of its corresponding final reasoning output\.Stage III:Decision\-Flow Backward Propagation\.Terminal evaluation utility are propagated backward through the tree, enabling intermediate reasoning states to incorporate and obtain globally node\-level utilities or utility\.Stage IV:Posterior Path Selection\.A reasoning path is selected according to a posterior decision rule that integrates prior generation tendencies with the propagated utility, and the final answer is derived from the selected chain\. A toy example is shown in Figure[4](https://arxiv.org/html/2609.12317#S4.F4)to illustrate the four stages\.

### 4\.1Hierarchical Reasoning Tree Construction

Given an input problemqq, we construct a reasoning tree𝒯\\mathcal\{T\}of depthLLby hierarchical sampling\. At each node of depthℓ−1\\ell\-1, we independently sampleKKcandidate extensions from the language model, forming total child nodes𝒞⁡\(vℓ−1\)=\{vℓ1,vℓ2,…,vℓd​K\}\\mathcal\{C\}\(v\_\{\\ell\-1\}\)=\\\{v^\{1\}\_\{\\ell\},v^\{2\}\_\{\\ell\},\\ldots,v^\{dK\}\_\{\\ell\}\\\}\(dddenotes the number of nodes at depthℓ−1\\ell\-1\)\. Theii\-th childvℓiv^\{i\}\_\{\\ell\}at depthℓ\\ellcorresponds to one reasoning stepsvℓis\_\{v^\{i\}\_\{\\ell\}\}generated conditioned on the problem and the path from the root to the parent ofvℓiv\_\{\\ell\}^\{i\}\.

For each nodevv, we record both the generated reasoning textsvs\_\{v\}and the accumulated log\-probability along the path from the root:

log⁡p⁡\(sv∣q,s<v\)=∑t=1\|sv\|log⁡p⁡\(wt∣q,s<v,w<t\)\\log p\(s\_\{v\}\\mid q,\\,s\_\{<v\}\)=\\sum\_\{t=1\}^\{\|s\_\{v\}\|\}\\log p\(w\_\{t\}\\mid q,\\,s\_\{<v\},\\,w\_\{<t\}\)\(2\)The full tree is constructed before any selection is made, enabling subsequent global evaluation over the entire reasoning space, a key departure from step\-wise decoding strategies\.

![Refer to caption](https://arxiv.org/html/2609.12317v1/fig4.png)Figure 5:Final node exhibits strong summarization and conclusion\-oriented characteristics\.
### 4\.2Terminal Node Energy Evaluation

We evaluate the quality of complete reasoning chains by scoring their terminal nodes\. This design is motivated by a consistent empirical observation as shown in Figure[5](https://arxiv.org/html/2609.12317#S4.F5): the final reasoning step typically exhibits strong summarization and conclusion\-oriented properties, condensing the chain’s outcome into a concise statement\. Evaluating the terminal node therefore serves as an efficient proxy for trajectory quality\.

Define a terminal energy function over leaf nodes:

E\(τ\)=−α⋅log⁡p⁡\(svL∣q,s<vL\)\|svL\|\+R\(svL\),E\(\\tau\)\\;=\\;\-\\alpha\\cdot\\frac\{\\log p\\\!\\left\(s\_\{v\_\{L\}\}\\mid q,s\_\{<v\_\{L\}\}\\right\)\}\{\|s\_\{v\_\{L\}\}\|\}\+R\(s\_\{v\_\{L\}\}\),\(3\)whereα\\alphais a temperature coefficient controlling the influence of model likelihood,\|svL\|\|s\_\{v\_\{L\}\}\|denotes the number of tokens of the final reasoning step, andR⁡\(svL\)R\(s\_\{v\_\{L\}\}\)is a quality score for the final output\. A lowerR⁡\(svL\)R\(s\_\{v\_\{L\}\}\)indicates higher output quality and a lower energyEEindicates a higher\-quality path\.

To evaluate the significance of leaf nodes, we define the terminal utility of each leaf nodevLv\_\{L\}asU⁡\(vL\)U\(v\_\{L\}\)\. This metric facilitates the propagation of path\-level evaluations back through the entire reasoning tree and is formulated as

U⁡\(vL\)=exp⁡\(−E⁡\(vL\)\)U\(v\_\{L\}\)=\\exp\\\!\\left\(\-E\(v\_\{L\}\)\\right\)\(4\)where higherU⁡\(vL\)U\(v\_\{L\}\)indicates greater terminal utility\. These leaf\-level scores are then propagated backward through the three in Stage III\.

### 4\.3Decision\-Flow Backward Propagation

Having scored all terminal nodes, we propagrate utility signals backward through the tree in a bottom\-up pass\. For any non\-leaf nodevℓ−1tv\_\{\\ell\-1\}^\{t\}, its utility is defined as the prior\-weighted expected utility of itsKKchildren, a recursive aggregation that carries trajectory\-level quality information to each intermediate decision state:

U⁡\(vℓ−1t\)=∑i=t​K\+1\(t\+1\)​Kpprior​\(vℓi∣vℓ−1t\)​U​\(vℓi\),U\(v\_\{\\ell\-1\}^\{t\}\)=\\sum\_\{i=tK\+1\}^\{\(t\+1\)K\}p\_\{\\text\{prior\}\}\(v\_\{\\ell\}^\{i\}\\mid v\_\{\\ell\-1\}^\{t\}\)\\,U\(v\_\{\\ell\}^\{i\}\),\(5\)wherevℓiv\_\{\\ell\}^\{i\}represents theii\-th node at Levelℓ\\ell\(the \(ii%KK\)\-th child ofvℓ−1tv\_\{\\ell\-1\}^\{t\}\)\.

The prior transition probabilitypprior​\(vℓi∣vℓ−1t\)p\_\{\\text\{prior\}\}\(v\_\{\\ell\}^\{i\}\\mid v\_\{\\ell\-1\}^\{t\}\)is defined by normalizing the language model’s generation probabilities over theKKsampled children at each node:

pprior​\(vℓi∣vℓ−1t\)=exp⁡\(log⁡p⁡\(svℓi∣vℓ−1t,s<vℓ−1t\)\)∑j=t​K\+1\(t\+1\)​Kexp⁡\(log⁡p⁡\(svℓj∣vℓ−1t,s<vℓ−1t\)\)p\_\{\\text\{prior\}\}\(v\_\{\\ell\}^\{i\}\\mid v\_\{\\ell\-1\}^\{t\}\)=\\frac\{\\exp\\\!\\left\(\\log p\(s\_\{v\_\{\\ell\}^\{i\}\}\\mid v\_\{\\ell\-1\}^\{t\},s\_\{<v\_\{\\ell\-1\}^\{t\}\}\)\\right\)\}\{\\sum\_\{j=tK\+1\}^\{\(t\+1\)K\}\\exp\\\!\\left\(\\log p\(s\_\{v\_\{\\ell\}^\{j\}\}\\mid v\_\{\\ell\-1\}^\{t\},s\_\{<v\_\{\\ell\-1\}^\{t\}\}\)\\right\)\}

\(6\)This backward propagation mechanism is what distinguishes DF\-Sample from locally\-guided methods: every intermediate node accumulates information about the global quality of the paths it can reach, rather than making decisions based solely on the local next\-step probability distribution\.

### 4\.4Posterior Path Selection

With utilities computed for all nodes, we define a posterior selection policy that integrates generation priors with propagated utilities\. For nodevℓ−1tv\_\{\\ell\-1\}^\{t\}, the probability of selecting itsii\-th childvℓiv\_\{\\ell\}^\{i\}is defined as:

π∗​\(vℓi∣vℓ−1t\)=pprior​\(vℓi∣vℓ−1t\)​U​\(vℓi\)∑j=t​K\+1\(t\+1\)​Kpprior​\(vℓj∣vℓ−1t\)​U​\(vℓj\)\.\\pi^\{\*\}\(v\_\{\\ell\}^\{i\}\\mid v\_\{\\ell\-1\}^\{t\}\)=\\frac\{p\_\{\\text\{prior\}\}\(v\_\{\\ell\}^\{i\}\\mid v\_\{\\ell\-1\}^\{t\}\)\\,U\(v\_\{\\ell\}^\{i\}\)\}\{\\sum\_\{j=tK\+1\}^\{\(t\+1\)K\}p\_\{\\text\{prior\}\}\(v\_\{\\ell\}^\{j\}\\mid v\_\{\\ell\-1\}^\{t\}\)\\,U\(v\_\{\\ell\}^\{j\}\)\}\.\(7\)
Starting from the root node, we recursively sample children according to the posterior distributionπ∗\(⋅∣⋅\)\\pi^\{\*\}\(\\cdot\\mid\\cdot\)until reaching a final leaf node, yielding a complete reasoning pathτ^\\hat\{\\tau\}\. Ifτ^\\hat\{\\tau\}contains an explicit final answer, we output it directly; otherwise,τ^\\hat\{\\tau\}is appended to the original prompt and used as context for generating a final answer\.

The posterior selection rule reduces the myopic bias of purely local accept\-reject strategies by weighting each child not only by how likely it is but by how good the paths it leads to tend to be\. A locally improbable step that consistently leads to correct conclusions will be favored over a locally probable step that leads to wrong ones\.

Algorithm 1Decision\-Flow Sample \(DF\-Sample\)1:Input:Question

qq, model

ℳ\\mathcal\{M\}, branching factor

KK, total depth

NN, block size

BB, temperature

α\\alpha
2:Output:Selected reasoning chain

τ^\\hat\{\\tau\}
3:Initialize context

c←qc\\leftarrow q\(question with prompt\)

4:Initialize final chain

τ^←∅\\hat\{\\tau\}\\leftarrow\\emptyset
5:

t←0t\\leftarrow 0
6:while

t<Nt<Ndo

7:

L←min⁡\(B,N−t\)L\\leftarrow\\min\(B,N\-t\)
8:Initialize reasoning tree

𝒯\\mathcal\{T\}with root node

v0v\_\{0\}\(using context

cc\)

9:// Phase 1: Hierarchical Reasoning Tree Construction

10:for

ℓ=1\\ell=1to

LLdo

11:foreach node

vvat depth

ℓ−1\\ell\-1do

12:Create leaf nodes

\{vℓ1,…,vℓK\}\\\{v\_\{\\ell\}^\{1\},\\dots,v\_\{\\ell\}^\{K\}\\\}
13:Sample

KKcandidate reasoning steps

\{svℓ1,…,svℓK\}\\\{s\_\{v\_\{\\ell\}^\{1\}\},\\ldots,s\_\{v\_\{\\ell\}^\{K\}\}\\\}from

ℳ\\mathcal\{M\}
14:Add leaves to

𝒯\\mathcal\{T\}
15:endfor

16:endfor

17:// Phase 2: Terminal Node Energy Evaluation

18:foreach leaf node

vLv\_\{L\}do

19:Compute terminal energy

E⁡\(vL\)E\(v\_\{L\}\)using Eq\. \(3\)

20:

U⁡\(vL\)←exp⁡\(−E⁡\(vL\)\)U\(v\_\{L\}\)\\leftarrow\\exp\(\-E\(v\_\{L\}\)\)
21:endfor

22:// Phase 3: Decision\-Flow Backward Propagation

23:for

ℓ=L−1\\ell=L\-1downto

00do

24:foreach node

vvat depth

ℓ\\elldo

25:

U⁡\(vℓ\)=∑ipprior​\(vℓ\+1i∣vℓ\)​U​\(vℓ\+1i\)U\(v\_\{\\ell\}\)=\\sum\_\{i\}p\_\{\\text\{prior\}\}\(v\_\{\\ell\+1\}^\{i\}\\mid v\_\{\\ell\}\)\\,U\(v\_\{\\ell\+1\}^\{i\}\)
26:endfor

27:endfor

28:// Phase 4: Posterior Path Selection

29:

τb←∅\\tau\_\{b\}\\leftarrow\\emptyset,

v←v0v\\leftarrow v\_\{0\}
30:while

ℓ<L\\ell<Ldo

31:Sample

vℓiv\_\{\\ell\}^\{i\}with probability

π∗​\(vℓi∣v\)∝pprior​\(vℓi∣v\)​U​\(vℓi\)\\pi^\{\*\}\(v\_\{\\ell\}^\{i\}\\mid v\)\\propto p\_\{\\text\{prior\}\}\(v\_\{\\ell\}^\{i\}\\mid v\)\\,U\(v\_\{\\ell\}^\{i\}\)
32:Append

svℓis\_\{v\_\{\\ell\}^\{i\}\}to

τb\\tau\_\{b\}
33:

v←vℓiv\\leftarrow v\_\{\\ell\}^\{i\}
34:endwhile

35:Append

τb\\tau\_\{b\}to

τ^\\hat\{\\tau\}
36:Update context

c←\(q,τ^\)c\\leftarrow\(q,\\hat\{\\tau\}\)
37:

t←t\+Lt\\leftarrow t\+L
38:endwhile

39:return

τ^\\hat\{\\tau\}

Table 1:Performance comparison of DF\-Sample and baselines on MATH500, HumanEval, GPQA\-Diamond, and AlpacaEval 2\.0\.Bolddenotes the best performance\. DF\-Sample outperforms all baselines in most settings\.
### 4\.5Efficient Inference via Block\-wise Sampling

The algorithmic pipeline involves two key design considerations\. First, before constructing the reasoning tree, the model estimates the required number of reasoning stepsNN, which determines the tree depth needed to produce a complete reasoning trajectory\. Second, asNNgrows, the number of candidate nodes expands exponentially, incurring substantial computational overhead\.

To mitigate this complexity, we introduce a block\-wise sampling strategy\. Specifically, whenNNexceeds a predefined thresholdBB, we first construct and evaluate the reasoning tree for the initial block of steps and apply Decision\-Flow selection to identify the best partial trajectory\. The selected partial reasoning path is then appended to the original problem as context for generating subsequent reasoning blocks\. This process repeats until the full reasoning depthNNis reached\. The Pseudocode of Decision\-Flow Sampling is shown inAlgorithm 1\.

## 5Experiments

### 5\.1Setup

Datasets\.We evaluate on four benchmarks spanning maths, codes, science, and general helpfulness:MATH500:500 competition\-level mathematics problems from the MATH dataset[Lightman et al\. \(2023\)](https://arxiv.org/html/2609.12317#bib.bib3), covering algebra, geometry, and number theory\.HumanEval:164 hand\-written programming tasks[Chen et al\. \(2021\)](https://arxiv.org/html/2609.12317#bib.bib2)\. Solutions are evaluated by executing unit tests and a problem is correct only if all tests pass\.GPQA\-Diamond:198 graduate\-level multiple\-choice questions in physics, chemistry, and biology[Rein et al\. \(2024\)](https://arxiv.org/html/2609.12317#bib.bib28), the most difficult split of GPQA\.AlpacaEval 2\.0:805 open\-ended instruction\-following prompts[Dubois et al\. \(2024\)](https://arxiv.org/html/2609.12317#bib.bib1)\. Responses are judged by GPT\-4\-Turbo and reported as a length\-normalized win rate\.

Models and Baselines\.We evaluate on three base model families: Qwen2\.5\-Math\-7B, Qwen2\.5\-7B, and Phi\-3\.5\-mini\-instruct\. We compare against four methods:Base model:Standard greedy decoding from the base model, with no sampling modification;Low\-temperature sampling:Exponentiated conditional next\-token distributions at each step[Wang et al\. \(2020\)](https://arxiv.org/html/2609.12317#bib.bib29), which sharpens the output distribution without constructing a tree;GRPO:Group Relative Policy Optimization[Shao et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib25), an RL\-based fine\-tuning method;Power sampling:improves reasoning by sampling many candidate reasoning segments from base model and selecting better replacement segment via MCMC[Karan and Du \(2025\)](https://arxiv.org/html/2609.12317#bib.bib31)\. All baseline results are taken directly from the corresponding papers[Karan and Du \(2025\)](https://arxiv.org/html/2609.12317#bib.bib31)\.

Implementations\.In our implementation, following prior work[Karan and Du \(2025\)](https://arxiv.org/html/2609.12317#bib.bib31), we setα=4\.0\\alpha=4\.0to encourage the model to produce outputs with higher confidence\. We useK=3K=3andB=3B=3in all experiments\. All other hyperparameters are kept consistent with the Power Sampling baseline\. The value ofRRis evaluated using GPT\-4o\. All experiments are run on 2 NVIDIA A6000 GPUs\.

### 5\.2Main Results

DF\-Sample consistently improves over the base model across all settings and outperforms the training\-free baseline power sampling in most configurations\. The gains are particularly striking on GPQA\-Diamond, where DF\-Sample with Qwen2\.5\-Math\-7B achieves 45\.6% compared to GRPO’s 39\.9%\. This is notable because GPQA\-Diamond problems require multi\-step scientific reasoning where globally\-correct paths are most likely to be locally counterintuitive — precisely the regime where global trajectory evaluation provides the largest advantage over step\-wise methods\. On MATH500, DF\-Sample achieves 81\.8% with Qwen2\.5\-Math\-7B, exceeding GRPO \(78\.5%\) by 3\.3 percentage points without any parameter updates\. On AlpacaEval 2\.0, DF\-Sample’s win rate improvements generalize beyond verifiable reasoning tasks, suggesting that the latent path hypothesis extends to general instruction following\.

Figure 6:The average log\-likelihood of Ours, power sampling, and GRPO responses over MATH500\.Figure 7:The average token confidence of Ours, power sampling, and GRPO responses over MATH500\.

### 5\.3Analysis

Token likelihood distribution\.Figure[7](https://arxiv.org/html/2609.12317#S5.F7)shows the histogram of length\-normalized sequence log\-likelihoods for DF\-Sample, GRPO, and power sampling on MATH500\. GRPO concentrates mass in a narrow high\-likelihood region, consistent with distribution sharpening\. DF\-Sample is biased toward higher likelihood but retains a broader spread than GRPO, indicating it accesses a wider region of the base model’s distribution while still favoring higher\-quality paths\.

Token\-level confidence\.Figure[7](https://arxiv.org/html/2609.12317#S5.F7)compares the distribution of token\-level confidence, defined as the average negative confidence of next\-token predictions\.

Conf\(x0:T\)=1T\+1∑t=0T∑x∈𝒳p\(x∣x<t\)logp\(x∣x<t\)\.\\mathrm\{Conf\}\(x\_\{0:T\}\)=\\frac\{1\}\{T\+1\}\\sum\_\{t=0\}^\{T\}\\sum\_\{x\\in\\mathcal\{X\}\}p\(x\\mid x\_\{<t\}\)\\log p\(x\\mid x\_\{<t\}\)\.

\(8\)GRPO responses concentrate near the highest\-confidence region, indicating that the generated tokens are typically produced in locally confident contexts under the base model\. Power sampling shows a somewhat broader distribution of confidence values\. In contrast, DF\-Sample shows the largest spread and extends furthest into low\-confidence regions\. This reveals that DF\-Sample is selecting paths through parts of the reasoning space where the model is locally uncertain but globally correct, precisely the latent paths the method is designed to recover\.

Figure 8:DF\-Sample \(Ours\) vs\. baselines on average generated tokens with Qwen2\.5\-Math\-7B on MATH500 dataset\.Average response lengths and latency\.Figure[8](https://arxiv.org/html/2609.12317#S5.F8)compares the average number of tokens generated for DF\-Sample, power sampling, and GRPO relative to the base model on the MATH500 dataset\. Power sampling produces the longest responses, followed by GRPO, indicating that these methods tend to generate longer reasoning trajectories\. In contrast, DF\-Sample generates responses with lengths of about 619 comparable to the base model of about 600\. This suggests that DF\-Sample tends to select relatively shorter reasoning trajectories while still preserving relatively correct answers\. On the MATH500 dataset with Qwen2\.5\-Math\-7B, power sampling requires about 340 seconds per question, while DF\-Sample takes around 384 seconds\. Despite the slightly longer inference time, DF\-Sample achieves 81\.8% accuracy, outperforming power sampling by 7\.0%\.

Figure 9:Accuracy comparison of DF\-Sample \(with differentKK\) and Power\-Sampling with Qwen2\.5\-Math\-7B on GPQA\-Diamond\.
### 5\.4Ablation Study

Impact ofKK\.Figure[9](https://arxiv.org/html/2609.12317#S5.F9)shows the performance of DF\-Sample on GPQA with Qwen2\.5\-Math\-7B under differentKK\. The model is more likely to find correct reasoning paths with largerKK, achieving higher accuracy\. For example, the accuracy is only about 1% higher than power sampling whenK=2K=2, while whenK=4K=4it improves by about 18%\. However, largerKKincreases inference latency, so we setK=3K=3to balance accuracy and efficiency\.

Figure 10:DF\-Sample \(Ours\) vs\. baselines on relative Pass@k accuracy to base model \(Qwen2\.5\-Math\- 7B\) on MATH500 dataset\.Table 2:Ablation study onα\\alphaon the GPQA\-Diamond dataset using Qwen2\.5\-Math\-7B\.Pass@k\.Figure[10](https://arxiv.org/html/2609.12317#S5.F10)compares the pass@k accuracy of DF\-Sample and baselines on the MATH500 dataset\. DF\-Sample achieves the highest pass@1 and pass@2 accuracy, with advantages particularly pronounced in the low\-k regime\. Askkincreases, the performance of the two methods becomes comparable, with power sampling slightly surpassing DF\-Sample at a few largerkkvalues\. Nevertheless, the differences remain small, indicating that DF\-Sample is highly effective at extracting correct reasoning paths, especially when only a small number of samples are available\.

Impact of temperature coefficientα\\alpha\.Table[2](https://arxiv.org/html/2609.12317#S5.T2)reports the ablation on the temperature coefficientα\\alphaon GPQA with Qwen2\.5\-Math\-7B\. Whenα\\alphais too large, sampling overly favors high\-probability reasoning paths and degrades accuracy\. Whenα\\alphais too small, the distribution becomes overly flat and lacks sufficient discrimination among candidate reasoning paths\. We observe the best performance whenα\>1\\alpha\>1with moderate values \(around44or55\)\.

## 6Related works

RL for LLM reasoning\.RL\-based approaches improve reasoning through post\-training\. Early work applies RLHF[Dong et al\. \(2024\)](https://arxiv.org/html/2609.12317#bib.bib19)to align outputs with human preferences, while more recent methods adopt reinforcement learning with verifiable rewards \(RLVR\) that directly optimize task correctness using automated verifiers[Lambert et al\. \(2024\)](https://arxiv.org/html/2609.12317#bib.bib11);[Guo et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib4);[Zeng et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib10)\. GRPO[Shao et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib25)is a representative RLVR method that improves reasoning via group\-relative policy updates\. Despite strong results, such approaches require expensive fine\-tuning and may reduce output diversity[Shao et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib25);[He et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib30)\. In contrast, DF\-Sample achieves similar improvements without parameter updates\.

Inference\-Time Reasoning Search\.Another line of work improves reasoning by allocating additional inference\-time computation\. Methods such as Tree of Thoughts[Yao et al\. \(2023\)](https://arxiv.org/html/2609.12317#bib.bib23), HyperTree[Gui et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib22), and ReST\-MCTS[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.12317#bib.bib21)explore tree\-structured reasoning search, while adaptive branching dynamically allocates compute during reasoning[Inoue et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib20)\. Power Sampling[Karan and Du \(2025\)](https://arxiv.org/html/2609.12317#bib.bib31)reshapes reasoning distributions via MCMC\-style sampling\. In contrast, DF\-Sample evaluates reasoning trajectories using global path\-level utilities\.

Sampling\-Based Methods\.Sampling\-based approaches such as MCMC and Generative Flow Networks \(GFlowNets\)[Bengio et al\. \(2021\)](https://arxiv.org/html/2609.12317#bib.bib7)reshape or approximate a model’s output distribution to favor high\-reward trajectories[Neal \(2001\)](https://arxiv.org/html/2609.12317#bib.bib9)\. For example, QUEST[Faria et al\. \(2024\)](https://arxiv.org/html/2609.12317#bib.bib8)applies a Metropolis–Hastings variant to iteratively resample model outputs, while annealed sampling[Karan and Du \(2025\)](https://arxiv.org/html/2609.12317#bib.bib31)sharpens the output distribution via temperature scheduling\. The Decision Flow framework[Chertkov et al\. \(2025\)](https://arxiv.org/html/2609.12317#bib.bib6)formulates trajectory sampling using a GFlowNet\-style approach with backward regression over sequence graphs\. Building on this idea, DF\-Sample applies decision\-flow\-style evaluation to reasoning trajectories, improving reasoning accuracy in LLMs\.

## 7Conclusion

In this paper, we proposeDecision\-Flow Sampling \(DF\-Sample\), a training\-free inference\-time sampling framework for improving reasoning performance in LLMs\. Unlike conventional decoding strategies that rely on local step\-wise probabilities, DF\-Sample constructs a hierarchical reasoning tree and performs global trajectory\-level evaluation through terminal\-node utility estimation and backward utility propagation\. By integrating generation probabilities with propagated utilities, our method effectively identifies high\-quality yet low\-probability reasoning paths that are often overlooked during standard decoding\. Extensive experiments across diverse reasoning benchmarks demonstrate that DF\-Sample consistently improves over base models and competitive sampling baselines, and achieves performance comparable to RL\-based approaches without any parameter updates\.

## Limitations

DF\-Sample operates purely at inference time and does not modify model parameters\. While this design enables training\-free improvements, it also raises the question of whether the reasoning patterns discovered during sampling could be distilled back into the model through training or fine\-tuning\. For example, future work could explore integrating DF\-Sample\-style trajectory selection with learning\-based approaches, allowing models to internalize effective reasoning strategies rather than relying solely on search during inference\.

## References

- Bengioet al\.\(2021\)E\. Bengio, M\. Jain, M\. Korablyov, D\. Precup, and Y\. BengioFlow network based generative models for non\-iterative diverse candidate generation\.Advances in neural information processing systems34,pp\. 27381–27394\.Cited by:[§6](https://arxiv.org/html/2609.12317#S6.p3.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. de Oliveira Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, B\. Chan, S\. Gray, N\. Ryder, M\. Pavlov, A\. Power, L\. Kaiser, M\. Bavarian, C\. Winter, P\. Tillet, F\. P\. Such, D\. Cummings, M\. Plappert, F\. Chantzis, E\. Barnes, A\. Herbert\-Voss, W\. H\. Guss, A\. Nichol, A\. Paino, N\. Tezak, J\. Tang, I\. Babuschkin, S\. Balaji, S\. Jain, W\. Saunders, C\. Hesse, A\. N\. Carr, J\. Leike, J\. Achiam, V\. Misra, E\. Morikawa, A\. Radford, M\. Knight, M\. Brundage, M\. Murati, K\. Mayer, P\. Welinder, B\. McGrew, D\. Amodei, S\. McCandlish, I\. Sutskever, and W\. ZarembaEvaluating large language models trained on code\.External Links:2107\.03374,[Link](https://arxiv.org/abs/2107.03374)Cited by:[§5\.1](https://arxiv.org/html/2609.12317#S5.SS1.p1.1)\.
- Chertkovet al\.\(2025\)M\. Chertkov, S\. Ahn, and H\. BehjooSampling decisions\.External Links:2503\.14549,[Link](https://arxiv.org/abs/2503.14549)Cited by:[§6](https://arxiv.org/html/2609.12317#S6.p3.1)\.
- Donget al\.\(2024\)H\. Dong, W\. Xiong, B\. Pang, H\. Wang, H\. Zhao, Y\. Zhou, N\. Jiang, D\. Sahoo, C\. Xiong, and T\. ZhangRlhf workflow: from reward modeling to online rlhf\.arXiv preprint arXiv:2405\.07863\.Cited by:[§6](https://arxiv.org/html/2609.12317#S6.p1.1)\.
- Dragoiet al\.\(2025\)M\. Dragoi, I\. Pintilie, F\. Gogianu, and F\. BradBeyond pass@ k: breadth\-depth metrics for reasoning boundaries\.arXiv preprint arXiv:2510\.08325\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p3.1)\.
- Duboiset al\.\(2024\)Y\. Dubois, B\. Galambosi, P\. Liang, and T\. B\. HashimotoLength\-controlled alpacaeval: a simple way to debias automatic evaluators\.arXiv preprint arXiv:2404\.04475\.Cited by:[§5\.1](https://arxiv.org/html/2609.12317#S5.SS1.p1.1)\.
- Fariaet al\.\(2024\)G\. Faria, S\. Agrawal, A\. Farinhas, R\. Rei, J\. de Souza, and A\. MartinsQUEST: quality\-aware metropolis\-hastings sampling for machine translation\.Advances in Neural Information Processing Systems37,pp\. 89042–89068\.Cited by:[§6](https://arxiv.org/html/2609.12317#S6.p3.1)\.
- Guiet al\.\(2025\)R\. Gui, Z\. Wang, J\. Wang, C\. Ma, H\. Zhen, M\. Yuan, J\. Hao, D\. Lian, E\. Chen, and F\. WuHyperTree planning: enhancing llm reasoning via hierarchical thinking\.External Links:2505\.02322,[Link](https://arxiv.org/abs/2505.02322)Cited by:[§6](https://arxiv.org/html/2609.12317#S6.p2.1)\.
- Guoet al\.\(2025\)D\. Guo, D\. Yang, H\. Zhang, J\. Song, R\. Zhang, R\. Xu, Q\. Zhu, S\. Ma, P\. Wang, X\. Bi,et al\.Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p1.1),[§6](https://arxiv.org/html/2609.12317#S6.p1.1)\.
- Haoet al\.\(2025\)Z\. Hao, H\. Wang, H\. Liu, J\. Luo, J\. Yu, H\. Dong, Q\. Lin, C\. Wang, and J\. ChenRethinking entropy interventions in rlvr: an entropy change perspective\.arXiv preprint arXiv:2510\.10150\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p2.1),[§2](https://arxiv.org/html/2609.12317#S2.p2.1)\.
- Heet al\.\(2025\)A\. W\. He, D\. Fried, and S\. WelleckRewarding the unlikely: lifting grpo beyond distribution sharpening\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 25559–25571\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p1.1),[§1](https://arxiv.org/html/2609.12317#S1.p2.1),[§6](https://arxiv.org/html/2609.12317#S6.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the math dataset\.arXiv preprint arXiv:2103\.03874\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p1.1)\.
- Huet al\.\(2025\)J\. Hu, Y\. Zhang, Q\. Han, D\. Jiang, X\. Zhang, and H\. ShumOpen\-reasoner\-zero: an open source approach to scaling up reinforcement learning on the base model\.arXiv preprint arXiv:2503\.24290\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p1.1)\.
- Inoueet al\.\(2025\)Y\. Inoue, K\. Misaki, Y\. Imajuku, S\. Kuroki, T\. Nakamura, and T\. AkibaWider or deeper? scaling llm inference\-time compute with adaptive branching tree search\.External Links:2503\.04412,[Link](https://arxiv.org/abs/2503.04412)Cited by:[§6](https://arxiv.org/html/2609.12317#S6.p2.1)\.
- Karan and Du \(2025\)A\. Karan and Y\. DuReasoning with sampling: your base model is smarter than you think\.arXiv preprint arXiv:2510\.14901\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p1.1),[§1](https://arxiv.org/html/2609.12317#S1.p3.1),[§2](https://arxiv.org/html/2609.12317#S2.p2.1),[§2](https://arxiv.org/html/2609.12317#S2.p4.1),[§5\.1](https://arxiv.org/html/2609.12317#S5.SS1.p2.1),[§5\.1](https://arxiv.org/html/2609.12317#S5.SS1.p3.1),[§6](https://arxiv.org/html/2609.12317#S6.p2.1),[§6](https://arxiv.org/html/2609.12317#S6.p3.1)\.
- Lambertet al\.\(2024\)N\. Lambert, J\. Morrison, V\. Pyatkin, S\. Huang, H\. Ivison, F\. Brahman, L\. J\. V\. Miranda, A\. Liu, N\. Dziri, S\. Lyu,et al\.Tulu 3: pushing frontiers in open language model post\-training\.arXiv preprint arXiv:2411\.15124\.Cited by:[§6](https://arxiv.org/html/2609.12317#S6.p1.1)\.
- Liet al\.\(2022\)Y\. Li, D\. Choi, J\. Chung, N\. Kushman, J\. Schrittwieser, R\. Leblond, T\. Eccles, J\. Keeling, F\. Gimeno, A\. Dal Lago,et al\.Competition\-level code generation with alphacode\.Science378\(6624\),pp\. 1092–1097\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p1.1)\.
- Lianget al\.\(2025\)X\. Liang, Z\. Li, Y\. Gong, Y\. Shen, Y\. N\. Wu, Z\. Guo, and W\. ChenBeyond pass@ 1: self\-play with variational problem synthesis sustains rlvr\.arXiv preprint arXiv:2508\.14029\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p2.1)\.
- Lightmanet al\.\(2023\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.External Links:2305\.20050,[Link](https://arxiv.org/abs/2305.20050)Cited by:[§5\.1](https://arxiv.org/html/2609.12317#S5.SS1.p1.1)\.
- Neal \(2001\)R\. M\. NealAnnealed importance sampling\.Statistics and computing11\(2\),pp\. 125–139\.Cited by:[§6](https://arxiv.org/html/2609.12317#S6.p3.1)\.
- Reinet al\.\(2024\)D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. BowmanGpqa: a graduate\-level google\-proof q&a benchmark\.InFirst Conference on Language Modeling,Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p1.1),[§5\.1](https://arxiv.org/html/2609.12317#S5.SS1.p1.1)\.
- Shaoet al\.\(2025\)R\. Shao, S\. S\. Li, R\. Xin, S\. Geng, Y\. Wang, S\. Oh, S\. S\. Du, N\. Lambert, S\. Min, R\. Krishna,et al\.Spurious rewards: rethinking training signals in rlvr\.arXiv preprint arXiv:2506\.10947\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p2.1),[§5\.1](https://arxiv.org/html/2609.12317#S5.SS1.p2.1),[§6](https://arxiv.org/html/2609.12317#S6.p1.1)\.
- Songet al\.\(2025\)Y\. Song, J\. Kempe, and R\. MunosOutcome\-based exploration for llm reasoning\.arXiv preprint arXiv:2509\.06941\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p2.1)\.
- Vafaiiet al\.\(2025\)H\. Vafaii, D\. Galor, and J\. L\. YatesBrain\-like variational inference\.ArXiv,pp\. arXiv–2410\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p3.1)\.
- Wanget al\.\(2020\)P\. Wang, S\. Hsieh, S\. Chang, Y\. Chen, J\. Pan, W\. Wei, and D\. JuanContextual temperature for language modeling\.External Links:2012\.13575,[Link](https://arxiv.org/abs/2012.13575)Cited by:[§5\.1](https://arxiv.org/html/2609.12317#S5.SS1.p2.1)\.
- Wuet al\.\(2025\)C\. Wu, B\. Li, M\. Gao, and Z\. WangFrom efficiency to adaptivity: a deeper look at adaptive reasoning in large language models\.External Links:2511\.10788,[Link](https://arxiv.org/abs/2511.10788)Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p3.1)\.
- Yaoet al\.\(2023\)S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. NarasimhanTree of thoughts: deliberate problem solving with large language models\.External Links:2305\.10601,[Link](https://arxiv.org/abs/2305.10601)Cited by:[§6](https://arxiv.org/html/2609.12317#S6.p2.1)\.
- Yueet al\.\(2025\)Y\. Yue, Z\. Chen, R\. Lu, A\. Zhao, Z\. Wang, S\. Song, and G\. HuangDoes reinforcement learning really incentivize reasoning capacity in llms beyond the base model?\.arXiv preprint arXiv:2504\.13837\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p2.1)\.
- Zenget al\.\(2025\)W\. Zeng, Y\. Huang, Q\. Liu, W\. Liu, K\. He, Z\. Ma, and J\. HeSimplerl\-zoo: investigating and taming zero reinforcement learning for open base models in the wild\.arXiv preprint arXiv:2503\.18892\.Cited by:[§6](https://arxiv.org/html/2609.12317#S6.p1.1)\.
- Zhanget al\.\(2024\)D\. Zhang, S\. Zhoubian, Z\. Hu, Y\. Yue, Y\. Dong, and J\. TangReST\-mcts\*: llm self\-training via process reward guided tree search\.External Links:2406\.03816,[Link](https://arxiv.org/abs/2406.03816)Cited by:[§6](https://arxiv.org/html/2609.12317#S6.p2.1)\.
- Zhanget al\.\(2025\)K\. Zhang, Y\. Zuo, B\. He, Y\. Sun, R\. Liu, C\. Jiang, Y\. Fan, K\. Tian, G\. Jia, P\. Li,et al\.A survey of reinforcement learning for large reasoning models\.arXiv preprint arXiv:2509\.08827\.Cited by:[§1](https://arxiv.org/html/2609.12317#S1.p2.1)\.