State of Thought Enables Endogenous Reasoning

# State of Thought Enables Endogenous Reasoning
Source: [https://arxiv.org/html/2609.16055](https://arxiv.org/html/2609.16055)
Zhiren Gong Yikun Hou Zihao Zeng Ming XiaoAffiliation:College of Computing and Data Science, Nanyang Technological University, SingaporeAffiliation:Interdisciplinary Graduate Programme, Nanyang Technological University, SingaporeAffiliation:Information Science & Engineering, KTH Royal Institute of Technology, Swedenzhiren001@e\.ntu\.edu\.sgzihao\.zeng@ntu\.edu\.sgbryan\.limwy@ntu\.edu\.sgChau Yuen Wei Yang Bryan LimAffiliation:College of Computing and Data Science, Nanyang Technological University, SingaporeAffiliation:School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore

###### Abstract

Test\-time compute has emerged as a major approach to improving the capabilities of Large Language Models \(LLMs\)\. However, existing test\-time reasoning paradigms rely heavily on externally imposed control, either through fixed reasoning programs or through costly expansion in constrained search spaces, limiting both generalization and efficiency\. We proposeState of Thought \(SoT\), a new reasoning paradigm that enablesendogenous reasoningin LLMs, with the model’s internal reasoning state governing how reasoning unfolds\. Concretely, SoT extracts a compact*dynamics\-geometric state*from the model’s internal information transfer and uses a 582\-parameter controller on frozen backbones to*selectively activate historical reasoning support*useful under the current reasoning state, framing reasoning as a state\-conditioned process over evidence rather than an externally prescribed token chain\. Across quantitative \(1\.34×1\.34\\times\), general \(1\.62×1\.62\\times\), symbolic\-and\-code \(1\.76×1\.76\\times\), and long\-context \(2\.51×2\.51\\times\) reasoning on 3 LLMs and 16 datasets, SoT consistently improves mean\-baseline accuracy while reducing generated tokens by62\.6%62\.6\\%and end\-to\-end latency by44\.6%44\.6\\%\. Across 2 VLM scales and 3 reasoning tasks, it improves mean accuracy by 3\.8 points over reasoning baselines, with74\.9%74\.9\\%fewer completion tokens and73\.5%73\.5\\%lower latency than search\-based methods\. Under constrained access, SoT retains38\.2%38\.2\\%/36\.5%36\.5\\%mean accuracy gains in training\-free/embedding\-only settings, while trajectory\-only judging reaches84\.1%84\.1\\%agreement across 3 API models\. Together, endogenous state\-driven reasoning provides a generalizable and efficient alternative\.

![[Uncaptioned image]](https://arxiv.org/html/2609.16055v1/intro_overview.png)

Figure 1:SoT Performance Across Models, Tasks, and Efficiency Metrics\.
## 1Introduction

![Refer to caption](https://arxiv.org/html/2609.16055v1/intro_main.png)Figure 2:From External Control \(Top\) to State\-of\-Thought \(Bottom\)\.Reasoning in Large Language Models \(LLMs\) is fundamentally the process of organizing historical evidence context for subsequent decisions\([Press et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib3);[Trivedi et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib2);[Yao et al\., 2023b](https://arxiv.org/html/2609.16055#bib.bib1)\)\. From this perspective, progress in test\-time compute comes from better strategies for organizing the evidence context with additional computation, thereby improving reasoning capabilities\([Brown et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib10);[Snell et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib9)\)\. One dominant line relies on externally structured reasoning programs, requiring the model to follow pre\-specified reasoning formats such as step\-by\-step\([Wei et al\., 2022](https://arxiv.org/html/2609.16055#bib.bib4)\), planning\([Wang et al\., 2023a](https://arxiv.org/html/2609.16055#bib.bib6)\), or revision\([Madaan et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib7)\)\. Another line allocates more test\-time search, exploring additional samples\([Brown et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib10);[Wang et al\., 2023b](https://arxiv.org/html/2609.16055#bib.bib5)\), branches\([Yao et al\., 2023a](https://arxiv.org/html/2609.16055#bib.bib8)\), or trajectories\([Snell et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib9)\)to accumulate broader candidate evidence for subsequent reasoning\. Despite their empirical success in adapting to particular reasoning strategies, both lines impose reasoning control from outside the model \(i\.e\., through pre\-specified formats or search policies\) and ignore its evolving internal state as a signal for what evidence is actually needed at subsequent decisions, limiting generalization across heterogeneous reasoning structures\([Hassid et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib12);[Gong et al\., 2026a](https://arxiv.org/html/2609.16055#bib.bib68)\)and efficiency under constrained inference\([Zhou et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib11)\)\.

Prior work has shown that LLMs internals exhibit structured representation geometry\([Zou et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib28)\), trajectory\-level regularities\([Sun et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib13)\), and confidence dynamics\([Tikhonov et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib14)\)\. However, these signals are typically used descriptively, such as monitoring and reasoning\-trajectory interpretation\([Ballon et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib19);[Damirchi et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib20);[Feng et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib18);[Gong et al\., 2026b](https://arxiv.org/html/2609.16055#bib.bib69)\), or early stopping criteria\([Hosseini et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib21)\), rather than to govern reasoning itself\. Together, these observations point to the significance of the model’s*endogenous state*\(i\.e\., the internally evolving state induced by its own reasoning trajectory\)\. In parallel, reasoning also depends critically on whether decisions are steadily grounded in suitable and decision\-relevant historical evidence rather than generic or noisy history\([Creswell et al\., 2022](https://arxiv.org/html/2609.16055#bib.bib22);[Press et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib3);[Wang et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib23)\)\. Existing approaches address this through context retrieval and compression\([Jiang et al\., 2024b](https://arxiv.org/html/2609.16055#bib.bib17);[Li et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib16);[Packer et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib15)\), or latent\-space reasoning\([Hao et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib24);[Wang et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib25)\), both of which bypass endogenous state as a vital signal for organizing the evidence genuinely needed for subsequent steps\. Appendix[R](https://arxiv.org/html/2609.16055#A18)provides further related discussion\.

This motivates a reframing of reasoning itself:*effective reasoning is actually a dynamic process of organizing the right evidence context for the next decision under the current endogenous state, rather than following an externally prescribed token chain\.*We introduceState of Thought \(SoT\), a reasoning paradigm for*endogenous reasoning*\. At each reasoning step, SoT reads out a compact dynamics\-geometric state from the model’s internal information transfer and uses a lightweight 582\-parameter controller to selectively activate the historical evidence context most useful for the next decision\. The active reasoning context is therefore not generated by pre\-specified reasoning programs or broader searches, but by the model’s own evolving state\. In this way, SoT turns reasoning into a closed loop: endogenous state organizes active evidence, which conditions the next reasoning step, and the new step in turn updates the state\.

We evaluate SoT at scale on 3 LLM backbones over 16 datasets and 2 VLM scales over 3 reasoning tasks\. Across the 16\-dataset LLM suite, SoT improves accuracy by89\.8%\{89\.8\}\\%,58\.6%\{58\.6\}\\%, and46\.0%\{46\.0\}\\%over the mean baselines on Llama\-3\.1\-8B, Qwen2\.5\-14B, and Mixtral\-8x7B, while reducing tokens/latency by19\.7%\{19\.7\}\\%/56\.5%\{56\.5\}\\%,81\.1%\{81\.1\}\\%/62\.2%\{62\.2\}\\%, and86\.8%\{86\.8\}\\%/15\.1%\{15\.1\}\\%, respectively\. Across Qwen2\.5\-VL\-7B/32B and 3 reasoning tasks, SoT improves mean accuracy by 3\.8 points over reasoning baselines, leads in 5 of 6 model–task settings, and uses74\.9%74\.9\\%fewer completion tokens with73\.5%73\.5\\%lower latency than search\-based methods\. As exploratory extensions under constrained access, a training\-free controller and an embedding\-only controller still achieve38\.2%\{38\.2\\%\}and36\.5%\{36\.5\\%\}improvement over baselines on accuracy across models, while a trajectory\-level post\-hoc judge reaches84\.1%84\.1\\%agreement on three API\-accessed models\. Together, these results suggest that changing the control variable of reasoning—from external token programs to endogenous state\-conditioned evidence organization—offers a scalable and robust route to reasoning improvement\.

## 2State of Thought

### 2\.1Problem Formulation

We formulate test\-time reasoning of a frozen autoregressive LLMffas a policy\-governed multi\-step trajectory generation started from a problemxxpresented by promptpp\([Brown et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib10)\)\. The modelffdynamically generates the subsequent reasoning trajectoryy1:T=\(y1,…,yT\)y\_\{1:T\}=\(y\_\{1\},\\dots,y\_\{T\}\)leading to the final answeraa:

ℙπ\(a,y1:T∣p\)=∏t=1Tℙπ\(yt∣p,y<t\)ℙπ\(a∣p,y1:T\),\\mathbb\{P\}\_\{\\pi\}\(a,y\_\{1:T\}\\mid p\)=\\prod\_\{t=1\}^\{T\}\\mathbb\{P\}\_\{\\pi\}\(y\_\{t\}\\mid p,y\_\{<t\}\)\\;\\mathbb\{P\}\_\{\\pi\}\(a\\mid p,y\_\{1:T\}\),\(1\)with the policyπ\\pidetermining how evidence from prior steps is organized for subsequent decisions, while different reasoning paradigms differ in prompt construction and policy\([Snell et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib9)\)\.

State of Thought seeks such a policy that the model’s current endogenous state \(Sec\.[2\.2](https://arxiv.org/html/2609.16055#S2.SS2)\) governs both evidence\-context organization and reasoning continuation \(Sec\.[2\.3](https://arxiv.org/html/2609.16055#S2.SS3)\), with no external reasoning instructions imposed on the prompt\. This creates a direct bridge between*continuous*internal evolution and*discrete*, inspectable reasoning steps: the continuous state controls which explicit evidence is carried forward, without replacing language\-space reasoning by an opaque latent chain\.

![Refer to caption](https://arxiv.org/html/2609.16055v1/state_space_regimes.png)Figure 3:Endogenous reasoning state trajectories across various reasoning paradigmson Llama\-3\.1\-8B with breakdown in Figure[8](https://arxiv.org/html/2609.16055#A15.F8)and reasoning state definition in Figure[4](https://arxiv.org/html/2609.16055#S2.F4)\. Different externally induced reasoning strategies occupy distinguishable regions in the dynamics\-geometric state space, while SoT supports a broader and more adaptive state distribution\.
### 2\.2Endogenous Reasoning State

Effective reasoning policyπ\\pishould be governed not by external scripts but by a compact endogenous state that reflects the geometry and dynamics of ongoing reasoning\([Park et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib27);[Park et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib26);[Zou et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib28)\)\. Accordingly, we read out a compact*dynamics\-geometric state*mt∈ℝ4m\_\{t\}\\in\\mathbb\{R\}^\{4\}from the model’s internal information transfer of sentence\-level reasoning unityty\_\{t\}, with calculation details and design rationale in Appendix[B](https://arxiv.org/html/2609.16055#A2):

mt=\[δt,vt,ct,Ht\],m\_\{t\}=\\big\[\\delta\_\{t\},\\;v\_\{t\},\\;c\_\{t\},\\;H\_\{t\}\\big\],\(2\)whereδt\\delta\_\{t\}characterizes the internal organization distinguishing concentrated from diffuse local structure;vtv\_\{t\}captures stepwise progress through magnitude;ctc\_\{t\}captures directional consistency distinguishing stable continuation from deviation or redirection; andHtH\_\{t\}measures the model’s local predictive uncertainty\. Together, these four quantities summarize the current reasoning endogenous control state and condition how the evidence context should be organized for subsequent decisions\.

#### Interpretability of endogenous reasoning state\.

To verify thatmtm\_\{t\}can capture reasoning\-relevant internal structure rather than arbitrary hidden variation, we aggregate trajectories from multiple external reasoning paradigms in the constructed state space in Figure[3](https://arxiv.org/html/2609.16055#S2.F3)\. SoT spans a broader and more adaptive state distribution, while others with externally structured reasoning programs occupy distinguishable regions, collapsing to fixed scripted patterns\. This indicates that SoT is less constrained by externally imposed reasoning formats and more aligned with the model’s endogenous state evolution, enabling stronger generalization across heterogeneous reasoning structures\.

### 2\.3State\-Conditioned Reasoning

SoT instantiates the reasoning policyπ\\pithrough evidence\-organization operator𝒮\\mathcal\{S\}and stopping operator𝒯\\mathcal\{T\}as two state\-conditioned operators with details in Appendix[A](https://arxiv.org/html/2609.16055#A1)\.

*State\-conditioned evidence selection\.*Given the current statemtm\_\{t\}and the historical reasoning trajectoryy<ty\_\{<t\}, the evidence\-organization operator𝒮\\mathcal\{S\}selects the reasoning\-related sparse subsety~<t=𝒮⁡\(y<t,mt\)⊂y<t\\widetilde\{y\}\_\{<t\}=\\mathcal\{S\}\(y\_\{<t\};m\_\{t\}\)\\subset y\_\{<t\}as the evidence context for subsequent reasoning to generate the next step

ℙπ​\(yt∣p,y<t\)=ℙ⁡\(yt∣p,y~<t\),\\mathbb\{P\}\_\{\\pi\}\(y\_\{t\}\\mid p,y\_\{<t\}\)=\\mathbb\{P\}\\\!\\left\(y\_\{t\}\\mid p,\\widetilde\{y\}\_\{<t\}\\right\),\(3\)In this way, SoT improves reasoning not by broadening search over more trajectories\([Yao et al\., 2023a](https://arxiv.org/html/2609.16055#bib.bib8)\), but by sharpening the evidence context carried into the next decision\.

![Refer to caption](https://arxiv.org/html/2609.16055v1/state_conditioned_activation.png)Figure 4:State\-conditioned reasoning in SoTwith breakdown in Figure[9](https://arxiv.org/html/2609.16055#A15.F9)\. Different endogenous reasoning states induce different sparse activation patterns and different stop tendencies\.*State\-conditioned stopping\.*SoT also produces a continuation decisionzt=𝒯⁡\(mt\)∈\{0,1\}z\_\{t\}=\\mathcal\{T\}\(m\_\{t\}\)\\in\\\{0,1\\\}, wherezt=0z\_\{t\}=0continues reasoning andzt=1z\_\{t\}=1terminates reasoning and triggers final answer generation as

ℙπ\(a∣p,y1:t\)=ℙ\(a∣p,y~<t,yt\)\.\\mathbb\{P\}\_\{\\pi\}\(a\\mid p,y\_\{1:t\}\)=\\mathbb\{P\}\(a\\mid p,\\widetilde\{y\}\_\{<t\},y\_\{t\}\)\.\(4\)Stopping therefore emerges from the current reasoning endogenous state rather than from a fixed external schedule\([Hosseini et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib21)\)on reasoning depth\.

Overall, SoT treats evidence sparsification and stopping as two outputs of the same endogenous reasoning policy: one determines what evidence context remains active, and the other determines when sufficient support has been accumulated for answer generation\.

#### Interpretability of state\-conditioned reasoning\.

We aggregate reasoning trajectories, cluster endogenous states, and analyze the induced evidence\-selection and stopping patterns in each state regime in Figure[4](https://arxiv.org/html/2609.16055#S2.F4)\. We find that different state clusters consistently and stably correspond to distinct sparse activation profiles and stop probabilities, indicating that SoT organizes evidence context and continuation behavior in a state\-dependent manner, supporting the core view that effective reasoning is governed by endogenous states rather than externally prescribed token chains\. Cluster names summarize four\-coordinate centroids and measured activation–stopping profiles, not task labels; Appendix[O\.2](https://arxiv.org/html/2609.16055#A15.SS2)details their interpretation\.

#### Lightweight training\.

SoT learns only the state\-conditioned evidence\-organization operator𝒮\\mathcal\{S\}and stopping operator𝒯\\mathcal\{T\}, while the backbone remains frozen, with training details in Appendix[C](https://arxiv.org/html/2609.16055#A3); Sec\.[4\.1](https://arxiv.org/html/2609.16055#S4.SS1)further removes this controller training\. Supervision is constructed offline from multi\-step reasoning trajectories: each endogenous state is paired with \(i\) preference\-style signals indicating which historical evidence remains useful for subsequent reasoning, in the spirit of direct preference learning\([Hong et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib30);[Rafailov et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib29)\), and \(ii\) stop proxies indicating whether reasoning should continue, following findings that intermediate reasoning states can support learned early stopping\([Liu and Wang, 2025](https://arxiv.org/html/2609.16055#bib.bib31)\)\. This design is also consistent with context\-selection and compression methods that learn lightweight selection modules from offline trajectories or contextual supervision rather than retraining the backbone model\([Jiang et al\., 2024b](https://arxiv.org/html/2609.16055#bib.bib17);[Li et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib16)\)\. The resulting controller has only 582 parameters \(a×3216\\\!\\times\\\!32selection projection with a scalar output, plus a linear→14\\\!\\to\\\!1stopping head\); across 5 problem\-held\-out refits, its selector AUC is0\.779±0\.0050\.779\\pm 0\.005\. Thus, SoT learns a compact control interface rather than updating or duplicating backbone reasoning capacity\.

Table 1:Main results on Llama\-3\.1\-8B \(accuracy, %\)\.CategoryMethodQuantitative ReasoningSymbolic and CodeGSMMATHDROPavg\.FOLPWBBHHEMBPPavg\.GreedyVanilla79\.058\.26\.253\.929\.626\.026\.025\.055\.233\.0Reasoning ParadigmCoT80\.864\.21\.755\.54\.924\.570\.025\.051\.631\.2PS71\.240\.01\.443\.44\.915\.255\.010\.022\.017\.6SR75\.844\.815\.650\.432\.531\.046\.025\.036\.432\.9SC83\.654\.01\.653\.216\.828\.552\.010\.035\.227\.3CB48\.036\.519\.837\.140\.435\.834\.030\.049\.238\.6MCTS55\.233\.513\.237\.536\.031\.530\.035\.049\.236\.7Memory\-OrientedH2O79\.859\.22\.453\.66\.427\.848\.010\.042\.026\.3SNAP80\.460\.22\.354\.18\.426\.255\.015\.041\.227\.3STREAM70\.444\.21\.344\.41\.531\.81\.00\.05\.613\.0LatentCOCO81\.460\.03\.754\.845\.345\.251\.020\.047\.242\.5RL\-BasedGRPO\-SP0\.00\.50\.30\.22\.02\.27\.00\.00\.01\.8OursSoT81\.864\.840\.565\.849\.864\.878\.059\.146\.858\.4
CategoryMethodGeneral UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.GreedyVanilla48\.068\.859\.245\.650\.754\.27\.020\.132\.716\.2Reasoning ParadigmCoT32\.831\.537\.234\.050\.336\.32\.04\.526\.97\.1PS30\.518\.221\.531\.242\.028\.11\.73\.020\.45\.3SR57\.064\.072\.262\.462\.763\.617\.921\.725\.520\.6SC28\.534\.543\.231\.045\.735\.81\.94\.325\.86\.8CB46\.071\.582\.251\.256\.061\.115\.527\.526\.221\.8MCTS42\.065\.582\.246\.449\.757\.013\.527\.127\.821\.0Memory\-OrientedH2O33\.530\.839\.235\.019\.732\.42\.02\.00\.01\.7SNAP33\.531\.038\.534\.616\.731\.82\.02\.00\.01\.7STREAM34\.826\.238\.822\.40\.025\.62\.50\.00\.01\.1LatentCOCO40\.563\.570\.042\.843\.352\.04\.410\.834\.611\.9RL\-BasedGRPO\-SP10\.53\.89\.88\.212\.08\.70\.61\.15\.31\.6OursSoT69\.568\.882\.365\.067\.070\.424\.939\.034\.731\.9
Note\.Other model results are reported in Table[12](https://arxiv.org/html/2609.16055#A16.T12),[13](https://arxiv.org/html/2609.16055#A16.T13)\. Cells colored with rankings as1st,2nd, and3rd\.

## 3Experiments

### 3\.1Experimental Setup

#### Backbone models\.

We evaluate SoT on three frozen*LLMs*with different scales and architectures \(Llama\-3\.1\-8B\([Grattafiori et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib45)\), Qwen2\.5\-14B\([Yang et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib32)\), and Mixtral\-8x7B\([Jiang et al\., 2024a](https://arxiv.org/html/2609.16055#bib.bib33)\)\) and transfer to two frozen VLM scales, Qwen2\.5\-VL\-7B and Qwen2\.5\-VL\-32B\([Bai et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib67)\)\. Full details are deferred to Appendix[L](https://arxiv.org/html/2609.16055#A12)\.

#### Tasks and evaluation domains\.

For*LLMs*, we organize evaluation into*four reasoning domains*comprising*16 benchmarks*\.Quantitative Reasoning\(QS\) includes GSM8K \(GSM\)\([Cobbe et al\., 2021](https://arxiv.org/html/2609.16055#bib.bib40)\), MATH\([Hendrycks et al\., 2021b](https://arxiv.org/html/2609.16055#bib.bib41)\), and DROP\([Dua et al\., 2019](https://arxiv.org/html/2609.16055#bib.bib54)\)\.General Understanding\(GU\) includes CommonsenseQA \(CSQA\)\([Talmor et al\., 2019](https://arxiv.org/html/2609.16055#bib.bib43)\), StrategyQA \(StrQA\)\([Geva et al\., 2021](https://arxiv.org/html/2609.16055#bib.bib42)\), BoolQ\([Clark et al\., 2019](https://arxiv.org/html/2609.16055#bib.bib46)\), MMLU\([Hendrycks et al\., 2021a](https://arxiv.org/html/2609.16055#bib.bib47)\), and RACE\([Lai et al\., 2017](https://arxiv.org/html/2609.16055#bib.bib48)\)\.Symbolic and Code\(S&C\) includes FOLIO \(FOL\)\([Han et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib49)\), ProofWriter \(PW\)\([Tafjord et al\., 2021](https://arxiv.org/html/2609.16055#bib.bib50)\), BBH\-Temporal \(BBH\)\([Srivastava et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib44)\), HumanEval \(HE\)\([Chen et al\., 2021](https://arxiv.org/html/2609.16055#bib.bib51)\), and MBPP\([Austin et al\., 2021](https://arxiv.org/html/2609.16055#bib.bib52)\)\.Long\-Context Reasoning\(LCR\) includes HotpotQA \(HQA\)\([Yang et al\., 2018](https://arxiv.org/html/2609.16055#bib.bib53)\), NarrativeQA \(NarQA\)\([Kočiskỳ et al\., 2018](https://arxiv.org/html/2609.16055#bib.bib55)\), and LongBench MultiFieldQA \(LB\)\([Bai et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib56)\)\. For*VLM transfer*, we evaluate*three reasoning\-oriented benchmarks*that require knowledge\-grounded, diagrammatic, or multimodal chain\-of\-thought inference: A\-OKVQA\([Schwenk et al\., 2022](https://arxiv.org/html/2609.16055#bib.bib64)\), AI2D\([Kembhavi et al\., 2016](https://arxiv.org/html/2609.16055#bib.bib65)\), and M3CoT\([Chen et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib66)\)\. Full details are deferred to Appendix[K](https://arxiv.org/html/2609.16055#A11)\.

#### Baselines\.

We compare SoT against*5 baseline families*\.Greedy:Vanilla\.Reasoning paradigm:Chain\-of\-Thought \(CoT\)\([Wei et al\., 2022](https://arxiv.org/html/2609.16055#bib.bib4)\), Plan\-and\-Solve \(PS\)\([Wang et al\., 2023a](https://arxiv.org/html/2609.16055#bib.bib6)\), Self\-Refine \(SR\)\([Madaan et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib7)\)with externally scripted programs, and Self\-Consistency \(SC\)\([Wang et al\., 2023b](https://arxiv.org/html/2609.16055#bib.bib5)\), Best\-of\-N \(BoN\)\([Snell et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib9)\), Constrained Beam \(CB\)\([Hokamp and Liu, 2017](https://arxiv.org/html/2609.16055#bib.bib38)\), Monte Carlo Tree Search \(MCTS\)\([Xie et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib37)\)with sampling or search methods\.Memory\-oriented methods:H2O \(H2O\)\([Zhang et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib35)\), SnapKV \(SNAP\)\([Li et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib34)\), and StreamingLLM \(STREAM\)\([Xiao et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib36)\)\.Latent reasoning:COCONUT \(COCO\)\([Hao et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib24)\)\.RL\-based reasoning:GRPO\-inspired soft prompt \(GRPO\-SP\)\([Shao et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib39)\)\. All learned components are fitted exclusively on training\-side data, with evaluation examples excluded\. Full details are deferred to Appendix[M](https://arxiv.org/html/2609.16055#A13)and[N](https://arxiv.org/html/2609.16055#A14)\.

Figure 5:Accuracy–efficiency trade\-offon Llama\-3\.1\-8B with breakdown and other models in Figure[11](https://arxiv.org/html/2609.16055#A16.F11)\.
#### Evaluation metrics\.

We report task\-standard accuracy metrics and efficiency cost\.Exact Match \(EM\)is used for GSM8K, MATH, RACE, MMLU, CommonsenseQA, FOL, BBH, ProofWriter, BoolQ, StrategyQA, A\-OKVQA, AI2D, and M3CoT\.F1is used for HotpotQA, DROP, NarrativeQA, and LongBench MultiFieldQA\.Pass@1is used for HumanEval and MBPP\.Efficiencyis measured by generated tokens and end\-to\-end inference latency; hardware and timing details are provided in Appendix[L](https://arxiv.org/html/2609.16055#A12)\.

CategoryMethodQSS&CGULCRTok\.Lat\.Tok\.Lat\.Tok\.Lat\.Tok\.Lat\.ReasoningParadigmCoT271\.617\.5369\.827\.5212\.912\.8198\.119\.6PS314\.220\.7372\.126\.8319\.822\.2282\.333\.7SR368\.525\.9391\.728\.8292\.219\.2278\.833\.2SC736\.843\.1953\.963\.5626\.537\.6533\.848\.9CB423\.922\.0363\.919\.4341\.716\.8283\.719\.5MCTS422\.121\.4364\.719\.4335\.216\.5286\.620\.5Memory\-OrientedH2O226\.016\.7264\.434\.6185\.615\.8164\.621\.5SNAP226\.016\.6264\.434\.8185\.616\.1164\.620\.9STREAM92\.612\.564\.224\.093\.49\.8103\.110\.3LatentCOCO171\.511\.5214\.620\.487\.26\.7131\.530\.3RL\-BasedGRPO\-SP73\.17\.569\.77\.171\.47\.576\.226\.9OursSoT255\.75\.8302\.29\.574\.91\.8263\.022\.0Table 2:Efficiency comparison on Llama\-3\.1\-8B\.
Figure 6:Distribution\.Note\.Generated reasoning tokens \(Tok\.\) and end\-to\-end latency \(Lat\.\) are aggregated over four reasoning domains, with full breakdowns in Tables[14](https://arxiv.org/html/2609.16055#A16.T14)and[15](https://arxiv.org/html/2609.16055#A16.T15)\. Results for the other models appear in Tables[16](https://arxiv.org/html/2609.16055#A16.T16),[17](https://arxiv.org/html/2609.16055#A16.T17),[18](https://arxiv.org/html/2609.16055#A16.T18), and[19](https://arxiv.org/html/2609.16055#A16.T19)\.

### 3\.2Main Results

#### Overall performance\.

Table[1](https://arxiv.org/html/2609.16055#S2.T1)reports the main results on Llama\-3\.1\-8B across four reasoning domains\. SoT achieves the best domain\-level average in all domains\. Compared with the strongest non\-SoT baseline in each domain, it improves Quantitative Reasoning, Symbolic and Code, General Understanding, and Long\-Context Reasoning by\+10\.3/\+15\.9/\+6\.8/\+10\.1\{\+10\.3/\+15\.9/\+6\.8/\+10\.1\}points, corresponding to relative gains of18\.6%/37\.4%/10\.7%/46\.3%\{18\.6\\%/37\.4\\%/10\.7\\%/46\.3\\%\}\. It also obtains the best or tied\-best result on 13 out of 16 datasets, with an average domain\-level gain of10\.8\{10\.8\}points over the strongest competing methods\. This pattern suggests that reasoning gains need*not*come primarily from prescribing a stronger external reasoning program or from scaling search over more trajectories\. A closed\-loop controller driven by the model’s own endogenous reasoning state can itself serve as an effective mechanism for improving test\-time reasoning\. We further demonstrate the stable performance gains of our method through quality sensitivity experiments on the training dataset in the Appendix[P\.5](https://arxiv.org/html/2609.16055#A16.SS5)\.

#### Generalization\.

A distinctive property of SoT is that the same controller can be applied across heterogeneous reasoning structures with stable performance\. Compared with scripted reasoning, SoT avoids committing to a fixed external reasoning form; compared with sampling and search, it improves reasoning without relying mainly on broader trajectory expansion; compared with memory\-oriented and latent reasoning baselines, it organizes historical evidence through the model’s evolving endogenous state rather than generic context heuristics or hidden\-space\-only reasoning\. Consistent gains across these regimes therefore indicate*mechanism\-level generalization*: SoT transfers as a reasoning principle, not merely as a benchmark\-specific strategy\.

In general, these patterns suggest a more structural interpretation of the results\. Within the family of externally prescribed token\-chain reasoning methods, performance gains are typically obtained by strengthening scripts, increasing samples, or enlarging search\. By contrast, SoT improves reasoning by changing the control variable itself, shifting from external token programs to endogenous state\-conditioned evidence organization\. This may explain why it can surpass strong token\-chain baselines across heterogeneous regimes\. Appendix[F](https://arxiv.org/html/2609.16055#A6)to[J](https://arxiv.org/html/2609.16055#A10)provide in\-depth theoretical analysis\.

### 3\.3Efficiency and Trade\-off

Efficiency is central to inference\-time reasoning because most performance gains come from expanding computation through longer trajectories, repeated sampling, or explicit search\. Instead, SoT activates only the evidence context most suitable under the current endogenous state, so its advantage should appear not only in accuracy but also in efficiency\.

#### Efficiency\.

Table[2](https://arxiv.org/html/2609.16055#S3.T2)and Figure[6](https://arxiv.org/html/2609.16055#S3.F6)report generated reasoning tokens and end\-to\-end latency on Llama\-3\.1\-8B\. Averaged over the four reasoning domains, SoT uses223\.9\{223\.9\}tokens and9\.89\.8s latency, compared with472\.7472\.7tokens and29\.029\.0s latency for sampling/search methods, reducing tokens by52\.6%\{52\.6\\%\}and latency by66\.4%66\.4\\%\. The saving is especially clear in QS, S&C, and GU, where SoT attains strong accuracy with only5\.85\.8s,9\.59\.5s, and1\.81\.8s latency, respectively\. Compared with memory\-oriented methods, however, SoT is not merely a more aggressive compression rule: generic pruning can reduce tokens, but often sacrifices accuracy because it does not condition retained evidence on the current reasoning state\. Thus, SoT improves efficiency by removing unnecessary computation while preserving the support needed for the next decision\.

#### Accuracy–efficiency trade\-off\.

Figure[5](https://arxiv.org/html/2609.16055#S3.F5)further shows that SoT moves the accuracy–efficiency frontier rather than simply trading accuracy for shorter outputs\. It reaches the highest accuracy while using far fewer tokens and lower latency than external strategies and sampling/search baselines\. This pattern clarifies the core advantage of SoT: test\-time compute is no longer spent on uniformly longer chains, larger candidate sets, or generic history retention, but is redirected toward state\-conditioned evidence organization\. The efficiency results therefore reinforce the main claim that endogenous reasoning state provides a better control signal for both reasoning quality and inference cost\.

Table 3:Ablation results on Llama\-3\.1\-8B,with other models provided in Tables[20](https://arxiv.org/html/2609.16055#A16.T20),[21](https://arxiv.org/html/2609.16055#A16.T21)\.VariantQSS&CGULCRacc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.Full SoT62\.8219\.45\.854\.5294\.09\.571\.264\.01\.833\.0181\.622\.0\+ Threshold Tuning62\.8219\.45\.854\.3294\.09\.571\.264\.01\.831\.8229\.022\.0w/o Evid\. Org\.𝒮\\mathcal\{S\}29\.6469\.512\.944\.5536\.216\.572\.4193\.94\.926\.9506\.514\.1w/o Stopping𝒯\\mathcal\{T\}60\.61769\.680\.757\.11705\.1110\.370\.24148\.599\.531\.62937\.4157\.3w/o Geometryδt\\delta\_\{t\}42\.568\.63\.346\.448\.02\.973\.025\.40\.924\.547\.22\.2w/o Dynamics\(vt,ct\)\(v\_\{t\},c\_\{t\}\)57\.01039\.229\.146\.2846\.725\.771\.21272\.333\.731\.61000\.330\.8w/o UncertaintyHtH\_\{t\}54\.1244\.96\.452\.1281\.88\.873\.661\.31\.932\.8555\.516\.6w/o Sent\.\-level Units54\.1244\.97\.252\.1281\.88\.873\.661\.31\.935\.1224\.111\.9

### 3\.4Ablation Studies

The ablations in Table[3](https://arxiv.org/html/2609.16055#S3.T3)indicate that the components of SoT interact in a way that controls distinct reasoning failure modes rather than acting as independent optimizations\. For instance, removing the evidence\-organization operatorSSleads to a significant drop in performance, especially in QS \(from 62\.8 to 29\.6\) and LCR \(from 33\.0 to 26\.9\), highlighting the critical role of state\-conditioned evidence selection\. Conversely, removing the stopping operatorTTimproves S&C \(from 54\.5 to 57\.1\), but this comes at the expense of drastically increased tokens \(from 189\.8 to 2640\.2\) and latency \(from 9\.8s to 112\.0s\), making the improvement unsustainable due to excessive computation\. Ablating state components such as geometryδt\\delta\_\{t\}, dynamics\(vt,ct\)\(v\_\{t\},c\_\{t\}\), and uncertaintyHtH\_\{t\}shows that they contribute complementary signals: removing geometryδt\\delta\_\{t\}reduces performance \(from 62\.8 to 42\.5 in QS\), while removing dynamics\(vt,ct\)\(v\_\{t\},c\_\{t\}\)increases computational cost and leads to a drop in LCR accuracy \(from 33\.0 to 31\.6\)\. Removing uncertaintyHtH\_\{t\}results in only a minor performance change in most domains \(from 54\.5 to 52\.1 in S&C\), but still weakens the overall accuracy–cost balance\. Finally, replacing the sentence\-level reasoning units does not significantly improve performance, but suggests that the default sentence\-level granularity provides stable control across domains\. SoT’s strength lies in the synergy between its components, rather than in any single one\.

### 3\.5Transfer to Vision\-Language Reasoning

Table 4:Vision–language reasoning across scales\(accuracy, %; completion tokens; latency, s\)\.MethodQwen2\.5\-VL\-7BQwen2\.5\-VL\-32BA\-OKVQAAI2DM3CoTA\-OKVQAAI2DM3CoTacc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.CoT81\.01373\.082\.71893\.980\.71784\.484\.026712\.887\.332117\.087\.332916\.0PS84\.01693\.582\.72455\.981\.32115\.880\.531116\.972\.743627\.178\.738717\.9SR78\.02135\.180\.02646\.183\.32546\.484\.541422\.289\.346526\.184\.046924\.3SC83\.569115\.987\.395519\.984\.090921\.384\.0131964\.086\.0166987\.687\.3161878\.2BoN84\.569114\.988\.796121\.786\.791921\.085\.0132265\.587\.3168088\.885\.3163380\.0SoT87\.51914\.486\.72155\.487\.31984\.585\.542523\.290\.036419\.288\.040920\.1Across these 6 model–reasoning\-task settings, SoT averages87\.5%87\.5\\%accuracy and is best in 5\. On 7B AI2D, it is2\.02\.0points below BoN but uses4\.5×4\.5\\timesfewer completion tokens\. At 32B, it exceeds the strongest alternative by0\.50\.5–0\.70\.7points on all 3 tasks while using3\.13\.1–4\.6×4\.6\\timesfewer tokens than search\-based methods\. Across both scales, it also reduces end\-to\-end latency by73\.5%73\.5\\%relative to SC and BoN\. The gains across knowledge\-grounded, diagrammatic, and multimodal chain\-of\-thought tasks show that state\-conditioned evidence organization transfers beyond text\-only reasoning and persists across scale without expanding the controller or relying on wider sampling\.

### 3\.6Case study

Figure[7](https://arxiv.org/html/2609.16055#S3.F7)shows a representative example with activated historical evidence at each step, and the corresponding stop probability\. SoT does not keep the full trajectory active, nor does it rely on a fixed external reasoning template\. Instead, as constraints are progressively resolved, the endogenous state selectively retains the evidence still needed for the next decision, suppresses obsolete support, and raises stop readiness only when the remaining context becomes sufficient\. This example therefore concretely illustrates the closed\-loop view of SoT: state organizes support, support shapes the next step, and the updated step in turn changes the state\.

#### Overall findings\.

Across accuracy, efficiency, ablations, scale transfer, and case evidence, SoT improves reasoning by reallocating existing inference computation rather than enlarging search or backbone capacity\. Its endogenous state acts as a compact control interface that couples evidence retention with termination, translating continuous internal evolution into discrete, inspectable decisions\. This shared mechanism explains why the same lightweight design advances the accuracy–efficiency frontier across heterogeneous text tasks and multimodal scales\.

![Refer to caption](https://arxiv.org/html/2609.16055v1/app_case.png)Figure 7:A case study of SoT,with more case studies in Appendix[Q](https://arxiv.org/html/2609.16055#A17)\.

## 4Exploratory Extensions of State of Thought

### 4\.1SoT under Limited Access

#### Training\-Free SoT\.

We first ask*whether SoT requires learned control at all*\. We instantiate the evidence\-organization operator𝒮\\mathcal\{S\}and stopping operator𝒯\\mathcal\{T\}without training, using fixed rules derived from the same dynamics\-geometric state\. Historical evidence is selected by state compatibility with the current reasoning regime, while stopping is triggered when the trajectory becomes sufficiently stable and low\-uncertainty\. This yields a fully training\-free SoT controller that preserves the same closed\-loop formulation while removing learned control entirely\.

#### SoT\-Embed under Limited Access\.

We next ask*whether SoT depends on privileged access to internal information transfer*\. We embed the model’s multi\-step segmented output with a fixed encoder and feed the embedding trajectory into the control loop in place of the internal state\. This tests whether endogenous reasoning control remains viable when its observable interface is restricted to text\.

SoT\-Training\-free evaluates whether the benefits of SoT come solely from the learned controller, while SoT\-Embed tests whether it relies on white\-box access\. Table[5](https://arxiv.org/html/2609.16055#S4.T5)summarizes that both variants maintain competitive performance, with macro\-averaged accuracy lower relative to full SoT by approximately17\.5%\{17\.5\\%\}for SoT\-Training\-free and17\.9%\{17\.9\\%\}for SoT\-Embed\. In comparison, the Top\-3 baseline methods show an average accuracy of47\.5%47\.5\\%on those cells, with SoT variants achieving comparable or slightly better results in many tasks\. These results suggest that SoT’s core advantage lies in the principle of state\-conditioned evidence organization, not in a specific implementation\. Despite these reductions relative to full SoT, both variants remain highly competitive, often outperforming the Top\-3 baselines on this Llama table\. For further details, see Appendix[D](https://arxiv.org/html/2609.16055#A4)\.

Table 5:Boundary studies of SoT on Llama\-3\.1\-8B,with other models provided in Tables[22](https://arxiv.org/html/2609.16055#A16.T22),[23](https://arxiv.org/html/2609.16055#A16.T23)\.Quantitative ReasoningSymbolic and CodeGSM8KMATHDROPavg\.FOLIOPWBBH\-THEMBPPavg\.Top\-3 Baseline avg\.81\.961\.516\.253\.240\.637\.660\.030\.052\.044\.0SoT\-Training\-free81\.860\.534\.659\.043\.430\.277\.053\.050\.850\.1SoT\-Embedding82\.861\.832\.359\.033\.830\.574\.057\.952\.849\.8
General UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.Top\-3 Baseline avg\.50\.368\.678\.953\.356\.561\.515\.625\.431\.724\.2SoT\-Training\-free37\.555\.270\.045\.453\.052\.021\.834\.224\.826\.9SoT\-Embedding37\.556\.871\.241\.852\.051\.928\.530\.324\.827\.9

### 4\.2SoT as a Trajectory\-Level Judge

We further study*whether information visible only in the completed reasoning trace can support black\-box quality assessment*, without online control or access to hidden states\. Given sentence\-segmented model outputs, we map each step to a standard sentence embedding and summarize the resulting sequence using compact trajectory statistics, with details in Appendix[E](https://arxiv.org/html/2609.16055#A5)\. A small trainable predictor \(SoT\-Judge\) then estimates whether the final answer matches reference correctness\. The goal is a clean trajectory\-only readout of reasoning health, parallel in spirit to SoT’s use of trajectory organization, but reduced to post\-hoc inputs that any API model exposes\. We evaluate three closed or API\-accessed models: GPT\-5\.4\([OpenAI, 2026](https://arxiv.org/html/2609.16055#bib.bib58)\), Claude Opus 4\.7\([Anthropic, 2026](https://arxiv.org/html/2609.16055#bib.bib59)\), and Qwen\-Max\([Alibaba Cloud Model Studio, 2026](https://arxiv.org/html/2609.16055#bib.bib60)\)\.

Table 6:Black\-box judging with trajectory\-based SoT\(agreement with reference correctness\)\.ModelMethodQRGUS&CLCRavg\.GPT\-5\.4Self\-Consistency58\.381\.479\.216\.758\.9Self\-Verification58\.380\.678\.216\.058\.3LLM\-as\-a\-Judge58\.382\.079\.216\.359\.0SoT\-Judge86\.070\.069\.881\.076\.7ClaudeOpus 4\.7Self\-Consistency56\.383\.675\.25\.055\.0Self\-Verification59\.462\.780\.39\.352\.9LLM\-as\-a\-Judge56\.783\.675\.45\.355\.2SoT\-Judge86\.364\.867\.099\.079\.3Qwen\-MaxSelf\-Consistency57\.984\.080\.014\.559\.1Self\-Verification29\.386\.577\.911\.651\.3LLM\-as\-a\-Judge59\.384\.879\.814\.859\.7SoT\-Judge83\.677\.469\.682\.278\.2

We compare against three representative black\-box judging strategies, including Self\-Consistency\([Wang et al\., 2023b](https://arxiv.org/html/2609.16055#bib.bib5)\), Self\-Verification\([Weng et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib61)\), and LLM\-as\-a\-Judge\([Zheng et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib57)\), covering sample\-based agreement, self\-verification, and external judging\. By contrast,SoT\-Judgepredicts correctness only from the embedded sentence trajectory, without extra rollouts, self\-critique passes, or stronger external judge calls\. Table[6](https://arxiv.org/html/2609.16055#S4.T6), with breakdown in Table[24](https://arxiv.org/html/2609.16055#A16.T24), shows thatSoT\-Judgeimproves the average agreement over the strongest baseline by\+17\.7\{\+17\.7\},\+24\.1\{\+24\.1\}, and\+18\.5\{\+18\.5\}points on GPT\-5\.4, Claude Opus 4\.7, and Qwen\-Max, respectively\. The gain is not uniform across domains: on GU and S&C, trajectory\-only judging is weaker than the strongest standard judges, while the largest improvements come from QR and especially LCR, where agreement of black\-box baselines drops to5\.0​–​16\.7%\{5\.0\\text\{\-\-\}16\.7\\%\}butSoT\-Judgeremains at81\.0​–​99\.0%\{81\.0\\text\{\-\-\}99\.0\\%\}\. This suggests that trajectory structure provides a useful correctness signal, especially when black\-box judging becomes unreliable under sparse or long\-range evidence\.

## 5Conclusion

We presented State of Thought \(SoT\), a reasoning paradigm that changes the control variable of test\-time reasoning from external token programs to endogenous state\-conditioned evidence organization\. Rather than prescribing longer chains, larger search budgets, or fixed memory rules, SoT uses the model’s evolving representational state to dynamically gate which evidence remains active and when reasoning should terminate\. This closed\-loop formulation yields measurable gains on the accuracy–efficiency frontier across text and multimodal benchmarks, and generalizes robustly to limited\-access and trajectory\-level evaluation settings\. By using a compact continuous state to govern discrete, inspectable reasoning steps, SoT connects latent internal evolution with explicit reasoning without sacrificing either controllability or interpretability\. Our results point to a broader principle: reasoning systems need not rely on increasingly elaborate external scaffolding — adaptivity and efficiency can emerge directly from the model’s internal state geometry\.

## Acknowledgements

This research is supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier 2 \(Award MOE\-T2EP20125\-0005\)\.

## References

- Alibaba Cloud Model StudioModel list\.Note:Alibaba Cloud DocumentationOfficial model catalog documenting Qwen\-MaxExternal Links:[Link](https://help.aliyun.com/zh/model-studio/models)Cited by:[§4\.2](https://arxiv.org/html/2609.16055#S4.SS2.p1.1)\.
- Anthropic \(2026\)AnthropicClaude opus 4\.7\.Note:Anthropic DocumentationOfficial model documentationExternal Links:[Link](https://docs.anthropic.com/en/docs/about-claude/models/overview)Cited by:[§4\.2](https://arxiv.org/html/2609.16055#S4.SS2.p1.1)\.
- Austinet al\.\(2021\)J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le, and C\. SuttonProgram synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[3rd item](https://arxiv.org/html/2609.16055#A11.I1.i3.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Baiet al\.\(2025\)S\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge, S\. Song, K\. Dang, P\. Wang, S\. Wang, J\. Tang,et al\.Qwen2\.5\-vl technical report\.arXiv preprint arXiv:2502\.13923\.Cited by:[Appendix L](https://arxiv.org/html/2609.16055#A12.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px1.p1.1)\.
- Baiet al\.\(2024\)Y\. Bai, X\. Lv, J\. Zhang, H\. Lyu, J\. Tang, Z\. Huang, Z\. Du, X\. Liu, A\. Zeng, L\. Hou,et al\.Longbench: a bilingual, multitask benchmark for long context understanding\.InProceedings of the 62nd annual meeting of the association for computational linguistics \(volume 1: Long papers\),pp\. 3119–3137\.Cited by:[4th item](https://arxiv.org/html/2609.16055#A11.I1.i4.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Ballonet al\.\(2026\)M\. Ballon, B\. Verbeken, V\. Ginis, and A\. AlgabaProbing the trajectories of reasoning traces in large language models\.arXiv preprint arXiv:2601\.23163\.Cited by:[§R\.2](https://arxiv.org/html/2609.16055#A18.SS2.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1)\.
- Bogdanet al\.\(2025\)P\. C\. Bogdan, U\. Macar, N\. Nanda, and A\. ConmyThought anchors: which LLM reasoning steps matter?\.InMechanistic Interpretability Workshop at NeurIPS 2025,External Links:[Link](https://openreview.net/forum?id=VnSlfeRCaU)Cited by:[Appendix A](https://arxiv.org/html/2609.16055#A1.SS0.SSS0.Px1.p1.1)\.
- Brownet al\.\(2024\)B\. Brown, J\. Juravsky, R\. Ehrlich, R\. Clark, Q\. V\. Le, C\. Ré, and A\. MirhoseiniLarge language monkeys: scaling inference compute with repeated sampling\.arXiv preprint arXiv:2407\.21787\.Cited by:[§1](https://arxiv.org/html/2609.16055#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.16055#S2.SS1.p1.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. d\. O\. Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman,et al\.Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[3rd item](https://arxiv.org/html/2609.16055#A11.I1.i3.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Chenet al\.\(2024\)Q\. Chen, L\. Qin, J\. Zhang, Z\. Chen, X\. Xu, and W\. CheM3\{\}^\{3\}cot: a novel benchmark for multi\-domain multi\-step multi\-modal chain\-of\-thought\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Cited by:[§K\.1](https://arxiv.org/html/2609.16055#A11.SS1.p3.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Clarket al\.\(2019\)C\. Clark, K\. Lee, M\. Chang, T\. Kwiatkowski, M\. Collins, and K\. ToutanovaBoolQ: exploring the surprising difficulty of natural yes/no questions\.arXiv preprint arXiv:1905\.10044\.Cited by:[2nd item](https://arxiv.org/html/2609.16055#A11.I1.i2.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[1st item](https://arxiv.org/html/2609.16055#A11.I1.i1.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Creswellet al\.\(2022\)A\. Creswell, M\. Shanahan, and I\. HigginsSelection\-inference: exploiting large language models for interpretable logical reasoning\.arXiv preprint arXiv:2205\.09712\.Cited by:[§1](https://arxiv.org/html/2609.16055#S1.p2.1)\.
- Damirchiet al\.\(2026\)H\. Damirchi, I\. Meza De la Jara, E\. Abbasnejad, A\. Shamsi, Z\. Zhang, and J\. ShiTruth as a trajectory: what internal representations reveal about large language model reasoning\.arXiv preprint arXiv:2603\.01326\.Cited by:[§R\.2](https://arxiv.org/html/2609.16055#A18.SS2.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1)\.
- Duaet al\.\(2019\)D\. Dua, Y\. Wang, P\. Dasigi, G\. Stanovsky, S\. Singh, and M\. GardnerDROP: a reading comprehension benchmark requiring discrete reasoning over paragraphs\.InProceedings of NAACL\-HLT,pp\. 2368–2378\.Cited by:[1st item](https://arxiv.org/html/2609.16055#A11.I1.i1.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Fenget al\.\(2025\)J\. Feng, S\. Russell, and J\. SteinhardtMonitoring latent world states in language models with propositional probes\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§R\.2](https://arxiv.org/html/2609.16055#A18.SS2.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1)\.
- Gevaet al\.\(2021\)M\. Geva, D\. Khashabi, E\. Segal, T\. Khot, D\. Roth, and J\. BerantDid aristotle use a laptop? a question answering benchmark with implicit reasoning strategies\.Transactions of the Association for Computational Linguistics9,pp\. 346–361\.Cited by:[2nd item](https://arxiv.org/html/2609.16055#A11.I1.i2.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Gonget al\.\(2026a\)Z\. Gong, T\. Wu, J\. Zhang, F\. Zhang, C\. Wang, Y\. Hao, Y\. Hou, F\. Ping, Y\. Zhao, F\. Huang, C\. Yuen, and W\. Y\. B\. LimXDomainbench: diagnosing reasoning collapse in high\-dimensional scientific knowledge composition\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=U8x5SYtT5b)Cited by:[§1](https://arxiv.org/html/2609.16055#S1.p1.1)\.
- Gonget al\.\(2026b\)Z\. Gong, Z\. Zeng, C\. Yuen, and W\. Y\. B\. LimConditional co\-ablation: recovering self\-repair backups in transformer circuits\.External Links:2607\.01940,[Link](https://arxiv.org/abs/2607.01940)Cited by:[§1](https://arxiv.org/html/2609.16055#S1.p2.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[Appendix L](https://arxiv.org/html/2609.16055#A12.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px1.p1.1)\.
- Hanet al\.\(2024\)S\. Han, H\. Schoelkopf, Y\. Zhao, Z\. Qi, M\. Riddell, W\. Zhou, J\. Coady, D\. Peng, Y\. Qiao, L\. Benson,et al\.Folio: natural language reasoning with first\-order logic\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 22017–22031\.Cited by:[3rd item](https://arxiv.org/html/2609.16055#A11.I1.i3.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Haoet al\.\(2025\)S\. Hao, S\. Sukhbaatar, D\. Su, X\. Li, Z\. Hu, J\. E\. Weston, and Y\. TianTraining large language models to reason in a continuous latent space\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=Itxz7S4Ip3)Cited by:[§M\.4](https://arxiv.org/html/2609.16055#A13.SS4.SSS0.Px1.p1.1),[§R\.4](https://arxiv.org/html/2609.16055#A18.SS4.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1)\.
- Hassidet al\.\(2025\)M\. Hassid, G\. Synnaeve, Y\. Adi, and R\. SchwartzDon’t overthink it\. preferring shorter thinking chains for improved llm reasoning\.arXiv preprint arXiv:2505\.17813\.Cited by:[§R\.4](https://arxiv.org/html/2609.16055#A18.SS4.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p1.1)\.
- Hendryckset al\.\(2021a\)D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. SteinhardtMeasuring massive multitask language understanding\.InInternational Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2609.16055#A11.I1.i2.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Hendryckset al\.\(2021b\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the math dataset\.NeurIPS34,pp\. 5327–5344\.Cited by:[1st item](https://arxiv.org/html/2609.16055#A11.I1.i1.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Hokamp and Liu \(2017\)C\. Hokamp and Q\. LiuLexically constrained decoding for sequence generation using grid beam search\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1535–1546\.Cited by:[§M\.2](https://arxiv.org/html/2609.16055#A13.SS2.SSS0.Px6.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1)\.
- Honget al\.\(2024\)J\. Hong, N\. Lee, and J\. ThorneORPO: monolithic preference optimization without reference model\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 11170–11189\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.626/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.626)Cited by:[§2\.3](https://arxiv.org/html/2609.16055#S2.SS3.SSS0.Px2.p1.1)\.
- Hosseiniet al\.\(2026\)P\. Hosseini, S\. Nawathe, M\. Salmani, M\. Razaviyayn, and S\. FeiziEarly stopping for large reasoning models via confidence dynamics\.arXiv preprint arXiv:2604\.04930\.Cited by:[§R\.2](https://arxiv.org/html/2609.16055#A18.SS2.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1),[§2\.3](https://arxiv.org/html/2609.16055#S2.SS3.p3.2)\.
- Jianget al\.\(2024a\)A\. Q\. Jiang, A\. Sablayrolles, A\. Roux, A\. Mensch, B\. Savary, C\. Bamford, D\. S\. Chaplot, D\. d\. l\. Casas, E\. B\. Hanna, F\. Bressand,et al\.Mixtral of experts\.arXiv preprint arXiv:2401\.04088\.Cited by:[Appendix L](https://arxiv.org/html/2609.16055#A12.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px1.p1.1)\.
- Jianget al\.\(2024b\)H\. Jiang, Q\. Wu, X\. Luo, D\. Li, C\. Lin, Y\. Yang, and L\. QiuLongllmlingua: accelerating and enhancing llms in long context scenarios via prompt compression\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1658–1677\.Cited by:[§R\.3](https://arxiv.org/html/2609.16055#A18.SS3.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1),[§2\.3](https://arxiv.org/html/2609.16055#S2.SS3.SSS0.Px2.p1.1)\.
- Kembhaviet al\.\(2016\)A\. Kembhavi, M\. Salvato, E\. Kolve, M\. Seo, H\. Hajishirzi, and A\. FarhadiA diagram is worth a dozen images\.InEuropean Conference on Computer Vision \(ECCV\),Cited by:[§K\.1](https://arxiv.org/html/2609.16055#A11.SS1.p3.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Kočiskỳet al\.\(2018\)T\. Kočiskỳ, J\. Schwarz, P\. Blunsom, C\. Dyer, K\. M\. Hermann, G\. Melis, and E\. GrefenstetteThe narrativeqa reading comprehension challenge\.Transactions of the Association for Computational Linguistics6,pp\. 317–328\.Cited by:[4th item](https://arxiv.org/html/2609.16055#A11.I1.i4.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Laiet al\.\(2017\)G\. Lai, Q\. Xie, H\. Liu, Y\. Yang, and E\. HovyRACE: large\-scale ReAding comprehension dataset from examinations\.InProceedings of EMNLP,pp\. 785–794\.Cited by:[2nd item](https://arxiv.org/html/2609.16055#A11.I1.i2.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Liet al\.\(2023\)Y\. Li, B\. Dong, F\. Guerin, and C\. LinCompressing context to enhance inference efficiency of large language models\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 6342–6353\.Cited by:[§R\.3](https://arxiv.org/html/2609.16055#A18.SS3.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1),[§2\.3](https://arxiv.org/html/2609.16055#S2.SS3.SSS0.Px2.p1.1)\.
- Liet al\.\(2024\)Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. ChenSnapkv: llm knows what you are looking for before generation\.Advances in Neural Information Processing Systems37,pp\. 22947–22970\.Cited by:[§M\.3](https://arxiv.org/html/2609.16055#A13.SS3.SSS0.Px2.p1.1),[§R\.3](https://arxiv.org/html/2609.16055#A18.SS3.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1)\.
- Liu and Wang \(2025\)X\. Liu and L\. WangAnswer convergence as a signal for early stopping in reasoning\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,Suzhou, China,pp\. 17896–17907\.External Links:ISBN 979\-8\-89176\-332\-6,[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.904),[Link](https://aclanthology.org/2025.emnlp-main.904/)Cited by:[§2\.3](https://arxiv.org/html/2609.16055#S2.SS3.SSS0.Px2.p1.1)\.
- Liuet al\.\(2026\)Z\. Liu, Y\. Li, and Y\. XuThink in sentences: explicit sentence boundaries enhance language model’s capabilities\.arXiv preprint arXiv:2604\.10135\.Cited by:[Appendix A](https://arxiv.org/html/2609.16055#A1.SS0.SSS0.Px1.p1.1)\.
- Madaanet al\.\(2023\)A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. ClarkSelf\-refine: iterative refinement with self\-feedback\.arXiv preprint arXiv:2303\.17651\.Cited by:[§M\.2](https://arxiv.org/html/2609.16055#A13.SS2.SSS0.Px3.p1.1),[§R\.1](https://arxiv.org/html/2609.16055#A18.SS1.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1)\.
- OpenAI \(2026\)OpenAIIntroducing gpt\-5\.4\.Note:OpenAIOfficial model announcementExternal Links:[Link](https://openai.com/index/introducing-gpt-5-4/)Cited by:[§4\.2](https://arxiv.org/html/2609.16055#S4.SS2.p1.1)\.
- Packeret al\.\(2023\)C\. Packer, V\. Fang, S\. Patil, K\. Lin, S\. Wooders, and J\. GonzalezMemGPT: towards llms as operating systems\.\.Cited by:[§1](https://arxiv.org/html/2609.16055#S1.p2.1)\.
- Parket al\.\(2025\)K\. Park, Y\. J\. Choe, Y\. Jiang, and V\. VeitchThe geometry of categorical and hierarchical concepts in large language models\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§R\.2](https://arxiv.org/html/2609.16055#A18.SS2.p1.1),[§2\.2](https://arxiv.org/html/2609.16055#S2.SS2.p1.1)\.
- Parket al\.\(2024\)K\. Park, Y\. J\. Choe, and V\. VeitchThe linear representation hypothesis and the geometry of large language models\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 39643–39666\.Cited by:[§R\.2](https://arxiv.org/html/2609.16055#A18.SS2.p1.1),[§2\.2](https://arxiv.org/html/2609.16055#S2.SS2.p1.1)\.
- Presset al\.\(2023\)O\. Press, M\. Zhang, S\. Min, L\. Schmidt, N\. A\. Smith, and M\. LewisMeasuring and narrowing the compositionality gap in language models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 5687–5711\.Cited by:[§1](https://arxiv.org/html/2609.16055#S1.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, S\. Ermon, C\. D\. Manning, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.arXiv preprint arXiv:2305\.18290\.External Links:2305\.18290,[Link](https://arxiv.org/abs/2305.18290)Cited by:[§2\.3](https://arxiv.org/html/2609.16055#S2.SS3.SSS0.Px2.p1.1)\.
- Schwenket al\.\(2022\)D\. Schwenk, A\. Khandelwal, C\. Clark, K\. Marino, and R\. MottaghiA\-okvqa: a benchmark for visual question answering using world knowledge\.InEuropean Conference on Computer Vision \(ECCV\),Cited by:[§K\.1](https://arxiv.org/html/2609.16055#A11.SS1.p3.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§M\.4](https://arxiv.org/html/2609.16055#A13.SS4.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1)\.
- Snellet al\.\(2024\)C\. Snell, J\. Lee, K\. Xu, and A\. KumarScaling llm test\-time compute optimally can be more effective than scaling model parameters\.arXiv preprint arXiv:2408\.03314\.Cited by:[§M\.2](https://arxiv.org/html/2609.16055#A13.SS2.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.16055#S2.SS1.p1.2),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1)\.
- Srivastavaet al\.\(2023\)A\. Srivastava, A\. Rastogi, A\. Rao, A\. A\. M\. Shoeb, A\. Abid, A\. Fisch, A\. R\. Brown, A\. Santoro, A\. Gupta, A\. Garriga\-Alonso,et al\.Beyond the imitation game: quantifying and extrapolating the capabilities of language models\.Transactions on machine learning research\.Cited by:[3rd item](https://arxiv.org/html/2609.16055#A11.I1.i3.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Sunet al\.\(2026\)L\. Sun, H\. Dong, B\. Qiao, Q\. Lin, D\. Zhang, and S\. RajmohanLLM reasoning as trajectories: step\-specific representation geometry and correctness signals\.arXiv preprint arXiv:2604\.05655\.Cited by:[§R\.2](https://arxiv.org/html/2609.16055#A18.SS2.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1)\.
- Tafjordet al\.\(2021\)O\. Tafjord, B\. Dalvi, and P\. ClarkProofwriter: generating implications, proofs, and abductive statements over natural language\.InFindings of the Association for Computational Linguistics: ACL\-IJCNLP 2021,pp\. 3621–3634\.Cited by:[3rd item](https://arxiv.org/html/2609.16055#A11.I1.i3.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Talmoret al\.\(2019\)A\. Talmor, J\. Herzig, N\. Lourie, and J\. BerantCommonsenseqa: a question answering challenge targeting commonsense knowledge\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),pp\. 4149–4158\.Cited by:[2nd item](https://arxiv.org/html/2609.16055#A11.I1.i2.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Tikhonovet al\.\(2026\)P\. Tikhonov, I\. Oseledets, and E\. TutubalinaConfidence leaps in llm reasoning: early stopping and cross\-model transfer\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 2: Short Papers\),pp\. 602–616\.Cited by:[§R\.2](https://arxiv.org/html/2609.16055#A18.SS2.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1)\.
- Trivediet al\.\(2023\)H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. SabharwalInterleaving retrieval with chain\-of\-thought reasoning for knowledge\-intensive multi\-step questions\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 10014–10037\.Cited by:[§R\.3](https://arxiv.org/html/2609.16055#A18.SS3.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p1.1)\.
- Wanget al\.\(2023a\)L\. Wang, W\. Xu, Y\. Lan, Z\. Hu, Y\. Lan, R\. K\. Lee, and E\. LimPlan\-and\-solve prompting: improving zero\-shot chain\-of\-thought reasoning by large language models\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 2609–2634\.Cited by:[Appendix A](https://arxiv.org/html/2609.16055#A1.SS0.SSS0.Px1.p1.1),[§M\.2](https://arxiv.org/html/2609.16055#A13.SS2.SSS0.Px2.p1.1),[§R\.1](https://arxiv.org/html/2609.16055#A18.SS1.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1)\.
- Wanget al\.\(2025\)X\. Wang, S\. Wang, Y\. Zhu, and B\. LiuSystem\-1\.5 reasoning: traversal in language and latent spaces with dynamic shortcuts\.arXiv preprint arXiv:2505\.18962\.Cited by:[§R\.4](https://arxiv.org/html/2609.16055#A18.SS4.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1)\.
- Wanget al\.\(2023b\)X\. Wang, J\. Wei, D\. Schuurmans, Q\. V\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. ZhouSelf\-consistency improves chain of thought reasoning in language models\.InThe Eleventh International Conference on Learning Representations,Cited by:[§M\.2](https://arxiv.org/html/2609.16055#A13.SS2.SSS0.Px4.p1.1),[§R\.1](https://arxiv.org/html/2609.16055#A18.SS1.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1),[§4\.2](https://arxiv.org/html/2609.16055#S4.SS2.p4.1)\.
- Wanget al\.\(2024\)Z\. Wang, A\. Liu, H\. Lin, J\. Li, X\. Ma, and Y\. LiangRAT: retrieval augmented thoughts elicit context\-aware reasoning in long\-horizon generation\.arXiv preprint arXiv:2403\.05313\.Cited by:[§R\.3](https://arxiv.org/html/2609.16055#A18.SS3.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p2.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. ZhouChain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 24824–24837\.Cited by:[§M\.2](https://arxiv.org/html/2609.16055#A13.SS2.SSS0.Px1.p1.1),[§R\.1](https://arxiv.org/html/2609.16055#A18.SS1.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1)\.
- Wenget al\.\(2023\)Y\. Weng, M\. Zhu, F\. Xia, B\. Li, S\. He, S\. Liu, B\. Sun, K\. Liu, and J\. ZhaoLarge language models are better reasoners with self\-verification\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 2550–2575\.Cited by:[§4\.2](https://arxiv.org/html/2609.16055#S4.SS2.p4.1)\.
- Xiaoet al\.\(2023\)G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. LewisEfficient streaming language models with attention sinks\.arXiv preprint arXiv:2309\.17453\.Cited by:[§M\.3](https://arxiv.org/html/2609.16055#A13.SS3.SSS0.Px3.p1.1),[§R\.3](https://arxiv.org/html/2609.16055#A18.SS3.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1)\.
- Xieet al\.\(2024\)Y\. Xie, A\. Goyal, W\. Zheng, M\. Kan, T\. Lillicrap, K\. Kawaguchi, and M\. ShiehMonte carlo tree search boosts reasoning via iterative preference learning\.arXiv preprint arXiv:2405\.00451\.Cited by:[§M\.2](https://arxiv.org/html/2609.16055#A13.SS2.SSS0.Px7.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1)\.
- Yanget al\.\(2024\)A\. Yang, B\. Yang, B\. Hui, B\. Zheng, B\. Yu, C\. Zhou, D\. Fan, F\. Huang, H\. Wei, J\. Xiong,et al\.Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[Appendix L](https://arxiv.org/html/2609.16055#A12.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px1.p1.1)\.
- Yanget al\.\(2018\)Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. W\. Cohen, R\. Salakhutdinov, and C\. D\. ManningHotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of EMNLP,pp\. 2369–2380\.Cited by:[4th item](https://arxiv.org/html/2609.16055#A11.I1.i4.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px2.p1.1)\.
- Yaoet al\.\(2023a\)S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. NarasimhanTree of thoughts: deliberate problem solving with large language models\.arXiv preprint arXiv:2305\.10601\.Cited by:[§R\.1](https://arxiv.org/html/2609.16055#A18.SS1.p1.1),[§1](https://arxiv.org/html/2609.16055#S1.p1.1),[§2\.3](https://arxiv.org/html/2609.16055#S2.SS3.p2.2)\.
- Yaoet al\.\(2023b\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InThe Eleventh International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.16055#S1.p1.1)\.
- Zhanget al\.\(2023\)Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng, R\. Cai, Z\. Song, Y\. Tian, C\. Ré, C\. Barrett,et al\.H2o: heavy\-hitter oracle for efficient generative inference of large language models\.Advances in Neural Information Processing Systems36,pp\. 34661–34710\.Cited by:[§M\.3](https://arxiv.org/html/2609.16055#A13.SS3.SSS0.Px1.p1.1),[§R\.3](https://arxiv.org/html/2609.16055#A18.SS3.p1.1),[§3\.1](https://arxiv.org/html/2609.16055#S3.SS1.SSS0.Px3.p1.1)\.
- Zhenget al\.\(2023\)L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing,et al\.Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.Advances in neural information processing systems36,pp\. 46595–46623\.Cited by:[§4\.2](https://arxiv.org/html/2609.16055#S4.SS2.p4.1)\.
- Zhouet al\.\(2023\)D\. Zhou, N\. Schärli, L\. Hou, J\. Wei, N\. Scales, X\. Wang, D\. Schuurmans, C\. Cui, O\. Bousquet, Q\. Le, and E\. H\. ChiLeast\-to\-most prompting enables complex reasoning in large language models\.InThe Eleventh International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.16055#S1.p1.1)\.
- Zouet al\.\(2023\)A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. HendrycksRepresentation engineering: a top\-down approach to ai transparency\.External Links:2310\.01405Cited by:[§1](https://arxiv.org/html/2609.16055#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.16055#S2.SS2.p1.1)\.

Appendix Overview

This appendix provides the complete algorithmic details, theoretical analysis, experimental setup, extended results, and discussion forState of Thought \(SoT\)\. Concretely, the appendix serves two complementary purposes:*\(i\)*to make the actual controller precise enough to be reproducible, and*\(ii\)*to make the underlying reasoning claim precise enough to be evaluable\.

#### Part A: Algorithmic Details\.

This part expands the method into explicit components and aligns each component with the actual implementation used in our experiments\.

- •Appendix[A](https://arxiv.org/html/2609.16055#A1): the complete SoT inference loop as state\-conditioned reasoning\.
- •Appendix[B](https://arxiv.org/html/2609.16055#A2): the endogenous reasoning state, its geometric meaning, and the controller\-facing state interface\.
- •Appendix[C](https://arxiv.org/html/2609.16055#A3): the controller fitting protocol from offline trajectories\.
- •Appendix[D](https://arxiv.org/html/2609.16055#A4): limited\-access SoT variants, including training\-free SoT and SoT\-Embed with sentence\-embedding trajectory features\.
- •Appendix[E](https://arxiv.org/html/2609.16055#A5):SoT\-Judgeas a trajectory\-level black\-box evaluator\.

#### Part B: Theoretical Analysis\.

This part formalizes the central claim of SoT as a reasoning\-control paradigm\.

- •Appendix[F](https://arxiv.org/html/2609.16055#A6): why externally prescribed token\-chain policy classes can be strictly weaker than state\-conditioned control\.
- •Appendix[G](https://arxiv.org/html/2609.16055#A7): why state\-conditioned evidence organization is necessary on heterogeneous reasoning distributions\.
- •Appendix[H](https://arxiv.org/html/2609.16055#A8): why sparse, state\-aligned evidence activation can improve both decision quality and efficiency\.
- •Appendix[I](https://arxiv.org/html/2609.16055#A9): why feedback evidence control dominates open\-loop reasoning schedules under regime variation\.
- •Appendix[J](https://arxiv.org/html/2609.16055#A10): why state\-driven stopping is a rational component of bounded\-cost reasoning\.

#### Part C: Experimental Setup and Implementation Details\.

This part expands the evaluation protocol, dataset organization, baseline taxonomy, and hyperparameter structure\.

- •Appendix[K](https://arxiv.org/html/2609.16055#A11): datasets, task types, reasoning families, and metric conventions\.
- •Appendix[L](https://arxiv.org/html/2609.16055#A12): backbones, evaluation protocol, and inference accounting\.
- •Appendix[M](https://arxiv.org/html/2609.16055#A13): detailed baseline taxonomy and comparison rationale\.
- •Appendix[N](https://arxiv.org/html/2609.16055#A14): controller, search, and evaluation hyperparameters\.

#### Part D: Additional Experimental Results and Analysis\.

This part collects interpretability analyses, additional results, and qualitative trajectory inspections that support the main paper’s claims\.

- •Appendix[O](https://arxiv.org/html/2609.16055#A15): extended state\-space and cluster\-based evidence for endogenous structure\.
- •Appendix[P](https://arxiv.org/html/2609.16055#A16): additional tables, efficiency breakdowns, and supplementary comparisons\.
- •Appendix[Q](https://arxiv.org/html/2609.16055#A17): qualitative trajectory\-level case studies\.

#### Part E: Related Work and Discussion\.

This part expands the paper’s broader positioning and the main limitations and future opportunities opened by SoT\.

- •Appendix[R](https://arxiv.org/html/2609.16055#A18): expanded related\-work positioning\.
- •Appendix[S](https://arxiv.org/html/2609.16055#A19): limitations and future work\.

Part A: Algorithmic Details

## Appendix AImplementation Details of State\-Conditioned Reasoning

Algorithm 1State of Thought \(SoT\) inference0:Frozen backbone

ff, problem

xx, evidence operator

𝒮\\mathcal\{S\}, stopping operator

𝒯\\mathcal\{T\}, context budget

BB, maximum steps

TmaxT\_\{\\max\}
0:Reasoning trajectory

y1:Ty\_\{1:T\}and final answer

aa
1:Initialize prompt

p←xp\\leftarrow x, history

y<1←∅y\_\{<1\}\\leftarrow\\varnothing, active evidence

y~<1←∅\\widetilde\{y\}\_\{<1\}\\leftarrow\\varnothing
2:for

t=1,2,…,Tmaxt=1,2,\\dots,T\_\{\\max\}do

3:Generate the next reasoning unit

yt∼ℙ\(⋅∣p,y~<t\)y\_\{t\}\\sim\\mathbb\{P\}\(\\cdot\\mid p,\\widetilde\{y\}\_\{<t\}\)
4:Read out the endogenous state

mtm\_\{t\}from

yty\_\{t\}
5:Update active evidence by

y~<t\+1←𝒮⁡\(y≤t,mt\)\\widetilde\{y\}\_\{<t\+1\}\\leftarrow\\mathcal\{S\}\(y\_\{\\leq t\};m\_\{t\}\)under budget

BB
6:Compute stopping decision

zt←𝒯⁡\(mt\)z\_\{t\}\\leftarrow\\mathcal\{T\}\(m\_\{t\}\)
7:if

zt=1z\_\{t\}=1then

8:Generate the final answer

a∼ℙ\(⋅∣p,y~<t\+1,yt\)a\\sim\\mathbb\{P\}\(\\cdot\\mid p,\\widetilde\{y\}\_\{<t\+1\},y\_\{t\}\)andbreak

9:endif

10:endfor

This appendix makes the inference\-time realization of SoT explicit\. As summarized in Algorithm[1](https://arxiv.org/html/2609.16055#alg1), the framework is fully specified by three components: the endogenous statemtm\_\{t\}, the evidence\-organization operator𝒮\\mathcal\{S\}, and the stopping operator𝒯\\mathcal\{T\}\.

#### Sentence\-level inference policy\.

SoT executes reasoning at the sentence level, because the sentence is typically the smallest unit that still carries a locally coherent reasoning intention while preserving sufficient temporal resolution for stepwise control\[[Bogdan et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib63),[Liu et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib62),[Wang et al\., 2023a](https://arxiv.org/html/2609.16055#bib.bib6)\]\. Accordingly, at steptt, the frozen backbone generates a sentence\-level reasoning unityty\_\{t\}, from which SoT reads out the endogenous statemt∈ℝ4m\_\{t\}\\in\\mathbb\{R\}^\{4\}as defined in Eq\. \([2](https://arxiv.org/html/2609.16055#S2.E2)\) and detailed in Appendix[B](https://arxiv.org/html/2609.16055#A2)\. The running history is therefore represented by the trajectory\(y1,m1\),…,\(yt,mt\)\(y\_\{1\},m\_\{1\}\),\\dots,\(y\_\{t\},m\_\{t\}\), and the controller operates directly on this evolving trajectory rather than on a handcrafted reasoning template\.

At each step, SoT applies the two state\-conditioned operators introduced in Sec\.[2\.3](https://arxiv.org/html/2609.16055#S2.SS3):

y~<t=𝒮⁡\(y<t,mt\),zt=𝒯⁡\(mt\)∈\{0,1\},\\widetilde\{y\}\_\{<t\}=\\mathcal\{S\}\(y\_\{<t\};m\_\{t\}\),\\qquad z\_\{t\}=\\mathcal\{T\}\(m\_\{t\}\)\\in\\\{0,1\\\},\(5\)where𝒮\\mathcal\{S\}determines which historical evidence remains active for the next decision, and𝒯\\mathcal\{T\}determines whether reasoning should continue \(zt=0z\_\{t\}=0\) or terminate \(zt=1z\_\{t\}=1\)\. Inference therefore follows

ℙπ\(yt∣p,y<t\)=ℙ\(yt∣p,y~<t\),ℙπ\(a∣p,y1:t\)=ℙ\(a∣p,y~<t,yt\),\\mathbb\{P\}\_\{\\pi\}\(y\_\{t\}\\mid p,y\_\{<t\}\)=\\mathbb\{P\}\\\!\\left\(y\_\{t\}\\mid p,\\widetilde\{y\}\_\{<t\}\\right\),\\qquad\\mathbb\{P\}\_\{\\pi\}\(a\\mid p,y\_\{1:t\}\)=\\mathbb\{P\}\(a\\mid p,\\widetilde\{y\}\_\{<t\},y\_\{t\}\),\(6\)so that both evidence carrying and stopping become consequences of the current endogenous state\.

#### State\-conditioned evidence organization\.

In the implemented controller,𝒮\\mathcal\{S\}scores each historical reasoning stepyiy\_\{i\}through its paired statemim\_\{i\}relative to the current statemtm\_\{t\}\. For eachi<ti<t, we form the pairwise descriptor

dt,i=\[mt;mi;mt−mi;mt⊙mi\]∈ℝ16,d\_\{t,i\}=\\bigl\[m\_\{t\};\\,m\_\{i\};\\,m\_\{t\}\-m\_\{i\};\\,m\_\{t\}\\odot m\_\{i\}\\bigr\]\\in\\mathbb\{R\}^\{16\},\(7\)which compactly captures the current regime, the historical regime, their displacement, and their channelwise interaction\. A lightweight evidence headϕsel\\phi\_\{\\mathrm\{sel\}\}maps this descriptor to a scalar selection score

gt,i=ϕsel​\(dt,i,θ\),g\_\{t,i\}=\\phi\_\{\\mathrm\{sel\}\}\(d\_\{t,i\};\\theta\),\(8\)whereθ\\thetadenotes the controller parameters learned offline as described in Appendix[C](https://arxiv.org/html/2609.16055#A3)\. Under the context budgetBB, accepted steps are then inserted in chronological order to form

y~<t=𝒮⁡\(y<t,mt\)⊆y<t\.\\widetilde\{y\}\_\{<t\}=\\mathcal\{S\}\(y\_\{<t\};m\_\{t\}\)\\subseteq y\_\{<t\}\.\(9\)Hencey~<t\\widetilde\{y\}\_\{<t\}contains exactly the historical reasoning units selected to remain active for the next decision\. No extra recency\-only or semantic\-only heuristic is used at runtime, so evidence organization remains grounded in endogenous state rather than generic memory management\.

#### State\-conditioned stopping\.

The stopping operator is implemented by a second lightweight head on the same statemtm\_\{t\}:

ot=ϕstop\(mt;ϕ\),ptstop=σ\(ot\),zt=𝕀\[ptstop\>τ\],o\_\{t\}=\\phi\_\{\\mathrm\{stop\}\}\(m\_\{t\};\\phi\),\\qquad p\_\{t\}^\{\\mathrm\{stop\}\}=\\sigma\(o\_\{t\}\),\\qquad z\_\{t\}=\\mathbb\{I\}\\bigl\[p\_\{t\}^\{\\mathrm\{stop\}\}\>\\tau\\bigr\],\(10\)whereϕ\\phidenotes the stop\-head parameters,σ⁡\(⋅\)\\sigma\(\\cdot\)is the sigmoid, andτ∈\(0,1\)\\tau\\in\(0,1\)is the stop threshold learned\. Stopping is therefore determined by the current reasoning regime rather than by a fixed external depth schedule\.

Overall, SoT runs as a closed\-loop state\-conditioned controller:

mt→y~<t=𝒮⁡\(y<t,mt\)→yt→mt\+1,zt=𝒯⁡\(mt\)\.m\_\{t\}\\;\\rightarrow\\;\\widetilde\{y\}\_\{<t\}=\\mathcal\{S\}\(y\_\{<t\};m\_\{t\}\)\\;\\rightarrow\\;y\_\{t\}\\;\\rightarrow\\;m\_\{t\+1\},\\qquad z\_\{t\}=\\mathcal\{T\}\(m\_\{t\}\)\.\(11\)Because the evidence carried at one step changes the next reasoning unit, and the new reasoning unit in turn changes the next endogenous state, the trajectory is path\-dependent by construction\. This is the operational sense in which SoT acts as a feedback controller for reasoning rather than as a fixed reasoning script or a generic history\-compression rule\.

#### Threshold robustness\.

The gate and stop thresholds are part of the fitted controller bundle and could in principle raise sensitivity concerns\. For this reason, the main ablation study in Sec\.[3\.4](https://arxiv.org/html/2609.16055#S3.SS4)and Appendix[P\.3](https://arxiv.org/html/2609.16055#A16.SS3)additionally reports a calibrated variant that adjusts these thresholds post hoc\. Its effect is consistently secondary to the full SoT mechanism, indicating that the observed gains are not driven by brittle threshold tuning alone\.

## Appendix BEndogenous Reasoning State

This section expands Sec\.[2\.2](https://arxiv.org/html/2609.16055#S2.SS2)and makes the state construction explicit\. The goal is to expose the smallest controller state that still captures the regime information needed for online reasoning control\.

### B\.1State construction

#### Sentence center\.

Letht,i\(L\)∈ℝDh^\{\(L\)\}\_\{t,i\}\\in\\mathbb\{R\}^\{D\}denote the final\-layer hidden state of tokeniiin reasoning sentenceyty\_\{t\}, and letntn\_\{t\}be the sentence length\. We define the sentence center by

ut=1nt​∑i=1ntht,i\(L\)∈ℝD\.u\_\{t\}=\\frac\{1\}\{n\_\{t\}\}\\sum\_\{i=1\}^\{n\_\{t\}\}h\_\{t,i\}^\{\(L\)\}\\in\\mathbb\{R\}^\{D\}\.\(12\)This provides a coarse\-grained anchor of the current reasoning unit in hidden space through simple sentence\-level pooling\.

#### Controller state\.

From the sentence\-center trajectory, SoT constructs the raw controller signature

st=\[δt,vt,ct,Ht\]⊤∈ℝ4,s\_\{t\}=\[\\delta\_\{t\},v\_\{t\},c\_\{t\},H\_\{t\}\]^\{\\top\}\\in\\mathbb\{R\}^\{4\},\(13\)which combines geometry, dynamics, and uncertainty into a compact endogenous control surface\.

The geometric coordinate is

δt=1nt​∑i=1nt‖ht,i\(L\)‖22−‖ut‖22,\\delta\_\{t\}=\\frac\{1\}\{n\_\{t\}\}\\sum\_\{i=1\}^\{n\_\{t\}\}\\\|h\_\{t,i\}^\{\(L\)\}\\\|\_\{2\}^\{2\}\-\\\|u\_\{t\}\\\|\_\{2\}^\{2\},\(14\)which measures within\-sentence dispersion around the sentence center and thus characterizes whether the current reasoning unit is internally concentrated or diffuse\.

The two dynamical coordinates are

vt=‖ut−ut−1‖2,v\_\{t\}=\\\|u\_\{t\}\-u\_\{t\-1\}\\\|\_\{2\},\(15\)and

ct=⟨ut−ut−1,ut−1−ut−2⟩‖ut−ut−1‖2⋅‖ut−1−ut−2‖2\+ε,c\_\{t\}=\\frac\{\\langle u\_\{t\}\-u\_\{t\-1\},\\,u\_\{t\-1\}\-u\_\{t\-2\}\\rangle\}\{\\\|u\_\{t\}\-u\_\{t\-1\}\\\|\_\{2\}\\cdot\\\|u\_\{t\-1\}\-u\_\{t\-2\}\\\|\_\{2\}\+\\varepsilon\},\(16\)wherevtv\_\{t\}measures the magnitude of state movement between adjacent reasoning units, whilectc\_\{t\}measures cosine alignment between consecutive displacements, namely whether the current step continues the recent trajectory direction or instead departs toward a new regime\. Together,vtv\_\{t\}andctc\_\{t\}separate movement from directionally stable movement\.

The uncertainty coordinate is

Ht=1nt​∑i=1ntℋ⁡\(pt,i\),H\_\{t\}=\\frac\{1\}\{n\_\{t\}\}\\sum\_\{i=1\}^\{n\_\{t\}\}\\mathcal\{H\}\(p\_\{t,i\}\),\(17\)whereℋ⁡\(⋅\)\\mathcal\{H\}\(\\cdot\)denotes Shannon entropy andpt,ip\_\{t,i\}is the next\-token predictive distribution at positionii\. It summarizes the model’s local predictive uncertainty over the current reasoning unit\.

These four coordinates are then normalized componentwise:

s^t=st−μσ\+ε,\\hat\{s\}\_\{t\}=\\frac\{s\_\{t\}\-\\mu\}\{\\sigma\+\\varepsilon\},\(18\)whereμ,σ∈ℝ4\\mu,\\sigma\\in\\mathbb\{R\}^\{4\}are the componentwise mean and standard deviation estimated from the offline training trajectories, andε\>0\\varepsilon\>0is a small constant for numerical stability\. The controller\-facing endogenous state is finally

mt=s^t∈ℝ4\.m\_\{t\}=\\hat\{s\}\_\{t\}\\in\\mathbb\{R\}^\{4\}\.\(19\)For boundary steps, we setv1=c1=0v\_\{1\}=c\_\{1\}=0andc2=0c\_\{2\}=0\.

The resulting state is intentionally minimal:δt\\delta\_\{t\}captures geometry,\(vt,ct\)\(v\_\{t\},c\_\{t\}\)capture dynamics, andHtH\_\{t\}captures uncertainty\. It is therefore not a handcrafted reasoning program, but a compact endogenous interface that answers four controller\-relevant questions: whether the current step is internally organized, whether the trajectory is moving, whether it remains directionally stable, and whether the model is locally uncertain\.

### B\.2Why four coordinates suffice for control

#### Control requirements\.

We define sufficiency with respect to SoT’s control problem: the backbone supplies semantic reasoning, while the state retains the feedback variables needed to organize prior evidence and decide whether reasoning should continue\. These two decisions require a snapshot of the current unit, a description of how the trajectory arrived there, and a readout of predictive readiness\. The four coordinates provide this control basis with distinct, complementary observables\.

Table 7:Complementary control roles of the four endogenous coordinates\.Coord\.ObservableControl question resolvedδt\\delta\_\{t\}Within\-unit dispersionIs the current representation internally concentrated or diffuse?vtv\_\{t\}Step displacementIs the trajectory advancing or nearly stationary?ctc\_\{t\}Directional persistenceIs it continuing along the preceding transition or redirecting at comparable speed?HtH\_\{t\}Predictive entropyIs the model’s local prediction decisive or unresolved?

The distinctions in Table[7](https://arxiv.org/html/2609.16055#A2.T7)cannot be recovered from one another\. In particular,vtv\_\{t\}measures how far the state moves but cannot distinguish continuation from reversal, which is the role ofctc\_\{t\}\. Likewise,δt\\delta\_\{t\}measures organization inside the current representation, whereasHtH\_\{t\}measures uncertainty in the predictive distribution; one describes representation geometry and the other output readiness\. Thus\(δt,Ht\)\(\\delta\_\{t\},H\_\{t\}\)provide complementary state snapshots, while\(vt,ct\)\(v\_\{t\},c\_\{t\}\)resolve the magnitude and direction of temporal change\.

#### From state to control\.

For evidence organization, SoT compares the current statemtm\_\{t\}with each historical statemjm\_\{j\}through\[mt;mj;mt−mj;mt⊙mj\]\[m\_\{t\};m\_\{j\};m\_\{t\}\-m\_\{j\};m\_\{t\}\\odot m\_\{j\}\]\. The differences expose regime change and the products expose coordinate\-wise compatibility, turning the four observables into a 16\-dimensional relational descriptor\. The stopping head readsmtm\_\{t\}directly\. The same compact state therefore supports two distinct actions: selecting history compatible with the trajectory’s present regime and terminating once its geometry, dynamics, and uncertainty indicate sufficient progress\.

#### Functional validation\.

We test whether each coordinate actually affects the deployed control law\. On held\-out selector decisions, one normalized coordinate is replaced by zero—its training\-population mean—while the other three coordinates, candidate texts, backbone, controller weights, and context budget are fixed\. We then rerun the selector and record whether its top\-ranked historical unit changes\. This intervention measures controller behavior directly and requires no task label\.

Table 8:Single\-coordinate interventions on evidence selection\.Coordinate fixedδt\\delta\_\{t\}vtv\_\{t\}ctc\_\{t\}HtH\_\{t\}Top\-1 selection changed26\.5%39\.1%56\.2%44\.7%

Every coordinate changes a substantial fraction of selections, with direction and uncertainty producing the largest rerouting rates\. A second intervention connects these decisions to generation outcomes\. On 120 examples from each of GSM8K, DROP, and MATH, zeroingδt\\delta\_\{t\}during generation lowers accuracy by 6\.7–22\.5 points on both Llama\-3\.1\-8B and Qwen2\.5\-14B\. Zeroingvtv\_\{t\}, by contrast, makes trajectories 4–5×\\timeslonger\. The separation matches the intended roles: dispersion is strongly coupled to answer quality, whereas displacement supplies a principal halting signal\. Together, the two intervention levels trace a coherent path from individual coordinates, through evidence selection, to accuracy and stopping\.

#### Excluding shortcut explanations\.

We examine alternatives at both the representation and controller levels\. First, token composition and sentence length explain at mostR2=0\.46R^\{2\}=0\.46ofδt\\delta\_\{t\}across eight backbones spanning 2B–72B\. Over the same scale range,ct≈−0\.43c\_\{t\}\\approx\-0\.43and remains negative after projecting out the mean drift and top five shared directions\. Hence dispersion is not reducible to surface composition, and directional redirection is not an artifact of a dominant shared direction\. Second, a diagnostic decomposition adds the four\-coordinate state to recency and semantic similarity when recovering the training\-side relevance rule; the state contributes0\.0440\.044–0\.1380\.138held\-out AUC on every tested backbone over 2B–47B\. This decomposition identifies information beyond the two generic heuristics, while an end\-task permutation supplies the independent compatibility test: we permute historical states across candidates while preserving each candidate’s text, position, and the inference budget\. Breaking only the state–unit correspondence reduces accuracy from 79\.0 to 62\.4 on GSM8K, 38\.6 to 24\.6 on DROP, and 60\.0 to 38\.5 on MATH\. The gains therefore arise from matching evidence to the evolving endogenous regime rather than from recency or surface similarity alone\.

#### Sufficiency and economy\.

The design argument and interventions support the same conclusion\. The four\-coordinate basis covers the controller’s required snapshot and transition variables; the relational map composes them for evidence selection, and the direct readout composes them for stopping\. This yields a 577\-parameter selection head plus a 5\-parameter stopping head—582 trainable parameters in total—without updating the backbone\. Its sufficiency is borne out operationally across heterogeneous reasoning structures: a compact endogenous state can organize evidence and termination while the backbone continues to supply the semantic reasoning itself\.

## Appendix CLightweight Training of State\-Conditioned Operators

This section closes the loop between Sec\.[2\.3](https://arxiv.org/html/2609.16055#S2.SS3), the runtime parameterizationsϕsel\\phi\_\{\\mathrm\{sel\}\}andϕstop\\phi\_\{\\mathrm\{stop\}\}, and the offline data used to fit them\. The backbone LLM remains frozen throughout; learning attaches only to the lightweight operators that instantiate𝒮\\mathcal\{S\}and𝒯\\mathcal\{T\}\.

#### Offline trajectory pool\.

We first roll out the frozen backbone on training problems, segment each transcript into sentence\-level reasoning unitsy1,…,yTy\_\{1\},\\dots,y\_\{T\}, and compute the corresponding statesmtm\_\{t\}as in Sec\.[B](https://arxiv.org/html/2609.16055#A2)\. The same trajectory collection is also used once to estimate the normalization statistics\(μ,σ\)\(\\mu,\\sigma\)in Eq\. \([18](https://arxiv.org/html/2609.16055#A2.E18)\), so training and deployment share a single state geometry\. For VLM transfer, the same train–evaluation separation is applied to multimodal offline trajectories: the image–text backbone remains frozen, evaluation examples are excluded, and fitting is confined to the lightweight controller bundle\.

Each stepttis stored together with its trajectory identity, step index, endogenous statemtm\_\{t\}, final outcome \(e\.g\., correctness\), and the paired historical set\{\(yj,mj\)\}j<t\\\{\(y\_\{j\},m\_\{j\}\)\\\}\_\{j<t\}\. This pairing is essential because evidence selection depends on\(mt,mj\)\(m\_\{t\},m\_\{j\}\), not on past text alone\. From lightweight offline replay on these trajectories, we derive two forms of supervision\. For evidence organization, replay compresses whether a historical unityjy\_\{j\}is both state\-compatible and incrementally useful for downstream reasoning, yielding a soft or hard retention targetb^t,j∈\[0,1\]\\hat\{b\}\_\{t,j\}\\in\[0,1\]\. For stopping, replay yields a targetzt∗∈\[0,1\]z\_\{t\}^\{\*\}\\in\[0,1\]that reflects sufficiency of accumulated support: on correct trajectories, stop mass appears only once the trace is mature and locally confident; on incorrect trajectories, it is deferred until the trace is clearly late and stagnating\. This prevents the stop head from degenerating into a trivial “stop whenever uncertain” rule\.

*Note on trajectory quality and generalization:*Trajectory quality affects how sharply the controller can be supervised, but SoT is not trained to mimic any fixed reasoning trace\. It only learns compact decisions about state\-conditioned evidence utility and stop readiness\. The trajectory pool and the evaluation suite are not identical, and SoT still transfers to datasets outside the trajectory\-collection set, which suggests that the learned control signal is not reducible to benchmark\-specific trajectory memorization\. This point is further strengthened by the training\-free variant in Sec\.[4\.1](https://arxiv.org/html/2609.16055#S4.SS1), where the same state\-conditioned principle remains effective even without learned controller fitting\. We additionally report a dedicated sensitivity study to trajectory\-pool quality in Appendix[P\.5](https://arxiv.org/html/2609.16055#A16.SS5)\.

#### Training the evidence\-organization operator\.

For each pair\(t,j\)\(t,j\)withj<tj<t, letdt,jd\_\{t,j\}be the descriptor in Eq\. \([7](https://arxiv.org/html/2609.16055#A1.E7)\)\. The trainable evidence score is

gt,j=ϕsel​\(dt,j,θ\),g\_\{t,j\}=\\phi\_\{\\mathrm\{sel\}\}\(d\_\{t,j\};\\theta\),\(20\)whereθ\\thetacollects the parameters of the lightweight selection head\. The evidence operator is fit by

ℒsel=∑t∑j<twt,j​ℓBCE​\(σ⁡\(gt,j\),b^t,j\)\+λsp​ℛsp\+λ2​‖θ‖22,\\mathcal\{L\}\_\{\\mathrm\{sel\}\}=\\sum\_\{t\}\\sum\_\{j<t\}w\_\{t,j\}\\,\\ell\_\{\\mathrm\{BCE\}\}\\\!\\bigl\(\\sigma\(g\_\{t,j\}\),\\hat\{b\}\_\{t,j\}\\bigr\)\+\\lambda\_\{\\mathrm\{sp\}\}\\mathcal\{R\}\_\{\\mathrm\{sp\}\}\+\\lambda\_\{2\}\\\|\\theta\\\|\_\{2\}^\{2\},\(21\)wherewt,jw\_\{t,j\}reweights rare positive supervision when needed,ℛsp\\mathcal\{R\}\_\{\\mathrm\{sp\}\}penalizes excessive batch\-level gate mass to prevent trivial dense retention, andλ2\\lambda\_\{2\}regularizes the controller parameters\. Supervision is concentrated on later steps of sufficiently long traces, where evidence selection materially changes the carried context rather than merely preserving the most recent sentence\.

#### Training the stopping operator\.

The stopping head uses the same interface as at inference:

ot=ϕstop​\(mt,ϕ\),ptstop=σ⁡\(ot\),o\_\{t\}=\\phi\_\{\\mathrm\{stop\}\}\(m\_\{t\};\\phi\),\\qquad p\_\{t\}^\{\\mathrm\{stop\}\}=\\sigma\(o\_\{t\}\),\(22\)whereϕ\\phidenotes the stop\-head parameters\. Given the replay\-derived stop targetzt∗z\_\{t\}^\{\*\}, the stopping loss is

ℒstop=∑tℓBCE​\(ptstop,zt∗\)\.\\mathcal\{L\}\_\{\\mathrm\{stop\}\}=\\sum\_\{t\}\\ell\_\{\\mathrm\{BCE\}\}\\\!\\bigl\(p\_\{t\}^\{\\mathrm\{stop\}\},z\_\{t\}^\{\*\}\\bigr\)\.\(23\)This target encodes evidence sufficiency rather than compliance with any external reasoning template\.

#### Overall objective\.

The final training objective is

ℒSoT=ℒsel\+λstop​ℒstop\.\\mathcal\{L\}\_\{\\mathrm\{SoT\}\}=\\mathcal\{L\}\_\{\\mathrm\{sel\}\}\+\\lambda\_\{\\mathrm\{stop\}\}\\mathcal\{L\}\_\{\\mathrm\{stop\}\}\.\(24\)Only the controller parameters\(θ,ϕ\)\(\\theta,\\phi\)are updated, while all backbone weights remain fixed\. Training therefore identifies a lightweight control law on the endogenous interface\(mt,\{mj\}j<t\)\(m\_\{t\},\\\{m\_\{j\}\\\}\_\{j<t\}\): it neither rewrites the model’s internal representations nor imitates a fixed multi\-step reasoning script\. The selector and stopping heads contain only 582 trainable parameters in total\. They are fit for 12 warm\-up epochs followed by 400 lightweight refinement steps over offline trajectories; no backbone gradients, evaluation examples, or test\-time rewards are used\. Across 5 problem\-held\-out refits, selector AUC is0\.779±0\.0050\.779\\pm 0\.005, indicating that the small control interface is stable across trajectory partitions rather than dependent on a favorable fit\. The trajectory\-quality analysis in Appendix[P\.5](https://arxiv.org/html/2609.16055#A16.SS5)further shows where weaker supervision primarily increases computation, while the training\-free results establish that the state\-conditioned principle does not depend on learned fitting alone\.

## Appendix DSoT under Limited Access and Limited Training

This section expands Section[4\.1](https://arxiv.org/html/2609.16055#S4.SS1)\. The goal of these variants is not to introduce new SoT formulations, but to probe the boundary of the SoT principle under weaker implementation conditions\. Concretely, we ask whether state\-conditioned evidence organization remains useful when either \(i\) the controller is not learned, or \(ii\) privileged hidden\-state access is unavailable\.

#### Training\-free SoT\.

Training\-free SoT preserves the same inference loop, the same sentence\-level reasoning unitsyty\_\{t\}, and the same endogenous state interfacemtm\_\{t\}, but removes learned controller parameters from both𝒮\\mathcal\{S\}and𝒯\\mathcal\{T\}\. Instead of using the learned evidence operator in Appendix[A](https://arxiv.org/html/2609.16055#A1), it scores each historical stepyjy\_\{j\}by a fixed state\-compatibility rule:

st,jtf=α1​⟨mt,mj⟩−α2​‖mt−mj‖2\+α3​cj−α4​Hj,s^\{\\mathrm\{tf\}\}\_\{t,j\}=\\alpha\_\{1\}\\langle m\_\{t\},m\_\{j\}\\rangle\-\\alpha\_\{2\}\\\|m\_\{t\}\-m\_\{j\}\\\|\_\{2\}\+\\alpha\_\{3\}c\_\{j\}\-\\alpha\_\{4\}H\_\{j\},\(25\)where⟨mt,mj⟩\\langle m\_\{t\},m\_\{j\}\\rangleencourages compatibility between the current and historical reasoning regimes,‖mt−mj‖2\\\|m\_\{t\}\-m\_\{j\}\\\|\_\{2\}penalizes mismatched states,cjc\_\{j\}favors directionally stable historical steps, andHjH\_\{j\}downweights locally uncertain ones\. Under the same context budgetBB, the deterministic evidence operator keeps the top\-scoring historical steps:

y~<ttf=𝒮tf​\(y<t,mt\)⊆y<t\.\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{tf\}\}=\\mathcal\{S\}\_\{\\mathrm\{tf\}\}\(y\_\{<t\};m\_\{t\}\)\\subseteq y\_\{<t\}\.\(26\)
Stopping is likewise made deterministic\. Rather than using the learned stop head, Training\-free SoT terminates only when the current state remains in a stable low\-uncertainty regime forrstopr\_\{\\mathrm\{stop\}\}consecutive steps:

zttf=𝒯tf\(mt−rstop\+1:t\)∈\{0,1\}\.z\_\{t\}^\{\\mathrm\{tf\}\}=\\mathcal\{T\}\_\{\\mathrm\{tf\}\}\(m\_\{t\-r\_\{\\mathrm\{stop\}\}\+1:t\}\)\\in\\\{0,1\\\}\.\(27\)In practice, this means that stopping requires jointly lowHtH\_\{t\}, sufficiently small state movementvtv\_\{t\}, and positive directional consistencyctc\_\{t\}over a short trailing window, so that the controller does not terminate on transient confidence alone\. This variant therefore isolates whether the*state\-conditioned reasoning principle*already helps before any controller fitting\.

#### SoT\-Embed \(sentence\-embedding trajectory\)\.

SoT\-Embed targets the same limited\-access setting as in the main text: internal hidden states are not exposed to the controller\. We adopt sentence\-level unitsy1,…,yTy\_\{1\},\\dots,y\_\{T\}that are mapped by a fixed sentence encoder to vectors

ut=Enc⁡\(yt\)∈ℝd,u\_\{t\}=\\mathrm\{Enc\}\(y\_\{t\}\)\\in\\mathbb\{R\}^\{d\},\(28\)and the matrixU∈ℝT×dU\\in\\mathbb\{R\}^\{T\\times d\}stacks the trajectory\. Control inputs are built*directly*from the embedding trajectory: the controller conditions onutu\_\{t\}and on compact trajectory summaries derived fromUU, such as a low\-dimensional subspace fit on the training distribution \(PCA coordinates per step\)\. Those features feed the same evidence and stopping interface as full SoT, so the loop remains state\-conditioned, but the state signal is read only from public text and embeddings\.

#### What these variants isolate\.

The two variants probe different boundary conditions\. Training\-free SoT asks whether SoT already has value as a fixed control principle before learning\. SoT\-Embed asks whether the paradigm fundamentally depends on privileged internal access when the control interface is replaced by embedding\-based trajectory features\. Together, they separate*endogenous reasoning as a principle*from*the strongest open\-weight implementation of that principle*\.

## Appendix ESoT\-Judge

This section expands Section[4\.2](https://arxiv.org/html/2609.16055#S4.SS2)\.SoT\-Judgetransfers the trajectory perspective of SoT from online reasoning control to post\-hoc black\-box evaluation\. The central question is whether the*observable*organization of a finished trace—without hidden states or extra generator calls—still carries enough signal to predict reference correctness\.

#### Trajectory from text only\.

Given a completed rollout from a closed or API\-accessed model, we segment the generated text into sentence\-level unitsy1,…,yTy\_\{1\},\\dots,y\_\{T\}and map them with a fixed sentence encoder to vectorsu1,…,uT∈ℝdu\_\{1\},\\dots,u\_\{T\}\\in\\mathbb\{R\}^\{d\}\. No architectural hooks are required: the judge consumes the same surface trace a human or downstream tool would see\.

#### Compact trajectory readout\.

LetU∈ℝT×dU\\in\\mathbb\{R\}^\{T\\times d\}stack the embeddings\. On the training split we fit a standard low\-dimensional subspace \(PCA\) and represent each step by its coordinates in that subspace\. We then pool the sequence into a short feature vectorφtraj\\varphi\_\{\\mathrm\{traj\}\}using elementary summaries along time \(e\.g\., coordinate\-wise means and standard deviations\) plus a few optional length statistics of the segmented text\. Auxiliary trajectory summaries can be appended for ablations, but the default recipe stays deliberately small: one fixed embedder, one subspace fit, and pooled statistics\.

#### Judge model\.

The judge is a lightweight predictorJψJ\_\{\\psi\}trained only on reference correctness labels:

r^=Jψ​\(φtraj\),\\hat\{r\}=J\_\{\\psi\}\\\!\\left\(\\varphi\_\{\\mathrm\{traj\}\}\\right\),\(29\)wherer^∈\[0,1\]\\hat\{r\}\\in\[0,1\]scores how likely the final answer is to be correct\. ThusSoT\-Judgereads*trajectory health*from the embedded sentence path, not from sample agreement, self\-critique, or an external judge prompt\.

#### What SoT\-Judge tests\.

The value ofSoT\-Judgeis both practical and conceptual\. Practically, it provides a lightweight black\-box alternative to majority\-vote style selection or stronger\-model judging\. Conceptually, it tests whether the SoT view survives the loss of online control: even when the model’s internals are hidden and intervention is impossible, reasoning may still be assessed through the organization of its trajectory rather than only through output agreement or a stronger judge prompt\. In this sense,SoT\-Judgeis not a separate reasoning paradigm, but a boundary\-case extension of the same SoT principle to post\-hoc evaluation\.

Part B: Theoretical Analysis

This part formalizes the theoretical logic underlying SoT as a reasoning\-control paradigm\. Here, we show a sharp structural point: once reasoning control is restricted to an external token\-chain policy class, the attainable optimum is constrained by the information available to that class, whereas endogenous state\-conditioned control can access a strictly richer decision interface on heterogeneous reasoning distributions\.

The argument proceeds in five steps\. Appendix[F](https://arxiv.org/html/2609.16055#A6)establishes policy\-class separation as the theoretical motivation\. Appendix[G](https://arxiv.org/html/2609.16055#A7)shows why evidence organization should depend on the realized reasoning state\. Appendix[H](https://arxiv.org/html/2609.16055#A8)shows why the selected support should generally be sparse rather than transcript\-complete\. Appendix[I](https://arxiv.org/html/2609.16055#A9)shows why the mechanism should be closed loop rather than one\-shot or fixed in advance\. Finally, Appendix[J](https://arxiv.org/html/2609.16055#A10)shows why stopping is not an auxiliary trick but part of the same bounded\-cost feedback control problem\.

## Appendix FPolicy\-Class Separation

We first formalize the theoretical starting point of the paper: the empirical gains of SoT suggest that reasoning improvements may saturate within a shared family of externally prescribed token\-chain policies, whereas state\-conditioned evidence organization opens a distinct control axis\. The goal of this section is not to claim that every external method is universally bounded by a single numeric ceiling, but to show that once the control variable is restricted to*external token\-chain policy classes*, the attainable optimum can be strictly below that of a state\-conditioned policy class\.

#### Setup\.

Letppdenote the input problem,y1:Ty\_\{1:T\}the reasoning trajectory, andaathe final answer\. For each steptt, letmtm\_\{t\}be the endogenous controller state\. We compare two policy classes\.

*External token\-chain policies*choose the next reasoning step from the prompt, observed token history, and an externally specified control variablec∈𝒞c\\in\\mathcal\{C\}:

ℙπ​\(yt∣p,y<t,c\)\.\\mathbb\{P\}\_\{\\pi\}\(y\_\{t\}\\mid p,y\_\{<t\},c\)\.\(30\)Hereccmay represent a prompting template, a planning script, a search budget, a branching rule, or any other externally imposed strategy variable, but it is fixed independently of the realized endogenous statemtm\_\{t\}\.

*State\-conditioned policies*instead choose the next\-step support through the current endogenous state:

ℙπ​\(yt∣p,y<t,mt\)\.\\mathbb\{P\}\_\{\\pi\}\(y\_\{t\}\\mid p,y\_\{<t\},m\_\{t\}\)\.\(31\)This is the policy class instantiated by SoT through the state\-conditioned operators\.

#### One\-step control utility\.

For a realized steptt, letutu\_\{t\}be a bounded one\-step utility, which may be instantiated as expected correctness gain, negative step loss, or any equivalent bounded progress signal\. For any admissible control variablextx\_\{t\}, define

Jt\(xt\)=𝔼\[ut∣p,y<t,xt\]\.J\_\{t\}\(x\_\{t\}\)=\\mathbb\{E\}\\\!\\left\[u\_\{t\}\\mid p,y\_\{<t\},x\_\{t\}\\right\]\.\(32\)Whenxt=cx\_\{t\}=c, this is the utility under an external token\-chain policy; whenxt=mtx\_\{t\}=m\_\{t\}, it is the utility under a state\-conditioned policy\.

#### Assumption F\.1 \(State\-relevant heterogeneity\)\.

There exists a positive\-probability set of realized prefixes\(p,y<t\)\(p,y\_\{<t\}\)for which two statesm,m′m,m^\{\\prime\}can arise such that the Bayes\-optimal next\-step support differs:

argmaxy~<t𝔼\[ut∣p,y<t,mt=m,y~<t\]≠argmaxy~<t𝔼\[ut∣p,y<t,mt=m′,y~<t\]\.\\arg\\max\_\{\\widetilde\{y\}\_\{<t\}\}\\mathbb\{E\}\\\!\\left\[u\_\{t\}\\mid p,y\_\{<t\},m\_\{t\}=m,\\widetilde\{y\}\_\{<t\}\\right\]\\neq\\arg\\max\_\{\\widetilde\{y\}\_\{<t\}\}\\mathbb\{E\}\\\!\\left\[u\_\{t\}\\mid p,y\_\{<t\},m\_\{t\}=m^\{\\prime\},\\widetilde\{y\}\_\{<t\}\\right\]\.\(33\)In words, even under the same problem and token prefix, different endogenous reasoning regimes can require different evidence support for the next decision\.

#### Assumption F\.2 \(External\-state mismatch\)\.

There exists a positive\-probability set of realized prefixes\(p,y<t\)\(p,y\_\{<t\}\)such that no externally specified control variablec∈𝒞c\\in\\mathcal\{C\}uniquely identifies the realized state\-relevant optimum in \([33](https://arxiv.org/html/2609.16055#A6.E33)\)\. Equivalently, within that set, at least two realizations with different optimal evidence support are indistinguishable to the external policy class \([30](https://arxiv.org/html/2609.16055#A6.E30)\)\.

#### Proposition F\.1 \(Strict policy\-class separation\)\.

Under Assumptions F\.1 and F\.2, the optimal expected one\-step utility achievable by the external token\-chain policy class is strictly below that achievable by the state\-conditioned policy class:

supπ∈Πext𝔼⁡\[ut\]<supπ∈Πstate𝔼⁡\[ut\],\\sup\_\{\\pi\\in\\Pi\_\{\\mathrm\{ext\}\}\}\\mathbb\{E\}\[u\_\{t\}\]\\;<\\;\\sup\_\{\\pi\\in\\Pi\_\{\\mathrm\{state\}\}\}\\mathbb\{E\}\[u\_\{t\}\],\(34\)whereΠext\\Pi\_\{\\mathrm\{ext\}\}denotes the class in \([30](https://arxiv.org/html/2609.16055#A6.E30)\) andΠstate\\Pi\_\{\\mathrm\{state\}\}the class in \([31](https://arxiv.org/html/2609.16055#A6.E31)\)\.

#### Proof\.

Fix a realized prefix\(p,y<t\)\(p,y\_\{<t\}\)in the positive\-probability set from Assumptions F\.1 and F\.2\. By Assumption F\.1, there exist two possible realized statesm≠m′m\\neq m^\{\\prime\}under this same prefix for which the Bayes\-optimal evidence support differs\. By Assumption F\.2, the external class cannot distinguish these two cases through any externally specified variablecc\. Hence any policy inΠext\\Pi\_\{\\mathrm\{ext\}\}must assign the same control decision to both realizations once\(p,y<t,c\)\(p,y\_\{<t\},c\)is fixed\. Therefore it cannot be simultaneously optimal for both state realizations\. As a result, on this positive\-probability set, every external token\-chain policy incurs a strictly positive conditional utility gap relative to the Bayes\-optimal state\-aware decision\.

By contrast, the state\-conditioned classΠstate\\Pi\_\{\\mathrm\{state\}\}conditions directly onmtm\_\{t\}, so it can choose the Bayes\-optimal support separately for the realizationmt=mm\_\{t\}=mand for the realizationmt=m′m\_\{t\}=m^\{\\prime\}\. HenceΠstate\\Pi\_\{\\mathrm\{state\}\}contains a policy that attains the Bayes\-optimal conditional utility on that set\.

Since the set has positive probability and the utility gap is strict on that set, the gap remains strict after averaging over the data\-generating distribution\. This yields \([34](https://arxiv.org/html/2609.16055#A6.E34)\)\.□\\square

#### Implication\.

Proposition F\.1 gives the theoretical motivation for the paper\. It does not say that any particular baseline is weak in isolation, nor that external token\-chain reasoning can never improve performance\. Rather, it shows that when reasoning control is restricted to an external policy class, the attainable optimum is constrained by the information available to that class\. If the next reasoning decision depends on the realized endogenous state, then changing prompts, adding scripts, or enlarging search may still leave the controller inside the same external class\. SoT improves along an orthogonal axis: it changes the control variable itself, from externally prescribed token programs to endogenous state\-conditioned evidence organization\.

The remaining sections characterize the structure of that advantage: state conditioning, sparsity, feedback, and stopping\.

## Appendix GState\-Conditioned Evidence Organization

This section formalizes the first theoretical claim of SoT: when the usefulness of historical reasoning evidence varies across the model’s current reasoning regime, a fixed evidence strategy that ignores the endogenous state is generally suboptimal, whereas a state\-conditioned operator𝒮⁡\(y<t,mt\)\\mathcal\{S\}\(y\_\{<t\};m\_\{t\}\)can be optimal ifmtm\_\{t\}is sufficient for control\.

#### Setup\.

At reasoning steptt, lety<ty\_\{<t\}denote the available reasoning history, and lety~<t⊆y<t\\widetilde\{y\}\_\{<t\}\\subseteq y\_\{<t\}be the subset retained for the next decision\. To make explicit that different hidden reasoning regimes may require different support, introduce a latent regime variableωt\\omega\_\{t\}\. For any candidate subsety~<t⊆y<t\\widetilde\{y\}\_\{<t\}\\subseteq y\_\{<t\}, define the one\-step conditional utility

Vt\(y~<t;y<t,ωt\):=𝔼\[ut∣y<t,ωt,y~<t\],V\_\{t\}\(\\widetilde\{y\}\_\{<t\};y\_\{<t\},\\omega\_\{t\}\):=\\mathbb\{E\}\\\!\\left\[u\_\{t\}\\mid y\_\{<t\},\\ \\omega\_\{t\},\\ \\widetilde\{y\}\_\{<t\}\\right\],\(35\)whereutu\_\{t\}is any bounded one\-step utility for the next reasoning decision, such as negative next\-step loss, expected correctness gain, or any equivalent bounded surrogate\. The regime\-optimal retained subset is

y~<t⋆​\(y<t,ωt\)∈arg⁡maxy~<t⊆y<t​Vt​\(y~<t,y<t,ωt\)\.\\widetilde\{y\}\_\{<t\}^\{\\star\}\(y\_\{<t\},\\omega\_\{t\}\)\\in\\arg\\max\_\{\\widetilde\{y\}\_\{<t\}\\subseteq y\_\{<t\}\}V\_\{t\}\(\\widetilde\{y\}\_\{<t\};y\_\{<t\},\\omega\_\{t\}\)\.\(36\)

#### Assumption G\.1 \(Heterogeneous evidence demand\)\.

There exist a realized historyy¯\\bar\{y\}, two distinct regimesω≠ω′\\omega\\neq\\omega^\{\\prime\}, and positive\-probability events\{y<t=y¯,ωt=ω\}\\\{y\_\{<t\}=\\bar\{y\},\\omega\_\{t\}=\\omega\\\}and\{y<t=y¯,ωt=ω′\}\\\{y\_\{<t\}=\\bar\{y\},\\omega\_\{t\}=\\omega^\{\\prime\}\\\}, such that the corresponding optimizers in Eq\. \([36](https://arxiv.org/html/2609.16055#A7.E36)\) are unique and distinct:

y~⋆​\(y¯,ω\)≠y~⋆​\(y¯,ω′\)\.\\widetilde\{y\}^\{\\star\}\(\\bar\{y\},\\omega\)\\neq\\widetilde\{y\}^\{\\star\}\(\\bar\{y\},\\omega^\{\\prime\}\)\.\(37\)

#### Assumption G\.2 \(State sufficiency for evidence control\)\.

There exists a measurable mapψ\\psisuch that, almost surely,

y~<t⋆​\(y<t,ωt\)=ψ⁡\(mt,y<t\)\.\\widetilde\{y\}\_\{<t\}^\{\\star\}\(y\_\{<t\},\\omega\_\{t\}\)=\\psi\(m\_\{t\},y\_\{<t\}\)\.\(38\)This assumption does not requiremtm\_\{t\}to reconstruct the full internal state of the backbone\. It only requiresmtm\_\{t\}to be sufficient for choosing the evidence subset relevant to the next decision\.

#### Proposition G\.1\.

Under Assumptions G\.1–G\.2, any fixed evidence operator𝒮fix:y<t↦y~<t\\mathcal\{S\}\_\{\\mathrm\{fix\}\}:y\_\{<t\}\\mapsto\\widetilde\{y\}\_\{<t\}that ignoresmtm\_\{t\}is strictly suboptimal on the heterogeneous distribution over\(y<t,ωt\)\(y\_\{<t\},\\omega\_\{t\}\)\. By contrast, the class of state\-conditioned operators\(mt,y<t\)↦y~<t\(m\_\{t\},y\_\{<t\}\)\\mapsto\\widetilde\{y\}\_\{<t\}contains a Bayes\-optimal evidence selector\.

#### Proof\.

Consider the realized historyy¯\\bar\{y\}from Assumption G\.1\. A fixed evidence operator must output a single subset𝒮fix​\(y¯\)\\mathcal\{S\}\_\{\\mathrm\{fix\}\}\(\\bar\{y\}\), independent of the realized regime\. Because the optimizersy~⋆​\(y¯,ω\)\\widetilde\{y\}^\{\\star\}\(\\bar\{y\},\\omega\)andy~⋆​\(y¯,ω′\)\\widetilde\{y\}^\{\\star\}\(\\bar\{y\},\\omega^\{\\prime\}\)are unique and distinct,𝒮fix​\(y¯\)\\mathcal\{S\}\_\{\\mathrm\{fix\}\}\(\\bar\{y\}\)cannot equal both\. Hence at least one of the following strict inequalities must hold:

Vt​\(𝒮fix​\(y¯\),y¯,ω\)<Vt​\(y~⋆​\(y¯,ω\),y¯,ω\),V\_\{t\}\\\!\\bigl\(\\mathcal\{S\}\_\{\\mathrm\{fix\}\}\(\\bar\{y\}\);\\bar\{y\},\\omega\\bigr\)<V\_\{t\}\\\!\\bigl\(\\widetilde\{y\}^\{\\star\}\(\\bar\{y\},\\omega\);\\bar\{y\},\\omega\\bigr\),\(39\)or

Vt​\(𝒮fix​\(y¯\),y¯,ω′\)<Vt​\(y~⋆​\(y¯,ω′\),y¯,ω′\)\.V\_\{t\}\\\!\\bigl\(\\mathcal\{S\}\_\{\\mathrm\{fix\}\}\(\\bar\{y\}\);\\bar\{y\},\\omega^\{\\prime\}\\bigr\)<V\_\{t\}\\\!\\bigl\(\\widetilde\{y\}^\{\\star\}\(\\bar\{y\},\\omega^\{\\prime\}\);\\bar\{y\},\\omega^\{\\prime\}\\bigr\)\.\(40\)Since both regime\-history events occur with positive probability, averaging over the data\-generating distribution yields a strictly positive expected utility gap between any fixed operator and the Bayes\-optimal rule\.

Now consider the state\-conditioned operator class\. By Assumption G\.2, the mapψ⁡\(mt,y<t\)\\psi\(m\_\{t\},y\_\{<t\}\)recovers the optimal subsety~<t⋆​\(y<t,ωt\)\\widetilde\{y\}\_\{<t\}^\{\\star\}\(y\_\{<t\},\\omega\_\{t\}\)almost surely\. Therefore the class\(mt,y<t\)↦y~<t\(m\_\{t\},y\_\{<t\}\)\\mapsto\\widetilde\{y\}\_\{<t\}contains a Bayes\-optimal selector\.□\\square

#### Implication\.

Proposition G\.1 gives the basic control\-theoretic justification for SoT\. If the evidence required for the next reasoning step changes with the current reasoning regime, then no single fixed evidence strategy can be uniformly optimal\. The control variable must therefore depend on the current endogenous state\. In this sense, SoT is not merely replacing one reasoning template with another; it is changing the problem from*fixed strategy execution*to*state\-conditioned evidence organization*\.

## Appendix HSparse Evidence Organization

This section formalizes the second theoretical claim of SoT: once evidence organization is conditioned on the current reasoning regime, the optimal operator should in general be sparse rather than full\-history preserving\. The reason is not only computational\. Historical evidence can be simultaneously helpful, neutral, or actively interfering for the next decision, and carrying the full history may therefore reduce both step utility and efficiency\.

#### Setup\.

We continue the one\-step formulation from Appendix[G](https://arxiv.org/html/2609.16055#A7)\. At steptt, lety<ty\_\{<t\}be the available reasoning history, letωt\\omega\_\{t\}denote the latent reasoning regime, and lety~<t⊆y<t\\widetilde\{y\}\_\{<t\}\\subseteq y\_\{<t\}be the subset retained for the next decision\. For any retained subset, the one\-step conditional utility is

Vt\(y~<t;y<t,ωt\):=𝔼\[ut∣y<t,ωt,y~<t\]\.V\_\{t\}\(\\widetilde\{y\}\_\{<t\};y\_\{<t\},\\omega\_\{t\}\):=\\mathbb\{E\}\\\!\\left\[u\_\{t\}\\mid y\_\{<t\},\\ \\omega\_\{t\},\\ \\widetilde\{y\}\_\{<t\}\\right\]\.\(41\)

#### Assumption H\.1 \(Helpful and interfering evidence\)\.

For each realized pair\(y<t,ωt\)\(y\_\{<t\},\\omega\_\{t\}\), the history admits a disjoint decomposition

y<t=y<t\+​\(ωt\)​∪˙​y<t−​\(ωt\),y\_\{<t\}=y\_\{<t\}^\{\+\}\(\\omega\_\{t\}\)\\,\\dot\{\\cup\}\\,y\_\{<t\}^\{\-\}\(\\omega\_\{t\}\),\(42\)wherey<t\+​\(ωt\)y\_\{<t\}^\{\+\}\(\\omega\_\{t\}\)contains evidence that is helpful for the next decision under regimeωt\\omega\_\{t\}, andy<t−​\(ωt\)y\_\{<t\}^\{\-\}\(\\omega\_\{t\}\)contains evidence that is irrelevant, misleading, or distracting under that regime\.

#### Assumption H\.2 \(Utility decomposition\)\.

There exist functionsRtR\_\{t\},ItI\_\{t\}, and a constantλ\>0\\lambda\>0such that

Vt​\(y~<t,y<t,ωt\)=Rt​\(y~<t∩y<t\+​\(ωt\),y<t,ωt\)−It​\(y~<t∩y<t−​\(ωt\),y<t,ωt\)−λ​\|y~<t\|,V\_\{t\}\(\\widetilde\{y\}\_\{<t\};y\_\{<t\},\\omega\_\{t\}\)=R\_\{t\}\\\!\\bigl\(\\widetilde\{y\}\_\{<t\}\\cap y\_\{<t\}^\{\+\}\(\\omega\_\{t\}\);\\,y\_\{<t\},\\omega\_\{t\}\\bigr\)\-I\_\{t\}\\\!\\bigl\(\\widetilde\{y\}\_\{<t\}\\cap y\_\{<t\}^\{\-\}\(\\omega\_\{t\}\);\\,y\_\{<t\},\\omega\_\{t\}\\bigr\)\-\\lambda\|\\widetilde\{y\}\_\{<t\}\|,\(43\)whereRtR\_\{t\}is monotone nondecreasing in its set argument andItI\_\{t\}is monotone nondecreasing in its set argument\. Thus, retaining more helpful evidence cannot reduce reward, retaining more interfering evidence cannot reduce interference, and carrying more context incurs a positive efficiency cost\.

#### Proposition H\.1\.

Fix a realized pair\(y<t,ωt\)\(y\_\{<t\},\\omega\_\{t\}\)\. Let the full\-history operator retain

y~<tfull=y<t,\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{full\}\}=y\_\{<t\},\(44\)and let a sparse operator retain somey~<tsp⊆y<t\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{sp\}\}\\subseteq y\_\{<t\}such that

y~<tsp∩y<t\+​\(ωt\)=y<t\+​\(ωt\),\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{sp\}\}\\cap y\_\{<t\}^\{\+\}\(\\omega\_\{t\}\)=y\_\{<t\}^\{\+\}\(\\omega\_\{t\}\),\(45\)and

y~<tsp∩y<t−​\(ωt\)⊊y<t−​\(ωt\)\.\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{sp\}\}\\cap y\_\{<t\}^\{\-\}\(\\omega\_\{t\}\)\\subsetneq y\_\{<t\}^\{\-\}\(\\omega\_\{t\}\)\.\(46\)Then

Vt​\(y~<tsp,y<t,ωt\)\>Vt​\(y~<tfull,y<t,ωt\)\.V\_\{t\}\(\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{sp\}\};y\_\{<t\},\\omega\_\{t\}\)\>V\_\{t\}\(\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{full\}\};y\_\{<t\},\\omega\_\{t\}\)\.\(47\)

#### Proof\.

Under the full\-history operator, Eq\. \([43](https://arxiv.org/html/2609.16055#A8.E43)\) gives

Vt​\(y~<tfull,y<t,ωt\)=Rt​\(y<t\+​\(ωt\),y<t,ωt\)−It​\(y<t−​\(ωt\),y<t,ωt\)−λ​\|y<t\|\.V\_\{t\}\(\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{full\}\};y\_\{<t\},\\omega\_\{t\}\)=R\_\{t\}\\\!\\bigl\(y\_\{<t\}^\{\+\}\(\\omega\_\{t\}\);y\_\{<t\},\\omega\_\{t\}\\bigr\)\-I\_\{t\}\\\!\\bigl\(y\_\{<t\}^\{\-\}\(\\omega\_\{t\}\);y\_\{<t\},\\omega\_\{t\}\\bigr\)\-\\lambda\|y\_\{<t\}\|\.\(48\)For the sparse operator, condition \([45](https://arxiv.org/html/2609.16055#A8.E45)\) implies that the helpful\-evidence term is unchanged:

Rt​\(y~<tsp∩y<t\+​\(ωt\),y<t,ωt\)=Rt​\(y<t\+​\(ωt\),y<t,ωt\)\.R\_\{t\}\\\!\\bigl\(\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{sp\}\}\\cap y\_\{<t\}^\{\+\}\(\\omega\_\{t\}\);y\_\{<t\},\\omega\_\{t\}\\bigr\)=R\_\{t\}\\\!\\bigl\(y\_\{<t\}^\{\+\}\(\\omega\_\{t\}\);y\_\{<t\},\\omega\_\{t\}\\bigr\)\.\(49\)Condition \([46](https://arxiv.org/html/2609.16055#A8.E46)\) implies that fewer interfering units are retained, so by monotonicity ofItI\_\{t\},

It​\(y~<tsp∩y<t−​\(ωt\),y<t,ωt\)≤It​\(y<t−​\(ωt\),y<t,ωt\)\.I\_\{t\}\\\!\\bigl\(\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{sp\}\}\\cap y\_\{<t\}^\{\-\}\(\\omega\_\{t\}\);y\_\{<t\},\\omega\_\{t\}\\bigr\)\\leq I\_\{t\}\\\!\\bigl\(y\_\{<t\}^\{\-\}\(\\omega\_\{t\}\);y\_\{<t\},\\omega\_\{t\}\\bigr\)\.\(50\)Moreover,

\|y~<tsp\|<\|y<t\|,\|\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{sp\}\}\|<\|y\_\{<t\}\|,\(51\)because at least one interfering unit is removed while all helpful units are preserved\. Substituting Eqs\. \([49](https://arxiv.org/html/2609.16055#A8.E49)\)–\([51](https://arxiv.org/html/2609.16055#A8.E51)\) into Eq\. \([43](https://arxiv.org/html/2609.16055#A8.E43)\) shows that the reward term is unchanged, the interference term does not increase, and the cost term strictly decreases byλ⁡\(\|y<t\|−\|y~<tsp\|\)\>0\\lambda\\bigl\(\|y\_\{<t\}\|\-\|\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{sp\}\}\|\\bigr\)\>0\. Hence

Vt​\(y~<tsp,y<t,ωt\)−Vt​\(y~<tfull,y<t,ωt\)≥λ⁡\(\|y<t\|−\|y~<tsp\|\)\>0,V\_\{t\}\(\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{sp\}\};y\_\{<t\},\\omega\_\{t\}\)\-V\_\{t\}\(\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{full\}\};y\_\{<t\},\\omega\_\{t\}\)\\geq\\lambda\\bigl\(\|y\_\{<t\}\|\-\|\\widetilde\{y\}\_\{<t\}^\{\\mathrm\{sp\}\}\|\\bigr\)\>0,\(52\)which proves Eq\. \([47](https://arxiv.org/html/2609.16055#A8.E47)\)\.□\\square

#### Implication\.

Proposition H\.1 clarifies why SoT should be sparse\. Once evidence usefulness is regime\-dependent, carrying the full history is generally not neutral: it preserves helpful support, but it also preserves regime\-mismatched interference and incurs additional cost\. A state\-aligned sparse operator improves the next reasoning decision not by generic compression, but by preserving the evidence that remains decision\-relevant under the current endogenous state while removing distracting history\. This is precisely why SoT can improve accuracy and efficiency jointly rather than trading one for the other\.

## Appendix IClosed\-Loop Reasoning Control

This section formalizes the third theoretical claim of SoT: once evidence organization is state\-conditioned and sparse, it should also be*closed loop*\. That is, the reasoning controller should adapt its evidence\-organization and continuation decisions to the realized endogenous statemtm\_\{t\}, rather than committing in advance to a fixed schedule\.

#### Setup\.

At reasoning steptt, let the control action be

at:=\(y~<t,zt\),a\_\{t\}:=\\bigl\(\\widetilde\{y\}\_\{<t\},z\_\{t\}\\bigr\),\(53\)wherey~<t⊆y<t\\widetilde\{y\}\_\{<t\}\\subseteq y\_\{<t\}is the retained historical evidence andzt∈\{0,1\}z\_\{t\}\\in\\\{0,1\\\}is the continuation decision, withzt=0z\_\{t\}=0meaning continue andzt=1z\_\{t\}=1meaning stop\. The endogenous state evolves according to the controlled stochastic dynamics

mt\+1=Ft​\(mt,at,εt\),m\_\{t\+1\}=F\_\{t\}\(m\_\{t\},a\_\{t\},\\varepsilon\_\{t\}\),\(54\)whereεt\\varepsilon\_\{t\}collects the randomness induced by token generation and environment variation\. Letrt​\(mt,at\)r\_\{t\}\(m\_\{t\},a\_\{t\}\)denote the one\-step reward, which may incorporate reasoning progress, answer quality, interference reduction, and compute cost\. For a finite horizonTT, define the value of a policyπ\\pifrom statemmat stepttas

Vtπ​\(m\):=𝔼π​\[∑τ=tTrτ​\(mτ,aτ\)\|mt=m\]\.V\_\{t\}^\{\\pi\}\(m\):=\\mathbb\{E\}^\{\\pi\}\\\!\\left\[\\sum\_\{\\tau=t\}^\{T\}r\_\{\\tau\}\(m\_\{\\tau\},a\_\{\\tau\}\)\\,\\middle\|\\,m\_\{t\}=m\\right\]\.\(55\)

#### Open\-loop and closed\-loop policies\.

An*open\-loop*policy fixes its actions in advance as a deterministic schedule depending only on the step index:

atopen=a¯t,a\_\{t\}^\{\\mathrm\{open\}\}=\\bar\{a\}\_\{t\},\(56\)wherea¯t\\bar\{a\}\_\{t\}does not depend on the realized statemtm\_\{t\}\. A*closed\-loop*policy instead adapts to the realized endogenous state:

atcl=πt​\(mt\)\.a\_\{t\}^\{\\mathrm\{cl\}\}=\\pi\_\{t\}\(m\_\{t\}\)\.\(57\)Thus, open\-loop control corresponds to a fixed external reasoning schedule, while closed\-loop control conditions evidence organization and stopping on how reasoning has actually unfolded\.

#### Proposition I\.1\.

Let

Vtcl​\(m\):=supπ∈ΠclVtπ​\(m\),Vtopen​\(m\):=supπ∈ΠopenVtπ​\(m\),V\_\{t\}^\{\\mathrm\{cl\}\}\(m\):=\\sup\_\{\\pi\\in\\Pi\_\{\\mathrm\{cl\}\}\}V\_\{t\}^\{\\pi\}\(m\),\\qquad V\_\{t\}^\{\\mathrm\{open\}\}\(m\):=\\sup\_\{\\pi\\in\\Pi\_\{\\mathrm\{open\}\}\}V\_\{t\}^\{\\pi\}\(m\),\(58\)whereΠcl\\Pi\_\{\\mathrm\{cl\}\}andΠopen\\Pi\_\{\\mathrm\{open\}\}denote the closed\-loop and open\-loop policy classes defined above\. Then for every stepttand every statemm,

Vtcl​\(m\)≥Vtopen​\(m\)\.V\_\{t\}^\{\\mathrm\{cl\}\}\(m\)\\geq V\_\{t\}^\{\\mathrm\{open\}\}\(m\)\.\(59\)Moreover, if there exists a positive\-probability set of realized statesAAsuch that the continuation\-optimal action at stepttis not constant overAA, then the inequality is strict in expectation over that set\.

#### Proof\.

Every open\-loop policy is a special case of a closed\-loop policy\. Indeed, for any fixed schedule\{a¯τ\}τ=tT∈Πopen\\\{\\bar\{a\}\_\{\\tau\}\\\}\_\{\\tau=t\}^\{T\}\\in\\Pi\_\{\\mathrm\{open\}\}, define the state\-independent closed\-loop policy

πτ​\(m\)≡a¯τfor all​m​and​τ≥t\.\\pi\_\{\\tau\}\(m\)\\equiv\\bar\{a\}\_\{\\tau\}\\qquad\\text\{for all \}m\\text\{ and \}\\tau\\geq t\.\(60\)This policy belongs toΠcl\\Pi\_\{\\mathrm\{cl\}\}and induces exactly the same controlled process and therefore the same value as the original open\-loop schedule\. Hence

Πopen⊆Πcl,\\Pi\_\{\\mathrm\{open\}\}\\subseteq\\Pi\_\{\\mathrm\{cl\}\},\(61\)which immediately implies Eq\. \([59](https://arxiv.org/html/2609.16055#A9.E59)\)\.

To show when the inequality is strict, consider the optimal state\-action value at steptt,

Qt\(m,a\):=rt\(m,a\)\+𝔼\[Vt\+1cl\(mt\+1\)\|mt=m,at=a\]\.Q\_\{t\}\(m,a\):=r\_\{t\}\(m,a\)\+\\mathbb\{E\}\\\!\\left\[V\_\{t\+1\}^\{\\mathrm\{cl\}\}\(m\_\{t\+1\}\)\\,\\middle\|\\,m\_\{t\}=m,\\ a\_\{t\}=a\\right\]\.\(62\)Suppose there exists a measurable setAAwith positive probability such that for some two statesm,m′∈Am,m^\{\\prime\}\\in A,

arg⁡maxa​Qt​\(m,a\)≠arg⁡maxa​Qt​\(m′,a\)\.\\arg\\max\_\{a\}Q\_\{t\}\(m,a\)\\neq\\arg\\max\_\{a\}Q\_\{t\}\(m^\{\\prime\},a\)\.\(63\)Then no single open\-loop actiona¯t\\bar\{a\}\_\{t\}can be optimal for all realized states inAA\. A closed\-loop policy can instead choose, at each realized statemt∈Am\_\{t\}\\in A, an action inarg⁡maxa​Qt​\(mt,a\)\\arg\\max\_\{a\}Q\_\{t\}\(m\_\{t\},a\), while matching the open\-loop schedule outsideAA\. This yields a strictly largerQt​\(mt,at\)Q\_\{t\}\(m\_\{t\},a\_\{t\}\)on a positive\-probability subset ofAA, and no smaller value elsewhere\. Therefore the expected value under the best closed\-loop policy is strictly larger than that under the best open\-loop policy\.□\\square

#### Implication\.

Proposition I\.1 explains why SoT should be closed loop rather than one\-shot\. When the best evidence support or continuation decision varies with the realized endogenous state, any fixed external reasoning schedule is necessarily unable to adapt to all realized reasoning regimes\. Closed\-loop control strictly enlarges the feasible policy class and becomes strictly better whenever different realized states require different actions\. This is the formal reason SoT is not merely another pre\-specified reasoning program: it treats reasoning as adaptive control over evidence organization and stopping under the evolving endogenous state\.

## Appendix JState\-Driven Stopping

This section formalizes the final theoretical claim of SoT: stopping is not an auxiliary heuristic, but part of the same state\-conditioned control problem as evidence organization\. Once reasoning incurs cost, the controller should decide from the current endogenous statemtm\_\{t\}not only what evidence to retain, but also whether another reasoning step is worth taking\.

#### Setup\.

At steptt, letzt∈\{0,1\}z\_\{t\}\\in\\\{0,1\\\}denote the stopping decision, wherezt=0z\_\{t\}=0means continue reasoning andzt=1z\_\{t\}=1means stop and emit the final answeraa\. LetVt⋆​\(m\)V\_\{t\}^\{\\star\}\(m\)denote the optimal bounded\-cost value starting from statemmat steptt\. We decompose the value of the two admissible actions as

Vt⋆​\(m\)=max⁡\{Vtstop​\(m\),Vtcont​\(m\)\},V\_\{t\}^\{\\star\}\(m\)=\\max\\\!\\left\\\{V\_\{t\}^\{\\mathrm\{stop\}\}\(m\),\\;V\_\{t\}^\{\\mathrm\{cont\}\}\(m\)\\right\\\},\(64\)whereVtstop​\(m\)V\_\{t\}^\{\\mathrm\{stop\}\}\(m\)is the value of stopping immediately and answering from the current active context, and

Vtcont\(m\)=supπ𝔼π\[rtcont\+Vt\+1⋆\(mt\+1\)\|mt=m,zt=0\]V\_\{t\}^\{\\mathrm\{cont\}\}\(m\)=\\sup\_\{\\pi\}\\mathbb\{E\}^\{\\pi\}\\\!\\left\[r\_\{t\}^\{\\mathrm\{cont\}\}\+V\_\{t\+1\}^\{\\star\}\(m\_\{t\+1\}\)\\,\\middle\|\\,m\_\{t\}=m,\\ z\_\{t\}=0\\right\]\(65\)is the optimal value of continuing for one more step and then behaving optimally thereafter\. Herertcontr\_\{t\}^\{\\mathrm\{cont\}\}already includes the marginal computation cost of taking another reasoning step\.

#### Continuation advantage\.

Define the continuation advantage

Δt​\(m\):=Vtcont​\(m\)−Vtstop​\(m\)\.\\Delta\_\{t\}\(m\):=V\_\{t\}^\{\\mathrm\{cont\}\}\(m\)\-V\_\{t\}^\{\\mathrm\{stop\}\}\(m\)\.\(66\)The sign ofΔt​\(m\)\\Delta\_\{t\}\(m\)determines whether an additional step is worthwhile under the current endogenous state: continuation is beneficial whenΔt​\(m\)\>0\\Delta\_\{t\}\(m\)\>0, and stopping is optimal whenΔt​\(m\)≤0\\Delta\_\{t\}\(m\)\\leq 0\.

#### Proposition J\.1\.

Assume thatmtm\_\{t\}is sufficient for the stopping decision in the sense that bothVtstopV\_\{t\}^\{\\mathrm\{stop\}\}andVtcontV\_\{t\}^\{\\mathrm\{cont\}\}are measurable functions ofmtm\_\{t\}\. Then an optimal bounded\-cost stopping rule is state\-conditioned and can be written as

zt⋆\(mt\)=\{Δt\(mt\)≤0\}\.z\_\{t\}^\{\\star\}\(m\_\{t\}\)=\\mathbf\{1\}\\\!\\left\\\{\\Delta\_\{t\}\(m\_\{t\}\)\\leq 0\\right\\\}\.\(67\)Equivalently, the stop region

Ωstop:=\{m∈ℝ4:Δt​\(m\)≤0\}\\Omega\_\{\\mathrm\{stop\}\}:=\\\{m\\in\\mathbb\{R\}^\{4\}:\\Delta\_\{t\}\(m\)\\leq 0\\\}\(68\)is optimal: the controller should stop exactly onΩstop\\Omega\_\{\\mathrm\{stop\}\}and continue on its complement\.

#### Proof\.

By Eq\. \([64](https://arxiv.org/html/2609.16055#A10.E64)\), the optimal action at statemmis whichever attains the larger ofVtstop​\(m\)V\_\{t\}^\{\\mathrm\{stop\}\}\(m\)andVtcont​\(m\)V\_\{t\}^\{\\mathrm\{cont\}\}\(m\)\. IfΔt​\(m\)≤0\\Delta\_\{t\}\(m\)\\leq 0, then by Eq\. \([66](https://arxiv.org/html/2609.16055#A10.E66)\),

Vtcont​\(m\)≤Vtstop​\(m\),V\_\{t\}^\{\\mathrm\{cont\}\}\(m\)\\leq V\_\{t\}^\{\\mathrm\{stop\}\}\(m\),\(69\)so stopping weakly dominates continuation and is therefore optimal\. IfΔt​\(m\)\>0\\Delta\_\{t\}\(m\)\>0, then

Vtcont​\(m\)\>Vtstop​\(m\),V\_\{t\}^\{\\mathrm\{cont\}\}\(m\)\>V\_\{t\}^\{\\mathrm\{stop\}\}\(m\),\(70\)so continuation is strictly optimal\. Hence the optimal decision is exactly

zt⋆​\(m\)=1⇔Δt​\(m\)≤0,z\_\{t\}^\{\\star\}\(m\)=1\\iff\\Delta\_\{t\}\(m\)\\leq 0,\(71\)which proves Eq\. \([67](https://arxiv.org/html/2609.16055#A10.E67)\)\. BecauseVtstopV\_\{t\}^\{\\mathrm\{stop\}\}andVtcontV\_\{t\}^\{\\mathrm\{cont\}\}are measurable inmtm\_\{t\}, the resulting stopping rule is a measurable state\-conditioned policy\.□\\square

#### Corollary J\.1\.

Any stopping rule that depends only on a fixed external depth budget, a pre\-specified schedule, or the step indextt, but not on the realized endogenous statemtm\_\{t\}, is generally suboptimal wheneverΔt​\(m\)\\Delta\_\{t\}\(m\)changes sign over a positive\-probability set of realized states\.

#### Proof\.

IfΔt​\(m\)\\Delta\_\{t\}\(m\)changes sign across realized states, then there exist statesmmandm′m^\{\\prime\}with positive probability such that stopping is optimal atmmbut continuation is optimal atm′m^\{\\prime\}\. Any state\-agnostic rule must assign the same decision to both states and therefore errs on at least one positive\-probability subset\. By Proposition I\.1, the state\-conditioned rule in Eq\. \([67](https://arxiv.org/html/2609.16055#A10.E67)\) attains the optimal action on both\. Thus the state\-agnostic rule is strictly suboptimal in expectation\.□\\square

#### Implication\.

Proposition J\.1 shows that stopping is the bounded\-cost counterpart of evidence organization\. Both are decisions about whether additional reasoning support is worth carrying forward under the current endogenous state\. Corollary J\.1 further explains why SoT does not use a fixed reasoning depth or an external stopping schedule: when the marginal value of another step varies across realized reasoning regimes, rational stopping must itself be state\-conditioned\. This is why the stop controller is part of the same closed\-loop reasoning mechanism rather than an afterthought\.

Part C: Experimental Setup and Implementation

## Appendix KDataset

Table 9:Dataset\-level organization, split size, and primary metric\.CategoryDatasetTask typeTrainTestMetricQuantitativeReasoning \(QS\)GSM8KMath QA—500EMMATHCompetition math—400EMDROPDiscrete reasoning QA—300F1GeneralUnderstanding\(GU\)CommonsenseQA \(CSQA\)Multiple choice QA52400EMStrategyQA \(StrQA\)Binary QA52400EMBoolQBoolean QA—400EMMMLUMulti\-domain MCQA52500EMRACEReading comprehension—300EMSymbolic andCode \(S&C\)FOLIOLogical entailment—203EMProofWriter \(PW\)Deductive QA52400EMBBH\-Temporal \(BBH\-T\)Temporal reasoning52100EMHumanEval \(HE\)Code generation—164P@1MBPPCode generation52250P@1Long\-ContextReasoning \(LCR\)HotpotQA \(HQA\)Multi\-hop QA52300F1NarrativeQA \(NarQA\)Narrative QA—250F1LongBench MultiFieldQA \(LB\)Long\-Context QA—110F1Vision–languagemultimodal \(VLM\)A\-OKVQAKnowledge\-intensive VQA—200EMAI2DScience\-diagram reasoning—150EMM3CoTMulti\-step multimodal reasoning—150EM
Note:EM= exact match,F1= token\-level F1,P@1= pass@1\.

### K\.1Category organization

The LLM datasets are grouped into four categories:

- •Quantitative Reasoning \(QS\):GSM8K\[[Cobbe et al\., 2021](https://arxiv.org/html/2609.16055#bib.bib40)\]\(grade\-school arithmetic word problems\), MATH\[[Hendrycks et al\., 2021b](https://arxiv.org/html/2609.16055#bib.bib41)\]\(competition\-level mathematical reasoning\), and DROP\[[Dua et al\., 2019](https://arxiv.org/html/2609.16055#bib.bib54)\]\(discrete reading comprehension with numerical operations\)\.
- •General Understanding \(GU\):CommonsenseQA\[[Talmor et al\., 2019](https://arxiv.org/html/2609.16055#bib.bib43)\]\(commonsense multiple\-choice QA\), StrategyQA\[[Geva et al\., 2021](https://arxiv.org/html/2609.16055#bib.bib42)\]\(implicit multi\-hop yes/no reasoning\), BoolQ\[[Clark et al\., 2019](https://arxiv.org/html/2609.16055#bib.bib46)\]\(passage\-grounded boolean QA\), MMLU\[[Hendrycks et al\., 2021a](https://arxiv.org/html/2609.16055#bib.bib47)\]\(multi\-domain professional and academic knowledge\), and RACE\[[Lai et al\., 2017](https://arxiv.org/html/2609.16055#bib.bib48)\]\(exam\-style reading comprehension\)\.
- •Symbolic and Code \(S&C\):FOLIO\[[Han et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib49)\]\(first\-order logical entailment\), ProofWriter\[[Tafjord et al\., 2021](https://arxiv.org/html/2609.16055#bib.bib50)\]\(multi\-step rule\-based deduction\), BBH\-Temporal\[[Srivastava et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib44)\]\(temporal symbolic reasoning\), HumanEval\[[Chen et al\., 2021](https://arxiv.org/html/2609.16055#bib.bib51)\]\(function\-level code generation\), and MBPP\[[Austin et al\., 2021](https://arxiv.org/html/2609.16055#bib.bib52)\]\(program synthesis from natural\-language specifications\)\.
- •Long\-Context Reasoning \(LCR\):HotpotQA\[[Yang et al\., 2018](https://arxiv.org/html/2609.16055#bib.bib53)\]\(multi\-hop evidence aggregation\), NarrativeQA\[[Kočiskỳ et al\., 2018](https://arxiv.org/html/2609.16055#bib.bib55)\]\(long\-form story understanding\), and LongBench MultiFieldQA\[[Bai et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib56)\]\(long\-context question answering across heterogeneous documents\)\.

For multimodal transfer, we use A\-OKVQA\[[Schwenk et al\., 2022](https://arxiv.org/html/2609.16055#bib.bib64)\]for knowledge\-intensive visual question answering, AI2D\[[Kembhavi et al\., 2016](https://arxiv.org/html/2609.16055#bib.bib65)\]for science\-diagram reasoning, and M3CoT\[[Chen et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib66)\]for multi\-domain, multi\-step multimodal reasoning\.

### K\.2Metrics

Table[9](https://arxiv.org/html/2609.16055#A11.T9)summarizes the dataset\-level organization, task type, split size, and primary metric used in our experiments\. The headline “accuracy” in the main text is a unified label for the*task\-standard final metric*of each benchmark\.

We use the following primary metrics:

- •EMfor classification\-style, short\-answer, logical\-label, and visual recognition tasks;
- •F1for long\-form QA and evidence\-aggregation tasks;
- •P@1for code\-generation benchmarks\.

For efficiency, we report generated tokens and end\-to\-end latency for all models:

- •Tokens: the number of generated reasoning tokens per example, averaged over the evaluation set;
- •Latency: the end\-to\-end wall\-clock inference time under the same hardware and decoding\.

## Appendix LModel Configuration and Evaluation Protocol

#### Backbone configuration\.

For text\-only reasoning, we evaluate frozen Llama\-3\.1\-8B\[[Grattafiori et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib45)\], Qwen2\.5\-14B\[[Yang et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib32)\], and Mixtral\-8×\\times7B\[[Jiang et al\., 2024a](https://arxiv.org/html/2609.16055#bib.bib33)\]\. For multimodal transfer, we evaluate frozen Qwen2\.5\-VL\-7B and Qwen2\.5\-VL\-32B backbones\[[Bai et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib67)\]; the 32B block uses BF16 weights throughout\. In all cases, backbone weights remain fixed throughout; any learned component is confined to the lightweight SoT controller introduced in Section[2\.3](https://arxiv.org/html/2609.16055#S2.SS3)\.

#### Unified evaluation protocol\.

All methods are evaluated from the same benchmark splits under a shared protocol that fixes decoding settings, inference budgets, answer extraction, and metric computation\. For each run, the evaluation subset is fixed before inference, with the same decoding caps and comparable search or sampling budgets when applicable; completed\-example counts are retained with the evaluation artifacts and used for sample\-weighted aggregates\. Task outputs are then passed through the same dataset\-specific answer extraction, normalization, and scoring functions\. Accordingly, SoT changes only the internal organization of evidence during reasoning; it does not benefit from a different evaluator, parser, or metric definition relative to the baselines\. Text\-backbone latency is measured on H200 GPUs\. VLM latency is measured on one dedicated 80GB A100\.

#### Consistency across baseline families\.

The comparison is controlled at three levels\. First, all methods use the same frozen backbone within each experiment\. Second, all methods follow the same prompt–generation–parsing pipeline except for the reasoning mechanism specific to that baseline family\. Third, efficiency statistics are collected under the same runtime interface\. Generated tokens are counted as the total reasoning tokens produced per example, including internal generations for multi\-pass methods, and latency is measured as end\-to\-end wall\-clock inference time under the same hardware and decoding setup\. This ensures that both quality and efficiency comparisons are mechanism\-level rather than implementation\-level\.

#### Training\-side consistency for methods with learned components\.

Whenever a method introduces a learned component beyond the frozen backbone, it is fit only on a pre\-specified training\-side corpus, with evaluation questions excluded in advance\. SoT follows the same rule: its evidence\-selection and stopping operators are trained only from offline reasoning trajectories derived from the frozen backbone, without access to evaluation instances\. This keeps train–evaluation separation uniform across all methods requiring lightweight fitting and prevents gains from arising from unequal data exposure\.

#### Prompting and post\-processing\.

For baseline families that share a common reasoning prompt style, SoT uses the same user\-level prompt format; differences are confined to the internal control policy rather than surface prompt advantages\. Final predictions for all methods are extracted by the same dataset\-specific post\-processing rules, and the primary metrics reported in Table[9](https://arxiv.org/html/2609.16055#A11.T9)are always computed by the same evaluation routines\.

## Appendix MBaseline Methods

### M\.1Direct generation

#### Vanilla\.

Vanilla performs direct greedy answering without explicit intermediate reasoning control\. It serves as the minimal frozen\-backbone reference\.

### M\.2Reasoning paradigms

#### CoT\.

Chain\-of\-Thought \(CoT\) prompting elicits reasoning through an externally specified step\-by\-step textual scaffold\[[Wei et al\., 2022](https://arxiv.org/html/2609.16055#bib.bib4)\]\. It is the canonical baseline for scripted single\-trajectory reasoning\.

#### PS\.

Plan\-and\-Solve \(PS\) separates reasoning into an explicit planning stage followed by execution\[[Wang et al\., 2023a](https://arxiv.org/html/2609.16055#bib.bib6)\]\. Compared with CoT, it imposes a more structured external decomposition before answering\.

#### SR\.

Self\-Refine \(SR\) uses an externally scripted draft–feedback–revision loop to iteratively improve an initial response\[[Madaan et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib7)\]\. Its improvement comes from explicit textual self\-correction rather than endogenous control of carried evidence\.

#### SC\.

Self\-Consistency \(SC\) samples multiple reasoning traces and selects the final answer by agreement across sampled chains\[[Wang et al\., 2023b](https://arxiv.org/html/2609.16055#bib.bib5)\]\. It represents sample\-based test\-time scaling through breadth rather than state\-conditioned control\.

#### BoN\.

Best\-of\-NN\(BoN\) samples five candidate solutions and uses a fixed zero\-temperature selector pass to choose the final answer from those candidates\[[Snell et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib9)\]\. It therefore tests candidate\-level selection beyond SC’s answer\-agreement rule; all candidate and selector completions are included in its reported cost\.

#### CB\.

Constrained Beam \(CB\) allocates extra inference budget through beam\-style structured decoding under explicit search constraints\[[Hokamp and Liu, 2017](https://arxiv.org/html/2609.16055#bib.bib38)\]\. It provides a search\-oriented comparison point for whether better reasoning must come from broader trajectory exploration\.

#### MCTS\.

Monte Carlo Tree Search \(MCTS\) performs explicit search over reasoning branches using rollout\-based lookahead and selection\[[Xie et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib37)\]\. It is a strong representative of search\-heavy reasoning methods that trade additional compute for broader exploration\.

### M\.3Memory\-oriented methods

#### H2O\.

H2O prunes attention history by retaining tokens estimated to be important for generation\[[Zhang et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib35)\], reducing effective context usage\.

#### SNAP\.

SnapKV accelerates long\-context inference by compressing KV cache content through observation of stable attention patterns\[[Li et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib34)\]\. Its objective is efficient context management rather than reasoning\-specific evidence control\.

#### STREAM\.

StreamingLLM maintains stable long\-context decoding by preserving a small set of attention anchors together with a streaming context window\[[Xiao et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib36)\]\. It is an important efficiency baseline because it also avoids full\-history accumulation\.

Table 10:Deployment\-side inference hyperparameters of SoT\.We report only externally specified quantities used at test time\.ParameterValueNotesContext budget1536Active evidence budgetMaximum reasoning steps32Safety cap onlyObservation window4096Visible context spanFinal answer budget512Answer generation capStop threshold0\.5Default stop ruleGate threshold0\.5Default keep ruleScore mixing coefficient0\.5Score fusion onlyScore temperature0\.8Ranking smoothing only
Table 11:Inference hyperparameters of baseline methods\.BaselineParameterValueVanillaMaximum new tokens512Temperature0CoTMaximum new tokens768Temperature0PSPlanning tokens96Solving tokens320Temperature0SRDraft tokens256Critique tokens96Revision tokens256Temperature0SCNumber of samples3 \(LLM\); 5 \(VLM\)Tokens per sample384Temperature0\.7BoNNumber of candidates5Candidate token cap768Candidate temperature0\.7Selector token cap64Selector temperature0CBSearch depth3Beam width3Branches per beam2Step tokens56Scoring tokens12Answer tokens256Expansion temp\.0\.4Tree token budget3200MCTSSearch iterations6
BaselineParameterValueMCTSExpansion branches2Step tokens56Scoring tokens12Answer tokens256Exploration const\.1\.2Expansion temp\.0\.5Tree token budget3200H2OSink tokens4Recent tokens256Context budget512Maximum new tokens512Temperature0SNAPSink tokens4Recent tokens256Context budget512Maximum new tokens512Temperature0STREAMSink tokens4Recent tokens256Context budget512Maximum new tokens512Temperature0COCOLatent steps3Maximum new tokens512Temperature0GRPO\-SPMaximum new tokens768Temperature0

### M\.4Latent and RL\-based reasoning

#### COCO\.

COCONUT performs reasoning partly in continuous latent space rather than purely through explicit textual chains\[[Hao et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib24)\]\. It is the closest latent\-reasoning comparison, but differs from SoT in that it shifts the reasoning substrate itself rather than using endogenous state to organize evidence\.

#### GRPO\-SP\.

GRPO\-SP is a lightweight RL\-based comparison inspired by grouped relative policy optimization\[[Shao et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib39)\]\. It freezes the backbone and applies grouped reward\-normalized policy\-gradient updates only to a four\-token continuous prompt, testing whether a compact reward\-trained inference prefix can recover reasoning gains without backbone updates\.

#### Comparison logic\.

These baselines probe complementary alternatives to SoT\. Vanilla tests whether explicit reasoning control is necessary\. CoT, PS, and SR test externally prescribed single\-trajectory programs\. SC, BoN, CB, and MCTS test whether gains instead come from larger test\-time search\. H2O, SNAP, and STREAM test whether generic context reduction is sufficient\. COCO and GRPO\-SP test latent\-space or optimization\-based alternatives to inference\-time state\-conditioned control\. Together, they provide the comparison surface needed to show that SoT is not reducible to prompting, search, memory compression, latent reasoning, or RL\-style behavioral optimization\.

#### Implementation and fairness\.

Names follow the implemented algorithmic roles: SC selects by answer agreement, BoN uses a fixed selector over five sampled candidates, CB and MCTS perform structured search, and GRPO\-SP is the grouped\-advantage soft\-prompt variant defined above\. Each family uses one pre\-specified configuration across datasets; sample counts, temperatures, and search budgets are reported in Table[11](https://arxiv.org/html/2609.16055#A13.T11)\. All generated stages count toward efficiency, and all methods share the same task\-specific answer extraction and metric\.

## Appendix NHyperparameter Configuration

Table[11](https://arxiv.org/html/2609.16055#A13.T11)reports the deployment\-side hyperparameters of SoT, while Table[11](https://arxiv.org/html/2609.16055#A13.T11)summarizes the corresponding hyperparameters for all baselines\.

The two score\-smoothing parameters used by SoT—the attention mixing coefficient and attention temperature—are deployment\-side stabilizers rather than additional reasoning modules: they only smooth or fuse ranking signals before thresholding, and do not change the state interface, control form, or backbone generation budget\. The maximum reasoning\-step budget is a deployment\-side safeguard only\. Empirically, nearly all trajectories terminate through the learned stop policy before the hard cap is reached \(about 99\.6% in our runs\), so the reported efficiency gains are not driven by forced truncation\.

For fairness, all methods are evaluated under the same backbone, data split, decoding interface, and scoring pipeline\. Each method is assigned only the minimal family\-specific hyperparameters required for execution\. In particular, SoT does not receive additional model capacity, longer unrestricted generation, or a privileged prompt path relative to the baselines\. Accordingly, any improvement should be attributed to its state\-conditioned control mechanism rather than to favorable hyperparameter budgeting\. Supplementary ablations also help verify that the observed gains are not driven by fragile score calibration or deployment\-side smoothing choices, but by the full state\-conditioned control mechanism\.

Part D: Additional Experimental Results and Analysis

## Appendix OInterpretable Evidence for State\-Conditioned Reasoning

This section extends the interpretability evidence in Secs\.[2\.2](https://arxiv.org/html/2609.16055#S2.SS2)and[2\.3](https://arxiv.org/html/2609.16055#S2.SS3), together with SoT as a trajectory\-level judge in Sec\.[4\.2](https://arxiv.org/html/2609.16055#S4.SS2)\. Our goal is to show that SoT is not merely a strong inference\-time heuristic: its gains are accompanied by stable and interpretable state\-space structure, regime\-dependent evidence organization, and measurable differences between successful and unsuccessful reasoning trajectories\.

### O\.1State\-space trajectories across reasoning paradigms

We first extend the state\-space visualization in Sec\.[2\.2](https://arxiv.org/html/2609.16055#S2.SS2)by covering all four task categories and a broader set of reasoning paradigms\. For each method, we project the trajectory ofmtm\_\{t\}into a shared two\-dimensional space and visualize the stepwise evolution across representative examples\. The comparison is designed to answer two questions: whether different external reasoning paradigms induce distinct trajectory regions or motion patterns, and whether SoT occupies a broader and more adaptive portion of the state space rather than collapsing to a fixed scripted mode\.

![Refer to caption](https://arxiv.org/html/2609.16055v1/app_state_regimes.png)Figure 8:Extended state\-space trajectories across reasoning paradigms\.This figure supplements the main\-text trajectory view with additional paradigms and task categories\.#### Interpretation\.

As shown in Figure[8](https://arxiv.org/html/2609.16055#A15.F8), this analysis does not claim that each paradigm maps to a discrete symbolic state label\. Instead, it shows that the compact statemtm\_\{t\}carries stable geometric structure: different paradigms trace distinguishable trajectory regions, while SoT occupies a broader and more adaptive regime rather than collapsing to a fixed externally scripted pattern\. The key insight is that SoT’s gain is associated with controllable state coverage—reaching multiple useful regimes when needed—rather than with a single pre\-committed reasoning template\.

### O\.2State clusters, sparse memory, and stopping

We next extend the operator\-level analysis of Sec\.[2\.3](https://arxiv.org/html/2609.16055#S2.SS3)\. After aggregating SoT states across tasks, we cluster the observedmtm\_\{t\}and summarize, for each cluster, \(i\) its occupancy across task categories, \(ii\) the average sparsity pattern of𝒮⁡\(y<t,mt\)\\mathcal\{S\}\(y\_\{<t\};m\_\{t\}\), and \(iii\) the stop tendency induced by𝒯⁡\(mt\)\\mathcal\{T\}\(m\_\{t\}\)\. This links the geometry of the state space to concrete control behavior\.

![Refer to caption](https://arxiv.org/html/2609.16055v1/app_state_clusters.png)Figure 9:Clustered state regimes and operator profiles\.#### Interpretation\.

Figure[9](https://arxiv.org/html/2609.16055#A15.F9)tests whether clustered states correspond to distinct control behavior, not just visual separability\. The observed cluster\-specific sparsity and stop profiles indicate regime\-dependent evidence organization and stopping, rather than generic recency bias, static compression, or a fixed\-depth schedule\. This suggests that SoT’s policy is conditionally compositional: the state first selects*what*evidence remains active and then implicitly calibrates*when*to terminate\. The four labels—*exploit\-dense*,*explore\-dense*,*early\-anchor*, and*sparse\-retrieval*—are assigned only after clustering\. We inspect each normalized centroid\(δ,v,c,H\)\(\\delta,v,c,H\)together with three measured properties: activation density, the temporal location of retained units, and stop probability\. “Dense” and “sparse” therefore describe the activation matrix; “early\-anchor” denotes concentration on early units; and the explore–exploit distinction additionally requires the centroid dynamics and stop profile to indicate continued movement or consolidation\. These labels never enter fitting or evaluation\. The evidence for distinct regimes is the operator profiles themselves, not the names attached to them\.

### O\.3Correct versus incorrect trajectories under internal and embedding\-trajectory SoT

Finally, we compare successful and unsuccessful trajectories under both the full SoT state and the SoT\-Embed variant from Sec\.[2\.2](https://arxiv.org/html/2609.16055#S2.SS2)and Sec\.[4\.2](https://arxiv.org/html/2609.16055#S4.SS2)\. For each of the four task categories, we aggregate trajectories by correctness and visualize the resulting state\-space distributions\. We perform the same analysis twice: once using the internal state derived from internal information transfer, and once using features derived from the sentence\-embedding trajectory \(Appendix[D](https://arxiv.org/html/2609.16055#A4)\)\. This allows us to test whether the separation between healthy and unhealthy reasoning trajectories is preserved even when privileged internal access is removed\.

![Refer to caption](https://arxiv.org/html/2609.16055v1/app_correct_incorrect.png)Figure 10:Correct and incorrect trajectory regimes under full\-state and embedding\-trajectory SoT\.This figure compares successful and unsuccessful reasoning trajectories across all four task categories, under both the internal\-state SoT and SoT\-Embed\.#### Interpretation\.

Figure[10](https://arxiv.org/html/2609.16055#A15.F10)serves two roles\. For full SoT, it links gains to trajectory quality by showing whether correct runs occupy more stable and favorable state regimes than incorrect runs\. For SoT\-Embed, it tests whether this correct–incorrect separation persists under limited access; the preserved gap supports that SoT is a transferable trajectory\-level principle rather than a privileged internal\-state trick\. The practical insight is that trajectory\-state separability can function as a lightweight health signal for reliability monitoring, even when white\-box internals are unavailable\.

## Appendix PAdditional Results

This section provides supplementary result tables and figures omitted from the main text for space\. Specifically, we provide backbone\-specific main\-result tables, token and latency breakdowns, and expanded boundary studies for the limited\-access variants\.

### P\.1Additional backbone results

Table[12](https://arxiv.org/html/2609.16055#A16.T12)and Table[13](https://arxiv.org/html/2609.16055#A16.T13)report the full per\-dataset accuracy results for the two LLM backbones not shown in the main text\. Two patterns are notable\. On Qwen2\.5\-14B, SoT achieves the best domain averages in QS \(66\.166\.1\), S&C \(76\.876\.8\), and GU \(81\.481\.4\); the GU gain is especially clear, with all five GU datasets reaching the highest scores in the table \(CSQA80\.580\.5, StrQA75\.075\.0, BoolQ86\.886\.8, MMLU80\.080\.0, RACE86\.086\.0\)\. On Mixtral\-8x7B, SoT is near\-best in QS \(46\.346\.3, close to46\.446\.4\) and leads clearly in S&C \(51\.751\.7\), GU \(72\.672\.6\), and LCR \(23\.523\.5\), indicating that the same state\-conditioned control remains effective across substantially different backbone characteristics\. Taken together, these cross\-backbone results support a mechanism\-level claim: SoT’s advantage is not tied to one architecture or one dataset family, but to a transferable control principle that improves diverse reasoning regimes under heterogeneous model priors\.

Table 12:Main results on Qwen2\.5\-14B \(accuracy, %\)\.CategoryMethodQuantitative ReasoningSymbolic and CodeGSMMATHDROPavg\.FOLPWBBHHEMBPPavg\.ReasoningParadigmCoT94\.477\.51\.965\.60\.055\.584\.075\.066\.049\.9PS88\.051\.52\.154\.40\.565\.592\.050\.047\.249\.6SR92\.068\.217\.665\.561\.183\.479\.060\.050\.468\.1SC92\.861\.01\.959\.50\.559\.587\.055\.048\.847\.2CB60\.041\.230\.846\.462\.185\.051\.050\.060\.869\.8MCTS66\.250\.019\.349\.163\.188\.553\.060\.063\.672\.6Memory\-OrientedH2O92\.667\.02\.661\.60\.514\.881\.072\.845\.233\.4SNAP92\.866\.52\.761\.52\.013\.083\.073\.544\.033\.1STREAM78\.045\.81\.848\.25\.919\.53\.068\.43\.219\.1LatentCOCO93\.674\.82\.864\.62\.540\.575\.071\.260\.045\.6RL\-BasedGRPO\-SP0\.65\.00\.01\.93\.43\.05\.058\.70\.010\.8OursSoT91\.872\.215\.166\.166\.587\.891\.075\.662\.876\.8
CategoryMethodGeneral UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.ReasoningParadigmCoT42\.851\.261\.539\.836\.846\.62\.04\.618\.45\.7PS58\.058\.442\.048\.640\.249\.92\.55\.623\.97\.3SR75\.071\.681\.874\.063\.473\.716\.325\.030\.822\.0SC40\.255\.861\.838\.648\.648\.52\.04\.422\.56\.3CB71\.070\.284\.567\.863\.871\.725\.833\.526\.328\.8MCTS71\.269\.485\.865\.862\.671\.117\.029\.028\.523\.5Memory\-OrientedH2O41\.847\.261\.537\.024\.743\.12\.02\.00\.01\.7SNAP41\.847\.261\.037\.427\.043\.42\.02\.00\.01\.7STREAM39\.846\.252\.833\.20\.036\.12\.40\.00\.01\.1LatentCOCO49\.045\.561\.853\.665\.754\.52\.37\.425\.38\.0RL\-BasedGRPO\-SP10\.05\.85\.28\.212\.78\.20\.10\.10\.30\.1OursSoT80\.575\.086\.880\.086\.081\.424\.225\.427\.325\.2

Table 13:Main results on Mixtral\-8x7B \(accuracy, %\)\.CategoryMethodQuantitative ReasoningSymbolic and CodeGSMMATHDROPavg\.FOLPWBBHHEMBPPavg\.ReasoningParadigmCoT64\.841\.59\.943\.323\.243\.047\.020\.040\.438\.1PS60\.635\.810\.639\.812\.838\.252\.025\.042\.435\.1SR49\.237\.89\.735\.527\.654\.551\.015\.032\.041\.9SC69\.843\.09\.845\.925\.645\.256\.020\.037\.239\.7CB50\.030\.210\.133\.448\.360\.539\.010\.034\.448\.0MCTS45\.626\.88\.029\.950\.757\.239\.020\.038\.448\.4Memory\-OrientedH2O65\.243\.09\.443\.923\.643\.852\.015\.031\.236\.6SNAP64\.843\.29\.143\.723\.643\.855\.015\.030\.836\.8STREAM59\.038\.51\.137\.78\.939\.00\.00\.02\.818\.6LatentCOCO60\.838\.09\.840\.529\.129\.055\.05\.018\.028\.4RL\-BasedGRPO\-SP1\.23\.80\.01\.84\.43\.88\.00\.00\.03\.3OursSoT69\.242\.014\.046\.343\.867\.555\.033\.543\.251\.7
CategoryMethodGeneral UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.ReasoningParadigmCoT46\.553\.572\.849\.847\.754\.114\.515\.337\.818\.7PS51\.538\.546\.849\.647\.346\.911\.117\.433\.217\.2SR65\.562\.080\.263\.261\.066\.59\.912\.233\.014\.6SC44\.557\.275\.855\.050\.056\.814\.515\.338\.118\.8CB67\.269\.582\.058\.649\.365\.813\.016\.637\.418\.4MCTS60\.068\.281\.053\.654\.063\.49\.517\.336\.416\.9Memory\-OrientedH2O45\.851\.773\.552\.011\.749\.013\.16\.60\.08\.5SNAP45\.851\.773\.852\.412\.049\.113\.16\.70\.08\.5STREAM47\.050\.230\.239\.40\.035\.415\.20\.00\.06\.9LatentCOCO48\.041\.257\.548\.048\.348\.615\.220\.417\.417\.5RL\-BasedGRPO\-SP11\.88\.27\.211\.613\.310\.30\.10\.20\.60\.2OursSoT71\.067\.881\.568\.476\.072\.617\.222\.243\.723\.5

### P\.2Extended efficiency and trade\-off results

Here we provide full per\-dataset efficiency breakdowns for Llama\-3\.1\-8B \(Tables[14](https://arxiv.org/html/2609.16055#A16.T14),[15](https://arxiv.org/html/2609.16055#A16.T15)\), Qwen2\.5\-14B \(Tables[16](https://arxiv.org/html/2609.16055#A16.T16),[17](https://arxiv.org/html/2609.16055#A16.T17)\), and Mixtral\-8x7B \(Tables[18](https://arxiv.org/html/2609.16055#A16.T18),[19](https://arxiv.org/html/2609.16055#A16.T19)\)\. Two consistent trends emerge: \(i\) search\-heavy methods incur very large token/latency overhead, especially on long\-context tasks \(e\.g\., Qwen LCR token average4901\.54901\.5for SC vs\.254\.0254\.0for SoT; Mixtral LCR token average7546\.27546\.2for CB vs\.260\.6260\.6for SoT\), and \(ii\) SoT remains in a low\-cost regime across backbones, with particularly strong Llama efficiency in GU \(74\.9\{74\.9\}tokens,1\.81\.8s\) and QS \(255\.7\{255\.7\}tokens,5\.85\.8s\)\. This indicates that SoT improves the accuracy–efficiency frontier through selective state\-conditioned computation rather than brute\-force trajectory expansion, so its efficiency gains reflect better control allocation, not merely shorter outputs by default\.

Table 14:Generated reasoning tokens on Llama\-3\.1\-8B\.CategoryMethodQuantitative ReasoningSymbolic and CodeGSMMATHDROPavg\.FOLPWBBHHEMBPPavg\.ReasoningParadigmCoT230\.1381\.5194\.1271\.6342\.9332\.0264\.4490\.1415\.4369\.8PS311\.0358\.9259\.9314\.2369\.9354\.4346\.0404\.6391\.4372\.1SR346\.5441\.6307\.6368\.5337\.8330\.4336\.4517\.6473\.0391\.7SC678\.4928\.6578\.2736\.8913\.0875\.4798\.11135\.31056\.2953\.9CB441\.8521\.0264\.5423\.9325\.8324\.6290\.0424\.8447\.3363\.9MCTS462\.2482\.3275\.0422\.1327\.0326\.9286\.7435\.1440\.7364\.7Memory\-OrientedH2O222\.4293\.8141\.6226\.0264\.6249\.5223\.6267\.9302\.1264\.4SNAP222\.4293\.8141\.6226\.0264\.6249\.5223\.6267\.9302\.1264\.4STREAM104\.8122\.832\.192\.666\.366\.06\.154\.489\.364\.2LatentCOCO133\.4260\.0117\.0171\.5159\.4186\.2192\.0318\.9245\.4214\.6RL\-BasedGRPO\-SP71\.573\.974\.873\.163\.965\.369\.279\.075\.569\.7OursSoT186\.1452\.0129\.1255\.7253\.7254\.9197\.2404\.0405\.2302\.2
CategoryMethodGeneral UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.ReasoningParadigmCoT194\.7221\.7164\.8279\.4178\.8212\.9275\.4165\.960\.7198\.1PS283\.7344\.4310\.1353\.0292\.8319\.8316\.1310\.6125\.8282\.3SR278\.3304\.0260\.9326\.8278\.8292\.2326\.9284\.1135\.3278\.8SC568\.3660\.8488\.6790\.3569\.0626\.5671\.0541\.9141\.3533\.8CB392\.8426\.7298\.7364\.2180\.3341\.7443\.4196\.446\.7283\.7MCTS375\.6423\.3301\.9353\.5178\.0335\.2446\.1196\.656\.2286\.6Memory\-OrientedH2O190\.3219\.6157\.0254\.057\.8185\.6237\.4126\.353\.0164\.6SNAP190\.3219\.6157\.0254\.057\.8185\.6237\.4126\.353\.0164\.6STREAM117\.5149\.560\.790\.535\.093\.4148\.779\.033\.4103\.1LatentCOCO56\.199\.572\.4120\.876\.087\.2175\.8104\.971\.0131\.5RL\-BasedGRPO\-SP70\.069\.670\.771\.176\.971\.471\.977\.684\.976\.2OursSoT107\.5121\.221\.897\.626\.474\.9435\.2207\.1146\.6263\.0

Table 15:End\-to\-end latency on Llama\-3\.1\-8B\.CategoryMethodQuantitative ReasoningSymbolic and CodeGSMMATHDROPavg\.FOLPWBBHHEMBPPavg\.ReasoningParadigmCoT12\.125\.615\.717\.524\.023\.918\.941\.330\.427\.5PS18\.722\.122\.320\.726\.224\.427\.732\.327\.326\.8SR21\.630\.327\.225\.922\.621\.626\.245\.135\.528\.8SC34\.851\.845\.543\.158\.955\.757\.485\.168\.163\.5CB20\.925\.619\.222\.016\.516\.217\.926\.323\.019\.4MCTS21\.922\.119\.821\.416\.616\.517\.626\.622\.119\.4Memory\-OrientedH2O13\.321\.915\.516\.723\.446\.220\.538\.128\.434\.6SNAP13\.322\.015\.116\.623\.247\.220\.637\.228\.434\.8STREAM11\.016\.49\.912\.518\.133\.911\.422\.319\.224\.0LatentCOCO6\.716\.013\.411\.510\.228\.014\.024\.916\.020\.4RL\-BasedGRPO\-SP6\.66\.89\.97\.56\.26\.57\.78\.77\.47\.1OursSoT4\.88\.14\.45\.87\.27\.15\.615\.013\.39\.5
CategoryMethodGeneral UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.ReasoningParadigmCoT9\.39\.89\.616\.719\.112\.815\.218\.533\.819\.6PS15\.818\.721\.822\.934\.822\.216\.638\.070\.733\.7SR14\.715\.117\.020\.032\.119\.217\.033\.776\.333\.2SC26\.028\.627\.844\.966\.037\.629\.460\.974\.948\.9CB16\.616\.416\.117\.118\.016\.817\.419\.226\.219\.5MCTS15\.916\.316\.316\.717\.716\.517\.219\.232\.620\.5Memory\-OrientedH2O9\.910\.610\.417\.035\.715\.813\.435\.611\.521\.5SNAP9\.910\.610\.418\.435\.616\.113\.334\.311\.420\.9STREAM8\.411\.07\.215\.34\.59\.811\.213\.41\.010\.3LatentCOCO2\.53\.94\.16\.918\.96\.79\.627\.593\.330\.3RL\-BasedGRPO\-SP6\.65\.87\.36\.812\.47\.56\.112\.7115\.626\.9OursSoT2\.42\.60\.81\.71\.61\.86\.17\.897\.522\.0

Table 16:Generated reasoning tokens on Qwen2\.5\-14B\.CategoryMethodQuantitative ReasoningSymbolic and CodeGSMMATHDROPavg\.FOLPWBBHHEMBPPavg\.ReasoningParadigmCoT399\.4521\.3544\.5476\.3503\.3416\.7456\.0644\.4597\.8490\.0PS653\.4698\.41021\.3760\.4639\.7633\.2896\.3935\.2829\.9718\.3SR1326\.71454\.91804\.91489\.01496\.61528\.61601\.11891\.51717\.51625\.0SC1188\.41366\.31646\.31362\.21430\.41227\.51433\.21798\.61614\.91402\.2CB3868\.33866\.04187\.83947\.43846\.23823\.34166\.93991\.84045\.23923\.9MCTS3899\.33884\.14184\.73965\.63896\.83822\.24170\.84040\.73998\.43923\.3Memory\-OrientedH2O390\.5446\.4490\.7434\.2447\.9398\.1453\.6509\.5488\.4440\.5SNAP390\.5446\.4490\.7434\.2447\.9398\.1453\.6509\.5488\.4440\.5STREAM260\.7262\.7371\.5289\.1263\.1256\.1272\.2279\.4267\.2262\.8LatentCOCO371\.8466\.9520\.8440\.7473\.8314\.2441\.7597\.3579\.5436\.6RL\-BasedGRPO\-SP861\.6839\.91065\.7905\.4920\.6893\.4966\.5902\.2888\.4905\.4OursSoT885\.0392\.7184\.6545\.8338\.4233\.7310\.4569\.2678\.2408\.4
CategoryMethodGeneral UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.ReasoningParadigmCoT263\.1251\.8314\.9401\.0612\.3358\.1322\.1729\.96597\.5529\.6PS490\.2472\.6686\.0660\.5638\.2590\.6507\.31386\.614498\.23172\.2SR985\.7938\.41139\.91361\.6702\.11058\.51156\.62363\.821847\.25062\.3SC810\.6771\.2951\.11206\.6685\.4911\.0990\.22208\.321690\.84901\.7CB3706\.43592\.73862\.33802\.03688\.43736\.03814\.44637\.421703\.17107\.6MCTS3844\.03728\.33915\.23853\.43715\.63818\.23724\.24612\.721714\.87059\.2Memory\-OrientedH2O264\.0263\.9313\.3389\.6572\.2351\.5321\.7624\.77141\.21573\.1SNAP264\.0263\.9313\.3389\.6572\.2351\.5321\.7624\.77141\.21573\.1STREAM238\.4231\.8264\.0259\.1550\.5294\.2257\.2565\.47141\.21521\.3LatentCOCO242\.7207\.4300\.5345\.7652\.5334\.4311\.1671\.27246\.61603\.4RL\-BasedGRPO\-SP859\.0823\.0945\.2882\.91264\.8935\.9819\.91280\.97806\.12158\.9OursSoT149\.2126\.445\.1199\.4127\.8133\.2229\.2175\.1500\.9254\.0

Table 17:End\-to\-end latency on Qwen2\.5\-14B\.CategoryMethodQuantitative ReasoningSymbolic and CodeGSMMATHDROPavg\.FOLPWBBHHEMBPPavg\.ReasoningParadigmCoT25\.047\.426\.632\.939\.225\.823\.060\.756\.736\.9PS27\.936\.732\.231\.922\.419\.638\.050\.146\.129\.5SR37\.856\.343\.545\.441\.149\.839\.687\.180\.459\.6SC73\.0100\.481\.784\.3107\.672\.879\.9149\.6129\.496\.9CB40\.643\.139\.041\.035\.033\.040\.149\.852\.639\.5MCTS43\.746\.841\.544\.234\.830\.540\.050\.652\.838\.5Memory\-OrientedH2O16\.225\.415\.018\.923\.516\.315\.733\.830\.021\.8SNAP16\.225\.514\.918\.923\.516\.115\.734\.630\.921\.9STREAM14\.521\.711\.916\.219\.214\.613\.528\.127\.019\.0LatentCOCO12\.921\.212\.315\.519\.57\.312\.327\.929\.016\.5RL\-BasedGRPO\-SP66\.364\.085\.570\.371\.768\.976\.068\.968\.470\.1OursSoT21\.116\.89\.516\.814\.810\.214\.327\.432\.819\.0
CategoryMethodGeneral UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.ReasoningParadigmCoT10\.111\.49\.225\.321\.815\.718\.135\.2254\.126\.7PS11\.913\.216\.825\.123\.618\.220\.138\.3262\.867\.5SR18\.920\.618\.935\.430\.225\.132\.057\.2366\.397\.2SC32\.233\.827\.572\.465\.846\.756\.4110\.1555\.9160\.0CB27\.228\.427\.334\.929\.729\.839\.242\.0162\.460\.8MCTS32\.634\.131\.935\.930\.533\.340\.143\.3172\.163\.3Memory\-OrientedH2O7\.08\.85\.915\.810\.79\.912\.614\.87\.412\.6SNAP6\.98\.75\.915\.811\.710\.012\.615\.37\.412\.8STREAM6\.38\.35\.715\.33\.68\.411\.212\.85\.410\.8LatentCOCO5\.35\.04\.410\.510\.67\.110\.711\.7126\.230\.3RL\-BasedGRPO\-SP65\.762\.274\.167\.9105\.573\.261\.9107\.8787\.9200\.3OursSoT6\.15\.02\.28\.27\.05\.810\.210\.289\.623\.4

Table 18:Generated reasoning tokens on Mixtral\-8x7B\.CategoryMethodQuantitative ReasoningSymbolic and CodeGSMMATHDROPavg\.FOLPWBBHHEMBPPavg\.ReasoningParadigmCoT348\.8439\.0534\.7425\.3394\.8353\.3471\.7617\.2574\.4436\.3PS665\.9703\.11094\.4785\.4801\.5728\.8944\.91000\.4868\.1807\.6SR1222\.41336\.41779\.21399\.61403\.31283\.21782\.41970\.01709\.81483\.3SC1038\.61205\.61612\.71237\.81168\.61070\.21408\.31817\.21588\.21273\.9CB3872\.73857\.24205\.33950\.73913\.13867\.34130\.24221\.44093\.73969\.3MCTS3901\.83917\.44191\.33979\.43991\.93915\.24148\.54170\.14070\.74000\.4Memory\-OrientedH2O339\.4392\.5485\.2393\.5378\.2354\.1443\.5500\.1480\.9403\.9SNAP339\.4392\.5485\.2393\.5378\.2354\.1443\.5500\.1480\.9403\.9STREAM256\.9259\.2386\.4290\.0261\.4254\.0279\.1288\.6268\.8262\.7LatentCOCO295\.1327\.0498\.1356\.5313\.1343\.1422\.9591\.0553\.2404\.2RL\-BasedGRPO\-SP892\.3882\.31138\.4950\.5954\.7943\.21045\.8967\.9952\.0958\.9OursSoT208\.0361\.9163\.7248\.2192\.0155\.6184\.4605\.5597\.6329\.8
CategoryMethodGeneral UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.ReasoningParadigmCoT249\.9210\.3303\.7328\.0678\.5336\.6251\.4726\.37933\.41711\.6PS564\.3520\.0776\.7698\.31504\.5772\.5504\.21532\.916024\.93480\.7SR1028\.7930\.41183\.01336\.82407\.11323\.71021\.22314\.324064\.05351\.5SC759\.4642\.2914\.4999\.02068\.01023\.1765\.52173\.823802\.65138\.5CB3859\.33821\.03966\.23903\.24678\.44006\.83831\.24804\.923908\.37546\.2MCTS3884\.23752\.53961\.43924\.74651\.33998\.53721\.64781\.423933\.47491\.7Memory\-OrientedH2O248\.2208\.3300\.8323\.5597\.7322\.0249\.8645\.07872\.41670\.0SNAP248\.2208\.3300\.8323\.5597\.7322\.0249\.8645\.07872\.41670\.0STREAM223\.8198\.1265\.1247\.1582\.8286\.6231\.2602\.87872\.41645\.5LatentCOCO260\.0268\.5365\.4329\.5728\.5370\.4231\.2713\.48116\.81728\.1RL\-BasedGRPO\-SP881\.4832\.7968\.3919\.01323\.5964\.7835\.61341\.98574\.62317\.2OursSoT135\.0123\.895\.7182\.5180\.2143\.6185\.1281\.3419\.8260\.6

Table 19:End\-to\-end latency on Mixtral\-8x7B\.CategoryMethodQuantitative ReasoningSymbolic and CodeGSMMATHDROPavg\.FOLPWBBHHEMBPPavg\.ReasoningParadigmCoT26\.740\.518\.029\.128\.619\.327\.049\.746\.729\.7PS31\.241\.123\.832\.636\.325\.137\.146\.742\.733\.6SR44\.652\.229\.943\.551\.739\.054\.172\.864\.150\.4SC76\.4130\.456\.589\.470\.866\.072\.4132\.9123\.983\.9CB63\.265\.435\.557\.056\.153\.852\.171\.572\.059\.1MCTS66\.370\.539\.961\.160\.655\.757\.170\.771\.261\.2Memory\-OrientedH2O42\.864\.036\.748\.447\.536\.238\.396\.789\.653\.7SNAP43\.665\.335\.848\.942\.336\.538\.496\.488\.552\.5STREAM38\.656\.926\.841\.735\.631\.833\.385\.881\.346\.6LatentCOCO33\.542\.126\.334\.623\.434\.630\.782\.679\.444\.4RL\-BasedGRPO\-SP190\.2278\.6206\.3223\.7191\.5203\.6204\.7186\.2186\.4196\.4OursSoT37\.760\.132\.143\.838\.330\.335\.3115\.1110\.262\.6
CategoryMethodGeneral UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.ReasoningParadigmCoT15\.615\.111\.621\.315\.816\.219\.315\.215\.117\.1PS22\.225\.627\.230\.734\.327\.822\.523\.331\.324\.3SR33\.735\.427\.844\.337\.436\.136\.226\.440\.033\.1SC46\.246\.430\.863\.943\.447\.259\.344\.833\.849\.5CB97\.166\.449\.356\.437\.162\.266\.728\.623\.045\.0MCTS65\.372\.455\.661\.140\.360\.068\.231\.125\.347\.0Memory\-OrientedH2O26\.325\.616\.235\.430\.027\.034\.835\.64\.330\.0SNAP26\.326\.116\.336\.031\.327\.433\.235\.33\.929\.1STREAM25\.224\.414\.732\.415\.123\.232\.927\.52\.525\.8LatentCOCO28\.840\.431\.834\.440\.234\.830\.729\.2118\.644\.8RL\-BasedGRPO\-SP184\.1185\.1197\.3198\.7242\.5199\.4172\.2232\.8357\.4226\.0OursSoT26\.823\.019\.434\.037\.428\.035\.352\.7160\.862\.8

\(a\)Llama\-3\.1\-8B\(b\)Qwen2\.5\-14B\(c\)Mixtral\-8x7B\(d\)Qwen2\.5\-VL\-7B/32B

Figure 11:Extended accuracy–efficiency trade\-off plots across backbones\.Figure[11](https://arxiv.org/html/2609.16055#A16.F11)shows a consistent cross\-backbone pattern: methods that gain accuracy mainly through broader sampling/search move upward to high\-token and high\-latency regions, while SoT stays on or near the Pareto frontier\. On text backbones, SoT combines top\-tier accuracy with substantially lower compute than search\-heavy alternatives, indicating that its gains come from better control allocation rather than budget expansion\. The same trend also appears in multimodal transfer, supporting the view that state\-conditioned evidence organization is a mechanism\-level efficiency advantage rather than a backbone\-specific tuning effect\.

Table 20:Additional ablation results on Qwen2\.5\-14B\.VariantQSS&CGULCRacc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.Full SoT66\.1545\.816\.876\.8408\.419\.081\.3133\.25\.825\.2254\.023\.4\+ Threshold Tuning66\.1545\.816\.876\.8408\.419\.081\.3133\.25\.825\.2254\.023\.4w/o Evid\. Org\.𝒮\\mathcal\{S\}30\.0532\.121\.528\.4556\.121\.772\.4234\.79\.426\.5489\.414\.3w/o Stopping𝒯\\mathcal\{T\}76\.01401\.937\.432\.42326\.259\.670\.24175\.0100\.931\.72815\.872\.8w/o Geometryδt\\delta\_\{t\}56\.577\.93\.923\.145\.82\.573\.025\.40\.921\.248\.22\.2w/o Dynamics\(vt,ct\)\(v\_\{t\},c\_\{t\}\)70\.01220\.932\.130\.8910\.026\.970\.61170\.531\.831\.2869\.027\.8w/o UncertaintyHtH\_\{t\}62\.5306\.07\.630\.6255\.67\.673\.061\.31\.933\.6190\.26\.6w/o Sent\.\-level Units62\.5306\.07\.630\.6255\.67\.673\.061\.31\.933\.6190\.26\.6
### P\.3Additional ablation results

To test whether the main ablation pattern is specific to Llama\-3\.1\-8B or reflects a more stable mechanism, we report additional ablations on Qwen2\.5\-14B, Mixtral\-8×\\times7B in Table[20](https://arxiv.org/html/2609.16055#A16.T20)and Table[21](https://arxiv.org/html/2609.16055#A16.T21)\. The cross\-backbone pattern is consistent with the main\-text ablation conclusion: removing evidence organization \(𝒮\\mathcal\{S\}\) causes clear accuracy degradation \(e\.g\., Qwen QS→30\.066\.1\\\!\\rightarrow\\\!30\.0, S&C→28\.476\.8\\\!\\rightarrow\\\!28\.4; Mixtral QS→31\.346\.3\\\!\\rightarrow\\\!31\.3, LCR→3\.023\.5\\\!\\rightarrow\\\!3\.0\)\. Removing stopping \(𝒯\\mathcal\{T\}\) can occasionally increase one domain score, but it breaks efficiency by orders of magnitude \(e\.g\., Qwen GU tokens→4175\.0133\.2\\\!\\rightarrow\\\!4175\.0\)\. Meanwhile, threshold tuning is negligible on text backbones \(identical to Full SoT on Qwen/Mixtral\), and state\-component removals typically trade short outputs for weaker accuracy, supporting the same mechanism\-level claim as the main text: SoT gains come from the full closed\-loop control, not from isolated calibration or trivial truncation\.

Table 21:Additional ablation results on Mixtral\-8×\\times7B\.VariantQSS&CGULCRacc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.Full SoT46\.3248\.243\.851\.7329\.862\.672\.5143\.628\.023\.5260\.662\.8\+ Threshold Tuning46\.3248\.243\.851\.7329\.862\.672\.5143\.628\.023\.5260\.662\.8w/o Evid\. Org\.𝒮\\mathcal\{S\}31\.3995\.070\.436\.01002\.665\.863\.2836\.851\.03\.0921\.190\.7w/o Stopping𝒯\\mathcal\{T\}31\.3995\.064\.036\.01002\.665\.463\.2836\.850\.73\.0921\.190\.6w/o Geometryδt\\delta\_\{t\}34\.058\.111\.830\.756\.913\.357\.650\.87\.63\.354\.013\.8w/o Dynamics\(vt,ct\)\(v\_\{t\},c\_\{t\}\)31\.3995\.064\.036\.01002\.665\.463\.2836\.850\.73\.0921\.190\.6w/o UncertaintyHtH\_\{t\}31\.3995\.064\.036\.01002\.665\.463\.2836\.850\.73\.0921\.190\.6w/o Sent\.\-level Units31\.3995\.064\.036\.01002\.665\.463\.2836\.850\.73\.0921\.190\.6
### P\.4Limited\-access variants and SoT\-Judge

We further expand the exploratory results in Section[4](https://arxiv.org/html/2609.16055#S4)with Tables[22](https://arxiv.org/html/2609.16055#A16.T22)and[23](https://arxiv.org/html/2609.16055#A16.T23)\.

On Qwen2\.5\-14B, Training\-free SoT reaches an LCR average of 26\.2 versus 25\.8 for the Top\-3 baselines, while SoT\-Embed remains competitive at 23\.1\. On Mixtral\-8x7B, both variants exceed the Top\-3 averages in quantitative, symbolic/code, and long\-context reasoning, reaching 26\.4 and 27\.9 versus 23\.7 on LCR\. The consistent HQA/NarQA gains across both backbones show that trajectory\-level state summaries retain useful long\-range structure when either controller fitting or internal\-state access is removed\.

Table 22:Boundary studies of SoT on Qwen2\.5\-14B \(accuracy, %\)\.Quantitative ReasoningSymbolic and CodeGSM8KMATHDROPavg\.FOLIOPWBBH\-THEMBPPavg\.Top\-3 Baseline avg\.93\.673\.522\.663\.262\.185\.687\.773\.863\.574\.5SoT\-Training\-free81\.860\.834\.659\.149\.530\.277\.053\.050\.452\.0SoT\-Embedding82\.658\.334\.858\.651\.259\.571\.048\.849\.656\.0
General UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.Top\-3 Baseline avg\.72\.470\.484\.069\.264\.372\.119\.729\.228\.525\.8SoT\-Training\-free59\.355\.270\.045\.452\.756\.522\.934\.221\.626\.2SoT\-Embedding57\.043\.064\.842\.848\.751\.321\.229\.818\.223\.1

Table 23:Boundary studies of SoT on Mixtral\-8x7B \(accuracy, %\)\.Quantitative ReasoningSymbolic and CodeGSM8KMATHDROPavg\.FOLIOPWBBH\-THEMBPPavg\.Top\-3 Baseline avg\.66\.643\.110\.240\.042\.757\.455\.321\.740\.443\.5SoT\-Training\-free75\.252\.234\.453\.931\.350\.563\.056\.149\.250\.0SoT\-Embedding76\.052\.036\.754\.930\.850\.563\.055\.549\.249\.8
General UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.Top\-3 Baseline avg\.64\.266\.681\.158\.955\.065\.215\.018\.437\.823\.7SoT\-Training\-free49\.554\.566\.550\.848\.353\.924\.334\.320\.526\.4SoT\-Embedding49\.344\.352\.551\.247\.549\.024\.334\.025\.527\.9

We report full judge behavior in Table[24](https://arxiv.org/html/2609.16055#A16.T24)\. The same insight as the main section is reinforced: SoT\-Judge’s largest gains concentrate in trajectory\-structure\-sensitive regimes \(especially LCR, e\.g\., GPT\-5\.481\.0\{81\.0\}vs\.≤16\.7\\leq\{16\.7\}for baselines, Claude 4\.799\.0\{99\.0\}vs\.≤9\.3\\leq\{9\.3\}, Qwen\-Max82\.2\{82\.2\}vs\.≤14\.8\\leq\{14\.8\}\), while trading some GU/S&C agreement for much stronger QR/LCR agreement, suggesting that trajectory\-state signals are most informative where long\-horizon coherence, not local answer matching, is the primary bottleneck\.

Table 24:Black\-box judging with trajectory\-based SoT \(dataset breakdown, %\)\.ModelMethodQuantitative ReasoningSymbolic and CodeGSM8KMATHDROPavg\.FOLIOPWBBH\-THEMBPPavg\.GPT\-5\.4Self\-Consistency92\.078\.05\.058\.369\.077\.086\.087\.077\.079\.2Self\-Verification92\.078\.05\.058\.362\.079\.085\.087\.078\.078\.2LLM\-as\-a\-Judge92\.079\.04\.058\.370\.076\.086\.086\.078\.079\.2SoT\-Judge85\.079\.094\.086\.050\.060\.077\.086\.076\.069\.8Claude 4\.7Self\-Consistency90\.078\.01\.056\.365\.073\.058\.094\.086\.075\.2Self\-Verification94\.180\.04\.059\.471\.480\.057\.198\.494\.480\.3LLM\-as\-a\-Judge90\.078\.02\.056\.766\.074\.057\.094\.086\.075\.4SoT\-Judge90\.078\.091\.086\.350\.078\.023\.097\.087\.067\.0Qwen\-MaxSelf\-Consistency88\.082\.83\.057\.976\.086\.081\.083\.873\.080\.0Self\-Verification46\.735\.36\.029\.380\.682\.974\.581\.570\.077\.9LLM\-as\-a\-Judge92\.082\.83\.059\.374\.086\.081\.083\.874\.079\.8SoT\-Judge77\.075\.998\.083\.668\.064\.074\.075\.067\.069\.6
ModelMethodGeneral UnderstandingLong\-Context ReasoningCSQAStrQABoolQMMLURACEavg\.HQANarQALBavg\.GPT\-5\.4Self\-Consistency85\.082\.094\.076\.070\.081\.47\.09\.034\.016\.7Self\-Verification84\.081\.093\.075\.070\.080\.67\.09\.032\.016\.0LLM\-as\-a\-Judge86\.085\.094\.075\.070\.082\.07\.09\.033\.016\.3SoT\-Judge85\.054\.067\.074\.070\.070\.098\.087\.058\.081\.0Claude 4\.7Self\-Consistency83\.086\.095\.076\.078\.083\.611\.01\.03\.05\.0Self\-Verification46\.587\.294\.023\.362\.762\.723\.01\.04\.09\.3LLM\-as\-a\-Judge82\.085\.095\.077\.079\.083\.611\.01\.04\.05\.3SoT\-Judge81\.049\.053\.067\.074\.064\.8100\.099\.098\.099\.0Qwen\-MaxSelf\-Consistency82\.082\.091\.077\.088\.084\.011\.01\.031\.514\.5Self\-Verification86\.786\.293\.880\.685\.086\.51\.91\.431\.511\.6LLM\-as\-a\-Judge84\.082\.093\.077\.088\.084\.812\.01\.031\.514\.8SoT\-Judge80\.074\.075\.072\.086\.077\.495\.0100\.051\.782\.2

### P\.5Sensitivity to trajectory quality

We test whether SoT remains informative when the offline trajectory pool is collected or curated differently, with Llama\-3\.1\-8B results in Table[25](https://arxiv.org/html/2609.16055#A16.T25)\(default HQ pool vs\. weaker variants and training\-free SoT\)\. The table shows a consistent split: weaker pools largely leave QS/S&C accuracy near plateau but disproportionately hurt GU/LCR while greatly inflating tokens and latency relative to HQ; training\-free SoT sits in the same heavy\-trajectory regime with domain\-mixed accuracy shifts\. Overall, curation affects whether supervision yields a cheap, reliable trajectory\-level control signal more than it acts as a uniform rescaling of backbone capability\.

Table 25:Sensitivity to trajectory\-pool quality on Llama\-3\.1\-8B\.Trajectory poolQSS&CGULCRacc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.acc\.tok\.lat\.Full HQ pool70\.8247\.96\.357\.1324\.211\.175\.558\.01\.637\.3121\.197\.5No filter71\.3931\.330\.057\.71080\.884\.149\.2886\.029\.521\.5900\.991\.5Weak filter71\.3931\.324\.357\.71080\.834\.549\.2886\.017\.721\.3900\.990\.7No pipeline71\.3931\.326\.157\.71080\.853\.449\.2886\.049\.321\.5900\.989\.3Training\-free72\.3917\.426\.058\.31078\.935\.053\.8906\.619\.728\.1872\.995\.8

## Appendix QCase Studies

### Q\.1Case 1: Quantitative Reasoning \(QS\)

Figure[12](https://arxiv.org/html/2609.16055#A17.F12)follows a staged numeric workflow \(per\-store accounting, then comparison\)\. Intermediate steps reuse only the quantities that remain algebraically active in the next operation, andp⁡\(stop\)p\(\\mathrm\{stop\}\)ramps once the competing subtotals are both fixed: the mechanism reads as*derivation support*that is pruned as subgoals close, not as uniform context replay\.

![Refer to caption](https://arxiv.org/html/2609.16055v1/app_case_arithmetic.png)Figure 12:Case study on Quantitative Reasoning \(QS\)\.
### Q\.2Case 2: General Understanding \(GU\)

Figure[13](https://arxiv.org/html/2609.16055#A17.F13)is a commonsense elimination problem: late steps depend on the prompt and the locally decisive intermediates while dropping earlier scaffolding—a pattern that matters when verbal distractors would otherwise pollute a full\-history transcript\.

![Refer to caption](https://arxiv.org/html/2609.16055v1/app_case_cki.png)Figure 13:Case study on General Understanding \(GU\)\.
### Q\.3Case 3: Symbolic and Code \(S&C\)

Figure[14](https://arxiv.org/html/2609.16055#A17.F14)chains explicit rules forward but the evidence matrix is non\-monotone—later steps re\-anchor on conclusions and skip obsolete rule instantiations—so correctness is tied to*structural*carry \(what still entails the next judgment\), not to preserving the full forward chain verbatim\.

![Refer to caption](https://arxiv.org/html/2609.16055v1/app_case_formal.png)Figure 14:Case study on Symbolic and Code \(S&C\)\.
### Q\.4Case 4: Long\-Context Reasoning \(LCR\)

Figure[15](https://arxiv.org/html/2609.16055#A17.F15)integrates constraints spread across a long narrative; the trajectory lingers in exploration\-like regimes while facts are harvested, then compresses into a sparse\-retrieval phase that fuses only the clauses still relevant to the contrastive*why*question—i\.e\., compositional support rather than verbatim long\-context carry\.

![Refer to caption](https://arxiv.org/html/2609.16055v1/app_case_longcontext.png)Figure 15:Case study on Long\-Context Reasoning \(LCR\)\.
### Q\.5Case 5: Visual\-language reasoning

Figure[16](https://arxiv.org/html/2609.16055#A17.F16)\(Qwen3\-VL\-8B\) shows the same interface on chart QA: panel\-wise extraction, numeric gap reasoning, risingp⁡\(stop\)p\(\\mathrm\{stop\}\)through the computation, and sparse dependencies—evidence that SoT remains a trajectory\-level organizer when evidence is grounded in pixels rather than text alone\.

![Refer to caption](https://arxiv.org/html/2609.16055v1/app_case_vlm.png)Figure 16:Case study on visual\-language reasoning\.Part E: Related Work and Discussion

## Appendix RRelated Work

This section expands the compressed related\-work discussion from the main paper\. SoT sits at the intersection of four research directions, but is not reducible to any one of them\.

### R\.1Externally scripted reasoning

One major line of work improves reasoning by imposing explicit inference\-time structure: step\-by\-step prompting\[[Wei et al\., 2022](https://arxiv.org/html/2609.16055#bib.bib4)\], self\-consistency\[[Wang et al\., 2023b](https://arxiv.org/html/2609.16055#bib.bib5)\], plan\-and\-solve decomposition\[[Wang et al\., 2023a](https://arxiv.org/html/2609.16055#bib.bib6)\], self\-refinement\[[Madaan et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib7)\], and tree\-style search\[[Yao et al\., 2023a](https://arxiv.org/html/2609.16055#bib.bib8)\]\. These methods substantially advance empirical reasoning performance, but they share a common modeling assumption: reasoning improves when we prescribe a better*external*program for how the model should think\.

SoT differs at the level of problem formulation\. It does not ask for a better externally specified chain\. It asks whether the model’s current endogenous regime can decide what evidence should remain active for the next decision\. In this sense, SoT shifts the focus from*external reasoning choreography*to*endogenous reasoning control*\.

### R\.2Internal reasoning trajectories

Another relevant line studies hidden\-state geometry, representation dynamics, and correctness\-related trajectory structure in language models\[[Ballon et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib19),[Damirchi et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib20),[Park et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib27),[Park et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib26),[Sun et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib13)\]\. Related work also shows that internal signals can support monitoring and early\-exit decisions\[[Feng et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib18),[Hosseini et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib21),[Tikhonov et al\., 2026](https://arxiv.org/html/2609.16055#bib.bib14)\]\. These works motivate the view that reasoning\-relevant structure exists inside the model\.

However, most of this literature is primarily descriptive: it interprets hidden trajectories, monitors confidence, or probes latent variables\. SoT draws on similar geometric\-dynamic structure but uses it differently\. Rather than analyzing the state after the fact, SoT engages it online as the control variable that determines evidence organization and continuation\.

### R\.3Context augmentation and compression

Retrieval\-augmented and context\-management methods also intersect with SoT\. Examples include retrieval\-interleaved reasoning\[[Trivedi et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib2),[Wang et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib23)\], context compression\[[Jiang et al\., 2024b](https://arxiv.org/html/2609.16055#bib.bib17),[Li et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib16)\], and KV\-cache or memory\-selection methods such as H2O, SnapKV, and StreamingLLM\[[Li et al\., 2024](https://arxiv.org/html/2609.16055#bib.bib34),[Xiao et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib36),[Zhang et al\., 2023](https://arxiv.org/html/2609.16055#bib.bib35)\]\. These methods are close to SoT in execution form because they do not always carry the entire history forward\.

The key distinction lies in*what drives selection*\. Most prior methods rely on semantic relevance, recency, generic compression objectives, or infrastructure efficiency\. SoT instead conditions historical activation on the model’s current reasoning regime\. This turns memory selection from a general context\-management tool into a component of reasoning control itself\.

### R\.4Latent reasoning and test\-time control

Recent work also explores reasoning in latent space or with alternative test\-time control\[[Hao et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib24),[Hassid et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib12),[Wang et al\., 2025](https://arxiv.org/html/2609.16055#bib.bib25)\]\. Some methods push more reasoning inside hidden computation; others show that shorter or differently allocated reasoning traces can be beneficial\. These directions are highly relevant because they challenge the naive idea that better reasoning must always mean longer explicit token chains\.

SoT is complementary to this line\. It does not replace explicit reasoning with a fully latent process, nor does it merely shorten chains\. Instead, it treats explicit sentence\-level reasoning as a trajectory whose supporting context should be routed by a compact endogenous controller\. This makes SoT a bridge between explicit\-token and latent\-control views of reasoning\.

### R\.5Positioning summary

Overall, SoT should not be read as another CoT variant, another search baseline, or another memory\-pruning heuristic\. Its central contribution is to formulate reasoning\-time control as a closed loop in which endogenous state governs sparse evidence activation and stopping\. That is a different object of study from externally scripted reasoning, descriptive hidden\-state analysis, or generic context compression\.

## Appendix SLimitations and Future Work

This paper focuses on a concrete instantiation of SoT across multiple text backbones and two scales of one multimodal backbone family; several scope boundaries naturally remain\. From a broader\-impact perspective, improving reasoning under fixed inference budgets could make capable models more usable in cost\- or latency\-sensitive settings \(e\.g\., education, healthcare\-related assistance, research tooling\), where efficiency and reliability both matter\. As with any method that can improve fluent reasoning, deployment should follow domain\-appropriate safeguards \(e\.g\., policy constraints, human oversight in high\-stakes use, and monitoring for misuse\)\.

### S\.1Current limitations

- •Performance depends on the state interface\.SoT routes memory and stopping through the extracted endogenous state; if that interface is a poor summary of the regime relevant to the next decision, control quality can degrade\.
- •The current state is intentionally compact\.Our implemented controller usesmt∈ℝ4m\_\{t\}\\in\\mathbb\{R\}^\{4\}for clarity and portability, but this choice may under\-specify unusually long, highly branching, or richly grounded trajectories where additional structure could help\.
- •Mechanistic evidence is supportive but not uniquely identifying\.Our ablations and interventions are consistent with state\-conditioned routing playing a central role, yet—as in most systems that couple representation, memory, and generation—alternative narratives cannot be ruled out in every corner case without heavier instrumentation\.
- •Empirical coverage matches the paper’s claims, not every deployment regime\.We report results across a broad multi\-task suite, but stronger proprietary models, different tool\-use/agent stacks, and safety\-critical deployments are outside our evaluated scope\.
- •Limited\-access variants are informative but not exhaustive\.Training\-free SoT, embedding\-based features, andSoT\-Judgeprobe the same interface under restricted supervision or observability; they are evaluated at a smaller footprint than the full supervised controller and are best read as exploratory complements\.

### S\.2Future work

- •Richer but still compact states\.A natural direction is to add state dimensions or structured summaries only when they measurably improve control \(e\.g\., branching, retrieval confidence, multimodal grounding\), without bloating the feedback loop\.
- •Tighter identification of what the controller uses\.Additional counterfactual replay, routing interventions, and closed\-loop stress tests could further clarify which components of the interface matter most across task families\.
- •Adaptive memory units\.Sentence\-level units are easy to audit, but variable segmentation or hybrid symbolic/latent memories may better match heterogeneous reasoning traces\.
- •Unified white\-box and black\-box trajectory control\.SoT\-Embed andSoT\-Judgesuggest a continuum between internals\-aware controllers and trajectory\-only signals; a more systematic comparison of that continuum is an interesting next step\.
- •Broader multimodal and interactive settings\.Environments with dynamic evidence—long multimodal contexts, tool use, and interactive decisions—are a natural fit for state\-conditioned organization, but will require task suites and training protocols beyond this paper\.

#### Final perspective\.

We intend SoT as a precise formulation of reasoning\-time control—not a claim that any single architecture is final\. The underlying point is empirical and conceptual: reasoning benefits from treating context activation and continuation as a feedback process coupled to the model’s evolving internal regime, rather than only as prompt programming over a growing transcript\.