Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs

# Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs
Source: [https://arxiv.org/html/2606.00726](https://arxiv.org/html/2606.00726)
Jiakang Li\*1,Guanyu Zhu\*2,Can Jin\*1,Chenxi Huang3,Dexu Yu4,Ronghao Chen5 Yang Zhou1,Hongwu Peng6,Xuanqi Lan7,Dimitris N\. Metaxas†\\dagger1,Youhua Li†\\dagger8

1Rutgers University2South China Agricultural University3Columbia University 4Fenz\.AI5QuantaAlpha6Adobe 7Santa Clara University8City University of Hong Kong Contact:\{jiakang\.li@rutgers\.edu\} \*Equal contribution\.†\\daggerEqual corresponding authors

###### Abstract

Strong reasoning depends not only on model knowledge but also on how effectively cognitive behaviors are deployed during generation\. Existing methods often rely on explicit behavior\-level control, making them insufficiently adaptive when failures and required corrections vary across reasoning states, tasks, and models\. To this end, we propose Latent Reward Steering \(Lrs\), an adaptive inference\-time framework that promotes cognitive behaviors by optimizing the sparse\-autoencoder \(SAE\) latent states that implicitly carry them\. Rather than relying on predefined cognitive behaviors or steering directions derived from them,Lrstrains a latent reward model on reasoning traces by final answer correctness to estimate the quality of intermediate latent states\. During inference, reward gradients provide state\-specific correction directions for fragile latent states, while a reward and confidence gate restricts intervention to states the reward signal flags as fragile\. Experiments on multiple reasoning LLM backbones and benchmarks show thatLrsconsistently improves performance over various baselines, and post\-hoc analyses further indicate thatLrsimplicitly promotes good cognitive behaviors that fix the original reasoning errors\. Code is available at:[https://github\.com/jiakanglee/Latent\-Reward\-Steering](https://github.com/jiakanglee/Latent-Reward-Steering)\.

Latent Reward Steering: An Adaptive Inference\-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs

Jiakang Li\*1, Guanyu Zhu\*2, Can Jin\*1, Chenxi Huang3, Dexu Yu4, Ronghao Chen5Yang Zhou1,Hongwu Peng6,Xuanqi Lan7,Dimitris N\. Metaxas†\\dagger1,Youhua Li†\\dagger81Rutgers University2South China Agricultural University3Columbia University4Fenz\.AI5QuantaAlpha6Adobe7Santa Clara University8City University of Hong KongContact:\{jiakang\.li@rutgers\.edu\}\*Equal contribution\.†\\daggerEqual corresponding authors\.

## 1Introduction

Performing step\-by\-step reasoning to solve complex problems has become a central research focus in large language models\(Weiet al\.,[2022](https://arxiv.org/html/2606.00726#bib.bib8); Kojimaet al\.,[2022](https://arxiv.org/html/2606.00726#bib.bib19)\)\. Yet even strong reasoning models remain brittle: a single early mistake, such as a flawed assumption or a skipped verification step, can gradually derail an otherwise promising reasoning chain\(Ganet al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib51); Huanget al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib36); Tyenet al\.,[2024](https://arxiv.org/html/2606.00726#bib.bib52)\)\. Recent work highlights cognitive behaviors such as verification, backtracking, and subgoal setting as important ingredients of successful reasoning\(Gandhiet al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib10)\)\. This suggests that some reasoning failures are not purely failures of model knowledge, but failures to induce cognitive behaviors at the right moments within an ongoing reasoning chain\.

![Refer to caption](https://arxiv.org/html/2606.00726v1/x1.png)Figure 1:Motivation\. A fragile reasoning state can derail reasoning, while explicit behavior\-level control may suffer from fixed labels and directions\.Lrsinstead optimizes fragile latent states with a learned reward signal and implicitly promotes useful cognitive behaviors\.The importance of cognitive behaviors in reasoning LLMs has motivated a line of work on controlling such behaviors, most of which involve explicit behavior\-level control\. Prompt\-based methods elicit desired cognitive behaviors through textual instructions, from few\-shot in\-context learning\(Brownet al\.,[2020](https://arxiv.org/html/2606.00726#bib.bib71)\)and chain\-of\-thought \(COT\) prompting\(Weiet al\.,[2022](https://arxiv.org/html/2606.00726#bib.bib8); Kojimaet al\.,[2022](https://arxiv.org/html/2606.00726#bib.bib19)\)to more specific behaviors such as sub\-goal decomposition\(Zhouet al\.,[2022](https://arxiv.org/html/2606.00726#bib.bib63); Wanget al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib67)\), strategic planning\(Zhenget al\.,[2024](https://arxiv.org/html/2606.00726#bib.bib64)\), and verification\(Wenget al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib65); Miaoet al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib68); Dhuliawalaet al\.,[2024](https://arxiv.org/html/2606.00726#bib.bib69)\)\. Representation\-level steering methods instead intervene directly on latent states\(Turneret al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib16); Zouet al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib17)\), by associating cognitive behaviors with steering directions\(Chenet al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib11)\), head\-specific interventions\(Zhanget al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib33)\), or routed behavior\-vector libraries\(Yeet al\.,[2026](https://arxiv.org/html/2606.00726#bib.bib13)\)\.

Compared to prompt\-based methods, which explicitly designate particular cognitive behaviors in text before generation begins and cannot target the step where an error actually occurs, representation\-level steering methods intervene directly during decoding, offering a more direct and promising space for cognitive behavior control\. However, this advantage in interface does not translate into adaptivity: representation\-level methods still follow the same explicit behavior\-level paradigm as prompt\-based ones, where the behaviors to control are predefined and represented as behavior\-specific intervention objects such as steering directions, selected heads, or vector libraries\. Such a paradigm is not adaptive, since it relies on predefined cognitive behaviors \(e\.g\., verification, backtracking\) that may not apply uniformly across models\(Gandhiet al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib10)\), and on steering directions derived from these predefined behaviors that may not match the local reasoning state\(Chenet al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib11)\)\. As a result, although latent\-level intervention is a promising direction, both prompt\-based and representation\-level methods remain insufficiently adaptive when failures and required corrections vary across reasoning tasks and models\.

This motivates an interesting research question:*Can we promote good cognitive behaviors adaptively at the latent level without committing to predefined behaviors or their derived steering directions?*Recent findings make this question plausible\. First, sparse latent states can be viewed as an internal space where the model’s ongoing deployment of cognitive behaviors is implicitly represented\(Wanget al\.,[2026](https://arxiv.org/html/2606.00726#bib.bib73)\)\. Second, useful cognitive mechanisms often already exist in the model’s latent space, and gains can come from deploying them better rather than from injecting new behaviors\(Venhoffet al\.,[2025a](https://arxiv.org/html/2606.00726#bib.bib18); Wardet al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib34)\)\. Third, recent work shows that latent representations themselves encode reward\-like quality signals that can be recovered by a learned model\(Duet al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib70)\)\. Building on these observations, we therefore hypothesize that reward\-guided optimization of latent states can adaptively promote the good cognitive behaviors already encoded in latent states during reasoning, without ever using explicit behavior control\.

Based on this hypothesis, we proposeLrs, an adaptive inference\-time framework that promotes cognitive behaviors by directly optimizing SAE latent states\(Cunninghamet al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib27); Templetonet al\.,[2024](https://arxiv.org/html/2606.00726#bib.bib28)\)with a learned reward signal\. Rather than relying on predefined cognitive behaviors or steering directions derived from them,Lrstrains a latent reward model on successful and unsuccessful reasoning traces to estimate the quality of intermediate latent states\. During inference, the reward gradient supplies a state\-specific correction direction for the current latent, while a reward and confidence gate restricts intervention to states the reward signal flags as fragile\-prone, helping preserve reasoning steps that are already likely to be healthy\.

Our main contributions are summarized below:

- •To the best of our knowledge, we are the first to frame cognitive behavior control for LLM reasoning as*implicit latent\-state optimization*, shifting the focus from explicitly selecting predefined behaviors to adaptively optimizing latent states that represent ongoing cognitive behavior deployment\.
- •We introduceLrs, an adaptive inference\-time framework that steers fragile SAE latent states through reward\-guided correction together with reward and confidence gating, without relying on predefined cognitive behaviors or their derived steering directions\.
- •We show thatLrsconsistently improves inference\-time reasoning across multiple LLMs and challenging benchmarks, while qualitative and case\-level analyses suggest that it implicitly promotes helpful cognitive behaviors such as solution verification and course correction\.

## 2Related Work

#### Inference\-time reasoning and prompt\-based behavior control\.

Inference\-time reasoning has become an important way to improve LLM performance on complex tasks\. Few\-shot in\-context learning and COT prompting elicit general step\-by\-step reasoning behavior\(Brownet al\.,[2020](https://arxiv.org/html/2606.00726#bib.bib71); Weiet al\.,[2022](https://arxiv.org/html/2606.00726#bib.bib8); Kojimaet al\.,[2022](https://arxiv.org/html/2606.00726#bib.bib19); Jinet al\.,[2025b](https://arxiv.org/html/2606.00726#bib.bib82)\)\. Later prompting methods target more specific cognitive behaviors, including sub\-goal decomposition\(Zhouet al\.,[2022](https://arxiv.org/html/2606.00726#bib.bib63); Wanget al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib67); Jinet al\.,[2025a](https://arxiv.org/html/2606.00726#bib.bib83)\), strategic planning\(Zhenget al\.,[2024](https://arxiv.org/html/2606.00726#bib.bib64)\), and verification\(Wenget al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib65); Miaoet al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib68); Dhuliawalaet al\.,[2024](https://arxiv.org/html/2606.00726#bib.bib69); Jinet al\.,[2025c](https://arxiv.org/html/2606.00726#bib.bib85); Zhanget al\.,[2026b](https://arxiv.org/html/2606.00726#bib.bib87)\)\.

#### Representation\-level steering for reasoning\.

Activation steering and representation engineering provide a more direct way to influence model behavior by modifying internal states during generation\(Turneret al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib16); Zouet al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib17); Jinet al\.,[2026](https://arxiv.org/html/2606.00726#bib.bib84); Zhanget al\.,[2026a](https://arxiv.org/html/2606.00726#bib.bib86)\)\. Recent work applies this idea to reasoning control\. SEAL decomposes reasoning traces into components such as execution, reflection, and transition, and learns steering vectors to calibrate them\(Chenet al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib11)\)\. CREST identifies attention heads associated with behaviors such as verification and backtracking, and derives head\-specific steering directions\(Zhanget al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib33)\)\. RISER builds a reusable library of reasoning vectors and learns a router to compose them during inference\(Yeet al\.,[2026](https://arxiv.org/html/2606.00726#bib.bib13)\)\. These methods make representation\-level intervention a promising interface for cognitive behavior control, but remain tied to predefined behaviors, selected heads, fixed directions, or finite vector libraries\.

#### Cognitive behaviors and implicit latent\-state optimization\.

Recent studies highlight the role of cognitive behaviors in LLM reasoning\. Cognitive behaviors such as verification, backtracking, and subgoal setting are important ingredients of strong reasoning performance\(Gandhiet al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib10)\)\. Other work suggests that useful reasoning mechanisms may already exist in base models, and that gains can come from better deployment of these mechanisms rather than from adding new knowledge\(Venhoffet al\.,[2025a](https://arxiv.org/html/2606.00726#bib.bib18); Wardet al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib34); Venhoffet al\.,[2025b](https://arxiv.org/html/2606.00726#bib.bib25)\)\. These findings motivate cognitive behavior control, but also reveal the limitation of explicit behavior\-level methods: predefined behaviors may not apply uniformly across models, and fixed intervention directions may not match the current reasoning state\. In contrast,Lrsframes cognitive behavior control as implicit latent\-state optimization, using latent states to adaptively steer fragile reasoning states during decoding\.

## 3Method

### 3\.1Problem Setup

![Refer to caption](https://arxiv.org/html/2606.00726v1/x2.png)Figure 2:Framework ofLrs\. It first constructs SAE latent traces from reasoning trajectories, then trains a latent reward model to estimate intermediate\-state quality\. During inference, a reward and confidence gate identifies fragile states, and reward\-guided latent correction updates the hidden state before decoding continues\.We formalize the problem of*adaptive cognitive behavior promotion for LLM reasoning*as an inference\-time intervention on the model’s latent states\. Let a parameter\-frozen reasoning modelℳ\\mathcal\{M\}process an input promptxxand autoregressively generate a responsey=\(y1,…,yT\)y=\(y\_\{1\},\\dots,y\_\{T\}\)token by token\. At generation steptt, letht∈ℝdhh\_\{t\}\\in\\mathbb\{R\}^\{d\_\{h\}\}denote the hidden activation at layerℓ\\ell\. As motivated in Section[1](https://arxiv.org/html/2606.00726#S1), we viewhth\_\{t\}as an internal state that implicitly carries the model’s intrinsic deployment of cognitive behaviors\.

#### SAE\-based intervention space\.

Directly steering the dense activationhth\_\{t\}is difficult because it is high\-dimensional and entangled\. We therefore use a pretrained sparse autoencoder \(SAE\) to maphth\_\{t\}into a low\-dimensional sparse latent representation:

zt=fenc​\(ht\),h^t=fdec​\(zt\),z\_\{t\}=f\_\{\\mathrm\{enc\}\}\(h\_\{t\}\),\\qquad\\hat\{h\}\_\{t\}=f\_\{\\mathrm\{dec\}\}\(z\_\{t\}\),\(1\)wherezt∈ℝdzz\_\{t\}\\in\\mathbb\{R\}^\{d\_\{z\}\}is adzd\_\{z\}\-dimensional sparse code\. We use the pretrained SAEs released byVenhoffet al\.\([2025a](https://arxiv.org/html/2606.00726#bib.bib18)\), which were trained on each reasoning model’s hidden activations and provide a model\-specific sparse code:dz=10d\_\{z\}=10for Open\-Reasoner\-7BHuet al\.\([2026](https://arxiv.org/html/2606.00726#bib.bib75)\)anddz=5d\_\{z\}=5for Open\-Reasoner\-1\.5BHuet al\.\([2026](https://arxiv.org/html/2606.00726#bib.bib75)\)\. Prior work suggests that SAE latents expose interpretable internal features in language models\(Cunninghamet al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib27); Templetonet al\.,[2024](https://arxiv.org/html/2606.00726#bib.bib28)\), and that SAE latent dimensions can be used to identify and analyze reasoning\-relevant cognitive behaviors such as verification, backtracking, and constraint checking in the model’s internal states\(Venhoffet al\.,[2025a](https://arxiv.org/html/2606.00726#bib.bib18); Wardet al\.,[2025](https://arxiv.org/html/2606.00726#bib.bib34); Wanget al\.,[2026](https://arxiv.org/html/2606.00726#bib.bib73)\)\. We therefore use SAE latents as the intervention space forLrs\.

#### Inference\-time steering objective\.

Given a decoding steptt, our goal is to obtain a corrected latent statezt′=zt\+Δ​ztz^\{\\prime\}\_\{t\}=z\_\{t\}\+\\Delta z\_\{t\}that improves the likelihood of a correct final answer when decoding continues from the corresponding hidden state\. The update is adaptive to the current state, free from predefined cognitive behavior labels or behavior\-specific steering directions, and selective enough to avoid disrupting already healthy reasoning states\.

### 3\.2Method Overview

As shown in Figure[2](https://arxiv.org/html/2606.00726#S3.F2),Lrshas three stages\. It first constructs SAE latent reasoning traces from a frozen reasoning model and labels each trace by final\-answer correctness\. It then trains a latent reward model to estimate the quality of intermediate latent states from successful and unsuccessful traces, without using explicit cognitive behavior annotations\. During inference,Lrsuses the learned reward signal to identify fragile states and applies reward\-guided latent correction only when the reward and confidence gate triggers intervention\. The decoded latent residual is added back to the hidden activation, enabling adaptive steering of fragile reasoning states without predefined behavior labels or behavior\-specific directions\.

### 3\.3Latent Reward Training

We first construct a dataset of sparse latent reasoning traces\. For each solved example, we store a latent sequence together with a binary label indicating whether the final answer is correct\. Formally, each example is represented as

𝒮i=\(zi,si,zi,si\+1,…,zi,Ti\),yi∈\{0,1\},\\mathcal\{S\}\_\{i\}=\(z\_\{i,s\_\{i\}\},z\_\{i,s\_\{i\}\+1\},\\dots,z\_\{i,T\_\{i\}\}\),\\qquad y\_\{i\}\\in\\\{0,1\\\},\(2\)wheresis\_\{i\}is the beginning of the reasoning segment\.

We train a lightweight Transformer reward modelRθR\_\{\\theta\}over these latent sequences:

pi,t=Rθ​\(zi,si:t\)∈\(0,1\),p\_\{i,t\}=R\_\{\\theta\}\(z\_\{i,s\_\{i\}:t\}\)\\in\(0,1\),\(3\)wherepi,tp\_\{i,t\}is the predicted probability that the reasoning trace belongs to a correct sample\. In implementation, the binary labelyiy\_\{i\}is repeated over all positions of the sequence, and the model is trained using binary cross\-entropy:

ℒRM=−∑i∑t=siTi\[yi​log⁡pi,t\+\(1−yi\)​log⁡\(1−pi,t\)\]\.\\mathcal\{L\}\_\{\\mathrm\{RM\}\}=\-\\sum\_\{i\}\\sum\_\{t=s\_\{i\}\}^\{T\_\{i\}\}\\Big\[y\_\{i\}\\log p\_\{i,t\}\+\(1\-y\_\{i\}\)\\log\(1\-p\_\{i,t\}\)\\Big\]\.\(4\)To mitigate class imbalance, we apply weighted random sampling during training\.

The reward model itself is a small Transformer encoder with LayerNorm on the latent input, a learned embedding layer, positional encoding, two Transformer blocks, and an MLP head\.

### 3\.4Online Reward\-Guided Latent Correction

At generation time, we intervene only on generation tokens, not on the prompt prefix\. Given the current activationhth\_\{t\}, we encode it into a sparse latent vector built from the SAE encoder:

zt\(0\)=fenc​\(ht\)\.z\_\{t\}^\{\(0\)\}=f\_\{\\mathrm\{enc\}\}\(h\_\{t\}\)\.\(5\)We then evaluate the reward model on the current latent \(implemented as a sequence of length11for efficiency\) and obtain an initial reward score

rt\(0\)=Rθ​\(zt\(0\)\)\.r\_\{t\}^\{\(0\)\}=R\_\{\\theta\}\(z\_\{t\}^\{\(0\)\}\)\.\(6\)If steering is triggered at this step \(𝕀steer​\(t\)=1\\mathbb\{I\}\_\{\\mathrm\{steer\}\}\(t\)=1, with the gating rule defined in Section[3\.5](https://arxiv.org/html/2606.00726#S3.SS5)\), we optimize the latent by normalized gradient ascent forKKsteps:

zt\(k\+1\)=zt\(k\)\+α​∇zt\(k\)Rθ​\(zt\(k\)\)‖∇zt\(k\)Rθ​\(zt\(k\)\)‖2\+ε,z\_\{t\}^\{\(k\+1\)\}=z\_\{t\}^\{\(k\)\}\+\\alpha\\frac\{\\nabla\_\{z\_\{t\}^\{\(k\)\}\}R\_\{\\theta\}\(z\_\{t\}^\{\(k\)\}\)\}\{\\left\\\|\\nabla\_\{z\_\{t\}^\{\(k\)\}\}R\_\{\\theta\}\(z\_\{t\}^\{\(k\)\}\)\\right\\\|\_\{2\}\+\\varepsilon\},\(7\)whereα\\alphais the step size andKKis the number of steering iterations\.

Instead of decoding the entire optimized latent, we only decode the latent*difference*

Δ​zt=zt\(K\)−zt\(0\),\\Delta z\_\{t\}=z\_\{t\}^\{\(K\)\}\-z\_\{t\}^\{\(0\)\},\(8\)and project it back to activation space with the SAE decoder matrixWdecW\_\{\\mathrm\{dec\}\}:

Δ​ht=Δ​zt​Wdec,ht′=ht\+Δ​ht\.\\Delta h\_\{t\}=\\Delta z\_\{t\}W\_\{\\mathrm\{dec\}\},\\qquad h\_\{t\}^\{\\prime\}=h\_\{t\}\+\\Delta h\_\{t\}\.\(9\)The steered hidden stateht′h\_\{t\}^\{\\prime\}is then fed into the subsequent layers of the LLM\.

### 3\.5Selective Reward and Confidence Gating

Applying latent updates at every generation step may disrupt already\-correct reasoning, as the ungated variantLrsBasicdegrades on several benchmarks \(Table[1](https://arxiv.org/html/2606.00726#S4.T1)\)\. We therefore introduce a selective gate based on two signals: the current reward scorert=Rθ​\(zt\(0\)\)r\_\{t\}=R\_\{\\theta\}\(z\_\{t\}^\{\(0\)\}\)and the previous\-token decoding confidencect−1c\_\{t\-1\}, defined as the maximum softmax probability at stept−1t\-1\.

Steering is triggered when either signal suggests a fragile local state:

𝕀steer​\(t\)=\{1,if​rt<τr,1,if​rt≥τr​and​ct−1<τc,0,otherwise,\\mathbb\{I\}\_\{\\mathrm\{steer\}\}\(t\)=\\begin\{cases\}1,&\\text\{if \}r\_\{t\}<\\tau\_\{r\},\\\\ 1,&\\text\{if \}r\_\{t\}\\geq\\tau\_\{r\}\\text\{ and \}c\_\{t\-1\}<\\tau\_\{c\},\\\\ 0,&\\text\{otherwise\},\\end\{cases\}\(10\)whereτr\\tau\_\{r\}andτc\\tau\_\{c\}are the reward and confidence thresholds\. Low reward indicates that the current latent state is unlikely to lead to a correct answer, while low decoding confidence suggests local uncertainty even when the reward is acceptable\. Thus,Lrsintervenes only when correction is needed and preserves states that appear already on track\.

## 4Experiments

In this paper, we study the following research questions:

- •RQ1:CanLrsimprove reasoning performance?
- •RQ2:Does the latent reward model provide a meaningful correction signal?
- •RQ3:DoLrs\-steered traces implicitly promote useful cognitive behaviors?

We further conduct ablation and sensitivity analysis to examine the role of selective intervention and steering strength\.

\\rowcolorgray\!10DatasetOpen\-Reasoner\-7BOpen\-Reasoner\-1\.5B\\rowcolorgray\!10Base0\-shotCoT5\-shotLrsBasic0\-shotLrs0\-shotBase0\-shotCoT5\-shotLrsBasic0\-shotLrs0\-shotMATH\-50079\.481\.481\.083\.0\(\+3\.6\)83\.8\(\+4\.4\)59\.258\.657\.259\.0\(−\-0\.2\)60\.8\(\+1\.6\)AIME2416\.613\.313\.316\.6\(\+0\.0\)26\.6\(\+10\.0\)3\.36\.76\.70\.0\(−\-3\.3\)13\.3\(\+10\.0\)AIME2516\.613\.310\.020\.0\(\+3\.4\)26\.6\(\+10\.0\)3\.30\.03\.30\.0\(−\-3\.3\)6\.6\(\+3\.3\)GPQA\-Diamond32\.335\.938\.430\.8\(−\-1\.5\)39\.4\(\+7\.1\)18\.217\.217\.215\.7\(−\-2\.5\)22\.8\(\+4\.6\)AMC2350\.055\.065\.045\.0\(−\-5\.0\)60\.0\(\+10\.0\)30\.032\.530\.032\.5\(\+2\.5\)37\.5\(\+7\.5\)IneqMath46\.048\.048\.052\.0\(\+6\.0\)60\.0\(\+14\.0\)29\.030\.028\.028\.0\(−\-1\.0\)34\.0\(\+5\.0\)

Table 1:Main results are all reported under greedy decoding with maximum token budget 4000\. Values in parentheses denote absolute gains ofLrsBASIC /Lrsover the corresponding Base model\. CoT and few\-shot columns report accuracies under chain\-of\-thought and few\-shot prompting, respectively\.LrsBASIC is ungated, while fullLrsuses reward and confidence gating\.### 4\.1Experimental Setup

We evaluateLrswithOpen\-Reasoner\-7Bas the primary base model and additionally includeOpen\-Reasoner\-1\.5Bto assess generalization across model scales\. We adopt the pretrained SAE checkpoints released byVenhoffet al\.\([2025a](https://arxiv.org/html/2606.00726#bib.bib18)\): steering is applied at layer 20 in a 10\-dimensional SAE latent space forOpen\-Reasoner\-7B, and in a 5\-dimensional SAE latent space forOpen\-Reasoner\-1\.5BHuet al\.\([2026](https://arxiv.org/html/2606.00726#bib.bib75)\)\. The main results in Table[1](https://arxiv.org/html/2606.00726#S4.T1)are reported for both backbones, while most qualitative and diagnostic analyses \(reward\-score separation, post\-hoc cognitive behavior annotation, SAE interpretability, efficiency, and case studies\) are conducted on the primaryOpen\-Reasoner\-7BmodelHuet al\.\([2026](https://arxiv.org/html/2606.00726#bib.bib75)\)\.

We evaluate on six reasoning benchmarks:MATH\-500\(Hendryckset al\.,[2021](https://arxiv.org/html/2606.00726#bib.bib59)\),AIME24Art of Problem Solving \([2024a](https://arxiv.org/html/2606.00726#bib.bib77),[b](https://arxiv.org/html/2606.00726#bib.bib78)\),AIME25Art of Problem Solving \([2025a](https://arxiv.org/html/2606.00726#bib.bib79),[b](https://arxiv.org/html/2606.00726#bib.bib80)\),AMC23math\-ai \([2025](https://arxiv.org/html/2606.00726#bib.bib81)\),IneqMath\(Shenget al\.,[2026](https://arxiv.org/html/2606.00726#bib.bib61)\), andGPQA\-Diamond\(Reinet al\.,[2023](https://arxiv.org/html/2606.00726#bib.bib62)\)\. All benchmarks are evaluated in a zero\-shot setting with greedy decoding and batch size11\. More detailed training and experimental settings are provided in Appendix[A](https://arxiv.org/html/2606.00726#A1)

### 4\.2Main Results:LrsImproves Reasoning Performance \(RQ1\)

#### Lrsimproves reasoning across model scales\.

As shown in Table[1](https://arxiv.org/html/2606.00726#S4.T1), fullLrsimproves over standard zero\-shot decoding on*all*six reasoning benchmarks forOpen\-Reasoner\-7B, with gains ranging from\+4\.4\+4\.4on theMATH\-500to\+14\.0\+14\.0on theIneqMath, and we observe the same trend on the smaller same family modelOpen\-Reasoner\-1\.5B, indicating that the method generalizes across model scales\. These improvements all come from inference\-time latent steering alone, without any training on the model weights\.

#### Recovery exceeds degradation\.

To better understandLrs’s effect on reasoning improvement at the trace level, we perform a matched\-pair analysis between base andLrsreasoning chains \(N=398N=398,Open\-Reasoner\-7B\)\. As shown in Table[2](https://arxiv.org/html/2606.00726#S4.T2),Lrsrecovers 17\.3% of examples from wrong to correct while degrading only 7\.8% from correct to wrong, with a net positive effect of\+9\.5%\+9\.5\\%\. This indicates that the gains in Table[1](https://arxiv.org/html/2606.00726#S4.T1)come fromLrsactively recovering failed reasoning traces rather than randomly perturbing them: whenLrsintervenes, it is more than twice as likely to fix a wrong reasoning chain as to break a correct one\.

#### Lrsconsistently outperforms prompt\-based behavior control\.

Lrsoutperforms both CoT and few\-shot prompting on five of the six benchmarks forOpen\-Reasoner\-7Band on all six forOpen\-Reasoner\-1\.5B\. In contrast, prompt\-based baselines are unstable: CoT and few\-shot prompting underperform standard zero\-shot decoding on several benchmarks \(e\.g\.,−3\.3\-3\.3for CoT onAIME24andAIME25withOpen\-Reasoner\-7B\), consistent with observations that reasoning\-tuned models already internalize CoT\-style reasoning behavior so adding explicit instructions does not necessarily help\(Venhoffet al\.,[2025a](https://arxiv.org/html/2606.00726#bib.bib18)\)\. These results empirically answer RQ1:Lrsimproves reasoning performance across models and benchmarks\.

TransitionBase→\\rightarrowLrsRateImprovedWrong→\\rightarrowCorrect17\.3%DegradedCorrect→\\rightarrowWrong7\.8%PreservedCorrect→\\rightarrowCorrect28\.6%UnresolvedWrong→\\rightarrowWrong46\.3%Table 2:Matched outcome transitions between base andLrstrace \(N=398N=398,Open\-Reasoner\-7B\)\.![Refer to caption](https://arxiv.org/html/2606.00726v1/x3.png)Figure 3:Latent reward scores distinguish successful and failed reasoning traces\. We compare final\-token rewardplastp\_\{\\mathrm\{last\}\}and trace\-mean rewardpmeanp\_\{\\mathrm\{mean\}\}between correct and incorrect generations\.

### 4\.3Latent Reward Signal Quality \(RQ2\)

We next examine whether the learned latent reward model captures a signal that is informative of reasoning quality\. For each generated reasoning trace, the reward model assigns token\-level sigmoid scores over the SAE latent sequence\. Although trained only with trace\-level correctness, the reward model uses token\-level scores because local reasoning errors often propagate, making intermediate latent quality useful for selective correction\. We summarize these scores using two diagnostic statistics: the final\-token rewardplastp\_\{\\mathrm\{last\}\}, which reflects the reward estimate at the end of generation, and the trace\-level mean rewardpmeanp\_\{\\mathrm\{mean\}\}, which reflects the average quality signal across the reasoning process\.

Figure[3](https://arxiv.org/html/2606.00726#S4.F3)shows that correct reasoning traces generally receive higher reward scores than incorrect reasoning traces across the evaluated datasets\. The separation is most pronounced onAMC23,AIME24, andAIME25, whileGPQA\-DiamondandIneqMathshow the same ordering with smaller gaps\. This suggests that the reward model captures reasoning\-quality differences in latent space\. Importantly, this analysis is not final\-answer verification and not behavior classification: the reward model is trained only with final correctness labels over latent traces, without behavior annotations, answer\-extraction labels, or predefined cognitive categories\. This empirically answers RQ2, showing that the latent reward model provides a meaningful correction signal\.

### 4\.4LrsImplicitly Promotes Useful Cognitive Behaviors \(RQ3\)

We finally examine whether reward\-guided latent steering is associated with interpretable changes in cognitive behavior\. This analysis is diagnostic only: behavior labels are never used to train the reward model, construct steering vectors, trigger the reward and confidence gate, or guide inference\. Instead, we annotate matched base andLrsreasoning chains post hoc to examine whether latent reward steering changes the observable reasoning process by promoting useful cognitive behaviors\. We conduct a matched\-pair analysis between base andLrsreasoning chains \(N=398N=398,Open\-Reasoner\-7B\) and annotate five cognitive behaviors followingGandhiet al\.\([2025](https://arxiv.org/html/2606.00726#bib.bib10)\):*Strategic Planning*,*Structured Decomposition*,*Constraint Grounding*,*Course Correction*, and*Solution Verification*\. Annotations are produced by GPT\-4o using a fixed post\-hoc judging prompt, which is provided in Appendix[B](https://arxiv.org/html/2606.00726#A2)\. For each trace, a behavior is counted as present if it appears at least once\.

#### Post\-hoc behavior frequency\.

Figure[5](https://arxiv.org/html/2606.00726#S4.F5)reports the occurrence rates of the five annotated cognitive behaviors in matched base andLrsreasoning chains\.Lrs\-steered traces show higher occurrence of*Course Correction*\(\+0\.20\+0\.20\),*Constraint Grounding*\(\+0\.10\+0\.10\), and*Solution Verification*\(\+0\.10\+0\.10\)\. These behaviors are closely related to local reasoning correction: revisiting potentially wrong steps, checking consistency with problem constraints, and auditing the final derivation or answer\. This pattern is consistent with the mechanism ofLrs\. SinceLrsdoes not use behavior annotations, behavior\-specific vectors, or predefined behavior triggers, these shifts should not be interpreted as explicit behavior selection\. Rather, they suggest that reward\-guided latent optimization changes fragile reasoning steps in ways associated with implicitly promoting more frequent helpful cognitive behaviors through latent\-state optimization\.

#### Case study\.

Case Study: AIME25 Q2 — Counting with Strict ConstraintsQuestionNine players choose chocolate, vanilla, or strawberry; each flavor appears and the counts satisfyc\>v\>sc\>v\>s\. Count assignments modulo10001000\.Base ErrorThe base sets upc\+v\+s=9c\+v\+s=9but keeps only\(5,3,1\)\(5,3,1\), giving\(95,3,1\)=504\\binom\{9\}\{5,3,1\}=504\. It rejects\(6,2,1\)\(6,2,1\)and\(4,3,2\)\(4,3,2\)despite6\>2\>16\>2\>1and4\>3\>24\>3\>2\.LrsCorrectionLrsrevisits the enumeration and keeps all valid partitions:\(6,2,1\)\(6,2,1\),\(5,3,1\)\(5,3,1\), and\(4,3,2\)\(4,3,2\), with counts252252,504504, and12601260; hence2016≡16\(mod1000\)2016\\equiv 16\\pmod\{1000\}\.Diagnostic PathwayEarly reward and confidence interventions occur during count branching and after steering, the trace rechecks discarded cases and strict ordering\.InterpretationThe case illustrates*Structured Decomposition*,*Constraint Grounding*, and*Course Correction*\. SAE dimension names remain post\-hoc priors, not deterministic causal labels\.Figure 4:Lrsrepairs an incomplete enumeration by recovering the missed valid partitions\.The compact case below illustrates the mechanism suggested by the aggregate analyses\. The base reasoning commits to an incomplete enumeration and fails to revisit excluded cases, leading to an incorrect answer\. In contrast, theLrs\-steered trace receives early latent interventions when the reward model signal indicates that enumeration steps are fragile\. After reward\-guided optimization at the fragile enumeration stage, cognitive behavior relevant SAE latent dimensions associated with algebraic execution and variable extraction become more active, and the trace later revisits the missing cases to produce a more complete solution\. Additional qualitative cases with fuller diagnostics are provided in Appendix[D](https://arxiv.org/html/2606.00726#A4)and Figure[12](https://arxiv.org/html/2606.00726#A4.F12)\. Together with the aggregate behavior\-frequency analysis, this case provides along with more cases[D](https://arxiv.org/html/2606.00726#A4)provide post\-hoc evidence for RQ3:Lrs\-steered reasoning traces are associated with implicitly promoting more frequent useful cognitive behaviors\.

![Refer to caption](https://arxiv.org/html/2606.00726v1/x4.png)Figure 5:Post\-hoc analysis shows thatLrs\-steered traces more often exhibit course correction, constraint grounding, and solution verification, with behavior labels used only for analysis\.

### 4\.5Additional Analyses: Stability, Interpretability, and Efficiency

We include 3 supporting analyses to examine the stability, interpretability, and practical cost ofLrs\.

#### Selective intervention and steering strength\.

Table[1](https://arxiv.org/html/2606.00726#S4.T1)shows thatLrsBasicimproves some datasets but degrades others, indicating that reward\-guided gradients should not be applied indiscriminately\. FullLrsuses the reward and confidence gate as a selective repair mechanism, and Figure[6](https://arxiv.org/html/2606.00726#S4.F6)shows that stronger updates do not monotonically improve accuracy and moderate intervention is more stable\.

#### SAE interpretability\.

Table[3](https://arxiv.org/html/2606.00726#S4.T3)summarizes max\-activating\-context interpretations of the 10 SAE dimensions onMATH\-500\. These names provide post\-hoc priors for analyzing latent changes, not deterministic mappings from dimensions to behaviors\.

![Refer to caption](https://arxiv.org/html/2606.00726v1/x5.png)Figure 6:Steering\-strength sensitivity onAIME24\. Accuracy peaks under moderate updates rather than increasing monotonically and the dashed line indicates standard decoding\.Dim\.Max Act\.Interpreted Patternz0z\_\{0\}0\.218Geometric / structural modelingz1z\_\{1\}0\.287Theorem invocation / logical branchingz2z\_\{2\}0\.477Algebraic flow / step executionz3z\_\{3\}0\.582Symbolic math / formattingz4z\_\{4\}0\.552Variable initialization / constant extractionz5z\_\{5\}0\.438Property definition / formula groundingz6z\_\{6\}0\.405Conclusion / answer consolidationz7z\_\{7\}0\.256Constraint checking / boundary validationz8z\_\{8\}0\.455Strategy selection / planningz9z\_\{9\}0\.283Complexity / asymptotic reasoning

Table 3:Compact interpretation of SAE latent dimensions from max\-activating contexts onMATH\-500\.
#### Inference efficiency\.

Table[4](https://arxiv.org/html/2606.00726#S4.T4)reports the inference overhead ofLrs\. Average wall\-clock time increases from 115\.3s to 156\.2s per problem \(1\.35×1\.35\\times\), while the reward and confidence gate skips 72\.1% of tokens and steers 27\.9%\.

MetricBaseLrsAvg\. generated tokens25952596Avg\. wallclock / problem \(s\)115\.3156\.2Avg\. generation cost \(ms / token\)44\.560\.2Slowdown ratio1\.00×\\times1\.35×\\timesSteered tokens \(%\)—27\.9Unsteered tokens \(%\)—72\.1Avg\. steering triggers / problem—725Triggers on correct answers—526Triggers on wrong answers—924Avg\.‖Δ​z‖\\\|\\Delta z\\\|per steered token—1\.24SAE latent dimension—10Table 4:Inference efficiency ofBaseandLrson 200 problems fromAIME24,AIME25,AMC23, andIneqMath\.

## 5Conclusion

We presentedLatent Reward Steering, an adaptive inference\-time framework that improves LLM reasoning by applying reward\-guided optimization to SAE latent states\.Lrslearns a latent reward model from successful and unsuccessful reasoning traces, uses reward gradients to correct fragile token\-level states, and applies a reward–confidence gate to keep intervention selective\. Experiments onOpen\-Reasoner\-7BandOpen\-Reasoner\-1\.5Bshow consistent gains across reasoning benchmarks without updating model weights\. Reward\-score separation, post\-hoc behavior analysis, and case studies further suggest thatLrs\-steered traces are associated with more frequent useful cognitive behaviors\. These findings support latent\-state optimization as a promising direction for inference\-time reasoning improvement\.

## Limitations

This work has several limitations as follows\.

- •Inference overhead\.Lrsrequires reward\-model evaluation and latent\-gradient updates during decoding\. Selective gating reduces unnecessary interventions, butLrsremains slower than standard decoding\.
- •Training–inference mismatch\.The reward model is trained on latent reasoning sequences but queried on local token\-level states for efficiency\. Future work could explore prefix\-level or memory\-augmented reward estimation\.

## Ethical considerations

Lrsperforms inference\-time latent steering based on a learned reward signal, which may introduce risks if the reward model is misaligned or poorly calibrated\. In such cases, steering could amplify undesirable reasoning patterns, increase overconfidence in incorrect answers, or produce behavioral shifts that are difficult to interpret\. Since latent\-space interventions are less transparent than explicit prompting, their effects in open\-ended and safety\-sensitive scenarios remain uncertain\. Our evaluation is limited to mathematical and scientific reasoning benchmarks, and does not fully characterize these risks\. Future use of LRS should involve reward\-model auditing, broader safety evaluation, and monitoring for unintended changes in model cognitive behavior\.

## References

- 2024 aime i problems and solutions\.Note:[https://artofproblemsolving\.com/wiki/index\.php/2024\_AIME\_I](https://artofproblemsolving.com/wiki/index.php/2024_AIME_I)Accessed: 2026\-05\-24Cited by:[§4\.1](https://arxiv.org/html/2606.00726#S4.SS1.p2.1)\.
- Art of Problem Solving \(2024b\)2024 aime ii problems and solutions\.Note:[https://artofproblemsolving\.com/wiki/index\.php/2024\_AIME\_II](https://artofproblemsolving.com/wiki/index.php/2024_AIME_II)Accessed: 2026\-05\-24Cited by:[§4\.1](https://arxiv.org/html/2606.00726#S4.SS1.p2.1)\.
- Art of Problem Solving \(2025a\)2025 aime i problems and solutions\.Note:[https://artofproblemsolving\.com/wiki/index\.php/2025\_AIME\_I](https://artofproblemsolving.com/wiki/index.php/2025_AIME_I)Accessed: 2026\-05\-24Cited by:[§4\.1](https://arxiv.org/html/2606.00726#S4.SS1.p2.1)\.
- Art of Problem Solving \(2025b\)2025 aime ii problems and solutions\.Note:[https://artofproblemsolving\.com/wiki/index\.php/2025\_AIME\_II](https://artofproblemsolving.com/wiki/index.php/2025_AIME_II)Accessed: 2026\-05\-24Cited by:[§4\.1](https://arxiv.org/html/2606.00726#S4.SS1.p2.1)\.
- T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. Amodei \(2020\)Language models are few\-shot learners\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 1877–1901\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf)Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- R\. Chen, Z\. Zhang, J\. Hong, S\. Kundu, and Z\. Wang \(2025\)SEAL: steerable reasoning calibration of large language models for free\.InConference on Language Modeling \(COLM\),Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§1](https://arxiv.org/html/2606.00726#S1.p3.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px2.p1.1)\.
- H\. Cunningham, A\. Ewart, L\. Riggs, R\. Huben, and L\. Sharkey \(2023\)Sparse autoencoders find highly interpretable features in language models\.arXiv preprint arXiv:2309\.08600\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p5.1),[§3\.1](https://arxiv.org/html/2606.00726#S3.SS1.SSS0.Px1.p1.6)\.
- S\. Dhuliawala, M\. Komeili, J\. Xu, R\. Raileanu, X\. Li, A\. Celikyilmaz, and J\. Weston \(2024\)Chain\-of\-verification reduces hallucination in large language models\.InFindings of the association for computational linguistics: ACL 2024,pp\. 3563–3578\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- H\. Du, Y\. Dong, and X\. Ning \(2025\)Latent thinking optimization: your latent reasoning language model secretly encodes reward signals in its latent thoughts\.arXiv preprint arXiv:2509\.26314\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p4.1)\.
- Z\. Gan, Y\. Liao, and Y\. Liu \(2025\)Rethinking external slow\-thinking: from snowball errors to probability of correct reasoning\.arXiv preprint arXiv:2501\.15602\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p1.1)\.
- K\. Gandhi, A\. Chakravarthy, A\. Singh, N\. Lile, and N\. D\. Goodman \(2025\)Cognitive behaviors that enable self\-improving reasoners, or, four habits of highly effective stars\.InConference on Language Modeling \(COLM\),Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p1.1),[§1](https://arxiv.org/html/2606.00726#S1.p3.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px3.p1.1),[§4\.4](https://arxiv.org/html/2606.00726#S4.SS4.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021\)Measuring mathematical problem solving with the math dataset\.arXiv preprint arXiv:2103\.03874\.Cited by:[§4\.1](https://arxiv.org/html/2606.00726#S4.SS1.p2.1)\.
- J\. Hu, Y\. Zhang, Q\. Han, D\. Jiang, X\. Zhang, and H\. Shum \(2026\)Open\-reasoner\-zero: an open source approach to scaling up reinforcement learning on the base model\.Advances in Neural Information Processing Systems38,pp\. 162239–162262\.Cited by:[§3\.1](https://arxiv.org/html/2606.00726#S3.SS1.SSS0.Px1.p1.6),[§4\.1](https://arxiv.org/html/2606.00726#S4.SS1.p1.1)\.
- J\. Huang, X\. Chen, S\. Mishra, H\. S\. Zheng, A\. W\. Yu, X\. Song, and D\. Zhou \(2023\)Large language models cannot self\-correct reasoning yet\.arXiv preprint arXiv:2310\.01798\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p1.1)\.
- C\. Jin, H\. Peng, Q\. Zhang, Y\. Tang, T\. Che, and D\. N\. Metaxas \(2025a\)Two heads are better than one: test\-time scaling of multi\-agent collaborative reasoning\.InWorkshop on Scaling Environments for Agents,External Links:[Link](https://openreview.net/forum?id=aLGgp4FK0A)Cited by:[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Jin, H\. Peng, S\. Zhao, Z\. Wang, W\. Xu, L\. Han, J\. Zhao, K\. Zhong, S\. Rajasekaran, and D\. N\. Metaxas \(2025b\)Apeer: automatic prompt engineering enhances large language model reranking\.InCompanion Proceedings of the ACM on Web Conference 2025,pp\. 2494–2502\.Cited by:[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Jin, R\. Wu, T\. Che, Q\. Zhang, H\. Peng, J\. Zhao, Z\. Wang, W\. Wei, L\. Han, Z\. Zhang,et al\.\(2026\)Reasoning over precedents alongside statutes: case\-augmented deliberative alignment for llm safety\.arXiv preprint arXiv:2601\.08000\.Cited by:[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px2.p1.1)\.
- C\. Jin, Y\. Zhou, Q\. Zhang, H\. Peng, D\. Zhang, Z\. Dong, M\. Pavone, L\. Han, Z\. Hong, T\. Che,et al\.\(2025c\)Your reward function for rl is your best prm for search: unifying rl and search\-based tts\.arXiv preprint arXiv:2508\.14313\.Cited by:[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- T\. Kojima, S\. S\. Gu, M\. Reid, Y\. Matsuo, and Y\. Iwasawa \(2022\)Large language models are zero\-shot reasoners\.arXiv preprint arXiv:2205\.11916\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p1.1),[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- math\-ai \(2025\)AMC 2023 dataset\.Hugging Face\.Note:[https://huggingface\.co/datasets/math\-ai/amc23](https://huggingface.co/datasets/math-ai/amc23)Accessed: 202\-05\-24Cited by:[§4\.1](https://arxiv.org/html/2606.00726#S4.SS1.p2.1)\.
- N\. Miao, Y\. W\. Teh, and T\. Rainforth \(2023\)Selfcheck: using llms to zero\-shot check their own step\-by\-step reasoning\.arXiv preprint arXiv:2308\.00436\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. Bowman \(2023\)Gpqa: a graduate\-level google\-proof q&a benchmark\.arXiv preprint arXiv:2311\.12022\.Cited by:[§4\.1](https://arxiv.org/html/2606.00726#S4.SS1.p2.1)\.
- J\. Sheng, L\. Lyu, J\. Jin, T\. Xia, A\. Gu, J\. Zou, and P\. Lu \(2026\)Solving inequality proofs with large language models\.Advances in Neural Information Processing Systems38\.Cited by:[§4\.1](https://arxiv.org/html/2606.00726#S4.SS1.p2.1)\.
- A\. Templeton, T\. Conerly, J\. Marcus, J\. Lindsey, T\. Bricken, B\. Chen, A\. Pearce, C\. Citro, E\. Ameisen, A\. Jones, H\. Cunningham, N\. L\. Turner, C\. McDougall, M\. MacDiarmid, C\. D\. Freeman, T\. R\. Sumers, E\. Rees, J\. Batson, A\. Jermyn, S\. Carter, C\. Olah, and T\. Henighan \(2024\)Scaling monosemanticity: extracting interpretable features from claude 3 sonnet\.Note:Transformer Circuits ThreadOnlineCited by:[§1](https://arxiv.org/html/2606.00726#S1.p5.1),[§3\.1](https://arxiv.org/html/2606.00726#S3.SS1.SSS0.Px1.p1.6)\.
- A\. M\. Turner, L\. Thiergart, D\. Udell, G\. Leech, U\. Mini, and M\. MacDiarmid \(2023\)Steering language models with activation engineering\.arXiv preprint arXiv:2308\.10248\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px2.p1.1)\.
- G\. Tyen, H\. Mansoor, V\. Cărbune, Y\. P\. Chen, and T\. Mak \(2024\)LLMs cannot find reasoning errors, but can correct them given the error location\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 13894–13908\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p1.1)\.
- C\. Venhoff, I\. Arcuschin, P\. Torr, A\. Conmy, and N\. Nanda \(2025a\)Base models know how to reason, thinking models learn when\.Note:Under review as a conference paper at ICLR 2026; arXiv:2510\.07364Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p4.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px3.p1.1),[§3\.1](https://arxiv.org/html/2606.00726#S3.SS1.SSS0.Px1.p1.6),[§4\.1](https://arxiv.org/html/2606.00726#S4.SS1.p1.1),[§4\.2](https://arxiv.org/html/2606.00726#S4.SS2.SSS0.Px3.p1.1)\.
- C\. Venhoff, I\. Arcuschin, P\. Torr, A\. Conmy, and N\. Nanda \(2025b\)Understanding reasoning in thinking language models via steering vectors\.InWorkshop on Reasoning and Planning for Large Language Models,Cited by:[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px3.p1.1)\.
- L\. Wang, W\. Xu, Y\. Lan, Z\. Hu, Y\. Lan, R\. K\. Lee, and E\. Lim \(2023\)Plan\-and\-solve prompting: improving zero\-shot chain\-of\-thought reasoning by large language models\.InProceedings of the 61st annual meeting of the association for computational linguistics \(volume 1: long papers\),pp\. 2609–2634\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Wang, H\. Chen, Y\. Tian, C\. Geng, D\. Liang, and X\. Chen \(2026\)Beyond dense states: elevating sparse transcoders to active operators for latent reasoning\.arXiv preprint arXiv:2602\.01695\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p4.1),[§3\.1](https://arxiv.org/html/2606.00726#S3.SS1.SSS0.Px1.p1.6)\.
- J\. Ward, C\. Lin, C\. Venhoff, and N\. Nanda \(2025\)Reasoning\-finetuning repurposes latent representations in base models\.arXiv preprint arXiv:2507\.12638\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p4.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px3.p1.1),[§3\.1](https://arxiv.org/html/2606.00726#S3.SS1.SSS0.Px1.p1.6)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. Chi, Q\. Le, and D\. Zhou \(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p1.1),[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Weng, M\. Zhu, F\. Xia, B\. Li, S\. He, S\. Liu, B\. Sun, K\. Liu, and J\. Zhao \(2023\)Large language models are better reasoners with self\-verification\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 2550–2575\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- W\. Ye, X\. Yuan, Y\. Bin, H\. Jin, L\. Peng, P\. Zeng, and H\. T\. Shen \(2026\)RISER: orchestrating latent reasoning skills for adaptive activation steering\.arXiv preprint arXiv:2601\.09269\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Zhang, X\. He, W\. Yan, A\. Shen, C\. Zhao, and X\. Wang \(2026a\)Soft thinking: unlocking the reasoning potential of llms in continuous concept space\.Advances in Neural Information Processing Systems38,pp\. 168990–169012\.Cited by:[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Zhang, K\. Song, X\. Wang, Y\. Hu, W\. Yan, C\. Zhao, H\. P\. Zou, H\. Deng, S\. R\. Indurthi, S\. Liu,et al\.\(2026b\)CM2: reinforcement learning with checklist rewards for multi\-turn and multi\-step agentic tool use\.arXiv preprint arXiv:2602\.12268\.Cited by:[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Zhang, X\. Wu, Z\. Zhou, Q\. Wu, Y\. Zhang, P\. Ponnusamy, H\. Subbaraj, J\. Wang, S\. L\. Song, and B\. Athiwaratkun \(2025\)Understanding and steering the cognitive behaviors of reasoning models at test\-time\.arXiv preprint arXiv:2512\.24574\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px2.p1.1)\.
- H\. S\. Zheng, S\. Mishra, X\. Chen, H\. Cheng, E\. H\. Chi, Q\. V\. Le, and D\. Zhou \(2024\)Take a step back: evoking reasoning via abstraction in large language models\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 20279–20316\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- D\. Zhou, N\. Schärli, L\. Hou, J\. Wei, N\. Scales, X\. Wang, D\. Schuurmans, C\. Cui, O\. Bousquet, Q\. Le,et al\.\(2022\)Least\-to\-most prompting enables complex reasoning in large language models\.arXiv preprint arXiv:2205\.10625\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Zou, Z\. Wang, N\. Carlini, M\. Nasr, J\. Z\. Kolter, and M\. Fredrikson \(2023\)Representation engineering: a top\-down approach to ai transparency\.arXiv preprint arXiv:2310\.01405\.Cited by:[§1](https://arxiv.org/html/2606.00726#S1.p2.1),[§2](https://arxiv.org/html/2606.00726#S2.SS0.SSS0.Px2.p1.1)\.

## Appendix ATraining and Experimental Details

#### Models and intervention layer\.

We useOpen\-Reasoner\-7Bas the primary model andOpen\-Reasoner\-1\.5Bas a smaller\-model comparison within the same family\. The base model parameters are never updated\.

#### Zero\-shot decoding and scoring\.

All benchmarks are evaluated in a zero\-shot setting with greedy decoding and batch size11\. During generation, the model receives only the original problem statement, without few\-shot demonstrations or task\-specific initial reasoning prompts\. All final answers are parsed from full output and parsed answers are then scored with the benchmark\-specific evaluator: math\-style benchmarks use rule\-based or math\-verification scoring where applicable, while multiple\-choice tasks use extracted option letters against the gold answer\.

#### Prompt baselines\.

The CoT baseline uses the same problem statement with a fixed instruction prefix:

CoT PromptLet’s solve this problem step by step, problem:<problem\>Figure 7:Zero\-shot chain\-of\-thought prompt used as a baseline\.For few\-shot prompting, we use fixed five\-example prompts before the test problem\. Math\-style datasets use elementary algebra, LCM, geometry, counting, and summation examples\.GPQA\-Diamonduses five multiple\-choice science examples andIneqMathuses five inequality examples covering AM\-GM, triangle inequality, and algebraic nonnegativity\. The full prompt templates are shown below\.

Five\-Shot Math PromptHere are 5 example problems and their solutions\.Example 1\.Problem: If3​x\+2=113x\+2=11, what is the value ofx2x^\{2\}? Solution: From3​x\+2=113x\+2=11we get3​x=93x=9, sox=3x=3\. Thereforex2=9x^\{2\}=9\. Final answer:9\\boxed\{9\}Example 2\.Problem: What is the smallest positive integer divisible by both 6 and 8? Solution: Prime factorize:6=2⋅36=2\\cdot 3and8=238=2^\{3\}\. Solcm​\(6,8\)=23⋅3=24\\mathrm\{lcm\}\(6,8\)=2^\{3\}\\cdot 3=24\. Final answer:24\\boxed\{24\}Example 3\.Problem: A right triangle has legs of length 5 and 12\. What is the length of the hypotenuse? Solution: By the Pythagorean theorem,c=52\+122=169=13c=\\sqrt\{5^\{2\}\+12^\{2\}\}=\\sqrt\{169\}=13\. Final answer:13\\boxed\{13\}Example 4\.Problem: How many ways are there to choose 3 books from a shelf of 7 distinct books? Solution: This is\(73\)=7⋅6⋅53⋅2⋅1=35\\binom\{7\}\{3\}=\\frac\{7\\cdot 6\\cdot 5\}\{3\\cdot 2\\cdot 1\}=35\. Final answer:35\\boxed\{35\}Example 5\.Problem: What is the sum of the first 100 positive integers? Solution: Using∑k=1nk=n​\(n\+1\)2\\sum\_\{k=1\}^\{n\}k=\\frac\{n\(n\+1\)\}\{2\}withn=100n=100, the sum is100⋅1012=5050\\frac\{100\\cdot 101\}\{2\}=5050\. Final answer:5050\\boxed\{5050\}Now solve this problem\.Problem:<problem\>Figure 8:Five\-shot prompt template for math\-style benchmarks\.Five\-Shot GPQA PromptHere are 5 example multiple\-choice problems and their solutions\.Example 1\.Problem: A ball is dropped from a height of 20 m\. How long does it take to hit the ground? Ignore air resistance,g=10​m/s2g=10\\,\\mathrm\{m/s^\{2\}\}\. Choices: \(A\) 1 s, \(B\) 2 s, \(C\) 4 s, \(D\) 5 s\. Solution: Usingh=12​g​t2h=\\frac\{1\}\{2\}gt^\{2\}, solve20=5​t220=5t^\{2\}, sot=2t=2\. Final answer:B\\boxed\{B\}Example 2\.Problem: Which of the following has the highest electronegativity? Choices: \(A\) Carbon, \(B\) Nitrogen, \(C\) Oxygen, \(D\) Fluorine\. Solution: Electronegativity increases across a period and decreases down a group\. Fluorine has the highest value\. Final answer:D\\boxed\{D\}Example 3\.Problem: Which organelle is responsible for ATP production via oxidative phosphorylation in eukaryotic cells? Choices: \(A\) Nucleus, \(B\) Ribosome, \(C\) Mitochondrion, \(D\) Endoplasmic reticulum\. Solution: The mitochondrion houses the electron transport chain and ATP synthase\. Final answer:C\\boxed\{C\}Example 4\.Problem: A 2 kg object moves at 3 m/s\. What is its kinetic energy? Choices: \(A\) 3 J, \(B\) 6 J, \(C\) 9 J, \(D\) 12 J\. Solution:K​E=12​m​v2=12​\(2\)​\(3\)2=9​JKE=\\frac\{1\}\{2\}mv^\{2\}=\\frac\{1\}\{2\}\(2\)\(3\)^\{2\}=9\\,\\mathrm\{J\}\. Final answer:C\\boxed\{C\}Example 5\.Problem: What is the pH of a10−310^\{\-3\}M HCl solution? Choices: \(A\) 1, \(B\) 3, \(C\) 7, \(D\) 11\. Solution: HCl fully dissociates, so\[H\+\]=10−3\[\\mathrm\{H\}^\{\+\}\]=10^\{\-3\}M andpH=3\\mathrm\{pH\}=3\. Final answer:B\\boxed\{B\}Now solve this problem\.Problem:<problem\>Figure 9:Five\-shot prompt template for GPQA\-Diamond\.Five\-Shot IneqMath PromptHere are 5 example inequality problems and their solutions\.Example 1\.Problem: Find the largest constantCCsuch thatx2\+y2≥C​x​yx^\{2\}\+y^\{2\}\\geq Cxyfor all realx,yx,y\. Solution:x2\+y2−2​x​y=\(x−y\)2≥0x^\{2\}\+y^\{2\}\-2xy=\(x\-y\)^\{2\}\\geq 0, soC=2C=2is tight\. Final answer:2\\boxed\{2\}Example 2\.Problem: For all positive realsa,ba,b, determine the relation betweena\+ba\+band2​a​b2\\sqrt\{ab\}\. Solution: By AM\-GM,a\+b2≥a​b\\frac\{a\+b\}\{2\}\\geq\\sqrt\{ab\}, soa\+b≥2​a​ba\+b\\geq 2\\sqrt\{ab\}\. Final answer:≥\\boxed\{\\geq\}Example 3\.Problem: Find the smallest constantCCsuch that\|x\+y\|≤C​\(\|x\|\+\|y\|\)\|x\+y\|\\leq C\(\|x\|\+\|y\|\)for all realx,yx,y\. Solution: By the triangle inequality,C=1C=1works and is tight\. Final answer:1\\boxed\{1\}Example 4\.Problem: For positive realsa,b,ca,b,c, determine the relation betweena2\+b2\+c2a^\{2\}\+b^\{2\}\+c^\{2\}anda​b\+b​c\+c​aab\+bc\+ca\. Solution:a2\+b2\+c2−a​b−b​c−c​a=12​\(\(a−b\)2\+\(b−c\)2\+\(c−a\)2\)≥0a^\{2\}\+b^\{2\}\+c^\{2\}\-ab\-bc\-ca=\\frac\{1\}\{2\}\(\(a\-b\)^\{2\}\+\(b\-c\)^\{2\}\+\(c\-a\)^\{2\}\)\\geq 0\. Final answer:≥\\boxed\{\\geq\}Example 5\.Problem: Find the largest constantCCsuch that\(a\+b\)2≥C​a​b\(a\+b\)^\{2\}\\geq Cabfor all positive realsa,ba,b\. Solution: By AM\-GM,a\+b≥2​a​ba\+b\\geq 2\\sqrt\{ab\}, so\(a\+b\)2≥4​a​b\(a\+b\)^\{2\}\\geq 4ab, with equality ata=ba=b\. Final answer:4\\boxed\{4\}Now solve this problem\.Problem:<problem\>Figure 10:Five\-shot prompt template for IneqMath\.
#### Reward model training\.

The reward model is trained on sparse latent reasoning sequences obtained by encoding the model’s hidden activations through the pretrained SAE \(Section[3\.1](https://arxiv.org/html/2606.00726#S3.SS1)\)\. For each generation, we keep only the reasoning segment starting from the model’s “think” marker, discarding the prompt prefix, and pair the sequence with a binary final\-answer correctness label that is repeated over all positions for token\-level supervision\. Architecturally, the reward model is a lightweight Transformer encoder operating directly ondzd\_\{z\}\-dimensional SAE latents: it applies a LayerNorm on the raw SAE input \(essential, since unnormalized SAE activations are often near\-zero and easily dominated by positional encoding\), a linear projection to hidden sized=128d=128, sinusoidal positional encoding, two Transformer encoder blocks \(4 attention heads, feedforward width4​d4d, dropout0\.10\.1\), and an MLP head \(Linear–ReLU–Dropout–Linear–Sigmoid\) producing a per\-position probabilitypi,t∈\(0,1\)p\_\{i,t\}\\in\(0,1\)\. We optimize with AdamW \(learning rate5×10−45\\times 10^\{\-4\}, weight decay10−410^\{\-4\}\), binary cross\-entropy loss, gradient clipping at max\-norm1\.01\.0, and batch size11\(one variable\-length trace per step\)\. To counter class imbalance between correct and incorrect trajectories, we apply weighted random sampling, with each trace sampled with probability inversely proportional to its class frequency\. We train for3030epochs and select the checkpoint with the lowest training loss for inference\-time steering\. No behavior labels, GPT\-4o annotations, or predefined cognitive categories are used at any stage and the only supervision is final\-answer correctness\.

#### Steering variants and gate\.

Lrs\(basic\)applies reward\-guided latent steering without the reward and confidence gate\. FullLrsuses the gate to selectively intervene: steering is triggered when the reward score is below the reward threshold, or when the reward score is above threshold but the previous\-token decoding confidence, measured as the last\-token maximum probability, is below the confidence threshold\. In the experiments below, most conditions are instantiated as reward<0\.9<0\.9or reward≥0\.9\\geq 0\.9with last\-token maximum probability<0\.72<0\.72, except for configurations whose thresholds are listed separately in Table[5](https://arxiv.org/html/2606.00726#A1.T5)\.

DatasetModelKKα\\alphaRewardConfidenceDeviceAMC23Open\-Reasoner\-7B11\.4000\.90\.72RTX A4500AMC23Open\-Reasoner\-1\.5B40\.3000\.90\.72RTX A6000AIME24Open\-Reasoner\-7B20\.2950\.90\.72RTX A4500AIME24Open\-Reasoner\-1\.5B40\.9000\.90\.72RTX A4500AIME25Open\-Reasoner\-7B31\.3200\.90\.72RTX A4500AIME25Open\-Reasoner\-1\.5B20\.4000\.90\.72RTX A6000GPQA DiamondOpen\-Reasoner\-7B41\.1500\.90\.72RTX A5000GPQA DiamondOpen\-Reasoner\-1\.5B31\.0000\.90\.72RTX A5000IneqMathOpen\-Reasoner\-7B20\.7000\.90\.72RTX A5000IneqMathOpen\-Reasoner\-1\.5B21\.1000\.90\.72RTX A6000MATH\-500Open\-Reasoner\-7B11\.4000\.80\.69RTX A4500MATH\-500Open\-Reasoner\-1\.5B10\.1000\.90\.72RTX A6000Table 5:Experimental configurations for different datasets and models\.KKdenotes the number of latent optimization steps, andα\\alphadenotes the step size\. The reward and confidence columns report the corresponding gate thresholds\. The device column reports the GPU used for each dataset–model configuration\.

## Appendix BBehavior Annotation Protocol

We use GPT\-4o only for post\-hoc diagnostic annotation of generated reasoning traces\. The same fixed prompt is applied to base andLrsoutputs\. These annotations are never used for reward\-model training, steering\-vector construction, or gate triggering\.

#### Annotation prompt\.

We apply the same fixed GPT\-4o prompt for both base andLrstraces, also we use the following behavior criteria:

GPT\-4o Behavior Annotation PromptInput\.You will receive two fields:problem, the original problem statement, andreasoning\_trace, the generated solution\.Task\.For each behavior below, decide whether it appears at least once anywhere in the reasoning trace\. Do not judge final\-answer correctness except when the trace explicitly verifies a result against the original problem\.Output\.Return only a JSON object with one key per behavior\. Each key must contain a Boolean fieldpresentand a shortevidencestring\. If a behavior is absent, setpresenttofalseand use an empty evidence string\.Schema\.\{StrategicPlanning: \{present: bool, evidence: str\}, StructuredDecomposition: \{present: bool, evidence: str\}, ConstraintGrounding: \{present: bool, evidence: str\}, CourseCorrection: \{present: bool, evidence: str\}, SolutionVerification: \{present: bool, evidence: str\}\}Figure 11:Prompt used for post\-hoc GPT\-4o behavior annotation\.- •Strategic Planning:mark true if the trace selects a method, theorem, or overall strategy before detailed computation and explains why it is appropriate\.
- •Structured Decomposition:mark true if the trace breaks the problem into cases, subproblems, lemmas, branches, or explicitly named intermediate goals\.
- •Constraint Grounding:mark true if the trace actively uses problem constraints to check domains, exclude invalid solutions, validate boundary conditions, or restrict the search space\.
- •Course Correction:mark true if the trace detects an error, contradiction, missing case, or uncertainty and then revises the computation, switches methods, or redirects the solution path\.
- •Solution Verification:mark true if the trace substitutes an intermediate result or final answer back into the original problem, or otherwise checks that the derived answer satisfies the required conditions\.

#### Occurrence and improvement\.

For each trace, a behavior has occurrence value11if GPT\-4o marks it as present at least once and0otherwise\. The occurrence rate is the average of this indicator over the matched trace set\. Behavior improvement is computed as theLrsoccurrence rate minus the Base occurrence rate\. Because the labels are post\-hoc diagnostics, they support process\-level interpretation rather than direct claims thatLrsexplicitly controls predefined behaviors\.

BehaviorTrace\-level criterionAssociated SAE dimensionsStrategic PlanningSelects a method or theorem and explains the rationale before detailed computation, e\.g\., choosing a Diophantine strategy before algebraic execution\.z8z\_\{8\}meta\-cognitive strategy;z1z\_\{1\}logical branching\.Structured DecompositionBreaks the problem into independent branches, such as case splits, named lemmas, or formally defined intermediate sub\-problems\.z1z\_\{1\}theorem invocation;z0z\_\{0\}structural decomposition\.Constraint GroundingUses problem constraints to guide or limit the solution process, such as checking domains, excluding invalid solutions, or verifying boundary conditions\.z7z\_\{7\}constraint processing;z6z\_\{6\}deductive conclusion\.Course CorrectionDetects a problem during reasoning and adjusts direction, such as switching methods after a contradiction or revising a flawed assumption\.z8z\_\{8\}meta\-cognitive re\-planning;z7z\_\{7\}violation detection;z1z\_\{1\}logical branching\.Solution VerificationSubstitutes the final answer back into the original problem or checks whether the derived result satisfies the required conditions\.z7z\_\{7\}constraint processing;z6z\_\{6\}answer consolidation\.Table 6:Cognitive behavior categories used for matched base\-vs\.\-Lrstrace analysis\. Associated SAE dimensions are used only as interpretability priors for analyzingΔ​z\\Delta z, not as deterministic behavior mappings\.

## Appendix CCase Diagnostic Notes

This appendix clarifies how to read the diagnostics used in the qualitative cases\. The case\-study boxes report steering\-event counts, early trigger positions, and the largest corrected SAE dimensions\. These diagnostics are derived from per\-question SAE steering traces and support a cautious process\-level reading: they show where latent updates were applied and which sparse dimensions changed most, but they do not provide token\-level reward values or prove a causal mechanism\. We therefore avoid numeric reward claims and interpret dimension names only as post\-hoc priors from Table[3](https://arxiv.org/html/2606.00726#S4.T3)\.

## Appendix DQualitative Case Studies with Reward\-Signal Diagnostics

We provide representative examples whereLrschanges an initially incorrect single generation into a correct answer\. Each case highlights the critical divergence between the base andLrs\-steered trajectories\. These examples cover different failure modes, including combinatorial overcounting, missing cases, incorrect constraint handling, scientific concept confusion, and algebraic simplification errors\. The highlighted failure modes are used only for post\-hoc analysis\.Lrsitself does not rely on behavior labels or predefined steering directions\. Excerpts are shortened for readability while preserving the original reasoning error, the corrected reasoning step, and the final answer\. SAE dimension interpretations are post\-hoc priors and should not be read as deterministic labels, especially for non\-mathematical tasks such as GPQA\.

![Refer to caption](https://arxiv.org/html/2606.00726v1/x6.png)Figure 12:Qualitative example whereLrssteers an incorrect reasoning trace toward verification and a correct answer\.Case Study 1: AMC23 Q38 — Combinatorics \(Subset Counting\)QuestionDetermine the number of nonempty subsetsBBof\{0,1,2,…,12\}\\\{0,1,2,\\ldots,12\\\}such that the number of elements inBBis equal to the least element ofBB\.Base OutputThe base model letskkdenote the least element ofBBand correctly requires\|B\|=k\|B\|=kwithk≥1k\\geq 1\. It then choosesk−1k\-1additional elements from\{k,k\+1,…,12\}\\\{k,k\+1,\\ldots,12\\\}, which contains13−k13\-kelements, yielding\(13−kk−1\)\{\\color\[rgb\]\{0\.70703125,0\.1953125,0\.1953125\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.70703125,0\.1953125,0\.1953125\}\\binom\{13\-k\}\{k\-1\}\}\. Summing fork=1k=1to77:1\+11\+45\+84\+70\+21\+1=2331\+11\+45\+84\+70\+21\+1=\{\\color\[rgb\]\{0\.70703125,0\.1953125,0\.1953125\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.70703125,0\.1953125,0\.1953125\}233\}\. The error is thatkkitself is already required to be inBB, so the remainingk−1k\-1elements must be chosen from\{k\+1,…,12\}\\\{k\{\+\}1,\\ldots,12\\\}, which has12−k12\-kelements, not from\{k,…,12\}\\\{k,\\ldots,12\\\}\.LrsSteered OutputTheLrs\-steered trace uses the correct selection pool:\(12−kk−1\)\{\\color\[rgb\]\{0\.1171875,0\.46875,0\.296875\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.1171875,0\.46875,0\.296875\}\\binom\{12\-k\}\{k\-1\}\}\. “The subsetBBmust includekkandk−1k\{\-\}1other elements, all of which must be greater thankk\. The elements ofBBcan be chosen from the set\{k\+1,k\+2,…,12\}\\\{k\{\+\}1,k\{\+\}2,\\ldots,12\\\}\. The number of elements in this set is12−k12\-k\.” Summing fork=1k=1to66:1\+10\+36\+56\+35\+6=1441\+10\+36\+56\+35\+6=\{\\color\[rgb\]\{0\.1171875,0\.46875,0\.296875\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.1171875,0\.46875,0\.296875\}144\}\.Key DifferenceBaseuses\(13−kk−1\)\\binom\{13\-k\}\{k\-1\}, which includeskkin the selection pool and causes overcounting\.Lrsuses\(12−kk−1\)\\binom\{12\-k\}\{k\-1\}, correctly excludingkk\.Steering Info184 steering events and the first recorded steering events occur at generation steps 24, 59, and 69\. Top corrected dimensions:z2z\_\{2\}↑\\uparrow\(algebraic execution\),z4z\_\{4\}↑\\uparrow\(variable extraction\),z9z\_\{9\}↓\\downarrow\(complexity evaluation\)\.Reward\-Signal DiagnosticThe base reasoning trace begins to fail when it constructs the counting pool\{k,…,12\}\\\{k,\\ldots,12\\\}and thereby counts the required minimum elementkktwice\. The reward and confidence gate is associated with early interventions before the final summation, suggesting that the local latent state around the combinatorial setup was treated as fragile\. In the steered trace, the solution grounds the constraint thatkkis already included inBBand changes the pool to\{k\+1,…,12\}\\\{k\+1,\\ldots,12\\\}\. This change is consistent with*Constraint Grounding*and*Course Correction*, rather than a behavior label explicitly supplied toLrs\.Latent\-Change InterpretationThe largest average latent changes up\-regulatez2z\_\{2\}andz4z\_\{4\}, which Table[3](https://arxiv.org/html/2606.00726#S4.T3)associates with algebraic execution and variable initialization, while down\-regulatingz9z\_\{9\}, associated with complexity evaluation\. This pattern is consistent with the steered trace correcting the counting setup and avoiding an overcomplicated selection pool\. These dimension names are post\-hoc interpretability priors, not deterministic causal labels\.Figure 13:AMC23 Q38:Lrscorrects an overcount caused by includingkkin the selection pool\.Case Study 2: AIME25 Q2 — Counting \(Multinomial Partitions\)QuestionNine players each choose one of three flavors: chocolate, vanilla, or strawberry\. Each flavor is chosen by at least one player, and the number choosing chocolate is strictly greater than the number choosing vanilla, which is strictly greater than the number choosing strawberry\. Find the number of valid assignments modulo10001000\.Base OutputSettingc\>v\>s≥1c\>v\>s\\geq 1withc\+v\+s=9c\+v\+s=9, the base model enumerates:•s=1s=1: finds\(5,3,1\)\(5,3,1\)as valid, butrejects\(6,2,1\)\(6,2,1\)by incorrectly claimingv\>sv\>sfails, although2\>12\>1holds\.•s=2s=2: tests\(4,3,2\)\(4,3,2\)butrejects it, writing “not valid sincec\>vc\>v”, which misapplies the constraint\.Only the single partition\(5,3,1\)\(5,3,1\)is retained:\(95,3,1\)=504\\binom\{9\}\{5,3,1\}=504, so the answer is504\{\\color\[rgb\]\{0\.70703125,0\.1953125,0\.1953125\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.70703125,0\.1953125,0\.1953125\}504\}\.LrsSteered OutputTheLrs\-steered reasoning trace correctly enumerates all three valid partitions:•\(6,2,1\)\(6,2,1\):valid since6\>2\>16\>2\>1,\(96,2,1\)=252\\binom\{9\}\{6,2,1\}=252\.•\(5,3,1\)\(5,3,1\): valid,\(95,3,1\)=504\\binom\{9\}\{5,3,1\}=504\.•\(4,3,2\)\(4,3,2\):valid since4\>3\>24\>3\>2,\(94,3,2\)=1260\\binom\{9\}\{4,3,2\}=1260\.Total:252\+504\+1260=2016252\+504\+1260=2016\. Remainder modulo10001000:16\{\\color\[rgb\]\{0\.1171875,0\.46875,0\.296875\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.1171875,0\.46875,0\.296875\}16\}\.Key DifferenceBasemisses two of three valid partitions due to incorrect constraint checks\.Lrscorrectly enumerates all three\.Steering Info134 steering events and the first recorded steering events occur at generation steps 14, 15, and 18\. Top corrected dimensions:z2z\_\{2\}↑\\uparrow\(algebraic execution\),z3z\_\{3\}↑\\uparrow\(symbolic reasoning\),z0z\_\{0\}↓\\downarrow\(geometric modeling\)\.Reward\-Signal DiagnosticThe base reasoning trace starts to fail during case enumeration: it rejects valid partitions such as\(6,2,1\)\(6,2,1\)and\(4,3,2\)\(4,3,2\)despite the strict ordering constraints being satisfied\. The early reward and confidence gate activations suggest that the latent state during the enumeration phase was flagged as locally fragile\. The steered trace revisits the missing cases and evaluates all valid multinomial partitions before applying the modulo operation\. This pattern is associated with*Structured Decomposition*,*Constraint Grounding*, and*Course Correction*\.Latent\-Change InterpretationThe dominant changes up\-regulatez2z\_\{2\}andz3z\_\{3\}, associated with algebraic execution and symbolic formatting, while down\-regulatingz0z\_\{0\}, associated with structural modeling\. This is consistent with the steered trace moving from an incomplete structural enumeration to explicit validation of each multinomial case\. These dimension names are post\-hoc interpretability priors, not deterministic causal labels\.Figure 14:AIME25 Q2:Lrsrecovers the missed valid partitions and the correct remainder\.Case Study 3: AMC23 Q25 — Complex Number AlgebraQuestionDefine the operationu⊗v=a​c\+b​d​iu\\otimes v=ac\+bdi, whereu=a\+b​iu=a\+biandv=c\+d​iv=c\+di\. Find\|z\|2\|z\|^\{2\}given thatz⊗z=z2\+40z\\otimes z=z^\{2\}\+40\.Base OutputBoth models setz=x\+y​iz=x\+yiand correctly derive the real\-part constrainty2=40y^\{2\}=40\. For the imaginary part, they reachy2​i=2​x​yy^\{2\}i=2xy\. The base model argues: “The left sidey2​iy^\{2\}iis purely imaginary, and the right side2​x​y2xyis a real number\.For these to be equal … this implies2​x​y=02xy=0andy2=0y^\{2\}=0\.However, we already found thaty2=40y^\{2\}=40, which meansy≠0y\\neq 0\. Therefore, the only way … is ifx=0x=0\.” Withx=0x=0andy2=40y^\{2\}=40, it concludes\|z\|2=0\+40=40\|z\|^\{2\}=0\+40=\{\\color\[rgb\]\{0\.70703125,0\.1953125,0\.1953125\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.70703125,0\.1953125,0\.1953125\}40\}\.LrsSteered OutputTheLrs\-steered reasoning trace handles the same equation correctly: “This impliesy2=2​x​yy^\{2\}=2xy\.Ify≠0y\\neq 0, we can divide both sides byyy:y=2​xy=2x\.” Combined withy2=40y^\{2\}=40, this gives\(2​x\)2=40⇒x2=10\(2x\)^\{2\}=40\\Rightarrow x^\{2\}=10\. Therefore,\|z\|2=x2\+y2=10\+40=50\|z\|^\{2\}=x^\{2\}\+y^\{2\}=10\+40=\{\\color\[rgb\]\{0\.1171875,0\.46875,0\.296875\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.1171875,0\.46875,0\.296875\}50\}\.Key DifferenceBasetreatsy2​i=2​x​yy^\{2\}i=2xyas requiring both real and imaginary parts to vanish independently, forcingx=0x=0\.Lrscorrectly compares the imaginary coefficients to obtainy=2​xy=2x\.Steering Info247 steering events and the first recorded steering events occur at generation steps 9, 37, and 48\. Top corrected dimensions:z2z\_\{2\}↑\\uparrow\(algebraic execution\),z9z\_\{9\}↓\\downarrow\(complexity evaluation\),z4z\_\{4\}↑\\uparrow\(variable extraction\)\.Reward\-Signal DiagnosticThe base reasoning trace starts to fail when it separates the imaginary\-part equation incorrectly and forcesx=0x=0instead of comparing imaginary coefficients\. The reward and confidence gate is associated with interventions near the algebraic manipulation stage, suggesting low reliability for the local equation\-handling state\. In the steered trace, the solution compares imaginary coefficients directly, derivesy=2​xy=2x, and then completes the norm computation\. This shift is consistent with*Constraint Grounding*and*Course Correction*\.Latent\-Change InterpretationThe largest average changes up\-regulatez2z\_\{2\}andz4z\_\{4\}, associated with algebraic execution and variable extraction, while down\-regulatingz9z\_\{9\}, associated with complexity evaluation\. This pattern is consistent with the steered trace replacing an invalid coefficient split with direct manipulation of the real and imaginary constraints\. These dimension names are post\-hoc interpretability priors, not deterministic causal labels\.Figure 15:AMC23 Q25:Lrsfixes the imaginary\-part constraint and obtains\|z\|2=50\|z\|^\{2\}=50\.Case Study 4: GPQA Diamond Q39 — Chemistry \(Separation Science\)QuestionA synthetic organic chemist tells a colleague: “My compounds are on top of each other\.” What is the second chemist most likely referring to?\(A\)The compounds have similar polarities\.\(B\)The compounds are bonding through non\-covalent interactions\.\(C\)The compounds have similar optical rotations\.\(D\)The compounds have similar boiling points\.Base OutputThe base model considers both chromatography and distillation but ultimately favors distillation: “The most likely reason for the compounds being ‘on top of each other’ … is that they havesimilar boiling points\. This would make it difficult to separate them using distillation\.” Answer:D\.LrsSteered OutputTheLrs\-steered trace identifies the chromatography context: “Similar polarities \(option A\)is a broad property that can affect the solubility and separation of compounds, making it a plausible explanation for the observed issue\. The phrase ‘on top of each other’ suggests that the compounds are not well\-separated or are difficult to distinguish from one another\.” Answer:A\.Key DifferenceBaseinterprets “on top of each other” through the lens of distillation and boiling points\.Lrsconnects the phrase to chromatographic separation and polarities\.Steering Info653 steering events and the first recorded steering events occur at generation steps 2, 3, and 6\. Top corrected dimensions:z3z\_\{3\}↑\\uparrow\(symbolic reasoning\),z2z\_\{2\}↑\\uparrow\(algebraic execution\),z0z\_\{0\}↓\\downarrow\(geometric modeling\)\.Reward\-Signal DiagnosticThe base reasoning trace begins to drift when it interprets “on top of each other” through a distillation frame and selects boiling points, rather than grounding the phrase in chromatographic separation\. The very early gate activations suggest that the reward and confidence signal treated the initial conceptual framing as unstable\. The steered trace redirects the interpretation toward compounds co\-eluting or poorly separating on a chromatographic medium, leading to similar polarities\. This change is associated with*Strategic Planning*at the conceptual\-framing level and*Constraint Grounding*in the domain\-specific clue\.Latent\-Change InterpretationThe largest average changes up\-regulatez3z\_\{3\}andz2z\_\{2\}and down\-regulatez0z\_\{0\}\. Although Table[3](https://arxiv.org/html/2606.00726#S4.T3)names these dimensions using math\-heavy max\-activating contexts, for this non\-mathematical GPQA case they should be read only as post\-hoc latent\-change summaries\. The observed pattern is consistent with an early reframing of the domain clue, not with a deterministic “algebraic” mechanism\.Figure 16:GPQA Diamond Q39:Lrsreframes the clue as chromatographic separation\.Case Study 5: IneqMath Q84 — Inequality Proof \(AM\-GM\)QuestionDetermine the correct relation forS=a​1\+b−c3\+b​1\+c−a3\+c​1\+a−b3S=a\\,\\sqrt\[3\]\{1\{\+\}b\{\-\}c\}\+b\\,\\sqrt\[3\]\{1\{\+\}c\{\-\}a\}\+c\\,\\sqrt\[3\]\{1\{\+\}a\{\-\}b\}versus11, givena\+b\+c=1a\+b\+c=1witha,b,c\>0a,b,c\>0\. Options include≤\\leq,==,≥\\geq, etc\.Base OutputTestinga=b=c=13a=b=c=\\frac\{1\}\{3\}givesS=1S=1\. The base model then attempts to rewrite the terms using the constraint:1\+b−c=a\+2​b,1\+c−a=b\+2​c,1\+a−b=2​a\+c\.\{\\color\[rgb\]\{0\.70703125,0\.1953125,0\.1953125\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.70703125,0\.1953125,0\.1953125\}1\+b\-c=a\+2b\},\\qquad 1\+c\-a=b\+2c,\\qquad 1\+a\-b=2a\+c\.This simplification is incorrect\. For example,1\+b−c1\+b\-cdoes not simplify toa\+2​ba\+2bundera\+b\+c=1a\+b\+c=1\. The model then attempts AM\-GM on the wrong expressions and ultimately concludes: “Given these inequalities, the only consistent relationship isS=1S=1\.” No valid option letter is extracted\.LrsSteered OutputTheLrs\-steered reasoning trace avoids the false simplification and reasons directly: “Letx=1\+b−cx=1\+b\-c,y=1\+c−ay=1\+c\-a,z=1\+a−bz=1\+a\-b\. Note thatx\+y\+z=3x\+y\+z=3\. By the AM\-GM inequality:1\+b−c3≤x\+y\+z3=1\{\\color\[rgb\]\{0\.1171875,0\.46875,0\.296875\}\\definecolor\[named\]\{pgfstrokecolor\}\{rgb\}\{0\.1171875,0\.46875,0\.296875\}\\sqrt\[3\]\{1\+b\-c\}\\leq\\frac\{x\+y\+z\}\{3\}=1\}\. Thereforea​1\+b−c3≤aa\\,\\sqrt\[3\]\{1\+b\-c\}\\leq a, and similarly for the other terms\. Summing:S≤a\+b\+c=1S\\leq a\+b\+c=1\.” Combined withS=1S=1at the symmetric point, the answer isA:≤\\leq\.Key DifferenceBasemakes an incorrect algebraic simplification and then conjectures equality without proof\.Lrsapplies AM\-GM directly to the original cube\-root terms and correctly establishesS≤1S\\leq 1\.Steering Info559 steering events and the first recorded steering events occur at generation steps 4, 5, and 6\. Top corrected dimensions:z4z\_\{4\}↑\\uparrow\(variable extraction\),z2z\_\{2\}↑\\uparrow\(algebraic execution\),z9z\_\{9\}↓\\downarrow\(complexity evaluation\)\.Reward\-Signal DiagnosticThe base reasoning trace starts to fail when it rewrites1\+b−c1\+b\-casa\+2​ba\+2b, creating an invalid algebraic premise for the later AM\-GM argument\. The early reward and confidence gate activations are associated with this fragile symbolic setup, suggesting that the latent reward model treated the local manipulation state as fragile\. The steered trace avoids the false simplification, defines auxiliary variables directly from the original cube\-root terms, and applies AM\-GM to the valid quantities\. This behavior is consistent with*Constraint Grounding*and*Course Correction*\.Latent\-Change InterpretationThe largest average changes up\-regulatez4z\_\{4\}andz2z\_\{2\}, associated with variable extraction and algebraic execution, while down\-regulatingz9z\_\{9\}, associated with complexity evaluation\. This pattern is consistent with the steered trace avoiding the false rewrite and operating directly on the original cube\-root terms\. These dimension names are post\-hoc interpretability priors, not deterministic causal labels\.Figure 17:IneqMath Q84:Lrsavoids a false simplification and applies AM\-GM correctly\.#### Cross\-case latent dimension patterns\.

Across all five cases,Lrsreward\-guided steering consistently up\-regulates concrete algebraic manipulation dimensions, especiallyz2z\_\{2\}for algebraic step execution andz4z\_\{4\}for variable initialization, while down\-regulating dimensions associated with premature conclusion formation or complexity evaluation, such asz6z\_\{6\}andz9z\_\{9\}\. The dimensionz2z\_\{2\}appears as a top\-corrected dimension in all five cases, suggesting that the latent reward signal primarily promotes step\-by\-step algebraic reasoning over abstract planning or premature answer selection\. This pattern holds across math competition tasks, graduate\-level science questions, and inequality\-proof settings\.