Steering Vectors as a Training Signal in LLM Post-Training

# Steering Vectors as a Training Signal in LLM Post-Training
Source: [https://arxiv.org/html/2610.40998](https://arxiv.org/html/2610.40998)
Maunil Vyas††thanks:This work was conducted independently by the author and was not undertaken as part of their role at Amazon\.Affiliation:AmazonHuaiyuan YaoAffiliation:Arizona State UniversityHua WeiAffiliation:Arizona State University

###### Abstract

Steering vectors \(SVs\) are residual\-stream directions that shift the behavior of LLMs when added to hidden states at inference time\. While prior work has mainly used SVs as post\-hoc controls, we ask whether the same directions can also guide parameter updates during post\-training\. We study this question across supervised learning, including SFT and distillation, and reinforcement learning with verifiable rewards \(RLVR\)\. Our framework considers two intervention regimes: a hidden\-state hook during teacher\-forced supervised training, and an advantage\-repair mechanism on wrong rollouts in GRPO\. For each regime, we analyze three design axes, which are the steering direction, its placement, and the operation used to combine it with training\. In supervised settings, SV injection improves over plain SFT and distillation, with gains scaling with the off\-policy gap and reaching up to a12%12\\%improvement when distilling from thinking\-mode teachers\. These gains hold across three random seeds\. In RL settings, applying a correctness\-contrast direction as advantage repair on wrong rollouts, including otherwise zero\-gradient all\-wrong groups, recovers a learning signal that transfers to held\-out math benchmarks, and acts as a composable auxiliary signal that adds to existing zero\-variance remedies such as entropy\-aware advantage shaping\. We further identify three failure boundaries\. In detail, code generation can break the direction, cross\-family layer transfer can break the placement, and inference\-time application on an SV\-trained checkpoint can break the intervention regime\. Together, these results establish steering vectors as a conditional but effective training signal for LLM post\-training\. Our code is available[here](https://github.com/tiejin98/sv_training)\.

## 1Introduction

A steering vector \(SV\) is a direction in an LLM’s residual stream that can shift the model’s behavior when added to the hidden state at inference time\[[1](https://arxiv.org/html/2610.40998#bib.bib2),[2](https://arxiv.org/html/2610.40998#bib.bib4),[3](https://arxiv.org/html/2610.40998#bib.bib5)\]\. Prior work has mainly used steering vectors as post\-hoc controls for frozen models: the model parameters remain unchanged, and the vector is injected only during generation\. This raises a natural question:*if a steering vector captures a reusable behavioral direction, can it also be used to guide parameter updates during post\-training?*

This question is important because post\-training often relies on fixed reference data or sparse reward signals\. In supervised fine\-tuning and distillation\[[4](https://arxiv.org/html/2610.40998#bib.bib18),[5](https://arxiv.org/html/2610.40998#bib.bib31)\], the model is optimized on reference trajectories rather than the trajectories induced by its own current policy, which creates a mismatch between training\-time and inference\-time behavior\[[6](https://arxiv.org/html/2610.40998#bib.bib22)\]\. In reinforcement learning with verifiable rewards, the reward signal may be sparse, especially when all sampled responses in a group are incorrect\. Steering vectors offer a representation\-level intervention: instead of changing only the final output distribution at inference time, they can modify the hidden states used during training and thereby influence the gradients used to update the model\. This turns steering from a temporary control mechanism into a possible training signal\.

In this paper, we study this possibility systematically\. We investigate how steering vectors can be integrated into both supervised post\-training \(SFT and knowledge distillation\) and reinforcement learning with verifiable rewards \(RLVR\)\[[7](https://arxiv.org/html/2610.40998#bib.bib37),[8](https://arxiv.org/html/2610.40998#bib.bib35)\]\. We organize the study around two intervention regimes that correspond to where the SV enters the optimization pipeline, namely a*hidden\-state hook*during supervised post\-training and an*advantage repair*on wrong rollouts during GRPO\. Within each regime we describe the design along three axes, namely the*direction*itself, its*placement*, and the*operation*that combines it with the existing computation\. Our findings reveal a structured, conditional picture\. On the direction choice, per\-prompt vectors are competitive with or outperform global means in supervised post\-training, and the gain scales with the size of the off\-policy gap, reaching up to a12%12\\%improvement when distilling from thinking\-mode teachers\. In the RL regime, the correctness\-contrast direction applied as advantage repair on wrong rollouts, including otherwise zero\-gradient all\-wrong groups, recovers a learning signal that transfers cleanly to held\-out math benchmarks\. Overall, our contributions are as follows:

∙\\bulletWe frame SV injection as a training signal across two intervention regimes \(a hidden\-state hook for supervised post\-training and an advantage repair on wrong GRPO rollouts\), and describe each regime along three axes, namely the*direction*itself, its*placement*, and the*operation*that combines it with the existing computation\. To our knowledge, this is the first systematic study of SVs as a persistent training signal rather than only an inference\-time control\.

∙\\bulletAcross multiple students, datasets, and teachers, we identify configurations that reliably help in both regimes\. In the supervised regime, we find that adding SVs during teacher\-forced training generally improves over Plain SFT and distillation\. In the RL regime, an advantage repair on wrong GRPO rollouts using a correctness\-contrast direction \(SV\-Rescue\) recovers a useful learning signal as a composable auxiliary signal that stacks with entropy\-aware advantage shaping\[[9](https://arxiv.org/html/2610.40998#bib.bib40)\]\.

∙\\bulletWe map the corresponding failure boundaries\. Code generation breaks the*direction*, where the reference direction conflicts with the test\-passing direction; cross\-family layer choice breaks the*placement*; and inference\-time application on an SV\-trained checkpoint breaks the*regime*, removing rather than amplifying the gain\.

## 2Related Work

##### Steering Vectors\.

Steering vectors are directions in the residual stream of an LLM that, when added or otherwise mixed into the hidden state, predictably shift high\-level model behavior without changing parameters\[[1](https://arxiv.org/html/2610.40998#bib.bib2),[10](https://arxiv.org/html/2610.40998#bib.bib3),[2](https://arxiv.org/html/2610.40998#bib.bib4),[3](https://arxiv.org/html/2610.40998#bib.bib5)\]\. Related formulations include function vectors\[[11](https://arxiv.org/html/2610.40998#bib.bib6)\]and weight\-space task arithmetic\[[12](https://arxiv.org/html/2610.40998#bib.bib7)\], and a recent line of work refines their reliability, scaling, and geometry\[[13](https://arxiv.org/html/2610.40998#bib.bib8),[14](https://arxiv.org/html/2610.40998#bib.bib9),[15](https://arxiv.org/html/2610.40998#bib.bib10),[16](https://arxiv.org/html/2610.40998#bib.bib11),[17](https://arxiv.org/html/2610.40998#bib.bib12),[18](https://arxiv.org/html/2610.40998#bib.bib13),[19](https://arxiv.org/html/2610.40998#bib.bib14),[20](https://arxiv.org/html/2610.40998#bib.bib15),[21](https://arxiv.org/html/2610.40998#bib.bib16),[22](https://arxiv.org/html/2610.40998#bib.bib17)\]\. All of these approaches use the steering vector as an inference\-time control signal on a frozen model, and none tests its influence during the training stage\. Our work fills this gap\.

##### Post\-training of LLMs\.

Modern LLM post\-training starts from supervised fine\-tuning \(SFT\) on reference data\[[4](https://arxiv.org/html/2610.40998#bib.bib18),[23](https://arxiv.org/html/2610.40998#bib.bib20),[24](https://arxiv.org/html/2610.40998#bib.bib21)\], often with parameter\-efficient adapters such as LoRA\[[25](https://arxiv.org/html/2610.40998#bib.bib23)\]\. SFT is also the main tool used to elicit chain\-of\-thought\-style reasoning before reinforcement learning\[[26](https://arxiv.org/html/2610.40998#bib.bib24),[27](https://arxiv.org/html/2610.40998#bib.bib25),[28](https://arxiv.org/html/2610.40998#bib.bib29),[29](https://arxiv.org/html/2610.40998#bib.bib30)\]\. Knowledge distillation, which trains a student to imitate a teacher’s output distribution\[[5](https://arxiv.org/html/2610.40998#bib.bib31),[30](https://arxiv.org/html/2610.40998#bib.bib32)\], can be viewed as a special case of SFT on teacher\-produced targets\. A well\-known limitation of teacher\-forced objectives is off\-policy gap\[[6](https://arxiv.org/html/2610.40998#bib.bib22)\]\. DFT\[[31](https://arxiv.org/html/2610.40998#bib.bib50)\]addresses this gap at the level of the objective by reweighting each token’s loss with its detached probability, whereas our SV acts on the hidden representation\. None of the previous SFT works considers using a steering vector during training\. Beyond supervised methods, modern post\-training pipelines increasingly adopt reinforcement learning with verifiable rewards \(RLVR\) to improve model capabilities\[[32](https://arxiv.org/html/2610.40998#bib.bib33),[33](https://arxiv.org/html/2610.40998#bib.bib34),[34](https://arxiv.org/html/2610.40998#bib.bib36),[7](https://arxiv.org/html/2610.40998#bib.bib37),[35](https://arxiv.org/html/2610.40998#bib.bib38)\]\. Prominent algorithms in this space include GRPO\[[8](https://arxiv.org/html/2610.40998#bib.bib35)\]as a widely used critic\-free variant\. However, a known weakness of GRPO is the zero variance on the all\-wrong group\. Several recent methods address these groups through dynamic sampling, entropy\-guided advantage shaping, or gold\-rollout injection\[[36](https://arxiv.org/html/2610.40998#bib.bib39),[9](https://arxiv.org/html/2610.40998#bib.bib40),[37](https://arxiv.org/html/2610.40998#bib.bib41)\]\. Our work fits into this family of rescue methods, but uniquely leverages SVs\. Because it only shifts the advantage of wrong rollouts, it is complementary to these methods and can be stacked on top of them\.

## 3Preliminaries

Letπθ\\pi\_\{\\theta\}be an autoregressive language model withLLtransformer blocks and hidden sizedd\. For a promptxxand a responsey=\(y1,…,yT\)y=\(y\_\{1\},\\dots,y\_\{T\}\), leth\(ℓ\)​\(x,y<t\)∈ℝdh^\{\(\\ell\)\}\(x,y\_\{<t\}\)\\in\\mathbb\{R\}^\{d\}denote the residual\-stream hidden state at the output of blockℓ∈\{1,…,L\}\\ell\\in\\\{1,\\dots,L\\\}when processing the token at positiontt\. We writeht\(ℓ\)h^\{\(\\ell\)\}\_\{t\}when the context is clear, andm∈\{0,1\}Tm\\in\\\{0,1\\\}^\{T\}for the binary mask selecting answer tokens\.

### 3\.1Steering vectors

A broad empirical observation in the interpretability literature is that many high\-level behaviors of LLMs, such as sentiment, truthfulness, refusal, and task identity, are encoded along approximately linear directions in the residual stream\[[1](https://arxiv.org/html/2610.40998#bib.bib2),[10](https://arxiv.org/html/2610.40998#bib.bib3),[2](https://arxiv.org/html/2610.40998#bib.bib4),[3](https://arxiv.org/html/2610.40998#bib.bib5)\]\. This motivates the goal of finding, at each layerℓ\\ell, a single directionv\(ℓ\)∈ℝdv^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{d\}such that shifting the hidden state alongv\(ℓ\)v^\{\(\\ell\)\}predictably shifts the corresponding behavior in the model’s output\.

Given two matched response populations𝒟\+\\mathcal\{D\}\_\{\+\}and𝒟−\\mathcal\{D\}\_\{\-\}that differ in the target behavior, the standard recipe\[[2](https://arxiv.org/html/2610.40998#bib.bib4),[3](https://arxiv.org/html/2610.40998#bib.bib5)\]setsv\(ℓ\)v^\{\(\\ell\)\}to the difference of the mean hidden states at layerℓ\\ellover answer tokens:

v\(ℓ\)=𝔼i∈𝒟\+​\[h¯\(ℓ\)​\(xi,yi\+\)\]−𝔼i∈𝒟−​\[h¯\(ℓ\)​\(xi,yi−\)\],v^\{\(\\ell\)\}\\;=\\;\\mathbb\{E\}\_\{i\\in\\mathcal\{D\}\_\{\+\}\}\\\!\\\!\\left\[\\bar\{h\}^\{\(\\ell\)\}\(x\_\{i\},y\_\{i\}^\{\+\}\)\\right\]\\;\-\\;\\mathbb\{E\}\_\{i\\in\\mathcal\{D\}\_\{\-\}\}\\\!\\\!\\left\[\\bar\{h\}^\{\(\\ell\)\}\(x\_\{i\},y\_\{i\}^\{\-\}\)\\right\],\(1\)whereh¯\(ℓ\)​\(x,y\)\\bar\{h\}^\{\(\\ell\)\}\(x,y\)denotes the mean hidden state at layerℓ\\ellover the answer tokens ofyy\. Appendix[A](https://arxiv.org/html/2610.40998#A1)gives the explicit pooling formula and the intuition\.

##### Injection\.

There are multiple primitives for applyingv\(ℓ\)v^\{\(\\ell\)\}to the hidden state at a chosen layerℓ⋆\\ell^\{\\star\}\. We introduce two here: \(1\)Additive injection\[[1](https://arxiv.org/html/2610.40998#bib.bib2),[2](https://arxiv.org/html/2610.40998#bib.bib4)\]takes:

h~t\(ℓ⋆\)=ht\(ℓ⋆\)\+α​v\(ℓ⋆\),α∈ℝ,\\tilde\{h\}^\{\(\\ell^\{\\star\}\)\}\_\{t\}\\;=\\;h^\{\(\\ell^\{\\star\}\)\}\_\{t\}\\;\+\\;\\alpha\\,v^\{\(\\ell^\{\\star\}\)\},\\qquad\\alpha\\in\\mathbb\{R\},\(2\)but changes both the direction and the magnitude of the hidden state\. \(2\)Spherical linear interpolation \(Slerp\)\[[20](https://arxiv.org/html/2610.40998#bib.bib15),[21](https://arxiv.org/html/2610.40998#bib.bib16),[22](https://arxiv.org/html/2610.40998#bib.bib17)\]avoids changing magnitude by rotating the hidden state on the unit sphere toward a target direction along the shortest great\-circle arc:

Slerp⁡\(u,w,α\)=sin⁡\(\(1−α\)​ω\)sin⁡ω​u\+sin⁡\(α​ω\)sin⁡ω​w,ω=arccos⁡\(u⊤​w\)\.\\mathrm\{Slerp\}\(u,w;\\alpha\)\\;=\\;\\frac\{\\sin\\\!\\big\(\(1\-\\alpha\)\\omega\\big\)\}\{\\sin\\omega\}\\,u\\;\+\\;\\frac\{\\sin\(\\alpha\\omega\)\}\{\\sin\\omega\}\\,w,\\qquad\\omega=\\arccos\(u^\{\\\!\\top\}w\)\.\(3\)More details can be found at Appendix[A](https://arxiv.org/html/2610.40998#A1)\.

### 3\.2Group Relative Policy Optimization \(GRPO\)

For reasoning tasks with a verifier \(e\.g\. final\-answer matching on GSM8K\[[27](https://arxiv.org/html/2610.40998#bib.bib25)\]\), GRPO\[[8](https://arxiv.org/html/2610.40998#bib.bib35)\]is a PPO\-style\[[33](https://arxiv.org/html/2610.40998#bib.bib34)\]policy\-gradient algorithm that replaces the value network with a within\-prompt baseline\. For each promptxix\_\{i\}, it samplesnnrollouts\{yi,k\}k=1n∼πθold\(⋅∣xi\)\\\{y\_\{i,k\}\\\}\_\{k=1\}^\{n\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid x\_\{i\}\), receives verifier rewards\{ri,k\}k=1n\\\{r\_\{i,k\}\\\}\_\{k=1\}^\{n\}, and forms the group\-relative advantage

Ai,kGRPO=ri,k−μiσi\+ϵ,μi=1n​∑k′ri,k′,σi=std⁡\(\{ri,k′\}\)\.A^\{\\mathrm\{GRPO\}\}\_\{i,k\}\\;=\\;\\frac\{r\_\{i,k\}\-\\mu\_\{i\}\}\{\\sigma\_\{i\}\+\\epsilon\},\\qquad\\mu\_\{i\}=\\tfrac\{1\}\{n\}\\\!\\sum\_\{k^\{\\prime\}\}r\_\{i,k^\{\\prime\}\},\\;\\;\\sigma\_\{i\}=\\mathrm\{std\}\(\\\{r\_\{i,k^\{\\prime\}\}\\\}\)\.\(4\)The policy is then updated with a clipped PPO surrogate and a KL penalty to a reference policy,

ℒGRPO\(θ\)=−𝔼i,k,t\[min\(ρi,k,t\(θ\)Ai,kGRPO,clip\(ρi,k,t\(θ\),−ε,\+ε\)Ai,kGRPO\)\]\+βDKL\(πθ∥πref\),\\mathcal\{L\}\_\{\\mathrm\{GRPO\}\}\(\\theta\)\\;=\\;\-\\,\\mathbb\{E\}\_\{i,k,t\}\\\!\\left\[\\min\\\!\\big\(\\rho\_\{i,k,t\}\(\\theta\)\\,A^\{\\mathrm\{GRPO\}\}\_\{i,k\},\\;\\mathrm\{clip\}\(\\rho\_\{i,k,t\}\(\\theta\),1\\\!\-\\\!\\varepsilon,1\\\!\+\\\!\\varepsilon\)\\,A^\{\\mathrm\{GRPO\}\}\_\{i,k\}\\big\)\\right\]\+\\beta\\,D\_\{\\mathrm\{KL\}\}\\\!\\left\(\\pi\_\{\\theta\}\\\|\\pi\_\{\\mathrm\{ref\}\}\\right\),\(5\)whereρi,k,t​\(θ\)=πθ​\(yi,k,t∣xi,yi,k,<t\)/πθold​\(yi,k,t∣xi,yi,k,<t\)\\rho\_\{i,k,t\}\(\\theta\)=\\pi\_\{\\theta\}\(y\_\{i,k,t\}\\mid x\_\{i\},y\_\{i,k,<t\}\)/\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{i,k,t\}\\mid x\_\{i\},y\_\{i,k,<t\}\)andε\\varepsilonis the clip range\. A known weakness of Equation \([4](https://arxiv.org/html/2610.40998#S3.E4)\) is the zero\-variance case\. When every rollout for a prompt receives the same reward, most commonly when all rollouts are wrong, and then the prompt contributes no gradient\.

![Refer to caption](https://arxiv.org/html/2610.40998v1/framework.png)Figure 1:Overall framework\. Panel \(A\) precomputes the off\-policy displacement vector, which serves as the injection signal for the Supervised Regime in \(B\)\. In parallel, the Reinforcement Learning Regime in \(C\) applies the steering mechanism independently to rescue zero\-variance rollout groups\.

## 4Steering Vector Post\-training: Supervised Regime

### 4\.1Method

Since the source of the problem above is that teacher\-forced training is off\-policy, our goal is to construct a steering direction whose injection during teacher\-forced training makes the labels appear less off\-policy to the policy under update\. Following the framework introduced in Section[1](https://arxiv.org/html/2610.40998#S1), we describe the hidden\-state\-hook regime along three axes, namely the*direction*we use, the*placement*where we inject it, and the*operation*that combines it with the existing hidden state\.

Direction\.To close the off\-policy gap during teacher forcing, we need a direction that points from where the model’s own activations naturally sit to where the reference activations sit\. Reading this gap directly off the residual stream gives us such a direction by construction, and avoids any extra learning\. Concretely, before training we sample one greedy completionyiPredy\_\{i\}^\{\\mathrm\{Pred\}\}from the initial model on each training promptxix\_\{i\}, while the reference completionyiGTy\_\{i\}^\{\\mathrm\{GT\}\}is read from the dataset \(in SFT\) or produced by an external teacher \(in distillation\)\. Fixing a layerℓ⋆\\ell^\{\\star\}, we then run teacher forcing on both completions to read out the answer\-token\-pooled hidden states

hrefℓ⋆​\(xi\)=h¯ℓ⋆​\(xi,yiGT\),hpredℓ⋆​\(xi\)=h¯ℓ⋆​\(xi,yiPred\)\.h^\{\\ell^\{\\star\}\}\_\{\\mathrm\{ref\}\}\(x\_\{i\}\)\\;=\\;\\bar\{h\}^\{\\ell^\{\\star\}\}\\\!\\left\(x\_\{i\},\\,y\_\{i\}^\{\\mathrm\{GT\}\}\\right\),\\qquad h^\{\\ell^\{\\star\}\}\_\{\\mathrm\{pred\}\}\(x\_\{i\}\)\\;=\\;\\bar\{h\}^\{\\ell^\{\\star\}\}\\\!\\left\(x\_\{i\},\\,y\_\{i\}^\{\\mathrm\{Pred\}\}\\right\)\.\(6\)The prompt\-wise directionδi=hrefℓ⋆​\(xi\)−hpredℓ⋆​\(xi\)\\delta\_\{i\}=h^\{\\ell^\{\\star\}\}\_\{\\mathrm\{ref\}\}\(x\_\{i\}\)\-h^\{\\ell^\{\\star\}\}\_\{\\mathrm\{pred\}\}\(x\_\{i\}\)is, by definition, the off\-policy gap measured at layerℓ⋆\\ell^\{\\star\}on promptxix\_\{i\}\. We use eitherδi\\delta\_\{i\}directly to retain heterogeneity across prompts, or its dataset\-level mean:

vℓ⋆=𝔼i​\[hrefℓ⋆​\(xi\)−hpredℓ⋆​\(xi\)\],v^\{\\ell^\{\\star\}\}\\;=\\;\\mathbb\{E\}\_\{i\}\\\!\\left\[h^\{\\ell^\{\\star\}\}\_\{\\mathrm\{ref\}\}\(x\_\{i\}\)\\;\-\\;h^\{\\ell^\{\\star\}\}\_\{\\mathrm\{pred\}\}\(x\_\{i\}\)\\right\],\(7\)which captures the direction that drives the off\-policy gap on average\. Both are computed once before training and fixed throughout, so the direction is not a moving target chasing the policy as it updates\.

Placement\.We inject at the middle layerℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\+1, following prior steering\-vector work that has consistently found mid\-stack layers most effective for behavioral interventions\[[2](https://arxiv.org/html/2610.40998#bib.bib4),[1](https://arxiv.org/html/2610.40998#bib.bib2)\], and we use the same layer for both constructing and injecting the direction\. We further restrict injection to answer\-token positions because the loss is computed only on answer tokens, so the SV signal should be concentrated where it can flow into the gradient\. Injecting on prompt tokens would corrupt the shared prompt encoding without contributing to the loss\. During the forward pass of training, we therefore inject the residual stream at layerℓ⋆\\ell^\{\\star\}and answer tokens only:

h~tℓ⋆=\{Inject⁡\(htℓ⋆,vℓ⋆\)if​mi,t=1​\(answer token\),htℓ⋆otherwise,\\tilde\{h\}^\{\\ell^\{\\star\}\}\_\{t\}\\;=\\;\\begin\{cases\}\\mathrm\{Inject\}\\\!\\left\(h^\{\\ell^\{\\star\}\}\_\{t\},\\,v^\{\\ell^\{\\star\}\}\\right\)&\\text\{if \}m\_\{i,t\}=1\\text\{ \(answer token\)\},\\\\ h^\{\\ell^\{\\star\}\}\_\{t\}&\\text\{otherwise\},\\end\{cases\}\(8\)
Operation\.The remaining choice is how to combine the direction with the existing hidden state\. The most natural primitive is the additive operation of Equation \([2](https://arxiv.org/html/2610.40998#S3.E2)\),h~=h\+α​v\\tilde\{h\}=h\+\\alpha v, which gives a clean linear correction along the direction at smallα\\alpha\. The additive primitive does inflate the norm of the residual stream asα\\alphagrows, which can push the activation off the manifold the downstream layers were trained on, so we also test spherical linear interpolation \(Slerp, Equation \([3](https://arxiv.org/html/2610.40998#S3.E3)\)\) as a complementary primitive that rotates the hidden state toward the reference direction without changing its norm\. We instantiateInject⁡\(⋅,⋅\)\\mathrm\{Inject\}\(\\cdot,\\cdot\)with both\.*Mean\-SV*uses the additive primitive with the globalvℓ⋆v^\{\\ell^\{\\star\}\}from Equation \([7](https://arxiv.org/html/2610.40998#S4.E7)\), and*Per\-sample Slerp*uses the spherical primitive and substitutes the per\-promptδi\\delta\_\{i\}forvℓ⋆v^\{\\ell^\{\\star\}\}\. The steered hidden state propagates through the downstream layers to the logits, and the SFT objective becomes:

ℒSV​\-​SFT\(θ\)=−∑i∑tmi,tlogπθSV\(yi,t\|xi,yi,<t\),\\mathcal\{L\}\_\{\\mathrm\{SV\\text\{\-\}SFT\}\}\(\\theta\)\\;=\\;\-\\sum\_\{i\}\\sum\_\{t\}m\_\{i,t\}\\,\\log\\pi\_\{\\theta\}^\{\\mathrm\{SV\}\}\\\!\\left\(y\_\{i,t\}\\,\\big\|\\,x\_\{i\},\\,y\_\{i,<t\}\\right\),\(9\)whereπθSV\\pi\_\{\\theta\}^\{\\mathrm\{SV\}\}denotes the model whose residual stream at blockℓ⋆\\ell^\{\\star\}has been steered by Equation \([8](https://arxiv.org/html/2610.40998#S4.E8)\), and the gradient flows throughh~ℓ⋆\\tilde\{h\}^\{\\ell^\{\\star\}\}back to all trainable parameters\.ℒSV​\-​SFT\\mathcal\{L\}\_\{\\mathrm\{SV\\text\{\-\}SFT\}\}collapses toℒSFT\\mathcal\{L\}\_\{\\mathrm\{SFT\}\}when the injected direction is zero\. The same procedure transfers verbatim to distillation, with the reference completion now produced by an external teacher\. The full framework is shown in[Fig\.1](https://arxiv.org/html/2610.40998#S3.F1)\(A\) and \(B\)\.

Does the SV leak target information?δi\\delta\_\{i\}summarizes the full reference and is injected at every answer position, so it gives each position information that plain teacher forcing does not, which is the intended training signal\. It is not evaluation leakage, because the SV is built only from training\-set references, and*no vector is injected at test time*, so any gain must be stored in the parameters\.

### 4\.2Supervised Fine\-Tuning

#### 4\.2\.1Experimental Setting

Models and datasets\.We train four models that span family and scale, namely Qwen2\.5\-1\.5B\-Instruct and Qwen2\.5\-7B\-Instruct\[[38](https://arxiv.org/html/2610.40998#bib.bib19)\], Qwen3\-4B \(no\-thinking mode\)\[[39](https://arxiv.org/html/2610.40998#bib.bib1)\], and llama3\.2\-3B\[[40](https://arxiv.org/html/2610.40998#bib.bib49)\], on four datasets that span grade\-school math \(GSM8K\)\[[27](https://arxiv.org/html/2610.40998#bib.bib25)\], competition\-level math \(MATH\)\[[41](https://arxiv.org/html/2610.40998#bib.bib26)\], code synthesis \(MBPP\)\[[42](https://arxiv.org/html/2610.40998#bib.bib27)\], and discrete reasoning \(StrategyQA\)\[[43](https://arxiv.org/html/2610.40998#bib.bib28)\]\.

Training and evaluation protocol\.Following common parameter\-efficient fine\-tuning practice for LLMs\[[44](https://arxiv.org/html/2610.40998#bib.bib48)\],all runs use LoRA\[[25](https://arxiv.org/html/2610.40998#bib.bib23)\]with rankr=8r=8, learning rate5×10−55\\times 10^\{\-5\}, batch size88, and one epoch\. Steering injection uses Per\-sample Slerp atα=0\.1\\alpha=0\.1, Mean\-SV atα=0\.5\\alpha=0\.5and layerℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\+1over the full answer span\. All evaluations \(including results in other parts as well\) use vLLM with greedy decoding unless otherwise specified\. All numbers reported below are accuracies on the official test split\. Experiments are done using 2 Nvidia L40\-S\.

Hyperparameter selection\.The layerℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\+1is fixed in advance by the mid\-stack rule of inference\-time steering\[[2](https://arxiv.org/html/2610.40998#bib.bib4)\]\. The operation andα\\alphaare selected once on Qwen2\.5\-1\.5B with GSM8K, training on6,0006\{,\}000training examples and validating on the remaining ones, and this single configuration is reused unchanged for every model and dataset\.

#### 4\.2\.2Main Results

To comprehensively compare normal SFT and SFT with steering vector, we show the results for four different models and four different datasets in[Table1](https://arxiv.org/html/2610.40998#S4.T1)\. The results show that using SV during SFT improves the performance after SFT in general, and we make the following observations:

Table 1:Comparison of test accuracy \(%\) after SFT across four models and datasets\. The best result per row is shown inbold\. SV\-augmented SFT improves over plain SFT in general\.ModelDatasetPlainMean\-SVPer\-sample SlerpQwen2\.5\-1\.5BGSM8K59\.1462\.8564\.14MATH24\.6027\.0027\.20MBPP47\.0045\.8045\.60StrategyQA62\.7464\.4863\.61Qwen2\.5\-7BGSM8K76\.8880\.1480\.21MATH43\.2044\.0047\.40MBPP64\.8065\.2065\.20StrategyQA78\.1779\.6279\.91Qwen3\-4BGSM8K80\.0681\.7384\.61MATH44\.8047\.8047\.60MBPP57\.4059\.6059\.40StrategyQA71\.6271\.1872\.93Llama\-3\.2\-3BGSM8K62\.0963\.0861\.79MATH25\.0025\.0027\.40MBPP48\.2048\.8048\.60StrategyQA66\.2365\.0764\.63

∙\\bulletAdding a steering vector during SFT is positive over Plain on the three Qwen models in1111of1212cells\. The only negative result among Qwen models is Qwen2\.5\-1\.5B on MBPP, and we analyze the cause in Appendix[AppendixC](https://arxiv.org/html/2610.40998#A3)\.

∙\\bulletPer\-sample Slerp generally beats Mean\-SV\. Per\-sample Slerp wins in88of1212Qwen cells over Mean\-SV despite using a5×5\\timessmaller injection coefficient\.

∙\\bulletThe default configuration transfers poorly to Llama\-3\.2\-3B\. Per\-sample Slerp gives a mean gain of only\+0\.23\+0\.23pp across the four Llama datasets\. We analyze this in detail in the following analysis section, and the results indicate the failure is due to layer choice rather than the SV mechanism itself\.

#### 4\.2\.3Analysis

We run a series of ablations on Qwen2\.5\-1\.5B with GSM8K to understand what drives the gain\.

Sensitivity to the injection coefficientα\\alpha\.To understand the dynamics of SFT with SV, we first sweep the injection coefficientα\\alphafor both Per\-sample Slerp and Mean\-SV additive at a fixed layerℓ⋆=15\\ell^\{\\star\}=15\. As is shown in[Fig\.2](https://arxiv.org/html/2610.40998#S4.F2)\(a\), on Qwen2\.5\-1\.5B, Per\-sample Slerp collapses to0%0\\%atα=1\.0\\alpha=1\.0because the hidden state is fully rotated away from the prediction direction, while Mean\-SV additive has a broad plateau\. Per\-sample Slerp therefore achieves a higher peak but is more sensitive to the choice ofα\\alpha\. This suggests that Slerp unlocks superior performance provided thatα\\alphais carefully tuned to avoid representation collapse, whereas Mean\-SV serves as a safer, more forgiving alternative when extensive hyperparameter search is prohibitive\.

Influence of injection layerℓ⋆\\ell^\{\\star\}\.Next, we investigate the influence of the injection layer by fixingα=0\.1\\alpha=0\.1for Per\-sample Slerp\. As is shown in[Fig\.2](https://arxiv.org/html/2610.40998#S4.F2)\(b\), Llama\-3\.2\-3B has a strikingly different layer profile compared to Qwen:ℓ⋆∈\{10,13,19\}\\ell^\{\\star\}\\in\\\{10,13,19\\\}all surpass Plain SFT, andℓ⋆=10\\ell^\{\\star\}=10andℓ⋆=19\\ell^\{\\star\}=19are tied at the peak\. The negative Llama row at the defaultℓ⋆=15\\ell^\{\\star\}=15in[Table1](https://arxiv.org/html/2610.40998#S4.T1)is therefore not a failure of SV on Llama but a failure of layer transfer across families, indicating that the optimal injection layer or placement choice is highly model\-specific\. What transfers from inference\-time practice is the mid\-stack region, since on both families neither shallow nor deep layers are ever optimal\. For a new backbone, a0\.250\.25\-epoch layer sweep validated on the last500500training examples recovers the best layer on both Qwen2\.5\-1\.5B and Llama\-3\.2\-3B without test data, as detailed in[AppendixB](https://arxiv.org/html/2610.40998#A2)\.

\(a\)α\\alpha\-sensitivity curves of Per\-sample Slerp and Mean\-SV at fixedℓ⋆=15\\ell^\{\\star\}=15\. Slerp peaks sharply while Mean\-SV has a broad plateau\.\(b\)Accuracy of Per\-sample Slerp as the injection layerℓ⋆\\ell^\{\\star\}is varied withα=0\.1\\alpha=0\.1\. The best layer for Qwen and Llama is quite different\.\(c\)Accuracy under inference\-time SV injection on three checkpoints, showing that adding SV at inference time does not help\.
Figure 2:Analysis of our SV\-augmented SFT on Qwen2\.5\-1\.5B with GSM8K\.Table 2:Four representative SV injection variants for SV SFT on Qwen2\.5\-1\.5B with GSM8K\. The full set of variants we tested is reported in Appendix[AppendixC](https://arxiv.org/html/2610.40998#A3)\. The best result is shown inbold\. Per\-sample Slerp outperforms every alternative SV construction we tested\.Methodα\\alphaAccuracyΔ\\DeltaPlain LoRA \(no SV\)–59\.140Random per\-sample Gaussian \(control\)0\.5054\.82−4\.32\-4\.32PCA SV0\.5059\.29\+0\.15\+0\.15Multi\-layer \(L=10/15/20\)0\.50/0\.50/0\.5060\.85\+1\.71\+1\.71Multi\-layer \(L=10/15/20\)0\.10/0\.50/0\.1062\.62\+3\.48\+3\.48Per\-sample additive0\.5062\.93\+3\.79\+3\.79Per\-sample Slerp \(full answer\)0\.1064\.14\+5\.00\\mathbf\{\+5\.00\}

Direction and injection variants\.To check that our findings are generalizable, we compare our method against four representative alternatives in[Table2](https://arxiv.org/html/2610.40998#S4.T2)and defer the full set of variants we tested to Appendix[SectionC\.2](https://arxiv.org/html/2610.40998#A3.SS2)\.*Random per\-sample Gaussian*replacesδi\\delta\_\{i\}with a Gaussian noise and serves as a control for whether SV causes the improvement\.*PCA SV*\[[3](https://arxiv.org/html/2610.40998#bib.bib5)\]replaces the dataset\-level mean with the principal component \(first 5\) of the per\-sampleδi\\delta\_\{i\}matrix to reduce the influence of confounding\.*Multi\-layer*inject Mean\-SV at three transformer blocksL∈\{10,15,20\}L\\in\\\{10,15,20\\\}simultaneously\.*Per\-sample additive*keeps the per\-sampleδi\\delta\_\{i\}but injects it through the additive primitive of Equation \([2](https://arxiv.org/html/2610.40998#S3.E2)\) rather than the spherical primitive of Equation \([3](https://arxiv.org/html/2610.40998#S3.E3)\), isolating the contribution of the Slerp injection geometry from the per\-sample direction\. Overall, we have the following conclusions:

∙\\bulletThe random Gaussian control hurts the model, which shows that what our method exploits is the specific GT\-Pred direction in residual space and not the existence of an additive perturbation\.

∙\\bulletPCA gains little over Plain LoRA, indicating that confounding is not a major problem here\.

∙\\bulletMulti\-layer Mean\-SV is sensitive to theα\\alphaschedule\. The uniform0\.50/0\.50/0\.500\.50/0\.50/0\.50schedule underperforms the single\-layer Mean\-SV, while the tuned schedule that concentrates the weight on L15 beats it\. Multi\-layer injection can therefore help, but only with careful per\-layer tuning\.

∙\\bulletPer\-sample additive clearly improves but still trails Per\-sample Slerp, which decomposes the headline gain into per\-sample heterogeneity and the spherical injection geometry to show that both ingredients are needed\.

Inference\-time SV is qualitatively different from training\-time SV\.The original use of SVs is at inference time, and we test the same direction at inference, with results in \(c\) of[Fig\.2](https://arxiv.org/html/2610.40998#S4.F2)\. This experiment serves two purposes\. First, the base and Plain\-SFT curves form a control test\. If our direction were simply a good inference\-time steer, injecting it into a model that never saw it during training should already help\. It does not, so the training\-time gain is not a post\-hoc steering effect\. Second, the SV\-trained curve forms a distribution\-match test\. Since this checkpoint is trained with the SV on every answer token, keeping the SV at inference matches its training distribution, yet it performs worse than dropping the SV\. We attribute this to the SV offloading the style and format component of the reference shift during training\. In support of this account, the change that training makes to the layer\-ℓ⋆\\ell^\{\\star\}hidden state is nearly orthogonal to the SV \(cosine≈−0\.05\\approx\-0\.05\), whereas SV\-free training aligns positively with it, so the gradient concentrates on the correctness\-relevant remainder and re\-injecting the SV at inference only perturbs the hidden state\. Results on more models and datasets are in[SectionC\.1](https://arxiv.org/html/2610.40998#A3.SS1)\.

### 4\.3Distillation

#### 4\.3\.1Experimental Setting

Teachers, students, and datasets\.We use three students and four datasets from[Section4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1)and pair them with four teachers chosen to vary both mode and family, namely Qwen3\-4B\-Instruct\-2507 in non\-thinking mode and Qwen3\-8B in thinking mode\[[38](https://arxiv.org/html/2610.40998#bib.bib19)\], and Gemma\-4\-26B\-A4B\[[45](https://arxiv.org/html/2610.40998#bib.bib47)\]in non\-thinking and thinking mode, a different\-family model\. Each teacher is asked to produce one completion per training prompt, and we keep only the completions that pass the dataset’s verifier\.

Distillation protocol\.The student is trained with the same LoRA configuration as in Section[4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1)\. The steering vector is now constructed from the contrast between the student’s hidden state on the teacher’s completion, and the student’s hidden state on its own completion of the same prompt\.

#### 4\.3\.2Main Results

To comprehensively compare distillation and distillation with SV, we show the results for three different models in[Table3](https://arxiv.org/html/2610.40998#S4.T3)\. We make the following observations:

Table 3:Comparison of test accuracy \(%\) across three students, four teachers, and four datasets\. The best result is shown inbold\. SV\-augmented distillation improves over plain distillation on math, with the largest gains on thinking\-mode teachers\.StudentDatasetQwen3\-4B\-InstructQwen3\-8BGemma\-4\-26B \(no\-thinking\)Gemma\-4\-26B \(thinking\)PlainMean\-SVSlerpPlainMean\-SVSlerpPlainMean\-SVSlerpPlainMean\-SVSlerpQwen2\.5\-1\.5BGSM8K74\.8374\.8374\.6865\.8167\.5567\.4070\.5871\.1170\.4349\.5860\.4260\.50MATH41\.6041\.4042\.2021\.2031\.2033\.6036\.6037\.6037\.4011\.8015\.0014\.00MBPP49\.0048\.8050\.0046\.0045\.6045\.4048\.8048\.0048\.8027\.0031\.8037\.40StrategyQA56\.9158\.0855\.3155\.3159\.8360\.8461\.8660\.9958\.5220\.8219\.0721\.11Qwen3\-4BGSM8K87\.8788\.0287\.9576\.4275\.5178\.6286\.8887\.3486\.7362\.7052\.7767\.63MATH67\.6067\.2067\.0066\.0065\.8066\.2066\.0064\.8067\.8019\.4027\.2026\.60MBPP65\.0066\.4066\.4065\.4065\.2065\.4065\.2065\.0065\.0064\.0064\.4064\.40StrategyQA65\.2164\.7764\.4862\.0162\.8865\.5071\.1872\.2069\.1445\.4150\.5139\.88Llama\-3\.2\-3BGSM8K77\.4876\.7276\.6567\.1768\.2367\.8571\.4272\.6372\.9347\.6145\.7946\.63MATH38\.0038\.0038\.009\.8021\.0019\.6034\.2034\.4036\.0012\.2015\.4017\.00MBPP49\.4049\.4049\.2044\.6045\.6045\.4050\.4050\.2050\.6047\.2047\.6046\.80StrategyQA59\.6860\.5562\.4564\.1960\.4162\.5965\.7965\.7967\.9834\.5033\.7733\.77

∙\\bulletTeacher mode dominates teacher family in determining the SV gain\. Both thinking teachers \(Qwen3\-8B and Gemma\-thinking\) yield the biggest gains over Plain, up to12%12\\%, while both no\-thinking teachers are essentially flat\. A plausible reason is that no\-thinking teachers, even from a different family, produce outputs whose style is close to students, so the contrast between the teacher’s completion and the student’s own completion is small, and the SV has little to correct\. Thinking teachers, by contrast, produce long reasoning traces that the student would hardly emit, which widens the contrast between the two populations and gives the SV a much richer correction signal\.

∙\\bulletLlama\-3\.2\-3B under\-uses Per\-sample Slerp under all four teachers, with mean SV gains well below the corresponding Qwen\-student gains on every teacher\. We attribute this to the same layer\-choice mismatch identified in the SFT main table, not a failure of the SV mechanism itself\.

#### 4\.3\.3Analysis

This subsection presents the two analyses that directly support our observation about the distillation results\. Further analyses, including normalized SV vs\. unnormalized SV, are deferred to[AppendixD](https://arxiv.org/html/2610.40998#A4)\.

Mitigating the off\-policy gap\.If the SV truly closes the off\-policy gap, then injecting it into the base model under teacher forcing should raise the probability of the reference completion without any parameter update\. We apply the training\-time SV hook on500500training samples and report two statistics on all answer tokens by teacher forcing\. The first,Δ​NLLsum\\Delta\\mathrm\{NLL\}\_\{\\mathrm\{sum\}\}, measures the log\-probability gain on the full label sequence\. The second,Δ​Psum\\Delta P\_\{\\mathrm\{sum\}\}, measures the linear probability mass moved onto label tokens\. Both metrics tell the same story in[Figs\.3\(a\)](https://arxiv.org/html/2610.40998#S4.F3.sf1)and[3\(b\)](https://arxiv.org/html/2610.40998#S4.F3.sf2): the evidence is consistent with gap mitigation, with large and consistent effects on thinking\-teacher cells and much weaker effects on non\-thinking\-teacher cells\. The full ten\-cell sweep in[AppendixD](https://arxiv.org/html/2610.40998#A4)shows that the cells without activation\-level mitigation are exactly the cells whose SV\-distill accuracy gain is small\.

Influence of teacher mode and family\.The cross\-teacher cosine similarity shown in[Fig\.3\(c\)](https://arxiv.org/html/2610.40998#S4.F3.sf3)provides the geometric explanation of the main\-text observation that mode dominates family\. Same\-mode teachers from different families remain well\-aligned in the SV direction, while different\-mode pairs align poorly even within the same family\. Switching the teacher’s mode, therefore, reshapes the SV direction far more than switching the teacher’s family does, which is consistent with the results pattern that teacher mode dominates teacher family in determining the SV gain\.

\(a\)Δ​NLLsum\\Delta\\mathrm\{NLL\}\_\{\\mathrm\{sum\}\}on Qwen3\-4B across four distillation teachers\. Numbers in parentheses arefrac⁡\(Δ​NLLsum\>0\)\\mathrm\{frac\}\(\\Delta\\mathrm\{NLL\}\_\{\\mathrm\{sum\}\}\>0\)for samples\.![Refer to caption](https://arxiv.org/html/2610.40998v1/fig_distill_offpolicy_main_psum.png)\(b\)Δ​Psum\\Delta P\_\{\\mathrm\{sum\}\}on the same four cells, in linear probability units\. The ordering matches figure \(a\)\.
![Refer to caption](https://arxiv.org/html/2610.40998v1/fig_distill_cross_teacher.png)\(c\)Cosine similarity of the SV directionv\(ℓ\)v^\{\(\\ell\)\}between the four teachers, averaged over the 12 \(student, dataset\) pairs of each teacher\.

Figure 3:Analysis on distillation settings\. SV\-augmented distillation closes the off\-policy gap, and the SV direction is reshaped more by switching teacher mode than by switching teacher family\.

### 4\.4Multi\-Seed Reliability

Since the main tables use a single seed, we rerun representative SFT and distillation cells with seeds\{42,1,2\}\\\{42,1,2\\\}, using the same pipeline and the fixed configuration\. As shown in[Table4](https://arxiv.org/html/2610.40998#S4.T4), every reasoning and distillation cell improves over Plain on all three seeds\. The Qwen2\.5\-1\.5B GSM8K gain is about eight times the larger seed standard deviation, and distillation from the thinking teacher nearly doubles plain distillation, while MBPP stays neutral within noise, consistent with the reference direction being the wrong target for code\.

Table 4:Accuracy \(%, mean±\\pmstd over three seeds\) of Plain and Per\-sample Slerp\. “Distill” uses the Qwen3\-8B thinking teacher\.1\.5B GSM8K7B GSM8K1\.5B MATHQwen3\-4B MATH7B MATH1\.5B MATH \(Distill\)1\.5B MBPPPlain58\.81±0\.3758\.81\\pm 0\.3777\.66±1\.7377\.66\\pm 1\.7325\.80±0\.5325\.80\\pm 0\.5342\.47±2\.2042\.47\\pm 2\.2042\.47±1\.2242\.47\\pm 1\.2218\.13±2\.3018\.13\\pm 2\.3046\.13±0\.3146\.13\\pm 0\.31Per\-sample Slerp62\.75±0\.5062\.75\\pm 0\.5080\.74±0\.4980\.74\\pm 0\.4927\.00±0\.2027\.00\\pm 0\.2047\.07±1\.7047\.07\\pm 1\.7046\.20±1\.4046\.20\\pm 1\.4033\.47±1\.1433\.47\\pm 1\.1446\.47±1\.0146\.47\\pm 1\.01Δ\\Delta\+3\.94\+3\.94\+3\.08\+3\.08\+1\.20\+1\.20\+4\.60\+4\.60\+3\.73\+3\.73\+15\.34\+15\.34\+0\.34\+0\.34

## 5Steering Vector Post\-training: Reinforcement Learning Regime

### 5\.1Method

As reviewed in Section[3\.2](https://arxiv.org/html/2610.40998#S3.SS2), GRPO collapses on zero\-variance groups\. One case in practice is the all\-wrong group\. These groups concentrate on the hardest prompts, exactly where learning is most needed, and the training budget is therefore wasted on prompts that produce zero gradient\. In this section, we use advantage\-repair to fix this problem\. Following the framework introduced in Section[1](https://arxiv.org/html/2610.40998#S1), we describe the advantage\-repair regime along three axes as well\.

Direction\.To recover a learning signal on zero\-variance groups, we need a continuous proxy for correctness that does not rely on the verifier\. The natural choice is the residual\-stream direction along which correct rollouts differ from wrong ones\. Projecting a rollout onto this direction yields a continuous proxy score that is expected to correlate with the correct\-vs\-wrong contrast, even when the group\-level GRPO advantage is uninformative\. Concretely, before GRPO training, we run the reference model on the training prompts to collect a buffer of rollouts, label each as correct or wrong with the verifier, and define:

vcorrℓ⋆=𝔼\(i,k\)∈Correct​\[h¯i,kℓ⋆\]−𝔼\(i,k\)∈Wrong​\[h¯i,kℓ⋆\],v^\{\\ell^\{\\star\}\}\_\{\\mathrm\{corr\}\}\\;=\\;\\mathbb\{E\}\_\{\(i,k\)\\in\\mathrm\{Correct\}\}\\\!\\left\[\\bar\{h\}^\{\\ell^\{\\star\}\}\_\{i,k\}\\right\]\\;\-\\;\\mathbb\{E\}\_\{\(i,k\)\\in\\mathrm\{Wrong\}\}\\\!\\left\[\\bar\{h\}^\{\\ell^\{\\star\}\}\_\{i,k\}\\right\],\(10\)the answer\-token\-pooled mean\-activation difference at layerℓ⋆\\ell^\{\\star\}between correct and wrong rollouts\. Recomputing this direction online during RL would introduce a moving target as the policy shifts and would add a per\-step encoding cost, so we precompute it once and freeze it\.

Placement\.Following the SFT setup, we use the middle layerℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\+1\. The score for rollout\(i,k\)\(i,k\)is the average projection over the latter halfWi,kW\_\{i,k\}of its answer tokens:

zi,ksv=1\|Wi,k\|​∑t∈Wi,k⟨hi,k,tℓ⋆,v^corrℓ⋆⟩,z^\{\\mathrm\{sv\}\}\_\{i,k\}\\;=\\;\\frac\{1\}\{\|W\_\{i,k\}\|\}\\sum\_\{t\\in W\_\{i,k\}\}\\big\\langle h^\{\\ell^\{\\star\}\}\_\{i,k,t\},\\,\\hat\{v\}^\{\\ell^\{\\star\}\}\_\{\\mathrm\{corr\}\}\\big\\rangle,\(11\)Intuitively, rollouts whose hidden states align more closely with the correct\-rollout pattern receive higher scores, so even within an all\-wrong group, the score distinguishes\.

Operation\.The remaining choice is how to feedzi,ksvz^\{\\mathrm\{sv\}\}\_\{i,k\}back into GRPO\. The GRPO advantage separates correct from wrong rollouts but treats all wrong rollouts of a prompt identically, and on all\-wrong groups it vanishes entirely\. We therefore leave correct rollouts untouched and act on every rollout the verifier has scored as wrong\. We standardizezi,ksvz^\{\\mathrm\{sv\}\}\_\{i,k\}within each group toz¯i,ksv\\bar\{z\}^\{\\mathrm\{sv\}\}\_\{i,k\}to match the scale of the GRPO advantage, and shift the advantage on wrong rollouts only:

Ai,k=Ai,kGRPO\+λ⋅z¯i,ksv⋅\[ri,k=0\],A\_\{i,k\}\\;=\\;A^\{\\mathrm\{GRPO\}\}\_\{i,k\}\\;\+\\;\\lambda\\cdot\\bar\{z\}^\{\\mathrm\{sv\}\}\_\{i,k\}\\cdot\\mathbf\{1\}\\\!\\left\[r\_\{i,k\}=0\\right\],\(12\)On all\-wrong groups,Ai,kGRPO=0A^\{\\mathrm\{GRPO\}\}\_\{i,k\}=0and the advantage reduces to the SV ranking, turning an otherwise silent prompt into a learning signal; in mixed\-reward groups, the term additionally ranks the wrong rollouts by their alignment with the correctness direction\. We call this advantage modification*SV\-Rescue*\. Because it touches only the advantage estimator, it composes cleanly with orthogonal rollout\-side mechanisms\. For example, the sampling temperature can be raised after steps that produced many all\-wrong groups, encouraging more exploration on hard prompts\. We call SV\-Rescue applied on top of such a temperature schedule*SV\-Explore*\. The full framework is shown in[Fig\.1](https://arxiv.org/html/2610.40998#S3.F1)\(C\)\.

Table 5:Comparison of final\-step validation accuracy \(%\) for GRPO post\-training\. The best result per row is shown inbold\. SV\-augmented GRPO consistently beats Plain GRPO on the math datasets\.StudentTraining datasetPlainSV\-RescueSV\-ExploreQwen2\.5\-1\.5BGSM8K77\.7278\.3579\.18DeepScaleR40\.2241\.7642\.30DeepCoder13\.9816\.1312\.90Qwen3\-4BGSM8K90\.7691\.0491\.99DeepScaleR52\.3354\.1255\.96DeepCoder37\.1935\.1836\.18Llama\-3\.2\-3BGSM8K81\.8384\.4986\.71DeepScaleR36\.6938\.8938\.92DeepCoder21\.0521\.0524\.21

Table 6:Out\-of\-distribution \(OOD\) evaluation accuracy \(%\) of the GRPO checkpoints from[Table6](https://arxiv.org/html/2610.40998#S5.T6)\. The best result is shown inbold\. The in\-domain SV gain transfers cleanly to OOD\.StudentMethodMATH\-500AIME\-24AIME\-25LiveCodeBenchQwen2\.5\-1\.5BPlain34\.601\.670\.337\.75SV\-Rescue35\.803\.331\.007\.75SV\-Explore37\.002\.330\.678\.25Qwen3\-4BPlain55\.2022\.0021\.0034\.00SV\-Rescue56\.8023\.3322\.6633\.00SV\-Explore56\.4024\.6622\.0036\.00Llama\-3\.2\-3BPlain26\.604\.000\.3320\.50SV\-Rescue32\.004\.330\.6722\.50SV\-Explore30\.804\.331\.0021\.25

### 5\.2Experiments

#### 5\.2\.1Experimental Setting

Models, datasets, and initial checkpoints\.We run RL on the same three models in[Section4\.3](https://arxiv.org/html/2610.40998#S4.SS3)\. Each student is paired with three training datasets that span easy math \(GSM8K\)\[[27](https://arxiv.org/html/2610.40998#bib.bib25)\], hard math \(DeepScaleR\)\[[46](https://arxiv.org/html/2610.40998#bib.bib43)\], and code \(DeepCoder\)\[[47](https://arxiv.org/html/2610.40998#bib.bib44)\]\.

GRPO configuration\.All runs use the verl\[[48](https://arxiv.org/html/2610.40998#bib.bib42)\]GRPO implementation withn=8n=8rollouts per prompt, KL coefficientβ=5×10−3\\beta=5\\times 10^\{\-3\}, base sampling temperatureTlow=0\.7T\_\{\\mathrm\{low\}\}=0\.7, learning rate5×10−55\\times 10^\{\-5\}\. The actor is updated through LoRA with rankr=8r=8on the Qwen students, and through full\-parameter fine\-tuning on Llama\-3\.2\-3B\. The SV used by both SV\-Rescue and SV\-Explore is the same precomputed correctness direction using 500 samples in total\. We useλ=0\.1\\lambda=0\.1for our method\.

In\-domain and out\-of\-distribution evaluation\.For each \(student, training\-dataset, method\) cell we report the final\-step validation accuracy on the training dataset’s official test split \(GSM8K, DeepScaleR\-val, DeepCoder TACO\-Verified\)\. For out\-of\-distribution evaluation, we run the trained checkpoint zero\-shot on a held\-out math benchmark: GSM8K\-trained checkpoints are evaluated on MATH\-500\[[41](https://arxiv.org/html/2610.40998#bib.bib26),[49](https://arxiv.org/html/2610.40998#bib.bib45)\], while DeepScaleR\-trained checkpoints are evaluated on AIME\-24 and 25\[[50](https://arxiv.org/html/2610.40998#bib.bib46)\]\.

#### 5\.2\.2Main Results

To comprehensively compare GRPO and GRPO with SV, we show the in\-domain validation accuracy for three different students in[Table6](https://arxiv.org/html/2610.40998#S5.T6)\. We make the following observations:

SV\-Explore is the strongest method on math, and SV\-Rescue provides a smaller but positive lift over Plain GRPO in general\. The gain is largest on Llama\-3\.2\-3B, the weakest base model and therefore the student that produces the most wrong rollouts and all\-wrong groups, giving the SV term the most rollouts to act on\. On code, no method wins consistently across the three students, which is the same as the SFT regime for code, suggesting that code calls for a correctness direction different from the direction that works on math\.

#### 5\.2\.3Out\-of\-Distribution Evaluation

[Table6](https://arxiv.org/html/2610.40998#S5.T6)reports the OOD accuracy of every trained checkpoint on its corresponding held\-out benchmark, with MATH\-500 evaluating the GSM8K\-trained checkpoint, AIME\-24 and AIME\-25 evaluating the DeepScaleR\-trained checkpoint, and LiveCodeBench evaluating the DeepCoder\-trained checkpoint\. AIME accuracy is averaged over1010independent samples per problem using temperature 0\.7\. The in\-domain advantage of the SV\-augmented methods carries over cleanly to the OOD regime\. The largest reasoning OOD gain is on Llama\-3\.2\-3B’s GSM8K\-to\-MATH\-500 transfer, where both SV variants clear Plain by a wide margin\. Due to the page limit, we defer more analysis, including ablation of RL experiments in Appendix[AppendixE](https://arxiv.org/html/2610.40998#A5)\.

## 6Conclusion

In this paper, we present the first systematic investigation of steering vectors as persistent training signals rather than mere post\-hoc inference controls\. By integrating SVs into both supervised learning and RL pipelines, we demonstrate that internal representations can effectively improve post\-training\. In the supervised regime, our mechanism mitigates the off\-policy gap inherent in teacher forcing, yielding consistent reasoning improvements, especially when distilling from thinking\-mode teachers, and these gains hold over three seeds\. In the RL regime, SV\-Rescue recovers a learning signal on wrong rollouts, including all\-wrong groups, and works best as a composable auxiliary signal on top of existing remedies and exploration schedules \(SV\-Explore\)\. While our comprehensive analysis maps specific boundary conditions, such as limited efficacy on code generation and the necessity of family\-specific layer tuning, our results establish steering vectors as an effective training signal for advancing modern LLM post\-training\.

## References

- \[1\]A\. M\. Turner, L\. Thiergart, G\. Leech, D\. Udell, J\. J\. Vazquez, U\. Mini, and M\. MacDiarmid\(2023\)Steering language models with activation engineering\.arXiv preprint arXiv:2308\.10248\.Cited by:[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px1.p1.1),[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px3.p1.1),[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.40998#S1.p1.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.40998#S3.SS1.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.40998#S3.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.40998#S4.SS1.p3.1)\.
- \[2\]N\. Rimsky, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. Turner\(2024\)Steering llama 2 via contrastive activation addition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15504–15522\.External Links:[Link](https://aclanthology.org/2024.acl-long.828/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.828)Cited by:[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px1.p1.1),[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px3.p1.1),[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px4.p1.1),[1st item](https://arxiv.org/html/2610.40998#A2.I1.i1.p1.1),[§1](https://arxiv.org/html/2610.40998#S1.p1.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.40998#S3.SS1.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.40998#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2610.40998#S3.SS1.p2.1),[§4\.1](https://arxiv.org/html/2610.40998#S4.SS1.p3.1),[§4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1.p3.1)\.
- \[3\]A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski,et al\.\(2023\)Representation engineering: a top\-down approach to ai transparency\.arXiv preprint arXiv:2310\.01405\.Cited by:[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px1.p1.1),[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2610.40998#S1.p1.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.40998#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2610.40998#S3.SS1.p2.1),[§4\.2\.3](https://arxiv.org/html/2610.40998#S4.SS2.SSS3.p4.1)\.
- \[4\]L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[§1](https://arxiv.org/html/2610.40998#S1.p2.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[5\]G\. Hinton, O\. Vinyals, and J\. Dean\(2015\)Distilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.Cited by:[§1](https://arxiv.org/html/2610.40998#S1.p2.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[6\]M\. Ranzato, S\. Chopra, M\. Auli, and W\. Zaremba\(2016\)Sequence level training with recurrent neural networks\.In4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2\-4, 2016, Conference Track Proceedings,Y\. Bengio and Y\. LeCun \(Eds\.\),External Links:[Link](http://arxiv.org/abs/1511.06732)Cited by:[§1](https://arxiv.org/html/2610.40998#S1.p2.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[7\]D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.\(2025\)Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§1](https://arxiv.org/html/2610.40998#S1.p3.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[8\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§1](https://arxiv.org/html/2610.40998#S1.p3.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2610.40998#S3.SS2.p1.1)\.
- \[9\]T\. V\. Le, M\. Jeon, K\. Vu, V\. Lai, and E\. Yang\(2025\)No prompt left behind: exploiting zero\-variance prompts in llm reinforcement learning via entropy\-guided advantage shaping\.arXiv preprint arXiv:2509\.21880\.Cited by:[Appendix E](https://arxiv.org/html/2610.40998#A5.SS0.SSS0.Px3.p1.1),[Appendix F](https://arxiv.org/html/2610.40998#A6.SS0.SSS0.Px4),[§1](https://arxiv.org/html/2610.40998#S1.p5.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[10\]K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. Wattenberg\(2023\)Inference\-time intervention: eliciting truthful answers from a language model\.Advances in Neural Information Processing Systems36,pp\. 41451–41530\.Cited by:[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.40998#S3.SS1.p1.1)\.
- \[11\]E\. Todd, M\. Li, A\. Sharma, A\. Mueller, B\. C\. Wallace, and D\. Bau\(2024\)Function vectors in large language models\.InInternational conference on learning representations,Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1)\.
- \[12\]G\. Ilharco, M\. T\. Ribeiro, M\. Wortsman, L\. Schmidt, H\. Hajishirzi, and A\. FarhadiEditing models with task arithmetic\.InThe Eleventh International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1)\.
- \[13\]J\. Braun, C\. Eickhoff, D\. Krueger, S\. A\. Bahrainian, and D\. Krasheninnikov\(2025\)Understanding \(un\) reliability of steering vectors in language models\.arXiv preprint arXiv:2505\.22637\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1)\.
- \[14\]A\. Wang, X\. Wu, D\. Shu, Y\. Ma, and N\. Liu\(2025\)Enhancing llm steering through sparse autoencoder\-based vector refinement\.arXiv preprint arXiv:2509\.23799\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1)\.
- \[15\]J\. Postmus and S\. Abreu\(2024\)Steering large language models using conceptors: improving addition\-based activation engineering\.arXiv preprint arXiv:2410\.16314\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1)\.
- \[16\]X\. Kang, D\. Shi, and L\. Chen\(2026\)Model whisper: steering vectors unlock large language models’ potential in test\-time\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 31392–31400\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1)\.
- \[17\]W\. H\. Gan, D\. Fu, J\. Asilis, O\. Liu, D\. Yogatama, V\. Sharan, R\. Jia, and W\. Neiswanger\(2025\)Textual steering vectors can improve visual understanding in multimodal large language models\.arXiv preprint arXiv:2505\.14071\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1)\.
- \[18\]C\. Venhoff, I\. Arcuschin, P\. Torr, A\. Conmy, and N\. Nanda\(2025\)Understanding reasoning in thinking language models via steering vectors\.arXiv preprint arXiv:2506\.18167\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1)\.
- \[19\]D\. V\. Nguyen, H\. M\. Vu, N\. Y\. Pham, L\. Zhang, and T\. M\. Nguyen\(2025\)Activation steering with a feedback controller\.arXiv preprint arXiv:2510\.04309\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1)\.
- \[20\]K\. Shoemake\(1985\)Animating rotation with quaternion curves\.InProceedings of the 12th annual conference on Computer graphics and interactive techniques,pp\. 245–254\.Cited by:[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px5.p1.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.40998#S3.SS1.SSS0.Px1.p1.2)\.
- \[21\]C\. Goddard, S\. Siriwardhana, M\. Ehghaghi, L\. Meyers, V\. Karpukhin, B\. Benedict, M\. McQuade, and J\. Solawetz\(2024\)Arcee’s mergekit: a toolkit for merging large language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track,pp\. 477–485\.Cited by:[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px5.p1.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.40998#S3.SS1.SSS0.Px1.p1.2)\.
- \[22\]Z\. You, C\. Deng, and H\. Chen\(2026\)Spherical steering: geometry\-aware activation rotation for language models\.arXiv preprint arXiv:2602\.08169\.Cited by:[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px4.p1.2),[Appendix A](https://arxiv.org/html/2610.40998#A1.SS0.SSS0.Px5.p1.1),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.40998#S3.SS1.SSS0.Px1.p1.2)\.
- \[23\]Y\. Wang, Y\. Kordi, S\. Mishra, A\. Liu, N\. A\. Smith, D\. Khashabi, and H\. Hajishirzi\(2023\)Self\-instruct: aligning language models with self\-generated instructions\.InProceedings of the 61st annual meeting of the association for computational linguistics \(volume 1: long papers\),pp\. 13484–13508\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[24\]R\. Taori, I\. Gulrajani, T\. Zhang, Y\. Dubois, X\. Li, C\. Guestrin, P\. Liang, and T\. B\. Hashimoto\(2023\)Stanford alpaca: an instruction\-following llama model\.Stanford, CA, USA\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[25\]E\. J\. Hu, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, W\. Chen,et al\.LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1),[§4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1.p2.1)\.
- \[26\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, I\. Brian, F\. Xia, E\. Chi, Q\. V\. Le, and D\. Zhou\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[27\]K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2610.40998#S3.SS2.p1.1),[§4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1.p1.1),[§5\.2\.1](https://arxiv.org/html/2610.40998#S5.SS2.SSS1.p1.1)\.
- \[28\]E\. Zelikman, Y\. Wu, J\. Mu, and N\. Goodman\(2022\)Star: bootstrapping reasoning with reasoning\.Advances in Neural Information Processing Systems35,pp\. 15476–15488\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[29\]H\. Dong, W\. Xiong, D\. Goyal, Y\. Zhang, W\. Chow, R\. Pan, S\. Diao, J\. Zhang, K\. Shum, and T\. Zhang\(2023\)Raft: reward ranked finetuning for generative foundation model alignment\.arXiv preprint arXiv:2304\.06767\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[30\]Y\. Gu, L\. Dong, F\. Wei, and M\. Huang\(2024\)Minillm: knowledge distillation of large language models\.InThe twelfth international conference on learning representations,Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[31\]Y\. Wu, Y\. Zhou, Z\. Zhou, Y\. Peng, X\. Ye, X\. Hu, W\. Zhu, L\. Qi, M\. Yang,et al\.\(2026\)On the generalization of sft: a reinforcement learning perspective with reward rectification\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 27550–27571\.Cited by:[Appendix F](https://arxiv.org/html/2610.40998#A6.SS0.SSS0.Px1.p1.1),[Appendix F](https://arxiv.org/html/2610.40998#A6.SS0.SSS0.Px2),[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[32\]P\. F\. Christiano, J\. Leike, T\. Brown, M\. Martic, S\. Legg, and D\. Amodei\(2017\)Deep reinforcement learning from human preferences\.Advances in neural information processing systems30\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[33\]J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov\(2017\)Proximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2610.40998#S3.SS2.p1.1)\.
- \[34\]N\. Lambert, J\. Morrison, V\. Pyatkin, S\. Huang, H\. Ivison, F\. Brahman, L\. J\. V\. Miranda, A\. Liu, N\. Dziri, S\. Lyu,et al\.\(2024\)Tulu 3: pushing frontiers in open language model post\-training\.arXiv preprint arXiv:2411\.15124\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[35\]X\. Wen, Z\. Liu, S\. Zheng, S\. Ye, Z\. Wu, Y\. Wang, Z\. Xu, X\. Liang, J\. Li, Z\. Miao,et al\.\(2025\)Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms\.arXiv preprint arXiv:2506\.14245\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[36\]Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, W\. Dai, T\. Fan, G\. Liu, L\. Liu,et al\.\(2025\)Dapo: an open\-source llm reinforcement learning system at scale\.arXiv preprint arXiv:2503\.14476\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[37\]H\. Wu and W\. Liu\(2025\)GCPO: when contrast fails, go gold\.arXiv preprint arXiv:2510\.07790\.Cited by:[§2](https://arxiv.org/html/2610.40998#S2.SS0.SSS0.Px2.p1.1)\.
- \[38\]B\. Hui, J\. Yang, Z\. Cui, J\. Yang, D\. Liu, L\. Zhang, T\. Liu, J\. Zhang, B\. Yu, K\. Lu,et al\.\(2024\)Qwen2\. 5\-coder technical report\.arXiv preprint arXiv:2409\.12186\.Cited by:[§4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1.p1.1),[§4\.3\.1](https://arxiv.org/html/2610.40998#S4.SS3.SSS1.p1.1)\.
- \[39\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1.p1.1)\.
- \[40\]A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1.p1.1)\.
- \[41\]D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the math dataset\.InThirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 2\),Cited by:[§4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1.p1.1),[§5\.2\.1](https://arxiv.org/html/2610.40998#S5.SS2.SSS1.p3.1)\.
- \[42\]J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le,et al\.\(2021\)Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1.p1.1)\.
- \[43\]M\. Geva, D\. Khashabi, E\. Segal, T\. Khot, D\. Roth, and J\. Berant\(2021\)Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies\.Transactions of the Association for Computational Linguistics9,pp\. 346–361\.Cited by:[§4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1.p1.1)\.
- \[44\]Y\. Xu, R\. Zhang, X\. Jiang, Y\. Feng, Y\. Xiao, X\. Ma, R\. Zhu, X\. Chu, J\. Zhao, and Y\. Wang\(2025\)Parenting: optimizing knowledge selection of retrieval\-augmented language models with parameter decoupling and tailored tuning\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 11643–11662\.Cited by:[§4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1.p2.1)\.
- \[45\]Google\(2026\)Gemma 4: byte for byte, the most capable open models\.Note:[https://blog\.google/innovation\-and\-ai/technology/developers\-tools/gemma\-4/](https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/)Google Blog, The Keyword\. Accessed: 2026\-05\-04Cited by:[§4\.3\.1](https://arxiv.org/html/2610.40998#S4.SS3.SSS1.p1.1)\.
- \[46\]M\. Luo, S\. Tan, J\. Wong, X\. Shi, W\. Y\. Tang, M\. Roongta, C\. Cai, J\. Luo, L\. E\. Li, R\. A\. Popa, and I\. Stoica\(2025\)DeepScaleR: surpassing o1\-preview with a 1\.5b model by scaling rl\.Note:[https://pretty\-radio\-b75\.notion\.site/DeepScaleR\-Surpassing\-O1\-Preview\-with\-a\-1\-5B\-Model\-by\-Scaling\-RL\-19681902c1468005bed8ca303013a4e2](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2)Notion BlogCited by:[§5\.2\.1](https://arxiv.org/html/2610.40998#S5.SS2.SSS1.p1.1)\.
- \[47\]M\. Luo, S\. Tan, R\. Huang, A\. Patel, A\. Ariyak, Q\. Wu, X\. Shi, R\. Xin, C\. Cai, M\. Weber, C\. Zhang, L\. E\. Li, R\. A\. Popa, and I\. Stoica\(2025\)DeepCoder: a fully open\-source 14b coder at o3\-mini level\.Note:[https://pretty\-radio\-b75\.notion\.site/DeepCoder\-A\-Fully\-Open\-Source\-14B\-Coder\-at\-O3\-mini\-Level\-1cf81902c14680b3bee5eb349a512a51](https://pretty-radio-b75.notion.site/DeepCoder-A-Fully-Open-Source-14B-Coder-at-O3-mini-Level-1cf81902c14680b3bee5eb349a512a51)Notion BlogCited by:[§5\.2\.1](https://arxiv.org/html/2610.40998#S5.SS2.SSS1.p1.1)\.
- \[48\]G\. Sheng, C\. Zhang, Z\. Ye, X\. Wu, W\. Zhang, R\. Zhang, Y\. Peng, H\. Lin, and C\. Wu\(2025\)Hybridflow: a flexible and efficient rlhf framework\.InProceedings of the Twentieth European Conference on Computer Systems,pp\. 1279–1297\.Cited by:[§5\.2\.1](https://arxiv.org/html/2610.40998#S5.SS2.SSS1.p2.1)\.
- \[49\]H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe\(2024\)Let’s verify step by step\.InThe twelfth international conference on learning representations,Cited by:[§5\.2\.1](https://arxiv.org/html/2610.40998#S5.SS2.SSS1.p3.1)\.
- \[50\]Mathematical Association of AmericaAmerican invitational mathematics examination \(aime\)\.Note:[https://maa\.org/maa\-invitational\-competitions/](https://maa.org/maa-invitational-competitions/)Accessed: 2026\-05\-03Cited by:[§5\.2\.1](https://arxiv.org/html/2610.40998#S5.SS2.SSS1.p3.1)\.

## Appendix ASteering Vectors: Detailed Construction

This appendix gives the complete construction of the steering\-vector primitives used throughout the paper\. The abbreviated version is in Section[3\.1](https://arxiv.org/html/2610.40998#S3.SS1)of the main text\.

##### Linear representation hypothesis and goal\.

A broad empirical observation in the interpretability literature is that many high\-level behaviors of LLMs, such as sentiment, truthfulness, refusal, and task identity, are encoded along approximately linear directions in the residual stream\[[1](https://arxiv.org/html/2610.40998#bib.bib2),[10](https://arxiv.org/html/2610.40998#bib.bib3),[2](https://arxiv.org/html/2610.40998#bib.bib4),[3](https://arxiv.org/html/2610.40998#bib.bib5)\]\. This motivates the goal of finding, at each layerℓ\\ell, a single directionv\(ℓ\)∈ℝdv^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{d\}such that shifting the hidden state alongv\(ℓ\)v^\{\(\\ell\)\}predictably shifts the corresponding behavior in the model’s output without any parameter update\. Buildingv\(ℓ\)v^\{\(\\ell\)\}requires two ingredients: a way to summarize the hidden state produced on a response as one vector per layer, and a pair of response populations whose summaries contrast in the target behavior\.

##### Answer\-token pooling\.

A common choice for the per\-response summary is the mean hidden state over the response tokens, masked to exclude the prompt\. For a response of lengthTTwith answer maskm∈\{0,1\}Tm\\in\\\{0,1\\\}^\{T\}, the answer\-token mean hidden state at layerℓ\\ellis

h¯\(ℓ\)​\(x,y\)=∑t=1Tmt​h\(ℓ\)​\(x,y<t\)∑t=1Tmt∈ℝd\.\\bar\{h\}^\{\(\\ell\)\}\(x,y\)\\;=\\;\\frac\{\\sum\_\{t=1\}^\{T\}m\_\{t\}\\,h^\{\(\\ell\)\}\(x,y\_\{<t\}\)\}\{\\sum\_\{t=1\}^\{T\}m\_\{t\}\}\\;\\in\\;\\mathbb\{R\}^\{d\}\.\(13\)This collapses token\-level variation into a single direction per layer and is the standard summary used in representation\-level analyses of LLMs\.

##### Difference\-in\-means steering vector\.

Given two matched response populations𝒟\+\\mathcal\{D\}\_\{\+\}and𝒟−\\mathcal\{D\}\_\{\-\}that differ in the target behavior, the standard recipe\[[1](https://arxiv.org/html/2610.40998#bib.bib2),[2](https://arxiv.org/html/2610.40998#bib.bib4),[3](https://arxiv.org/html/2610.40998#bib.bib5)\]setsv\(ℓ\)v^\{\(\\ell\)\}to the difference of their answer\-token\-pooled means,

v\(ℓ\)=𝔼i∈𝒟\+​\[h¯\(ℓ\)​\(xi,yi\+\)\]−𝔼i∈𝒟−​\[h¯\(ℓ\)​\(xi,yi−\)\]\.v^\{\(\\ell\)\}\\;=\\;\\mathbb\{E\}\_\{i\\in\\mathcal\{D\}\_\{\+\}\}\\\!\\\!\\left\[\\bar\{h\}^\{\(\\ell\)\}\(x\_\{i\},y\_\{i\}^\{\+\}\)\\right\]\\;\-\\;\\mathbb\{E\}\_\{i\\in\\mathcal\{D\}\_\{\-\}\}\\\!\\\!\\left\[\\bar\{h\}^\{\(\\ell\)\}\(x\_\{i\},y\_\{i\}^\{\-\}\)\\right\]\.\(14\)We writev^\(ℓ\)=v\(ℓ\)/∥v\(ℓ\)∥2\\hat\{v\}^\{\(\\ell\)\}=v^\{\(\\ell\)\}/\\lVert v^\{\(\\ell\)\}\\rVert\_\{2\}for the unit\-normalized version\. Under the linear representation assumption, hidden states in𝒟\+\\mathcal\{D\}\_\{\+\}carry a larger component along the behavior\-relevant direction than those in𝒟−\\mathcal\{D\}\_\{\-\}, while they share the same distribution along unrelated axes\. Averaging over many samples in each population washes out the behavior\-unrelated variation, so the difference of the two means concentrates on the direction that discriminates the two populations\. Shifting the residual stream alongv\(ℓ\)v^\{\(\\ell\)\}at inference time then targets the same direction that was already correlated with the desired behavior in the training\-set activations\.

##### Additive injection\.

The simplest way to applyv\(ℓ\)v^\{\(\\ell\)\}to the hidden state at a chosen layerℓ⋆\\ell^\{\\star\}is the additive primitive\[[1](https://arxiv.org/html/2610.40998#bib.bib2),[2](https://arxiv.org/html/2610.40998#bib.bib4)\],

h~t\(ℓ⋆\)=ht\(ℓ⋆\)\+α​v\(ℓ⋆\),α∈ℝ\.\\tilde\{h\}^\{\(\\ell^\{\\star\}\)\}\_\{t\}\\;=\\;h^\{\(\\ell^\{\\star\}\)\}\_\{t\}\\;\+\\;\\alpha\\,v^\{\(\\ell^\{\\star\}\)\},\\qquad\\alpha\\in\\mathbb\{R\}\.\(15\)Additive injection changes both the direction and the magnitude of the hidden state, and has been reported to collapse the activation manifold at largeα\\alpha\[[22](https://arxiv.org/html/2610.40998#bib.bib17)\]\.

##### Spherical \(Slerp\) injection\.

Spherical linear interpolation \(Slerp\) was introduced for quaternion animation by[Shoemake \[20\]](https://arxiv.org/html/2610.40998#bib.bib15)and later adopted for weight\-space model merging\[[21](https://arxiv.org/html/2610.40998#bib.bib16)\]and for activation steering\[[22](https://arxiv.org/html/2610.40998#bib.bib17)\]\. It avoids the magnitude\-drift issue of additive injection by rotating the normalized hidden stateut=ht\(ℓ⋆\)/∥ht\(ℓ⋆\)∥2u\_\{t\}=h^\{\(\\ell^\{\\star\}\)\}\_\{t\}/\\lVert h^\{\(\\ell^\{\\star\}\)\}\_\{t\}\\rVert\_\{2\}toward a target directionwwalong the shortest great\-circle arc on the unit sphere,

Slerp⁡\(u,w,α\)=sin⁡\(\(1−α\)​ω\)sin⁡ω​u\+sin⁡\(α​ω\)sin⁡ω​w,ω=arccos⁡\(u⊤​w\)\.\\mathrm\{Slerp\}\(u,w;\\alpha\)\\;=\\;\\frac\{\\sin\\\!\\big\(\(1\-\\alpha\)\\omega\\big\)\}\{\\sin\\omega\}\\,u\\;\+\\;\\frac\{\\sin\(\\alpha\\omega\)\}\{\\sin\\omega\}\\,w,\\qquad\\omega=\\arccos\(u^\{\\\!\\top\}w\)\.\(16\)Takingw∝ht\(ℓ⋆\)\+v\(ℓ⋆\)w\\propto h^\{\(\\ell^\{\\star\}\)\}\_\{t\}\+v^\{\(\\ell^\{\\star\}\)\}gives the Slerp counterpart of Equation \([15](https://arxiv.org/html/2610.40998#A1.E15)\),

h~t\(ℓ⋆\)=∥ht\(ℓ⋆\)∥2⋅Slerp⁡\(ut,w,α\),\\tilde\{h\}^\{\(\\ell^\{\\star\}\)\}\_\{t\}\\;=\\;\\lVert h^\{\(\\ell^\{\\star\}\)\}\_\{t\}\\rVert\_\{2\}\\cdot\\mathrm\{Slerp\}\\\!\\left\(u\_\{t\},\\,w;\\,\\alpha\\right\),\(17\)which preserves the norm ofht\(ℓ⋆\)h^\{\(\\ell^\{\\star\}\)\}\_\{t\}by construction\.

## Appendix BHyperparameter Selection Protocol and Practical Guide

This section expands the selection protocol summarized in[Section4\.2\.1](https://arxiv.org/html/2610.40998#S4.SS2.SSS1)\.

##### What was fixed, selected, and analyzed post hoc\.

- •Fixed in advance\.The injection layerℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\+1\(mid\-stack rule from inference\-time steering\[[2](https://arxiv.org/html/2610.40998#bib.bib4)\]\) for every model, in both regimes\. No reported number comes from a per\-model layer search\.
- •Selected on a development split\.The injection operation \(Per\-sample Slerp vs\. Mean\-SV\) andα\\alphawere selected once on Qwen2\.5\-1\.5B with GSM8K, training on6,0006\{,\}000training examples and validating on the remaining training examples\. This gives Per\-sample Slerp atα=0\.1\\alpha=0\.1and Mean\-SV atα=0\.5\\alpha=0\.5, which are then reused for every cell of[Tables1](https://arxiv.org/html/2610.40998#S4.T1)and[3](https://arxiv.org/html/2610.40998#S4.T3)\. For RL,λ=0\.1\\lambda=0\.1is used throughout\.

##### Validation\-based layer selection for a new backbone\.

When adapting to a new architecture, we recommend a quarter\-epoch \(0\.250\.25\) sweep over candidate layers, using the last500500training examples as the validation set and picking the best\-validation layer\. The total cost is roughly one extra epoch of training spread over the candidates\. On Qwen2\.5\-1\.5B this recoversℓ⋆=15\\ell^\{\\star\}=15, which is also the test optimum, and on Llama\-3\.2\-3B it recoversℓ⋆=10\\ell^\{\\star\}=10, one of its two tied peaks, without access to test data\. The same protocol applies toα\\alpha\.

##### Practical heuristics\.

- •Layer\.Start mid\-stack; across both families we tested, effective layers lie in the middle of the network and neither shallow nor deep layers are ever optimal\. Use the quarter\-epoch sweep above if budget allows\.
- •Operation andα\\alpha\.With no tuning budget, Mean\-SV atα=0\.5\\alpha=0\.5is the safer choice because of its broadα\\alphaplateau\. Per\-sample Slerp has a higher peak atα=0\.1\\alpha=0\.1but degrades quickly beyondα=0\.3\\alpha=0\.3, falling below Plain atα=0\.5\\alpha=0\.5and collapsing atα=1\\alpha=1\([Fig\.2\(a\)](https://arxiv.org/html/2610.40998#S4.F2.sf1)\), so we advise keepingα≤0\.3\\alpha\\leq 0\.3for Slerp\.
- •Large off\-policy gaps\.With thinking\-mode teachers,∥v∥\\lVert v\\rVertis roughly3×3\\timeslarger \([Fig\.5](https://arxiv.org/html/2610.40998#A4.F5)\), and the raw\-norm default under\-injects; a unit\-normalized direction withα\\alphaaround11–3×3\\timesthe raw norm is a good starting point\.
- •When not to use the reference direction\.If the per\-sample directions are dispersed \(low meancos⁡\(δi,v\)\\cos\(\\delta\_\{i\},v\), e\.g\.0\.480\.48on MBPP versus0\.810\.81on GSM8K\), the reference direction likely encodes style rather than correctness; an outcome\-conditioned direction \(pass minus fail\) is preferable\.

## Appendix CMore Analysis on SFT Experiments

Here, we provide more ablation studies and further analysis\.

MBPP redesign with execution\-conditioned SV\.The reference\-based direction is a poor target on MBPP for two reasons\. First, the per\-sampleδi\\delta\_\{i\}alignment is only0\.4830\.483on MBPP versus0\.8080\.808on GSM8K, so the direction is dispersed\. Second, the cosine between the reference directionvrefv\_\{\\mathrm\{ref\}\}and the execution\-conditioned directionvexec=𝔼⁡\[h∣pass\]−𝔼⁡\[h∣fail\]v\_\{\\mathrm\{exec\}\}=\\mathbb\{E\}\[h\\mid\\text\{pass\}\]\-\\mathbb\{E\}\[h\\mid\\text\{fail\}\], which partitions the population by whether the bootstrap prediction passes the unit tests, is−0\.447\-0\.447atℓ⋆=15\\ell^\{\\star\}=15on Qwen2\.5\-1\.5B and−0\.437\-0\.437on Qwen2\.5\-7B\. In other words, reference\-SV actively pushes the model away from the test\-passing direction, and100%100\\%of passing bootstrap predictions differ syntactically from the reference solution, so the reference target carries arbitrary style on top of correctness\. Switching tovexecv\_\{\\mathrm\{exec\}\}atα=0\.3\\alpha=0\.3on Qwen2\.5\-7B raises MBPP accuracy from64\.80%64\.80\\%\(Plain SFT\) to66\.00%66\.00\\%, a\+1\.20\+1\.20pp gain that is the first non\-noise positive on MBPP in the entire study, withα=0\.1\\alpha=0\.1andα=0\.5\\alpha=0\.5giving smaller positives \(65\.20%65\.20\\%and65\.40%65\.40\\%respectively\)\.

### C\.1Training\-Time vs\. Inference\-Time SV

##### Setup\.

The inference\-time SV is deliberately the*same*direction used in training, namely the Mean\-SV reference\-minus\-prediction direction at layerℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\+1, added with coefficientα\\alpha\. We do not claim this is a good inference\-time steering vector; the purpose of the experiment is to separate the two uses of one direction\.[Table7](https://arxiv.org/html/2610.40998#A3.T7)extends panel \(c\) of[Fig\.2](https://arxiv.org/html/2610.40998#S4.F2)to six cells\.

Table 7:The same SV used during training versus at inference\. “SV at inference” injects the training direction \(α=0\.5\\alpha=0\.5\) into the Plain\-SFT checkpoint at test time\. Training with the SV helps in every cell; injecting it at inference hurts in every cell\.Model, datasetPlainSV during trainingSV at inferenceQwen2\.5\-1\.5B, GSM8K59\.1464\.14 \(\+5\.00\+5\.00\)44\.43 \(−14\.71\-14\.71\)Qwen2\.5\-1\.5B, MATH24\.6027\.20 \(\+2\.60\+2\.60\)19\.60 \(−5\.00\-5\.00\)Qwen2\.5\-7B, GSM8K76\.8880\.21 \(\+3\.33\+3\.33\)71\.87 \(−5\.01\-5\.01\)Qwen3\-4B, GSM8K80\.0684\.61 \(\+4\.55\+4\.55\)77\.03 \(−3\.03\-3\.03\)Qwen2\.5\-1\.5B, distill from Qwen3\-8B, GSM8K65\.8167\.55 \(\+1\.74\+1\.74\)63\.53 \(−2\.28\-2\.28\)Qwen2\.5\-1\.5B, distill from Qwen3\-8B, MATH21\.2031\.20 \(\+10\.00\+10\.00\)10\.80 \(−10\.40\-10\.40\)
##### Evidence for the offload account\.

\(i\)*Activation change\.*With the SV active during training, the change that training makes to the layer\-15 hidden state on training answer tokens has cosine≈−0\.05\\approx\-0\.05with the SV direction, whereas SV\-free training produces a clearly positive alignment\. \(ii\)*LoRA absorption check\.*A natural worry is that the adapter simply learns to undo the injection\. Plain and SV\-trained LoRA deltas have identical Frobenius norm to within1%1\\%across all three students \(1\.5B19\.3019\.29=1\.000\\frac\{19\.30\}\{19\.29\}=1\.000, 7B27\.8528\.19=0\.988\\frac\{27\.85\}\{28\.19\}=0\.988, Qwen3\-4B35\.3835\.55=0\.995\\frac\{35\.38\}\{35\.55\}=0\.995\), and the projection of the SV\-trained LoRA delta ontovvis≈10−6\\approx 10^\{\-6\}\. The adapter therefore learns corrections overwhelmingly orthogonal to the SV, so the gain comes from redirected gradients rather than from magnitude inflation or from cancelling the injection\. Together, these suggest that the SV supplies the component of the reference shift along its own direction \(largely style and format\) during training, so the parameters need not learn it; re\-adding it at inference then only perturbs the hidden states\.

##### A falsifiable prediction\.

If the offload account is right, a configuration whose vector steers more strongly at inference supplies the target component more completely during the training forward pass, so offloading should also be more complete\. Fixing the SV construction and varying the configuration \(injection layer or strength\), the inference\-time steering effect of each configuration on the frozen model should therefore correlate positively with how much of the target behavior the corresponding trained model offloads, measured by the alignment of its learned update with the SV\. We leave this causal test to future work\.

##### Implication for placement\.

The same account suggests why the best layer is model\-specific\. If the SV helps by offloading correctness\-irrelevant information, the most effective injection layer is the one where the vector offloads the most such information\. The best layer is then not purely a property of the architecture, since it also depends on how the training\-time vector is constructed, since what can be offloaded is set by the vector itself\.

### C\.2Full Variant Sweep on SV\-Augmented SFT

This appendix reports the full set of SV injection variants we tested on Qwen2\.5\-1\.5B with GSM8K atℓ⋆=15\\ell^\{\\star\}=15, in addition to the representative variants discussed in the main text\. All variants below apply some form of steering vector throughout training; methods that disable the SV hook or use no SV at all are excluded by construction\. The full numerical results are in[Table8](https://arxiv.org/html/2610.40998#A3.T8); we group the variants by which of the three design axes from Section[4\.1](https://arxiv.org/html/2610.40998#S4.SS1)they perturb, namely direction, placement, or operation\.

##### Direction\-axis variants\.

The variants below modify how the injected directionvℓ⋆v^\{\\ell^\{\\star\}\}is constructed\.

- •Cos\-weighted Mean\-SVreweights each sample’s contribution tovmeanv\_\{\\mathrm\{mean\}\}bycos⁡\(δi,vmean\)\\cos\(\\delta\_\{i\},v\_\{\\mathrm\{mean\}\}\), so that samples whose per\-sample direction is well aligned with the global mean count more\.
- •Calc\-wrong\-only Mean\-SVbuildsvmeanv\_\{\\mathrm\{mean\}\}from only the samples whose bootstrap prediction has a wrong final answer, on the hypothesis that wrong samples carry the most useful correction direction\.
- •Clean\-SV \(consolidated\)keeps only the samples withcos⁡\(δi,vmean\)\>0\.3\\cos\(\\delta\_\{i\},v\_\{\\mathrm\{mean\}\}\)\>0\.3and recomputes the mean from this clean subset, then injects the consolidated direction on every training sample\.
- •PCA PC1 SV\(also reported in the main text\) replacesvmeanv\_\{\\mathrm\{mean\}\}with the first principal component of the per\-sampleδi\\delta\_\{i\}matrix\.
- •Dynamic SVrecomputes the steering vector online during training\. Every200200optimizer steps we re\-run the training set through the current LoRA\-adapted model with and without GT teacher forcing, recompute both the GT\-mean and the Pred\-mean, and replace the injected direction with the freshly recomputedvmeanv\_\{\\mathrm\{mean\}\}\.

##### Placement\-axis variants\.

The variants below modify where the SV is injected, namely which token positions or which layers\.

- •Slerp first\-KKrestricts the Slerp hook to only the firstKKanswer tokens, varying the front\-loaded scope; we testedK=32K=32andK=64K=64\.
- •Confidence\-gated SVapplies the SV only to answer\-token positions where the next\-token entropy under the current model exceeds a threshold, on the hypothesis that low\-confidence tokens are where correction matters\.
- •Multi\-layer \(uniform\)\(also reported in the main text\) injects Mean\-SV atL∈\{10,15,20\}L\\in\\\{10,15,20\\\}with uniform per\-layerα=0\.50/0\.50/0\.50\\alpha=0\.50/0\.50/0\.50\.
- •Multi\-layer \(tuned\)injects at the same three layers withα=0\.10/0\.50/0\.10\\alpha=0\.10/0\.50/0\.10, putting more weight on the middle layer\.
- •All\-layer injectioninjects at every transformer block with the single\-layer coefficientα=0\.5\\alpha=0\.5\. Training collapses, and the model emits an empty or truncated final answer on about92%92\\%of GSM8K test problems and scores near zero\. We attribute this to the total intervention strength, since reusing a single\-layer coefficient at every layer multiplies the effective perturbation, rather than to multi\-layer injection itself\. The three\-layer schedules above train stably, but the directions at layers1010,1515, and2020have pairwise cosine≈0\.78\\approx 0\.78, so multi\-layer injection largely amplifies one shared direction rather than adding independent corrections\. Single\-layer injection is therefore the simpler and more robust default\.

##### Operation\-axis variants\.

The variants below modify how the SV combines with the existing hidden state, including the injection primitive, the sample\-level gating, and theα\\alphaschedule\.

- •Mean\-SV additiveis the standard mean\-direction baseline that uses the globalvmeanv\_\{\\mathrm\{mean\}\}with the additive primitive of Equation \([2](https://arxiv.org/html/2610.40998#S3.E2)\)\.
- •Per\-sample additive\(also reported in the main text\) keeps the per\-sampleδi\\delta\_\{i\}but injects it through the additive primitive rather than through Slerp\.
- •Negated Mean\-SV additiveflips the sign of the steering vector, subtractingα​vmean\\alpha v\_\{\\mathrm\{mean\}\}instead of adding, to test whether the gain depends on the direction of the SV rather than on the presence of any in\-place perturbation\.
- •Gated Slerpdrops samples whosecos⁡\(δi,vmean\)\\cos\(\\delta\_\{i\},v\_\{\\mathrm\{mean\}\}\)falls below a threshold and applies SV only on the remaining samples; we testedτ=0\.5\\tau=0\.5and the more aggressiveτ=0\.7\\tau=0\.7\.
- •α\\alphadecay schedulestarts atα=0\.5\\alpha=0\.5and linearly decays toα=0\\alpha=0over the epoch, asking whether SV is more useful in early training\.

Table 8:Full sweep of SV injection variants on Qwen2\.5\-1\.5B with GSM8K atℓ⋆=15\\ell^\{\\star\}=15\. Methods are sorted by accuracy\.Methodα\\alphaAccuracyΔ\\DeltaPlain LoRA \(no SV\)–59\.140Negated Mean\-SV additive \(sign\-flip control\)−0\.50\-0\.5054\.58−4\.56\-4\.56Random per\-sample Gaussian \(control\)0\.5054\.82−4\.32\-4\.32α\\alphadecay schedule \(linear0\.5→00\.5\\to 0\)0\.5→00\.5\\to 058\.07−1\.07\-1\.07Gated Slerp \(dropcos<0\.7\\cos<0\.7\)0\.1058\.45−0\.69\-0\.69Slerp first\-K=64K\{=\}64\(extended scope\)0\.1058\.61−0\.53\-0\.53Clean\-SV \(consolidated mean,cos\>0\.3\\cos\>0\.3subset\)0\.1058\.76−0\.38\-0\.38Confidence\-gated SV \(mask high\-conf tokens\)0\.5058\.98−0\.16\-0\.16PCA PC1 SV0\.5059\.29\+0\.15\+0\.15Gated Slerp \(dropcos<0\.5\\cos<0\.5\)0\.1059\.36\+0\.22\+0\.22Dynamic SV \(online recomputation every 200 steps\)0\.5060\.23\+1\.09\+1\.09Multi\-layer \(L=10/15/20, uniform\)0\.50/0\.50/0\.5060\.85\+1\.71\+1\.71Mean\-SV additive0\.5061\.11\+1\.97\+1\.97Slerp first\-K=32K\{=\}32\(front\-loaded\)0\.1062\.02\+2\.88\+2\.88Cos\-weighted Mean\-SV additive0\.5062\.40\+3\.26\+3\.26Calc\-wrong\-only Mean\-SV additive0\.5062\.55\+3\.41\+3\.41Multi\-layer \(L=10/15/20, tuned\)0\.10/0\.50/0\.1062\.62\+3\.48\+3\.48Per\-sample additive0\.5062\.93\+3\.79\+3\.79Per\-sample Slerp \(full answer\)0\.1064\.14\+5\.00\\mathbf\{\+5\.00\}
##### Discussion of the full sweep\.

The full sweep reinforces the four conclusions drawn in the main text and adds three further ones\. None of the direction\-construction variants we tested exceeds plain Mean\-SV by a meaningful margin, which we read as a consequence of the per\-sampleδi\\delta\_\{i\}’s being already99\.9%99\.9\\%aligned with the global mean on GSM8K \(meancos=0\.808\\cos=0\.808\); when the per\-sample directions all point the same way, there is essentially nothing for a filter or a cleaner aggregation to recover\. Sample\-level and token\-level gating \(Gated Slerp at both thresholds, Confidence\-gated SV\) all under\-perform the no\-gating baseline, which we read as evidence that selectively skipping samples removes useful gradient signal more than it removes noise\. Scope restrictions \(Slerp first\-KKatK=32K=32orK=64K=64\) hurt at everyKKwe tried, confirming that the SV signal is broadly distributed across the answer span rather than concentrated at the prefix\. The tuned multi\-layer schedule outperforms the uniform multi\-layer schedule by a substantial margin but still falls short of single\-layer Per\-sample Slerp, which is consistent with the main\-text reading that multi\-layer injection demands careful per\-layerα\\alphatuning rather than reusing a sharedα\\alphaacross layers\. Finally, the negated Mean\-SV additive control matches the random\-Gaussian control at roughly55pp below Plain, ruling out the alternative that the SV gain comes from any in\-place hidden\-state perturbation rather than from the specific GT\-vs\-Pred direction\.

## Appendix DMore Analysis on Distillation Experiments

This appendix collects the additional distillation analyses we deferred from the main text\. All results are on the same3×4×43\\times 4\\times 4grid reported in[Table3](https://arxiv.org/html/2610.40998#S4.T3)\.

##### Full off\-policy mitigation sweep across all 10 cells\.

The main text reports the four Qwen3\-4B distillation cells of[Figs\.3\(a\)](https://arxiv.org/html/2610.40998#S4.F3.sf1)and[3\(b\)](https://arxiv.org/html/2610.40998#S4.F3.sf2)\.[Figure4](https://arxiv.org/html/2610.40998#A4.F4)extends the test to all88distillation cells of the headline grid \(two students×\\timesfour teachers\) plus the two SFT cells, in both the log\-probability metricΔ​NLLsum\\Delta\\mathrm\{NLL\}\_\{\\mathrm\{sum\}\}and the linear\-probability metricΔ​Psum\\Delta P\_\{\\mathrm\{sum\}\}\. The protocol matches the main text: we apply the trained\-time SV hook \(additive,α=0\.5\\alpha=0\.5,ℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\{\+\}1\) to the base model under teacher forcing on500500sampled training prompts and report the per\-prompt sums and the fraction of prompts on whichΔ​NLLsum\\Delta\\mathrm\{NLL\}\_\{\\mathrm\{sum\}\}is positive\. The two cells without off\-policy mitigation \(Qwen2\.5\-1\.5B distilled from a non\-thinking teacher\) are exactly the cells where the headline accuracy table shows no SV\-distill gain\.

\(a\)Δ​NLLsum\\Delta\\mathrm\{NLL\}\_\{\\mathrm\{sum\}\}across all 10 cells, sorted descending\. Numbers in parentheses arefrac⁡\(Δ​NLLsum\>0\)\\mathrm\{frac\}\(\\Delta\\mathrm\{NLL\}\_\{\\mathrm\{sum\}\}\>0\)\.![Refer to caption](https://arxiv.org/html/2610.40998v1/fig_distill_offpolicy_full_psum.png)\(b\)Δ​Psum\\Delta P\_\{\\mathrm\{sum\}\}on the same 10 cells in the same row order as panel \(a\), in linear probability units\.

Figure 4:Full off\-policy mitigation sweep on all1010cells \(eight distillation cells of the headline grid plus the two SFT cells\), in two complementary scales\. Eight of ten cells are positive atfrac⁡\(Δ​NLLsum\>0\)≥73%\\mathrm\{frac\}\(\\Delta\\mathrm\{NLL\}\_\{\\mathrm\{sum\}\}\>0\)\\geq 73\\%; the two near\-zero/negative cells in panel \(a\) are exactly the Qwen2\.5\-1\.5B×\\timesnon\-thinking\-teacher distillation cells that also show no SV\-distill accuracy gain in[Table3](https://arxiv.org/html/2610.40998#S4.T3)\. The linear\-probability metric in panel \(b\) is non\-negative on every cell because the per\-token probability mass moved onto label tokens is positive even on the cells where token\-level NLL gains and losses cancel\.
##### SV direction statistics by teacher\.

[Figure5](https://arxiv.org/html/2610.40998#A4.F5)reports three statistics forv\(ℓ\)v^\{\(\\ell\)\}atℓ⋆=15\\ell^\{\\star\}=15averaged over the1212cells of each teacher, namely the norm‖v‖\\\|v\\\|, the mean alignmentcos⁡\(δi,v\)\\cos\(\\delta\_\{i\},v\), and Cohen’sddfor theprojGT−projPred\\mathrm\{proj\}\_\{\\mathrm\{GT\}\}\-\\mathrm\{proj\}\_\{\\mathrm\{Pred\}\}split\. Both thinking teachers \(Qwen3 thinking and Gemma thinking\) produce roughly3×3\\timesthe norm,∼\\sim2×2\\timesthe alignment, and∼\\sim2×2\\timesthe Cohen’sddof either non\-thinking teacher, and the two non\-thinking teachers are statistically indistinguishable on all three quantities despite being from different families\. The two thinking teachers, by contrast, agree in magnitude on these statistics but produce different downstream gains, which is consistent with the cosine\-similarity finding in the main text that their directions are not interchangeable\.

\(a\)Direction norm‖v\(ℓ\)‖\\\|v^\{\(\\ell\)\}\\\|atℓ⋆=15\\ell^\{\\star\}=15, averaged over the 12 \(student, dataset\) cells of each teacher\.\(b\)Mean per\-sample alignmentcos⁡\(δi,v\(ℓ\)\)\\cos\(\\delta\_\{i\},v^\{\(\\ell\)\}\)atℓ⋆=15\\ell^\{\\star\}=15, averaged over the 12 cells of each teacher\.\(c\)Cohen’sddfor the projection of the GT and Pred populations ontov\(ℓ\)v^\{\(\\ell\)\}atℓ⋆=15\\ell^\{\\star\}=15, averaged over the 12 cells of each teacher\.
Figure 5:Three statistics of the SV direction atℓ⋆=15\\ell^\{\\star\}=15broken down by teacher mode and averaged over the 12 \(student, dataset\) cells of each teacher, namely the direction norm in panel \(a\), the per\-sample alignment in panel \(b\), and Cohen’sddfor the GT\-versus\-Pred projection in panel \(c\)\. Both thinking teachers dominate the two non\-thinking teachers on every axis, while the two non\-thinking teachers \(different families\) are statistically indistinguishable on all three quantities\.
##### Normalized SV vs\. unnormalized SV\.

Both Mean\-SV and Per\-sample Slerp inject the raw directionvvat a fixed coefficientα\\alpha, so the actual injection magnitude isα⋅‖v‖\\alpha\\cdot\\\|v\\\|and confounds two factors, namely how strongly we steer and how large the underlying direction happens to be\. Because thinking teachers produce SV directions with substantially larger‖v‖\\\|v\\\|than non\-thinking teachers \([Fig\.5\(a\)](https://arxiv.org/html/2610.40998#A4.F5.sf1)\), the same nominalα\\alphais not comparable across teachers\. We therefore try*Normalized SV*, which replacesvvwith the unit\-normalizedv^=v/‖v‖\\hat\{v\}=v/\\\|v\\\|, so thatα\\alphabecomes the injection magnitude in unit\-norm space\. On the Qwen3\-thinking×\\timesMATH cell that produced the largest gain in[Table3](https://arxiv.org/html/2610.40998#S4.T3), Normalized SV at a tunedα=15\\alpha=15reaches a\+21\.40\+21\.40pp gain over Plain on Qwen2\.5\-1\.5B, up from\+12\.40\+12\.40pp under Per\-sample Slerp atα=0\.1\\alpha=0\.1\. The full sweep across both Qwen2\.5\-1\.5B and Qwen2\.5\-7B is shown in[Fig\.6](https://arxiv.org/html/2610.40998#A4.F6): theα\\alpha\-curve is monotonic and concave on both students, with the optimum in the rangeα∈\[10,30\]\\alpha\\in\[10,30\], which corresponds to roughly11to33times the raw norm‖vraw‖\\\|v\_\{\\mathrm\{raw\}\}\\\|\. Normalized SV at this tunedα\\alphabeats Per\-sample Slerp atα=0\.1\\alpha=0\.1on every think\-teacher MATH cell we tested, so the headline number is bottlenecked by under\-injection of the raw direction rather than by the direction itself\.

\(a\)Normalized Mean\-SVα\\alpha\-sweep on Qwen2\.5\-1\.5B distilled from the Qwen3 thinking teacher on MATH\.\(b\)Normalized Mean\-SVα\\alpha\-sweep on Qwen2\.5\-7B distilled from the Qwen3 thinking teacher on MATH\.
Figure 6:Accuracy under a normalized Mean\-SVα\\alpha\-sweep on the Qwen3\-thinking\-teacher MATH cells, where the steering direction is unit\-normalized so thatα\\alphais interpretable as the literal injection magnitude rather than a Slerp rotation fraction\. Plain SFT is shown as a dashed grey line and the Per\-sample Slerpα=0\.1\\alpha=0\.1result from[Table3](https://arxiv.org/html/2610.40998#S4.T3)as a dotted orange line\. Both panels show that a tunedα∈\[10,30\]\\alpha\\in\[10,30\]on the normalized direction outperforms Per\-sample Slerp atα=0\.1\\alpha=0\.1\.
##### Cluster separation as a predictor of SV gain\.

The story from[Section4\.2\.3](https://arxiv.org/html/2610.40998#S4.SS2.SSS3)that “higher Cohen’sddimplies more SV gain” breaks down on distillation\. The three cells with the highest Cohen’sddin the distillation grid are all Qwen3\-thinking×\\timesMBPP cells, withd∈\{5\.85,6\.77,7\.10\}d\\in\\\{5\.85,6\.77,7\.10\\\}, but the SV gain on those same cells is in\{−1\.80,0\.00,−0\.60\}\\\{\-1\.80,0\.00,\-0\.60\\\}\.[Table9](https://arxiv.org/html/2610.40998#A4.T9)reports the top\-six cells by Cohen’sddalongside their SV gain\. A highddseparates GT from Pred populations well, but only on the training distribution\. For the gain to transfer to test time, the direction must encode signal that survives on\-policy decoding, not just the GT\-vs\-Pred gap on training prompts\. On thinking\-teacher MBPP, the dominant component of the GT\-Pred gap is verbose reasoning prefix versus terse code, which is a surface artifact rather than a transferable correctness signal\.

Table 9:The six \(student, teacher, dataset\) cells with the largest Cohen’sddin the distillation grid, alongside the direction norm‖v‖\\\|v\\\|, the per\-sample alignmentcos\\cos, and the Per\-sample Slerp gain over Plain\. A high Cohen’sddon the training distribution does not imply a positive gain at test time, as illustrated by the three top\-ranked MBPP cells whose gain is at most zero\. The cell with the largest gain in this subset is shown inbold\.Cell \(student×\\timesteacher×\\timesdataset\)‖v‖\\\|v\\\|mean cosCohen’sddSVΔ\\Delta\(pp\)7B×\\timesQwen3\-thinking×\\timesMBPP22\.390\.8017\.10−1\.80\-1\.80Qwen3\-4B×\\timesQwen3\-thinking×\\timesMBPP12\.930\.8146\.770\.000\.001\.5B×\\timesQwen3\-thinking×\\timesMBPP18\.330\.8355\.85−0\.60\-0\.60Qwen3\-4B×\\timesQwen3\-thinking×\\timesGSM8K5\.510\.7835\.84−2\.20\-2\.207B×\\timesQwen3\-thinking×\\timesGSM8K10\.570\.7785\.79\+3\.94\+3\.947B×\\timesQwen3\-thinking×\\timesMATH11\.230\.7494\.76\+19\.80\\mathbf\{\+19\.80\}
##### Why Llama\-3\.2\-3B underperforms under the canonical recipe\.

The cross\-family Llama\-3\.2\-3B student gives essentially neutral mean Per\-sample SlerpΔ\\Deltaacross teachers in[Table3](https://arxiv.org/html/2610.40998#S4.T3), in contrast to the consistently positive Qwen students\. This is the same layer\-choice issue that the SFT layer sweep in panel \(b\) of[Fig\.2](https://arxiv.org/html/2610.40998#S4.F2)already exposes, and not a failure specific to distillation\. The Qwen\-default rule setsℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\+1on every Qwen student and lands on a representation regime where the GT\-Pred direction is dense and well\-aligned, but the same fractional depth on Llama\-3\.2\-3B’s2828blocks \(ℓ⋆=15\\ell^\{\\star\}=15\) sits in a valley between two off\-center optima\. On Llama with GSM8K, Plain SFT is62\.09%62\.09\\%, the canonicalℓ⋆=15\\ell^\{\\star\}=15drops the model by−0\.30\-0\.30pp to61\.79%61\.79\\%, whileℓ⋆=10\\ell^\{\\star\}=10andℓ⋆=19\\ell^\{\\star\}=19both lift Llama by\+1\.22\+1\.22pp to63\.31%63\.31\\%\. A tuned distill layer ofℓ⋆=13\\ell^\{\\star\}=13on the Gemma\-thinking teacher reaches\+9\.0\+9\.0pp over the canonicalℓ⋆=15\\ell^\{\\star\}=15on the same cell\. The optimal SV injection layer therefore depends jointly on student family, dataset, and teacher mode, and is not a fixed fraction of network depth\. Reusing a single\-layer index across architectures is a confound that systematically suppresses Llama’s true potential rather than evidence that SV transfer fails\.

## Appendix EAnalysis on Reinforcement Learning Experiments

This appendix collects the additional RL analyses that we deferred from the main text, namely two ablations that isolate the contribution of the SV direction and of the SV\-Rescue advantage modification, and a layer\-wise correlation study that supports the canonical injection layer\.

##### Ablation 1: SV direction vs random direction\.

We replace the precomputed correctness direction in SV\-Rescue with a same\-norm random Gaussian vector, holding every other hyperparameter fixed\.[Table10](https://arxiv.org/html/2610.40998#A5.T10)shows that the precomputed correctness direction beats the random control on both students\. The SV\-Rescue gain therefore depends on the direction being correctness\-aligned and is not an artifact of shifting the advantage of wrong rollouts per se\.

Table 10:SV\-Rescue with the precomputed correctness direction \(ours\) vs a same\-norm random Gaussian direction on GSM8K\. The SV direction is what carries the rescue gain\.StudentRandom directionSV\-Rescue \(ours\)Qwen2\.5\-1\.5B77\.5478\.35Qwen3\-4B90\.3591\.04
##### Ablation 2: SV\-Explore without SV\-Rescue\.

SV\-Explore stacks SV\-Rescue on top of a per\-prompt temperature schedule\. To check that SV\-Rescue is not redundant once exploration is added, we run SV\-Explore on GSM8K with the SV\-Rescue advantage disabled \(temperature schedule only\)\.[Table11](https://arxiv.org/html/2610.40998#A5.T11)shows that removing SV\-Rescue drops accuracy on both students, so the headline SV\-Explore gain in[Table6](https://arxiv.org/html/2610.40998#S5.T6)is the joint contribution of SV\-Rescue and the exploration schedule rather than the schedule alone\.

Table 11:SV\-Explore \(full\) vs SV\-Explore with the SV\-Rescue advantage disabled \(temperature schedule only\) on GSM8K\. SV\-Rescue is not redundant once exploration is added\.StudentTemperature schedule onlySV\-Explore \(full\)Qwen2\.5\-1\.5B78\.7279\.18Qwen3\-4B91\.7991\.99
##### Combination with entropy\-aware shaping\.

Entropy\-aware advantage shaping\[[9](https://arxiv.org/html/2610.40998#bib.bib40)\]also assigns learning signal to zero\-variance groups\. Over three seeds on Qwen2\.5\-1\.5B with GSM8K, GRPO reaches77\.60±0\.5177\.60\\pm 0\.51, GRPO with SV\-Rescue78\.13±0\.9378\.13\\pm 0\.93, GRPO with entropy\-aware shaping78\.64±0\.8078\.64\\pm 0\.80, and GRPO with both78\.87±0\.5978\.87\\pm 0\.59\. Entropy\-aware shaping alone is stronger than SV\-Rescue alone, while stacking SV\-Rescue on top of it gives the best result\. SV\-Rescue is therefore best viewed as a composable auxiliary signal rather than a standalone solution to the zero\-variance problem\.

##### Layer\-wise SV signal\.

SV\-Rescue usesℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\+1as the layer at which the SV alignment score is read\. To check whether this is the best layer for the RL setting, we compute the per\-rollout Pearson correlation between the SV alignment score at every transformer block and the binary verifier reward on a20002000\-rollout GSM8K hold\-out, both before and after RL \([Fig\.7](https://arxiv.org/html/2610.40998#A5.F7)\)\. The peak correlation lies slightly aboveℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\+1, near2/32/3depth, although the canonicalL/2\+1L/2\+1retains8080\-90%90\\%of the peak\. To test whether the higher hold\-out correlation translates into stronger RL gain, we re\-trained Qwen2\.5\-1\.5B GSM8K SV\-Rescue at the peak\-correlation layer; the peak\-layer run reaches78\.66%78\.66\\%, marginally above the canonical baseline of78\.35%78\.35\\%\. Layer choice is therefore a small additional lever rather than a major one, and the canonicalL/2\+1L/2\+1default works well across students\.

![Refer to caption](https://arxiv.org/html/2610.40998v1/fig_rl_layer_correlation.png)Figure 7:Per\-rollout Pearson correlation between the SV alignment score at each transformer block and the binary verifier reward, on a20002000\-rollout GSM8K hold\-out, both before RL \(init\) and after RL \(SV\-Rescue\)\. The vertical dashed line marks the canonical injection layerℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\+1used by SV\-Rescue and the dotted line marks the peak\-correlation layer\. The correlation peaks slightly aboveL/2\+1L/2\+1on both students, but the canonicalL/2\+1L/2\+1retains8080\-90%90\\%of the peak\.

## Appendix FAdditional Baselines

All baselines below use the same model \(Qwen2\.5\-1\.5B\), data, LoRA configuration, one\-epoch budget, and evaluation protocol as our method\.

##### Supervised baselines\.

[Table12](https://arxiv.org/html/2610.40998#A6.T12)compares the SV with two baselines that also target the off\-policy mismatch of SFT\. DFT\[[31](https://arxiv.org/html/2610.40998#bib.bib50)\]acts at the level of the objective, whereas the SV acts on the representation\. With its original hyperparameters, DFT barely changes Plain under our LoRA setting; re\-tuned for LoRA, it reaches61\.09%61\.09\\%, still below the SV\. Adding the SV on top of DFT yields\+3\.76\+3\.76pp over DFT, so the representation\-level correction is not subsumed by the objective\-level one\. A per\-token hidden\-state matching loss instead hurts accuracy, so explicitly pulling hidden states toward the reference does not replace injecting the direction\.

Table 12:SFT\-side baselines on Qwen2\.5\-1\.5B with GSM8K \(mean±\\pmstd over three seeds unless marked†\)\.MethodAccuracyΔ\\Deltavs\. PlainPlain SFT58\.81±0\.3758\.81\\pm 0\.3700DFT \(original hyperparameters\)58\.88±0\.8058\.88\\pm 0\.80\+0\.07\+0\.07DFT \(re\-tuned for LoRA\)61\.09†61\.09^\{\\dagger\}\+2\.28\+2\.28Hidden\-state distillation53\.68†53\.68^\{\\dagger\}−5\.13\-5\.13SV \(Per\-sample Slerp\)62\.75±0\.50\\mathbf\{62\.75\\pm 0\.50\}\+3\.94\\mathbf\{\+3\.94\}DFT \(original\) \+ SV62\.64±0\.7362\.64\\pm 0\.73\+3\.83\+3\.83
##### DFT\[[31](https://arxiv.org/html/2610.40998#bib.bib50)\]\.

DFT rescales each token’s cross\-entropy loss by the token’s detached probability under the current model, which corrects the off\-policy mismatch at the level of the objective\. We first used the hyperparameters recommended in the original paper, which were developed for full\-parameter fine\-tuning on different models and datasets and transfer poorly to our LoRA setting \(58\.88%58\.88\\%\)\. We then re\-searched DFT’s hyperparameters under our setting, reaching61\.09%61\.09\\%\.*DFT \+ SV*uses DFT with its original hyperparameters, applying our Per\-sample Slerp hook in the forward pass and the DFT loss in the objective\.

##### Hidden\-state distillation\.

This baseline asks whether explicitly pulling hidden states toward the reference with an auxiliary loss can replace injecting the direction; it reduces accuracy to53\.68%53\.68\\%\.

##### Entropy\-aware advantage shaping\[[9](https://arxiv.org/html/2610.40998#bib.bib40)\]\.

This method assigns token\-level advantages to zero\-variance prompts based on token entropy, so all\-wrong groups also contribute gradient\.*Entropy\-aware \+ SV\-Rescue*adds the SV\-Rescue term of[Eq\.12](https://arxiv.org/html/2610.40998#S5.E12)on top of the entropy\-aware advantages\.

##### Relation of the three\-seed RL runs to[Table6](https://arxiv.org/html/2610.40998#S5.T6)\.

The three\-seed GRPO runs are an independent rerun with the same configuration; their means \(77\.6077\.60for GRPO and78\.1378\.13for SV\-Rescue\) differ slightly from the single\-seed values in[Table6](https://arxiv.org/html/2610.40998#S5.T6)\(77\.7277\.72and78\.3578\.35\)\.

## Appendix GDiscussion

Our results reveal a consistent set of conditions under which steering vectors provide a useful training signal, as well as clear boundaries where they do not\.

##### When does SV injection help?

The largest and most reliable gains appear on reasoning tasks \(GSM8K, MATH\) where the off\-policy gap between the model’s native style and the reference completion is large and directionally coherent\. Per\-sample Slerp exploits per\-instance heterogeneity in this gap and outperforms the global Mean\-SV direction on most cells, while Mean\-SV remains competitive on the rest\. In distillation, thinking\-mode teachers amplify the benefit because their long reasoning traces widen the stylistic contrast and give the SV a richer correction signal\. In RL, SV\-Rescue recovers gradient on all\-wrong groups that would otherwise contribute nothing to learning, and composes cleanly with orthogonal rollout\-side mechanisms and with entropy\-aware advantage shaping\. Its standalone gain is small and below entropy\-aware shaping alone, so we view it as an auxiliary signal rather than a standalone remedy\.

##### When does SV injection fail?

The two failure modes we observe line up with two of the design axes from Section[4\.1](https://arxiv.org/html/2610.40998#S4.SS1)\. First, code generation breaks the*direction*, where the reference\-based SV direction is anti\-correlated with the execution\-conditioned correctness direction, so the SV actively pushes the model away from test\-passing behavior\. Second, cross\-family layer transfer breaks the*placement*, where the canonicalℓ⋆=L/2\+1\\ell^\{\\star\}=L/2\+1that works for Qwen lands in a valley for Llama\-3\.2\-3B and requires a per\-architecture layer search\.

##### Training\-time SV is internalized rather than memorized\.

A separate observation that does not fit cleanly into the three\-axis decomposition is that a model trained with Per\-sample Slerp performs best at inference*without*SV, even though it was trained with SV active at every answer\-token position\. This indicates that the SV does not act as a permanent crutch the model leans on at decode time\. Instead, the SV redirects the gradient during training so that the corrected behavior is internalized into the parameters themselves, and the resulting checkpoint generalizes beyond the steered training distribution rather than being tied to it\.[Section4\.2\.3](https://arxiv.org/html/2610.40998#S4.SS2.SSS3)and[SectionC\.1](https://arxiv.org/html/2610.40998#A3.SS1)analyze this shift and show that the learned update is nearly orthogonal to the SV, which is consistent with the SV offloading a correctness\-irrelevant component during training\. A causal test of this offload account is the main open question we leave to future work\.

##### Future directions: finer\-grained steering vectors\.

The directions we use in this work are coarse correctness summaries, namely the GT\-vs\-Pred displacement in SFT and the correct\-vs\-wrong contrast in RL\. A natural next step is to partition the contrast populations more finely, for example, by execution outcome on code, by reasoning style, by task subtype, or by specific failure mode\. Each finer direction could be activated selectively to shape*what*the model learns at a per\-skill or per\-mistake level rather than uniformly correcting toward a single reference style\. The same machinery could also control the*learning dynamics*themselves, varying which directions fire and at what strength as training progresses, rather than fixing a singleα\\alphaand direction throughout\. We see steering vectors as a general controller of post\-training learning dynamics, of which the two regimes in this paper are only the first instances\.

##### Limitations\.

Our study has several limitations\. First, all experiments use student models at or below 7B parameters; whether the findings transfer to larger scales remains open\. Second, the best injection layer is model\-specific; the fixed mid\-stack rule is a reasonable default and the quarter\-epoch validation sweep of[AppendixB](https://arxiv.org/html/2610.40998#A2)finds a better layer cheaply, but there is no principled criterion that predicts it without training\. Third, our baseline comparison does not cover on\-policy distillation, scheduled sampling, or gold\-rollout repair, and in RL, SV\-Rescue alone is weaker than entropy\-aware shaping alone\.

## Appendix HBroader Impact

Our work studies activation\-steering as a training signal for LLM post\-training\. The positive impact is a potentially better way of post\-training\. The negative impact is the general dual\-use concern that any improvement to LLM reasoning capability can, in principle, be applied to harmful tasks\. Our methods do not introduce a new direct path to such misuse beyond what already exists for the public base models we fine\-tune\. The SV directions we use encode model\-vs\-reference correctness rather than safety\-relevant attributes such as refusal or compliance, so the technique is not a method for bypassing safety alignment\.

## NeurIPS Paper Checklist

1. 1\.Claims
2. Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
3. Answer:\[Yes\]
4. Justification: The abstract and Section 1 accurately reflect the contribution\.
5. Guidelines: - •The answer\[N/A\]means that the abstract and introduction do not include the claims made in the paper\. - •The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations\. A\[No\]or\[N/A\]answer to this question will not be perceived well by the reviewers\. - •The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings\. - •It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper\.
6. 2\.Limitations
7. Question: Does the paper discuss the limitations of the work performed by the authors?
8. Answer:\[Yes\]
9. Justification: We show the negative results in the experimental part and also discuss more limitations in the appendix discussion part\.
10. Guidelines: - •The answer\[N/A\]means that the paper has no limitation while the answer\[No\]means that the paper has limitations, but those are not discussed in the paper\. - •The authors are encouraged to create a separate “Limitations” section in their paper\. - •The paper should point out any strong assumptions and how robust the results are to violations of these assumptions \(e\.g\., independence assumptions, noiseless settings, model well\-specification, asymptotic approximations only holding locally\)\. The authors should reflect on how these assumptions might be violated in practice and what the implications would be\. - •The authors should reflect on the scope of the claims made, e\.g\., if the approach was only tested on a few datasets or with a few runs\. In general, empirical results often depend on implicit assumptions, which should be articulated\. - •The authors should reflect on the factors that influence the performance of the approach\. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting\. Or a speech\-to\-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon\. - •The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size\. - •If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness\. - •While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper\. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community\. Reviewers will be specifically instructed to not penalize honesty concerning limitations\.
11. 3\.Theory assumptions and proofs
12. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete \(and correct\) proof?
13. Answer:\[N/A\]
14. Justification: No theory
15. Guidelines: - •The answer\[N/A\]means that the paper does not include theoretical results\. - •All the theorems, formulas, and proofs in the paper should be numbered and cross\-referenced\. - •All assumptions should be clearly stated or referenced in the statement of any theorems\. - •The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition\. - •Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material\. - •Theorems and Lemmas that the proof relies upon should be properly referenced\.
16. 4\.Experimental result reproducibility
17. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper \(regardless of whether the code and data are provided or not\)?
18. Answer:\[Yes\]
19. Justification: We provide the information in the experimental part\.
20. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •If the paper includes experiments, a\[No\]answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not\. - •If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable\. - •Depending on the contribution, reproducibility can be accomplished in various ways\. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model\. In general\. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model \(e\.g\., in the case of a large language model\), releasing of a model checkpoint, or other means that are appropriate to the research performed\. - •While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution\. For example 1. \(a\)If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm\. 2. \(b\)If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully\. 3. \(c\)If the contribution is a new model \(e\.g\., a large language model\), then there should either be a way to access this model for reproducing the results or a way to reproduce the model \(e\.g\., with an open\-source dataset or instructions for how to construct the dataset\)\. 4. \(d\)We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility\. In the case of closed\-source models, it may be that access to the model is limited in some way \(e\.g\., to registered users\), but it should be possible for other researchers to have some path to reproducing or verifying the results\.
21. 5\.Open access to data and code
22. Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
23. Answer:\[Yes\]
24. Justification: The code link is given in the footnote on the first page\.
25. Guidelines: - •The answer\[N/A\]means that paper does not include experiments requiring code\. - • - •While we encourage the release of code and data, we understand that this might not be possible, so\[No\]is an acceptable answer\. Papers cannot be rejected simply for not including code, unless this is central to the contribution \(e\.g\., for a new open\-source benchmark\)\. - •The instructions should contain the exact command and environment needed to run to reproduce the results\. See the NeurIPS code and data submission guidelines \([https://neurips\.cc/public/guides/CodeSubmissionPolicy](https://neurips.cc/public/guides/CodeSubmissionPolicy)\) for more details\. - •The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc\. - •The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines\. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why\. - •At submission time, to preserve anonymity, the authors should release anonymized versions \(if applicable\)\. - •Providing as much information as possible in supplemental material \(appended to the paper\) is recommended, but including URLs to data and code is permitted\.
26. 6\.Experimental setting/details
27. Question: Does the paper specify all the training and test details \(e\.g\., data splits, hyperparameters, how they were chosen, type of optimizer\) necessary to understand the results?
28. Answer:\[Yes\]
29. Justification: In the experimental part
30. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them\. - •The full details can be provided either with the code, in appendix, or as supplemental material\.
31. 7\.Experiment statistical significance
32. Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
33. Answer:\[Yes\]
34. Justification: Section[4\.4](https://arxiv.org/html/2610.40998#S4.SS4)and Section[5\.2\.3](https://arxiv.org/html/2610.40998#S5.SS2.SSS3)report mean±\\pmstandard deviation over three training seeds for representative supervised, distillation, code, and RL settings\. The full result grids in Tables[1](https://arxiv.org/html/2610.40998#S4.T1)–[6](https://arxiv.org/html/2610.40998#S5.T6)are single\-seed, which we state in the limitations\.
35. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The authors should answer\[Yes\]if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper\. - •The factors of variability that the error bars are capturing should be clearly stated \(for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions\)\. - •The method for calculating the error bars should be explained \(closed form formula, call to a library function, bootstrap, etc\.\) - •The assumptions made should be given \(e\.g\., Normally distributed errors\)\. - •It should be clear whether the error bar is the standard deviation or the standard error of the mean\. - •It is OK to report 1\-sigma error bars, but one should state it\. The authors should preferably report a 2\-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified\. - •For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range \(e\.g\., negative error rates\)\. - •If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text\.
36. 8\.Experiments compute resources
37. Question: For each experiment, does the paper provide sufficient information on the computer resources \(type of compute workers, memory, time of execution\) needed to reproduce the experiments?
38. Answer:\[Yes\]
39. Justification: in the SFT experimental part\. We use the same setting for every part\.
40. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage\. - •The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute\. - •The paper should disclose whether the full research project required more compute than the experiments reported in the paper \(e\.g\., preliminary or failed experiments that didn’t make it into the paper\)\.
41. 9\.Code of ethics
43. Answer:\[Yes\]
44. Justification:\[N/A\]
45. Guidelines: - •The answer\[N/A\]means that the authors have not reviewed the NeurIPS Code of Ethics\. - •If the authors answer\[No\], they should explain the special circumstances that require a deviation from the Code of Ethics\. - •The authors should make sure to preserve anonymity \(e\.g\., if there is a special consideration due to laws or regulations in their jurisdiction\)\.
46. 10\.Broader impacts
47. Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
48. Answer:\[Yes\]
49. Justification: In the appendix
50. Guidelines: - •The answer\[N/A\]means that there is no societal impact of the work performed\. - •If the authors answer\[N/A\]or\[No\], they should explain why their work has no societal impact or why the paper does not address societal impact\. - •Examples of negative societal impacts include potential malicious or unintended uses \(e\.g\., disinformation, generating fake profiles, surveillance\), fairness considerations \(e\.g\., deployment of technologies that could make decisions that unfairly impact specific groups\), privacy considerations, and security considerations\. - •The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments\. However, if there is a direct path to any negative applications, the authors should point it out\. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation\. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster\. - •The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from \(intentional or unintentional\) misuse of the technology\. - •If there are negative societal impacts, the authors could also discuss possible mitigation strategies \(e\.g\., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML\)\.
51. 11\.Safeguards
52. Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse \(e\.g\., pre\-trained language models, image generators, or scraped datasets\)?
53. Answer:\[N/A\]
54. Justification: This paper’s topic seems unrelated to this\.
55. Guidelines: - •The answer\[N/A\]means that the paper poses no such risks\. - •Released models that have a high risk for misuse or dual\-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters\. - •Datasets that have been scraped from the Internet could pose safety risks\. The authors should describe how they avoided releasing unsafe images\. - •We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort\.
56. 12\.Licenses for existing assets
57. Question: Are the creators or original owners of assets \(e\.g\., code, data, models\), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
58. Answer:\[Yes\]
59. Justification: All base models and datasets we use are publicly released and cited in the main text\.
60. Guidelines: - •The answer\[N/A\]means that the paper does not use existing assets\. - •The authors should cite the original paper that produced the code package or dataset\. - •The authors should state which version of the asset is used and, if possible, include a URL\. - •The name of the license \(e\.g\., CC\-BY 4\.0\) should be included for each asset\. - •For scraped data from a particular source \(e\.g\., website\), the copyright and terms of service of that source should be provided\. - •If assets are released, the license, copyright information, and terms of use in the package should be provided\. For popular datasets,[paperswithcode\.com/datasets](https://paperswithcode.com/datasets)has curated licenses for some datasets\. Their licensing guide can help determine the license of a dataset\. - •For existing datasets that are re\-packaged, both the original license and the license of the derived asset \(if it has changed\) should be provided\. - •If this information is not available online, the authors are encouraged to reach out to the asset’s creators\.
61. 13\.New assets
62. Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
63. Answer:\[Yes\]
64. Justification: We write a Readme in the code so that everyone can easy to use,
65. Guidelines: - •The answer\[N/A\]means that the paper does not release new assets\. - •Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates\. This includes details about training, license, limitations, etc\. - •The paper should discuss whether and how consent was obtained from people whose asset is used\. - •At submission time, remember to anonymize your assets \(if applicable\)\. You can either create an anonymized URL or include an anonymized zip file\.
66. 14\.Crowdsourcing and research with human subjects
67. Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation \(if any\)?
68. Answer:\[N/A\]
69. Justification: No human subjects
70. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper\. - •According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector\.
71. 15\.Institutional review board \(IRB\) approvals or equivalent for research with human subjects
72. Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board \(IRB\) approvals \(or an equivalent approval/review based on the requirements of your country or institution\) were obtained?
73. Answer:\[N/A\]
74. Justification: this paper does not involves any human subjects
75. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Depending on the country in which research is conducted, IRB approval \(or equivalent\) may be required for any human subjects research\. If you obtained IRB approval, you should clearly state this in the paper\. - •We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution\. - •For initial submissions, do not include any information that would break anonymity \(if applicable\), such as the institution conducting the review\.
76. 16\.Declaration of LLM usage
77. Question: Does the paper describe the usage of LLMs if it is an important, original, or non\-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does*not*impact the core methodology, scientific rigor, or originality of the research, declaration is not required\.
78. Answer:\[N/A\]
79. Justification: No original idea from LLM\.
80. Guidelines: - •The answer\[N/A\]means that the core method development in this research does not involve LLMs as any important, original, or non\-standard components\. - •Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described\.