Rethinking Privileged Information in On-Policy Self-Distillation

# Rethinking Privileged Information in On-Policy Self-Distillation
Source: [https://arxiv.org/html/2608.18271](https://arxiv.org/html/2608.18271)
###### Abstract

On\-policy self\-distillation \(OPSD\) trains a student on its own responses using token\-level supervision from the same model conditioned on privileged reference information\. We investigate whether performance gains from OPSD show that the student learned the information in the reference or instead reflect recovery of reasoning behavior already present in the base model\. We perform OPSD experiments on science and mathematics datasets using Qwen3 models ranging from 1\.7B to 8B\. Our analysis framework separates the supervision induced by the reference from the supervision provided by the teacher without the reference and measures how each aligns with changes in the student’s predictions\. The correct reference does not provide a consistent performance benefit across teacher generation modes, model sizes, and training datasets\. Students can improve without the correct reference, and a solution from another problem can outperform the correct solution on several mathematical reasoning benchmarks\. The student’s predictions align more strongly with the base model’s thinking behavior than with the supervision induced by the reference, but controls constructed from other problems reproduce much of both alignments\. Moreover, stronger alignment attributable to the correct reference does not reliably coincide with a greater performance benefit from the reference\. Performance gains and distributional alignment alone therefore cannot determine how privileged reference information contributes to student learning in OPSD\.

## 1Introduction

On\-policy distillation \(OPD\) has become a widely used method in the post\-training of large language models \(LLMs\), including recent releases such as DeepSeek\-V4\([3](https://arxiv.org/html/2608.18271#bib.bib16)\), GLM\-5\([5](https://arxiv.org/html/2608.18271#bib.bib17)\), Kimi K3\([17](https://arxiv.org/html/2608.18271#bib.bib18)\), Nemotron\-Cascade 2\([32](https://arxiv.org/html/2608.18271#bib.bib19)\), and Qwen3\([31](https://arxiv.org/html/2608.18271#bib.bib15)\)\. Unlike supervised fine\-tuning \(SFT\), which is prone to exposure bias from training on fixed target sequences\([2](https://arxiv.org/html/2608.18271#bib.bib4)\), OPD trains the student on trajectories sampled from its current policy\. A teacher provides dense, token\-level supervision on the states visited by the student\([1](https://arxiv.org/html/2608.18271#bib.bib5);[6](https://arxiv.org/html/2608.18271#bib.bib6);[21](https://arxiv.org/html/2608.18271#bib.bib7)\)\. On\-policy self\-distillation \(OPSD\) uses the same model as both teacher and student under different contexts\([34](https://arxiv.org/html/2608.18271#bib.bib8)\)\. The student generates a response from the problem alone, while the teacher evaluates the same response with access to privileged information, typically a reference solution unavailable to the student\.

The central premise of OPSD is that privileged reference information improves the teacher’s supervision and transfers useful information to the student\([34](https://arxiv.org/html/2608.18271#bib.bib8)\)\. However, increasing the teacher’s exposure to a reference is not consistently beneficial and can alter uncertainty expression or reasoning behavior\([7](https://arxiv.org/html/2608.18271#bib.bib29);[15](https://arxiv.org/html/2608.18271#bib.bib32);[13](https://arxiv.org/html/2608.18271#bib.bib33)\)\. Reference\-conditioned supervision may also contain information that the student cannot use when the reference is absent at inference time\([36](https://arxiv.org/html/2608.18271#bib.bib28);[25](https://arxiv.org/html/2608.18271#bib.bib34)\)\. Performance alone therefore cannot determine whether the student learned from the reference\. This question becomes more difficult when the teacher operates in thinking mode because the teacher then differs from the student in both its context and its generation mode\.

We therefore ask three questions\. First, is the correct reference necessary for OPSD to improve performance? Second, must the reference correspond to the problem being solved? Third, do the gains reflect learning from the reference or recovery of reasoning behavior already present in the base model? We study these questions using Qwen3\-8B, Qwen3\-4B, and Qwen3\-1\.7B students trained on NuminaMath and MegaScience\. For OPSD, pairing a non\-thinking student with a thinking teacher produces the largest performance gains among the tested mode combinations\([34](https://arxiv.org/html/2608.18271#bib.bib8)\)\. In our experiments, the student remains in non\-thinking mode during both training and evaluation, while we vary the teacher’s generation mode and reference context\. Holding the student’s generation mode fixed isolates these teacher\-side effects\. It also allows us to test whether supervision from a thinking teacher moves the student toward behavior associated with the base model’s thinking mode\.

We introduce an analysis framework that separates the supervision induced by the reference from the remaining teacher supervision\. We measure how each form of supervision aligns with changes in the student’s predictions during training and repeat the measurements using teacher contexts from other problems\. Figure[1](https://arxiv.org/html/2608.18271#S1.F1)summarizes the OPSD setup and the teacher and reference conditions used in our experiments\.

studentbase model \+ LoRAnon\-thinking modeseesxxonlysamples on\-policyteacherbase model \(frozen\)seesxx\+ referencetwo teacher modesthinkingnon\-thinkingproblemxxreference contextseen only by the teacherstudent response, evaluated token by tokennext\-token training targetsCentral question\.Does the student learn the information contained in the reference?Figure 1:OPSD setup and experimental conditions\. The student remains in non\-thinking mode while we vary the teacher’s generation mode and reference context\.Our contributions are as follows:

- •We show that the benefit of the correct reference depends on the teacher’s generation mode, the model size, and the training dataset\. Students can improve without the correct reference, and a solution from another problem can outperform the correct solution on several mathematical reasoning benchmarks\.
- •We introduce an analysis framework that separates reference\-induced supervision from the remaining teacher supervision and measures how each aligns with changes in the student’s predictions\. The change in the student’s predictions aligns more strongly with the base model’s thinking behavior than with the supervision induced by the reference\.
- •We compare these alignments with controls constructed from other problems\. The controls reproduce much of the apparent alignment, and stronger alignment attributable to the correct reference does not reliably coincide with a greater performance benefit from the reference\. Together, these results show that performance gains and distributional alignment alone cannot determine how privileged reference information contributes to student learning in OPSD\.

## 2Related work

#### On\-policy distillation\.

Knowledge distillation trains a student to match the output distribution of a teacher model\([9](https://arxiv.org/html/2608.18271#bib.bib1)\)\. Sequence\-level distillation adapts this approach to autoregressive models by training the student on complete responses generated by the teacher\([16](https://arxiv.org/html/2608.18271#bib.bib2)\)\. Since these responses are not generated by the student’s current policy, the prefixes seen during training may differ from those encountered during inference, creating exposure bias and allowing errors to accumulate throughout the generated response\([2](https://arxiv.org/html/2608.18271#bib.bib4);[1](https://arxiv.org/html/2608.18271#bib.bib5)\)\. Imitation\-based distillation addresses this mismatch by training on student\-generated trajectories, with the teacher providing supervision at the states visited by the student\([20](https://arxiv.org/html/2608.18271#bib.bib3)\)\. OPD applies this principle to LLMs by evaluating the teacher on trajectories sampled from the student’s current policy\([6](https://arxiv.org/html/2608.18271#bib.bib6);[21](https://arxiv.org/html/2608.18271#bib.bib7)\)\. Training minimizes a token\-level KL divergence between the teacher’s and student’s next\-token distributions, providing dense supervision at every position in the generated response\([1](https://arxiv.org/html/2608.18271#bib.bib5)\)\.

#### On\-policy self\-distillation\.

On\-policy self\-distillation \(OPSD\) removes the need for a separate teacher by instantiating the teacher and student from a single model under different contexts\([34](https://arxiv.org/html/2608.18271#bib.bib8)\)\. The student is conditioned on the problem alone, while the teacher also receives privileged information such as a reference solution\. The student generates the training trajectories, and the teacher provides token\-level supervision by evaluating the same prefixes with access to the additional context\. Closely related work applies this principle using expert demonstrations and textual feedback\([26](https://arxiv.org/html/2608.18271#bib.bib9);[11](https://arxiv.org/html/2608.18271#bib.bib10)\)\. Other methods extend the teacher’s context beyond reference solutions to include experiential knowledge extracted from historical solution traces and optimized system prompts\([33](https://arxiv.org/html/2608.18271#bib.bib11)\), action\-only privileged information\([23](https://arxiv.org/html/2608.18271#bib.bib12)\), source documents\([27](https://arxiv.org/html/2608.18271#bib.bib13)\), and natural\-language skills extracted from completed trajectories\([28](https://arxiv.org/html/2608.18271#bib.bib14)\)\. Together, these methods use contextual information available during training to create a more informed teacher policy\.

#### Privileged context in OPSD\.

A more informed teacher does not necessarily provide more useful supervision\. Privileged information specific to an individual problem may not transfer reliably to a student that lacks that information at inference time\([36](https://arxiv.org/html/2608.18271#bib.bib28)\)\. Providing more of the reference can increase the mismatch between teacher and student\([7](https://arxiv.org/html/2608.18271#bib.bib29)\), while rich teacher context can suppress uncertainty and impair out\-of\-distribution generalization\([15](https://arxiv.org/html/2608.18271#bib.bib32)\)\. Privileged\-context distillation can also degrade thinking models by suppressing verification and self\-correction\([13](https://arxiv.org/html/2608.18271#bib.bib33)\)\. Proposed remedies adapt the amount of reference shown to the teacher\([7](https://arxiv.org/html/2608.18271#bib.bib29)\), reduce the influence of high\-entropy token positions\([14](https://arxiv.org/html/2608.18271#bib.bib30)\), restrict distillation to identified reasoning errors\([35](https://arxiv.org/html/2608.18271#bib.bib31)\), or remove reference\-specific shortcuts from the teacher’s signal\([25](https://arxiv.org/html/2608.18271#bib.bib34)\)\. Concurrent work finds that a solution from another problem can preserve or improve OPSD performance\([12](https://arxiv.org/html/2608.18271#bib.bib35)\)and shows that likelihood changes induced by privileged context do not necessarily provide useful token credit\([22](https://arxiv.org/html/2608.18271#bib.bib36)\)\. Our study complements this work by separating reference\-dependent from reference\-free teacher supervision and comparing how each relates to changes in the student’s predictions using controls constructed from other problems\.

## 3Analysis Framework

### 3\.1Training Objective

In OPSD, a student modelπS\\pi\_\{S\}generates trajectories from its current policy, while a frozen teacherπT\\pi\_\{T\}instantiated from the same base model provides token\-level supervision with access to a reference solution\([1](https://arxiv.org/html/2608.18271#bib.bib5);[34](https://arxiv.org/html/2608.18271#bib.bib8)\)\. Let𝒟\\mathcal\{D\}denote the training dataset\. For a problemx∼𝒟x\\sim\\mathcal\{D\}with reference solutionrr, the student samples a trajectoryy∼πS\(⋅∣x\)y\\sim\\pi\_\{S\}\(\\cdot\\mid x\)\. The training objective is

ℒ\(πS\)=𝔼x∼𝒟,y∼πS\(⋅∣x\)\[∑t=1\|y\|DJSD\(πT\(⋅∣x,r,y<t\)∥πS\(⋅∣x,y<t\)\)\],\\mathcal\{L\}\(\\pi\_\{S\}\)=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\},\\;y\\sim\\pi\_\{S\}\(\\cdot\\mid x\)\}\\left\[\\sum\_\{t=1\}^\{\|y\|\}D\_\{\\mathrm\{JSD\}\}\\\!\\left\(\\pi\_\{T\}\(\\cdot\\mid x,r,y\_\{<t\}\)\\,\\\|\\,\\pi\_\{S\}\(\\cdot\\mid x,y\_\{<t\}\)\\right\)\\right\],\(1\)whereDJSDD\_\{\\mathrm\{JSD\}\}denotes the generalized Jensen–Shannon divergence\.

### 3\.2Reference\-Dependent and Reference\-Free Supervision

Equation[1](https://arxiv.org/html/2608.18271#S3.E1)specifies how the student learns from the teacher, but it does not identify which part of the teacher’s supervision comes from the reference\. Motivated by the decomposition approach introduced in Purified OPSD\([25](https://arxiv.org/html/2608.18271#bib.bib34)\), we isolate the contribution made by the reference by comparing the teacher’s next\-token distributions with and without it\.

To make this comparison, we let the student at checkpointccgenerate a responsey\(c\)∼πS\(c\)\(⋅∣x\)y^\{\(c\)\}\\sim\\pi\_\{S\}^\{\(c\)\}\(\\cdot\\mid x\)to problemxx\. At each positionttof the response, letScS\_\{c\}denote the student’s next\-token distribution\. LetTTandNNdenote the next\-token distributions of the teacher with and without referencerr, respectively\. Thus, for every tokenvvin the vocabulary𝒱\\mathcal\{V\},

T⁡\(v\)=πT​\(v∣x,r,y<t\(c\)\),N⁡\(v\)=πT​\(v∣x,y<t\(c\)\),Sc​\(v\)=πS\(c\)​\(v∣x,y<t\(c\)\)\.T\(v\)=\\pi\_\{T\}\(v\\mid x,r,y^\{\(c\)\}\_\{<t\}\),\\qquad N\(v\)=\\pi\_\{T\}\(v\\mid x,y^\{\(c\)\}\_\{<t\}\),\\qquad S\_\{c\}\(v\)=\\pi\_\{S\}^\{\(c\)\}\(v\\mid x,y^\{\(c\)\}\_\{<t\}\)\.\(2\)
We represent each distribution by its vector of log probabilities:

log⁡T:=\(log⁡T⁡\(v\)\)v∈𝒱∈ℝ\|𝒱\|,\\log T:=\\bigl\(\\log T\(v\)\\bigr\)\_\{v\\in\\mathcal\{V\}\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\},withlog⁡N\\log Nandlog⁡Sc\\log S\_\{c\}defined analogously\. The total differenceDtot\(c\)D\_\{\\mathrm\{tot\}\}^\{\(c\)\}between the log\-probability vectors of the teacher and the student can be decomposed as

log⁡T−log⁡Sc⏟Dtot\(c\)=log⁡T−log⁡N⏟Dref\(c\)\+log⁡N−log⁡Sc⏟Drec\(c\),\\underbrace\{\\log T\-\\log S\_\{c\}\}\_\{D\_\{\\mathrm\{tot\}\}^\{\(c\)\}\}=\\underbrace\{\\log T\-\\log N\}\_\{D\_\{\\mathrm\{ref\}\}^\{\(c\)\}\}\+\\underbrace\{\\log N\-\\log S\_\{c\}\}\_\{D\_\{\\mathrm\{rec\}\}^\{\(c\)\}\},\(3\)whereDref\(c\)D\_\{\\mathrm\{ref\}\}^\{\(c\)\}is the reference direction that measures the difference between the teacher with and without the reference, andDrec\(c\)D\_\{\\mathrm\{rec\}\}^\{\(c\)\}is the recovery direction that measures the difference between the teacher without the reference and the student\.

### 3\.3Composition of Teacher Supervision

Before analyzing how the student changes during training, we measure the component of the reference directionDrefD\_\{\\mathrm\{ref\}\}along the total teacher supervisionDtotD\_\{\\mathrm\{tot\}\}\. We compute this measurement at training step 0 using responses generated by the initial student\.

For each problemxx, the initial student generates a responsey\(0\)∼πS\(0\)\(⋅∣x\)y^\{\(0\)\}\\sim\\pi\_\{S\}^\{\(0\)\}\(\\cdot\\mid x\)\. Letℐ0\\mathcal\{I\}\_\{0\}denote the set of all problem\-position pairs\(x,t\)\(x,t\)in these responses\. For any two directionsAAandBB, letAx,t,Bx,t∈ℝ\|𝒱\|A^\{x,t\},B^\{x,t\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\}denote their values at positionttof the response to problemxx\. We define the inner product and its induced norm as

⟨A,B⟩ℐ0=1\|ℐ0\|​∑\(x,t\)∈ℐ0⟨Ax,t,Bx,t⟩,∥A∥ℐ0=⟨A,A⟩ℐ0,\\langle A,B\\rangle\_\{\\mathcal\{I\}\_\{0\}\}=\\frac\{1\}\{\|\\mathcal\{I\}\_\{0\}\|\}\\sum\_\{\(x,t\)\\in\\mathcal\{I\}\_\{0\}\}\\langle A^\{x,t\},B^\{x,t\}\\rangle,\\qquad\\lVert A\\rVert\_\{\\mathcal\{I\}\_\{0\}\}=\\sqrt\{\\langle A,A\\rangle\_\{\\mathcal\{I\}\_\{0\}\}\},\(4\)where∥A∥ℐ0\\lVert A\\rVert\_\{\\mathcal\{I\}\_\{0\}\}is the root\-mean\-square magnitude ofAAacross problem\-position pairs\.

We define the reference projection coefficient as

wref=⟨Dref,Dtot⟩ℐ0‖Dtot‖ℐ02\.w\_\{\\mathrm\{ref\}\}=\\frac\{\\left\\langle D\_\{\\mathrm\{ref\}\},D\_\{\\mathrm\{tot\}\}\\right\\rangle\_\{\\mathcal\{I\}\_\{0\}\}\}\{\\left\\lVert D\_\{\\mathrm\{tot\}\}\\right\\rVert\_\{\\mathcal\{I\}\_\{0\}\}^\{2\}\}\.\(5\)

### 3\.4Changes in Student Predictions on Fixed Responses

Our analysis so far has focused on the teacher’s supervision\. To measure how the student’s predictions change during training, we evaluate every checkpointccon the responses generated by the initial student at training step 0\.

At each positiontt, letS0S\_\{0\}andScS\_\{c\}denote the next\-token distributions of the initial student and the student at checkpointcc, conditioned on the same generated prefix:

S0​\(v\)=πS\(0\)​\(v∣x,y<t\(0\)\),Sc​\(v\)=πS\(c\)​\(v∣x,y<t\(0\)\)\.S\_\{0\}\(v\)=\\pi\_\{S\}^\{\(0\)\}\(v\\mid x,y^\{\(0\)\}\_\{<t\}\),\\qquad S\_\{c\}\(v\)=\\pi\_\{S\}^\{\(c\)\}\(v\\mid x,y^\{\(0\)\}\_\{<t\}\)\.\(6\)We also compute the teacher distributionsTTandNNfrom Section[3\.2](https://arxiv.org/html/2608.18271#S3.SS2)on these prefixes\. LetBBdenote the distribution produced by the base model in thinking mode without the reference:

B⁡\(v\)=πTthink​\(v∣x,y<t\(0\)\)\.B\(v\)=\\pi\_\{T\}^\{\\mathrm\{think\}\}\(v\\mid x,y^\{\(0\)\}\_\{<t\}\)\.
We define the change in the student’s log probabilities as

Δ​Sc=log⁡Sc−log⁡S0\.\\Delta S\_\{c\}=\\log S\_\{c\}\-\\log S\_\{0\}\.\(7\)The teacher directions are defined on the same fixed responses:

Dref=log⁡T−log⁡N,Drec=log⁡N−log⁡S0,Dthink=log⁡B−log⁡S0\.D\_\{\\mathrm\{ref\}\}=\\log T\-\\log N,\\qquad D\_\{\\mathrm\{rec\}\}=\\log N\-\\log S\_\{0\},\\qquad D\_\{\\mathrm\{think\}\}=\\log B\-\\log S\_\{0\}\.The responses, generated prefixes, and teacher distributions remain fixed across checkpoints\. Only the student distributionScS\_\{c\}and the resulting changeΔ​Sc\\Delta S\_\{c\}vary withcc\. This allows us to compare changes in the student’s predictions while keeping the text on which the distributions are computed fixed\.

When the teacher operates in non\-thinking mode without the reference, its model and context are the same as those of the initial student\. Therefore,N=S0N=S\_\{0\}andDrec=0D\_\{\\mathrm\{rec\}\}=0\.

### 3\.5Relationship Between Teacher Supervision and Student Predictions

We characterize the student’s change in three ways\. We first measure its projection onto each teacher direction and its cosine similarity with that direction\. We then measure how much of the student’s change can be represented by a linear combination of two teacher directions\. Finally, we measure the magnitude ofΔ​Sc\\Delta S\_\{c\}and the KL divergence from the reference\-free teacher distributionNNtoScS\_\{c\}\. LetDDdenote any nonzero teacher direction defined in Section[3\.4](https://arxiv.org/html/2608.18271#S3.SS4)\.

We measure the component ofΔ​Sc\\Delta S\_\{c\}along directionDDusing the projection coefficient

βD\(c\)=⟨Δ​Sc,D⟩ℐ0∥D∥ℐ02\.\\beta\_\{D\}^\{\(c\)\}=\\frac\{\\langle\\Delta S\_\{c\},D\\rangle\_\{\\mathcal\{I\}\_\{0\}\}\}\{\\lVert D\\rVert\_\{\\mathcal\{I\}\_\{0\}\}^\{2\}\}\.\(8\)The projection coefficient depends on the relative magnitudes ofΔ​Sc\\Delta S\_\{c\}andDDas well as on their directions\. To compare the directions independently of magnitude, we use cosine similarity:

cos⁡θD\(c\)=⟨Δ​Sc,D⟩ℐ0∥Δ​Sc∥ℐ0​∥D∥ℐ0\.\\cos\\theta\_\{D\}^\{\(c\)\}=\\frac\{\\langle\\Delta S\_\{c\},D\\rangle\_\{\\mathcal\{I\}\_\{0\}\}\}\{\\lVert\\Delta S\_\{c\}\\rVert\_\{\\mathcal\{I\}\_\{0\}\}\\lVert D\\rVert\_\{\\mathcal\{I\}\_\{0\}\}\}\.\(9\)When comparing teacher directions, we compute the cosine similarity between the reference directionDrefD\_\{\\mathrm\{ref\}\}and the thinking directionDthinkD\_\{\\mathrm\{think\}\}\.

We next measure the fraction of∥Δ​Sc∥ℐ02\\lVert\\Delta S\_\{c\}\\rVert\_\{\\mathcal\{I\}\_\{0\}\}^\{2\}preserved by its orthogonal projection onto the span of two teacher directions\. We useDrefD\_\{\\mathrm\{ref\}\}andDrecD\_\{\\mathrm\{rec\}\}for a thinking teacher\. BecauseDrec=0D\_\{\\mathrm\{rec\}\}=0for a non\-thinking teacher, we useDrefD\_\{\\mathrm\{ref\}\}andDthinkD\_\{\\mathrm\{think\}\}instead\. At each problem\-position pair, we define the span as

𝒲x,t=\{span⁡\{Drefx,t,Drecx,t\},thinking teacher,span⁡\{Drefx,t,Dthinkx,t\},non\-thinking teacher\.\\mathcal\{W\}^\{x,t\}=\\begin\{cases\}\\operatorname\{span\}\\left\\\{D\_\{\\mathrm\{ref\}\}^\{x,t\},D\_\{\\mathrm\{rec\}\}^\{x,t\}\\right\\\},&\\text\{thinking teacher\},\\\\\[3\.0pt\] \\operatorname\{span\}\\left\\\{D\_\{\\mathrm\{ref\}\}^\{x,t\},D\_\{\\mathrm\{think\}\}^\{x,t\}\\right\\\},&\\text\{non\-thinking teacher\}\.\\end\{cases\}\(10\)LetP𝒲x,tP\_\{\\mathcal\{W\}^\{x,t\}\}denote the orthogonal projection onto this span\. We define this fraction as

Rc2=∑\(x,t\)∈ℐ0‖P𝒲x,t​Δ​Scx,t‖22∑\(x,t\)∈ℐ0‖Δ​Scx,t‖22\.R\_\{c\}^\{2\}=\\frac\{\\sum\_\{\(x,t\)\\in\\mathcal\{I\}\_\{0\}\}\\left\\lVert P\_\{\\mathcal\{W\}^\{x,t\}\}\\Delta S\_\{c\}^\{x,t\}\\right\\rVert\_\{2\}^\{2\}\}\{\\sum\_\{\(x,t\)\\in\\mathcal\{I\}\_\{0\}\}\\left\\lVert\\Delta S\_\{c\}^\{x,t\}\\right\\rVert\_\{2\}^\{2\}\}\.\(11\)
Finally, we measure the magnitude ofΔ​Sc\\Delta S\_\{c\}using∥Δ​Sc∥ℐ0\\lVert\\Delta S\_\{c\}\\rVert\_\{\\mathcal\{I\}\_\{0\}\}\. We measure the average token\-level KL divergence fromNNtoScS\_\{c\}as

KLℐ0\(N∥Sc\)=1\|ℐ0\|∑\(x,t\)∈ℐ0KL\(Nx,t∥Scx,t\)\.\\operatorname\{KL\}\_\{\\mathcal\{I\}\_\{0\}\}\(N\\\|S\_\{c\}\)=\\frac\{1\}\{\|\\mathcal\{I\}\_\{0\}\|\}\\sum\_\{\(x,t\)\\in\\mathcal\{I\}\_\{0\}\}\\operatorname\{KL\}\\\!\\left\(N^\{x,t\}\\\|S\_\{c\}^\{x,t\}\\right\)\.

### 3\.6Problem\-Specific Effects of Teacher Supervision

The metrics in Section[3\.5](https://arxiv.org/html/2608.18271#S3.SS5)compare the student’s change with the teacher directions\. Because the teacher and student share the same base model and evaluate the same generated prefix, some of the measured similarity may not be specific to the problem\. We control for this by repeating each comparison using the teacher context from a different problem\.

To construct the control, we keep the student\-generated response fixed and condition the teacher on a different problem\. If the teacher receives a reference, it receives the reference for this different problem\. This produces the control distributionsT∗T^\{\*\},N∗N^\{\*\}, andB∗B^\{\*\}\. The control directions are:

Dref∗=log⁡T∗−log⁡N∗,Drec∗=log⁡N∗−log⁡S0,Dthink∗=log⁡B∗−log⁡S0\.D\_\{\\mathrm\{ref\}\}^\{\*\}=\\log T^\{\*\}\-\\log N^\{\*\},\\qquad D\_\{\\mathrm\{rec\}\}^\{\*\}=\\log N^\{\*\}\-\\log S\_\{0\},\\qquad D\_\{\\mathrm\{think\}\}^\{\*\}=\\log B^\{\*\}\-\\log S\_\{0\}\.The student distributionsS0S\_\{0\}andScS\_\{c\}remain fixed, soΔ​Sc\\Delta S\_\{c\}does not change\. When comparing two teacher directions, we keep the problem and generated prefix fixed and replace only the reference with one from another problem\. This comparison measures whether the alignment between the two teacher directions depends on using the correct reference\.

For the measurements defined in Section[3\.5](https://arxiv.org/html/2608.18271#S3.SS5), we replace the original teacher directions with the control directions defined above\. The difference between the original and control measurements isolates dependence on the correct problem context\. Because assigning a different teacher context can change a direction’s magnitude, we use cosine similarity as the primary directional measure and report projection coefficients with the corresponding direction norms\. We estimate uncertainty using a paired bootstrap over problems\. To verify that the results do not depend on a particular pairing between evaluated and control problems, we repeat each control construction under three different pairings while keeping the student responses and original measurements fixed\.

## 4Experiments

### 4\.1Experimental Setup

#### Models and datasets\.

Our main experiments use Qwen3\-8B\([31](https://arxiv.org/html/2608.18271#bib.bib15)\)\. We repeat the core performance and distributional comparisons with Qwen3\-4B and Qwen3\-1\.7B\. Complete results for these models appear in Appendix[C](https://arxiv.org/html/2608.18271#A3)\. At each scale, the student and teacher are instantiated from the same base model\. Each student uses LoRA with rank 64 and alpha 128\([10](https://arxiv.org/html/2608.18271#bib.bib20)\)and operates in non\-thinking mode\. The teacher is frozen and operates in either thinking or non\-thinking mode\. We curate a 10,000\-problem subset of NuminaMath\([19](https://arxiv.org/html/2608.18271#bib.bib21)\)and another 10,000\-problem subset of MegaScience\([4](https://arxiv.org/html/2608.18271#bib.bib22)\)\. Both subsets contain worked solutions with extractable boxed answers, and we remove overlaps with the corresponding evaluation benchmarks\.

#### Training conditions and implementation\.

At all three scales, we train students using thinking and non\-thinking teachers\. For the thinking teacher, we compare training with and without the reference\. For the non\-thinking teacher, we compare the correct reference with a reference from another problem\. At 8B, we additionally compare answer\-only references, abstract hints, canonical solutions, and reasoning traces generated by Qwen3\-8B and Qwen3\-32B\. We use the same 4,953 NuminaMath problems for the answer\-only references, canonical solutions, and reasoning traces\. The abstract\-hint condition uses 4,898 of these problems after removing 55 hints that reveal the answer\. We train for 300 steps with a learning rate of5×10−65\\times 10^\{\-6\}and an effective batch size of 32\. We generate on\-policy student trajectories with vLLM\([18](https://arxiv.org/html/2608.18271#bib.bib27)\)at temperature 1\.1 with top\-pp0\.95 and top\-kk20\. We set the generalized Jensen–Shannon interpolation parameter toβ=0\\beta=0\. For experiments with a non\-thinking teacher, we set the maximum student completion length to 4,096 tokens on NuminaMath and 2,048 tokens on MegaScience, with pointwise clipping at10−710^\{\-7\}\. For experiments with a thinking teacher, we set the maximum student completion length to 1,024 tokens and use pointwise clipping at 0\.06\.

#### Evaluation\.

We evaluate mathematical reasoning on MATH\-500\([8](https://arxiv.org/html/2608.18271#bib.bib23)\), AIME 2024, AIME 2025, and HMMT 2025, and scientific reasoning on GPQA\-Diamond\([24](https://arxiv.org/html/2608.18271#bib.bib24)\), MMLU\-Pro\([29](https://arxiv.org/html/2608.18271#bib.bib25)\), and UGPhysics\([30](https://arxiv.org/html/2608.18271#bib.bib26)\)\. We report Avg@kk, the mean accuracy acrosskksampled responses\. The main AIME, HMMT, and GPQA\-Diamond comparisons usek=12k=12\. MATH\-500 usesk=4k=4, while the 8B reference\-type comparison in Table[9](https://arxiv.org/html/2608.18271#A1.T9)usesk=12k=12\. MMLU\-Pro and UGPhysics usek=1k=1\. The primary evaluations use non\-thinking mode to hold the student’s generation mode fixed\. Appendix[A\.5](https://arxiv.org/html/2608.18271#A1.SS5)reports an auxiliary Qwen3\-8B evaluation with thinking enabled\. All evaluations use temperature 1\.0, with top\-pp0\.8 in non\-thinking mode and 0\.95 in thinking mode\. We use a repetition penalty of 1\.3 for students trained with a non\-thinking teacher and 1\.0 for students trained with a thinking teacher\. We grade free\-response mathematics by symbolic equivalence and use benchmark\-specific answer matching for the remaining evaluations\. Uncertainty estimation, checkpoint\-selection procedures, and complete evaluation settings are reported in Appendices[A\.4](https://arxiv.org/html/2608.18271#A1.SS4)and[C](https://arxiv.org/html/2608.18271#A3)\. We describe one student as performing better than another only when the paired 95% interval for their score difference does not contain zero\.

### 4\.2Effects of Teacher Mode, Reference Information, and Model Scale on Performance

Table[1](https://arxiv.org/html/2608.18271#S4.T1)reports the complete Qwen3\-8B results\. On NuminaMath, students trained with either teacher mode score above the non\-thinking base model on average\. On MegaScience, students trained with a thinking teacher score above the non\-thinking base model on all four benchmarks, whereas students trained with a non\-thinking teacher show smaller and less consistent differences\. Adding the reference to a thinking teacher provides no consistent advantage at 8B\. On NuminaMath, the student trained with the reference scores higher on AIME 2024 and AIME 2025, while the student trained without it scores higher on MATH\-500 and HMMT 2025\. On MegaScience, their scores differ by at most 0\.3 points across the four benchmarks\.

Table 1:Main benchmark results for Qwen3\-8B\. The base model and all OPSD students are evaluated in non\-thinking mode\. Ref\. denotes the reference solution\. Avg\. is the unweighted mean across benchmarks\. AIME, HMMT, and GPQA\-Diamond use Avg@12, MATH\-500 uses Avg@4, and MMLU\-Pro and UGPhysics use Avg@1\.At 4B and 1\.7B, the effect of the reference depends on the training dataset\. On NuminaMath, paired comparisons support improvements on AIME 2024, AIME 2025, and HMMT 2025 at 4B, and on AIME 2024 and HMMT 2025 at 1\.7B\. For students trained on MegaScience, the student trained with the reference does not outperform the student trained without it at 4B\. At 1\.7B, removing the reference improves all three science benchmarks\. Complete results appear in Tables[16](https://arxiv.org/html/2608.18271#A3.T16)and[17](https://arxiv.org/html/2608.18271#A3.T17)\.

Under a non\-thinking teacher, replacing the correct solution with a solution from another problem improves AIME 2025 and HMMT 2025 at 8B, and all three competition mathematics benchmarks at 4B and 1\.7B\. For students trained on MegaScience, the student trained with a solution from another problem does not outperform the student trained with the correct solution on any of the three science benchmarks at 4B or 1\.7B\. The corresponding 8B scores and paired comparisons appear in Table[10](https://arxiv.org/html/2608.18271#A1.T10)\. Across the tested models, the performance value of the reference therefore depends on the teacher’s generation mode, the model scale, and the training dataset\.

### 4\.3Effects of Reference Information on the Teacher’s Predictions

Performance does not reveal how the reference changes the teacher’s predictions\. We measure this effect at Qwen3\-8B using the decomposition defined in Section[3\.2](https://arxiv.org/html/2608.18271#S3.SS2)\. At training step 0, the reference projection coefficientwrefw\_\{\\mathrm\{ref\}\}is 0\.46 on NuminaMath and 0\.39 on MegaScience\. The reference therefore changes the teacher’s predictions even though adding it provides no consistent performance advantage at this scale\. On NuminaMath, under a non\-thinking teacher, we compare the reference directionDrefD\_\{\\mathrm\{ref\}\}, induced by adding the reference, with the thinking directionDthinkD\_\{\\mathrm\{think\}\}, induced by enabling thinking mode\. We construct the controlDref∗D\_\{\\mathrm\{ref\}\}^\{\*\}by replacing the correct reference with a reference from another problem\.

Figure 2:Cosine similarity between the reference and thinking directions on NuminaMath under a non\-thinking teacher, across five reference types\. Error bars show 1\.96 bootstrap standard errors across problems\.Figure[2](https://arxiv.org/html/2608.18271#S4.F2)reports this comparison across five reference types\. In every condition, the cosine similarity betweenDrefD\_\{\\mathrm\{ref\}\}andDthinkD\_\{\\mathrm\{think\}\}is lower than the cosine similarity betweenDref∗D\_\{\\mathrm\{ref\}\}^\{\*\}andDthinkD\_\{\\mathrm\{think\}\}\. This ordering holds under all three control assignments\. The direction induced by the correct reference therefore does not align more closely withDthinkD\_\{\\mathrm\{think\}\}than the direction induced by a reference from another problem\.

### 4\.4Effects of Teacher Supervision on the Student’s Predictions

We next measure how the student’s next\-token predictions change during training\. We denote this change byΔ​Sc\\Delta S\_\{c\}\. In addition to the reference directionDrefD\_\{\\mathrm\{ref\}\}introduced above, we use the recovery directionDrecD\_\{\\mathrm\{rec\}\}, which compares the predictions of the reference\-free teacher with those of the initial student\. For each directionDD, we compare the cosine similarity betweenΔ​Sc\\Delta S\_\{c\}andDDwith the corresponding similarity for the control directionD∗D^\{\*\}constructed from another problem\.

Figure 3:Alignment between changes in the Qwen3\-8B student’s predictions and the teacher directions during training\.Under a thinking teacher, Figures[3](https://arxiv.org/html/2608.18271#S4.F3)\(a\) and[3](https://arxiv.org/html/2608.18271#S4.F3)\(b\) show higher alignment withDrecD\_\{\\mathrm\{rec\}\}than withDrefD\_\{\\mathrm\{ref\}\}\. The directions for the evaluated problem and their controls follow similar trajectories on both datasets\. The higher alignment withDrecD\_\{\\mathrm\{rec\}\}therefore also occurs when the recovery direction is constructed from another problem\. Under a non\-thinking teacher, Figures[3](https://arxiv.org/html/2608.18271#S4.F3)\(c\) and[3](https://arxiv.org/html/2608.18271#S4.F3)\(d\) show lower alignment withDrefD\_\{\\mathrm\{ref\}\}than withDref∗D\_\{\\mathrm\{ref\}\}^\{\*\}\. At step 150, the difference between these cosine similarities is−0\.23±0\.03\-0\.23\\pm 0\.03on NuminaMath and−0\.38±0\.04\-0\.38\\pm 0\.04on MegaScience\.

Figure 4:Changes in the Qwen3\-8B student’s predictions measured against the teacher directions and their controls\. \(a\) Projection coefficients ontoDrefD\_\{\\mathrm\{ref\}\}andDrecD\_\{\\mathrm\{rec\}\}under a thinking teacher\. \(b\) Cosine similarity with the same directions\.Figure[4](https://arxiv.org/html/2608.18271#S4.F4)\(a\) reports larger projection coefficients forDrecD\_\{\\mathrm\{rec\}\}\. Because a projection coefficient also depends on the magnitude of the teacher direction, a larger coefficient does not necessarily indicate stronger alignment\. Figure[4](https://arxiv.org/html/2608.18271#S4.F4)\(b\) removes this dependence through cosine similarity\. At 8B, the cosine similarity withDrefD\_\{\\mathrm\{ref\}\}exceeds that withDref∗D\_\{\\mathrm\{ref\}\}^\{\*\}by 0\.05 on NuminaMath and 0\.10 on MegaScience\. The corresponding differences forDrecD\_\{\\mathrm\{rec\}\}are−0\.08\-0\.08and 0\.01\. The student therefore shows only weak additional alignment withDrefD\_\{\\mathrm\{ref\}\}beyond the alignment reproduced by its control, and no consistent additional alignment withDrecD\_\{\\mathrm\{rec\}\}\.

Table[2](https://arxiv.org/html/2608.18271#S4.T2)reportsRc2R\_\{c\}^\{2\}, the fraction of∥Δ​Sc∥2\\lVert\\Delta S\_\{c\}\\rVert^\{2\}that lies in the span of the teacher directions\. For directions constructed from the evaluated problem,Rc2R\_\{c\}^\{2\}ranges from 0\.88 to 0\.97\. For the control directions, it ranges from 0\.94 to 0\.98\. The control values match or exceed the values for the evaluated problem under both teacher modes and on both datasets\. A high value ofRc2R\_\{c\}^\{2\}therefore does not indicate dependence on the correct problem context\.

Table 2:FractionRc2R\_\{c\}^\{2\}of the squared change in the Qwen3\-8B student’s log probabilities that lies in the span of the teacher directions at checkpoint 300\.The corresponding results at 4B and 1\.7B appear in Table[18](https://arxiv.org/html/2608.18271#A3.T18)\. Under a thinking teacher, across the three tested model sizes, the differencecos⁡θDref\(c\)−cos⁡θDref∗\(c\)\\cos\\theta\_\{D\_\{\\mathrm\{ref\}\}\}^\{\(c\)\}\-\\cos\\theta\_\{D\_\{\\mathrm\{ref\}\}^\{\*\}\}^\{\(c\)\}increases in the order 8B, 4B, and 1\.7B on both datasets\. This ordering describes the three models tested here, but does not establish that model size caused the differences\. On NuminaMath, the larger cosine differences at 4B and 1\.7B coincide with higher performance for the student trained with the reference\. On MegaScience, the cosine difference is largest at 1\.7B, but the student trained without the reference performs better on all three science benchmarks\. A larger cosine difference therefore does not reliably indicate a larger performance benefit from the reference\.

Under a non\-thinking teacher, the cosine similarity betweenΔ​Sc\\Delta S\_\{c\}andDrefD\_\{\\mathrm\{ref\}\}, before comparison with its control, changes sign across model scales\. It is positive at 4B on both datasets and negative or near zero at 8B and 1\.7B\. However, at 8B, 4B, and 1\.7B on both NuminaMath and MegaScience,DrefD\_\{\\mathrm\{ref\}\}aligns less withΔ​Sc\\Delta S\_\{c\}thanDref∗D\_\{\\mathrm\{ref\}\}^\{\*\}\. The sign of the cosine similarity withDrefD\_\{\\mathrm\{ref\}\}therefore does not determine whetherDrefD\_\{\\mathrm\{ref\}\}aligns more strongly than its control\.

## 5Limitations

Our study has several limitations\. Our experiments use Qwen3 models at 8B, 4B, and 1\.7B\. Whether the results generalize to other model families remains unknown\. Our training datasets cover mathematical and scientific reasoning, so the results may not generalize to coding or other domains\. We train each condition once, so our results do not measure variation across repeated training runs\.

## 6Conclusion

We investigated what privileged reference information contributes to OPSD\. We developed an analysis framework that separates the supervision induced by the reference from the remaining teacher supervision and tests both against controls constructed from other problems\. Across Qwen3\-8B, Qwen3\-4B, and Qwen3\-1\.7B students trained on NuminaMath and MegaScience, we evaluate the performance benefit of the reference, its alignment with changes in the student’s predictions, and whether either depends on using the reference for the correct problem\.

The experiments provide three forms of evidence\. First, students can improve without the correct reference, and a solution from another problem can outperform the correct solution\. Whether the correct reference improves performance depends on the teacher’s generation mode, the model size, and the training dataset\. Second, we compare the change in the student’s predictions during training with the supervision induced by the reference and with the base model’s thinking behavior\. The student’s predictions align more strongly with the base model’s thinking behavior than with the supervision induced by the reference\. Because controls constructed from other problems reproduce much of the alignment with both forms of supervision, the stronger alignment with thinking behavior does not establish that recovering this behavior caused the performance gains\. Third, the alignment gained by using the correct reference instead of a reference from another problem does not imply a greater performance benefit\. Among the tested model sizes and training datasets, this increase in alignment is largest at 1\.7B on MegaScience\. Nevertheless, the student trained without the reference performs better\. Together, the performance experiments and distributional analysis show that the contribution of privileged reference information to OPSD cannot be inferred from performance gains or distributional alignment alone\.

## Acknowledgements

We thank Nate Woodward, Collin Farquhar, Shaghayegh Sadeghi, and Nawar Ismail for their feedback and suggestions on earlier drafts of this paper\.

## References

- Agarwalet al\.\(2024\)R\. Agarwal, N\. Vieillard, Y\. Zhou, P\. Stanczyk, S\. Ramos, M\. Geist, and O\. BachemOn\-policy distillation of language models: learning from self\-generated mistakes\.InInternational Conference on Learning Representations \(ICLR\),External Links:2306\.13649Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p1.1),[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2608.18271#S3.SS1.p1.1)\.
- Bengioet al\.\(2015\)S\. Bengio, O\. Vinyals, N\. Jaitly, and N\. ShazeerScheduled sampling for sequence prediction with recurrent neural networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),External Links:1506\.03099Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p1.1),[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px1.p1.1)\.
- DeepSeek\-AI \(2026\)DeepSeek\-AIDeepSeek\-V4: towards highly efficient million\-token context intelligence\.Technical reportDeepSeek\-AI\.External Links:[Link](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash)Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p1.1)\.
- Fanet al\.\(2025\)R\. Fan, Z\. Wang, and P\. LiuMegaScience: pushing the frontiers of post\-training datasets for science reasoning\.arXiv preprint arXiv:2507\.16812\.External Links:2507\.16812Cited by:[§A\.1](https://arxiv.org/html/2608.18271#A1.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2608.18271#S4.SS1.SSS0.Px1.p1.1)\.
- GLM\-5 Team \(2026\)GLM\-5 TeamGLM\-5: from vibe coding to agentic engineering\.arXiv preprint arXiv:2602\.15763\.External Links:2602\.15763Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p1.1)\.
- Guet al\.\(2024\)Y\. Gu, L\. Dong, F\. Wei, and M\. HuangMiniLLM: knowledge distillation of large language models\.InInternational Conference on Learning Representations \(ICLR\),External Links:2306\.08543,[Link](https://openreview.net/forum?id=5h0qf7IBZZ)Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p1.1),[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px1.p1.1)\.
- Hanet al\.\(2026\)Z\. Han, T\. Zhang, H\. Wang, and Y\. SunAdaptive teacher exposure for self\-distillation in LLM reasoning\.arXiv preprint arXiv:2605\.11458\.External Links:2605\.11458Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p2.1),[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px3.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the MATH dataset\.InAdvances in Neural Information Processing Systems \(NeurIPS\),External Links:2103\.03874Cited by:[§4\.1](https://arxiv.org/html/2608.18271#S4.SS1.SSS0.Px3.p1.1)\.
- Hintonet al\.\(2015\)G\. Hinton, O\. Vinyals, and J\. DeanDistilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.External Links:1503\.02531Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px1.p1.1)\.
- Huet al\.\(2022\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations \(ICLR\),External Links:2106\.09685Cited by:[§4\.1](https://arxiv.org/html/2608.18271#S4.SS1.SSS0.Px1.p1.1)\.
- Hübotteret al\.\(2026\)J\. Hübotter, F\. Lübeck, L\. Behric, A\. Baumann, M\. Bagatella, D\. Marta, I\. Hakimi, I\. Shenfeld, T\. Kleine Buening, C\. Guestrin, and A\. KrauseReinforcement learning via self\-distillation\.arXiv preprint arXiv:2601\.20802\.External Links:2601\.20802Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px2.p1.1)\.
- Ichiharaet al\.\(2026\)Y\. Ichihara, N\. Iwase, M\. A\. Quamar, and J\. KomiyamaPrivileged solutions or context\-induced teacher behavior? dissecting on\-policy self\-distillation\.arXiv preprint arXiv:2608\.09228\.External Links:2608\.09228Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px3.p1.1)\.
- Kauret al\.\(2026\)S\. Kaur, N\. Ri, Y\. He, L\. Fowl, and S\. AroraRethinking on\-policy self\-distillation for thinking models\.arXiv preprint arXiv:2607\.05184\.External Links:2607\.05184Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p2.1),[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px3.p1.1)\.
- Keet al\.\(2026\)J\. Ke, Z\. Wen, W\. Li, C\. He, and L\. ZhangRespecting self\-uncertainty in on\-policy self\-distillation for efficient LLM reasoning\.arXiv preprint arXiv:2605\.13255\.External Links:2605\.13255Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px3.p1.1)\.
- Kimet al\.\(2026\)J\. Kim, X\. Luo, M\. Kim, S\. Lee, D\. Kim, J\. Jeon, D\. Li, and Y\. YangWhy does self\-distillation \(sometimes\) degrade the reasoning capability of LLMs?\.arXiv preprint arXiv:2603\.24472\.External Links:2603\.24472Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p2.1),[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px3.p1.1)\.
- Kim and Rush \(2016\)Y\. Kim and A\. M\. RushSequence\-level knowledge distillation\.InProceedings of the 2016 Conference on Empirical Methods in Natural Language Processing,pp\. 1317–1327\.External Links:[Document](https://dx.doi.org/10.18653/v1/D16-1139),[Link](https://aclanthology.org/D16-1139/)Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px1.p1.1)\.
- Kimi Team \(2026\)Kimi TeamKimi K3: open frontier intelligence\.arXiv preprint arXiv:2607\.24653\.External Links:2607\.24653Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p1.1)\.
- Kwonet al\.\(2023\)W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. StoicaEfficient memory management for large language model serving with PagedAttention\.InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles \(SOSP\),External Links:2309\.06180Cited by:[§4\.1](https://arxiv.org/html/2608.18271#S4.SS1.SSS0.Px2.p1.1)\.
- Liet al\.\(2024\)J\. Li, E\. Beeching, L\. Tunstall, B\. Lipkin, R\. Soletskyi, S\. C\. Huang, K\. Rasul, L\. Yu, A\. Jiang, Z\. Shen, Z\. Qin, B\. Dong, L\. Zhou, Y\. Fleureau, G\. Lample, and S\. PoluNuminaMath\.Numina\.Note:Hugging Face dataset repositoryExternal Links:[Link](https://huggingface.co/datasets/AI-MO/NuminaMath-CoT)Cited by:[§A\.1](https://arxiv.org/html/2608.18271#A1.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.18271#S4.SS1.SSS0.Px1.p1.1)\.
- Linet al\.\(2020\)A\. Lin, J\. Wohlwend, H\. Chen, and T\. LeiAutoregressive knowledge distillation through imitation learning\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing,pp\. 6121–6133\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.494),[Link](https://aclanthology.org/2020.emnlp-main.494/)Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px1.p1.1)\.
- Lu and Thinking Machines Lab \(2025\)K\. Lu and Thinking Machines LabOn\-policy distillation\.Note:Thinking Machines Lab: ConnectionismExternal Links:[Link](https://thinkingmachines.ai/blog/on-policy-distillation/)Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p1.1),[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px1.p1.1)\.
- Nguyenet al\.\(2026\)X\. Nguyen, S\. Pandit, Y\. Zhao, A\. Koul, Z\. Liu, and S\. JotyPrivileged likelihood is not automatically value: three checks for token credit in on\-policy self\-distillation\.arXiv preprint arXiv:2608\.09263\.External Links:2608\.09263Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px3.p1.1)\.
- Penalozaet al\.\(2026\)E\. Penaloza, D\. Vattikonda, N\. Gontier, A\. Lacoste, L\. Charlin, and M\. CacciaPrivileged information distillation for language models\.arXiv preprint arXiv:2602\.04942\.External Links:2602\.04942Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px2.p1.1)\.
- Reinet al\.\(2023\)D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. BowmanGPQA: a graduate\-level google\-proof Q&A benchmark\.arXiv preprint arXiv:2311\.12022\.External Links:2311\.12022Cited by:[§4\.1](https://arxiv.org/html/2608.18271#S4.SS1.SSS0.Px3.p1.1)\.
- Shenet al\.\(2026\)Z\. Shen, J\. Tong, S\. Yan, C\. Shen, H\. Chen, W\. Ye, X\. Hu, R\. Miao, H\. Wang, J\. Zhao, G\. Chen, and J\. YePurified OPSD: on\-policy self\-distillation without losing how to think\.arXiv preprint arXiv:2607\.02234\.External Links:2607\.02234Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p2.1),[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2608.18271#S3.SS2.p1.1)\.
- Shenfeldet al\.\(2026\)I\. Shenfeld, M\. Damani, J\. Hübotter, and P\. AgrawalSelf\-distillation enables continual learning\.arXiv preprint arXiv:2601\.19897\.External Links:2601\.19897Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px2.p1.1)\.
- Steinet al\.\(2026\)A\. Stein, F\. Huang, and T\. GoldsteinGATES: self\-distillation under privileged context with consensus gating\.arXiv preprint arXiv:2602\.20574\.External Links:2602\.20574Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2026\)H\. Wang, G\. Wang, H\. Xiao, Y\. Zhou, Y\. Pan, J\. Wang, K\. Xu, Y\. Wen, X\. Ruan, X\. Chen, and H\. QiSkill\-SD: skill\-conditioned self\-distillation for multi\-turn LLM agents\.arXiv preprint arXiv:2604\.10674\.External Links:2604\.10674Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2024\)Y\. Wang, X\. Ma, G\. Zhang, Y\. Ni, A\. Chandra, S\. Guo, W\. Ren, A\. Arulraj, X\. He, Z\. Jiang, T\. Li, M\. Ku, K\. Wang, A\. Zhuang, R\. Fan, X\. Yue, and W\. ChenMMLU\-Pro: a more robust and challenging multi\-task language understanding benchmark\.InAdvances in Neural Information Processing Systems \(NeurIPS\), Datasets and Benchmarks Track,External Links:2406\.01574Cited by:[§4\.1](https://arxiv.org/html/2608.18271#S4.SS1.SSS0.Px3.p1.1)\.
- Xuet al\.\(2025\)X\. Xu, Q\. Xu, T\. Xiao, T\. Chen, Y\. Yan, J\. Zhang, S\. Diao, C\. Yang, and Y\. WangUGPhysics: a comprehensive benchmark for undergraduate physics reasoning with large language models\.InInternational Conference on Machine Learning \(ICML\),External Links:2502\.00334Cited by:[§4\.1](https://arxiv.org/html/2608.18271#S4.SS1.SSS0.Px3.p1.1)\.
- Yanget al\.\(2025\)A\. Yanget al\.Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.External Links:2505\.09388Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.18271#S4.SS1.SSS0.Px1.p1.1)\.
- Yanget al\.\(2026\)Z\. Yang, Z\. Liu, Y\. Chen, W\. Dai, B\. Wang, S\. Lin, C\. Lee, Y\. Chen, D\. Jiang, J\. He, R\. Pi, G\. Lam, N\. Lee, A\. Bukharin, M\. Shoeybi, B\. Catanzaro, and W\. PingNemotron\-Cascade 2: post\-training LLMs with cascade RL and multi\-domain on\-policy distillation\.arXiv preprint arXiv:2603\.19220\.External Links:2603\.19220Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p1.1)\.
- Yeet al\.\(2026\)T\. Ye, L\. Dong, X\. Wu, S\. Huang, and F\. WeiOn\-policy context distillation for language models\.arXiv preprint arXiv:2602\.12275\.External Links:2602\.12275Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2026a\)S\. Zhao, Z\. Xie, M\. Liu, J\. Huang, G\. Pang, F\. Chen, and A\. GroverSelf\-distilled reasoner: on\-policy self\-distillation for large language models\.arXiv preprint arXiv:2601\.18734\.External Links:2601\.18734Cited by:[§A\.5](https://arxiv.org/html/2608.18271#A1.SS5.p1.1),[§1](https://arxiv.org/html/2608.18271#S1.p1.1),[§1](https://arxiv.org/html/2608.18271#S1.p2.1),[§1](https://arxiv.org/html/2608.18271#S1.p3.1),[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2608.18271#S3.SS1.p1.1)\.
- Zhaoet al\.\(2026b\)Z\. Zhao, X\. Ma, L\. Yang, Y\. Feng, D\. Shi, J\. He, X\. Xin, Z\. Ren, and X\. WuROSD: reflective on\-policy self\-distillation for language model reasoning across domains\.arXiv preprint arXiv:2605\.28014\.External Links:2605\.28014Cited by:[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px3.p1.1)\.
- Zhuet al\.\(2026\)S\. Zhu, X\. Ye, H\. Lu, W\. Shi, and G\. LiuThe many faces of on\-policy distillation: pitfalls, mechanisms, and fixes\.arXiv preprint arXiv:2605\.11182\.External Links:2605\.11182Cited by:[§1](https://arxiv.org/html/2608.18271#S1.p2.1),[§2](https://arxiv.org/html/2608.18271#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix AAdditional Experimental Details

### A\.1Dataset Construction

#### NuminaMath\.

We construct the NuminaMath training set from the training split of NuminaMath\-CoT\([19](https://arxiv.org/html/2608.18271#bib.bib21)\)\. We retain problems with an extractable boxed answer and sample 10,000 examples using the source quotas in Table[A\.1](https://arxiv.org/html/2608.18271#A1.SS1.SSS0.Px2)\. Sampling within each source and the final shuffle use seed 42\. We remove exact overlaps with AIME 2024 and MATH\-500 after lowercasing the problem text and collapsing whitespace\. AIME 2025 and HMMT 2025 postdate the NuminaMath snapshot and are not included in the decontamination set\.

#### MegaScience\.

We construct the MegaScience training set from thetextbook\_reasoningportion of MegaScience\([4](https://arxiv.org/html/2608.18271#bib.bib22)\)\. We retain solutions that contain an extractable boxed answer and at least 250 characters\. We then sample 10,000 examples using the subject quotas in Table[A\.1](https://arxiv.org/html/2608.18271#A1.SS1.SSS0.Px2)\. Sampling uses seed 42\. We remove exact problem matches and shared word 8\-grams with GPQA\-Diamond, MMLU\-Pro, UGPhysics, SciBench, MATH\-500, and GSM8K\.

Table 3:Composition of the training subsets\.DatasetSource or subjectProblemsNuminaMathOlympiads6,500AoPS forum2,000Synthetic AMC1,000MATH500MegaSciencePhysics4,000Mathematics2,500Chemistry2,500Biology1,000
#### Comparison of reference types\.

Qwen3\-8B and Qwen3\-32B generate reasoning traces in thinking mode while receiving the original dataset solution as an answer anchor\. We retain traces whose final boxed answer agrees with the original answer and whose rendered teacher prompt contains at most 15,900 tokens\. The common set is the deduplicated intersection of the surviving 8B and 32B trace sets\. The agreement and length filters favor problems for which both models produce a correct trace within the limit\. Results on this set are therefore not directly comparable to results on the full 10,000\-problem set\.

Table[4](https://arxiv.org/html/2608.18271#A1.T4)gives the final artifact counts\. The original solution, two reasoning traces, and answer\-only condition use the same 4,953 problems in the same order\. The hint condition contains 4,898 of these problems because 55 answer\-leaking hints are removed\.

Table 4:Training\-set sizes for the reference\-type comparison\.

### A\.2Reference Conditions and Prompts

Table[5](https://arxiv.org/html/2608.18271#A1.T5)summarizes the information provided to the teacher\. In every condition except the no\-reference condition, the selected text occupies the reference field in the same teacher prompt\. The student does not receive this text\.

Table 5:Reference conditions used during training\.#### Generated references\.

The abstract hints are generated by Qwen3\-32B in non\-thinking mode at temperature 0\.7 with top\-pp0\.8, top\-kk20, and a 512\-token limit\. The model is prompted to produce three to five concise bullets\. Hints that contain\\\\backslashboxedor reproduce the extracted answer are removed\. The Qwen3\-8B and Qwen3\-32B traces are generated in thinking mode at temperature 0\.6 with top\-pp0\.95, top\-kk20, and a 32,768\-token limit\. All generated\-reference procedures use seed 42\.

#### Abstract\-hint generation prompt\.

Before applying the Qwen3\-32B chat template, we construct the following user message:

> You are a math tutor\. Given a problem and a reference solution, write an abstract hint that helps a student solve the problem without revealing the solution\. The hint should mention the core idea, theorem, transformation, invariant, case split, or caution needed to solve the problem\. It should be specific and useful, but not a full solution\. Rules: \- Do not reveal the final answer\. \- Do not use \\boxed\{\}\. \- Do not reproduce the solution steps\. \- Avoid long calculations or decisive intermediate numeric values\. \- Keep it to 3\-5 concise bullets\. Problem: \{problem\} Reference solution: \{solution\} Abstract hint:

We replace the placeholders with the problem and original dataset solution, render the prompt as a single user message, and disable thinking mode\.

#### Mismatched references\.

For NuminaMath, we apply a seed\-42 derangement to the solution column with no self\-pairs\. For MegaScience, we apply the same construction within each subject\. The problem and all other fields remain unchanged\.

#### OPSD training prompts\.

The student user message is

> Problem: \{problem\} Please reason step by step, and put your final answer within \\boxed\{\}\.

When a reference is used, the teacher user message is

> Problem: \{problem\} Here is a reference solution to this problem: === Reference Solution Begin === \{reference\} === Reference Solution End === After reading the reference solution above, make sure you truly understand the reasoning behind each step — do not copy or paraphrase it\. Now, using your own words and independent reasoning, derive the same final answer to the problem above\. Think step by step, explore different approaches, and don’t be afraid to backtrack or reconsider if something doesn’t work out: Please reason step by step, and put your final answer within \\boxed\{\}\.

In the no\-reference condition, the teacher receives the student user message\. We render both messages with the Qwen3 chat template and set thinking mode independently for the student and teacher\.

### A\.3Training Configuration

The teacher is the frozen Qwen3\-8B base model\. During teacher scoring, the student’s LoRA adapter is disabled\. We minimize the full\-vocabulary forward KL divergenceKL\(πT∥πS\)\\operatorname\{KL\}\(\\pi\_\{T\}\\\|\\pi\_\{S\}\)\. Both distributions are computed at temperature 1\.1\. Pointwise clipping is applied to each vocabulary coordinate’s contribution before summation, masking, and reduction\. We list the common hyperparameters in Table[6](https://arxiv.org/html/2608.18271#A1.T6)and the dataset\- and mode\-specific settings in Table[7](https://arxiv.org/html/2608.18271#A1.T7)\.

Table 6:Shared OPSD training configuration\.The standard microbatch contains two examples per GPU with two gradient\-accumulation steps across eight GPUs\. The full\-trace conditions use one example per GPU with four accumulation steps\. We use FlashAttention 2, gradient checkpointing, and colocated vLLM generation with tensor parallel size one\. The vLLM rollout engines use their data\-parallel process indices as seeds\.

The 20,000\-token setting limits the student rollout context\. Student prompts are truncated to leave room for the maximum completion\. Teacher prompts are separately truncated at 20,000 tokens before the sampled student completion is appended\. It is therefore not a universal limit on the combined teacher sequence\. For the full\-trace conditions, the 15,900\-token prompt filter and 4,096\-token completion limit keep the combined sequence below 20,000 tokens\.

Table 7:Settings that vary by dataset and teacher mode\.
### A\.4Evaluation

We define Avg@kkas the mean accuracy acrosskksampled responses\. Unparseable responses count as incorrect\. Table[8](https://arxiv.org/html/2608.18271#A1.T8)gives the main reported read and grader for each benchmark\.

Table 8:Evaluation benchmarks and grading procedures\.The reference\-type comparison uses Avg@12 on AIME 2024\. Every trained student is evaluated in non\-thinking mode at temperature 1\.0 and top\-pp0\.8\. Students trained with a non\-thinking teacher use repetition penalty 1\.3, while students trained with a thinking teacher use 1\.0\. Base non\-thinking evaluations use repetition penalty 1\.0\. Separate base\-model evaluations in thinking mode use temperature 1\.0 and top\-pp0\.95\.

#### Uncertainty estimation\.

We compare two students by resampling the same benchmark problems for both students 10,000 times and recomputing the difference in Avg@kk\. The paired 95% interval contains the 2\.5th through 97\.5th percentiles of these differences\. In Table[9](https://arxiv.org/html/2608.18271#A1.T9), each reference type is compared with the canonical solution\. In Table[10](https://arxiv.org/html/2608.18271#A1.T10), the solution from another problem is compared with the correct solution\. No other pairs are tested\.

Table 9:Reference\-type comparison on AIME 2024 under a non\-thinking teacher \(Avg@12\)\.Table 10:Correct and mismatched references under a non\-thinking teacher \(Avg@12\)\.
#### Checkpoint selection\.

For each trained condition, we evaluate saved checkpoints on two selection benchmarks using Avg@4 and select the checkpoint with the highest mean across them\. We use MATH\-500 and AIME 2024 for NuminaMath, and MATH\-500 and GPQA\-Diamond for MegaScience\. The remaining benchmarks are evaluated only after checkpoint selection\. Table[11](https://arxiv.org/html/2608.18271#A1.T11)gives the selected Qwen3\-8B checkpoints\.

Table 11:Checkpoints used for the main benchmark comparison\.
#### Generation limits\.

Table[12](https://arxiv.org/html/2608.18271#A1.T12)reports the maximum generated tokens in the main performance evaluations\. Every condition compared within a benchmark uses the same completion limit, so a single limit per benchmark applies to all conditions\.

Table 12:Generation limits for the main reported evaluations\.
#### UGPhysics coverage\.

The base, non\-thinking\-teacher, and mismatched\-reference rows use all 5,520 UGPhysics questions\. The thinking\-teacher row uses 5,477 questions after a transient dataset\-configuration load failure\. The corresponding no\-reference row uses 5,175 questions because one 345\-problem evaluation shard did not complete before the job limit\.

### A\.5Evaluation in Thinking Mode

We additionally evaluate the Qwen3\-8B base model and the students trained on NuminaMath with a thinking teacher, with and without the reference\. Thinking mode is enabled for every model, as in[34](https://arxiv.org/html/2608.18271#bib.bib8)\. We set the maximum generation length to 38,912 tokens, temperature to 1\.0, top\-ppto 0\.95, top\-kkto−1\-1, and the repetition penalty to 1\.0\. Table[13](https://arxiv.org/html/2608.18271#A1.T13)reports Avg@12 accuracy on AIME 2024, AIME 2025, and HMMT 2025\. The checkpoints were selected using the main non\-thinking evaluation\.

Table 13:Thinking\-mode evaluation of Qwen3\-8B on mathematical reasoning benchmarks\.
### A\.6Software and Compute

We train on one node with eight NVIDIA H200 GPUs\. Each training condition uses one training run\. Primary launchers request 64 CPU cores and 256 GB of host memory\. Table[14](https://arxiv.org/html/2608.18271#A1.T14)gives the principal software versions from the training environment\.

Table 14:Principal software versions\.

## Appendix BAdditional Distributional Analyses

This section reports additional measurements of the teacher’s signal and the student’s change\. The change\-magnitude and alignment analyses use the fixed responses generated by the initial student, following Section[3\.4](https://arxiv.org/html/2608.18271#S3.SS4)\. The policy\-entropy measurements are instead recorded from the on\-policy trajectories generated during training\.

### B\.1Sensitivity to Control Pairing

Each control pairs an evaluated problem with a different problem\. To determine whether the results depend on this pairing, we repeat each control construction under three different assignments while keeping the student responses and original measurements fixed\. Table[15](https://arxiv.org/html/2608.18271#A2.T15)reports the range of the difference between the evaluated\-problem and control cosine similarities across the three assignments\.

Table 15:Cosine\-similarity differences across three control assignments\. Bracketed entries show the minimum and maximum values, not confidence intervals\.The sign of every cosine difference is unchanged across the three assignments\. The magnitude\-sensitive reference projection in Figure[4](https://arxiv.org/html/2608.18271#S4.F4)\(a\) is less stable on MegaScience, where the difference between the evaluated\-problem and control measurements ranges from−0\.038\-0\.038to0\.0880\.088\. Because the projection coefficient also depends on the direction’s magnitude, we do not interpret this variation as a stable directional effect\. The corresponding cosine difference remains positive across all three assignments\.

### B\.2Change Magnitude and Recovery Divergence

As Figure[5](https://arxiv.org/html/2608.18271#A2.F5)shows, after the initial variation,∥Δ​Sc∥ℐ0\\lVert\\Delta S\_\{c\}\\rVert\_\{\\mathcal\{I\}\_\{0\}\}remains within a relatively narrow range on both datasets, whileKLℐ0\(N∥Sc\)\\operatorname\{KL\}\_\{\\mathcal\{I\}\_\{0\}\}\(N\\\|S\_\{c\}\)continues to increase\. The student therefore becomes increasingly different from the reference\-free teacher without a corresponding increase in the overall magnitude of its log\-probability change\.

Figure 5:Magnitude of the Qwen3\-8B student’s log\-probability change and its KL divergence from the reference\-free teacher during training under a thinking teacher\.
### B\.3Policy Entropy

Figure[6](https://arxiv.org/html/2608.18271#A2.F6)shows that entropy increases under a thinking teacher but falls sharply under a non\-thinking teacher\. The near\-identical non\-thinking curves show that replacing the reference does not explain the entropy collapse\.

Figure 6:Policy entropy of the Qwen3\-8B student during training on NuminaMath under thinking and non\-thinking teachers\.
### B\.4Alignment with the Thinking Direction

In Figure[7](https://arxiv.org/html/2608.18271#A2.F7), both the evaluated\-problem and control measurements remain negatively aligned with the thinking direction throughout training\. The evaluated\-problem measurement is less negative than its control, especially on NuminaMath\. Thus, the positive difference between the two measurements reflects weaker opposition to the thinking direction rather than positive alignment with it\.

Figure 7:Cosine similarity between changes in the Qwen3\-8B student’s predictions and the base model’s thinking direction under a non\-thinking teacher\.

## Appendix CResults at Smaller Model Scales

### C\.1Qwen3\-4B Performance

Table[16](https://arxiv.org/html/2608.18271#A3.T16)repeats the main performance comparison with Qwen3\-4B using the same training and evaluation procedure as for Qwen3\-8B\. On NuminaMath, the correct reference improves the thinking\-teacher student on all three competition mathematics benchmarks, with paired bootstrap intervals excluding zero\. Under a non\-thinking teacher, the mismatched reference outperforms the correct reference on the same benchmarks\. On MegaScience, removing the reference does not reduce thinking\-teacher performance\.

Table 16:Qwen3\-4B benchmark results under non\-thinking evaluation\. Evaluation reads follow Table[1](https://arxiv.org/html/2608.18271#S4.T1)\.
### C\.2Qwen3\-1\.7B Performance

Table[17](https://arxiv.org/html/2608.18271#A3.T17)repeats the comparison with Qwen3\-1\.7B under the same evaluation protocol\. On NuminaMath, the correct reference improves the thinking\-teacher student on AIME 2024 and HMMT 2025, with paired bootstrap intervals excluding zero; the interval for AIME 2025 includes zero\. Under a non\-thinking teacher, the mismatched reference outperforms the correct reference on all three competition mathematics benchmarks\. On MegaScience, removing the reference improves thinking\-teacher performance on all three science benchmarks\.

Table 17:Qwen3\-1\.7B benchmark results under non\-thinking evaluation\. Evaluation reads follow Table[1](https://arxiv.org/html/2608.18271#S4.T1)\.
### C\.3Distributional Results Across Model Scales

We repeat the cosine\-similarity analysis at 8B, 4B, and 1\.7B\. Table[18](https://arxiv.org/html/2608.18271#A3.T18)reports each cosine similarity and its difference from the corresponding control across three control assignments\. The checkpoint used for each measurement is listed in the table\.

Table 18:Cosine similarities at 8B, 4B, and 1\.7B\. Bracketed entries give the range across three control assignments, not confidence intervals\.\(a\) Thinking teacher

\(b\) Non\-thinking teacher