Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation

# Prompt-Level Teacher Gating for On-Policy Distillation
Source: [https://arxiv.org/html/2609.02998](https://arxiv.org/html/2609.02998)
## Verify Before You Distill: Prompt\-Level Teacher Gating for On\-Policy Distillation

\\setcitestyle

numbers,square,comma,sortcompress

\\fancyhead

\\outfitVerify Before You Distill: Prompt\-Level Teacher Gating for On\-Policy DistillationAllSpark TeamOn\-policy distillation \(OPD\) accelerates post\-training by providing dense token\-level supervision from a frozen teacher on the student’s own rollouts\. Vanilla OPD applies this supervision uniformly across prompts, without checking whether the teacher is reliable for each prompt\. Because reverse KL is mode\-seeking, a confidently wrong teacher can induce a strong yet misleading update\. Distributional proxies, such as entropy or teacher–student likelihood agreement, measure uncertainty or agreement but do not directly verify outcome correctness\. We introduce Teacher\-Gated On\-Policy Distillation \(TGOPD\), built on the principle that teacher reliability should be verified at the prompt level before dense supervision is admitted\. TGOPD estimates reliability from a small set of verifier\-scored teacher probes and routes each prompt exclusively to dense OPD when the reliability check passes or to verifier\-grounded GRPO otherwise\. Across 4B and 35B students in mathematics, code, and instruction following, TGOPD outperforms Vanilla OPD in all six single\-domain settings and achieves higher seven\-benchmark averages at both scales under multi\-domain training\. By using otherwise\-idle teacher capacity for reliability estimation, TGOPD also reduces teacher\-side compute waste in asynchronous OPD, increasing teacher\-node GPU utilization from9\.8%9\.8\\%to78\.9%78\.9\\%in the measured 4B single\-domain run\.\\outfitDate:August 28, 2026Status:Public Technical Report

## 1Introduction

Figure 1:Teacher\-side GPU underutilization in asynchronous OPD\.\(a\)Under Vanilla OPD, teacher\-node utilization remains below5%5\\%for59%59\\%of the measured hour; TGOPD uses this otherwise\-idle capacity for reliability probes\.\(b\)Mean teacher\-node utilization increases from9\.8%9\.8\\%to78\.9%78\.9\\%\.On\-policy distillation \(OPD\) is a compute\-efficient approach to post\-training language models\\citepagarwal2024gkd,gu2024minillm,thinkingmachines2025opd\. Reinforcement learning with verifiable rewards \(RLVR\) assigns a single scalar reward to an entire trajectory\. In contrast, OPD samples rollouts from the student and uses a stronger teacher to provide a reverse\-KL learning signal at every token, transforming sparse outcome supervision into dense per\-token guidance\. This dense feedback can enable the student to reach teacher\-level accuracy roughly an order of magnitude faster than RLVR\\citepthinkingmachines2025opd\.

Despite this training efficiency, asynchronous OPD leaves much of the teacher’s compute idle\. In a typical asynchronous deployment, the student generates rollouts on dedicated inference nodes, while a stronger, domain\-specialized frozen teacher scores completed rollouts on a separate node\. A scoring pass requires only a forward evaluation over tokens that have already been generated\. It is therefore much cheaper than autoregressive decoding and cannot begin until a batch of student rollouts is ready, leaving the teacher’s accelerators idle between scoring batches\. Figure[1](https://arxiv.org/html/2609.02998#S1.F1)quantifies this inefficiency in a 4B run: the teacher node averages9\.8%9\.8\\%GPU utilization and remains below5%5\\%utilization for59%59\\%of the measured hour, while the rollout and training nodes remain busy\. This observation motivates using the otherwise\-idle teacher capacity to improve the training signal itself\.

Figure 2:Teacher confidence does not consistently indicate prompt\-level reliability\.This diagnostic uses ten offline teacher responses for each of2,4002\{,\}400prompts per domain;qT\(10\)q\_\{T\}^\{\(10\)\}is their verifier pass rate and is distinct from theKT=3K\_\{T\}=3online estimator used for training\.\(a,b\)Teacher self\-confidence under higher \(qT\(10\)≥0\.7q\_\{T\}^\{\(10\)\}\\\!\\geq\\\!0\.7\) and lower \(qT\(10\)<0\.7q\_\{T\}^\{\(10\)\}\\\!<\\\!0\.7\) reliability\. Confidence separates the groups at AUROC0\.510\.51on code and0\.730\.73on math\.\(c\)In the low\-reliability regime \(qT\(10\)<0\.5q\_\{T\}^\{\(10\)\}\\\!<\\\!0\.5\), the teacher’s highest\-confidence sampled response is incorrect84%84\\%of the time on code and61%61\\%on math\.The idle capacity can be used to address a more fundamental weakness of Vanilla OPD: teacher supervision is admitted without a prompt\-level reliability check\. Reverse KL is mode\-seeking and concentrates the student on the teacher’s high\-probability behavior\\citepgu2024minillm,thinkingmachines2025opd\. This property makes OPD efficient when the teacher is reliable, but it can also amplify confident errors\. Reliability varies by prompt: a globally informative teacher signal may not be locally exploitable\\citeprethinkingopd2026, and a stronger teacher can induce negative transfer into a smaller student\\citepsmallmodels2025\. Domain routing alone does not resolve this issue because existing multi\-teacher frameworks use the selected teacher’s dense reward without verifying its reliability on the particular prompt\\citepmimo2026v2flash\.

Reliability\-aware OPD has developed along two complementary directions\. The first uses distributional evidence\. EOPD introduces forward KL at high\-entropy teacher tokens\\citepeopd2026\. TrOPD defines token\-level trust regions from teacher–student decoding agreement\\citeptropd2026\. REOPOLD clips and samples token rewards using likelihood ratios and student entropy\\citepreopold2026\. These signals capture uncertainty, compatibility, or optimization risk, but they do not directly test whether a teacher answer is correct\. Outcome evidence has also been used to regulate more local decisions\. RG\-OPD retains trajectory\-level distillation when verifier feedback agrees with the teacher–student likelihood gap\\citeprgopd2026\. RLSD uses environmental correctness to determine update direction while self\-distillation modulates its magnitude\\citeprlsd2026\. At the token level, SPOT evaluates teacher\-proposed branches through verifier\-scored student continuations\\citepspot2026\. Together, these methods show how outcome feedback can control individual trajectories or token branches\. TGOPD instead uses repeated teacher outcomes to make a prompt\-level admission decision before dense supervision is applied\.

Figure[2](https://arxiv.org/html/2609.02998#S1.F2)motivates this prompt\-level decision\. For the diagnostic, we draw ten offline teacher responses per prompt and defineqT\(10\)​\(x\)q\_\{T\}^\{\(10\)\}\(x\)as their verifier pass rate\. The larger sample count gives a finer\-grained analysis than the three fresh online probes used during training; both quantities measure the same verifier\-defined teacher reliability\. On code, self\-confidence barely separates higher\- and lower\-reliability prompts \(AUROC0\.510\.51\), compared with0\.730\.73on math\. Within the low\-reliability regime, the teacher’s highest\-confidence sampled response is still incorrect84%84\\%of the time on code and61%61\\%on math\. Because reverse KL emphasizes high\-probability teacher behavior, this is precisely the type of error that Vanilla OPD can propagate\.

Teacher\-Gated On\-Policy Distillation \(TGOPD\) uses the teacher’s idle capacity to estimate prompt\-level reliability before selecting a supervision signal\. As shown in Figure[3](https://arxiv.org/html/2609.02998#S2.F3), the teacher generatesKTK\_\{T\}probe rollouts for the same prompt; a verifier scores them; and their pass rateqT​\(x\)=KT−1​∑krkq\_\{T\}\(x\)=K\_\{T\}^\{\-1\}\\sum\_\{k\}r\_\{k\}provides the online reliability estimate\. IfqT​\(x\)≥τq\_\{T\}\(x\)\\\!\\geq\\\!\\tau, TGOPD uses dense OPD alone\. Otherwise, it rejects the teacher signal and uses verifier\-grounded GRPO alone, provided that the student rollout group contains reward variation\. The two branches are routed rather than blended\. Because the probes run primarily while the teacher would otherwise be idle, teacher\-node GPU utilization rises from9\.8%9\.8\\%to78\.9%78\.9\\%and cluster utilization from51\.5%51\.5\\%to69\.5%69\.5\\%, with modest measured end\-to\-end overhead\.

Across 4B and 35B students in mathematics, code, and instruction following, TGOPD outperforms Vanilla OPD in all six domain–scale settings\. The largest gains occur on code, where confidence is least informative about teacher reliability\. These results support the design choice: verify the teacher at the prompt level, retain its dense signal when the reliability check passes, and withdraw that signal when the check fails\.

## 2Preliminaries

#### Setup and notation\.

Letπθ\\pi\_\{\\theta\}denote the student policy andπT\\pi\_\{T\}a frozen teacher; in a multi\-teacher setting,πT=πT,dom⁡\(x\)\\pi\_\{T\}=\\pi\_\{T,\\operatorname\{dom\}\(x\)\}is the teacher routed to the domain of promptxx\. For each promptx∼𝒟x\\sim\\mathcal\{D\}, the student samples a group ofGGon\-policy rollouts\{yi\}i=1G∼πθ\(⋅∣x\)\\\{y^\{i\}\\\}\_\{i=1\}^\{G\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid x\), and a binary verifierr⁡\(x,y\)∈\{0,1\}r\(x,y\)\\in\\\{0,1\\\}scores each rollout against a ground\-truth outcome: unit tests for code and a rule\-based judge for mathematics and instruction following\. We writeytiy^\{i\}\_\{t\}for thett\-th token of rolloutiiandsg⁡\[⋅\]\\operatorname\{sg\}\[\\cdot\]for the stop\-gradient operator\.

#### On\-policy distillation \(OPD\)\.

OPD supervises every token of the student’s own rollouts by minimizing the mode\-seeking reverse KL𝕂𝕃\(πθ∥πT\)\\mathbb\{KL\}\(\\pi\_\{\\theta\}\\,\\\|\\,\\pi\_\{T\}\)\. Estimated from the single sampled token at each position, this objective takes the form of a policy\-gradient update whose per\-token advantage is the teacher–student log\-likelihood ratio\\citepgu2024minillm,thinkingmachines2025opd,mimo2026v2flash:

A^i,tOPD=β​sg⁡\[log⁡πT​\(yti∣x,y<ti\)−log⁡πθ​\(yti∣x,y<ti\)\],\\hat\{A\}^\{\\mathrm\{OPD\}\}\_\{i,t\}=\\beta\\operatorname\{sg\}\\\!\\left\[\\log\\pi\_\{T\}\\\!\\left\(y^\{i\}\_\{t\}\\mid x,y^\{i\}\_\{<t\}\\right\)\-\\log\\pi\_\{\\theta\}\\\!\\left\(y^\{i\}\_\{t\}\\mid x,y^\{i\}\_\{<t\}\\right\)\\right\],\(1\)whereβ\>0\\beta\>0sets the scale of the teacher signal\. Every token therefore receives its own dense, individually weighted learning signal, which contributes to OPD’s sample efficiency relative to learning from a single trajectory\-level reward\. Being mode\-seeking, the objective concentrates the student on the teacher’s highest\-probability behavior rather than spreading probability mass over alternatives\. Note that equation[1](https://arxiv.org/html/2609.02998#S2.E1)depends only onπT\\pi\_\{T\}andπθ\\pi\_\{\\theta\}, not on the verifierrr: it measures how far the student is from the teacher but does not directly assess whether the teacher’s output is correct\. WhenπT\\pi\_\{T\}is unreliable onxx, the resulting dense gradient can therefore be misleading\.

#### GRPO with verifiable rewards\.

When teacher supervision is unavailable or withheld, the student can instead learn from the verifier alone\. GRPO\\citepdeepseekmath2024 centers the reward within each group; we omit the standard\-deviation normalization\\citepliu2025drgrpo:

A^iGRPO=r\(x,yi\)−1G∑j=1Gr\(x,yj\),A^i,tGRPO=A^iGRPO∀t\.\\hat\{A\}^\{\\mathrm\{GRPO\}\}\_\{i\}=r\\\!\\left\(x,y^\{i\}\\right\)\-\\frac\{1\}\{G\}\\sum\_\{j=1\}^\{G\}r\\\!\\left\(x,y^\{j\}\\right\),\\qquad\\hat\{A\}^\{\\mathrm\{GRPO\}\}\_\{i,t\}=\\hat\{A\}^\{\\mathrm\{GRPO\}\}\_\{i\}\\quad\\forall\\,t\.\(2\)Unlike the token\-specific OPD advantage in equation[1](https://arxiv.org/html/2609.02998#S2.E1), this signal assigns the same scalar advantage to every token in a rollout\. It is therefore trajectory\-level rather than token\-specific, but remains verifier\-grounded: its sign is determined by the verifier outcome relative to the group mean\.

#### The gap\.

Vanilla OPD provides dense token\-level supervision without directly verifying teacher reliability, whereas GRPO provides verifier\-grounded but coarse trajectory\-level supervision\. Vanilla OPD and GRPO\-only training each apply a fixed supervision rule uniformly across prompts, even though teacher reliability—and hence the preferable learning signal—may vary from prompt to prompt\. TGOPD, described next, makes this choice per prompt by applying the same verifier to teacher probes as well as student rollouts\. It routes each prompt to exactly one signal; OPD and GRPO are never blended on the same prompt\.

![Refer to caption](https://arxiv.org/html/2609.02998v1/tgopd_framework_v2.png)Figure 3:TGOPD overview\.\(A\)Vanilla OPD: the student generatesGGon\-policy rollouts, and the frozen teacher scores every token\. The resulting teacher–student log\-probability gap provides dense token\-level supervision \(equation[1](https://arxiv.org/html/2609.02998#S2.E1)\) on every prompt\.\(B\)TGOPD: while the student decodes, the teacher uses its otherwise\-idle capacity to generate a small set of probe rollouts\. A verifier scores these probes, and their pass rateqT​\(x\)q\_\{T\}\(x\)\(equation[4](https://arxiv.org/html/2609.02998#S3.E4)\) estimates prompt\-level teacher reliability\. Teacher scoring and routine verifier evaluation of student rollouts make both candidate advantages available; the latter is omitted from the diagram for visual clarity\. The gate selects dense OPD whenqT​\(x\)≥τq\_\{T\}\(x\)\\geq\\tauand verifier\-grounded GRPO \(equation[2](https://arxiv.org/html/2609.02998#S2.E2)\) otherwise\. The two supervision branches are mutually exclusive\.

TGOPD represents teacher reliability as a per\-prompt quantity and uses it to choose between two supervision regimes\. A verifier audits the teacher on the current prompt\. A successful audit selects dense OPD; otherwise, teacher supervision is withheld and the update uses verifier\-grounded GRPO\. The two signals are never added or interpolated\. Figure[3](https://arxiv.org/html/2609.02998#S2.F3)contrasts this procedure with vanilla OPD\. In panel A, the teacher’s log\-probabilities enter every update\. In panel B, they enter only after the audit passes; a verifier\-grounded signal is used when the audit fails\. Section[3\.1](https://arxiv.org/html/2609.02998#S3.SS1)defines the reliability estimate and the gate it drives, §[3\.2](https://arxiv.org/html/2609.02998#S3.SS2)gives the objective the gate induces, and §[3\.3](https://arxiv.org/html/2609.02998#S3.SS3)shows how the audit overlaps with the asynchronous training loop’s otherwise\-idle teacher window\.

### 3\.1Teacher Reliability Probe and Gate

The gate requires an estimate of teacher reliability for each prompt, which the vanilla OPD loop does not provide\. We define this quantity below and estimate it with a small number of teacher samples\.

#### Reliability and its estimator\.

We define the teacher’s reliability on a prompt as its expected verifier reward,

RT\(x\)=𝔼y∼πT\(⋅∣x\)\[r\(x,y\)\]∈\[0,1\],R\_\{T\}\(x\)\\;=\\;\\mathbb\{E\}\_\{y\\sim\\pi\_\{T\}\(\\cdot\\mid x\)\}\\big\[r\(x,y\)\\big\]\\;\\in\\;\[0,1\],\(3\)that is, the probability that a sample from the teacher solvesxx\. SinceRTR\_\{T\}is not available in closed form, we estimate it with a small teacher probe: the teacher independently generatesKTK\_\{T\}rollouts\{y^k\}k=1KT∼πT\(⋅∣x\)\\\{\\hat\{y\}^\{k\}\\\}\_\{k=1\}^\{K\_\{T\}\}\\sim\\pi\_\{T\}\(\\cdot\\mid x\)on the same prompt\. The verifier used for student rollouts scores each teacher rollout, and their empirical pass rate is \(Figure[3](https://arxiv.org/html/2609.02998#S2.F3)B\)

qT​\(x\)=1KT​∑k=1KTr⁡\(x,y^k\)\.q\_\{T\}\(x\)\\;=\\;\\frac\{1\}\{K\_\{T\}\}\\sum\_\{k=1\}^\{K\_\{T\}\}r\\\!\\left\(x,\\hat\{y\}^\{k\}\\right\)\.\(4\)Because the probes are i\.i\.d\. draws fromπT\(⋅∣x\)\\pi\_\{T\}\(\\cdot\\mid x\)andrris binary,KT​qT​\(x\)∼Binomial⁡\(KT,RT​\(x\)\)K\_\{T\}\\,q\_\{T\}\(x\)\\sim\\mathrm\{Binomial\}\\\!\\left\(K\_\{T\},R\_\{T\}\(x\)\\right\)\. The estimator is therefore unbiased,𝔼⁡\[qT​\(x\)\]=RT​\(x\)\\mathbb\{E\}\[q\_\{T\}\(x\)\]=R\_\{T\}\(x\), for any probe budget, with varianceRT​\(x\)​\(1−RT​\(x\)\)/KTR\_\{T\}\(x\)\\big\(1\-R\_\{T\}\(x\)\\big\)/K\_\{T\}\. Unbiasedness alone does not imply that a particular finite probe budget yields reliable accept/reject decisions\. We empirically findKT=3K\_\{T\}=3sufficient in our main experiments; Section[4\.5](https://arxiv.org/html/2609.02998#S4.SS5)uses a larger budget to examine the threshold at a finer resolution\.

#### Verifier\-grounded reliability signal\.

The estimatorqT​\(x\)q\_\{T\}\(x\)uses task outcomes rather than the teacher’s own confidence\. Entropy and teacher–student likelihood agreement are functions of model distributions alone: they can quantify uncertainty or compatibility, but cannot by themselves distinguish equally confident correct and incorrect answers\. Evaluatingrron completed teacher outputs provides direct outcome evidence\. Unlike verifier\-aware token or trajectory gates,qT​\(x\)q\_\{T\}\(x\)aggregates repeated teacher outcomes into a prompt\-level reliability estimate before the supervision branch is selected\.

#### The reliability gate\.

A hard gate admits the teacher only when its estimated reliability reaches a thresholdτ∈\(0,1\]\\tau\\in\(0,1\]\(Figure[3](https://arxiv.org/html/2609.02998#S2.F3)B, “reliability gate”\):

g\(x\)=\[qT\(x\)≥τ\]∈\{0,1\}\.g\(x\)\\;=\\;\\mathbb\{1\}\\\!\\left\[\\,q\_\{T\}\(x\)\\;\\geq\\;\\tau\\,\\right\]\\;\\in\\;\\\{0,1\\\}\.\(5\)SinceqT​\(x\)q\_\{T\}\(x\)takes values in\{0,1/KT,…,1\}\\\{0,1/K\_\{T\},\\dots,1\\\}, the gate opens exactly when at least⌈τ​KT⌉\\lceil\\tau K\_\{T\}\\rceilof theKTK\_\{T\}probes pass; only the pair\(⌈τ​KT⌉,KT\)\\big\(\\lceil\\tau K\_\{T\}\\rceil,\\,K\_\{T\}\\big\), and notτ\\tauin isolation, is operationally meaningful\. We useKT=3K\_\{T\}=3andτ=2/3\\tau=2/3in all main experiments, i\.e\. a two\-of\-three majority\. Section[4\.5](https://arxiv.org/html/2609.02998#S4.SS5)sweeps the threshold atKT=5K\_\{T\}=5and finds a broad optimum around a simple majority\. This result suggests that a coarse reliability decision is sufficient; the gate need not rank prompts precisely\.

### 3\.2Gate\-Conditioned Supervision Routing

For each prompt, the gate selects one of the two supervision signals defined in §[2](https://arxiv.org/html/2609.02998#S2)\. When it is open, the dense OPD signal controls the update and the trajectory\-level verifier reward does not enter the gradient\. When the gate is closed, teacher supervision is withheld and the verifier\-grounded GRPO signal controls the update\. Withholding teacher supervision therefore does not necessarily discard the training prompt\. Whenever the student’s rollout group contains reward variation, the group\-relative fallback assigns positive advantage to above\-average attempts and negative advantage to below\-average ones \(Figure[3](https://arxiv.org/html/2609.02998#S2.F3)B, lower signal box\)\. If every rollout receives the same reward, however, the centered GRPO advantage is exactly zero and that prompt produces no update\. For pipeline regularity, both candidate advantages are formed for every prompt before the update\. This eager computation does not imply additive supervision: the binary gate selects exactly one candidate, giving the per\-token advantage

A^i,tTGOPD=g⁡\(x\)​A^i,tOPD\+\(1−g⁡\(x\)\)​A^i,tGRPO\.\\hat\{A\}^\{\\mathrm\{TGOPD\}\}\_\{i,t\}\\;=\\;g\(x\)\\,\\hat\{A\}^\{\\mathrm\{OPD\}\}\_\{i,t\}\\;\+\\;\\big\(1\-g\(x\)\\big\)\\,\\hat\{A\}^\{\\mathrm\{GRPO\}\}\_\{i,t\}\.\(6\)Becauseg⁡\(x\)∈\{0,1\}g\(x\)\\in\\\{0,1\\\}, equation[6](https://arxiv.org/html/2609.02998#S3.E6)is a selector rather than an interpolation: OPD and GRPO never jointly supervise the same prompt\. Vanilla OPD and pure GRPO are its two degenerate endpoints, obtained by fixingg⁡\(x\)=1g\(x\)=1andg⁡\(x\)=0g\(x\)=0, respectively, for every prompt\.

The resulting advantage enters the standard clipped policy\-gradient surrogate\. Letπold\\pi\_\{\\mathrm\{old\}\}denote the frozen rollout\-policy snapshot and defineρi,t​\(θ\)=πθ​\(yti∣x,y<ti\)/πold​\(yti∣x,y<ti\)\\rho\_\{i,t\}\(\\theta\)=\\pi\_\{\\theta\}\(y^\{i\}\_\{t\}\\mid x,y^\{i\}\_\{<t\}\)/\\pi\_\{\\mathrm\{old\}\}\(y^\{i\}\_\{t\}\\mid x,y^\{i\}\_\{<t\}\)\. We optimize

ℒTGOPD​\(θ\)=−𝔼x∼𝒟\{yi\}i=1G∼πold\(⋅∣x\)​\[1∑i\|yi\|​∑i=1G∑t=1\|yi\|min⁡\(ρi,t​\(θ\)​A^i,tTGOPD,clip⁡\(ρi,t​\(θ\),1−ϵ,1\+ϵ\)​A^i,tTGOPD\)\]\.\\mathcal\{L\}\_\{\\mathrm\{TGOPD\}\}\(\\theta\)=\-\\,\\mathbb\{E\}\_\{\\begin\{subarray\}\{c\}x\\sim\\mathcal\{D\}\\\\ \\\{y^\{i\}\\\}\_\{i=1\}^\{G\}\\sim\\pi\_\{\\mathrm\{old\}\}\(\\cdot\\mid x\)\\end\{subarray\}\}\\left\[\\frac\{1\}\{\\sum\_\{i\}\|y^\{i\}\|\}\\sum\_\{i=1\}^\{G\}\\sum\_\{t=1\}^\{\|y^\{i\}\|\}\\min\\\!\\left\(\\rho\_\{i,t\}\(\\theta\)\\hat\{A\}^\{\\mathrm\{TGOPD\}\}\_\{i,t\},\\operatorname\{clip\}\\\!\\left\(\\rho\_\{i,t\}\(\\theta\),1\-\\epsilon,1\+\\epsilon\\right\)\\hat\{A\}^\{\\mathrm\{TGOPD\}\}\_\{i,t\}\\right\)\\right\]\.\(7\)The advantage is held fixed within the update\. The asynchronous implementation additionally uses IcePop to suppress excessive train–inference mismatch; this system\-level correction does not alter the supervision\-routing objective above \(§[4\.1](https://arxiv.org/html/2609.02998#S4.SS1)\)\.

#### Un\-normalized fallback\.

Our implementation omits the standard\-deviation normalization of GRPO\\citepliu2025drgrpo\. This avoids rescaling the fallback by the within\-group reward standard deviation\. We use it as an implementation convention, not as a requirement of the gate\. A controlled comparison with normalized variants is left to future work\.

#### Design choices\.

The gate acts at the prompt level: a prompt is supervised either entirely by OPD or entirely by the verifier\-grounded fallback, never by a per\-token or additive mixture\. This avoids entangling two signals of different density and provenance within a single trajectory, where the effective supervision would depend on an arbitrary interpolation weight\. The gate also withdraws teacher supervision only where the audit fails, so on the majority of prompts the full direction and magnitude of the dense OPD signal survives intact\. Section[4\.2](https://arxiv.org/html/2609.02998#S4.SS2)shows the cost of reducing the teacher’s role on every prompt rather than only on those that fail the audit\.

### 3\.3Algorithm and System Realization

Algorithm[1](https://arxiv.org/html/2609.02998#alg1)summarizes one rollout\-update cycle\. Relative to vanilla OPD, TGOPD retains the same student rollout and teacher scoring operations and adds the probe in equation[4](https://arxiv.org/html/2609.02998#S3.E4), which determines the gate in equation[5](https://arxiv.org/html/2609.02998#S3.E5)\. Most of this added work runs in the shaded*idle teacher compute*region of Figure[3](https://arxiv.org/html/2609.02998#S2.F3)B; the measured residual overhead is summarized in Appendix[D](https://arxiv.org/html/2609.02998#A4)\.

Algorithm 1TGOPD: one asynchronous rollout–update cycle1:current student

πθ\\pi\_\{\\theta\}; frozen rollout snapshot

πold\\pi\_\{\\mathrm\{old\}\}; frozen teacher

πT\\pi\_\{T\}; verifier

rr
2:prompt batch

ℬ\\mathcal\{B\}; group size

GG; probe budget

KTK\_\{T\}; threshold

τ\\tau; PPO clip

ϵ\\epsilon; step size

η\\eta
3:updated student parameters

θ\\theta
4:foreach prompt

x∈ℬx\\in\\mathcal\{B\}do

5:concurrently:⊳\\trianglerightprobe fills the teacher’s idle window

6:*rollout nodes:*

\{yi\}i=1G∼πold\(⋅∣x\)\\\{y^\{i\}\\\}\_\{i=1\}^\{G\}\\sim\\pi\_\{\\mathrm\{old\}\}\(\\cdot\\mid x\); record rollout log\-probs⊳\\trianglerightnominal critical path

7:*teacher node:*

\{y^k\}k=1KT∼πT\(⋅∣x\)\\\{\\hat\{y\}^\{k\}\\\}\_\{k=1\}^\{K\_\{T\}\}\\sim\\pi\_\{T\}\(\\cdot\\mid x\)
8:

qT​\(x\)←1KT​∑k=1KTr⁡\(x,y^k\)q\_\{T\}\(x\)\\leftarrow\\frac\{1\}\{K\_\{T\}\}\\sum\_\{k=1\}^\{K\_\{T\}\}r\\\!\\left\(x,\\hat\{y\}^\{k\}\\right\)⊳\\trianglerightreliability estimate, equation[4](https://arxiv.org/html/2609.02998#S3.E4)

9:*// both candidate signals are formed for every prompt*

10:evaluate

log⁡πT​\(yti∣x,y<ti\)\\log\\pi\_\{T\}\\\!\\left\(y^\{i\}\_\{t\}\\mid x,y^\{i\}\_\{<t\}\\right\)for all

i,ti,t⊳\\trianglerightteacher scoring pass

11:score the student rollouts:

r⁡\(x,yi\)r\\\!\\left\(x,y^\{i\}\\right\)for

i=1,…,Gi=1,\\dots,G⊳\\trianglerightthe RLVR signal

12:if

qT​\(x\)≥τq\_\{T\}\(x\)\\geq\\tauthen⊳\\trianglerightteacher supervision admitted

13:

A^i,t←A^i,tOPD\\hat\{A\}\_\{i,t\}\\leftarrow\\hat\{A\}^\{\\mathrm\{OPD\}\}\_\{i,t\}for all

i,ti,t⊳\\trianglerightequation[1](https://arxiv.org/html/2609.02998#S2.E1)

14:else⊳\\trianglerightteacher supervision withheld

15:

A^i,t←A^i,tGRPO\\hat\{A\}\_\{i,t\}\\leftarrow\\hat\{A\}^\{\\mathrm\{GRPO\}\}\_\{i,t\}for all

i,ti,t⊳\\trianglerightequation[2](https://arxiv.org/html/2609.02998#S2.E2)

16:endif

17:endfor

18:form

ρi,t​\(θ\)\\rho\_\{i,t\}\(\\theta\)from current and recorded rollout log\-probs

19:

θ←θ−η​∇θℒTGOPD​\(θ\)\\theta\\leftarrow\\theta\-\\eta\\,\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{TGOPD\}\}\(\\theta\)over

ℬ\\mathcal\{B\}⊳\\trianglerightequation[7](https://arxiv.org/html/2609.02998#S3.E7)

#### Overlapping probes with teacher idle time\.

In an asynchronous deployment, the teacher’s standard task is a forward pass over already\-generated tokens\. This is far cheaper than the student’s autoregressive decoding and cannot begin until a batch of student rollouts is ready, so the teacher node idles for most of each cycle \(§[4\.4](https://arxiv.org/html/2609.02998#S4.SS4)\)\. TGOPD issues theKTK\_\{T\}probe rollouts at the start of the cycle, concurrently with student generation, so most probe work occupies this window\. If allKTK\_\{T\}decodes finish before the student\-side batch completes, the probe adds no wall\-clock cost; in practice the overlap is substantial but not perfect \(Appendix[D](https://arxiv.org/html/2609.02998#A4)\)\.

#### Scoring stays unconditional\.

Teacher scoring runs on every prompt, exactly as under vanilla OPD, and the verifier scores the student’s rollouts because that reward is the reinforcement\-learning signal in any case\. The scoring pass is a single forward evaluation over tokens the student has already produced\. It is batched as rollouts arrive and runs on a node that is not the throughput bottleneck\. Making this pass conditional for a minority of prompts would add a data\-dependent pipeline branch while saving little computation\. The additional work in TGOPD therefore consists of theKTK\_\{T\}probe decodes, which is why teacher utilization can rise from9\.8%9\.8\\%to78\.9%78\.9\\%with only modest end\-to\-end overhead \(§[4\.4](https://arxiv.org/html/2609.02998#S4.SS4)\): most audit work consumes capacity that was previously wasted, while the measured remainder slightly extends the cycle\.

## 4Experiments

### 4\.1Experimental Setup

#### Models and teachers\.

We evaluate at two scales: a dense 4B student \(Qwen3\.5\-4B\) and a mixture\-of\-experts 35B student \(Qwen3\.6\-35B\-A3B, with 3B active parameters\)\. For each of three domains \(mathematics, code, and instruction following, or IF\), we train a dedicated domain\-specialist teacher by applying GRPO\\citepdeepseekmath2024 to the same base model\. Teachers are frozen after training and used to generate reliability probes and score student rollouts; their own evaluation scores provide both a teacher\-performance reference and a GRPO\-only reference on the same architecture \(the “Teacher” row in Table[1](https://arxiv.org/html/2609.02998#S4.T1)\)\. All runs use the slime framework\\citepslime\_github with an asynchronous rollout–update loop that overlaps teacher scoring with student generation\. Student updates use the standard PPO\-clipped surrogate in equation[7](https://arxiv.org/html/2609.02998#S3.E7)\. The implementation additionally applies IcePop\\citepring1t2025 to stabilize the train–inference probability mismatch introduced by asynchronous rollout; this correction is shared by all compared methods and is not part of the TGOPD gate\. Within TGOPD, the OPD and GRPO advantages are mutually exclusive: the former is used only when the teacher passes the prompt\-level audit, and the latter only when it fails \(equation[6](https://arxiv.org/html/2609.02998#S3.E6)\)\. Appendix[B](https://arxiv.org/html/2609.02998#A2)summarizes the recorded optimization and infrastructure settings\.

#### Baselines\.

All comparisons share the same student initialization, frozen domain teacher, training corpus, and training pipeline\. Vanilla OPD\\citepthinkingmachines2025opd applies the sampled\-token reverse\-KL advantage in equation[1](https://arxiv.org/html/2609.02998#S2.E1)to every prompt\. TrOPD\\citeptropd2026 uses an adaptive token\-level trust region, applying reverse KL within the region, forward KL to outliers, and teacher\-prefix off\-policy guidance\. RG\-OPD\\citeprgopd2026 retains trajectory\-level distillation only when the verifier\-derived advantage agrees with the teacher–student likelihood gap; inconsistent trajectories are omitted from its distillation objective\. We also include an RLSD\-style baseline inspired by RLSD\\citeprlsd2026: the verifier\-derived advantage determines update direction, while the same external frozen teacher used by the other baselines rescales token\-level magnitude\. This variant adopts RLSD’s direction–magnitude decomposition without reproducing its privileged\-context self\-distillation setting\.

#### Training data and evaluation\.

Mathematics prompts are drawn from DAPO\-Math\-17K\\citepyu2026dapo; code prompts are the input–output prediction questions of CodeI/O\\citepli2025codeio; and IF prompts are filtered from Nemotron\-Cascade 2\\citepyang2026nemotron\. Each method is trained per domain from the same student initialization and the same prompt pool\. We report in\-domain benchmarks: AIME 2025 and AIME 2026 \(accuracy, 64\-run average\), HMMT\-Feb 2025 \(accuracy, 32\-run average\) for mathematics; the code\-generation subtask of LiveCodeBench \(pass@1, 6\-run average\) and OJBench overall \(C\+\+/Python aggregate\) for code; and IFBench and IFEval for instruction following\. Appendix[C](https://arxiv.org/html/2609.02998#A3)summarizes the evaluation protocols and the number of independent evaluation generations for each benchmark\.

### 4\.2Main Results

Table 1:In\-domain distillation results across two model scales and three domains\. Each domain group is an independent experiment with its own GRPO\-trained teacher \(πT\\pi\_\{T\}\), training corpus, and in\-domain evaluation benchmarks\. The*Teacher*row reports the domain teacher’s own scores, serving as both a teacher\-performance reference and the GRPO\-only reference\.Bold: best among distillation methods per column\.Underline: surpasses the teacher\.MathCodeIF\(πTmath\\pi\_\{T\}^\{\\,\\text\{math\}\}\)\(πTcode\\pi\_\{T\}^\{\\,\\text\{code\}\}\)\(πTIF\\pi\_\{T\}^\{\\,\\text\{IF\}\}\)MethodAIME 2025AIME 2026HMMT\-FebLiveCodeBenchOJBenchIFBenchIFEval*Qwen3\.5\-4B*Base Model47\.858\.140\.539\.414\.435\.985\.3Teacher63\.473\.454\.853\.317\.055\.985\.3\[2pt/3pt\] Vanilla OPD61\.171\.254\.342\.318\.848\.583\.9TrOPD62\.170\.556\.849\.618\.153\.284\.6RG\-OPD63\.573\.654\.145\.619\.454\.384\.1RLSD\-style48\.856\.740\.338\.415\.131\.579\.8TGOPD \(ours\)64\.873\.552\.747\.120\.050\.485\.2*Qwen3\.6\-35B\-A3B*Base Model69\.874\.464\.361\.026\.130\.986\.8Teacher76\.379\.865\.362\.727\.652\.190\.8\[2pt/3pt\] Vanilla OPD73\.277\.562\.460\.226\.741\.390\.1TrOPD75\.877\.765\.258\.526\.144\.590\.8RG\-OPD73\.477\.264\.757\.527\.844\.090\.2RLSD\-style71\.976\.661\.256\.925\.438\.387\.1TGOPD \(ours\)74\.078\.764\.164\.028\.743\.791\.5

Table[1](https://arxiv.org/html/2609.02998#S4.T1)reports results across two scales and three independently trained domain experiments\. TGOPD outperforms Vanilla OPD in all six domain×\\timesscale settings, with the largest gains on code \(\+3\.0\+3\.0average at 4B,\+2\.9\+2\.9at 35B\), followed by IF \(\+1\.6\+1\.6/\+1\.9\+1\.9\) and math \(\+1\.5\+1\.5/\+1\.2\+1\.2\)\. This ordering is consistent with the “confidently wrong” failure described in Figure[2](https://arxiv.org/html/2609.02998#S1.F2)\. The code teacher’s errors are virtually indistinguishable from its correct answers by confidence \(AUROC0\.510\.51\), so the harmful dense signal is least detectable in this domain\. Across the full table, TGOPD ranks first among distillation methods on 7 of 14 in\-domain benchmark columns and surpasses the domain teacher on six of them\.

The code results provide the clearest example\. At the 35B scale, every other distillation method causes negative transfer on LiveCodeBench: the student scores below the untrained base model \(OPD−0\.8\-0\.8, TrOPD−2\.5\-2\.5, RG\-OPD−3\.5\-3\.5, RLSD\-style−4\.1\-4\.1\)\. Yet TGOPD is the only method that achieves positive transfer \(\+3\.0\+3\.0over base\) and, in fact, surpasses the teacher itself \(\+1\.3\+1\.3LCB,\+1\.1\+1\.1OJBench\)\. This exceedance is consistent with how the gate partitions the supervision: on prompts that fail the reliability audit, the gate closes and withdraws the dense teacher signal\. When the student’s rollout group contains reward variation, the GRPO fallback can still reinforce the relatively better attempts; for uniform\-outcome groups, its centered advantage is zero\. This prompt\-wise routing performs better than applying either supervision policy uniformly across all prompts in this setting\. At the 4B scale, the pattern is the same in direction though smaller in magnitude: Vanilla OPD closes only21%21\\%of the base\-to\-teacher gap on LCB, while TGOPD closes55%55\\%\. This difference is consistent with the gate removing unreliable teacher signals\.

The RLSD\-style variant uses the teacher only to scale update magnitude and relies on verifier feedback for the update direction\. It applies this rule to all prompts rather than only those on which the teacher is unreliable, and falls below the untrained base model on 8 of 14 in\-domain columns, most severely on IF IFEval \(−5\.5\-5\.5at 4B\) and code LCB \(−4\.1\-4\.1at 35B\)\. TGOPD instead preserves the full OPD direction and magnitude on prompts that pass the audit\. These results suggest that reducing the teacher’s role uniformly can remove useful supervision along with harmful signals\.

### 4\.3Extension to Multi\-Domain OPD

Table 2:TGOPD applied to multi\-domain OPD \(MOPD\)\. Each run trains a single student on math, code, and IF simultaneously, routing prompts to domain\-specialist teachers\. Results are reported at 199 training steps withKT=3,τ=2/3K\_\{T\}\\\!=\\\!3,\\;\\tau\\\!=\\\!2/3\.Bold: better of the two methods per column\.MathCodeIFMethodAvgAIME 25AIME 26HMMT\-FebLCBOJBenchIFBenchIFEval*Qwen3\.5\-4B*MOPD53\.4058\.0763\.7550\.7350\.4818\.1048\.7883\.89MOPD \+ TGOPD54\.5458\.1367\.6051\.2550\.0018\.9751\.7284\.11*Qwen3\.6\-35B\-A3B*MOPD60\.9973\.1277\.2960\.7357\.4323\.2844\.4090\.68MOPD \+ TGOPD61\.9472\.9778\.0262\.4057\.8127\.5943\.9990\.78

The experiments above evaluate TGOPD in the single\-domain setting, where each run trains one student with one domain teacher\. We also consider multi\-domain OPD \(MOPD\)\\citepmimo2026v2flash,mopd2026, in which a single student is trained on math, code, and IF prompts simultaneously, with each prompt routed to a domain\-specialist teacher\. Because the gating decision is per\-prompt and per\-teacher, TGOPD applies to MOPD without modification: the routed teacher generatesKT=3K\_\{T\}\\\!=\\\!3probes on its assigned prompt, the domain verifier scores them, and the gate admits or withholds teacher supervision as before\.

Table[2](https://arxiv.org/html/2609.02998#S4.T2)reports results at 199 training steps\. At the 4B scale, TGOPD improves the seven\-benchmark average from53\.4053\.40to54\.5454\.54\(\+1\.14\+1\.14\), winning 6 of 7 columns; at the 35B scale the average rises from60\.9960\.99to61\.9461\.94\(\+0\.95\+0\.95\), winning 5 of 7 columns\. The largest single\-benchmark gains appear on AIME 2026 \(\+3\.85\+3\.85at 4B\), IFBench \(\+2\.94\+2\.94at 4B\), and OJBench \(\+0\.87\+0\.87at 4B,\+4\.31\+4\.31at 35B\); the code result in particular is consistent with the single\-domain finding, since confidence is least informative about teacher reliability on code\. The few regressions \(LCB at 4B,−0\.48\-0\.48; AIME 2025,−0\.15\-0\.15; IFBench,−0\.41\-0\.41, at 35B\) are all under half a point and within the run\-to\-run variance of early\-training checkpoints\. The average improvements at both scales show that the same per\-prompt gate can be used in multi\-domain distillation without reducing overall quality\.

### 4\.4Compute Efficiency: Reclaiming Idle Teacher Capacity

Figure 4:GPU utilization before and after TGOPD, across four configurations\.\(a\)Split\-violin view of teacher\-node utilization\. Under vanilla OPD the mass piles up near0%0\\%in every configuration; TGOPD shifts the whole distribution upward, showing that probes use the idle window\. The MOPD distributions under TGOPD are visibly bimodal because probing is domain\-routed: the teacher node is partitioned into three domain\-specialist engines and only the engine owning the current prompt’s domain is active, so the node alternates between partial and full occupancy rather than saturating as a single undivided teacher does\.\(b\)Cluster\-average utilization\. Gains track both the teacher’s share of the cluster \(1/51/5at 4B vs\.1/71/7at 35B\) and the saturation the probe achieves \(higher under SOPD than under MOPD\)\.Table 3:GPU utilization before and after TGOPD across four training configurations \(single\-domain OPD \(SOPD\) and multi\-domain OPD \(MOPD\), at both model scales\)\.*Idle*= fraction of 15\-second samples below5%5\\%; each run is a one\-hour window\. Rollout\-node utilization remains within normal run\-to\-run variation in all four cases, showing no sustained resource contention; utilization alone does not establish zero wall\-clock overhead\. In MOPD the single frozen teacher node hosts three domain\-specialist engines \(code 4 GPU, math 2 GPU, IF 2 GPU\); the rollout figure is the mean of the two student\-rollout nodes\.Figure[1](https://arxiv.org/html/2609.02998#S1.F1)showed low teacher utilization at the 4B scale\. We extend the analysis to both model scales and to single\- and multi\-domain OPD, giving four configurations in total\. We also examine whether probe generation affects other workloads\. Table[3](https://arxiv.org/html/2609.02998#S4.T3)and Figure[4](https://arxiv.org/html/2609.02998#S4.F4)report summary statistics; the underlying one\-hour utilization traces for all four configurations are plotted in Appendix[D](https://arxiv.org/html/2609.02998#A4)\.

#### The idle pattern is structural, not configuration\-specific\.

Table[3](https://arxiv.org/html/2609.02998#S4.T3)and Figure[4](https://arxiv.org/html/2609.02998#S4.F4)show the same profile in every configuration\. Without probes, the teacher node averages between5\.0%5\.0\\%and9\.8%9\.8\\%utilization, with5757–78%78\\%of all 15\-second samples falling below5%5\\%\. The cause is architectural rather than incidental\. The teacher’s scoring pass is a single forward evaluation over already\-generated tokens, so it completes in a small fraction of the time the student spends on autoregressive decoding, and it cannot begin until the student finishes\. The teacher therefore waits by construction, at every scale and in both training modes\. Multi\-domain OPD is the more wasteful of the two modes: its teacher node idles for78%78\\%\(4B\) and70%70\\%\(35B\) of samples, against5757–59%59\\%under SOPD\. This follows from how the node is partitioned: three domain\-specialist engines share it, and each runs only on prompts routed to its own domain\. Any individual engine, and hence the node average, sits at zero for a larger share of the run\.

#### Probes mostly fill the gap, with modest end\-to\-end overhead\.

Activating teacher probes raises teacher utilization to78\.9%78\.9\\%and82\.8%82\.8\\%under SOPD \(4B, 35B\) and to66\.6%66\.6\\%and57\.7%57\.7\\%under MOPD, with idle time falling to00–2%2\\%throughout\. The MOPD figures are lower for the same reason its baseline is more idle: probing is domain\-routed, so only the engine owning the current prompt’s domain is active and the node cannot reach the saturation a single undivided teacher achieves\. The traces in Appendix[D](https://arxiv.org/html/2609.02998#A4)make this concrete: the MOPD teacher oscillates between partial and full occupancy, whereas the SOPD teacher holds a tight band near saturation\. Rollout\-node utilization remains stable across the four configurations:→70\.569\.6\\\!\\to\\\!70\.5,→73\.873\.6\\\!\\to\\\!73\.8,→66\.364\.2\\\!\\to\\\!66\.3, and→63\.865\.6\\\!\\to\\\!63\.8percent, every change lying within the±2\\pm 2\-point run\-to\-run variance of these nodes\. This stability shows that probe generation does not create sustained resource contention on the student nodes, but it does not by itself prove wall\-clock neutrality\.

#### Cluster gains track teacher fraction and probe saturation\.

Cluster\-wide utilization improves by\+18\.0\+18\.0and\+11\.8\+11\.8points under SOPD and by\+12\.7\+12\.7and\+8\.6\+8\.6under MOPD \(4B, 35B respectively\)\. The ordering follows from two factors\. At 4B, the teacher accounts for1/51/5of the nodes, compared with1/71/7at 35B, so the same teacher\-side gain has a larger effect on the cluster average\. Probe saturation is also lower under MOPD because the teacher node is partitioned by domain\. Decomposing the cluster delta into per\-role contributions shows that the teacher term dominates in every case: it accounts for13\.813\.8of the\+18\.0\+18\.0at 4B SOPD,10\.610\.6of\+11\.8\+11\.8at 35B SOPD,11\.911\.9of\+12\.7\+12\.7at 4B MOPD, and7\.57\.5of\+8\.6\+8\.6at 35B MOPD\. The residual in each case comes from ordinary run\-to\-run drift on the train nodes and is largest at 4B SOPD \(\+3\.8\+3\.8\); in the two MOPD runs the train nodes move by−0\.0\-0\.0and\+1\.6\+1\.6points, so essentially the entire gain is reclaimed teacher capacity\.

#### Consistency across deployment topologies\.

The same pattern appears in single\- and multi\-domain OPD despite their different teacher allocations\. SOPD dedicates a full node to one teacher, whereas MOPD partitions that node across three domain specialists that are individually idle more often\. In both cases, most probe work uses previously idle capacity\. The utilization gain therefore does not depend on one particular teacher layout\.

### 4\.5Ablation: Gate Threshold

Figure 5:Gate\-threshold sweep on math benchmarks\(Qwen3\.5\-4B,KT=5K\_\{T\}\\\!=\\\!5, 99 steps\)\. Solid: TGOPD; dotted: Vanilla OPD\. All three benchmarks peak nearτ=3/5\\tau\\\!=\\\!3/5\.The main experiments fixτ=2/3\\tau\\\!=\\\!2/3andKT=3K\_\{T\}\\\!=\\\!3\. For this diagnostic ablation, we increase the probe budget toKT=5K\_\{T\}\\\!=\\\!5, yielding five nontrivial pass\-count cutoffs and thus a finer\-grained view of threshold sensitivity\. We sweepτ\\tauacross\{1/5,2/5,3/5,4/5,5/5\}\\\{1/5,\\;2/5,\\;3/5,\\;4/5,\\;5/5\\\}while keeping all other hyperparameters identical \(Qwen3\.5\-4B, math domain, 99 training steps\)\.

Figure[5](https://arxiv.org/html/2609.02998#S4.F5)shows an inverted\-U relationship between gate strictness and downstream accuracy\. At low thresholds \(τ≤2/5\\tau\\\!\\leq\\\!2/5\), the pass requirement admits many lower\-reliability teacher signals, including confidently wrong ones\. Performance still exceeds Vanilla OPD on AIME 2025 \(\+2\.2\+2\.2\) and HMMT\-Feb \(\+1\.0\+1\.0\), but the gains are smaller because the gate provides little filtering\. Atτ=3/5\\tau\\\!=\\\!3/5, all three benchmarks reach their peak: AIME 2026 scores73\.173\.1, AIME 2025 scores64\.464\.4, and HMMT\-Feb scores58\.058\.0\. These values exceed Vanilla OPD by\+1\.9\+1\.9,\+3\.3\+3\.3, and\+3\.7\+3\.7, respectively\. At high thresholds \(τ≥4/5\\tau\\\!\\geq\\\!4/5\), many useful teacher signals are rejected\. HMMT\-Feb falls below the Vanilla OPD baseline atτ=5/5\\tau\\\!=\\\!5/5\(52\.052\.0vs\.54\.354\.3\), which is consistent with useful OPD signal being discarded\.

The peak atτ=3/5\\tau=3/5\(pass rate60%60\\%\) is close to the defaultτ=2/3\\tau=2/3\(≈67%\\approx 67\\%\) used in the main experiments\. Under theKT=5K\_\{T\}=5diagnostic setting, the results indicate a broad performance optimum around a majority\-vote criterion\.

### 4\.6Ablation: What the Gate Does After It Closes

The threshold ablation determines when the gate closes\. We next compare two policies for closed\-gate prompts\. TGOPD uses a verifier\-grounded GRPO update \(equation[6](https://arxiv.org/html/2609.02998#S3.E6)\), which remains informative when the student rollout group contains both better and worse outcomes\. A simpler alternative masks the prompt by disabling the OPD branch and excluding all of its tokens from the update\. Both variants use the same gate \(KT=3K\_\{T\}\\\!=\\\!3,τ=2/3\\tau\\\!=\\\!2/3\) and differ only in what replaces the withdrawn teacher signal\. The comparison separates the effect of blocking an unreliable signal from the effect of providing a replacement\.

Table[5](https://arxiv.org/html/2609.02998#S4.T5)reports the single\-domain mathematics runs\. Both variants improve on Vanilla OPD at both scales, and the two are close: the GRPO fallback leads on average \(\+1\.47\+1\.47/\+1\.24\+1\.24at 4B / 35B\) against masking \(\+1\.32\+1\.32/\+0\.64\+0\.64\)\. The margin is carried entirely by the AIME sets, where the fallback is ahead in all four comparisons; masking is the better policy on HMMT\-Feb at both scales \(55\.5255\.52vs\.52\.7052\.70;64\.2764\.27vs\.64\.1064\.10\)\.

Table 4:Closed\-gate policy on multi\-domain runs at 199 training steps\. All three rows within a block share a training corpus and student initialization; the two gated rows additionally share the gate \(KT=3K\_\{T\}\\\!=\\\!3,τ=2/3\\tau\\\!=\\\!2/3\) and differ only in the update applied to gated\-off prompts\.Bold: best per column within a block\.MathCodeIFClosed\-gate policyAvgAIME 25AIME 26HMMT\-FebLCBOJBenchIFBenchIFEval*Qwen3\.5\-4B*Vanilla MOPD \(no gate\)53\.4058\.0763\.7550\.7350\.4818\.1048\.7883\.89GRPO fallback54\.5458\.1367\.6051\.2550\.0018\.9751\.7284\.11Mask only54\.9759\.9569\.7951\.9850\.7618\.5350\.8882\.88*Qwen3\.6\-35B\-A3B*Vanilla MOPD \(no gate\)60\.9973\.1277\.2960\.7357\.4323\.2844\.4090\.68GRPO fallback61\.9472\.9778\.0262\.4057\.8127\.5943\.9990\.78Mask only61\.7874\.2276\.7762\.5057\.1427\.1644\.6290\.05

Table[4](https://arxiv.org/html/2609.02998#S4.T4)repeats the comparison under multi\-domain training, where the seven\-benchmark average summarizes how each policy behaves across domains\. The ordering changes with scale: masking is stronger at 4B \(\+1\.57\+1\.57vs\.\+1\.14\+1\.14\), leading on all three mathematics columns and LCB, while the fallback recovers the lead at 35B \(\+0\.95\+0\.95vs\.\+0\.79\+0\.79\)\. The clearest domain\-level pattern appears in code: the GRPO fallback outperforms masking on OJBench at both scales \(18\.9718\.97vs\.18\.5318\.53;27\.5927\.59vs\.27\.1627\.16\), where execution\-based verification provides a direct correctness signal\.

Table 5:Closed\-gate policy on single\-domain mathematics runs\.#### Blocking the signal accounts for most of the gain\.

Across the four settings, masking improves on the ungated baseline by\+1\.08\+1\.08points on average, against\+1\.20\+1\.20for the full GRPO fallback\. Roughly90%90\\%of TGOPD’s benefit therefore remains when gated\-off prompts contribute no update\. This comparison suggests that blocking unreliable teacher signals accounts for most of the gain, while the GRPO replacement provides a smaller additional benefit\.

#### The fallback is the stronger default, though narrowly\.

The GRPO fallback outperforms masking in three of the four settings, with the largest margin occurring at 35B on single\-domain mathematics \(improvements of\+1\.24\+1\.24vs\.\+0\.64\+0\.64over vanilla OPD\)\. This pattern suggests that applying verifier\-grounded GRPO to prompts that fail the audit is generally preferable to omitting their updates altogether, although the margins remain small\. Masking wins only at 4B MOPD, but by the largest single margin in the table \(\+1\.57\+1\.57vs\.\+1\.14\+1\.14\)\. Since all four margins are small \(0\.150\.15–0\.600\.60points\), we adopt the fallback for its consistency rather than its magnitude\. A plausible reading of the split is that the fallback helps where the student can already solve a fair share of the gated\-off prompts on its own, because mixed\-outcome groups provide a meaningful relative signal\. It is exactly inert when every rollout in a group is incorrect \(or correct\), since centering then yields zero advantage\. The aggregate results do not isolate how often each regime occurs\. Future work should report this breakdown and examine whether the closed\-gate policy should vary by domain\.

## 5Related Work

### 5\.1On\-Policy Distillation and Multi\-Teacher Frameworks

On\-policy distillation \(OPD\) trains a student on its own rollouts while a stronger teacher provides token\-level supervision through a mode\-seeking reverse\-KL objective\\citepgu2024minillm,agarwal2024gkd,opdsurvey2026\. Compared with a single sequence\-level reward, this dense feedback can substantially improve sample efficiency\\citepthinkingmachines2025opd\. Vanilla OPD applies teacher supervision uniformly across prompts, while multi\-teacher OPD routes each prompt to a domain\-expert teacher to integrate multiple domain capabilities into one student\\citepmimo2026v2flash,mopd2026,sun2026d\.

Recent analyses show that teacher usefulness varies across instances\. OPD may fail when teacher knowledge is not locally exploitable by the student\\citeprethinkingopd2026, when student prefixes drift from teacher\-supported states\\citeprevisitingopd2026, or when an overly strong teacher induces negative transfer into a smaller student\\citepsmallmodels2025\. Existing multi\-teacher systems route prompts by domain but do not verify the selected teacher on each instance before using its token\-level rewards\. These studies motivate instance\-aware teacher selection, but do not estimate prompt\-level reliability from repeated verifier\-scored teacher rollouts\. TGOPD performs this verification before admitting dense teacher supervision\.

### 5\.2Reliability\-Aware Distillation and the Confidence–Correctness Gap

One line of reliability\-aware distillation adapts OPD using signals derived from model distributions\. EOPD introduces forward KL at high\-entropy teacher tokens\\citepeopd2026, while TrOPD defines token\-level trust regions using teacher–student decoding agreement\\citeptropd2026\. REOPOLD stabilizes implicit token rewards through mixture clipping and student\-entropy sampling\\citepreopold2026, whereas REOPD extrapolates token rewards from compatibility signals under a batch\-level budget, without an external verifier\\citepreopd2026\. When no external grader is available, GATES uses agreement among sampled tutor traces as a trajectory\-level reliability proxy\\citepgates2026\. These methods address uncertainty, diversity, distribution mismatch, or optimization stability, but do not directly verify outcome correctness\. PW\-OPSD further shows that high teacher entropy can reflect either non\-viable uncertainty or benign solution diversity\\citeppwopsd2026, while ESR finds that the position\-dependent degradation of off\-policy teacher guidance is not fully explained by KL or entropy\\citepesr2026\.

A complementary line explicitly incorporates outcome evidence\. RG\-OPD conditions trajectory\-level teacher supervision on agreement between verifier feedback and the teacher–student likelihood gap\\citeprgopd2026\. RLSD uses environmental correctness to determine the update direction while self\-distillation modulates its magnitude\\citeprlsd2026\. At a finer granularity, SPOT evaluates teacher\-proposed candidates using verifier\-scored student continuations and distills outcome\-calibrated local targets\\citepspot2026\. Related data\-selection methods verify static responses or tokens before training\\citepdong2023raft,singh2024restem,xu2025speculativekd\. TGOPD differs in the granularity and timing of its decision: it verifies repeated complete teacher rollouts to estimate prompt\-level reliability before dense supervision is admitted\. Each prompt is then routed exclusively to either OPD or GRPO, rather than blending the two signals or calibrating individual trajectories, tokens, or candidate branches\.

## 6Conclusion

This paper addresses vanilla OPD’s unconditional trust in teacher supervision through Teacher\-Gated On\-Policy Distillation \(TGOPD\), a prompt\-level gate grounded in verifier\-scored teacher probes\. TGOPD admits dense OPD only when the reliability audit passes and otherwise routes the prompt to verifier\-grounded GRPO, thereby withholding teacher supervision selectively rather than weakening it on every prompt\. Across 4B and 35B students in mathematics, code, and instruction following, TGOPD outperforms vanilla OPD in all six single\-domain settings and increases the seven\-benchmark average by1\.141\.14and0\.950\.95points in the corresponding multi\-domain settings\. The probes largely reuse idle teacher capacity, raising utilization from9\.8%9\.8\\%to78\.9%78\.9\\%in the measured 4B single\-domain run; a matched 35B code run incurs a5\.9%5\.9\\%increase in mean step time\. The current method requires an automatic verifier and makes a binary routing decision, so extending outcome\-grounded reliability estimation to open\-ended tasks and uncertainty\-aware gates is an important direction for future work\.

## References

- \[Agarwal et al\.\(2024\)Agarwal, Vieillard, Zhou, Stanczyk, Ramos, Geist, and Bachem\]Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem\.On\-policy distillation of language models: Learning from self\-generated mistakes\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.arXiv:2306\.13649\.
- \[Akhondzadeh et al\.\(2026\)Akhondzadeh, Lingam, Tejaswi, Ekbote, Sanghavi, and Bojchevski\]Mohammad Sadegh Akhondzadeh, Vijay Lingam, Atula Tejaswi, Chanakya Ekbote, Sujay Sanghavi, and Aleksandar Bojchevski\.Reward\-gated on\-policy distillation\.*arXiv preprint arXiv:2607\.04037*, 2026\.
- \[Dong et al\.\(2023\)\]Hanze Dong et al\.Raft: Reward ranked finetuning for generative foundation model alignment\.*arXiv preprint arXiv:2304\.06767*, 2023\.
- \[Fu et al\.\(2026\)Fu, Huang, Jiang, Liu, Jiang, Zhu, and Zhao\]Yuqian Fu, Haohuan Huang, Kaiwen Jiang, Jiacai Liu, Zhuo Jiang, Yuanheng Zhu, and Dongbin Zhao\.Revisiting on\-policy distillation: Empirical failure modes and simple fixes\.*arXiv preprint arXiv:2603\.25562*, 2026\.
- \[Gu et al\.\(2024\)Gu, Dong, Wei, and Huang\]Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang\.Minillm: Knowledge distillation of large language models\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.arXiv:2306\.08543\.
- \[Jin et al\.\(2026\)Jin, Min, Yang, Wei, Zhou, Kadhe, Baracaldo, and Lee\]Woogyeol Jin, Taywon Min, Yongjin Yang, Dennis Wei, Yi Zhou, Swanand Ravindra Kadhe, Nathalie Baracaldo, and Kimin Lee\.Entropy\-aware on\-policy distillation of language models\.*arXiv preprint arXiv:2603\.07079*, 2026\.
- \[Ko et al\.\(2026\)Ko, Abdali, Kim, Chen, and Cameron\]Jongwoo Ko, Sara Abdali, Young Jin Kim, Tianyi Chen, and Pashmina Cameron\.Scaling reasoning efficiently via relaxed on\-policy distillation\.*arXiv preprint arXiv:2603\.11137*, 2026\.
- \[Li et al\.\(2025a\)Li, Guo, Yang, Xu, Wu, and He\]Junlong Li, Daya Guo, Dejian Yang, Runxin Xu, Yu Wu, and Junxian He\.Codei/o: Condensing reasoning patterns via code input\-output prediction\.In*International Conference on Machine Learning \(ICML\)*\. PMLR, 2025a\.
- \[Li et al\.\(2026\)Li, Zuo, He, Zhang, Xiao, Qian, Yu, Gao, Yang, Liu, and Ding\]Yaxuan Li, Yuxin Zuo, Bingxiang He, Jinqian Zhang, Chaojun Xiao, Cheng Qian, Tianyu Yu, Huan\-ang Gao, Wenkai Yang, Zhiyuan Liu, and Ning Ding\.Rethinking on\-policy distillation of large language models: Phenomenology, mechanism, and recipe\.*arXiv preprint arXiv:2604\.13016*, 2026\.
- \[Li et al\.\(2025b\)\]Yuetai Li et al\.Small models struggle to learn from strong reasoners\.*arXiv preprint arXiv:2502\.12143*, 2025b\.
- \[Ling Team et al\.\(2025\)Ling Team, Shen, Li, Hu, Jing, Chen, Huang, Zhang, Yang, Lin, et al\.\]Ling Team, Anqi Shen, Baihui Li, Bin Hu, Bin Jing, Cai Chen, Chao Huang, Chao Zhang, Chaokun Yang, Cheng Lin, et al\.Every step evolves: Scaling reinforcement learning for trillion\-scale thinking model\.*arXiv preprint arXiv:2510\.18855*, 2025\.
- \[Liu et al\.\(2026\)Liu, Wang, Ma, Zhang, and Xiao\]Xiaogeng Liu, Xinyan Wang, Yingzi Ma, Yechao Zhang, and Chaowei Xiao\.When are teacher tokens reliable? position\-weighted on\-policy self\-distillation for reasoning\.*arXiv preprint arXiv:2605\.21606*, 2026\.
- \[Liu et al\.\(2025\)\]Zichen Liu et al\.Understanding r1\-zero\-like training: A critical perspective\.*arXiv preprint arXiv:2503\.20783*, 2025\.
- \[LLM\-Core Xiaomi\(2026\)\]LLM\-Core Xiaomi\.Mimo\-v2\-flash technical report\.*arXiv preprint arXiv:2601\.02780*, 2026\.
- \[Ma et al\.\(2026\)Ma, Wei, Zhao, Zhang, Xiao, Li, Yang, Gao, Wang, Li, Dong, Sui, and Luo\]Wenhan Ma, Jianyu Wei, Liang Zhao, Hailin Zhang, Bangjun Xiao, Lei Li, Qibin Yang, Bofei Gao, Yudong Wang, Rang Li, Jinhao Dong, Zhifang Sui, and Fuli Luo\.MOPD: Multi\-teacher on\-policy distillation for capability integration in LLM post\-training\.*arXiv preprint arXiv:2606\.30406*, 2026\.
- \[Qu et al\.\(2026\)Qu, Zhang, Kong, Shang, Chen, Ban, Qiu, and Dai\]Zikun Qu, Min Zhang, Mingze Kong, Zhiwei Shang, Zhengyu Chen, Yikun Ban, Shuang Qiu, and Zhongxiang Dai\.SPOT: Sparse probing and outcome calibration for on\-policy distillation\.*arXiv preprint arXiv:2608\.04419*, 2026\.
- \[Shao et al\.\(2024\)\]Zhihong Shao et al\.Deepseekmath: Pushing the limits of mathematical reasoning in open language models\.*arXiv preprint arXiv:2402\.03300*, 2024\.
- \[Singh et al\.\(2024\)\]Avi Singh et al\.Beyond human data: Scaling self\-training for problem\-solving with language models\.*Transactions on Machine Learning Research \(TMLR\)*, 2024\.arXiv:2312\.06585\.
- \[Song & Zheng\(2026\)Song and Zheng\]Mingyang Song and Mao Zheng\.A survey of on\-policy distillation for large language models\.*arXiv preprint arXiv:2604\.00626*, 2026\.
- \[Stein et al\.\(2026\)Stein, Huang, and Goldstein\]Alex Stein, Furong Huang, and Tom Goldstein\.GATES: Self\-distillation under privileged context with consensus gating\.*arXiv preprint arXiv:2602\.20574*, 2026\.
- \[Sun et al\.\(2026a\)Sun, Ma, Qin, Liu, Lu, Zhu, Cai, and Yan\]Yang Sun, Lichao Ma, Houyuan Qin, Yuxin Liu, Hanyang Lu, Yao Zhu, Pinlong Cai, and Guohang Yan\.REOPD: Reliability\-adaptive reward extrapolation for on\-policy distillation\.*arXiv preprint arXiv:2608\.11698*, 2026a\.
- \[Sun et al\.\(2026b\)Sun, Zhang, Zhao, Li, Mu, Deng, Zhan, Chen, Hu, and Zhang\]Zechen Sun, Zhiwei Zhang, Fei Zhao, Juntao Li, Chuan Mu, Huayu Deng, Guojian Zhan, Wenliang Chen, Yao Hu, and Min Zhang\.D3\-MOPD: Adaptive dynamic domain scheDuling for efficient multi\-teacher distillation\.*arXiv preprint arXiv:2608\.24987*, 2026b\.
- \[Thinking Machines Lab\(2025\)\]Thinking Machines Lab\.On\-policy distillation\.[https://thinkingmachines\.ai/blog/on\-policy\-distillation/](https://thinkingmachines.ai/blog/on-policy-distillation/), 2025\.
- \[Xing et al\.\(2026\)Xing, Wang, Gao, Li, and Tang\]Xingrun Xing, Haoqing Wang, Boyan Gao, Ziheng Li, and Yehui Tang\.Trust region on\-policy distillation\.*arXiv preprint arXiv:2606\.01249*, 2026\.
- \[Xu et al\.\(2025\)\]Wenda Xu et al\.Speculative knowledge distillation: Bridging the teacher\-student gap through interleaved sampling\.In*International Conference on Learning Representations \(ICLR\)*, 2025\.arXiv:2410\.11325\.
- \[Yang et al\.\(2026a\)Yang, Qin, Si, Chen, Gu, Yao, Lin, Wang, Wang, and Duan\]Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, Naibin Gu, Dingyu Yao, Zheng Lin, Weiping Wang, Jiaqi Wang, and Nan Duan\.Self\-distilled RLVR\.*arXiv preprint arXiv:2604\.03128*, 2026a\.
- \[Yang et al\.\(2026b\)Yang, Liu, Chen, Dai, Wang, Lin, Lee, Chen, Jiang, He, et al\.\]Zhuolin Yang, Zihan Liu, Yang Chen, Wenliang Dai, Boxin Wang, Sheng\-Chieh Lin, Chankyu Lee, Yangyi Chen, Dongfu Jiang, Jiafan He, et al\.Nemotron\-cascade 2: Post\-training llms with cascade rl and multi\-domain on\-policy distillation\.*arXiv preprint arXiv:2603\.19220*, 2026b\.
- \[Yu et al\.\(2026\)Yu, Zhang, Zhu, Yuan, Zuo, Yue, Dai, Fan, Liu, Liu, et al\.\]Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al\.Dapo: An open\-source llm reinforcement learning system at scale\.*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2026\.
- \[Zhou et al\.\(2026\)Zhou, Li, Tang, Wu, and Terzopoulos\]Ziheng Zhou, Jiaqi Li, Huacong Tang, Ying Nian Wu, and Demetri Terzopoulos\.Less is more: Early stopping rollout for on\-policy distillation\.*arXiv preprint arXiv:2605\.27028*, 2026\.
- \[Zhu et al\.\(2025\)Zhu, Xie, Lv, and slime Contributors\]Zilin Zhu, Chengxing Xie, Xin Lv, and slime Contributors\.slime: An llm post\-training framework for rl scaling\.[https://github\.com/THUDM/slime](https://github.com/THUDM/slime), 2025\.

## Appendix AContributors

Zhiwei Zhang, Zechen Sun, Fei Zhao, Kang Peng, Bin Liang, Huayu Deng, Yao Hu, Kam\-Fai Wong, Mu Chuan\.

## Appendix BTraining Hyperparameters and Infrastructure

Table[6](https://arxiv.org/html/2609.02998#A2.T6)reports the settings retained in the run records at both model scales\. Within each controlled comparison, Vanilla OPD and TGOPD share the student initialization, data, optimization settings, and system topology\. IcePop/TIS is an implementation\-level safeguard for the asynchronous engine; the method itself is defined by the gate and routed advantage in equation[6](https://arxiv.org/html/2609.02998#S3.E6)\.

Table 6:Training configuration by model scale\. Both scales follow the same optimization and online\-probing protocol; only model\-dependent system settings differ\.#### Infrastructure\.

The 4B runs use five 8\-GPU nodes \(40 GPUs\): two trainer nodes, two student\-rollout nodes, and one teacher node\. The 35B runs use seven 8\-GPU nodes \(56 GPUs\): four trainer nodes, two student\-rollout nodes, and one teacher node\. For the 4B 16K runs, we cap the SGLang static memory fraction at 0\.60 and both the maximum running requests and teacher\-server concurrency at 64\.

#### Probe realization\.

At both model scales, the frozen teacher draws three fresh probe responses online for each prompt and the gate opens when at least two pass the verifier\. Thus the 4B and 35B experiments instantiate the same reliability test described in §[3\.1](https://arxiv.org/html/2609.02998#S3.SS1); the resource topology changes with model scale, but the training rule does not\.

## Appendix CEvaluation Benchmark Details

Table[7](https://arxiv.org/html/2609.02998#A3.T7)lists the benchmarks used for in\-domain evaluation, together with the number of independent generation runs and the metric reported in the main tables\. For benchmarks evaluated with multiple runs, the reported score is the average over those runs\. For LiveCodeBench we report only the code\-generation subtask; for OJBench we report the overall score aggregated over the C\+\+ and Python splits\.

Table 7:Evaluation benchmark details\.*Runs*: number of independent generations per checkpoint; the score reported in all main\-text tables is the mean over these runs\.
## Appendix DPer\-Configuration GPU Utilization Traces

Section[4\.4](https://arxiv.org/html/2609.02998#S4.SS4)summarizes teacher\-node utilization by its mean and idle fraction \(Table[3](https://arxiv.org/html/2609.02998#S4.T3)\) and its marginal distribution \(Figure[4](https://arxiv.org/html/2609.02998#S4.F4)a\)\. Neither summary preserves the time axis\. Figure[6](https://arxiv.org/html/2609.02998#A4.F6)therefore plots the raw one\-hour trace for every configuration\. All eight traces are sampled at 15\-second resolution over a one\-hour window taken from steady\-state training, and the two curves within a panel are drawn on a common axis\.

#### Matched end\-to\-end timing\.

To avoid inferring wall\-clock neutrality from utilization alone, we additionally compare the completed 35B CodeIO TGOPD and OPD runs under identical settings\. Across500500aligned rollout cycles, TGOPD increases mean step time by5\.9%5\.9\\%, while mean decode throughput changes by less than0\.1%0\.1\\%\. Thus most probe work overlaps with idle teacher capacity, but the audit is not strictly free\.

Figure 6:Teacher\-node GPU utilization over a one\-hour window for each configuration\.Light blue: vanilla OPD\. Dark blue: TGOPD\. The shaded region indicates the teacher capacity reclaimed through probe generation\. Dashed lines indicate the mean utilization of each run\.\(a,b\)Single\-domain OPD at the 4B and 35B scales\.\(c,d\)Multi\-domain OPD at the corresponding scales\.The traces clarify three aspects of the utilization results in §[4\.4](https://arxiv.org/html/2609.02998#S4.SS4)\.

#### The baseline teacher is bursty, not uniformly slow\.

In all four panels the light\-blue curve alternates between brief spikes and long stretches pinned at zero, rather than hovering at a low but steady level\. Each spike is one scoring pass; the flat intervals are the student’s rollout phase, during which the teacher has no scoring work\. The idle fraction in Table[3](https://arxiv.org/html/2609.02998#S4.T3)measures these intervals\. Their duration is set by the student rollout rather than teacher scoring speed, so shrinking the teacher or batching its scoring work differently would not remove the gap\.

#### Probes fill the gaps rather than adding load\.

Under TGOPD, the dark\-blue curve occupies a high band and rarely returns to zero, but its peaks do not exceed the peaks the baseline already reached\. The probe is therefore consuming the interval between scoring passes rather than competing with them, which is the mechanism behind the stable rollout\-node utilization reported in Table[3](https://arxiv.org/html/2609.02998#S4.T3); the matched timing comparison above shows that the overlap is substantial but not complete\.

#### Multi\-domain probing oscillates because it is domain\-routed\.

The SOPD panels show a tight blue band around7979–83%83\\%, whereas the MOPD panels swing between roughly40%40\\%and85%85\\%\. The MOPD teacher node is partitioned into three domain\-specialist engines \(code 4 GPU, math 2 GPU, IF 2 GPU\), and only the engine owning the current prompt’s domain is active, so node occupancy steps between partial and full as the domain mix of the batch changes\. This oscillation is what produces the bimodal marginal distribution visible in Figure[4](https://arxiv.org/html/2609.02998#S4.F4)a and the lower MOPD means in Table[3](https://arxiv.org/html/2609.02998#S4.T3)\. It reflects how the node is divided rather than a failure of the probe to fill the window: the idle fraction still falls to00–2%2\\%\.