Consensus as Privileged Context for Label-Free Self-Distillation

# Consensus as Privileged Context for Label-Free Self-Distillation
Source: [https://arxiv.org/html/2607.13643](https://arxiv.org/html/2607.13643)
John Gkountouras1Josip Jukić1Ivan Titov1,2 1ILLC, University of Amsterdam 2ILCC, University of Edinburgh \{i\.gkountouras,j\.jukic\}@uva\.nlititov@inf\.ed\.ac\.uk

###### Abstract

Sampling multiple solutions and returning the majority answer is among the most reliable ways to improve the reasoning accuracy of large language models without labels, and a growing family of methods converts this consensus signal into training supervision\. However, existing approaches use consensus only in restricted forms: as a filter that selects solutions for fine\-tuning, as a preference between answers, or as a scalar reward for reinforcement learning, discarding most of the information that the agreeing solutions contain\. We presentCanon\(Consensus\-ANchored self\-distillatiON\), a label\-free training method that turns consensus into dense, token\-level supervision\. For each unlabeled prompt,Canonsamples multiple solutions, extracts the majority answer, and conditions a frozen snapshot of the model on a solution that reaches it; this consensus\-anchored teacher then supervises the model on its own rollouts at every token\. Experiments on mathematical and scientific reasoning benchmarks show thatCanonimproves pass@1 by up to 12 points, outperforming label\-free reinforcement learning by 6 points at a seventh of its compute and approaching a teacher conditioned on gold solutions; trained on pooled unlabeled data, it transfers to held\-out benchmarks, matching training methods that use gold labels\. Analysis suggests that the improvements are not pure distribution sharpening: after training, the model solves problems it previously never solved in 32 attempts, and its majority vote itself becomes more accurate\.

## 1Introduction

Reinforcement learning with verifiable rewards has become the standard recipe for improving the reasoning abilities of large language models\(Guoet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib11); Shaoet al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib10)\), but it presupposes gold answers to verify against\. Without labels, an informative signal a model produces about its own correctness is agreement: sampling several solutions and returning the most frequent answer, known as self\-consistency\(Wanget al\.,[2022](https://arxiv.org/html/2607.13643#bib.bib1)\), reliably outperforms single\-pass decoding\. Answers supported by many independent samples are more likely to be correct, so consistency acts as an unsupervised proxy for accuracy\(Wanget al\.,[2022](https://arxiv.org/html/2607.13643#bib.bib1); Prasadet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib4)\)\. Agreement is available on any unlabeled prompt at the cost of extra samples, and it identifies not only a likely answer but entire solutions that reach it\. This paper studies how to turn that signal into supervision\.

Existing methods use consensus as a filter that selects solutions for supervised fine\-tuning\(Huanget al\.,[2023](https://arxiv.org/html/2607.13643#bib.bib2)\), as a source of pairwise preferences\(Prasadet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib4)\), or, most prominently, as a scalar pseudo\-reward for reinforcement learning on unlabeled data\(Zuoet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib3)\)\. In every case the agreeing solutions are reduced to a selection or a single bit of feedback, and the reinforcement learning variants additionally require hundreds of optimization steps, each backed by fresh generation\. In parallel, on\-policy self\-distillation has emerged as a dense alternative to reinforcement learning in the labeled regime: a model conditioned on label\-derived privileged information \(i\.e\., the gold solution or a derivation for a gold answer\) serves as a token\-level teacher for its own unconditioned policy\(Zhaoet al\.,[2026a](https://arxiv.org/html/2607.13643#bib.bib6); Hübotteret al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib7)\)\. The difficulty is that this privileged context depends on the gold label and is therefore unavailable in the unsupervised setting\. This raises the question:*can the model’s own consensus play the role of this privileged context enabling dense self\-distillation without gold solutions?*

![Refer to caption](https://arxiv.org/html/2607.13643v1/x1.png)Figure 1:Canontraining framework\.Given an unlabeled prompt, the model samplesNNsolutions and extracts the majority answer\. A frozen snapshot of the model, conditioned on a consensus solution, serves as a dense per\-token teacher; the student is trained on its own rollouts to follow the lead line set by their consensus, without seeing it\. Gradients never flow into the teacher\. A single generation pass provides both the consensus and the distillation substrate\.We proposeCanon\(Consensus\-ANchored self\-distillatiON\),which turns the model’s own agreement into a dense training signal\. For each unlabeled prompt,CanonsamplesNNsolutions, extracts the majority answer, and selects a consensus solution from among the rollouts that reach it\. A frozen snapshot of the model, conditioned on this consensus solution, then acts as a teacher: the student minimizes a full\-vocabulary Jensen–Shannon divergence toward the teacher’s next\-token distributions over its own sampled tokens\. A single generation pass provides both the consensus and the distillation substrate, and training lasts one epoch\.

We evaluateCanonin two regimes\. In the*transductive*regime, we train directly on the unlabeled prompts of a test benchmark and evaluate on the same prompts, the setting popularized by test\-time reinforcement learning\(Zuoet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib3)\); labels are used exclusively for scoring\. In the*inductive*regime, we train on one unlabeled prompt set and evaluate on disjoint benchmarks, testing whether the model has genuinely improved rather than adapted\. Across both, we compare against label\-free alternatives, namely self\-consistency fine\-tuning\(Huanget al\.,[2023](https://arxiv.org/html/2607.13643#bib.bib2)\)and test\-time reinforcement learning\(Zuoet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib3)\), and against label\-full ones: rejection fine\-tuning\(Yuanet al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib21)\), GRPO with gold rewards\(Shaoet al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib10)\), and an oracle variant of our own method whose teacher is conditioned on gold\-verified solutions, mirroring the privileged context assumed by gold\-conditioned self\-distillation\(Zhaoet al\.,[2026a](https://arxiv.org/html/2607.13643#bib.bib6); Hübotteret al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib7)\)\. Our evaluation includes competition mathematics and graduate\-level multiple\-choice science: the method assumes only an extractable final answer and a reliable consensus\. Section[5\.2](https://arxiv.org/html/2607.13643#S5.SS2)characterizes this applicability condition\.

Our findings are consistent across models and benchmarks\. Transductively,Canonis the strongest of the label\-free methods we evaluate, improving pass@1 by up to twelve points on competition mathematics and graduate\-level science questions while training in about an hour of GPU time, whereas the reinforcement learning alternatives require six to ten\. It also approaches the oracle\-conditioned teacher, trailing by half a point on AMC, three on AIME 2024, and eight on GPQA: the model’s own consensus captures much of the value of a gold solution as privileged context\. Inductively, the improvements transfer to held\-out and harder benchmarks, whereCanonmatches both gold\-conditioned distillation and gold\-reward reinforcement learning trained on the same prompts\. Finally, the gains are not only concentration of probability mass on answers the model already produced: after training, the model solves problems it previously never solved in 32 attempts, and on the unsaturated benchmarks its majority vote itself becomes more accurate\.

The key contributions of our work are as follows\.

- •Canon, a label\-free post\-training method that distills a consensus\-anchored teacher into the model over its own rollouts, turning majority votes into dense token\-level supervision\.
- •A controlled comparison of supervision signals for unlabeled test\-time training, showing thatCanondominates the label\-free alternatives we evaluate on the accuracy\-compute frontier and recovers most of the gains of gold\-conditioned teachers\.
- •Evidence that the improvements transfer: gains persist on harder and cross\-domain held\-out benchmarks, whereCanonmatches training on gold labels\.
- •An analysis of when consensus supervision helps: per\-prompt gains track the state of the base model’s consensus, largest where the majority is correct but unconfident, near zero where it is saturated, and negative where it is confidently wrong\.

## 2Related Work

#### Test\-time scaling and test\-time training\.

Sampling multiple solutions and aggregating them by majority vote improves reasoning accuracy without any training\(Wanget al\.,[2022](https://arxiv.org/html/2607.13643#bib.bib1)\), at the cost of multiplying inference compute on every query\. Test\-time*training*instead spends compute once, updating parameters on the test distribution itself, an idea that has proven effective from vision\(Sunet al\.,[2020](https://arxiv.org/html/2607.13643#bib.bib20)\)to abstract reasoning with language models\(Akyüreket al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib14)\)\.Canonbelongs to this family but derives its training signal purely from the model’s own consensus, and amortizes the benefit of voting into the weights: after training, a single sample captures much of what previously required an ensemble\.

#### Label\-free reinforcement learning\.

The closest training paradigm is label\-free reinforcement learning\. TTRL\(Zuoet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib3)\)showed that the majority\-vote answer can serve as a pseudo\-label for verifiable\-reward RL on unlabeled test data, and a large family of descendants refines the pseudo\-reward or stabilizes its optimization\(e\.g\., Yanet al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib28); Linet al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib19)\); we evaluate three members of this family as baselines in Section[4\.2](https://arxiv.org/html/2607.13643#S4.SS2)\. Related intrinsic signals include semantic entropy and self\-certainty\(Zhanget al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib17); Zhaoet al\.,[2026b](https://arxiv.org/html/2607.13643#bib.bib18)\)\. These methods share two structural costs: the consensus is compressed into a scalar per\-trajectory reward, and the RL loop requires repeated generation across many optimization steps\. Recent analyses further argue that a substantial fraction of the resulting gains reflects sharpening of already\-solvable problems rather than new capability\(Linet al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib19); Shafayatet al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib16)\)\.Canonkeeps the consensus signal but discards the RL machinery: the vote conditions a teacher rather than defining a reward, yielding a full\-vocabulary target at every token from a single generation pass\.

#### Self\-training on model consensus\.

Using agreement across complementary views to exploit unlabeled data predate large language models: co\-training\(Blum and Mitchell,[1998](https://arxiv.org/html/2607.13643#bib.bib36)\)lets two learners with different views of the data supervise each other through their predictions, an approach with a long lineage in NLP\(Collins and Singer,[1999](https://arxiv.org/html/2607.13643#bib.bib35)\)\. Consensus self\-training replaces agreement between distinct views with agreement among rollouts from a single model\. Before the RL\-based variants, consensus was used to filter self\-generated solutions for supervised fine\-tuning\(Huanget al\.,[2023](https://arxiv.org/html/2607.13643#bib.bib2)\)and to build preference pairs for DPO\-style training\(Prasadet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib4)\); label\-full analogues filter by gold correctness instead\(Zelikmanet al\.,[2022](https://arxiv.org/html/2607.13643#bib.bib12); Singhet al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib13); Yuanet al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib21)\)\. These approaches supervise on the selected text with hard targets\.Canondiffers in both signal and mechanism: supervision covers all of the student’s rollouts, not only consensus\-consistent ones, and the target is the teacher’s full next\-token distribution rather than a token identity, a difference we find matters empirically \(Section[6](https://arxiv.org/html/2607.13643#S6)\)\.

#### Privileged\-context self\-distillation\.

Distilling the effect of a context into a model’s weights goes back to context distillation, which trains a model to reproduce, without the context, the behavior it exhibits with it\(Snellet al\.,[2023](https://arxiv.org/html/2607.13643#bib.bib37)\)\. On\-policy distillation extends the idea to training on the student’s own rollouts under a teacher’s dense token\-level feedback\(Agarwalet al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib5); Song and Zheng,[2026](https://arxiv.org/html/2607.13643#bib.bib9)\), and recent work instantiates the teacher as the same model conditioned on privileged information, typically the gold solution or rich environment feedback, removing the need for a separate teacher\(Zhaoet al\.,[2026a](https://arxiv.org/html/2607.13643#bib.bib6); Hübotteret al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib7)\)\. All of these assume the privileged context is given\. Closest to us, GATES\(Steinet al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib8)\)studies label\-free self\-distillation in document\-grounded QA, where an external document provides the privileged context and consensus among teacher traces gates the distillation signal\.Canonmakes the complementary move: consensus is not a reliability filter on external privileged information, it*is*the privileged information, manufactured by the model itself with no document, verifier, or label\. Our oracle\-recovery results quantify how little is lost by this substitution\.

## 3Methodology

### 3\.1Setting

We are given a policyπθ\\pi\_\{\\theta\}, initialized from a pretrained modelπ0\\pi\_\{0\}, and a set of*unlabeled*prompts𝒟=\{x1,…,xM\}\\mathcal\{D\}=\\\{x\_\{1\},\\ldots,x\_\{M\}\\\}with extractable final answers \(boxed mathematical answers or multiple\-choice letters\)\. No gold answers, reward models, or verifiers are available\. We consider two regimes\. In thetransductiveregime,𝒟\\mathcal\{D\}is the test set itself: we adapt on its prompts and evaluate on the same prompts, with gold labels used only for scoring\. In theinductiveregime, we train on𝒟\\mathcal\{D\}and evaluate on disjoint prompt sets, measuring whether label\-free training produces improvements that transfer\.

### 3\.2Canon

Canonruns in three steps, illustrated in Figure[1](https://arxiv.org/html/2607.13643#S1.F1): it extracts a consensus from sampled solutions, constructs a teacher anchored to that consensus, and distills the teacher back into the model over the model’s own rollouts\.

For each promptxx, we sampleNNrolloutsy\(1\),…,y\(N\)∼π0\(⋅∣x\)y^\{\(1\)\},\\ldots,y^\{\(N\)\}\\sim\\pi\_\{0\}\(\\cdot\\mid x\)and extract their final answersai=ans⁡\(y\(i\)\)a\_\{i\}=\\operatorname\{ans\}\(y^\{\(i\)\}\)\. The majority answer is

a^x=arg​maxa​∑i=1N𝟏​\[ai=a\],\\hat\{a\}\_\{x\}=\\operatorname\*\{arg\\,max\}\_\{a\}\\sum\_\{i=1\}^\{N\}\\mathbf\{1\}\\big\[a\_\{i\}=a\\big\],\(1\)and the*consensus solution*is the most confident member of the majority set𝒮x=\{y\(i\):ai=a^x\}\\mathcal\{S\}\_\{x\}=\\\{y^\{\(i\)\}:a\_\{i\}=\\hat\{a\}\_\{x\}\\\}, namely the rollout with the highest mean token log\-probability,sx=arg​maxy∈𝒮x⁡1\|y\|​∑t=1\|y\|log⁡π0​\(yt∣x,y<t\)s\_\{x\}=\\operatorname\*\{arg\\,max\}\_\{y\\in\\mathcal\{S\}\_\{x\}\}\\tfrac\{1\}\{\|y\|\}\\sum\_\{t=1\}^\{\|y\|\}\\log\\pi\_\{0\}\(y\_\{t\}\\mid x,y\_\{<t\}\)\. Prompts for which no rollout yields an extractable answer are skipped\.

The teacher is a frozen snapshotπ¯\\bar\{\\pi\}of the policy, taken at the start of the epoch; with a single training epoch, as in our recipe, this snapshot coincides with the initial policyπ0\\pi\_\{0\}, the same weights that generated the rollouts\. The teacher never generates; it only scores\. To score a rollouty\(i\)y^\{\(i\)\}, the consensus solution is inserted into the teacher’s context through a short template that presents it as a reference solution for the prompt \(Appendix[C](https://arxiv.org/html/2607.13643#A3)\), and the teacher computes, in a single teacher\-forced pass over the fixed rollout tokens, a next\-token distribution at every position,π¯\(⋅∣x,sx,y<t\(i\)\)\\bar\{\\pi\}\(\\cdot\\mid x,s\_\{x\},y^\{\(i\)\}\_\{<t\}\)\. Because the teacher reads a complete solution that reaches the consensus answer before scoring, these distributions are anchored to that line of reasoning\. The student computes distributions over the same rollout tokens from the prompt alone and is trained to match the teacher’s, by minimizing a full\-vocabulary Jensen\-Shannon divergence:

ℒ\(θ\)=𝔼x∼𝒟1N∑i=1N1\|y\(i\)\|∑t=1\|y\(i\)\|DJS\(πθ\(⋅∣x,y<t\(i\)\)∥π¯\(⋅∣x,sx,y<t\(i\)\)\),\\mathcal\{L\}\(\\theta\)=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\}\\;\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\frac\{1\}\{\|y^\{\(i\)\}\|\}\\sum\_\{t=1\}^\{\|y^\{\(i\)\}\|\}D\_\{\\mathrm\{JS\}\}\\\!\\left\(\\pi\_\{\\theta\}\\big\(\\cdot\\mid x,\\,y^\{\(i\)\}\_\{<t\}\\big\)\\,\\\|\\,\\bar\{\\pi\}\\big\(\\cdot\\mid x,\\,s\_\{x\},\\,y^\{\(i\)\}\_\{<t\}\\big\)\\right\),\(2\)whereDJS​\(p∥q\)=12​DKL​\(p∥m\)\+12​DKL​\(q∥m\)D\_\{\\mathrm\{JS\}\}\\\!\\left\(p\\,\\\|\\,q\\right\)=\\tfrac\{1\}\{2\}D\_\{\\mathrm\{KL\}\}\\\!\\left\(p\\,\\\|\\,m\\right\)\+\\tfrac\{1\}\{2\}D\_\{\\mathrm\{KL\}\}\\\!\\left\(q\\,\\\|\\,m\\right\)withm=12​\(p\+q\)m=\\tfrac\{1\}\{2\}\(p\+q\); the divergence was selected on a development slice, and its bounded, symmetric form is discussed in Appendix[A](https://arxiv.org/html/2607.13643#A1)\. Each rollout contributes its token\-mean divergence, so rollouts count equally in the objective regardless of their length\. Gradients flow only through the student; the teacher is a stop\-gradient snapshot\. TheNNrollouts sampled to form the consensus double as the distillation substrate, soCanonrequires exactly one generation pass, the same pass that self\-consistency decoding at inference would perform\. We train for a single epoch on these rollouts; a second epoch with regenerated rollouts and a re\-anchored teacher adds at most a point and a third reverses it \(Appendix[F](https://arxiv.org/html/2607.13643#A6)\)\. A formal statement of the setup, objective, and training procedure is given in Appendix[A](https://arxiv.org/html/2607.13643#A1)\.

Three properties distinguishCanonfrom prior uses of consensus\. First, the supervision is dense: where a majority\-vote reward assigns one bit per trajectory, the consensus\-anchored teacher provides a full next\-token distribution at every position of every rollout, including rollouts that disagree with the consensus\. For these, the teacher indicates where the trajectory departs from the consensus line rather than just the fact that it fails\. Second, the teacher is frozen: conditioning a snapshot rather than the live policy prevents the degenerate co\-adaptation we observe when teacher and student share parameters \(Appendix[F](https://arxiv.org/html/2607.13643#A6)\)\. Third, the method is cheap: no reward model, no repeated generation, no preference pairs; the marginal cost over self\-consistency decoding is one epoch of distillation\.

The mechanism also suggests where the method should and should not help\. The teacher can only anchor to what the consensus contains: on prompts where the majority answer is wrong and confidently held, the anchor points the wrong way, and on benchmarks where consensus is already saturated there is little left to distill\. Section[5](https://arxiv.org/html/2607.13643#S5)examines both regimes empirically and uses them to characterize when the method applies\.

## 4Experiments

Our experiments address four research questions\.

- •RQ1:How doesCanoncompare to existing label\-free methods when training on unlabeled test prompts, in accuracy and in compute? \(Section[4\.2](https://arxiv.org/html/2607.13643#S4.SS2)\)
- •RQ2:How much of the value of a gold\-conditioned teacher does consensus conditioning recover? \(Sections[4\.2](https://arxiv.org/html/2607.13643#S4.SS2)and[4\.3](https://arxiv.org/html/2607.13643#S4.SS3)\)
- •RQ3:Do the improvements transfer inductively to held\-out and harder benchmarks? \(Section[4\.4](https://arxiv.org/html/2607.13643#S4.SS4)\)
- •RQ4:Do the gains go beyond sharpening of the output distribution? \(Section[5](https://arxiv.org/html/2607.13643#S5)\)

### 4\.1Experimental Setup

Most experiments use Qwen3\-4B\-Instruct\-2507; Section[4\.3](https://arxiv.org/html/2607.13643#S4.SS3)extends the evaluation to six further models spanning the Qwen family from 2B to 9B, three other model families, and a mixture\-of\-experts architecture\. Hybrid\-thinking models are run in non\-thinking mode throughout, in both training and evaluation\. Small instruct models are the natural setting for this problem: they exhibit reliable consensus on the benchmarks we study while leaving a large gap between single\-sample and majority\-vote accuracy, which is precisely the headroomCanontargets \(Section[5\.2](https://arxiv.org/html/2607.13643#S5.SS2)\)\.

We evaluate on AMC 2023 \(83 prompts\), AIME 2024 and 2025 \(30 each\), GPQA\-Diamond in multiple\-choice format \(198\)\(Reinet al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib22)\), and MATH500 \(500\)\(Lightmanet al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib23)\)\. For inductive experiments, we additionally construct a pooled unlabeled training set of 339 mixed mathematics prompts \(composition in Appendix[D](https://arxiv.org/html/2607.13643#A4)\) and use OmniMath difficulty slices\(Gaoet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib24)\)\. All hyperparameter selection uses dedicated OmniMath development slices that are disjoint from every evaluation benchmark\.

We compare methods matched on training prompts and grouped by supervision\. The label\-free baselines are LMSI, self\-consistency\-filtered supervised fine\-tuning\(Huanget al\.,[2023](https://arxiv.org/html/2607.13643#bib.bib2)\); ScPO, self\-consistency preference optimization\(Prasadet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib4)\); TTRL, test\-time reinforcement learning with a majority\-vote pseudo\-reward\(Zuoet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib3)\); EMPO, reinforcement learning that minimizes the semantic entropy of sampled responses, clustered by answer equivalence\(Zhanget al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib17)\); SCRL, which grants the positive pseudo\-reward only under strict consensus criteria and adds entropy\-gated negative pseudo\-labels\(Yanet al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib28)\); and TTRL\-Guard, which monitors the stability of the vote during training and down\-weights the pseudo\-reward where the majority is unreliable\(Linet al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib19)\)\. The label\-full baselines are RFT, rejection fine\-tuning on the shortest gold\-correct rollout per prompt\(Yuanet al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib21)\); GRPO, identical to the TTRL pipeline but with the reward verified against gold answers\(Shaoet al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib10)\); and an oracle variant ofCanonwhose teacher is conditioned on a gold\-verified solution obtained by deep resampling, that is, by sampling each prompt up to 512 times and retaining a solution verified against the gold answer \(protocol and per\-benchmark coverage in Appendix[B](https://arxiv.org/html/2607.13643#A2)\)\. The oracle isolates the value of the conditioning content, since it differs fromCanononly in which solution anchors the teacher; remaining baseline configurations are detailed in Appendix[B](https://arxiv.org/html/2607.13643#A2)\.

All evaluations use a single frozen harness: temperature0\.60\.6, top\-pp0\.950\.95,n=32n\{=\}32samples per prompt with a fixed seed, maximum generation length 4608 tokens, and one greedy sample\. We report avg@32, the mean per\-prompt fraction of correct samples \(an unbiased estimate of pass@1 at the sampling temperature\), maj@32 \(majority\-vote accuracy\), and pass@32\. Answers are extracted from`\\boxed\{\}`and checked with math\-verify for mathematics, and by letter extraction for multiple choice\. Uncertainty for every comparison in the paper, from paired bootstraps over prompts \(B=10,000B\{=\}10\{,\}000, percentile intervals\), is reported in Appendix[E](https://arxiv.org/html/2607.13643#A5)\.

Several precautions ensure that the comparisons below are meaningful\. The same frozen harness evaluates every model, method, and checkpoint\. To rule out leakage through tuning, learning rates are selected once per model on the OmniMath development slices and then frozen \(Appendix[B](https://arxiv.org/html/2607.13643#A2)\); no hyperparameter is ever tuned on a reported benchmark\. Label\-free methods never observe gold answers at any point, and evaluation labels are used solely to compute metrics\. Finally,Canonuses a single configuration across every model and dataset \(N=32N\{=\}32, one consensus solution, JSD, snapshot teacher, one epoch, LoRA\(Huet al\.,[2022](https://arxiv.org/html/2607.13643#bib.bib27)\)rank 64\), so no per\-benchmark adjustment contributes to the reported gains; one epoch on an 83\-prompt benchmark amounts to roughly a dozen optimizer steps\.

### 4\.2Transductive Test\-Time Training \(RQ1, RQ2\)

Table 1:Method and supervision comparison\(Qwen3\-4B\-Instruct\-2507, avg@32\)\. All trained arms use the same 83 unlabeled AMC prompts; AMC is transductive, while the remaining columns evaluate the same checkpoints on benchmarks they never trained on\. GPU\-hours are measured wall\-clock and include generation\. Best label\-free result per column inbold; shaded rows use gold labels\. Seed variance and significance tests are reported in Appendices[H](https://arxiv.org/html/2607.13643#A8)and[E](https://arxiv.org/html/2607.13643#A5)\.Table[1](https://arxiv.org/html/2607.13643#S4.T1)compares all methods trained on the same 83 unlabeled AMC prompts, evaluated transductively on AMC and, without further training, on three other benchmarks\.

Among label\-free methods,Canonis the strongest by a wide margin\. It reaches 76\.5 avg@32 on the transductive cell over a base of 66\.0\. The four RL\-based alternatives cluster in a narrow band: TTRL at 70\.3, EMPO at 69\.6, SCRL at 71\.0, and TTRL\-Guard at 70\.5\. The two fine\-tuning\-based methods barely move the model: LMSI, which fine\-tunes on consensus\-filtered text with hard targets, reaches 66\.5, and ScPO reaches 66\.4, its vote\-margin filter retaining only 12 preference pairs from the 83 prompts, too few to train on at test\-time scale\. The four RL methods differ in how they turn agreement into a scalar reward, from the raw majority vote to semantic\-entropy minimization, consensus\-gated positive and negative labels, and stability\-weighted votes, yet they land within 1\.4 points of one another and 5 to 7 points belowCanon\. What separatesCanonis how much of the signal reaches the gradient\. Strengthening LMSI and ScPO beyond their papers’ protocols does not close the gap either \(Appendix[B](https://arxiv.org/html/2607.13643#A2)\)\. Across three training seeds,Canon’s AMC result varies by 0\.3 points, and across three evaluation seeds by 0\.2 \(Appendix[H](https://arxiv.org/html/2607.13643#A8)\)\.

Consensus conditioning also recovers much of the value of gold conditioning\. The oracle version, identical toCanonexcept that its teacher is anchored to deep\-resampled gold\-verified solutions, reaches 76\.9 againstCanon’s 76\.5, a difference within evaluation noise\. Trained transductively on AIME 2024 the two reach 48\.0 and 44\.6, and on GPQA 62\.5 and 54\.6 \(Table[6](https://arxiv.org/html/2607.13643#S6.T6)\); GPQA is the one benchmark where gold conditioning keeps a clear advantage \(Appendix[E](https://arxiv.org/html/2607.13643#A5)\)\. Much of what a gold solution provides as privileged conditioning, the model’s own consensus provides too\.

Gold\-reward reinforcement learning eventually surpassesCanontransductively, but at an order of magnitude more compute, and its extra gains do not transfer\. GRPO reaches 79\.6 at step 180, requiring 19\.2 GPU\-hours against 1\.2 and gold labels throughout; even so, its remaining advantage of 3\.1 points overCanonis not statistically significant under a paired bootstrap \(see[E](https://arxiv.org/html/2607.13643#A5)\)\. On the held\-out columns the picture inverts: the step\-180 GRPO checkpoint transfers no better thanCanonon AIME 2024 \(39\.7 against 39\.5\), and worse on AIME 2025 \(30\.4 against 32\.6\) and GPQA \(47\.6 against 48\.9\)\. The additional transductive points that gold\-reward RL buys with compute appear to be adaptation to the training prompts rather than transferable capability; we return to this in Section[4\.4](https://arxiv.org/html/2607.13643#S4.SS4)\.

Beyond the aggregate numbers, adaptation changes how the accuracy is delivered\. AfterCanon, a single greedy pass on AMC scores 76\.7 averaged over three evaluation seeds, up from 67\.1 for the base model and approaching the base model’s 32\-sample majority vote of 83\.1\. Much of the benefit of ensembling is amortized into the weights, which is the practically relevant regime when per\-query sampling budgets are constrained\.

![Refer to caption](https://arxiv.org/html/2607.13643v1/x2.png)\(a\)Accuracy against compute\.
![Refer to caption](https://arxiv.org/html/2607.13643v1/x3.png)\(b\)Oracle recovery\.

Figure 2:Transductive AMC, main comparisons\(Qwen3\-4B\-Instruct\-2507\)\. \(a\) Accuracy against measured wall\-clock on identical hardware, including generation\.Canonreaches within half a point of the gold\-conditioned oracle in 1\.2 GPU\-hours; gold\-reward GRPO needs roughly 11 hours to match it and 19 to surpass it, and TTRL plateaus belowCanonat every budget\. Filled markers and solid lines denote label\-free methods; open markers and dashed lines denote methods that use gold labels\. \(b\)Canon\(label\-free\) against the deep\-resampled oracle teacher \(gold labels\), per benchmark\. The model’s own consensus captures much of the gold solution’s value as privileged context, with the largest remaining gap on GPQA; variance and paired bootstrap intervals in Appendix[E](https://arxiv.org/html/2607.13643#A5)\.
### 4\.3Breadth Across Benchmarks, Models, and Families \(RQ2\)

Table 2:TransductiveCanonacross benchmarks, models, families, and scales\(avg@32\)\. One fixed recipe per model, selected once on development slices and never on a reported benchmark \(per\-model values in Appendix[B](https://arxiv.org/html/2607.13643#A2)\); each cell trains only on that benchmark’s unlabeled prompts\. Deltas over the corresponding base model in parentheses\. LFM2\.5\-8B\-A1B is a mixture\-of\-experts \(MoE\) model with 1B active parameters\. Majority\-vote and coverage metrics for the Qwen3\-4B\-Instruct\-2507 cells appear in Table[4](https://arxiv.org/html/2607.13643#S5.T4)\.Table[2](https://arxiv.org/html/2607.13643#S4.T2)appliesCanonacross five benchmarks and seven models spanning four families and scales from 2B to 9B, with one fixed recipe per model\. The gains are broad: 34 of the 35 reported cells improve, by\+5\.7\+5\.7to\+12\.1\+12\.1points on the four hard benchmarks for Qwen3\-4B\-Instruct\-2507 and by\+3\.1\+3\.1to\+9\.0\+9\.0for Qwen3\.5\-9B, the largest model\. MATH500 shows that saturation is a property of the model\-benchmark pair rather than the benchmark: Qwen3\-4B\-Instruct\-2507 already answers 89% of samples correctly and votes at 95%, leaving little room between expressed and consensus accuracy, and gains\+1\.8\+1\.8, while the unsaturated models gain\+3\.3\+3\.3to\+6\.6\+6\.6on the same benchmark \(Section[5\.2](https://arxiv.org/html/2607.13643#S5.SS2)\)\. AIME 2025 shows the smallest hard\-benchmark gain for Qwen3\-4B\-Instruct\-2507; its base consensus is also the weakest of the four \(43\.3 maj@32\), again tracking consensus quality\. Run transductively on AIME 2024 for comparison, TTRL reaches 39\.3 avg@32 at its best checkpoint againstCanon’s 44\.6\.

Evidence from a single model family can be weak on its own\.Shaoet al\.\([2026](https://arxiv.org/html/2607.13643#bib.bib34)\)show that reinforcement learning on Qwen models can produce large mathematical gains even from random or deliberately incorrect rewards, by amplifying reasoning behaviors already acquired during pretraining, that the same signals largely fail on other families, and accordingly recommend validating any proposed training signal across model families and against uninformative\-signal controls\. Our evidence addresses both recommendations\. Within the Qwen family, the content\-free conditioning control of Section[6](https://arxiv.org/html/2607.13643#S6)plays the role of the uninformative signal and yields no gain, so the improvements require the consensus content\. Across families, SmolLM3\-3B improves on all five benchmarks \(\+1\.6\+1\.6to\+5\.5\+5\.5\), and LFM2\.5\-8B\-A1B, a mixture\-of\-experts model, improves on four of five, with the largest majority\-vote gain of any reported cell on AMC \(\+4\.8\+4\.8points\) and a 3\.1\-point decline on MATH500\. Consensus\-level effects otherwise vary by model: the Qwen3\.5\-9B and SmolLM3 majority votes are unchanged on AMC, and the 9B AMC cell is stable across three training seeds \(mean 79\.2, sd 0\.7\)\. Gemma\-4\-E4B\-it, from a fourth family, improves on all five benchmarks, with the largest single\-cell gain in the table on AIME 2024 \(\+15\.4\+15\.4points\)\.

### 4\.4Inductive Transfer and Self\-Improvement \(RQ3\)

The transductive setting cannot distinguish genuine improvement from adaptation to the particular prompts being evaluated\. We therefore train label\-free on one prompt set and evaluate on disjoint benchmarks, in two configurations: a single\-source configuration, where the AMC\-trained checkpoints of Table[1](https://arxiv.org/html/2607.13643#S4.T1)are evaluated elsewhere, and a pooled configuration, where training uses a larger mixed\-mathematics pool\.

The held\-out columns of Table[1](https://arxiv.org/html/2607.13643#S4.T1)already contain the single\-source result: AMC\-trainedCanonimproves AIME 2024 by\+7\.0\+7\.0points over base, AIME 2025 by\+2\.5\+2\.5, and GPQA by\+5\.0\+5\.0, the last a transfer from competition mathematics to graduate\-level science questions\. The AIME 2025 gain is small on its own; on the combined AIME 2024\+2025 set \(n=60n\{=\}60\), AMC\-trainedCanonimproves by\+4\.7\+4\.7points against\+2\.7\+2\.7for TTRL and\+3\.8\+3\.8for GRPO\-180 \(Appendix[E](https://arxiv.org/html/2607.13643#A5)\)\. On AIME 2026, a post\-cutoff evaluation set constructed for this paper \(Appendix[I](https://arxiv.org/html/2607.13643#A9)\), AMC\-trainedCanonimproves majority\-vote accuracy from 51\.7 to 58\.6 with pass@1 moving within noise \(n=29n\{=\}29\)\. Qwen3\.5\-4B shows the same pattern, including transfer from easy to hard problems within OmniMath \(Appendix[G](https://arxiv.org/html/2607.13643#A7)\)\.

Table 3:Pooled label\-free training, held\-out evaluation\.Qwen3\-4B\-Instruct\-2507 trained on a 339\-prompt unlabeled pool \(Appendix[D](https://arxiv.org/html/2607.13643#A4)\) and evaluated on AIME 2024, which shares no prompts with the pool\. Trajectory methods \(TTRL, GRPO\) report their final checkpoint, since selecting an intermediate checkpoint would itself require labeled held\-out evaluation; full trajectories in Figure[3](https://arxiv.org/html/2607.13643#S4.F3)\. Shaded rows use gold labels\.![Refer to caption](https://arxiv.org/html/2607.13643v1/x4.png)
Figure 3:Held\-out AIME 2024 during pooled label\-free training\.The RL\-based methods climb slowly toward, and do not reach, the levelCanonattains in a single epoch \(dashed guide\); TTRL regresses past its peak checkpoint\. Filled markers and solid lines denote label\-free methods; open markers and dashed lines denote methods that use gold labels\.
Table[3](https://arxiv.org/html/2607.13643#S4.T3)scales the source data: all methods train on the 339\-prompt unlabeled pool and are evaluated on held\-out AIME 2024\.Canonreaches 41\.7 avg@32 in a single epoch, above the final checkpoints of both TTRL\-pool \(39\.4 after 34\.5 hours\) and gold\-reward GRPO\-pool \(39\.0\), and within a point of gold\-conditioned oracle distillation \(42\.4\); for scale, oneCanon\-pool epoch is roughly 55 optimizer steps at this batch size, while the horizontal axis of Figure[3](https://arxiv.org/html/2607.13643#S4.F3)counts the RL methods’ PPO steps\. TTRL does pass through a stronger intermediate checkpoint \(41\.4 at step 220, Figure[3](https://arxiv.org/html/2607.13643#S4.F3)\) before regressing, but identifying that checkpoint requires labeled held\-out evaluation that the label\-free setting does not provide;Canonneeds no such selection, since it trains for exactly one epoch\. That a label\-free method matches gold\-supervised training on held\-out data mirrors the single\-source finding above: dense consensus distillation appears to extract more transferable improvement per unit of supervision than sparse gold rewards, at roughly a seventh of the compute\. Two caveats apply: the evaluation set has 30 prompts, so one prompt moves maj@32 by 3\.3 points, andCanon\-pool’s majority\-vote accuracy lands below base here while GRPO\-pool’s does not\. Section[5\.1](https://arxiv.org/html/2607.13643#S5.SS1)examines consensus\-level effects where the statistics are adequate\.

Transfer also degrades gracefully with distance from the training distribution\. Training Qwen3\.5\-4B on one unlabeled OmniMath slice and evaluating the same checkpoint at increasing distance yields gains of\+9\.2\+9\.2points held\-in,\+4\.9\+4\.9on harder problems from the same source,\+0\.6\+0\.6on a different mathematics benchmark, and\+3\.9\+3\.9cross\-domain on GPQA \(Appendix[G](https://arxiv.org/html/2607.13643#A7)\)\. Label\-free self\-improvement is strongest near the training distribution but does not invert far from it\.

## 5Analysis

### 5\.1More than sharpening? \(RQ4\)

A natural concern with any self\-training signal is that it sharpens the output distribution, concentrating probability on answers the model already produces, rather than improving what the model can solve\(Huanget al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib15); Linet al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib19)\)\. Understood as concentration toward each prompt’s existing answers, sharpening can move the finite\-sample metrics in either direction, but two outcomes lie beyond it in expectation: it cannot make the majority answer correct on a prompt whose dominant answer was wrong, and it cannot systematically produce correct samples on prompts where no sampled answer was correct\.Canondoes concentrate the distribution, as expected: mean vote share on AMC rises from 0\.68 to 0\.81\. The question is whether concentration is all that happens\.

Table 4:Consensus and coverage effectsof transductiveCanon\(Qwen3\-4B\-Instruct\-2507\)\. Concentration toward existing answers cannot flip a wrong majority to a right one, and cannot cover a previously unsolved prompt\.Figure 4:Where the gains land\.Change in avg@32 by base\-model difficulty band \(AMC, transductive\)\. Gains concentrate on the hardest prompts, including a\+33\.3\+33\.3\-point pass@32 gain on never\-solved prompts \(hatched\)\.
![Refer to caption](https://arxiv.org/html/2607.13643v1/x5.png)
Concentration is not all of it\. Table[4](https://arxiv.org/html/2607.13643#S5.T4)shows majority\-vote accuracy improving on all four hard benchmarks, by\+2\.4\+2\.4to\+3\.3\+3\.3points, and pass@32, improving substantially on the mathematics benchmarks \(\+4\.8\+4\.8on AMC,\+10\.0\+10\.0on AIME 2024,\+16\.7\+16\.7on AIME 2025\)\. The model’s consensus becomes more often correct, and it solves problems it previously never solved in 32 attempts\. Per\-prompt transitions point the same way, with wrong\-to\-right majority flips outnumbering the reverse on every benchmark, though the per\-benchmark counts are small \(Appendix[J](https://arxiv.org/html/2607.13643#A10)\)\. Stratifying prompts by base difficulty makes the coverage effect explicit\. Gains are largest in the hardest bands,\+51\.9\+51\.9avg@32 points on prompts the base model solved in under a quarter of its samples and\+33\.3\+33\.3pass@32 points on prompts it never solved, and are near zero on already\-solved prompts \(Figure[4](https://arxiv.org/html/2607.13643#S5.F4)\)\. GPQA and MATH500 show flat coverage, consistent with their headroom profiles discussed below\. Further controls are provided in Appendix[J](https://arxiv.org/html/2607.13643#A10); binning prompts on independent base\-model evaluation draws preserves the difficulty profile, and on a fresh 64\-sample drawCanon’s pass@kkstays above the base model’s at everykk, showing no sign of the pass@k crossover reported for RL\-trained modelsYueet al\.\([2025](https://arxiv.org/html/2607.13643#bib.bib38)\)\.

### 5\.2When Does Consensus Supervision Help?

The supervisionCanonconstructs is only as good as the consensus behind it, which predicts where the gains should concentrate\. To test this, we pool per\-prompt gains across twelve reported transductive cells of Tables[1](https://arxiv.org/html/2607.13643#S4.T1)and[2](https://arxiv.org/html/2607.13643#S4.T2), giving 1,493 prompt\-level observations across the three models, and classify each prompt by the state of the base model’s consensus: whether the majority answer is correct, and whether it is held confidently \(vote share of at least0\.50\.5\)\. The prediction holds\. Gains concentrate where the majority is correct but unsure \(\+21\.5\+21\.5points, 239 prompts\); they are moderate where it is correct and saturated \(\+4\.3\+4\.3, 875 prompts\), small where it is wrong but weakly held \(\+2\.6\+2\.6, 338\), and negative where the samples collude on a wrong answer \(−3\.0\-3\.0, 41\), the one regime in which the anchor points the wrong way\. Confidently wrong consensus is also rare, which is why the aggregate effect is strongly positive\.

Table 5:Gains by base vote\-share band: change in avg@32 after transductive training, in fixed\-width bins of the base model’s vote share over its 32 evaluation samples, pooled over the twelve reported transductive cells\.Table[5](https://arxiv.org/html/2607.13643#S5.T5)shows the same pattern at finer grain\. Gains are largest through the middle of the vote\-share range, where the consensus is informative but not yet expressed in single samples; they are moderate at the bottom, where the consensus is often too weak to anchor reliably, and near zero at the saturated top, where there is nothing left to distill\.

This yields a practical selection rule\.Canonis indicated when base consensus accuracy meaningfully exceeds base pass@1, which is exactly the regime of small instruct models on competition mathematics and multiple\-choice science, and contra\-indicated where that gap has closed or the consensus is unreliable\. Saturation is a property of the model\-benchmark pair, not the benchmark: on MATH500 the gap is six points for Qwen3\-4B\-Instruct\-2507, which gains\+1\.8\+1\.8, while the unsaturated Qwen3\.5 models gain\+5\.3\+5\.3and\+6\.6\+6\.6\. The gap is measurable on a labeled development slice, and vote\-share statistics alone provide a label\-free proxy\.

## 6Ablations

The gains of Section[4](https://arxiv.org/html/2607.13643#S4)could in principle come from several sources: from the fine\-tuning procedure itself, from conditioning on generic solution\-shaped text, from the consensus answer leaking into the teacher, or from the reasoning content of the consensus solution\. To separate these, we hold the entire recipe fixed and vary only what the teacher is conditioned on, from nothing at all up to the gold solution\.

Table 6:Conditioning ladder\(Qwen3\-4B\-Instruct\-2507, transductive; avg@32 / maj@32\)\. All arms are identical toCanonexcept for the teacher’s conditioning content\. Both oracles use deep\-resampled gold solutions \(protocol and coverage in Appendix[B](https://arxiv.org/html/2607.13643#A2)\)\.Table 7:Design and mechanism ablations\(transductive AMC, avg@32; one change at a time from the locked recipe\)\. Teacher and rollout\-freshness ablations and further detail in Appendix[F](https://arxiv.org/html/2607.13643#A6)\.Table[6](https://arxiv.org/html/2607.13643#S6.T6)reports the ladder\. A content\-free placeholder leaves the model essentially unchanged, ruling out the training procedure itself as the source of gains\. Conditioning on the consensus answer alone recovers roughly a third ofCanon’s improvement \(\+3\.5\+3\.5AMC,\+3\.3\+3\.3GPQA over base\), and conditioning on another prompt’s consensus solution, correctly formatted but about the wrong problem, lands at the same level \(69\.9 and 47\.7\): solution\-shaped text and a bare answer each explain only the first rung\. The full consensus solution adds a further\+7\.0\+7\.0and\+7\.4\+7\.4on top\. The reasoning trace about the problem at hand, not the answer token or the format, carries most of the signal, consistent with the view that the teacher’s value lies in anchoring a line of reasoning rather than leaking a label\.

The remaining design choices, and a set of mechanism arms that vary one component of the supervision at a time, are summarized in Table[7](https://arxiv.org/html/2607.13643#S6.T7)and detailed in Appendix[F](https://arxiv.org/html/2607.13643#A6)\. Three findings carry the section\. First, the teacher must be a frozen snapshot: a live shared\-parameter teacher collapses training entirely, with generation entropy and the training loss reaching zero within a handful of steps \(Appendix[F](https://arxiv.org/html/2607.13643#A6)\)\. Second, the supervision must be dense and must cover all rollouts: sequence\-level SFT toward the same consensus solution, selected by the same rule, gains\+1\.4\+1\.4points where distillation gains\+10\.5\+10\.5, and masking out minority\-answer rollouts costs3\.53\.5points, so rollouts that disagree with the consensus still carry signal, consistent with the vote\-share\-gate ablation\. Third, the budget knobs are forgiving: increasing the rollout countNNimproves results monotonically up toN=32N\{=\}32and plateaus at 64, with evenN=4N\{=\}4recovering two thirds of the gain, and additional epochs with regenerated rollouts add at most a point before reversing \(Appendix[F](https://arxiv.org/html/2607.13643#A6)\)\.

## 7Conclusion

We showed that a model’s own consensus can serve as privileged context for self\-distillation, turning the signal behind majority voting into dense, token\-level, label\-free supervision\. Instantiated with a frozen snapshot teacher and a single training epoch,Canonoutperforms label\-free reinforcement learning and consensus fine\-tuning on transductive test\-time training at a fraction of their compute, approaches gold\-conditioned teachers, and transfers to held\-out benchmarks, where it matches training methods that use gold labels\. The gains go beyond pure sharpening: consensus accuracy and coverage rise alongside pass@1, concentrated on problems the base model rarely or never solved within its sampling budget\.

### 7\.1Limitations

Canonrequires an extractable final answer to form a consensus, restricting it to verifiable\-output domains; replacing exact\-match voting with minimum\-Bayes\-risk or medoid selection over sampled solutions would extend consensus extraction to free\-form generation, and we leave this to future work\. The method distills the gap between expressed and consensus accuracy, so it offers little on saturated model\-benchmark pairs and can reinforce errors where the samples collude on a wrong answer, a measurable but rare failure mode \(41 of 1493 pooled prompt\-level observations in our analysis\)\. We study a single round of self\-distillation; in preliminary experiments, iterating the procedure yields at most one additional productive round before returns turn negative\. Finally, the transductive regime assumes access to test prompts before answering, which is natural for batch workloads but not for interactive use; the inductive results suggest the assumption can be relaxed\.

### 7\.2Acknowledgments

The authors thank the Netherlands Organization for Scientific Research \(NWO\) for their support \(VICI grant VI\.C\.212\.053\)\. This work used the Dutch national e\-infrastructure with the support of the SURF Cooperative using grant no\. EINF\-15485\. We acknowledge EuroHPC Joint Undertaking for awarding us access to Leonardo at CINECA, Italy\.

## References

- On\-policy distillation of language models: learning from self\-generated mistakes\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 21246–21263\.Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px4.p1.1)\.
- E\. Akyürek, M\. Damani, A\. Zweiger, L\. Qiu, H\. Guo, J\. Pari, Y\. Kim, and J\. Andreas \(2024\)The surprising effectiveness of test\-time training for few\-shot learning\.arXiv preprint arXiv:2411\.07279\.Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Blum and T\. Mitchell \(1998\)Combining labeled and unlabeled data with co\-training\.InProceedings of the eleventh annual conference on Computational learning theory,pp\. 92–100\.Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px3.p1.1)\.
- M\. Collins and Y\. Singer \(1999\)Unsupervised models for named entity classification\.In1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora,External Links:[Link](https://aclanthology.org/W99-0613/)Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px3.p1.1)\.
- B\. Gao, F\. Song, Z\. Yang, Z\. Cai, Y\. Miao, Q\. Dong, L\. Li, C\. Ma, L\. Chen, R\. Xu, Z\. Tang, B\. Wang, D\. Zan, S\. Quan, G\. Zhang, L\. Sha, Y\. Zhang, X\. Ren, T\. Liu, and B\. Chang \(2025\)Omni\-MATH: a universal olympiad level mathematic benchmark for large language models\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=yaqPf0KAlN)Cited by:[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p2.1)\.
- D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.\(2025\)DeepSeek\-r1 incentivizes reasoning in llms through reinforcement learning\.Nature645\(8081\),pp\. 633–638\.Cited by:[§1](https://arxiv.org/html/2607.13643#S1.p1.1)\.
- E\. J\. Hu, yelong shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen \(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=nZeVKeeFYf9)Cited by:[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p5.1)\.
- A\. Huang, A\. Block, D\. J\. Foster, D\. Rohatgi, C\. Zhang, M\. Simchowitz, J\. T\. Ash, and A\. Krishnamurthy \(2025\)Self\-improvement in language models: the sharpening mechanism\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WJaUkwci9o)Cited by:[§5\.1](https://arxiv.org/html/2607.13643#S5.SS1.p1.1)\.
- J\. Huang, S\. Gu, L\. Hou, Y\. Wu, X\. Wang, H\. Yu, and J\. Han \(2023\)Large language models can self\-improve\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 1051–1068\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.67/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.67)Cited by:[Appendix B](https://arxiv.org/html/2607.13643#A2.SS0.SSS0.Px5.p1.10),[§1](https://arxiv.org/html/2607.13643#S1.p2.1),[§1](https://arxiv.org/html/2607.13643#S1.p4.1),[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p3.1)\.
- J\. Hübotter, F\. Lübeck, L\. Behric, A\. Baumann, M\. Bagatella, D\. Marta, I\. Hakimi, I\. Shenfeld, T\. K\. Buening, C\. Guestrin,et al\.\(2026\)Reinforcement learning via self\-distillation\.arXiv preprint arXiv:2601\.20802\.Cited by:[§1](https://arxiv.org/html/2607.13643#S1.p2.1),[§1](https://arxiv.org/html/2607.13643#S1.p4.1),[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px4.p1.1)\.
- H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe \(2024\)Let’s verify step by step\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 39578–39601\.Cited by:[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p2.1)\.
- H\. Lin, Z\. Kuai, E\. Xue, and L\. Wang \(2026\)When the majority votes wrong, the intervention timing for test\-time reinforcement learning hides in the extinction window\.arXiv preprint arXiv:2605\.19444\.Cited by:[Appendix B](https://arxiv.org/html/2607.13643#A2.SS0.SSS0.Px6.p1.1),[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p3.1),[§5\.1](https://arxiv.org/html/2607.13643#S5.SS1.p1.1)\.
- I\. Loshchilov and F\. Hutter \(2019\)Decoupled weight decay regularization\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Bkg6RiCqY7)Cited by:[Appendix A](https://arxiv.org/html/2607.13643#A1.p6.2)\.
- A\. Prasad, W\. Yuan, R\. Y\. Pang, J\. Xu, M\. Fazel\-Zarandi, M\. Bansal, S\. Sukhbaatar, J\. E\. Weston, and J\. Yu \(2025\)Self\-consistency preference optimization\.InProceedings of the 42nd International Conference on Machine Learning,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267,pp\. 49737–49751\.External Links:[Link](https://proceedings.mlr.press/v267/prasad25a.html)Cited by:[§1](https://arxiv.org/html/2607.13643#S1.p1.1),[§1](https://arxiv.org/html/2607.13643#S1.p2.1),[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p3.1)\.
- D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. Bowman \(2024\)GPQA: a graduate\-level google\-proof q&a benchmark\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=Ti67584b98)Cited by:[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p2.1)\.
- S\. Shafayat, F\. Tajwar, R\. Salakhutdinov, J\. Schneider, and A\. Zanette \(2026\)Can large reasoning models self\-train?\.External Links:[Link](https://openreview.net/forum?id=wkRrpXD8Np)Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px2.p1.1)\.
- R\. Shao, S\. S\. Li, R\. Xin, S\. Geng, Y\. Wang, S\. Oh, S\. S\. Du, N\. Lambert, S\. Min, R\. Krishna, Y\. Tsvetkov, H\. Hajishirzi, P\. W\. Koh, and L\. Zettlemoyer \(2026\)Spurious rewards: rethinking training signals in RLVR\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=tqTNOpkP5j)Cited by:[§4\.3](https://arxiv.org/html/2607.13643#S4.SS3.p2.4)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.CoRRabs/2402\.03300\.External Links:[Link](https://doi.org/10.48550/arXiv.2402.03300)Cited by:[§1](https://arxiv.org/html/2607.13643#S1.p1.1),[§1](https://arxiv.org/html/2607.13643#S1.p4.1),[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p3.1)\.
- G\. Sheng, C\. Zhang, Z\. Ye, X\. Wu, W\. Zhang, R\. Zhang, Y\. Peng, H\. Lin, and C\. Wu \(2024\)HybridFlow: a flexible and efficient rlhf framework\.arXiv preprint arXiv: 2409\.19256\.Cited by:[Appendix B](https://arxiv.org/html/2607.13643#A2.SS0.SSS0.Px1.p1.6)\.
- A\. Singh, J\. D\. Co\-Reyes, R\. Agarwal, A\. Anand, P\. Patil, X\. Garcia, P\. J\. Liu, J\. Harrison, J\. Lee, K\. Xu, A\. T\. Parisi, A\. Kumar, A\. A\. Alemi, A\. Rizkowsky, A\. Nova, B\. Adlam, B\. Bohnet, G\. F\. Elsayed, H\. Sedghi, I\. Mordatch, I\. Simpson, I\. Gur, J\. Snoek, J\. Pennington, J\. Hron, K\. Kenealy, K\. Swersky, K\. Mahajan, L\. A\. Culp, L\. Xiao, M\. Bileschi, N\. Constant, R\. Novak, R\. Liu, T\. Warkentin, Y\. Bansal, E\. Dyer, B\. Neyshabur, J\. Sohl\-Dickstein, and N\. Fiedel \(2024\)Beyond human data: scaling self\-training for problem\-solving with language models\.Transactions on Machine Learning Research\.Note:Expert CertificationExternal Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=lNAyUngGFK)Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px3.p1.1)\.
- C\. V\. Snell, D\. Klein, and R\. Zhong \(2023\)Learning by distilling context\.External Links:[Link](https://openreview.net/forum?id=am22IukDiKf)Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px4.p1.1)\.
- M\. Song and M\. Zheng \(2026\)A survey of on\-policy distillation for large language models\.arXiv preprint arXiv:2604\.00626\.Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px4.p1.1)\.
- A\. Stein, F\. Huang, and T\. Goldstein \(2026\)Gates: self\-distillation under privileged context with consensus gating\.arXiv preprint arXiv:2602\.20574\.Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px4.p1.1)\.
- Y\. Sun, X\. Wang, Z\. Liu, J\. Miller, A\. Efros, and M\. Hardt \(2020\)Test\-time training with self\-supervision for generalization under distribution shifts\.InInternational conference on machine learning,pp\. 9229–9248\.Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px1.p1.1)\.
- W\. Wang, F\. Wei, L\. Dong, H\. Bao, N\. Yang, and M\. Zhou \(2020\)Minilm: deep self\-attention distillation for task\-agnostic compression of pre\-trained transformers\.Advances in neural information processing systems33,pp\. 5776–5788\.Cited by:[Appendix I](https://arxiv.org/html/2607.13643#A9.p2.1)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2022\)Self\-consistency improves chain of thought reasoning in language models\.arXiv preprint arXiv:2203\.11171\.Cited by:[§1](https://arxiv.org/html/2607.13643#S1.p1.1),[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px1.p1.1)\.
- D\. Yan, J\. Liang, Y\. Wang, S\. Lu, R\. He, and T\. Tan \(2026\)What if consensus lies? selective\-complementary reinforcement learning at test time\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 28957–28970\.External Links:[Link](https://aclanthology.org/2026.acl-long.1337/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.1337),ISBN 979\-8\-89176\-390\-6Cited by:[Appendix B](https://arxiv.org/html/2607.13643#A2.SS0.SSS0.Px6.p1.1),[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p3.1)\.
- Z\. Yuan, H\. Yuan, C\. Li, G\. Dong, K\. Lu, C\. Tan, C\. Zhou, and J\. Zhou \(2024\)Scaling relationship on learning mathematical reasoning with large language models\.External Links:[Link](https://openreview.net/forum?id=cijO0f8u35)Cited by:[§1](https://arxiv.org/html/2607.13643#S1.p4.1),[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p3.1)\.
- Y\. Yue, Z\. Chen, R\. Lu, A\. Zhao, Z\. Wang, Y\. Yue, S\. Song, and G\. Huang \(2025\)Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=4OsgYD7em5)Cited by:[Appendix J](https://arxiv.org/html/2607.13643#A10.SS0.SSS0.Px2.p1.6),[§5\.1](https://arxiv.org/html/2607.13643#S5.SS1.p2.9)\.
- E\. Zelikman, Y\. Wu, J\. Mu, and N\. Goodman \(2022\)Star: bootstrapping reasoning with reasoning\.Advances in Neural Information Processing Systems35,pp\. 15476–15488\.Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px3.p1.1)\.
- Q\. Zhang, H\. Wu, C\. Zhang, P\. Zhao, and Y\. Bian \(2025\)Right question is already half the answer: fully unsupervised LLM reasoning incentivization\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=k8Mim6RI5O)Cited by:[Appendix B](https://arxiv.org/html/2607.13643#A2.SS0.SSS0.Px6.p1.1),[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p3.1)\.
- S\. Zhao, Z\. Xie, M\. Liu, J\. Huang, G\. Pang, F\. Chen, and A\. Grover \(2026a\)Self\-distilled reasoner: on\-policy self\-distillation for large language models\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=Jpxfof0EaS)Cited by:[§1](https://arxiv.org/html/2607.13643#S1.p2.1),[§1](https://arxiv.org/html/2607.13643#S1.p4.1),[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px4.p1.1)\.
- X\. Zhao, Z\. Kang, A\. Feng, S\. Levine, and D\. Song \(2026b\)Learning to reason without external rewards\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=OU9nFEYR2M)Cited by:[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Zuo, K\. Zhang, L\. Sheng, S\. Qu, G\. Cui, X\. Zhu, H\. Li, Y\. Zhang, X\. Long, E\. Hua, B\. Qi, Y\. Sun, Z\. Ma, L\. Yuan, N\. Ding, and B\. Zhou \(2025\)TTRL: test\-time reinforcement learning\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=VuVhgEiu20)Cited by:[§1](https://arxiv.org/html/2607.13643#S1.p2.1),[§1](https://arxiv.org/html/2607.13643#S1.p4.1),[§2](https://arxiv.org/html/2607.13643#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2607.13643#S4.SS1.p3.1)\.

## Appendix AFormal Setup, Objective, and Training

This appendix restates the method of Section[3](https://arxiv.org/html/2607.13643#S3)with full notation\.

Setup\.Let𝒱\\mathcal\{V\}be the vocabulary and𝒴=𝒱∗\\mathcal\{Y\}=\\mathcal\{V\}^\{\*\}the space of token sequences\. A policyπθ​\(y∣x\)=∏t=1\|y\|πθ​\(yt∣x,y<t\)\\pi\_\{\\theta\}\(y\\mid x\)=\\prod\_\{t=1\}^\{\|y\|\}\\pi\_\{\\theta\}\(y\_\{t\}\\mid x,y\_\{<t\}\)is initialized from a pretrained modelπ0\\pi\_\{0\}and adapted through a low\-rank parameterization ofθ\\theta\. We are given unlabeled prompts𝒟=\{x1,…,xM\}\\mathcal\{D\}=\\\{x\_\{1\},\\ldots,x\_\{M\}\\\}and a deterministic answer\-extraction mapans:𝒴→𝒜∪\{⊥\}\\operatorname\{ans\}:\\mathcal\{Y\}\\to\\mathcal\{A\}\\cup\\\{\\bot\\\}, where𝒜\\mathcal\{A\}is the answer space \(a canonical form for boxed expressions, or a choice letter\) and⊥\\botmarks sequences without an extractable answer\. No reward function, verifier, or gold answer is available to the learner\.

Consensus\.For each promptxx, drawNNrolloutsy\(1\),…,y\(N\)∼iidπ0\(⋅∣x\)y^\{\(1\)\},\\ldots,y^\{\(N\)\}\\stackrel\{\{\\scriptstyle\\text\{iid\}\}\}\{\{\\sim\}\}\\pi\_\{0\}\(\\cdot\\mid x\)and letai=ans⁡\(y\(i\)\)a\_\{i\}=\\operatorname\{ans\}\(y^\{\(i\)\}\)\. The majority answer is the plug\-in mode of the induced answer distribution,

a^x=arg​maxa∈𝒜​∑i=1N𝟏​\[ai=a\],\\hat\{a\}\_\{x\}=\\operatorname\*\{arg\\,max\}\_\{a\\in\\mathcal\{A\}\}\\sum\_\{i=1\}^\{N\}\\mathbf\{1\}\\big\[a\_\{i\}=a\\big\],\(3\)with size ties resolved to the first\-formed group and prompts whose rollouts are all unparseable removed from𝒟\\mathcal\{D\}\. Writing𝒮x=\{y\(i\):ai=a^x\}\\mathcal\{S\}\_\{x\}=\\\{y^\{\(i\)\}:a\_\{i\}=\\hat\{a\}\_\{x\}\\\}for the majority set, the consensus solution is its most confident member under the generating policy,

sx=arg​maxy∈𝒮x⁡1\|y\|​∑t=1\|y\|log⁡π0​\(yt∣x,y<t\)\.s\_\{x\}=\\operatorname\*\{arg\\,max\}\_\{y\\in\\mathcal\{S\}\_\{x\}\}\\;\\frac\{1\}\{\|y\|\}\\sum\_\{t=1\}^\{\|y\|\}\\log\\pi\_\{0\}\\big\(y\_\{t\}\\mid x,y\_\{<t\}\\big\)\.\(4\)Both selections are deterministic given the rollouts\.

Teacher\.Letc​\(x,s\)c\(x,s\)denote the augmented context that embeds a solutionssinto the prompt forxxthrough the fixed template of Appendix[C](https://arxiv.org/html/2607.13643#A3)\. The teacher is the conditional next\-token distribution of a frozen snapshotπ¯\\bar\{\\pi\}of the policy under this context,π¯\(⋅∣c\(x,sx\),y<t\)\\bar\{\\pi\}\(\\cdot\\mid c\(x,s\_\{x\}\),y\_\{<t\}\)\. The snapshot is taken at the start of the epoch; in the single\-epoch recipe,π¯=π0\\bar\{\\pi\}=\\pi\_\{0\}\. The teacher is never sampled from: it is evaluated by teacher forcing on externally supplied token sequences, and no gradient flows through it\.

Objective\.Training minimizes the empirical token\-mean Jensen\-Shannon divergence between student and teacher next\-token distributions over the sampled rollouts,

ℒ^\(θ\)=1N​M∑x∈𝒟∑i=1N1\|y\(i\)\|∑t=1\|y\(i\)\|DJS\(πθ\(⋅∣x,y<t\(i\)\)∥π¯\(⋅∣c\(x,sx\),y<t\(i\)\)\),\\hat\{\\mathcal\{L\}\}\(\\theta\)=\\frac\{1\}\{NM\}\\sum\_\{x\\in\\mathcal\{D\}\}\\sum\_\{i=1\}^\{N\}\\frac\{1\}\{\|y^\{\(i\)\}\|\}\\sum\_\{t=1\}^\{\|y^\{\(i\)\}\|\}D\_\{\\mathrm\{JS\}\}\\\!\\left\(\\pi\_\{\\theta\}\\big\(\\cdot\\mid x,\\,y^\{\(i\)\}\_\{<t\}\\big\)\\,\\\|\\,\\bar\{\\pi\}\\big\(\\cdot\\mid c\(x,s\_\{x\}\),\\,y^\{\(i\)\}\_\{<t\}\\big\)\\right\),\(5\)that is, each rollout contributes its token\-mean divergence with equal per\-sequence weight, so longer rollouts do not carry proportionally more gradient;DJS​\(p∥q\)=12​DKL​\(p∥m\)\+12​DKL​\(q∥m\)D\_\{\\mathrm\{JS\}\}\\\!\\left\(p\\,\\\|\\,q\\right\)=\\tfrac\{1\}\{2\}D\_\{\\mathrm\{KL\}\}\\\!\\left\(p\\,\\\|\\,m\\right\)\+\\tfrac\{1\}\{2\}D\_\{\\mathrm\{KL\}\}\\\!\\left\(q\\,\\\|\\,m\\right\)withm=12​\(p\+q\)m=\\tfrac\{1\}\{2\}\(p\+q\), and both arguments are full distributions over𝒱\\mathcal\{V\}, computed in the implementation as fp32 log\-softmax over bf16 logits with no temperature\. The divergence is symmetric, finite for any pair of distributions, and bounded bylog⁡2\\log 2, which makes the per\-token loss bounded regardless of how far teacher and student disagree; this contrasts with forward or reverse KL, either of which is unbounded under support mismatch\. Treating the teacher as constant, the per\-token gradient takes the simple form

∇θDJS​\(pθ∥q\)=12​∑v∈𝒱∇θpθ​\(v\)​log⁡pθ​\(v\)m​\(v\),\\nabla\_\{\\theta\}\\,D\_\{\\mathrm\{JS\}\}\\\!\\left\(p\_\{\\theta\}\\,\\\|\\,q\\right\)=\\frac\{1\}\{2\}\\sum\_\{v\\in\\mathcal\{V\}\}\\nabla\_\{\\theta\}\\,p\_\{\\theta\}\(v\)\\,\\log\\frac\{p\_\{\\theta\}\(v\)\}\{m\(v\)\},\(6\)which vanishes at the distributional optimumpθ=qp\_\{\\theta\}=q\. The log\-ratiolog⁡\(pθ​\(v\)m​\(v\)\)\\log\\left\(\\frac\{p\_\{\\theta\}\(v\)\}\{m\(v\)\}\\right\)is bounded above bylog⁡2\\log 2\.

Training\.One epoch of stochastic optimization over the fixed set of∑xN\\sum\_\{x\}Nsequences is performed with AdamW\[Loshchilov and Hutter,[2019](https://arxiv.org/html/2607.13643#bib.bib26)\]under the configuration of Appendix[B](https://arxiv.org/html/2607.13643#A2), with the teacher held fixed throughout\. The rollouts serving as the distillation substrate are the sameNNsamples that defined the consensus, drawn from the policy at the start of the epoch; their staleness relative to the evolving student is therefore bounded by a single epoch of optimization\. In the multi\-epoch generalization, rollouts are regenerated and the teacher re\-snapshotted at each epoch boundary, but all results in this paper use one epoch\.

## Appendix BImplementation Details

#### Training configuration\.

Canontrains LoRA adapters \(rank 64,α=128\\alpha\{=\}128, zero dropout, applied to all linear layers\) with AdamW \(β1=0\.9\\beta\_\{1\}\{=\}0\.9,β2=0\.999\\beta\_\{2\}\{=\}0\.999, weight decay 0\.01\), a constant learning rate of10−510^\{\-5\}with no warmup, gradient clipping at 1\.0, and micro\-batch size 1, for a single epoch over theN×MN\{\\times\}Msampled sequences; on an 83\-prompt benchmark withN=32N\{=\}32this is 2,656 sequences and roughly 13 optimizer steps\. Rollout generation for training uses vLLM at temperature 1\.0; the evaluation harness of Section[4\.1](https://arxiv.org/html/2607.13643#S4.SS1)\(temperature 0\.6\) is never used to produce training rollouts\. Training uses a response cap of 4,608 tokens \(3,072 where memory\-bound\)\. Training runs under FSDP via verl\[Shenget al\.,[2024](https://arxiv.org/html/2607.13643#bib.bib25)\]on four\-GPU nodes; wall\-clock figures are reported for a single reference node with four RTX 6000 Ada GPUs \(48 GB\)\. Each model’s generation mode is fixed across training and evaluation \(Table[8](https://arxiv.org/html/2607.13643#A2.T8)\), multiple\-choice training on GPQA uses a 2,048\-token prompt cap, and answers are extracted from`\\boxed\{\}`with math\-verify equivalence checking for mathematics and by letter extraction for multiple choice \(Appendix[C](https://arxiv.org/html/2607.13643#A3)\)\.

#### Consensus mechanics\.

In the consensus computation, vote share is measured over allNNrollouts including unparseable ones, and size ties between answer groups resolve to the first\-formed group; the selection of the consensus solution by mean token log\-probability is deterministic\.

#### Software\.

Training uses verl 0\.9\.0\.dev0 with vLLM 0\.22\.1, transformers 5\.12\.1, torch 2\.11\.0, and math\-verify 0\.9\.0; TTRL and GRPO instead run unmodified in TTRL’s own released stack \(verl 0\.4\.1, vLLM 0\.8\.4, transformers 4\.52\.4\), so the faithful baseline keeps its reference environment\.

#### Hyperparameter selection\.

Because transductive experiments offer no validation set, hyperparameter selection must happen elsewhere\. We therefore swept the learning rate once per model on a 64\-prompt OmniMath development slice \(difficulty 2 to 3\.5\), disjoint from every evaluation benchmark, and froze the selected value for all experiments in the paper\. Table[9](https://arxiv.org/html/2607.13643#A2.T9)reports the sweep;10−510^\{\-5\}wins on avg@32 for both models and is used throughout, so no benchmark result in the paper reflects benchmark\-specific tuning\. The remaining models of Table[2](https://arxiv.org/html/2607.13643#S4.T2)follow the same protocol, with the learning rate, training response cap, and generation mode selected once per model on dedicated development slices and then frozen; Table[8](https://arxiv.org/html/2607.13643#A2.T8)lists the selected values\.

Table 8:Per\-model recipe values \(all other recipe components identical across models; everything selected on development slices\)\.Table 9:Development learning\-rate sweep \(OmniMath dev slice, avg@32 / maj@32\)\. The selected value, in bold, is frozen for all reported experiments\.
#### Baselines\.

The baselines follow their reference implementations wherever one exists\. TTRL is run faithfully from the released codebase and configuration \(learning rate5⋅10−75\\cdot 10^\{\-7\}, GRPO advantage estimator, majority\-vote pseudo\-reward with 64 votes, 32 samples per prompt, response cap 3,072, no KL penalty\), with training and validation both set to the unlabeled target prompts\. GRPO is identical except that the reward verifies against gold answers, which makes the TTRL/GRPO pair a controlled comparison of pseudo\-reward against gold reward under the same optimizer\. LMSI follows the self\-consistency\-filtered SFT recipe\[Huanget al\.,[2023](https://arxiv.org/html/2607.13643#bib.bib2)\]: 32 rollouts per prompt are sampled at temperature 1\.0, prompts whose majority vote share reaches 0\.5 are kept \(56 of 83 on AMC\), and all majority\-group rollouts become SFT targets \(1,661 sequences\), trained for one epoch at learning rate10−510^\{\-5\}with the same LoRA configuration; the reported row is a faithful retrain of this configuration whose transductive result matches the original run to 0\.1 points, and its measured train pipeline \(generation, consensus filtering, and SFT\) takes 84 minutes end to end\. RFT samples 16 rollouts per prompt at temperature0\.60\.6, keeps prompts with at least one gold\-correct rollout \(70 of 83 on AMC\), and fine\-tunes on the shortest correct rollout per prompt \(LoRA rank 64, learning rate10−510^\{\-5\}, 3 epochs, response\-masked loss\)\. ScPO is implemented from the paper, as no official code is released: 32 rollouts per prompt are clustered by final\-answer equivalence with the repository’s extraction machinery; the chosen response is a random member of the highest\-vote cluster and the rejected response a random member of the least\-voted cluster; prompts are kept only if the majority vote reachesτ​k\\tau kwithτ=0\.5\\tau\{=\}0\.5, and each pair is weighted by the vote margin\. The objective is the paper’s weighted DPO plus a weighted length\-normalized NLL term \(β=0\.5\\beta\{=\}0\.5,α=1\\alpha\{=\}1, learning rate5⋅10−65\\cdot 10^\{\-6\}\), with the frozen base model as reference via adapter disabling\. Logged deviations from the paper: a single iteration and epoch, LoRA rank 64 rather than full fine\-tuning, and a fresh seeded rollout draw\. On the 83 AMC prompts the construction yields 12 pairs \(44 prompts are unanimous and 27 fail the vote filter\), which is itself an observation about consensus\-preference methods at test\-time scale\. Strengthening these baselines beyond their papers’ protocols does not change the picture: LMSI run for three epochs with fresh generation and re\-filtering each epoch reaches 68\.3, and a second ScPO iteration at the paper’s iteration\-two threshold \(τ=0\.7\\tau\{=\}0\.7\) retains 3 pairs and reaches 66\.0\.

#### EMPO, SCRL, and TTRL\-Guard\.

All three run from their official releases on the same 83 prompts at the TTRL arm’s budget: EMPO, which minimizes semantic entropy over answer clusters and serves as a confidence\-sharpening representative\[Zhanget al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib17)\]; SCRL\[Yanet al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib28)\]; and TTRL\-Guard\[Linet al\.,[2026](https://arxiv.org/html/2607.13643#bib.bib19)\], whose released code reduces to its confidence\-reweighting component in this setting, a caveat we log rather than repair\. Their avg@32 trajectories at steps 20/40/60 are 68\.2/70\.0/69\.6 \(EMPO\), 69\.1/70\.4/71\.0 \(SCRL\), and 68\.6/71\.6/70\.5 \(TTRL\-Guard\): all end between 69\.6 and 71\.0, inside the TTRL band and well belowCanon, and none exceeds the base model’s majority\-vote accuracy at its endpoint\. Held\-out, the three land in the TTRL band as well \(avg@32 on AIME 2024 / AIME 2025 / GPQA: EMPO 36\.9/31\.7/45\.0; SCRL 37\.3/31\.4/45\.0; TTRL\-Guard 34\.9/33\.2/45\.7\)\.

#### Oracle construction\.

Oracle distillation is identical toCanonexcept that the teacher is conditioned on a gold\-correct rollout, ranked by the same mean\-token\-log\-probability statistic that selects the consensus solution\. On AMC and AIME 2024 the conditioning solutions come from a dedicated deep\-resampling pass rather than the training rollouts: up to 256 samples per prompt at temperature 1\.0 under one seed, a second 256\-sample pass at a second seed for prompts still unsolved, and 512 for the remainder, retaining the most confident gold\-correct rollout per prompt\. This yields gold conditioning for 75 of 83 AMC prompts, 23 of 30 AIME 2024 prompts, and 179 of 198 GPQA prompts, with per\-prompt coverage records; the uncovered prompts \(0 correct in 256 to 512 samples\) fall back to consensus conditioning, and restricting the oracle comparisons to covered prompts changes no conclusion, since the fallback prompts are ones the base model never solves\. The pooled oracle arm uses trainer\-internal verification over theN=32N\{=\}32training rollouts instead\. The resampling pass is a one\-off cost excluded from the oracle’s GPU\-hours in Table[1](https://arxiv.org/html/2607.13643#S4.T1)\.

## Appendix CPrompt Templates and Answer Extraction

Every rollout, in training and evaluation, is generated from a single\-turn chat prompt with no system message: the user message is the problem followed by the standard suffix used by TTRL, wrapped with the tokenizer’s chat template \(generation prompt appended; thinking disabled for hybrid\-thinking models\)\. The student computes its distributions under this same plain prompt\.

Rollout and evaluation prompt \(user message\)``` {problem} Please reason step by step, and put your final answer within \boxed{}. ```

The teacher scores the identical rollout tokens under an augmented context: the consensus solution is inserted into a single user message that presents it as a reference solution, followed by the same assistant header, after which the rollout tokens are appended\. The full teacher\-side user message is the following \(long lines wrapped for display; the reference solution sits inside the context, before the assistant turn, and the student never sees it\):

Teacher conditioning prompt \(user message\)``` {problem} Please reason step by step, and put your final answer within \boxed{}. A correct solution to this problem is given below for your reference: <solution> {consensus solution} </solution> Guided by the reference solution, write your own step-by-step solution, and put your final answer within \boxed{}. ```

Final answers are read from the last`\\boxed\{\.\.\.\}`occurrence in a response, with brace\-depth matching for nested braces\. For multiple choice, a cascade applies: a boxed letter \(allowing`\\text`and`\\mathrm`wrappers\), then an “answer: X” pattern, then a trailing bare letter, searched within the final 400 characters\. Two answers are considered equivalent if they match as strings or if math\-verify’s symbolic check passes in either direction; responses without an extractable answer never match, and count in the vote\-share denominator\.

## Appendix DThe Pooled Training Set

The pooled experiments of Section[4\.4](https://arxiv.org/html/2607.13643#S4.SS4)are meant to test whether label\-free training on a broader unlabeled corpus improves a disjoint benchmark, so the pool must be large enough to matter and verifiably disjoint from the evaluation set\. The pool contains 339 mathematics prompts: the 83 AMC 2023 prompts taken whole, a 128\-prompt subset of MATH500 drawn uniformly at random \(seed 1234, the project\-wide seed\), and 128 OmniMath problems drawn with the same seeded sampler from the difficulty 2\.0 to 3\.5 band used for development slices\. All prompts enter training unlabeled; gold answers for pool prompts are never read by any label\-free method, and the label\-full arms \(GRPO\-pool, oracle\-pool\) use them only as rewards or for teacher\-rollout verification, exactly as in the single\-source experiments\.

The held\-out evaluation set, AIME 2024, shares no prompts with the pool; a 13\-gram overlap audit of all 339 pool prompts against every held\-out evaluation set finds zero matches, with the AMC\-transductive component as a positive control \(Appendix[I](https://arxiv.org/html/2607.13643#A9)\)\. One overlap inside the training side is disclosed for completeness: 43 of the pool’s 128 OmniMath prompts also appear in the 128\-prompt OmniMath development slice used for learning\-rate selection\. Both are training\-side sets, so no evaluation benchmark is affected and the protocol claim of Section[4\.1](https://arxiv.org/html/2607.13643#S4.SS1)is unchanged\.

## Appendix ESignificance Tests

The benchmark cells in this paper contain between 30 and 198 prompts, so point differences of a few points can be noise\. We quantify uncertainty on the central comparisons with a paired bootstrap over prompts \(B=10,000B\{=\}10\{,\}000\), reported in Table[10](https://arxiv.org/html/2607.13643#A5.T10)as avg@32 differences in points with 95% confidence intervals\.

Table 10:Paired bootstrap comparisons \(Qwen3\-4B\-Instruct\-2507;B=10,000B\{=\}10\{,\}000; avg@32 differences in points\)\. All comparisons use the lockedCanonartifact; transductive oracle comparisons use the deep\-resampled oracle arms\.Three conclusions follow\. The advantage ofCanonover the base model and over every label\-free and matched\-compute gold baseline is significant, with intervals clear of zero\. Against the oracle the picture splits by domain: on the mathematics benchmarks the difference is not distinguishable from zero in any setting tested, while on GPQA the deep\-resampled oracle keeps a significant transductive advantage and the held\-out interval sits exactly at the boundary\. Finally, GRPO\-180’s remaining transductive advantage, at sixteen times the compute and with gold labels throughout, is itself not significant\.

## Appendix FDesign Choices and Variance

The recipe of Section[3\.2](https://arxiv.org/html/2607.13643#S3.SS2)fixes four design choices: a frozen snapshot teacher, rollouts generated at the start of the epoch, a single consensus solution per prompt, and no vote\-share gating\. This appendix justifies each choice and reports seed variance; every experiment varies one choice at a time under an otherwise identical configuration\.

#### Freezing the teacher\.

Freezing is the one indispensable choice\. Table[12](https://arxiv.org/html/2607.13643#A6.T12)compares the snapshot teacher against a live teacher that shares parameters with the student, on a 390\-prompt OmniMath slice: the live\-teacher run collapses entirely, and Table[12](https://arxiv.org/html/2607.13643#A6.T12)shows the collapse signature from the training diagnostics, with loss and gradient norm reaching exactly zero within five steps while every sampled response saturates the length cap with no extractable answer\. Consensus anchoring is only stable against a fixed reference\. Rollout freshness matters for the same on\-policy reason, though far less dramatically: reusing stale rollouts from an earlier model version loses one to five points against regenerating at epoch start\.

Table 11:Teacher and rollout freshness\(390\-prompt OmniMath slice, avg@32; one change at a time\)\.
Table 12:Collapse signatureof the live\-teacher run \(training diagnostics at step 5\) against the snapshot run\.

#### Consensus width and gating\.

The remaining two choices are matters of simplicity rather than necessity \(values in Table[7](https://arxiv.org/html/2607.13643#S6.T7)\)\. Using three consensus solutions instead of one, implemented as a uniform log\-mixture of three consensus\-anchored teacher distributions rather than concatenated text, costs about a point on AMC \(76\.5 to 75\.6\), so we keep one\. Gating training on prompts with vote share of at least0\.50\.5costs 4\.4 points \(76\.5 to 72\.1\), consistent with the vote\-share\-band analysis of Section[5\.2](https://arxiv.org/html/2607.13643#S5.SS2): low\-confidence consensus still carries usable signal, so discarding it wastes training data\.

#### Rollout count\.

NNcontrols both the consensus estimate and the amount of distillation substrate\. Table[13](https://arxiv.org/html/2607.13643#A6.T13)variesNNwith everything else locked: gains grow monotonically toN=32N\{=\}32and plateau at 64, and evenN=4N\{=\}4recovers roughly two thirds of the full gain, so the method does not depend on a large sampling budget\.

#### Additional rounds\.

The recipe trains one epoch\. Iterating the full procedure, i\.e\., drawing fresh rollouts and re\-anchoring the teacher, buys little: on AMC a second round reaches 77\.4 \(\+0\.9\+0\.9, at twice the cost\) and a third gives the gain back \(76\.4\), and on the pooled configuration a second round declines on held\-out AIME 2024 \(38\.8 against the single\-round 41\.7\)\. One round captures the achievable gain\.

Table 13:Rollout\-count ablation\(transductive AMC; locked recipe otherwise\)\.
#### Mechanism arms\.

Two arms vary one component of the supervision at a time \(Table[7](https://arxiv.org/html/2607.13643#S6.T7)\)\. Restricting supervision to majority\-answer rollouts \(masking the minority\) costs 3\.5 points \(76\.5 to 73\.0\), so rollouts that disagree with the consensus carry usable signal, consistent with the gate ablation\. Replacing distillation with sequence\-level SFT toward the same consensus solution, selected by the same highest\-confidence rule, yields 67\.4 againstCanon’s 76\.5: the dense per\-token target, not the choice of solution, carries most of the improvement\. The divergence itself \(JSD against forward KL\) was selected on the OmniMath development slice, where JSD won; on the AMC test cell the divergence choice moves the result by under two points \(forward KL 78\.2, JSD 76\.5\), single cells at multi\-seed noise of 0\.3\.

#### Stability\.

Table[14](https://arxiv.org/html/2607.13643#A6.T14)reports three independentCanontraining runs on AMC with identical configuration; three identical reruns of the AIME 2024 transductive cell give 46\.0, 46\.2, and 42\.6 against the reported 44\.6 \(mean 44\.8, sd 1\.7\), the expected larger variance of a 30\-prompt set\. Three seeds of the Qwen3\.5\-9B AMC cell give mean 79\.2, sd 0\.7\. Evaluation\-seed variance is measured separately in Appendix[H](https://arxiv.org/html/2607.13643#A8)\.

Table 14:Training\-seed variance\(transductive AMC, three independent runs\)\.

## Appendix GAdditional Transfer Results

The main text establishes inductive transfer on Qwen3\-4B\-Instruct\-2507\. This appendix extends the evidence to Qwen3\.5\-4B and to two controlled notions of distance from the training distribution\.

Table 15:Additional inductive transfer on Qwen3\.5\-4B\(avg@32\)\. Left: AMC\-trained and pool\-trained checkpoints evaluated on disjoint benchmarks\. Right: one checkpoint trained label\-free on a 128\-prompt OmniMath slice, evaluated at increasing distance from the training distribution\.\(a\)Cross\-benchmark transfer\.
\(b\)Distance ladder\.

Table[15](https://arxiv.org/html/2607.13643#A7.T15)a shows the same qualitative picture as Qwen3\-4B\-Instruct\-2507: label\-free training on AMC or on the pool improves every held\-out benchmark tested, by two to three and a half points\. Majority\-vote accuracy on the 30\-prompt AIME sets is noisy at this scale; it rises on AIME 2024 \(from 60\.0 to 63\.3 for the AMC\-trained checkpoint\) and falls on AIME 2025 \(from 43\.3 to 36\.7, a two\-prompt movement\), so we do not read consensus\-level conclusions from these cells\. Transfer also crosses difficulty within a single source: training on an easy OmniMath slice \(difficulty 2 to 3\.5\) and evaluating on a difficulty 4\.0 to 5\.5 subset improves avg@32 from 29\.6 to 31\.3 with maj@32 unchanged\.

Table[15](https://arxiv.org/html/2607.13643#A7.T15)b evaluates one checkpoint at increasing distance\. Gains decay with distance but remain non\-negative everywhere; majority\-vote accuracy on the far transfers is noisier, and on AMC it falls from 86\.8 to 79\.5 while avg@32 still improves\. The pattern supports the reading of Section[4\.4](https://arxiv.org/html/2607.13643#S4.SS4): label\-free self\-improvement is strongest near the training distribution and attenuates, rather than inverts, away from it\.

## Appendix HEvaluation Robustness

The benchmark cells in this paper are single evaluations under the frozen harness, so we quantify the sampling noise a reader needs to interpret point differences\.

Table[16](https://arxiv.org/html/2607.13643#A8.T16)re\-evaluates fixed checkpoints under two additional vLLM sampling seeds\. The standard deviation over three seeds is 0\.2 to 0\.8 points on the 83\-prompt AMC cells and 0\.7 to 1\.2 points on the 30\-prompt AIME cells; differences of a point or less between arms on a 30\-prompt set are therefore within evaluation noise, which is how we read them throughout\. Training\-seed variance forCanonis of the same order \(0\.3 points over three independent runs, Appendix[F](https://arxiv.org/html/2607.13643#A6)\)\.

Table 16:Evaluation\-seed variance\(avg@32 under three vLLM sampling seeds; fixed checkpoints, frozen harness otherwise\)\.Finally, re\-evaluating every AMC arm under a stricter 3,072\-token generation cap preserves the method ordering: base 57\.3, TTRL 61\.5,Canon69\.2, GRPO\-180 74\.9\.

## Appendix IContamination Audits and a Post\-Cutoff Evaluation

This appendix addresses the possibility that inductive transfer reflects overlap between the training pool and the held\-out benchmarks\.

At the prompt level, an exact\-string and normalized 13\-gram overlap audit between all 339 pool prompts and each held\-out evaluation set \(AIME 2024, AIME 2025, GPQA\) finds zero matches; the audit’s positive control, AMC prompts against the pool’s AMC component, recovers all 83 with near\-duplicate pairs\. A sentence\-embedding audit \(all\-MiniLM\-L6\-v2\[Wanget al\.,[2020](https://arxiv.org/html/2607.13643#bib.bib29)\]\) over all pool\-eval pairs finds no pair with cosine similarity of 0\.70 or above \(maximum 0\.687, median top\-1 similarity 0\.51\)\.

Finally, we constructed a post\-cutoff evaluation set from the 2026 AIME, which post\-dates the training data of every model in this paper\. The set contains 29 of the 30 problems: every answer was cross\-checked against at least two independent sources, and one problem was excluded because its published statement is genuinely disputed\. The set has zero 13\-gram overlap with the training pool\. Table[17](https://arxiv.org/html/2607.13643#A9.T17)evaluates the transfer checkpoints on it\. Atn=29n\{=\}29, single\-sample accuracy is within evaluation noise for every arm; majority\-vote accuracy improves under the AMC\-trained checkpoint \(\+6\.9\+6\.9points\) and the oracle\-pool checkpoint \(\+6\.9\+6\.9\)\. We report this cell primarily as a contamination control: a benchmark that cannot appear in any training corpus shows the same qualitative consensus\-level movement as the pre\-cutoff held\-out sets\.

Table 17:Post\-cutoff evaluation: AIME 2026\(n=29n\{=\}29; avg@32 / maj@32\)\. Checkpoints as in Section[4\.4](https://arxiv.org/html/2607.13643#S4.SS4)\.
## Appendix JPer\-Prompt Analysis Details

This appendix defines the per\-prompt quantities used in Section[5](https://arxiv.org/html/2607.13643#S5)and reports the statistics that did not fit in the main text\. For each prompt we compare base and trained accuracy over the 32 evaluation samples and classify the movement\. Coverage transitions count prompts that move between zero and at least one correct sample \(cov\+for gained coverage, cov\-for lost\); sharpening transitions count accuracy movement within already\-covered prompts \(sharp\+and sharp\-\); and majority flips compare the correctness of the per\-prompt majority answer before and after training\.

Table 18:Per\-prompt transitionsunder transductiveCanon\(Qwen3\-4B\-Instruct\-2507\)\. Counts of prompts per category; the full per\-cell table over all reported checkpoints is released with the evaluation artifacts\.Table[18](https://arxiv.org/html/2607.13643#A10.T18)shows the transition counts on the three transductive benchmarks\. On AMC the picture is clean: coverage only grows, sharpening improvements outnumber regressions roughly three to one, and majority flips run three to the right against one to the wrong side\. AIME 2024 shows the same asymmetry at its smaller scale\. On GPQA the aggregate is similar but individual prompts are noisier, with coverage gains and losses nearly balanced and majority flips at 13 right against 8 wrong; this matches the flat pass@32 and positive maj@32 of Table[4](https://arxiv.org/html/2607.13643#S5.T4)\.

#### A regression\-to\-the\-mean control for the difficulty analysis\.

The difficulty bands of Figure[4](https://arxiv.org/html/2607.13643#S5.F4)are assigned from the base model’s observed accuracy over 32 samples, and the same finite\-sample estimate ordinarily serves as the baseline when computing gains\. Prompts that enter the hardest bands partly through an unlucky draw would then show spurious gains under any re\-evaluation, and the never\-solved band can only move up\. To control for this, we separate the roles: bands are assigned from one base evaluation draw, the baseline is computed from an independent second base draw, and the post\-training accuracy comes from the trained checkpoint \(the two additional evaluation seeds of Appendix[H](https://arxiv.org/html/2607.13643#A8)\)\. Table[19](https://arxiv.org/html/2607.13643#A10.T19)reports both estimates\. The regression component is small in every band except one, where it works against the naive estimate, and the aggregate gain is slightly larger under the control \(\+11\.2\+11\.2against\+9\.7\+9\.7points\)\. On the never\-solved band, the independent draw also scores zero\. Band membership and magnitudes differ from Figure[4](https://arxiv.org/html/2607.13643#S5.F4)because the draws differ\.

Table 19:Regression\-to\-the\-mean control\(AMC, transductiveCanon, avg@32 in percent\)\. Bands assigned from base draw A;Δ\\Deltanaive uses draw A as the baseline,Δ\\Deltaclean uses the independent base draw B\.
#### Coverage at a larger sampling budget\.

The coverage claims of Section[5\.1](https://arxiv.org/html/2607.13643#S5.SS1)rest on 32 samples per prompt\. pass@kkanalyses of RL\-trained models report a crossover: the trained model wins at smallkkbut falls below the base model at largekk, indicating a narrowed reasoning boundary\[Yueet al\.,[2025](https://arxiv.org/html/2607.13643#bib.bib38)\]\. Table[20](https://arxiv.org/html/2607.13643#A10.T20)extends the budget with a fresh 64\-sample draw per prompt\.Canonstays above the base model at everykk, with no crossover throughk=64k\{=\}64, and its majority\-vote advantage over pass@1 roughly halves, quantifying how much of the voting ensemble is amortized into the weights\. A second round ofCanonsits below the first at everykk, consistent with the rounds analysis of Appendix[F](https://arxiv.org/html/2607.13643#A6)\. Budgets beyond 64 samples, where the crossover ofYueet al\.\[[2025](https://arxiv.org/html/2607.13643#bib.bib38)\]typically emerges, are left to future work\.

Table 20:pass@kkat a larger budget\(AMC, fresh 64\-sample draw per prompt, so values differ slightly from the fixed\-seed harness\)\. maj is majority vote over the 64 samples\.