从 Pass@K 与 Pass@1 之间的差距中学习

# Learning from the Gap Between Pass@K and Pass@1
Source: [https://arxiv.org/html/2609.35793](https://arxiv.org/html/2609.35793)
###### Abstract

Large language models \(LLMs\) are increasingly trained with reinforcement learning from verifiable rewards \(RLVR\), in which sampled responses receive feedback from an exact verifier\. The same verifier can support test\-time scaling, one deployment option in which a system samples several responses and keeps one that passes\. Other deployments may spend extra inference compute on hard queries through beam search, adaptive sampling, or tool\-using loops\. We study single\-sample decoding, where each query receives one model response and no search chooses among alternatives, to ask whether search\-exposed behavior has been absorbed into the model\. Post\-training data recipes use verified responses to narrow this gap\. They can differ in how they sample or weight those responses, but they do not generally condition that choice on whether the source policy already solves the problem under the single\-sample decoding\. A fixed training budget then mixes redundant responses from first\-decode successes with responses that recover first\-decode failures\. Because search runs on every problem and the former group is usually larger, state\-unaware selection spends many examples repeating behavior the deployed policy already has, leaving fewer examples for new single\-sample targets\. To address this data\-selection gap, we design GapFT specifically to select evidence by the source checkpoint’s single\-sample outcome: it fine\-tunes on the Pass@KK–Pass@1 gap, namely problems the policy fails on a single sample but solves withinKKsamples\. We match training examples, processed tokens, and optimizer steps, and leave the objective unchanged\. GapFT labels each problem by its evidence state, fills the matched budget with recovered failures, and uses an exact decomposition to separate corrections of recovered and missed failures from regressions on deployed successes\. On LogiQA 2\.0 and ReClor with Llama\-3\.1\-8B, we empirically show that GapFT raises Pass@1 by14\.414\.4and13\.913\.9points over the source model, exceeds budget\-matched uniform verified RFT at the shared learning rate, and matches fine\-tuning on the entire verified pool with one third of the data\. A single decode reaches the accuracy of verifier\-selected Pass@4 from the source model\. A randomized control attributes the gain to covering distinct failures\. The final analysis relates the available gain to the amount of transferable failure support\. A three\-seed Qwen2\.5\-7B replication preserves the positive direction over uniform RFT on both logic tasks\.

## 1Introduction

Large language models \(LLMs\) are increasingly used to answer questions directly, so reliability matters whenever an incorrect response propagates to a later decision or computation\. A single decoding path can be unreliable on such problems, which has motivated additional inference computation: self\-consistency aggregates independent paths, while test\-time scaling studies how extra samples and verification improve accuracy\([Wang et al\., 2023](https://arxiv.org/html/2609.35793#bib.bib4);[Snell et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib21)\)\. Search may instead branch through changing environments, as in tool\-using agents\([Yao et al\., 2023b](https://arxiv.org/html/2609.35793#bib.bib2)\), or keep the input fixed and explore alternative responses that an exact verifier can check\([Chen et al\., 2021](https://arxiv.org/html/2609.35793#bib.bib1);[Li et al\., 2022](https://arxiv.org/html/2609.35793#bib.bib3)\)\. We study the latter setting\. For a problemxx, every sampled response receives the samexx; the source model is queried once for the deployment answer andKKtimes for discovery, with no feedback between generations\.

![Refer to caption](https://arxiv.org/html/2609.35793v1/gapft_overview_editable.png)Figure 1:GapFT selects verified evidence for single\-sample decoding\. \(a\) A fixed source policy and exact verifier assign each problem toGKG\_\{K\},SKS\_\{K\}, orUKU\_\{K\}; \(b\) under a matched training budget, GapFT selects one verified response for each distinctSKS\_\{K\}problem, whereas uniform verified RFT mixesGKG\_\{K\}andSKS\_\{K\}evidence; \(c\) the held\-out Pass@1 change decomposes into corrections and regressions\.Search improves an individual query by allocating additional inference\-time compute, a setting commonly called inference\-time scaling\([Snell et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib21)\)\. At deployment, using search incurs this cost again for every future problem\. Post\-training can amortize the cost by running search on a discovery set and using its verified responses to update the policy\. The updated model can then produce answers with fewer inference\-time searches on later problems\. Offline methods such as STaR and ReST retain successful model\-generated traces\([Zelikman et al\., 2022](https://arxiv.org/html/2609.35793#bib.bib6);[Gulcehre et al\., 2023](https://arxiv.org/html/2609.35793#bib.bib7)\), while RLVR methods such as ReFT and GRPO use verifier feedback during optimization\([Trung et al\., 2024](https://arxiv.org/html/2609.35793#bib.bib9);[Shao et al\., 2024](https://arxiv.org/html/2609.35793#bib.bib37)\)\. Other work distills a selected Best\-of\-NNdistribution or studies the value of distinct correct paths\([Sessa et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib12);[Yuan et al\., 2023](https://arxiv.org/html/2609.35793#bib.bib19);[Amini et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib13)\)\. Together, these methods establish that search\-derived correct responses can improve a policy\. They leave open a more basic data question: when the training budget is fixed, which verified responses should occupy its limited slots?

Correctness alone does not answer this allocation question\. A verified response from a problem that the source model already solves in a single sample records behavior the deployed policy already has\. A verified response from a problem that the source model misses in a single sample but recovers by search supplies a target that direct answering does not yet produce\. Both responses pass the same verifier, yet they have different potential value for single\-sample deployment\. Existing recipes usually filter, rank, or weight responses by correctness, reward, preference, or diversity\([Zelikman et al\., 2022](https://arxiv.org/html/2609.35793#bib.bib6);[Gulcehre et al\., 2023](https://arxiv.org/html/2609.35793#bib.bib7);[Sessa et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib12)\); these criteria do not reveal whether a response is redundant under the source policy’s deployment rule\. When the first group is larger, state\-unaware allocation can spend the fixed budget repeating known behavior and leave fewer slots for distinct recoverable failures\. This is the data\-selection gap we study, and it matters because it determines how much of search’s benefit can be transferred to a model that must answer with a single sample\.

To address this data\-selection gap, we design GapFT to condition evidence selection on the source policy’s single\-sample outcome\. A fixed source policy and exact verifier assign each discovery problem toGKG\_\{K\}\(single\-sample success\),SKS\_\{K\}\(single\-sample failure recovered by search\), orUKU\_\{K\}\(failure missed by the search budget; Figure[1](https://arxiv.org/html/2609.35793#S1.F1)\)\. GapFT fills a matched post\-training budget with one selected verified response for each chosenSKS\_\{K\}problem, while keeping the likelihood objective unchanged\. We evaluate the resulting policy under the same single\-sample decoding procedure and decompose its Pass@1 change into corrections onSKS\_\{K\}andUKU\_\{K\}and regressions onGKG\_\{K\}\. This design isolates the value of allocating verified evidence by source state, rather than introducing a new verifier, objective, or inference procedure\.

We test one allocation hypothesis, identify its causal mechanism, and measure the single boundary imposed by transferable failure support\. The contributions are:

- •We make the source checkpoint’s single\-sample outcome an explicit variable in verified\-data allocation\. This exposes the redundancy betweenGKG\_\{K\}andSKS\_\{K\}evidence that fixed\-budget recipes otherwise mix\.
- •We introduce GapFT, a matched\-budget recipe that selects verified responses fromSKS\_\{K\}while keeping the objective fixed, together with an exact decomposition of correction and regression and controls for search reachability\.
- •Across two logic benchmarks and a second model family, we empirically show that state\-based allocation improves single\-sample accuracy over strong budget\-matched baselines and reaches the performance of the full verified pool with substantially fewer examples\. The mechanism controls attribute this gain to covering distinct failures\.

## 2Related Work

#### Search and verification at inference time\.

Pass@KKmade repeated sampling an explicit source of performance\([Chen et al\., 2021](https://arxiv.org/html/2609.35793#bib.bib1)\); self\-consistency, Tree of Thoughts, and AlphaCode aggregate, branch, or filter many samples\([Wang et al\., 2023](https://arxiv.org/html/2609.35793#bib.bib4);[Yao et al\., 2023a](https://arxiv.org/html/2609.35793#bib.bib5);[Li et al\., 2022](https://arxiv.org/html/2609.35793#bib.bib3)\)\. Outcome verifiers score solutions\([Cobbe et al\., 2021](https://arxiv.org/html/2609.35793#bib.bib29);[Hosseini et al\., 2024](https://arxiv.org/html/2609.35793#bib.bib8)\); process verifiers score steps\([Lightman et al\., 2024](https://arxiv.org/html/2609.35793#bib.bib30)\)\. Recent work allocates inference compute by difficulty and measures the return from verification\([Snell et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib21);[Setlur et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib22)\)\. These methods spend compute at every query\. GapFT runs the same search once, on discovery problems, and asks what its verified output is worth after search is removed from deployment\.

#### What verified post\-training trains on\.

STaR, ReST, and rejection\-sampling fine\-tuning train on responses a verifier accepts\([Zelikman et al\., 2022](https://arxiv.org/html/2609.35793#bib.bib6);[Gulcehre et al\., 2023](https://arxiv.org/html/2609.35793#bib.bib7);[Shao et al\., 2024](https://arxiv.org/html/2609.35793#bib.bib37)\)\. ReFT uses online PPO with answer\-based rewards rather than simply retaining accepted offline responses\([Trung et al\., 2024](https://arxiv.org/html/2609.35793#bib.bib9)\)\.[Yuan et al\. \(2023\)](https://arxiv.org/html/2609.35793#bib.bib19)observe that the gain tracks the number of distinct correct paths rather than the number of samples\. Online methods filter implicitly: DAPO drops prompts whose sampled group is entirely correct or incorrect\([Yu et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib38)\), and prompt curricula choose which problems receive updates\([Gao et al\., 2026](https://arxiv.org/html/2609.35793#bib.bib15)\)\. Distillation imitates a Best\-of\-NNdistribution or a specified selection rule\([Sessa et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib12);[Amini et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib13);[Chow et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib23)\), SCoRe shows that correction traces collected in advance can mismatch the deployed policy\([Kumar et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib17)\), and Learning From Failure trains on unsuccessful trajectories\([Wang et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib18)\)\. These methods establish the general strategy of transferring search\-derived evidence into the policy, but none holds training examples, processed tokens, and optimizer steps fixed while measuring the Pass@1 value of a verified response as a function of whether the deployed policy already produces it\. This is the quantity GapFT isolates \(Appendix Table[10](https://arxiv.org/html/2609.35793#A2.T10)\)\.

#### Amplification, interference, and regression\.

Whether post\-training amplifies behavior already present or acquires new behavior is debated: large\-KKcomparisons and Echo Chamber report amplification\([Yue et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib11);[Zhao et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib10)\), whereas synthetic experiments show unseen compositions\([Yuan et al\., 2026](https://arxiv.org/html/2609.35793#bib.bib14)\)\.[Barakat et al\. \(2026\)](https://arxiv.org/html/2609.35793#bib.bib20)show that Pass@kkobjectives can degrade Pass@1 through prompt interference, ReGenesis links weak transfer to task\-specific self\-generated reasoning\([Peng et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib16)\), and the alignment\-tax and forgetting literature quantifies what adaptation costs\([Lin et al\., 2024](https://arxiv.org/html/2609.35793#bib.bib24);[Kalajdzievski, 2024](https://arxiv.org/html/2609.35793#bib.bib25)\)\. Our decomposition places these quantities in one identity measured on held\-out items: amplification is correction onSKS\_\{K\}, acquisition beyond observed support is correction onUKU\_\{K\}, and the cost is regression onGKG\_\{K\}\.

## 3GapFT

Section[3\.1](https://arxiv.org/html/2609.35793#S3.SS1)fixes notation for the offline rejection\- sampling fine\-tuning \(RFT\) setting used by GapFT\([Zelikman et al\., 2022](https://arxiv.org/html/2609.35793#bib.bib6);[Gulcehre et al\., 2023](https://arxiv.org/html/2609.35793#bib.bib7)\)\. GapFT then adds two parts: an intervention that decides which verified evidence fills a fixed post\-training budget, and a decomposition of the resulting Pass@1 change for each audit item \(Figure[1](https://arxiv.org/html/2609.35793#S1.F1)\)\. The training objective is unchanged\.

### 3\.1Problem Setup

LetV⁡\(x,y\)∈\{0,1\}V\(x,y\)\\in\\\{0,1\\\}be an exact outcome verifier for problemxxand responseyy, and letdπ​\(x\)d\_\{\\pi\}\(x\)be the response of policyπ\\piunder the single\-sample decoding \(a greedy answer for logic and MATH\([Hendrycks et al\., 2021](https://arxiv.org/html/2609.35793#bib.bib28)\), the final submission of one bounded episode for code\)\. Under the fixed\-input search setting, repeated sampling draws𝒴Kπ​\(x\)=\(y1,…,yK\)\\mathcal\{Y\}^\{\\pi\}\_\{K\}\(x\)=\(y\_\{1\},\\ldots,y\_\{K\}\)as conditionally independent samples fromπ\(⋅∣x\)\\pi\(\\cdot\\mid x\): every generation receives the samexxand no feedback from the other generations\. We countxxas solved if any sample passes\([Chen et al\., 2021](https://arxiv.org/html/2609.35793#bib.bib1)\)\. For a problem distribution𝒟\\mathcal\{D\}, deployment accuracy and search success are

Adep\(π\)=𝔼x∼𝒟\[V\(x,dπ\(x\)\)\],Asearch\(π;K\)=𝔼x∼𝒟,y1:K∼π\(⋅∣x\)\[maxk≤KV\(x,yk\)\],A\_\{\\mathrm\{dep\}\}\(\\pi\)=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\}\[V\(x,d\_\{\\pi\}\(x\)\)\],\\qquad A\_\{\\mathrm\{search\}\}\(\\pi;K\)=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\},\\,y\_\{1:K\}\\sim\\pi\(\\cdot\\mid x\)\}\\Big\[\\max\_\{k\\leq K\}V\(x,y\_\{k\}\)\\Big\],\(1\)Pass@1 and Pass@KKin the usual terminology, and their difference is the Pass@KK–Pass@1 gap\.

The offline RFT procedure studied here samples responses from a frozen source policyπsrc\\pi\_\{\\mathrm\{src\}\}on a discovery set𝒟disc\\mathcal\{D\}\_\{\\mathrm\{disc\}\}, keeps the pairs accepted by the verifier as a training set𝒯\\mathcal\{T\}, and fine\-tunes with the maximum\-likelihood objective

ℒ\(θ\)=−∑\(x,y\)∈𝒯∑t=1\|y\|logπθ\(yt∣x,y<t\)\\mathcal\{L\}\(\\theta\)=\-\\sum\_\{\(x,y\)\\in\\mathcal\{T\}\}\\sum\_\{t=1\}^\{\|y\|\}\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid x,y\_\{<t\}\)\(2\)\([Zelikman et al\., 2022](https://arxiv.org/html/2609.35793#bib.bib6);[Gulcehre et al\., 2023](https://arxiv.org/html/2609.35793#bib.bib7);[Yuan et al\., 2023](https://arxiv.org/html/2609.35793#bib.bib19);[Shao et al\., 2024](https://arxiv.org/html/2609.35793#bib.bib37)\)\. Standard RFT selection admits a response on the basis of correctness alone and, when the training budget is smaller than the accepted pool, subsamples without regard to whetherπsrc\\pi\_\{\\mathrm\{src\}\}already solves the problem in one decode\. We writeπ𝒯\\pi\_\{\\mathcal\{T\}\}for the policy trained on𝒯\\mathcal\{T\}and evaluate it on an audit set𝒟audit\\mathcal\{D\}\_\{\\mathrm\{audit\}\}disjoint from𝒟disc\\mathcal\{D\}\_\{\\mathrm\{disc\}\}\.

### 3\.2Evidence States

Relative to the source policy and a search budgetK≥1K\\geq 1, we place every problem in exactly one of three*evidence states*,

GK\\displaystyle G\_\{K\}=\{x:V⁡\(x,dsrc​\(x\)\)=1\},\\displaystyle=\\\{x:V\(x,d\_\{\\mathrm\{src\}\}\(x\)\)=1\\\},\(3\)SK\\displaystyle S\_\{K\}=\{x:V\(x,dsrc\(x\)\)=0,∃y∈𝒴Kπsrc\(x\):V\(x,y\)=1\},\\displaystyle=\\\{x:V\(x,d\_\{\\mathrm\{src\}\}\(x\)\)=0,\\ \\exists y\\in\\mathcal\{Y\}^\{\\pi\_\{\\mathrm\{src\}\}\}\_\{K\}\(x\):V\(x,y\)=1\\\},\(4\)UK\\displaystyle U\_\{K\}=\{x:V\(x,dsrc\(x\)\)=0,∀y∈𝒴Kπsrc\(x\):V\(x,y\)=0\},\\displaystyle=\\\{x:V\(x,d\_\{\\mathrm\{src\}\}\(x\)\)=0,\\ \\forall y\\in\\mathcal\{Y\}^\{\\pi\_\{\\mathrm\{src\}\}\}\_\{K\}\(x\):V\(x,y\)=0\\\},\(5\)the*deployed successes*, the*failures recovered by search*, and the*failures missed by search*\. The partition records observations under a finite search budget, not capability labels, soUKU\_\{K\}does not mean the source assigns zero probability to a correct response\. A verified response on a deployed success is redundant for deployment; one on a failure recovered by search is a target the deployed policy does not yet produce\. GapFT rests on the hypothesis that this distinction, not correctness alone, determines training value\.

### 3\.3Evidence Selection under a Matched Budget

Search runs only on𝒟disc\\mathcal\{D\}\_\{\\mathrm\{disc\}\}and yields two evidence sets,

ℰG=\{\(x,dsrc​\(x\)\):x∈𝒟disc∩GK\},ℰS,K=\{\(x,yK∗​\(x\)\):x∈𝒟disc∩SK\},\\mathcal\{E\}\_\{G\}=\\\{\(x,d\_\{\\mathrm\{src\}\}\(x\)\):x\\in\\mathcal\{D\}\_\{\\mathrm\{disc\}\}\\cap G\_\{K\}\\\},\\qquad\\mathcal\{E\}\_\{S,K\}=\\\{\(x,y\_\{K\}^\{\*\}\(x\)\):x\\in\\mathcal\{D\}\_\{\\mathrm\{disc\}\}\\cap S\_\{K\}\\\},\(6\)whereyK∗​\(x\)y\_\{K\}^\{\*\}\(x\)is one verified response chosen by a rule frozen before training \(the shortest verified response for logic and code; a frozen generation order for MATH\)\. A problem entersSKS\_\{K\}when its source decision is wrong and at least one of theKKsampled responses passes verification\. Even when several sampled responses pass, the recipe retains only\(x,yK∗​\(x\)\)\(x,y\_\{K\}^\{\*\}\(x\)\)for that problem\.

Letn=\|𝒯\|n=\|\\mathcal\{T\}\|denote the number of example slots before repetition, and letRRbe the number of times that fixed recipe is repeated during training\. The training budgetℬ=\(N,L,J\)\\mathcal\{B\}=\(N,L,J\)comprisesN=R​nN=Rntotal example exposures,LLprocessed tokens across those exposures, andJJoptimizer steps\. Processed tokens include the input and response after truncation, excluding padding; the loss is computed only on response tokens\. Replay may assign multiple slots to the same problem\. These quantities describe post\-training; the search budgetKKcounts candidate generations per problem and is reported separately\. In the main comparison, GapFT setsn=\|ℰS,K\|n=\|\\mathcal\{E\}\_\{S,K\}\|and uses𝒯gap=ℰS,K\\mathcal\{T\}\_\{\\mathrm\{gap\}\}=\\mathcal\{E\}\_\{S,K\}: each recovered problem contributes one verified response before repetition\. Two baselines draw the samennexamples without replacement from the same pool\.*Uniform verified RFT*draws fromℰG∪ℰS,K\\mathcal\{E\}\_\{G\}\\cup\\mathcal\{E\}\_\{S,K\}regardless of state, which is what the pipelines of Section[3\.1](https://arxiv.org/html/2609.35793#S3.SS1)do whenever the budget is smaller than the pool;*solved replay*, the no\-search conditionK=0K=0, draws fromℰG\\mathcal\{E\}\_\{G\}only\. Across conditions we enforce

N=R​\|𝒯\|,R​Tok⁡\(𝒯\)≃L,Step⁡\(𝒯;R\)=J,N=R\|\\mathcal\{T\}\|,\\qquad R\\operatorname\{Tok\}\(\\mathcal\{T\}\)\\simeq L,\\qquad\\operatorname\{Step\}\(\\mathcal\{T\};R\)=J,\(7\)whereStep⁡\(𝒯,R\)\\operatorname\{Step\}\(\\mathcal\{T\};R\)counts updates overRRrepetitions\. We report the permitted token discrepancy for each experiment\.*RFT on all verified items*usesn=\|ℰG\|\+\|ℰS,K\|n=\|\\mathcal\{E\}\_\{G\}\|\+\|\\mathcal\{E\}\_\{S,K\}\|with the sameRR, providing a reference with a larger training budget\. The mechanism controls retain a shared evidence core and replace replayed examples with distinct failures\. The replacement controls match\(N,L,J\)\(N,L,J\)within each comparison; the gold\-label control matches exposures, response tokens, and steps\. Their absolute budgets and evidence compositions are specified in Section[5](https://arxiv.org/html/2609.35793#S5)and Appendix Table[6](https://arxiv.org/html/2609.35793#A1.T6)\.

All conditions minimize Equation[2](https://arxiv.org/html/2609.35793#S3.E2)on their evidence set\. The effect of GapFT against a baseline evidence set𝒯b\\mathcal\{T\}\_\{b\}at the same budget isΔb=Adep​\(π𝒯gap\)−Adep​\(π𝒯b\)\\Delta\_\{b\}=A\_\{\\mathrm\{dep\}\}\(\\pi\_\{\\mathcal\{T\}\_\{\\mathrm\{gap\}\}\}\)\-A\_\{\\mathrm\{dep\}\}\(\\pi\_\{\\mathcal\{T\}\_\{b\}\}\), the deployment value of spending the budget on failures recovered by search rather than on the evidence the baseline selects\. Audit responses never enter evidence selection, training, or checkpoint selection; for nested budgets the firstKKsamples are reused as a prefix and the audit partition is rebuilt at eachKK\. Intervals resample paired audit problems and coupled training seeds with 2,000 bootstrap replicates unless noted\.

### 3\.4Decomposition of the Pass@1 Change

We introduce an identity that attributes any Pass@1 change to the three states\. On the audit set, partition problems intoGKG\_\{K\},SKS\_\{K\}, andUKU\_\{K\}using the source greedy decision andKKsource samples, and letpB=Prx∼𝒟audit\[x∈BK\]p\_\{B\}=\\Pr\_\{x\\sim\\mathcal\{D\}\_\{\\mathrm\{audit\}\}\}\[x\\in B\_\{K\}\]\. For a post\-training policyπ\\pidefine

κB=Pr⁡\[V⁡\(x,dπ​\(x\)\)=1∣x∈BK\]​for​B∈\{S,U\},ρG=Pr⁡\[V⁡\(x,dπ​\(x\)\)=0∣x∈GK\],\\kappa\_\{B\}=\\Pr\[V\(x,d\_\{\\pi\}\(x\)\)=1\\mid x\\in B\_\{K\}\]\\ \\text\{for \}B\\in\\\{S,U\\\},\\qquad\\rho\_\{G\}=\\Pr\[V\(x,d\_\{\\pi\}\(x\)\)=0\\mid x\\in G\_\{K\}\],\(8\)the correction rates on the two failure states and the regression rate on deployed successes\. Here,κS\\kappa\_\{S\}andκU\\kappa\_\{U\}are the fractions of source single\-sample errors that become correct under single\-sample evaluation after post\-training, conditioned onSKS\_\{K\}andUKU\_\{K\}, respectively\. The rateρG\\rho\_\{G\}is the fraction of source single\-sample successes that become incorrect after post\-training\. Because audit items are disjoint from training,κS\\kappa\_\{S\}measures generalization rather than reproduction\. Since the three states partition the audit set,

Adep​\(π\)−Adep​\(πsrc\)=pS​κS\+pU​κU−pG​ρGA\_\{\\mathrm\{dep\}\}\(\\pi\)\-A\_\{\\mathrm\{dep\}\}\(\\pi\_\{\\mathrm\{src\}\}\)=p\_\{S\}\\kappa\_\{S\}\+p\_\{U\}\\kappa\_\{U\}\-p\_\{G\}\\rho\_\{G\}\(9\)holds exactly, and any paired contrastΔb\\Delta\_\{b\}decomposes into the differences of the three terms\. The weights bound each term; in particularpS​κS≤pSp\_\{S\}\\kappa\_\{S\}\\leq p\_\{S\}, so no post\-training can gain more from correction within observed support than the share of failures recovered by search in the audit set\. The identity explains a measured contrast but does not assign a cause; causality comes from the matched\-budget intervention and the randomized control of Section[5](https://arxiv.org/html/2609.35793#S5)\(Appendix Figure[4](https://arxiv.org/html/2609.35793#A2.F4)\)\.

Remark\.Two closely related studies also examine the gap between multi\-sample success and single\-sample reliability on verifiable tasks\. Barakat et al\.\([Barakat et al\., 2026](https://arxiv.org/html/2609.35793#bib.bib20)\)study an objective\-level mismatch: optimizing Pass@KKreweights prompts and can create gradient conflict that lowers Pass@1, with theory and mathematical experiments\. Yue et al\.\([Yue et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib11)\)study a capability\-level question: by comparing RL\-trained and base models over largeKK, they find that RL often increases the probability of already\-supported correct paths without expanding the reasoning boundary\. Both motivate measuring what reaches single\-sample decoding, while GapFT studies a different intervention\. We hold the objective and training budget fixed, select evidence from a fixed source policy by its single\-sample outcome, and measure the resulting Pass@1 change with theGK/SK/UKG\_\{K\}/S\_\{K\}/U\_\{K\}decomposition and matched controls\. Their settings change the objective or compare RL and base\-model distributions overKK; ours starts from one frozen source policy and a fixed verified pool, and changes only which evidence enters offline RFT\. The contribution is therefore an evidence\-allocation recipe and audit for the data side of the gap, complementary to their objective and capability analyses\.

## 4Experiments

### 4\.1Setup

The main experiments use LogiQA 2\.0\([Liu et al\., 2023](https://arxiv.org/html/2609.35793#bib.bib26)\)and ReClor\([Yu et al\., 2020](https://arxiv.org/html/2609.35793#bib.bib27)\)with exact choice verification and greedy Pass@1 deployment, and Llama\-3\.1\-8B\-Instruct\([Grattafiori et al\., 2024](https://arxiv.org/html/2609.35793#bib.bib31)\)as the source policy, which decodes once and samplesK=8K=8times on the discovery split\. Each selected problem contributes one verified response\. We repeat these fixed input–response pairsR=8R=8times and train for one LoRA RFT epoch\([Hu et al\., 2022](https://arxiv.org/html/2609.35793#bib.bib36)\)over the resulting dataset using three coupled seeds\. We evaluate on a disjoint split with paired bootstrap intervals \(Appendix[A](https://arxiv.org/html/2609.35793#A1)\)\. The constrained protocol of Section[4\.2](https://arxiv.org/html/2609.35793#S4.SS2)setsn=\|ℰS,8\|n=\|\\mathcal\{E\}\_\{S,8\}\|, so that GapFT, uniform RFT, and solved replay draw the same number of problems from the same pool with identical optimizer steps and a processed\-token mismatch below0\.01%0\.01\\%; failures recovered by search make up34%34\\%and30%30\\%of the two pools, so uniform RFT contains that share of them by chance\. All conditions use the same learning rate5×10−55\{\\times\}10^\{\-5\}, with no per\-condition tuning\. Herennis 3,249 on LogiQA and 916 on ReClor, givingN=25,992/7,328N=25\{,\}992/7\{,\}328total training exposures andJ=1,625/458J=1\{,\}625/458updates\. RFT on all verified items uses the full pool with the same repeat count, providing a reference with a larger budget\. The controls in Section[5](https://arxiv.org/html/2609.35793#S5)preserve a shared evidence core and replace repeated examples with new failures\. Appendix Table[6](https://arxiv.org/html/2609.35793#A1.T6)lists their separate budgets and distinguishes processed\-token matching from response\-token matching\. The final analysis examines one boundary of the recipe: the amount of transferable failure support exposed by search\.

The experiments follow the study’s single question and its causal explanation\. The main comparison tests state\-based allocation at a fixed budget; the mechanism controls identify why the allocation changes Pass@1; and the final analysis measures when the exposed failure support leaves room for a gain\.

### 4\.2Main Results

Table 1:Pass@1 Under Matched Evidence Budgets#### Fixed\-budget allocation improves single\-sample accuracy\.

State\-targeted training reaches the accuracy of the full verified pool with one third as many problems \(Table[1](https://arxiv.org/html/2609.35793#S4.T1)\)\. At the same constrained budget, solved replay is lowest, uniform RFT is already strong, and GapFT adds3\.903\.90points on LogiQA and2\.332\.33points on ReClor over uniform RFT\. The full verified pool does not measurably improve over GapFT once the recovered failures are present\. The main result is therefore a data\-efficiency finding: state\-based selection captures the benefit of the full verified pool with far fewer training examples, while the gain over a strong state\-blind baseline is consistent across both tasks\.

#### The Gain Is Net of Higher Regression\.

The decomposition identifies the source of this ordering\. Relative to uniform RFT, GapFT increases the weighted corrections onSKS\_\{K\}andUKU\_\{K\}more than it increases regression onGKG\_\{K\}, so its net Pass@1 is higher\. Adding the deployed\-success examples back lowers regression but also lowers both correction terms, leaving the full pool and constrained GapFT at the same accuracy\. The extra evidence buys retention rather than additional correction\.

Figure 2:Source Inference Curves and One\-Decode Results
#### One Decode Replaces Four Verifier\-Selected Samples\.

To express the gain in inference terms, we compare each fine\-tuned Pass@1 with the source model’s Pass@KKand majority\-vote curves at the shared learning rate \(Figure[2](https://arxiv.org/html/2609.35793#S4.F2); Appendix Table[11](https://arxiv.org/html/2609.35793#A2.T11)\)\. GapFT reaches the source’s Pass@4 accuracy on both tasks; uniform RFT reaches Pass@3 and solved replay reaches Pass@2\. Majority voting over up to 16 samples remains below all fine\-tuned conditions\. Thus the allocation ordering also appears as the amount of verifier\-selected inference that a single sample can replace, although Pass@KKremains an oracle reference that requires search at inference\.

#### The allocation effect persists in a second model\.

The Qwen2\.5\-7B\([Qwen Team et al\., 2024](https://arxiv.org/html/2609.35793#bib.bib32)\)logic replication preserves the direction of the main result, with smaller margins that reflect its smaller exposed failure support \(Table[2](https://arxiv.org/html/2609.35793#S4.T2)\)\. It checks the same fixed\-budget question without introducing a second research object\.

The MATH row uses the matched\-exposure audit protocol, whereas the two logic rows use the fixed\-budget protocol\. The Qwen LogiQA split is ID\-disjoint; its source conversion contains 205 repeated prompt strings, with at most eight entering any condition’s training set\.

Table 2:Cross\-Model and Task ResultsValues are percentages; intervals are paired 95% CIs\.

## 5Mechanism Analysis

The mechanism controls test whether replacing repeated examples with distinct failures improves single\-sample accuracy\. They preserve a shared evidence core and use larger recipes than the main comparison\. Search and randomized inclusion match\(N,L,J\)\(N,L,J\); the gold\-label control matchesNN, response tokens, andJJ\(Appendix Table[6](https://arxiv.org/html/2609.35793#A1.T6)\)\. The cumulative search control keeps every example inℰG\\mathcal\{E\}\_\{G\}and fixesn=\|ℰG\|\+\|ℰS,8\|n=\|\\mathcal\{E\}\_\{G\}\|\+\|\\mathcal\{E\}\_\{S,8\}\|\. AtK=0K=0, the remaining slots replayℰG\\mathcal\{E\}\_\{G\}; atK=1K=1, recovered failures fill some slots and replay fills the rest; atK=8K=8, all slots contain distinct examples fromℰG∪ℰS,8\\mathcal\{E\}\_\{G\}\\cup\\mathcal\{E\}\_\{S,8\}\. Thus itsK=8K=8condition is the all\-verified reference in Table[1](https://arxiv.org/html/2609.35793#S4.T1), while the main GapFT recipe selects onlyℰS,8\\mathcal\{E\}\_\{S,8\}\. This control and the following interventions provide evidence about failure coverage under replacement\.

#### Design\.

The randomized control uses a separate discovery run and keeps all evidence verified within the first four samples\. It includes0/50/100%0/50/100\\%of problems first solved in samples five through eight and replaces each omitted item by replayed core evidence with matched input–response lengths, preservingNN,LL, andJJ; this changes how many verified problems are covered without changing the budget, so any difference is caused by the identities included\.

The gold\-label control compares two ways of selecting the failure portion of the cumulativeK=8K=8recipe\. Both retain the sameℰG\\mathcal\{E\}\_\{G\}core\. The search condition adds one verified response from each problem inS8S\_\{8\}\. The gold\-label condition samples the same number of distinct problems from all source\-decision failures,S8∪U8S\_\{8\}\\cup U\_\{8\}, without requiring a successful search response\. Sampling is uniform within each gold\-answer class, with class counts matched to the search condition\. Each selected problem receives one answer\-only target string \(e\.g\.,AorA\.\) from the search condition with the same correct option label, preserving the exact target\-string counts\. Both complete recipes repeat each input–response pair eight times, matchingNN, supervised response tokens, andJJ; processed input tokens differ with question length\. This comparison tests the restriction to search\-recoverable failures at a fixed number of distinct failure problems\. Its matched search reference is the cumulative control, which retainsℰG\\mathcal\{E\}\_\{G\}; the smaller main GapFT recipe contains onlyℰS,8\\mathcal\{E\}\_\{S,8\}\.

Two audit checks guard against artifacts of multiple\-choice tasks: post\-training answers are permuted within each stratum defined by the source answer and audit group, preserving answer marginals and change counts while breaking problem–answer pairing, and theSK/UKS\_\{K\}/U\_\{K\}split is recomputed from four independent source sample sets with all decisions fixed\.

Figure 3:Evidence Replacement at Fixed Training BudgetsTable 3:Pass@1 Decomposition and Inclusion ControlsDatasetContrastΔ\\DeltaPass@1 \(pp\)95% CIGGSSUUA\. Budget shift toward failures recovered by searchLogiQA 2\.0K=8−K=0K\{=\}8\-K\{=\}0\+9\.78\\mathbf\{\+9\.78\}\[8\.44, 11\.17\]\+0\.19\+0\.19\+6\.47\+6\.47\+3\.12\+3\.12ReClorK=8−K=0K\{=\}8\-K\{=\}0\+8\.67\\mathbf\{\+8\.67\}\[6\.27, 10\.93\]\+0\.13\+0\.13\+5\.07\+5\.07\+3\.47\+3\.47B\. Randomized inclusion of newly verified failuresLogiQA 2\.050%−0%50\\%\-0\\%\+1\.31\\mathbf\{\+1\.31\}\[0\.08, 2\.50\]−0\.30\-0\.30\+0\.74\+0\.74\+0\.87\+0\.87LogiQA 2\.0100%−0%100\\%\-0\\%\+2\.44\\mathbf\{\+2\.44\}\[1\.40, 3\.54\]−0\.15\-0\.15\+1\.29\+1\.29\+1\.29\+1\.29ReClor50%−0%50\\%\-0\\%\+1\.33\+1\.33\[−0\.27\-0\.27, 2\.93\]\+0\.13\+0\.13\+0\.60\+0\.60\+0\.60\+0\.60ReClor100%−0%100\\%\-0\\%\+2\.73\\mathbf\{\+2\.73\}\[1\.00, 4\.47\]\+0\.60\+0\.60\+1\.80\+1\.80\+0\.33\+0\.33
#### Coverage of Distinct Failures Causes Gain\.

Inclusion of distinct recovered failures increases Pass@1: full inclusion beats zero inclusion on both tasks, and half inclusion lies between the two \(Table[3](https://arxiv.org/html/2609.35793#S5.T3), panel B; Figure[3](https://arxiv.org/html/2609.35793#S5.F3), panel b\)\. The ReClor half\-inclusion interval includes zero, the only weaker comparison, but every seed favors full inclusion\. The gain flows throughS8S\_\{8\}on ReClor and through bothS8S\_\{8\}andU8U\_\{8\}on LogiQA, while theG8G\_\{8\}term stays near zero\. Replacing omitted failures with replayed core evidence therefore isolates the value of covering distinct failures; the matched\-diversity MATH control tests whether generic diversity alone explains the effect\.

#### The Useful Property Is Correcting a Deployed Failure\.

Gold\-label selection improves over solved replay by more than the cumulative search control on both tasks, and the search–gold contrast excludes zero \(Table[3](https://arxiv.org/html/2609.35793#S5.T3); Appendix Table[12](https://arxiv.org/html/2609.35793#A2.T12)\)\. Search therefore finds useful deployed failures, but it does not identify the best possible failures on these labeled tasks\. At the same failure count and target\-string counts, selecting from all source failures improves over restricting selection to failures recovered by search\.

#### Correction Concentrates on Specific Failures\.

Relative toK=0K=0, the gain flows throughS8S\_\{8\}andU8U\_\{8\}, while theG8G\_\{8\}contribution is unresolved \(Table[3](https://arxiv.org/html/2609.35793#S5.T3)\)\. Answer reshuffling substantially lowers the observedS8S\_\{8\}correction, showing that answer frequencies alone do not explain the effect\. Across four independent source sample sets, the combined correction remains fixed even though items move betweenSKS\_\{K\}andUKU\_\{K\}, so the aggregate effect is stable while the finerS/US/Uattribution has sampling uncertainty \(Appendix Tables[9](https://arxiv.org/html/2609.35793#A2.T9)and[14](https://arxiv.org/html/2609.35793#A2.T14)\)\. The controls support one conclusion: GapFT gains by covering distinct deployed failures, and search is the way those failures are found\.

## 6Failure Support and Applicability

GapFT can help only when search exposes failures that the model can learn and then produce under single\-sample decoding\. The decomposition bounds the contribution of observed support bypS​κSp\_\{S\}\\kappa\_\{S\}\. Llama\-3\.1\-8B and Qwen2\.5\-7B have similar conditional correction rates, but Qwen exposes an order\-of\-magnitude smallerS8S\_\{8\}share and therefore has less room for a gain \(Table[9](https://arxiv.org/html/2609.35793#A2.T9)\)\. The analysis below examines this support limit and separates it from the effect of repeating the same training problems\.

### 6\.1Limited Transferable Failure Support

On an independent 600\-problem MATH audit withK=8K=8source samples, Qwen2\.5\-7B provides a matched test of this limit\. Training on search\-recovered failures reaches66\.11%66\.11\\%, compared with64\.11%64\.11\\%for state\-blind matched uniform RFT, but the\+2\.00\+2\.00\-point contrast has a hierarchical interval that includes zero with three seeds\. The same\-budget comparison in Table[4](https://arxiv.org/html/2609.35793#S6.T4)also shows that 188 distinct problems with four exposures each reach65\.78%65\.78\\%, slightly above 94 distinct problems with eight exposures at65\.17%65\.17\\%\. These results support the design choice to prioritize distinctSKS\_\{K\}problems, while showing that the recipe cannot overcome a small transferable support or substantial regression onGKG\_\{K\}\.

Table 4:MATH Results with Matched Training Exposures
### 6\.2Coverage Is Preferable to Repetition

At the same number of training exposures, using 188 distinct problems reaches65\.78%65\.78\\%, compared with65\.17%65\.17\\%from 94 problems trained twice as often\. The−0\.61\-0\.61\-point contrast is small and its interval includes zero, but its direction supports allocating a fixed budget to distinct recoverable failures rather than increasing exposure to the same problems\. The table also shows that the search\-recovered condition is2\.002\.00points above state\-blind uniform RFT, while the corresponding interval remains seed\-limited\.

The adapter\-scaling checks in the appendix point to the same limitation\. Increasing update strength does not create additional support when the source search exposes few transferable failures, and can instead increase regression onGKG\_\{K\}\.

## 7Conclusion

A verified response is not equally useful training evidence under single\-sample deployment\. Existing recipes collect responses that pass a verifier, but do not determine whether those responses repeat behavior the source policy already produces or provide missing targets for direct answering, leaving a fixed\-budget data\-selection gap\. This gap matters because the allocation determines how many distinct failures can be converted into reliable single\-sample behavior\. GapFT resolves this gap by fixing the source policy, assigning each problem toGKG\_\{K\},SKS\_\{K\}, orUKU\_\{K\}according to its single\-sample and search outcomes, and filling a matched post\-training budget with one verified response from each selectedSKS\_\{K\}problem while keeping the training objective unchanged\. This allocation transfers search\-recovered failures into the single\-sample policy, and the held\-out decomposition makes its corrections and regressions explicit\. Across two logic benchmarks and a second model family, GapFT consistently improves single\-sample accuracy over state\-unaware allocation and reaches the performance of the full verified pool with far fewer examples\. The resulting gain is governed by the amount of transferableSKS\_\{K\}support exposed by search, which identifies when this recipe has room to help\.

### AI use statement

This study uses AI tools to write parts of the code, refine the idea, and adjust the LaTeX formatting\.

## References

- Aminiet al\.\(2025\)A\. Amini, T\. Vieira, E\. Ash, and R\. CotterellVariational best\-of\-N alignment\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/402542c2341e5d2eadc1dd0891275901-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.35793#S1.p2.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1)\.
- Barakatet al\.\(2026\)A\. Barakat, S\. Chakraborty, K\. Pahwa, and A\. S\. BediWhy pass@k optimization can degrade pass@1: prompt interference in LLM post\-training\.arXiv preprint arXiv:2602\.21189\.External Links:[Link](https://arxiv.org/abs/2602.21189)Cited by:[Table 10](https://arxiv.org/html/2609.35793#A2.T10.2.7.1.1.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px3.p1.1),[§3\.4](https://arxiv.org/html/2609.35793#S3.SS4.p2.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. d\. O\. Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman, A\. Ray, R\. Puri, G\. Krueger, M\. Petrov, H\. Khlaaf, G\. Sastry, P\. Mishkin, B\. Chan, S\. Gray, N\. Ryder, M\. Pavlov, A\. Power, L\. Kaiser, M\. Bavarian, C\. Winter, P\. Tillet, F\. P\. Such, D\. Cummings, M\. Plappert, F\. Chantzis, E\. Barnes, A\. Herbert\-Voss, W\. H\. Guss, A\. Nichol, A\. Paino, N\. Tezak, J\. Tang, I\. Babuschkin, S\. Balaji, S\. Jain, W\. Saunders, C\. Hesse, A\. N\. Carr, J\. Leike, J\. Achiam, V\. Misra, E\. Morikawa, A\. Radford, M\. Knight, M\. Brundage, M\. Murati, K\. Mayer, P\. Welinder, B\. McGrew, D\. Amodei, S\. McCandlish, I\. Sutskever, and W\. ZarembaEvaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.External Links:[Link](https://arxiv.org/abs/2107.03374)Cited by:[§1](https://arxiv.org/html/2609.35793#S1.p1.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2609.35793#S3.SS1.p1.1)\.
- Chowet al\.\(2025\)Y\. Chow, G\. Tennenholtz, I\. Gur, V\. Zhuang, B\. Dai, S\. Thiagarajan, C\. Boutilier, R\. Agarwal, A\. Kumar, and A\. FaustInference\-aware fine\-tuning for best\-of\-N sampling in large language models\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/c40bed606c51c8e827c1ba75aa2da054-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.External Links:[Link](https://arxiv.org/abs/2110.14168)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px1.p1.1.1)\.
- Gaoet al\.\(2026\)Z\. Gao, J\. Kim, W\. Sun, T\. Joachims, S\. Wang, R\. Y\. Pang, and L\. TanPrompt curriculum learning for efficient LLM post\-training\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2026/hash/601b42cebb3650ddb075770e2a85f428-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.External Links:[Link](https://arxiv.org/abs/2407.21783)Cited by:[§4\.1](https://arxiv.org/html/2609.35793#S4.SS1.p1.1)\.
- Gulcehreet al\.\(2023\)C\. Gulcehre, T\. L\. Paine, S\. Srinivasan, K\. Konyushkova, L\. Weerts, A\. Sharma, A\. Siddhant, A\. Ahern, M\. Wang, C\. Gu, W\. Macherey, A\. Doucet, O\. Firat, and N\. de FreitasReinforced self\-training \(ReST\) for language modeling\.arXiv preprint arXiv:2308\.08998\.External Links:[Link](https://arxiv.org/abs/2308.08998)Cited by:[§1](https://arxiv.org/html/2609.35793#S1.p2.1),[§1](https://arxiv.org/html/2609.35793#S1.p3.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2609.35793#S3.SS1.p2.2),[§3](https://arxiv.org/html/2609.35793#S3.p1.1)\.
- Guoet al\.\(2024\)D\. Guo, Q\. Zhu, D\. Yang, Z\. Xie, K\. Dong, W\. Zhang, G\. Chen, X\. Bi, Y\. Wu, Y\. K\. Li,et al\.DeepSeek\-Coder: when the large language model meets programming—the rise of code intelligence\.arXiv preprint arXiv:2401\.14196\.External Links:[Link](https://arxiv.org/abs/2401.14196)Cited by:[Table 8](https://arxiv.org/html/2609.35793#A1.T8.2.2.2.1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the MATH dataset\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=7Bywt2mQsCe)Cited by:[Table 7](https://arxiv.org/html/2609.35793#A1.T7.2.7.2.1.1),[§3\.1](https://arxiv.org/html/2609.35793#S3.SS1.p1.1)\.
- Hosseiniet al\.\(2024\)A\. Hosseini, X\. Yuan, N\. Malkin, A\. Courville, A\. Sordoni, and R\. AgarwalV\-STaR: training verifiers for self\-taught reasoners\.InConference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=stmqBSW2dV)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px1.p1.1.1)\.
- Huet al\.\(2022\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=nZeVKeeFYf9)Cited by:[Appendix A](https://arxiv.org/html/2609.35793#A1.p1.1),[§4\.1](https://arxiv.org/html/2609.35793#S4.SS1.p1.1)\.
- Huiet al\.\(2024\)B\. Hui, J\. Yang, Z\. Cui, J\. Yang, D\. Liu, L\. Zhang, T\. Liu, J\. Zhang, B\. Yu, K\. Lu,et al\.Qwen2\.5\-Coder technical report\.arXiv preprint arXiv:2409\.12186\.External Links:[Link](https://arxiv.org/abs/2409.12186)Cited by:[Table 8](https://arxiv.org/html/2609.35793#A1.T8.2.2.2.1.1)\.
- Jainet al\.\(2025\)N\. Jain, K\. Han, A\. Gu, W\. Li, F\. Yan, T\. Zhang, S\. Wang, A\. Solar\-Lezama, K\. Sen, and I\. StoicaLiveCodeBench: holistic and contamination free evaluation of large language models for code\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/94074dd5a072d28ff75a76dabed43767-Abstract-Conference.html)Cited by:[Table 8](https://arxiv.org/html/2609.35793#A1.T8)\.
- Kalajdzievski \(2024\)D\. KalajdzievskiScaling laws for forgetting when fine\-tuning large language models\.arXiv preprint arXiv:2401\.05605\.External Links:[Link](https://arxiv.org/abs/2401.05605)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px3.p1.1)\.
- Kumaret al\.\(2025\)A\. Kumar, V\. Zhuang, R\. Agarwal, Y\. Su, J\. D\. Co\-Reyes, A\. Singh, K\. Baumli, S\. Iqbal, C\. Bishop, R\. Roelofs, L\. M\. Zhang, K\. McKinney, D\. Shrivastava, C\. Paduraru, G\. Tucker, D\. Precup, F\. Behbahani, and A\. FaustTraining language models to self\-correct via reinforcement learning\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/871ac99fdc5282d0301934d23945ebaa-Abstract-Conference.html)Cited by:[Table 10](https://arxiv.org/html/2609.35793#A2.T10.2.6.1.1.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2022\)Y\. Li, D\. Choi, J\. Chung, N\. Kushman, J\. Schrittwieser, R\. Leblond, T\. Eccles, J\. Keeling, F\. Gimeno, A\. Dal Lago, T\. Hubert, P\. Choy, C\. de Masson d’Autume, I\. Babuschkin, X\. Chen, P\. Huang, J\. Welbl, S\. Gowal, A\. Cherepanov, J\. Molloy, D\. J\. Mankowitz, E\. Robson, P\. Kohli, N\. de Freitas, K\. Kavukcuoglu, and O\. VinyalsCompetition\-level code generation with AlphaCode\.Science378\(6624\),pp\. 1092–1097\.External Links:[Link](https://www.science.org/doi/10.1126/science.abq1158)Cited by:[§1](https://arxiv.org/html/2609.35793#S1.p1.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px1.p1.1)\.
- Lightmanet al\.\(2024\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/aca97732e30bcf1303bc22ac3924fd16-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px1.p1.1.2)\.
- Linet al\.\(2024\)Y\. Lin, H\. Lin, W\. Xiong, S\. Diao, J\. Liu, J\. Zhang, R\. Pan, H\. Wang, W\. Hu, H\. Zhang, H\. Dong, R\. Pi, H\. Zhao, N\. Jiang, H\. Ji, Y\. Yao, and T\. ZhangMitigating the alignment tax of RLHF\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 580–606\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.35),[Link](https://aclanthology.org/2024.emnlp-main.35/)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px3.p1.1)\.
- Liuet al\.\(2023\)H\. Liu, J\. Liu, L\. Cui, Z\. Teng, N\. Duan, M\. Zhou, and Y\. ZhangLogiQA 2\.0—an improved dataset for logical reasoning in natural language understanding\.IEEE/ACM Transactions on Audio, Speech, and Language Processing31,pp\. 2947–2962\.External Links:[Document](https://dx.doi.org/10.1109/TASLP.2023.3293046),[Link](https://doi.org/10.1109/TASLP.2023.3293046)Cited by:[§4\.1](https://arxiv.org/html/2609.35793#S4.SS1.p1.1)\.
- Penget al\.\(2025\)X\. Peng, C\. Xia, X\. Yang, C\. Xiong, C\. Wu, and C\. XingReGenesis: LLMs can grow into reasoning generalists via self\-improvement\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/9c77f2ce42151b2c2e26d2cf47f99564-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px3.p1.1)\.
- Qwen Teamet al\.\(2024\)Qwen Team, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang,et al\.Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.External Links:[Link](https://arxiv.org/abs/2412.15115)Cited by:[Table 5](https://arxiv.org/html/2609.35793#A1.T5.2.2.2.1.1),[§4\.2](https://arxiv.org/html/2609.35793#S4.SS2.SSS0.Px4.p1.1)\.
- Sessaet al\.\(2025\)P\. G\. Sessa, R\. Dadashi, L\. Hussenot, J\. Ferret, N\. Vieillard, A\. Ramé, B\. Shahriari, S\. Perrin, A\. Friesen, G\. Cideron, S\. Girgin, P\. Stanczyk, A\. Michi, D\. Sinopalnikov, S\. Ramos, A\. Héliou, A\. Severyn, M\. Hoffman, N\. Momchev, and O\. BachemBOND: aligning LLMs with best\-of\-N distillation\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/947f37882a394140f7add476bb99d1d3-Abstract-Conference.html)Cited by:[Table 10](https://arxiv.org/html/2609.35793#A2.T10.2.5.1.1.1),[§1](https://arxiv.org/html/2609.35793#S1.p2.1),[§1](https://arxiv.org/html/2609.35793#S1.p3.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1)\.
- Setluret al\.\(2025\)A\. Setlur, N\. Rajaraman, S\. Levine, and A\. KumarScaling test\-time compute without verification or RL is suboptimal\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 54058–54094\.External Links:[Link](https://proceedings.mlr.press/v267/setlur25a.html)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px1.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.External Links:[Link](https://arxiv.org/abs/2402.03300)Cited by:[§B\.4](https://arxiv.org/html/2609.35793#A2.SS4.p1.1),[Table 10](https://arxiv.org/html/2609.35793#A2.T10.2.2.1.1.1),[§1](https://arxiv.org/html/2609.35793#S1.p2.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2609.35793#S3.SS1.p2.2)\.
- Snellet al\.\(2025\)C\. Snell, J\. Lee, K\. Xu, and A\. KumarScaling LLM test\-time compute optimally can be more effective than scaling parameters for reasoning\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/1b623663fd9b874366f3ce019fdfdd44-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.35793#S1.p1.1),[§1](https://arxiv.org/html/2609.35793#S1.p2.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px1.p1.1)\.
- Trunget al\.\(2024\)L\. Trung, X\. Zhang, Z\. Jie, P\. Sun, X\. Jin, and H\. LiReFT: reasoning with reinforced fine\-tuning\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,pp\. 7601–7614\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.410),[Link](https://aclanthology.org/2024.acl-long.410/)Cited by:[§1](https://arxiv.org/html/2609.35793#S1.p2.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2025\)R\. Wang, X\. Han, Y\. Zhang, T\. Baldwin, and H\. LiNAT: enhancing agent tuning with negative samples\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 7385–7398\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.378),[Link](https://aclanthology.org/2025.naacl-long.378/)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2023\)X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. ZhouSelf\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=1PL1NIMMrw)Cited by:[§1](https://arxiv.org/html/2609.35793#S1.p1.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px1.p1.1)\.
- Yaoet al\.\(2023a\)S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. NarasimhanTree of thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/271db9922b8d1f4dd7aaef84ed5ac703-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px1.p1.1)\.
- Yaoet al\.\(2023b\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[§1](https://arxiv.org/html/2609.35793#S1.p1.1)\.
- Yuet al\.\(2025\)Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, T\. Fan, G\. Liu, L\. Liu, X\. Liu,et al\.DAPO: an open\-source LLM reinforcement learning system at scale\.InAdvances in Neural Information Processing Systems,Vol\.38\.External Links:[Document](https://dx.doi.org/10.52202/085713-3775),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/hash/a4277440d50f1f15d2cb4c14f7e0c0d2-Abstract-Conference.html)Cited by:[§B\.4](https://arxiv.org/html/2609.35793#A2.SS4.p1.1),[Table 10](https://arxiv.org/html/2609.35793#A2.T10.2.4.1.1.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1)\.
- Yuet al\.\(2020\)W\. Yu, Z\. Jiang, Y\. Dong, and J\. FengReClor: a reading comprehension dataset requiring logical reasoning\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=HJgJtT4tvB)Cited by:[§4\.1](https://arxiv.org/html/2609.35793#S4.SS1.p1.1)\.
- Yuanet al\.\(2026\)L\. Yuan, W\. Chen, Y\. Zhang, G\. Cui, H\. Wang, Z\. You, N\. Ding, Z\. Liu, M\. Sun, and H\. PengFromf⁡\(x\)f\(x\)andg⁡\(x\)g\(x\)tof⁡\(g⁡\(x\)\)f\(g\(x\)\): LLMs learn new skills in RL by composing old ones\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2026/hash/eeae43a68515325cad64c0f54b2d0c70-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px3.p1.1)\.
- Yuanet al\.\(2023\)Z\. Yuan, H\. Yuan, C\. Li, G\. Dong, K\. Lu, C\. Tan, C\. Zhou, and J\. ZhouScaling relationship on learning mathematical reasoning with large language models\.arXiv preprint arXiv:2308\.01825\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2308.01825),[Link](https://arxiv.org/abs/2308.01825)Cited by:[Table 10](https://arxiv.org/html/2609.35793#A2.T10.2.3.1.1.1),[§1](https://arxiv.org/html/2609.35793#S1.p2.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2609.35793#S3.SS1.p2.2)\.
- Yueet al\.\(2025\)Y\. Yue, Z\. Chen, R\. Lu, A\. Zhao, Z\. Wang, Y\. Yue, S\. Song, and G\. HuangDoes reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?\.InAdvances in Neural Information Processing Systems,Vol\.38\.External Links:[Document](https://dx.doi.org/10.52202/085713-1933),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/hash/537d5aa768c2d534016a4d06f87bc8fb-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px3.p1.1),[§3\.4](https://arxiv.org/html/2609.35793#S3.SS4.p2.1)\.
- Zelikmanet al\.\(2022\)E\. Zelikman, Y\. Wu, J\. Mu, and N\. D\. GoodmanSTaR: bootstrapping reasoning with reasoning\.InAdvances in Neural Information Processing Systems,Vol\.35\.External Links:[Document](https://dx.doi.org/10.52202/068431-1126),[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/639a9a172c044fbb64175b5fad42e9a5-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.35793#S1.p2.1),[§1](https://arxiv.org/html/2609.35793#S1.p3.1),[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2609.35793#S3.SS1.p2.2),[§3](https://arxiv.org/html/2609.35793#S3.p1.1)\.
- Zhaoet al\.\(2025\)R\. Zhao, A\. Meterez, S\. Kakade, C\. Pehlevan, S\. Jelassi, and E\. MalachEcho chamber: RL post\-training amplifies behaviors learned in pretraining\.InConference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=dp4KWuSDzj)Cited by:[§2](https://arxiv.org/html/2609.35793#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix AImplementation Details

Tables[5](https://arxiv.org/html/2609.35793#A1.T5)–[8](https://arxiv.org/html/2609.35793#A1.T8)list the settings needed to reproduce the logic, MATH, and code experiments\. In all three, cross\-entropy is computed only on the verified response tokens, and LoRA\([Hu et al\., 2022](https://arxiv.org/html/2609.35793#bib.bib36)\)is applied to the attention projectionsq\_proj,k\_proj,v\_proj, ando\_proj, and the MLP projectionsgate\_proj,up\_proj, anddown\_proj\. All final audits use deterministic decoding\.

#### Logic run families\.

The main comparison selectsn=\|ℰS,8\|n=\|\\mathcal\{E\}\_\{S,8\}\|distinct problems per condition, retains one response per problem, and repeats each fixed pair eight times\. ThusK=8K=8is the number of search candidates per problem, whereasR=8R=8is the number of training exposures per selected pair\. The cumulativeK=0/1/8K=0/1/8control retainsℰG\\mathcal\{E\}\_\{G\}and replaces repeated evidence within\|ℰG\|\+\|ℰS,8\|\|\\mathcal\{E\}\_\{G\}\|\+\|\\mathcal\{E\}\_\{S,8\}\|slots\. Its oracle control retains the sameℰG\\mathcal\{E\}\_\{G\}and replaces search\-selected failures with the same number of gold\-label failures, sampled without replacement within gold\-answer classes\. It assigns the search condition’s answer\-only target strings to selected problems with matching labels and repeats the complete recipe eight times, preserving target string counts and training exposures\. The randomized inclusion experiment uses another discovery run, fixes the evidence found within four samples, and varies inclusion of failures first recovered in samples five through eight\. Table[6](https://arxiv.org/html/2609.35793#A1.T6)separates recipe slots from total training exposures for these controls\. The scope matrix uses separately completed uniform RFT checkpoints and selects their adapter scales on disjoint calibration data\. Absolute post\-training accuracies are therefore compared only within, not across, these protocols\.

Table 5:Implementation Details for Logic ExperimentsTable 6:Training Budgets of the Llama Logic ExperimentsHerenncounts example slots before repetition,NNcounts total training exposures,LLcounts processed input and response tokens, andJJcounts optimizer steps\. The main, cumulative\-search, and inclusion rows give the processed\-token matching targets; replay replacement can introduce small token\-count discrepancies\. Gold\-label selection matches response tokens exactly but processes0\.33%0\.33\\%fewer total tokens on LogiQA and0\.48%0\.48\\%fewer on ReClor than the cumulativeK=8K=8control\. All rows useR=8R=8and learning rate5×10−55\\times 10^\{\-5\}\. The all\-verified reference in Table[1](https://arxiv.org/html/2609.35793#S4.T1)shares theK=8K=8checkpoint of the cumulative search control\.

Table 7:MATH Matched\-Diversity ControlTable 8:LiveCodeBench Allocation and Replication\([Jain et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib35)\)

## Appendix BExtended Experimental Results

### B\.1Full Logic Scope Matrix

Table[9](https://arxiv.org/html/2609.35793#A2.T9)reports both search budgets for the scope matrix with two tasks and two models\. Conditional rates with fewer than 30SKS\_\{K\}items are marked because their intervals are unstable\. The compactK=8K=8summary in the main text reports the main denominators; this table gives the complete accounting from the source to the post\-training policy\.

Table 9:Logic Scope Matrix and Answer\-Redistribution ControlFor example, at LogiQA/K=4K=4, Llama’sSSterm within observed support contributes9\.189\.18points and Qwen’s contributes only0\.910\.91points\. The difference primarily comes from the306306versus2828audit items inS4S\_\{4\}, rather than from a low Qwen conditional correction rate\. FromK=4K=4toK=8K=8, the number of discoverySSitems changes from2,3642\{,\}364to3,2493\{,\}249for LogiQA/Llama,655655to916916for ReClor/Llama,209209to279279for LogiQA/Qwen, and5050to6363for ReClor/Qwen\. These counts make newly available verified evidence, rather than nominalKKalone, the appropriate basis for reading the matrix\.

### B\.2Comparison with Closely Related Methods

Table[10](https://arxiv.org/html/2609.35793#A2.T10)contrasts GapFT with the methods most likely to be compared against it, along the dimensions of research question, assumption about verified evidence, mechanism, training regime, and the difference from GapFT\.

Table 10:Comparison with Closely Related Methods
### B\.3Evidence Replacement Controls

Panel A of Table[12](https://arxiv.org/html/2609.35793#A2.T12)reports the cumulative search and gold\-label controls\. They retainℰG\\mathcal\{E\}\_\{G\}, usen=\|ℰG\|\+\|ℰS,8\|n=\|\\mathcal\{E\}\_\{G\}\|\+\|\\mathcal\{E\}\_\{S,8\}\|slots, and replace replay with source\-failure evidence\. Search budgets match processed tokens; the gold\-label comparison matches response tokens as detailed in Table[6](https://arxiv.org/html/2609.35793#A1.T6)\. Panel B reports the separate randomized inclusion run, whose shared core includes all evidence verified within four samples\. Table[11](https://arxiv.org/html/2609.35793#A2.T11)lists the sample equivalents and headroom read from the curves of Figure[2](https://arxiv.org/html/2609.35793#S4.F2), Figure[5](https://arxiv.org/html/2609.35793#A2.F5)shows the code decomposition of Section[6](https://arxiv.org/html/2609.35793#S6), and Figure[4](https://arxiv.org/html/2609.35793#A2.F4)illustrates the decomposition flows\. For shared halfAA, new halfBB, and subset successaC​\(r\)a\_\{C\}\(r\)afterrrexposures, the code allocation contrast is

Δdiffuse−concentrated=12​\[aB​\(1\)−aB​\(0\)\]⏟coverage gain−12​\[aA​\(2\)−aA​\(1\)\]⏟repetition loss\.\\Delta\_\{\\mathrm\{diffuse\-concentrated\}\}=\\tfrac\{1\}\{2\}\\underbrace\{\[a\_\{B\}\(1\)\-a\_\{B\}\(0\)\]\}\_\{\\text\{coverage gain\}\}\-\\tfrac\{1\}\{2\}\\underbrace\{\[a\_\{A\}\(2\)\-a\_\{A\}\(1\)\]\}\_\{\\text\{repetition loss\}\}\.\(10\)
Table 11:Inference Samples Replaced by One Decode![Refer to caption](https://arxiv.org/html/2609.35793v1/fig1_audit_flow.png)Figure 4:Outcome\-Flow Decomposition by Source StateTable 12:Pass@1 Under the Full\-Pool ProtocolLogiQA 2\.0ReClorTraining conditionPass@1 \(%\)Δ\\Deltavs ref\. \(pp\)Pass@1 \(%\)Δ\\Deltavs ref\. \(pp\)A\. Fixed budget allocation\(reference: solved replay\)no post\-training \(source\)48\.98–55\.40–solved replay,K=0K\{=\}053\.310 \(ref\.\)59\.270 \(ref\.\)search evidence,K=1K\{=\}156\.53\+3\.22\\mathbf\{\+3\.22\}\[2\.08, 4\.47\]62\.40\+3\.13\\mathbf\{\+3\.13\}\[0\.47, 5\.60\]search evidence,K=8K\{=\}863\.08\+9\.78\\mathbf\{\+9\.78\}\[8\.44, 11\.17\]67\.93\+8\.67\\mathbf\{\+8\.67\}\[6\.27, 10\.93\]failures selected with gold labels \(oracle\)65\.25\+11\.94\\mathbf\{\+11\.94\}\[10\.01, 13\.99\]73\.20\+13\.93\\mathbf\{\+13\.93\}\[11\.33, 16\.67\]B\. Randomized inclusion of newly verified failures\(reference: 0%\)inclusion 0%62\.640 \(ref\.\)70\.730 \(ref\.\)inclusion 50%63\.95\+1\.31\\mathbf\{\+1\.31\}\[0\.08, 2\.50\]72\.07\+1\.33 \[−0\.27\-0\.27, 2\.93\]inclusion 100%65\.08\+2\.44\\mathbf\{\+2\.44\}\[1\.40, 3\.54\]73\.47\+2\.73\\mathbf\{\+2\.73\}\[1\.00, 4\.47\]

Figure 5:Code Coverage and Transfer Effects
### B\.4Online Optimization at Matched Update Steps

One control replaces the offline objective with online RL while holding the number of optimizer updates fixed\. On LogiQA 2\.0 with the same source model and audit set, GRPO\([Shao et al\., 2024](https://arxiv.org/html/2609.35793#bib.bib37)\)runs for the1,6251\{,\}625updates of the constrained protocol with group size88, LoRAr=8r=8, rule\-based reward on the option letter, and three seeds\. One condition uses every discovery prompt; the other masks groups whose rollouts all receive the same reward, the filtering rule of DAPO\([Yu et al\., 2025](https://arxiv.org/html/2609.35793#bib.bib38)\), after the rollouts are generated, so the two conditions differ in which groups reach the gradient and not in sampling cost\.

Two properties of this control limit what it can establish\. First, it fixes updates rather than generation: each seed produces about2\.82\.8M completion tokens against the6\.16\.1M processed tokens of a supervised condition, with a6464\-token cap, so the online conditions see less generation than the offline ones and complete about half an epoch\. Second, the reward uses a verifier that accepts an option letter followed by its text, whereas the audit protocol of Section[4\.1](https://arxiv.org/html/2609.35793#S4.SS1)accepts the letter alone\. The online conditions drift toward the longer format their reward permits, which the audit scores as invalid:19\.3%19\.3\\%and65\.2%65\.2\\%of their audit responses against0\.13%0\.13\\%for GapFT, and about half of the invalid responses open with the correct letter\. Table[13](https://arxiv.org/html/2609.35793#A2.T13)therefore scores every condition under both rules\.

Table 13:Online RL Control at Matched UpdatesBoth online conditions stay below GapFT under either rule, and the permissive rule places them near the source model rather than far below it, so the answer\-only gap is mostly format drift and not lost accuracy\. The masking rule also fails to reproduce evidence selection: deployed successes are29\.1%29\.1\\%of the groups it drops, far from the67\.9%67\.9\\%that dropping every deployed success and every failure search misses would give, because reward flatness is a property of the current policy at sampling temperature while the evidence state is a property of the source model at a single sample\. Neither observation bounds online RL in general: the control covers one algorithm at one generation budget, and evidence selection acts on which problems enter training rather than on the objective, so applying it to online prompt selection is untested\.

### B\.5Allocation When the Budget Covers the Pool

Atn=\|ℰG\|\+\|ℰS,8\|n=\|\\mathcal\{E\}\_\{G\}\|\+\|\\mathcal\{E\}\_\{S,8\}\|, a separate comparison trains on every verified problem once per repetition or samples uniformly with replacement from the same pool\. The latter includes duplicate problems, so the comparison tests coverage versus repetition at a larger budget\. The paired contrast, all verified minus uniform with replacement, is\+0\.91\+0\.91\[−0\.45\-0\.45,2\.212\.21\] on LogiQA and−2\.53\-2\.53\[−5\.13\-5\.13,0\.270\.27\] on ReClor\. The main comparison instead fixesn=\|ℰS,8\|n=\|\\mathcal\{E\}\_\{S,8\}\|and selects without replacement, testing whether source\-state selection improves accuracy when only part of the verified pool can be used\. The all\-verified condition remains a reference with a larger budget in Table[1](https://arxiv.org/html/2609.35793#S4.T1)\.

### B\.6Stability Across Independent Source Samples

TheSK/UKS\_\{K\}/U\_\{K\}distinction depends on a finite sample from the source model\. Table[14](https://arxiv.org/html/2609.35793#A2.T14)repeats that sample four times while holding greedy source and post\-training decisions fixed\.

Table 14:Stability of the Source\-State PartitionSource\-sample variation reallocates at most1\.381\.38LogiQA points and2\.802\.80ReClor points between theSSandUUterms\. Their combined correction on items the source gets wrong and the final accuracy are exactly invariant because the underlying decisions do not change\. Thus,κS\\kappa\_\{S\}andκU\\kappa\_\{U\}must be interpreted conditional on the sampled source partition, while the matched\-budget Pass@1 contrasts remain unchanged\.

## Appendix CAdditional Protocol Details

### C\.1Adapter Scaling with a Calibration Constraint

This rule is used for the scope matrix and its answer redistribution and source sample controls\. It is not used for the primary cumulativeK=0/1/8K=0/1/8comparison or the randomized inclusion experiment\.

For a completed LoRA updateΔ\\Delta, deployment at scaleα\\alphauses

θ⁡\(α\)=θ0\+α​Δ,α∈𝒜=\{0,\.25,\.5,\.75,1\}\.\\theta\(\\alpha\)=\\theta\_\{0\}\+\\alpha\\Delta,\\qquad\\alpha\\in\\mathcal\{A\}=\\\{0,\.25,\.5,\.75,1\\\}\.\(11\)No retraining occurs\. Each checkpoint selects its scale using a calibration split only\. For every nonzeroα\\alpha, letuαu\_\{\\alpha\}be a one\-sided Wilson upper confidence bound forρG​\(α\)\\rho\_\{G\}\(\\alpha\)with Bonferroni correction over the four nonzero comparisons\. A scale is*safe for every smaller scale*when

maxβ∈𝒜:0<β≤αuβ≤0\.10\.\\max\_\{\\beta\\in\\mathcal\{A\}:0<\\beta\\leq\\alpha\}u\_\{\\beta\}\\leq 0\.10\.\(12\)The rule selects the largest scale that is safe for every smaller scale using calibration data only and falls back toα=0\\alpha=0if no nonzero scale is safe\. Across 24 checkpoints from eight task–model–budget environments, 17 selectα=1\\alpha=1, six selectα=\.75\\alpha=\.75, and one selectsα=\.5\\alpha=\.5\. The equal\-environment mean improves over the source policy by8\.008\.00points with hierarchical 95% CI\[3\.35,12\.76\]\[3\.35,12\.76\], but differs from fixedα=\.75\\alpha=\.75by only\+0\.38\+0\.38points, CI\[−0\.17,0\.86\]\[\-0\.17,0\.86\]\. It reduces the number of checkpoints with aρG\>10%\\rho\_\{G\}\>10\\%point estimate from four to one, but the remaining case reaches10\.11%10\.11\\%on the final split\. Calibration scaling reduces observed regression but does not satisfy the universal risk criterion\.