Unlocking the Unsolvable: Teacher-Guided Curriculum for Data-Efficient RLVR

# Teacher-Guided Curriculum for Data-Efficient RLVR
Source: [https://arxiv.org/html/2609.13997](https://arxiv.org/html/2609.13997)
## Unlocking the Unsolvable: Teacher\-Guided Curriculum for Data\-Efficient RLVR

Zhen Han††thanks:This work was conducted independently of Amazon and does not relate to the author’s position there\.Affiliation:AmazonEmail:[zhenhz@amazon\.com](mailto:)

###### Abstract

Reinforcement Learning with Verifiable Rewards \(RLVR\) has shown remarkable success in improving the mathematical reasoning of large language models\. Yet problems beyond the model’s current capability, where rollouts uniformly fail and no learning signal is produced, are structurally wasted despite marking the most informative training frontier\. We show that these otherwise\-inert problems can be*unlocked*via teacher\-guided curriculum learning: partial reasoning traces from a stronger model create a graded difficulty landscape, and a backward\-chaining curriculum progressively withdraws guidance until the model solves problems unaided\. Training on only 128 unsolvable problems matches or exceeds GRPO trained on a full 2,000\-problem corpus \(∼\\sim16×\\timesdata efficiency\) on the nine\-benchmark average for both base models, while substantially expanding the reasoning boundary measured by pass@kkat largekk\. Furthermore, we identify a distribution\-shift cost that is particularly acute in the unsolvable\-only regime and proposeMonotone Frontier Curriculum \(MFC\), a method that monotonically drives training toward unguided solving, consistently outperforming existing curriculum methods\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2609.13997v1/overview.png)Figure 1:Overview of our unsolvable\-curriculum pipeline\.Top path: standard GRPO trained on the full2,0002\{,\}000\-problem mixed\-difficulty corpus\. Bottom path: we filter the same corpus to its147147unsolvable problems \(pass​@​64=0\\mathrm\{pass\}@64=0on the student\), generate a step\-structured trace with a stronger teacher, discard teacher failures \(136136remain\), and select the128128shortest examples as the curriculum\-training set\. We compare three backward\-chaining curricula: Mixture \(R3\), AdaBack, and ourMFC\. MFC is designed for the unsolvable\-only regime\. With∼\\sim16×\\timesless source data, the curriculum methods match or exceed the GRPO baseline on the cross\-benchmark average \(right; Qwen3\-1\.7B,99benchmarks\)\.Reasoning\-centric large language models \(LLMs\)[OpenAI \(2024\)](https://arxiv.org/html/2609.13997#bib.bib1);[Guo et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib2)have established Reinforcement Learning with Verifiable Rewards \(RLVR\)[Shao et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib3)as a powerful recipe for improving mathematical reasoning\. RLVR builds on the broader paradigm of reinforcement learning from human feedback[Ouyang et al\. \(2022\)](https://arxiv.org/html/2609.13997#bib.bib41)and policy gradient methods[Schulman et al\. \(2017\)](https://arxiv.org/html/2609.13997#bib.bib7), replacing costly human annotations with verifiable reward signals\. While most work focuses on algorithmic refinements[Yu et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib4);[Liu et al\. \(2025b\)](https://arxiv.org/html/2609.13997#bib.bib5);[Cui et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib6), a complementary question has received growing attention:*which training data matters most*?[Pikus et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib8)show that the hardest problems yield the largest gains, and[Foster et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib9)show that expected policy improvement is bounded by the variance of success\.[Wang et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib10)demonstrate that even a single example can unlock substantial reasoning gains\. These findings suggest a common theme: problems near the model’s reliable\-solving boundary, with mixed rollout outcomes, tend to provide the richest learning signal for RLVR\.

A separate line of inquiry concerns the*scope*of what RLVR can teach\.[Yue et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib14)find that standard RLVR raisespass​@​1\\mathrm\{pass\}@1by amplifying rewarded traces already in the base model, while at largekkthe base model matches or exceeds the RL\-trained model and coverage can shrink\.[Havrilla et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib15)find that RL fine\-tuning fails to explore far beyond solutions already available after supervised fine\-tuning\. The model’s*reasoning boundary*111Following[Yue et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib14), we use “reasoning boundary” to denote the set of problems a model can solve given sufficient sampling, operationalized aspass​@​k\\mathrm\{pass\}@kat largekk\.therefore tends to plateau or even shrink after standard training\.[Yao et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib16)treat expansion and shrinkage as two stages of probability\-mass reallocation: early exploitation can narrow coverage, and expansion requires training that continues into a later exploration stage\.

What, then, about problems*operationally beyond*that boundary? In challenging domains, a non\-trivial fraction of problems are*unsolvable*: every sampled rollout fails even under generous budgets \(*e\.g\.*,pass​@​64=0\\mathrm\{pass\}@64=0\)\. We find∼7%\{\\sim\}7\\%of a 2,000\-problem competition\-math corpus is unsolvable for Qwen3\-1\.7B\. These problems present a paradox: they mark the model’s reasoning boundary and carry significant learning potential, yet under standard GRPO, uniform failure produces vanishing advantage and near\-zero gradient\.

This motivates our central question:

> *Can unsolvable problems beunlocked222Throughout this paper,*unlocking*refers to converting a problem that produces vanishing gradient under RLVR \(due to uniform rollout failure\) into one that yields non\-trivial learning signal and meaningful group\-relative advantage\.for RLVR, and can they alone drive data\-efficient reasoning improvement and frontier expansion?*

We answer affirmatively\. Our key insight is that a partial reasoning trace from a stronger teacher transforms a single unsolvable problem into a*family of states*with graded difficulty, indexed by a guidance levelρ∈\[0,1\]\\rho\\in\[0,1\]controlling how much of the teacher’s reasoning is revealed\. A backward\-chaining curriculum overρ\\rhocan then be utilized to progressively withdraw guidance toward unaided solving\.

Figure[1](https://arxiv.org/html/2609.13997#S1.F1)summarizes the pipeline\. Training on128 unsolvable problemswith backward\-chaining curricula[Amani et al\. \(2026\)](https://arxiv.org/html/2609.13997#bib.bib18)matches or exceeds GRPO on the full 2,000\-problem corpus \(∼16×\{\\sim\}16\\timesdata reduction\) on the nine\-benchmark average for both base models, and*substantially expands*the reasoning boundary \(AIMEpass​@​k\\mathrm\{pass\}@kimproves at everykkup to256256\)\.

We further observe that existing backward\-chaining curricula are not optimized for the unsolvable\-only regime\. Every training step at guidance levelρ\>0\\rho\>0occurs under a distribution that differs from the test\-time distribution \(ρ=0\\rho=0\), and generic curricula have no explicit mechanism to maximize training atρ=0\\rho=0\. We formalize this distribution\-shift cost in §[3\.2](https://arxiv.org/html/2609.13997#S3.SS2)and proposeMonotone Frontier Curriculum \(MFC\), a method that maintains a per\-sample monotone non\-increasing guidance frontier and consistently outperforms existing curricula\.

Our contributions are as follows:

- •We demonstrate thatteacher\-guided curriculum learning unlocks unsolvable problems for RLVR\. Training on 128 such problems achieves data efficiency comparable to or exceeding GRPO on 2,000 mixed\-difficulty problems\.
- •We show that this paradigmexpands the reasoning boundary\(pass​@​k\\mathrm\{pass\}@kgains at everykkup to 256\), contrasting with prior observations that RLVR narrows coverage[Yue et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib14)\.
- •We proposeMonotone Frontier Curriculum \(MFC\), designed for the unsolvable\-only regime, which maximizes unguided training through a monotone per\-sample frontier and consistently outperforms existing curricula\.

## 2The Unsolvable Regime

We first establish the empirical foundation for our work: we formalize what it means for a problem to be*unsolvable*under RLVR \(§[2\.1](https://arxiv.org/html/2609.13997#S2.SS1)\), verify that most of such problems remain inert throughout training \(§[2\.2](https://arxiv.org/html/2609.13997#S2.SS2)\), and show that teacher guidance can unlock them as a rich source of learning signal \(§[2\.3](https://arxiv.org/html/2609.13997#S2.SS3)\)\.

### 2\.1Unsolvable Problems and Vanishing Gradient Signal

GRPO[Shao et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib3)trains a policyπθ\\pi\_\{\\theta\}by samplingnnrollouts per prompt and computing a group\-relative advantageAi=\(ri−μg\)/\(σg\+ϵ\)A\_\{i\}=\(r\_\{i\}\-\\mu\_\{g\}\)/\(\\sigma\_\{g\}\+\\epsilon\)\. When every rollout returns the same reward,σg=0\\sigma\_\{g\}=0and all advantages vanish, producing no gradient\. For binary rewards this occurs whenever a group is uniformly correct or uniformly incorrect\.

###### Definition 1\(Unsolvable problem\)\.

For a policyπ0\\pi\_\{0\}and a sampling budgetNN, a problemssis*NN\-unsolvable*ifpass​@​N​\(π0,s\)=0\\mathrm\{pass\}@N\(\\pi\_\{0\},s\)=0, i\.e\., no correct solution is found amongNNindependent rollouts\.

The budgetNNshould be large enough relative to the GRPO per\-prompt rollout countnnso thatNN\-unsolvability reliably predicts uniform failure within each training group\. We usen=8n=8rollouts per group and setN=64N=64: a problem with no success in 64 attempts is overwhelmingly likely to producen=8n=8failures at almost every training step\. From our 2,000\-problem competition\-mathematics corpus \(details in Appendix[C](https://arxiv.org/html/2609.13997#A3)\), approximately 7% of problems \(147147total\) are 64\-unsolvable for Qwen3\-1\.7B\. These problems potentially mark the model’s reasoning boundary, yet they contribute negligible gradient signal under GRPO\.

### 2\.2The Waste Hypothesis

One might expect training to gradually render some unsolvable problems productive, so that they begin generating learning signals\. We test this by tracking the147147unsolvable problems across all checkpoints of a 1,200\-step GRPO baseline on Qwen3\-1\.7B \(the same run described in §[4](https://arxiv.org/html/2609.13997#S4)\), evaluatingpass​@​8\\mathrm\{pass\}@8at each checkpoint\.

Figure 2:Averagepass​@​8\\mathrm\{pass\}@8on the147147initially unsolvable problems \(pass​@​64=0\\mathrm\{pass\}@64=0\) across GRPO training checkpoints\.As shown in Figure[2](https://arxiv.org/html/2609.13997#S2.F2),pass​@​8\\mathrm\{pass\}@8rises to∼\\sim24% by step 300–400 then plateaus; at step 1,200, more than 75% of initially unsolvable problems still yield zero successes\. The small fraction that becomes learnable can be primarily explained by the model’s modest capability growth during training\. This confirms thewaste hypothesis: under GRPO, the vast majority of unsolvable problems are*structurally excluded*from learning throughout training\.

### 2\.3Unlocking via Teacher Guidance

If the bottleneck is uniform failure, a natural remedy is to reduce effective difficulty so that some succeed\. We employ DeepSeek\-V3\.2[DeepSeek\-AI et al\. \(2025b\)](https://arxiv.org/html/2609.13997#bib.bib31);[DeepSeek\-AI et al\. \(2025a\)](https://arxiv.org/html/2609.13997#bib.bib43)as the teacher model throughout this work; for each unsolvable problemss, we prompt the teacher model to generate a structured reasoning traceT⁡\(s\)T\(s\)consisting of discrete steps \(details in Appendix[G](https://arxiv.org/html/2609.13997#A7)\)\. A*guidance level*ρ∈\[0,1\]\\rho\\in\[0,1\], then, controls the fraction of this trace revealed to the student\. Lettingnsteps​\(s\)n\_\{\\mathrm\{steps\}\}\(s\)denote the number of steps inT⁡\(s\)T\(s\), we expose the firstggsteps:

g\\displaystyle g=round⁡\(ρ⋅nsteps​\(s\)\),\\displaystyle=\\mathrm\{round\}\(\\rho\\cdot n\_\{\\mathrm\{steps\}\}\(s\)\),\(1\)sρ\\displaystyle s\_\{\\rho\}=s∪\{firstgsteps ofT⁡\(s\)\}\.\\displaystyle=s\\,\\cup\\,\\\{\\text\{first $g$ steps of $T\(s\)$\}\\\}\.The student receives the augmented promptsρs\_\{\\rho\}in which the teacher’s partial trace is presented as auxiliary context \(a “hint”\) and then generates a complete solution from scratch \(see Appendix[G](https://arxiv.org/html/2609.13997#A7)for templates\)\.

Table[1](https://arxiv.org/html/2609.13997#S2.T1)shows the*unlock curve*:pass​@​8\\mathrm\{pass\}@8as a function ofρ\\rhoon base Qwen3\-1\.7B over the147147unsolvable problems\. We measurepass​@​8\\mathrm\{pass\}@8because it mirrors the GRPO rollout regime \(n=8n=8\): a problem withpass​@​8\>0\\mathrm\{pass\}@8\>0yields non\-trivial group\-relative advantage\.

Table 1:pass​@​8\\mathrm\{pass\}@8of Qwen3\-1\.7B on the147147unsolvable problems at varying guidance levelsρ\\rho\(averaged over44independent evaluations\)\.The curve rises sharply\.pass​@​8\\mathrm\{pass\}@8increases from2\.2%2\.2\\%atρ=0\\rho=0to34\.0%34\.0\\%atρ=0\.25\\rho=0\.25and85\.4%85\.4\\%atρ=1\\rho=1, with a broad learnable band inρ∈\[0\.25,1\]\\rho\\in\[0\.25,1\]\. The2\.2%2\.2\\%residual means that the per\-problem success probability is small but not strictly zero, which remains consistent with the finite\-samplepass​@​64=0\\mathrm\{pass\}@64=0filter\. Teacher guidance converts the otherwise inert population into one that produces non\-trivial advantage at every step, the operational meaning of*unlocking*\.

Each unsolvable problem thus becomes a*family of training states*indexed byρ\\rho, naturally suggesting a*backward\-chaining curriculum*: begin at highρ\\rhowhere the student reliably succeeds, and progressively withdraw guidance towardρ=0\\rho=0, where the student must solve the problem entirely on its own\. We formalize this paradigm in the next section\.

## 3Curriculum Methods for Unsolvable Problems

### 3\.1The Backward\-Chaining Family

A curriculum is*backward\-chaining*if it places non\-trivial training mass atρ\>0\\rho\>0and progressively shifts that mass towardρ=0\\rho=0\. Three canonical sub\-families occupy this design space\.

#### Staged curricula\.

A staged curriculum maintains a single global scheduleρglobal​\(t\)\\rho\_\{\\mathrm\{global\}\}\(t\)advanced when an aggregate success threshold is met[Salimans and Chen \(2018\)](https://arxiv.org/html/2609.13997#bib.bib22)\. In practice, curriculum performance is dominated by scheduler design, with no settled best practice for when and how to advance the globalρ\\rho\. Vanilla staged reverse curricula can exhibit sharp drops at stage transitions and degradation of earlier\-stage skill[Xi et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib17)\. For the experiments of §[4](https://arxiv.org/html/2609.13997#S4)we therefore do not include a staged baseline\.

#### Mixture curricula\.

A mixture curriculum bypasses the scheduler by expanding the per\-sample state space: each problem is replicated atMMdiscretizedρ\\rho\-levels, and a single training pass uniformly samples across all levels\. For the experiments of §[4](https://arxiv.org/html/2609.13997#S4)we use R3[Xi et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib17)as the Mixture baseline withM=5M=5levels \(ρ∈\{0,0\.25,0\.5,0\.75,1\}\\rho\\in\\\{0,0\.25,0\.5,0\.75,1\\\}\), expanding the128128unsolvable problems into a640640\-row dataset trained with standard GRPO\.

#### Per\-sample adaptive curricula\.

A per\-sample adaptive curriculum maintains one schedule per sample and updates it based on rollout outcomes\. The reference instantiation is AdaBack[Amani et al\. \(2026\)](https://arxiv.org/html/2609.13997#bib.bib18), a per\-sample adaptive method that we compare with the R3\-style mixture below\. Originally proposed over a continuous guidance level, AdaBack is adapted here to share MFC’s \(§[3\.3](https://arxiv.org/html/2609.13997#S3.SS3)\) discrete step lattice,333From here on we operate on the integerg∈\{0,…,nsteps​\(s\)\}g\\in\\\{0,\\ldots,n\_\{\\mathrm\{steps\}\}\(s\)\\\}of Eq\. \([1](https://arxiv.org/html/2609.13997#S2.E1)\); the continuousρ\\rhois retained only for narrative description\.ensuring both methods sample from the same space \(Appendix[B](https://arxiv.org/html/2609.13997#A2)\)\. Per sampless, AdaBack maintains an integer interval\[gmin​\(s\),gmax​\(s\)\]⊆\{0,…,nsteps​\(s\)\}\[g\_\{\\mathrm\{min\}\}\(s\),g\_\{\\mathrm\{max\}\}\(s\)\]\\subseteq\\\{0,\\ldots,n\_\{\\mathrm\{steps\}\}\(s\)\\\}initialized to the full range\. The first query usesρ=0\.5\\rho=0\.5, and subsequent visits samplegused∼Uniform⁡\{gmin,…,gmax\}g\_\{\\mathrm\{used\}\}\\sim\\mathrm\{Uniform\}\\\{g\_\{\\mathrm\{min\}\},\\ldots,g\_\{\\mathrm\{max\}\}\\\}\. AdaBack then updates the interval*bidirectionally*: a success \(r¯≥τ\\bar\{r\}\\geq\\tau\) setsgmax←gusedg\_\{\\mathrm\{max\}\}\\leftarrow g\_\{\\mathrm\{used\}\}and resetsgmin←0g\_\{\\mathrm\{min\}\}\\leftarrow 0; a failure \(r¯<τ\\bar\{r\}<\\tau\) raisesgmin←gusedg\_\{\\mathrm\{min\}\}\\leftarrow g\_\{\\mathrm\{used\}\}\. This implements a binary search for the per\-sample*learnable band*around which AdaBack stabilizes\.

### 3\.2The Hidden Cost in the Unsolvable\-Only Regime

The unsolvable\-only regime exposes a hidden inefficiency shared by all three methods above\. We make this precise by separating the curriculum’s training\-time objective from the test\-time objective it ultimately serves\.

The training objective at stepttunder curriculum𝒞\\mathcal\{C\}samples\(s,ρ\)∼𝒞\(t\)\(s,\\rho\)\\sim\\mathcal\{C\}^\{\(t\)\}and conditions the policy on the augmented promptsρs\_\{\\rho\}of Eq\. \([1](https://arxiv.org/html/2609.13997#S2.E1)\):

J𝒞\(t\)\(θ\)=𝔼\(s,ρ\)∼𝒞\(t\)𝔼y∼πθ\(⋅∣sρ\)\[r\(s,y\)\],J\_\{\\mathcal\{C\}\}^\{\(t\)\}\(\\theta\)\\;=\\;\\mathbb\{E\}\_\{\(s,\\rho\)\\sim\\mathcal\{C\}^\{\(t\)\}\}\\,\\mathbb\{E\}\_\{y\\sim\\pi\_\{\\theta\}\(\\cdot\\mid s\_\{\\rho\}\)\}\\bigl\[r\(s,y\)\\bigr\],\(2\)wherer⁡\(s,y\)r\(s,y\)is based on the*original*problemssbut the policy seessρs\_\{\\rho\}wheneverρ\>0\\rho\>0\. The test\-time objective is the unguided case

J0\(θ\)=𝔼s∼𝒟𝔼y∼πθ\(⋅∣s\)\[r\(s,y\)\]\.J\_\{0\}\(\\theta\)\\;=\\;\\mathbb\{E\}\_\{s\\sim\\mathcal\{D\}\}\\,\\mathbb\{E\}\_\{y\\sim\\pi\_\{\\theta\}\(\\cdot\\mid s\)\}\\bigl\[r\(s,y\)\\bigr\]\.\(3\)Wheneverg⁡\(s,ρ\)\>0g\(s,\\rho\)\>0, the curriculum induces a*distribution shift*between training and test prompts\. ImprovingJ𝒞\(t\)J\_\{\\mathcal\{C\}\}^\{\(t\)\}does not automatically improveJ0J\_\{0\}: augmented visits transfer to the unguided regime through parameter sharing, but the transfer is sign\-indeterminate in general \(Appendix[F](https://arxiv.org/html/2609.13997#A6)\)\. A curriculum that aims to improveJ0J\_\{0\}should therefore allocate training mass to the unguided regime\.

We capture this via theunguided training mass

M0\(t\):=Pr\(s,ρ\)∼𝒞\(t\)\[g\(s,ρ\)=0\],M\_\{0\}^\{\(t\)\}\\;:=\\;\\Pr\_\{\(s,\\rho\)\\sim\\mathcal\{C\}^\{\(t\)\}\}\\bigl\[g\(s,\\rho\)=0\\bigr\],\(4\)the in\-batch fraction of visits at guidance length zero\. HigherM0\(t\)M\_\{0\}^\{\(t\)\}means more compute on prompts matching the test distribution\. Appendix[F](https://arxiv.org/html/2609.13997#A6)provides the formal first\-order decomposition ofJ0J\_\{0\}motivating this choice\.

This surrogate matters more in the unsolvable\-only regime than in mixed\-difficulty training\. In a mixed corpus, many samples are already feasible atρ=0\\rho=0, soM0\(t\)M\_\{0\}^\{\(t\)\}rises naturally\. In an unsolvable\-only corpus, samples produce near\-zero feasibility atρ=0\\rho=0initially, and none of the existing curricula carry an explicit mechanism to drive descent towardρ=0\\rho=0as the policy improves\. AdaBack, for instance, binary\-searches the per\-sample*learnable band*, which for an unsolvable problem typically remains aboveρ=0\\rho=0throughout training even as the policy improves substantially, soM0\(t\)M\_\{0\}^\{\(t\)\}plateaus\. Figure[3](https://arxiv.org/html/2609.13997#S3.F3)confirms this empirically: AdaBack’sM0\(t\)M\_\{0\}^\{\(t\)\}rises to∼\\sim0\.40 by step800800and remains essentially flat over the remaining steps\.

Figure 3:Unguided training massM0\(t\)M\_\{0\}^\{\(t\)\}of AdaBack on the128128\-problem unsolvable corpus \(Qwen3\-1\.7B, the same run in §[4](https://arxiv.org/html/2609.13997#S4)\)\.M0\(t\)M\_\{0\}^\{\(t\)\}rises initially but plateaus once per\-sample intervals settle in the learnable band\.
### 3\.3Monotone Frontier Curriculum \(MFC\)

The analysis above identifies a need for a curriculum that can continue increasing its unguided training mass as the policy improves\. MFC implements this goal by lowering a sample’s guidance frontier only after the student succeeds with less guidance\. As a result, each sample’s unguided sampling probability increases monotonically with demonstrated competence, and sampledρ\\rhoshifts toward00\. It restructures AdaBack’s update around an asymmetric principle:*success commits, failure is forgotten*\. AdaBack raisesgming\_\{\\mathrm\{min\}\}on failure\. MFC keeps only the success\-side ratchet, committinggcurr←gusedg\_\{\\mathrm\{curr\}\}\\leftarrow g\_\{\\mathrm\{used\}\}on a successful visit if smaller, while failures leave the per\-sample state untouched and still contribute to the GRPO advantage\. Withgming\_\{\\mathrm\{min\}\}removed, the state collapses to a single integergcurr​\(s\)g\_\{\\mathrm\{curr\}\}\(s\)that is monotonically non\-increasing intt\.

Algorithm 1Monotone Frontier Curriculum \(MFC\)1:Training corpus

𝒟\\mathcal\{D\}; for each prompt

s∈𝒟s\\in\\mathcal\{D\}, a teacher trace

T⁡\(s\)T\(s\)given as an ordered list of

nsteps​\(s\)n\_\{\\mathrm\{steps\}\}\(s\)reasoning steps; success threshold

τ\\tau; rollouts per prompt

nn\.

2:

gcurr​\(s\)←nsteps​\(s\)g\_\{\\mathrm\{curr\}\}\(s\)\\leftarrow n\_\{\\mathrm\{steps\}\}\(s\)for all

s∈𝒟s\\in\\mathcal\{D\}\.

3:foreach visit to

s∈𝒟s\\in\\mathcal\{D\}during GRPO trainingdo

4:

gused∼Uniform⁡\{0,1,…,gcurr​\(s\)\}g\_\{\\mathrm\{used\}\}\\sim\\mathrm\{Uniform\}\\\{0,1,\\ldots,g\_\{\\mathrm\{curr\}\}\(s\)\\\}
5:Roll out

nntrajectories from

sswith the first

gusedg\_\{\\mathrm\{used\}\}steps of

T⁡\(s\)T\(s\)prepended as hint

6:Compute mean reward

r¯\\bar\{r\}and apply the GRPO update

7:if

r¯≥τ\\bar\{r\}\\geq\\tauand

gused<gcurr​\(s\)g\_\{\\mathrm\{used\}\}<g\_\{\\mathrm\{curr\}\}\(s\)then

8:

gcurr​\(s\)←gusedg\_\{\\mathrm\{curr\}\}\(s\)\\leftarrow g\_\{\\mathrm\{used\}\}
9:endif

10:endfor

This asymmetry suits the unsolvable\-only regime: with a rapidly evolving policy, failure at lowρ\\rhotoday is weak evidence for failure tomorrow, whereas success is firm evidence that a lower frontier is feasible\. The monotone descent propagates directly to the surrogate: each successful visit at a lowergusedg\_\{\\mathrm\{used\}\}permanently raises MFC’s per\-sample on\-target probability1/\(gcurr​\(s\)\+1\)1/\(g\_\{\\mathrm\{curr\}\}\(s\)\+1\)via the discrete\-uniform draw of Algorithm[1](https://arxiv.org/html/2609.13997#alg1)\. Oncegcurr​\(s\)=0g\_\{\\mathrm\{curr\}\}\(s\)=0, all visits to that sample are atρ=0\\rho=0\(equivalently,ρmax​\(s\):=gcurr​\(s\)/nsteps​\(s\)=0\\rho\_\{\\mathrm\{max\}\}\(s\):=g\_\{\\mathrm\{curr\}\}\(s\)/n\_\{\\mathrm\{steps\}\}\(s\)=0\)\.

Algorithm[1](https://arxiv.org/html/2609.13997#alg1)gives the full procedure with a single hyperparameterτ\\tau\(success threshold\); we useτ=0\.5\\tau=0\.5throughout\.

## 4Experiments

Table 2:Main result\.pass​@​1\\mathrm\{pass\}@1\(%, average of88completions at temperature11\) on nine benchmarks for two base models\.Boldmarks the column\-wise best andunderlinethe column\-wise second\-best, both computed*within*each base\-model block\.### 4\.1Setup

#### Base models\.

We useQwen3\-1\.7B[Yang et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib30)as the primary base model andQwen3\-0\.6Bas a smaller\-scale replicate, both from the released base \(non\-instruct\) checkpoint\. Both models belong to the Qwen model family[Yang et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib42);[Yang et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib30)and use the same tokenizer\.

#### Datasets\.

All training and in\-domain evaluation data are subsets of OpenR1\-Math\-220k[Hugging Face \(2025\)](https://arxiv.org/html/2609.13997#bib.bib32), derived through the quality\-filter process of Appendix[C](https://arxiv.org/html/2609.13997#A3)\. We use a2,0002\{,\}000\-problem mixed\-difficulty training set \(OR1\-2k\) and a disjoint200200\-problem held\-out test set \(OR1\-200\)\. The unsolvable training pool is built directly from OR1\-2k: we evaluate Qwen3\-1\.7B atpass​@​64\\mathrm\{pass\}@64and retain the 147 problems withpass​@​64=0\\mathrm\{pass\}@64=0, generate teacher traces with DeepSeek\-V3\.2 and discard problems on which the teacher itself fails \(136 remaining\), and select the128128shortest by combined question\-plus\-trace length to bound prompt budget, yieldingUns\-128\. The same Uns\-128 is reused unchanged when training Qwen3\-0\.6B \(a fortiori unsolvable for the weaker sibling\)\. The analogous extraction on OR1\-200 yieldsUns\-22, an in\-domain unsolvable evaluation subset\. We release the full OR1\-2k, OR1\-200, Uns\-128, and Uns\-22 records, together with the Uns\-128 teacher traces, in a public Hugging Face dataset repository\.444[https://huggingface\.co/datasets/yukangzhu/unlocking\-the\-unsolvable](https://huggingface.co/datasets/yukangzhu/unlocking-the-unsolvable)

#### Algorithms\.

For each base model we compare a*Base*\(no\-RL\) reference and four trained algorithms, all run for1,2001\{,\}200GRPO steps under matched compute:\(1\) GRPOon the full OR1\-2k corpus with no teacher guidance;\(2\) Mixture Curriculum\(R3\-style; §[3\.1](https://arxiv.org/html/2609.13997#S3.SS1)\) on Uns\-128 expanded into a640640\-row dataset atM=5M=5uniformρ\\rho\-levels;\(3\) AdaBackon Uns\-128 with the discrete adaptation of §[3\.1](https://arxiv.org/html/2609.13997#S3.SS1)\(τ=0\.5\\tau=0\.5, default initialρ=0\.5\\rho=0\.5\); and\(4\) MFC\(ours\) on Uns\-128 withτ=0\.5\\tau=0\.5\(Algorithm[1](https://arxiv.org/html/2609.13997#alg1)\)\. All teacher\-guided runs provide the partial trace as a hint within the prompt \(§[2\.3](https://arxiv.org/html/2609.13997#S2.SS3)\); all four share an identical GRPO core \(batch size128128,n=8n=8rollouts at temperature11,1​e−61\\mathrm\{e\}\{\-6\}learning rate, low\-variance KL with coefficient1​e−31\\mathrm\{e\}\{\-3\},8,1928\{,\}192\-token response cap\) augmented with two DAPO[Yu et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib4)stabilizers:*clip\-higher*and an*overlong\-response penalty*\. Full hyperparameter tables and launcher pointers are in Appendix[B](https://arxiv.org/html/2609.13997#A2)\.

#### Benchmarks\.

Figure 4:Reasoning boundary measured bypass​@​k\\mathrm\{pass\}@kon the union of AIME24/25/26 \(9090problems\),k∈\{1,2,4,8,16,32,64,128,256\}k\\in\\\{1,2,4,8,16,32,64,128,256\\\}on alog2\\log\_\{2\}axis\. Left: Qwen3\-1\.7B; right: Qwen3\-0\.6B\. The unsolvable\-curriculum methods \(Mixture, AdaBack, MFC\) widen the gap over both*Base*and GRPO at largekk, with MFC uniformly best\.

### 4\.2Main Results

Table[2](https://arxiv.org/html/2609.13997#S4.T2)reportspass​@​1\\mathrm\{pass\}@1on all nine benchmarks for both base models and the four trained algorithms, with the un\-trained*Base*reference for context\. We summarize three results from Table[2](https://arxiv.org/html/2609.13997#S4.T2):

1. 1\.128128unsolvable problems match or exceed2,0002\{,\}000mixed\.On Qwen3\-1\.7B, AdaBack and MFC stay within1\.41\.4pp of the GRPO 2k baseline on OR1\-200 and MATH\-500\. On Qwen3\-0\.6B, the corresponding gaps range from0\.70\.7to3\.33\.3pp\. Both methods exceed GRPO on Uns\-22 and on the average across AIME24/25/26 \(e\.g\.\+7\.9\+7\.9/\+6\.8\+6\.8pp for MFC on Uns\-22\), with a∼\\sim16×\\timescut in source data\. Mixture is competitive on Qwen3\-1\.7B but trails GRPO on the Qwen3\-0\.6B average \(30\.630\.6vs31\.331\.3\), so uniform\-ρ\\rhomixing alone is not enough at the smaller scale\.
2. 2\.MFC has the best cross\-benchmark average on both base models\.Its averages are47\.347\.3and34\.034\.0\. On the AIME24/25/26 average it improves over GRPO by\+4\.7\+4\.7/\+3\.7\+3\.7pp on Qwen3\-1\.7B / 0\.6B, and over AdaBack by\+1\.0\+1\.0/\+1\.7\+1\.7pp\.
3. 3\.The ordering MFC\>\>AdaBack\>\>GRPO holds on both base modelson the cross\-benchmark average, on Uns\-22, and on the AIME average\. Mixture and GRPO trade positions across base models\. MFC’s lead over AdaBack holds on both models, so the advantage of monotone frontier descent \(§[3\.3](https://arxiv.org/html/2609.13997#S3.SS3)\) is not specific to one model size\.

The non\-math OOD benchmarks \(SciBench, GPQA\-Diamond\) tell a complementary story: training on128128math\-only unsolvable problems does not degrade non\-math reasoning and shows mild positive transfer for MFC \(\+1\.2\+1\.2to\+4\.4\+4\.4pp over Base across the two base models\), suggesting that the model’s improved reasoning capability potentially generalizes beyond mathematics\.

### 4\.3Reasoning Boundary Expansion

We evaluate whether unsolvable\-curriculum training expands the model’s reasoning*boundary*\. We measurepass​@​k\\mathrm\{pass\}@kon the union of AIME24/25/26 \(90 problems total\) fork∈\{1,2,4,8,16,32,64,128,256\}k\\in\\\{1,2,4,8,16,32,64,128,256\\\}, using the same five rows of Table[2](https://arxiv.org/html/2609.13997#S4.T2)\(*Base*, GRPO, Mixture, AdaBack, MFC\) on each base model\.

Plain GRPO shifts the frontier modestly\. On Qwen3\-1\.7B,pass​@​64\\mathrm\{pass\}@64rises from54\.4%54\.4\\%\(Base\) to57\.2%57\.2\\%\(\+2\.8\+2\.8pp\), andpass​@​256\\mathrm\{pass\}@256rises from57\.8%57\.8\\%to60\.0%60\.0\\%\(\+2\.2\+2\.2pp\)\. On Qwen3\-0\.6B, the corresponding gains are\+4\.1\+4\.1pp atpass​@​64\\mathrm\{pass\}@64and\+3\.9\+3\.9pp atpass​@​256\\mathrm\{pass\}@256\. These gains still trail every teacher\-guided method, consistent with prior reports[Yue et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib14)that standard RL primarily re\-weights existing solution paths\. Mixture, AdaBack, and MFC each add several further points at largekk, and their gains over Base tend to be larger at highkk\. Largekkis the setting that most directly measures reasoning coverage\. MFC is highest on both base models at everykk\. Atpass​@​256\\mathrm\{pass\}@256, it reaches70\.0%70\.0\\%on Qwen3\-1\.7B and53\.9%53\.9\\%on Qwen3\-0\.6B, exceeding GRPO by\+10\.0\+10\.0pp for both models\. Its lead over AdaBack widens at largekk, reaching\+3\.0\+3\.0pp atpass​@​256\\mathrm\{pass\}@256for Qwen3\-0\.6B and\+3\.3\+3\.3pp for Qwen3\-1\.7B\. Together with the analysis in §[3\.2](https://arxiv.org/html/2609.13997#S3.SS2), the higher unguided training mass accumulated by MFC is associated with broader reasoning coverage at largekk\.

### 4\.4Data Efficiency

The four algorithms span a∼\\sim16×\\timesrange in source\-data size:2,0002\{,\}000problems for GRPO,640640rows \(128×5128\\times 5\) for Mixture, and128128problems for AdaBack and MFC, all under matched compute and matched1,2001\{,\}200GRPO steps\. As summarized in the right panel of Figure[1](https://arxiv.org/html/2609.13997#S1.F1)and Table[2](https://arxiv.org/html/2609.13997#S4.T2), AdaBack and MFC match or exceed the GRPO 2k cross\-benchmark average on both base models\. Mixture exceeds GRPO on Qwen3\-1\.7B but trails it on Qwen3\-0\.6B\. Under matched compute, MFC on Uns\-128 attains a higher cross\-benchmark average than GRPO on OR1\-2k while using∼\\sim16×\\timesfewer source problems\.

## 5Analysis

### 5\.1Curriculum Dynamics

We now verify that this monotone update raises the unguided training massM0\(t\)M\_\{0\}^\{\(t\)\}\(Eq\. \([4](https://arxiv.org/html/2609.13997#S3.E4)\)\) and lowers the sampled guidance level\. Figure[5](https://arxiv.org/html/2609.13997#S5.F5)compares MFC and AdaBack on Uns\-128 \(Qwen3\-1\.7B\) acrossM0\(t\)M\_\{0\}^\{\(t\)\}andmean​\_​rho​\_​used\\mathrm\{mean\\\_rho\\\_used\}, the batch\-averaged guidance levelρ\\rhoactually sampled at each step \(lower is closer to the test\-time regime\)\. By step1,2001\{,\}200, MFC reaches a smoothedM0\(t\)≈0\.60M\_\{0\}^\{\(t\)\}\\approx 0\.60versus AdaBack’s≈0\.40\\approx 0\.40, and drivesmean​\_​rho​\_​used\\mathrm\{mean\\\_rho\\\_used\}down to≈0\.09\\approx 0\.09versus AdaBack’s≈0\.20\\approx 0\.20\. The pattern is consistent: MFC’s monotone frontier ratchetsρmax\\rho\_\{\\mathrm\{max\}\}downward on every successful visit, progressively shifting the sampling distribution towardg=0g=0; AdaBack’s bidirectional update stabilizes once a learnable band is established and has no explicit mechanism to push further\. The same separation holds on Qwen3\-0\.6B \(Appendix[D\.2](https://arxiv.org/html/2609.13997#A4.SS2)\)\.

Figure 5:Curriculum dynamics of MFC vs\. AdaBack on Uns\-128 \(Qwen3\-1\.7B\)\. Top: unguided training massM0\(t\)M\_\{0\}^\{\(t\)\}; bottom: average guidance level used per step\. MFC continues to drive training toward the unguided regime throughout the1,2001\{,\}200\-step run, whereas AdaBack largely plateaus after step∼\\sim800\.Appendix[F](https://arxiv.org/html/2609.13997#A6)decomposesΔ​J0\\Delta J\_\{0\}into an unguided term weighted byM0\(t\)M\_\{0\}^\{\(t\)\}and an augmented term whose sign is unconstrained\. MFC’s higherM0M\_\{0\}corresponds to a larger share of compute spent on prompts with the test\-time input format\. Empirically, this increase is associated with a higher cross\-benchmark average and broader reasoning coverage \(Figure[4](https://arxiv.org/html/2609.13997#S4.F4)\)\.

### 5\.2SFT Baseline vs\. Curriculum RL

A natural concern is that curriculum RL on teacher traces is effectively expensive distillation\. We test this with a supervised\-fine\-tuning \(SFT\) baseline: Qwen3\-1\.7B trained on the same Uns\-128 problems via next\-token prediction on the full teacher trace \(ρ=1\\rho=1\), using the identicalρ=0\\rho=0prompt template as the RL evaluation prompt\. We train for5050epochs \(lr1​e−51\\mathrm\{e\}\{\-5\}, cosine schedule\) and select the step\-4040checkpoint where loss has converged; full configuration in Appendix[E](https://arxiv.org/html/2609.13997#A5)\.

Table 3:Cross\-benchmark averagepass​@​1\\mathrm\{pass\}@1\(%\) on Qwen3\-1\.7B\. SFT uses the same Uns\-128 corpus and full teacher traces; curriculum RL methods use the same data under adaptive guidance\. Per\-benchmark breakdown in Appendix[E](https://arxiv.org/html/2609.13997#A5)\.Table[3](https://arxiv.org/html/2609.13997#S5.T3)shows that SFT improves over Base by only\+3\.0\+3\.0pp on average, far short of AdaBack \(\+9\.0\+9\.0\) and MFC \(\+10\.5\+10\.5\)\. SFT memorizes a single teacher path per problem and converges quickly \(under5050epochs, see Appendix[E](https://arxiv.org/html/2609.13997#A5)\) on this small corpus; curriculum RL, by contrast, drivesρ\\rhoto00under reward feedback and forces the policy to discover its own solutions\. The curriculum\-RL gains are therefore not attributable to teacher\-trace distillation: the same128128problems and the same teacher information yield qualitatively different outcomes under SFT versus curriculum RL\.

## 6Related Work

#### Problems beyond the RLVR frontier\.

Studies of reasoning boundaries find that standard RLVR often amplifies solution paths already available to the model, while many problems remain outside its sampled coverage[Yue et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib14);[Havrilla et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib15)\. Uniform failure creates an optimization problem because a GRPO group with identical rewards has no group\-relative advantage\. DAPO dynamic sampling removes groups with accuracy zero or one when constructing a training batch[Yu et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib4)\. RL\-ZVP keeps zero\-variance groups and supplies an entropy\-guided token\-level advantage[Le et al\. \(2026\)](https://arxiv.org/html/2609.13997#bib.bib52)\. SwS identifies weaknesses from persistent training failures and synthesizes new questions around the associated concepts[Liang et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib51)\. Adaptive Difficulty Curriculum Learning \(ADCL\) periodically re\-estimates problem difficulty and reorders upcoming batches as model capability changes[Zhang et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib20)\. For an unsolvable\-only pool, reordering alone does not change uniformly failed rollout groups\. Our setting keeps the original problems selected bypass​@​64=0\\mathrm\{pass\}@64=0\. Partial teacher traces turn each problem into guided states with mixed rollout outcomes, allowing the curriculum to train on the same fixed corpus\.

#### Backward\-chaining over partial solutions\.

When a solution trace is available, the amount of revealed reasoning becomes a curriculum variable\. R3moves the generation starting point through a correct demonstration and mixes examples from several stages[Xi et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib17)\. In our setup the trace is auxiliary context, and the student generates a complete solution from scratch\. AdaBack adapts the revealed portion per sample through bidirectional updates to a feasible interval[Amani et al\. \(2026\)](https://arxiv.org/html/2609.13997#bib.bib18)\. SEELE adjusts hint length to keep rollout accuracy near a target level using item response theory[Li et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib21)\. MFC keeps a single upper frontier and lowers it only after success\. This update is designed to increase unguided training mass when every original prompt is initially infeasible\.

#### Teacher traces and data\-efficient RLVR\.

Data\-centric studies show that hard problems, high reward variance, and even carefully chosen single examples can be especially useful for RLVR[Pikus et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib8);[Foster et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib9);[Wang et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib10)\. Our study considers the endpoint where the selected problems have no sampled success before guidance\. Teacher\-generated solutions are also widely used as supervised targets or to bootstrap reasoning training[Hinton et al\. \(2015\)](https://arxiv.org/html/2609.13997#bib.bib25);[Zelikman et al\. \(2022\)](https://arxiv.org/html/2609.13997#bib.bib29);[Gulcehre et al\. \(2023\)](https://arxiv.org/html/2609.13997#bib.bib27);[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib28)\. Here the teacher trace enters the prompt as context for an RL objective, and the student still produces a complete solution\. The SFT comparison in §[5\.2](https://arxiv.org/html/2609.13997#S5.SS2)isolates this use of the same Uns\-128 traces as supervised targets\. Appendix[A](https://arxiv.org/html/2609.13997#A1)reviews the broader data\-selection, curriculum\-learning, and teacher\-guidance literature\.

## 7Conclusion

We showed that*unsolvable problems*, defined bypass​@​64=0\\mathrm\{pass\}@64=0for the base model, can be turned into productive RLVR training signal via a stronger teacher’s structured trace combined with a backward\-chaining curriculum\. Training on only128128such problems matches or exceeds a2,0002\{,\}000\-problem GRPO baseline on the nine\-benchmark average for both base models \(∼\\sim16×\\timesdata efficiency\) and expands the reasoning boundary \(pass​@​k\\mathrm\{pass\}@kgains up tok=256k=256on AIME24/25/26\)\.

Within this regime we further identified a distribution\-shift cost inherent to guided curricula and proposedMonotone Frontier Curriculum \(MFC\), whose per\-sample frontier descends monotonically under a success\-only update\. MFC outperforms both the GRPO baseline and existing backward\-chaining curricula on the cross\-benchmark average for both base models\. Our findings suggest that teacher\-guided curricula, paired with the right scheduling mechanism, can unlock substantial reasoning gains from a remarkably small pool of hard problems\.

## Limitations

Our work relies on a single fixed teacher model \(DeepSeek\-V3\.2\) to generate all reasoning traces, and the quality of the resulting curriculum is inevitably tied to the quality of these traces\. We do not study how teacher capability interacts with curriculum design\. A stronger teacher may raise the attainableM0\(t\)M\_\{0\}^\{\(t\)\}per sample, and it may also introduce reasoning patterns whose hints transfer less cleanly to a smaller student\.

The framework is trained only on mathematical reasoning with single\-turn LLM outputs\. SciBench and GPQA\-Diamond provide out\-of\-domain probes, but they do not constitute a study of scientific reasoning\. Transfer to code generation[Li et al\. \(2022\)](https://arxiv.org/html/2609.13997#bib.bib48)or formal theorem proving is untested\. These domains introduce distinct challenges: code requires multi\-file coherence and execution\-based verification, while scientific reasoning often lacks the discrete step structure our hint mechanism relies on\. We have not applied the curriculum to*agentic*tasks[Jimenez et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib49);[Zhou et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib50), which need sequential environment actions and multi\-step tool\-use traces\.

## References

- Amaniet al\.\(2026\)M\. H\. Amani, A\. Lotfi, N\. Baldwin, S\. Bengio, M\. Farajtabar, E\. Abbe, and R\. WestRL for reasoning by adaptively revealing rationales\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=wdbgTG5kib)Cited by:[§B\.2](https://arxiv.org/html/2609.13997#A2.SS2.p1.1),[§1](https://arxiv.org/html/2609.13997#S1.p6.1),[§3\.1](https://arxiv.org/html/2609.13997#S3.SS1.SSS0.Px3.p1.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px2.p1.1)\.
- Bengioet al\.\(2009\)Y\. Bengio, J\. Louradour, R\. Collobert, and J\. WestonCurriculum learning\.InProceedings of the 26th annual international conference on machine learning,pp\. 41–48\.Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px2.p1.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.External Links:2110\.14168,[Link](https://arxiv.org/abs/2110.14168)Cited by:[Appendix C](https://arxiv.org/html/2609.13997#A3.SS0.SSS0.Px1.p1.1)\.
- Cuiet al\.\(2025\)G\. Cui, L\. Yuan, Z\. Wang, H\. Wang, Y\. Zhang, J\. Chen, W\. Li, B\. He, Y\. Fan, T\. Yu, Q\. Xu, W\. Chen, J\. Yuan, H\. Chen, K\. Zhang, X\. Lv, S\. Wang, Y\. Yao, X\. Han, H\. Peng, Y\. Cheng, Z\. Liu, M\. Sun, B\. Zhou, and N\. DingProcess reinforcement through implicit rewards\.External Links:2502\.01456,[Link](https://arxiv.org/abs/2502.01456)Cited by:[§1](https://arxiv.org/html/2609.13997#S1.p1.1)\.
- DeepSeek\-AIet al\.\(2025a\)DeepSeek\-AI, A\. Liu, B\. Feng, B\. Xue, B\. Wang, B\. Wu, C\. Lu, C\. Zhao, C\. Deng, C\. Zhang, C\. Ruan, D\. Dai, D\. Guo, D\. Yang, D\. Chen, D\. Ji, E\. Li, F\. Lin, F\. Dai, F\. Luo, G\. Hao, G\. Chen, G\. Li, H\. Zhang, H\. Bao, H\. Xu, H\. Wang, H\. Zhang, H\. Ding, H\. Xin, H\. Gao, H\. Li, H\. Qu, J\. L\. Cai, J\. Liang, J\. Guo, J\. Ni, J\. Li, J\. Wang, J\. Chen, J\. Chen, J\. Yuan, J\. Qiu, J\. Li, J\. Song, K\. Dong, K\. Hu, K\. Gao, K\. Guan, K\. Huang, K\. Yu, L\. Wang, L\. Zhang, L\. Xu, L\. Xia, L\. Zhao, L\. Wang, L\. Zhang, M\. Li, M\. Wang, M\. Zhang, M\. Zhang, M\. Tang, M\. Li, N\. Tian, P\. Huang, P\. Wang, P\. Zhang, Q\. Wang, Q\. Zhu, Q\. Chen, Q\. Du, R\. J\. Chen, R\. L\. Jin, R\. Ge, R\. Zhang, R\. Pan, R\. Wang, R\. Xu, R\. Zhang, R\. Chen, S\. S\. Li, S\. Lu, S\. Zhou, S\. Chen, S\. Wu, S\. Ye, S\. Ye, S\. Ma, S\. Wang, S\. Zhou, S\. Yu, S\. Zhou, S\. Pan, T\. Wang, T\. Yun, T\. Pei, T\. Sun, W\. L\. Xiao, W\. Zeng, W\. Zhao, W\. An, W\. Liu, W\. Liang, W\. Gao, W\. Yu, W\. Zhang, X\. Q\. Li, X\. Jin, X\. Wang, X\. Bi, X\. Liu, X\. Wang, X\. Shen, X\. Chen, X\. Zhang, X\. Chen, X\. Nie, X\. Sun, X\. Wang, X\. Cheng, X\. Liu, X\. Xie, X\. Liu, X\. Yu, X\. Song, X\. Shan, X\. Zhou, X\. Yang, X\. Li, X\. Su, X\. Lin, Y\. K\. Li, Y\. Q\. Wang, Y\. X\. Wei, Y\. X\. Zhu, Y\. Zhang, Y\. Xu, Y\. Xu, Y\. Huang, Y\. Li, Y\. Zhao, Y\. Sun, Y\. Li, Y\. Wang, Y\. Yu, Y\. Zheng, Y\. Zhang, Y\. Shi, Y\. Xiong, Y\. He, Y\. Tang, Y\. Piao, Y\. Wang, Y\. Tan, Y\. Ma, Y\. Liu, Y\. Guo, Y\. Wu, Y\. Ou, Y\. Zhu, Y\. Wang, Y\. Gong, Y\. Zou, Y\. He, Y\. Zha, Y\. Xiong, Y\. Ma, Y\. Yan, Y\. Luo, Y\. You, Y\. Liu, Y\. Zhou, Z\. F\. Wu, Z\. Z\. Ren, Z\. Ren, Z\. Sha, Z\. Fu, Z\. Xu, Z\. Huang, Z\. Zhang, Z\. Xie, Z\. Zhang, Z\. Hao, Z\. Gou, Z\. Ma, Z\. Yan, Z\. Shao, Z\. Xu, Z\. Wu, Z\. Zhang, Z\. Li, Z\. Gu, Z\. Zhu, Z\. Liu, Z\. Li, Z\. Xie, Z\. Song, Z\. Gao, and Z\. PanDeepSeek\-v3 technical report\.External Links:2412\.19437,[Link](https://arxiv.org/abs/2412.19437)Cited by:[§2\.3](https://arxiv.org/html/2609.13997#S2.SS3.p1.1)\.
- DeepSeek\-AIet al\.\(2025b\)DeepSeek\-AI, A\. Liu, A\. Mei, B\. Lin, B\. Xue, B\. Wang, B\. Xu, B\. Wu, B\. Zhang, C\. Lin, C\. Dong, C\. Lu, C\. Zhao, C\. Deng, C\. Xu, C\. Ruan, D\. Dai, D\. Guo, D\. Yang, D\. Chen, E\. Li, F\. Zhou, F\. Lin, F\. Dai, G\. Hao, G\. Chen, G\. Li, H\. Zhang, H\. Xu, H\. Li, H\. Liang, H\. Wei, H\. Zhang, H\. Luo, H\. Ji, H\. Ding, H\. Tang, H\. Cao, H\. Gao, H\. Qu, H\. Zeng, J\. Huang, J\. Li, J\. Xu, J\. Hu, J\. Chen, J\. Xiang, J\. Yuan, J\. Cheng, J\. Zhu, J\. Ran, J\. Jiang, J\. Qiu, J\. Li, J\. Song, K\. Dong, K\. Gao, K\. Guan, K\. Huang, K\. Zhou, K\. Huang, K\. Yu, L\. Wang, L\. Zhang, L\. Wang, L\. Zhao, L\. Yin, L\. Guo, L\. Luo, L\. Ma, L\. Wang, L\. Zhang, M\. S\. Di, M\. Y\. Xu, M\. Zhang, M\. Zhang, M\. Tang, M\. Zhou, P\. Huang, P\. Cong, P\. Wang, Q\. Wang, Q\. Zhu, Q\. Li, Q\. Chen, Q\. Du, R\. Xu, R\. Ge, R\. Zhang, R\. Pan, R\. Wang, R\. Yin, R\. Xu, R\. Shen, R\. Zhang, S\. H\. Liu, S\. Lu, S\. Zhou, S\. Chen, S\. Cai, S\. Chen, S\. Hu, S\. Liu, S\. Hu, S\. Ma, S\. Wang, S\. Yu, S\. Zhou, S\. Pan, S\. Zhou, T\. Ni, T\. Yun, T\. Pei, T\. Ye, T\. Yue, W\. Zeng, W\. Liu, W\. Liang, W\. Pang, W\. Luo, W\. Gao, W\. Zhang, X\. Gao, X\. Wang, X\. Bi, X\. Liu, X\. Wang, X\. Chen, X\. Zhang, X\. Nie, X\. Cheng, X\. Liu, X\. Xie, X\. Liu, X\. Yu, X\. Li, X\. Yang, X\. Li, X\. Chen, X\. Su, X\. Pan, X\. Lin, X\. Fu, Y\. Q\. Wang, Y\. Zhang, Y\. Xu, Y\. Ma, Y\. Li, Y\. Li, Y\. Zhao, Y\. Sun, Y\. Wang, Y\. Qian, Y\. Yu, Y\. Zhang, Y\. Ding, Y\. Shi, Y\. Xiong, Y\. He, Y\. Zhou, Y\. Zhong, Y\. Piao, Y\. Wang, Y\. Chen, Y\. Tan, Y\. Wei, Y\. Ma, Y\. Liu, Y\. Yang, Y\. Guo, Y\. Wu, Y\. Wu, Y\. Cheng, Y\. Ou, Y\. Xu, Y\. Wang, Y\. Gong, Y\. Wu, Y\. Zou, Y\. Li, Y\. Xiong, Y\. Luo, Y\. You, Y\. Liu, Y\. Zhou, Z\. F\. Wu, Z\. Z\. Ren, Z\. Zhao, Z\. Ren, Z\. Sha, Z\. Fu, Z\. Xu, Z\. Xie, Z\. Zhang, Z\. Hao, Z\. Gou, Z\. Ma, Z\. Yan, Z\. Shao, Z\. Huang, Z\. Wu, Z\. Li, Z\. Zhang, Z\. Xu, Z\. Wang, Z\. Gu, Z\. Zhu, Z\. Li, Z\. Zhang, Z\. Xie, Z\. Gao, Z\. Pan, Z\. Yao, B\. Feng, H\. Li, J\. L\. Cai, J\. Ni, L\. Xu, M\. Li, N\. Tian, R\. J\. Chen, R\. L\. Jin, S\. S\. Li, S\. Zhou, T\. Sun, X\. Q\. Li, X\. Jin, X\. Shen, X\. Chen, X\. Song, X\. Zhou, Y\. X\. Zhu, Y\. Huang, Y\. Li, Y\. Zheng, Y\. Zhu, Y\. Ma, Z\. Huang, Z\. Xu, Z\. Zhang, D\. Ji, J\. Liang, J\. Guo, J\. Chen, L\. Xia, M\. Wang, M\. Li, P\. Zhang, R\. Chen, S\. Sun, S\. Wu, S\. Ye, T\. Wang, W\. L\. Xiao, W\. An, X\. Wang, X\. Sun, X\. Wang, Y\. Tang, Y\. Zha, Z\. Zhang, Z\. Ju, Z\. Zhang, and Z\. QuDeepSeek\-v3\.2: pushing the frontier of open large language models\.External Links:2512\.02556,[Link](https://arxiv.org/abs/2512.02556)Cited by:[item 2](https://arxiv.org/html/2609.13997#A3.I2.i2.p1.1),[§2\.3](https://arxiv.org/html/2609.13997#S2.SS3.p1.1)\.
- Fosteret al\.\(2025\)T\. Foster, A\. Sims, J\. Forkel, and J\. N\. FoersterLILO: learning to reason at the frontier of learnability\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=8HYeWMf0W3)Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.13997#S1.p1.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px3.p1.1)\.
- Gulcehreet al\.\(2023\)C\. Gulcehre, T\. L\. Paine, S\. Srinivasan, K\. Konyushkova, L\. Weerts, A\. Sharma, A\. Siddhant, A\. Ahern, M\. Wang, C\. Gu, W\. Macherey, A\. Doucet, O\. Firat, and N\. de FreitasReinforced self\-training \(rest\) for language modeling\.External Links:2308\.08998,[Link](https://arxiv.org/abs/2308.08998)Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px3.p1.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px3.p1.1)\.
- Guoet al\.\(2025\)D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi, X\. Zhang, X\. Yu, Y\. Wu, Z\. F\. Wu, Z\. Gou, Z\. Shao, Z\. Li, Z\. Gao, A\. Liu, B\. Xue, B\. Wang, B\. Wu, B\. Feng, C\. Lu, C\. Zhao, C\. Deng, C\. Ruan, D\. Dai, D\. Chen, D\. Ji, E\. Li, F\. Lin, F\. Dai, F\. Luo, G\. Hao, G\. Chen, G\. Li, H\. Zhang, H\. Xu, H\. Ding, H\. Gao, H\. Qu, H\. Li, J\. Guo, J\. Li, J\. Chen, J\. Yuan, J\. Tu, J\. Qiu, J\. Li, J\. L\. Cai, J\. Ni, J\. Liang, J\. Chen, K\. Dong, K\. Hu, K\. You, K\. Gao, K\. Guan, K\. Huang, K\. Yu, L\. Wang, L\. Zhang, L\. Zhao, L\. Wang, L\. Zhang, L\. Xu, L\. Xia, M\. Zhang, M\. Zhang, M\. Tang, M\. Zhou, M\. Li, M\. Wang, M\. Li, N\. Tian, P\. Huang, P\. Zhang, Q\. Wang, Q\. Chen, Q\. Du, R\. Ge, R\. Zhang, R\. Pan, R\. Wang, R\. J\. Chen, R\. L\. Jin, R\. Chen, S\. Lu, S\. Zhou, S\. Chen, S\. Ye, S\. Wang, S\. Yu, S\. Zhou, S\. Pan, S\. S\. Li, S\. Zhou, S\. Wu, T\. Yun, T\. Pei, T\. Sun, T\. Wang, W\. Zeng, W\. Liu, W\. Liang, W\. Gao, W\. Yu, W\. Zhang, W\. L\. Xiao, W\. An, X\. Liu, X\. Wang, X\. Chen, X\. Nie, X\. Cheng, X\. Liu, X\. Xie, X\. Liu, X\. Yang, X\. Li, X\. Su, X\. Lin, X\. Q\. Li, X\. Jin, X\. Shen, X\. Chen, X\. Sun, X\. Wang, X\. Song, X\. Zhou, X\. Wang, X\. Shan, Y\. K\. Li, Y\. Q\. Wang, Y\. X\. Wei, Y\. Zhang, Y\. Xu, Y\. Li, Y\. Zhao, Y\. Sun, Y\. Wang, Y\. Yu, Y\. Zhang, Y\. Shi, Y\. Xiong, Y\. He, Y\. Piao, Y\. Wang, Y\. Tan, Y\. Ma, Y\. Liu, Y\. Guo, Y\. Ou, Y\. Wang, Y\. Gong, Y\. Zou, Y\. He, Y\. Xiong, Y\. Luo, Y\. You, Y\. Liu, Y\. Zhou, Y\. X\. Zhu, Y\. Huang, Y\. Li, Y\. Zheng, Y\. Zhu, Y\. Ma, Y\. Tang, Y\. Zha, Y\. Yan, Z\. Z\. Ren, Z\. Ren, Z\. Sha, Z\. Fu, Z\. Xu, Z\. Xie, Z\. Zhang, Z\. Hao, Z\. Ma, Z\. Yan, Z\. Wu, Z\. Gu, Z\. Zhu, Z\. Liu, Z\. Li, Z\. Xie, Z\. Song, Z\. Pan, Z\. Huang, Z\. Xu, Z\. Zhang, and Z\. ZhangDeepSeek\-R1 incentivizes reasoning in LLMs through reinforcement learning\.Nature645\(8081\),pp\. 633–638\.External Links:ISSN 1476\-4687,[Link](https://doi.org/10.1038/s41586-025-09422-z),[Document](https://dx.doi.org/10.1038/s41586-025-09422-z)Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2609.13997#S1.p1.1)\.
- Havrillaet al\.\(2024\)A\. Havrilla, Y\. Du, S\. C\. Raparthy, C\. Nalmpantis, J\. Dwivedi\-Yu, M\. Zhuravinskyi, E\. Hambro, S\. Sukhbaatar, and R\. RaileanuTeaching large language models to reason with reinforcement learning\.arXiv preprint arXiv:2403\.04642\.Cited by:[§1](https://arxiv.org/html/2609.13997#S1.p2.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px1.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the MATH dataset\.InThirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 2\),External Links:[Link](https://openreview.net/forum?id=7Bywt2mQsCe)Cited by:[§4\.1](https://arxiv.org/html/2609.13997#S4.SS1.SSS0.Px4.p1.1)\.
- Hintonet al\.\(2015\)G\. Hinton, O\. Vinyals, and J\. DeanDistilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px3.p1.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px3.p1.1)\.
- Hugging Face \(2025\)Hugging FaceOpen r1: a fully open reproduction of deepseek\-r1\.External Links:[Link](https://github.com/huggingface/open-r1)Cited by:[Appendix C](https://arxiv.org/html/2609.13997#A3.p1.1),[§4\.1](https://arxiv.org/html/2609.13997#S4.SS1.SSS0.Px2.p1.1)\.
- Jimenezet al\.\(2024\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. R\. NarasimhanSWE\-bench: can language models resolve real\-world github issues?\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=VTF8yNQM66)Cited by:[Limitations](https://arxiv.org/html/2609.13997#Sx1.p2.1)\.
- Kwonet al\.\(2023\)W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. Gonzalez, H\. Zhang, and I\. StoicaEfficient memory management for large language model serving with pagedattention\.InProceedings of the 29th symposium on operating systems principles,pp\. 611–626\.Cited by:[§B\.3](https://arxiv.org/html/2609.13997#A2.SS3.p1.1)\.
- Leet al\.\(2026\)T\. V\. Le, M\. Jeon, K\. Vu, V\. D\. Lai, and E\. YangNo prompt left behind: exploiting zero\-variance prompts in LLM reinforcement learning via entropy\-guided advantage shaping\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=kiXFIESZKv)Cited by:[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px1.p1.1)\.
- Liet al\.\(2022\)Y\. Li, D\. Choi, J\. Chung, N\. Kushman, J\. Schrittwieser, R\. Leblond, T\. Eccles, J\. Keeling, F\. Gimeno, A\. D\. Lago, T\. Hubert, P\. Choy, C\. de Masson d’Autume, I\. Babuschkin, X\. Chen, P\. Huang, J\. Welbl, S\. Gowal, A\. Cherepanov, J\. Molloy, D\. J\. Mankowitz, E\. S\. Robson, P\. Kohli, N\. de Freitas, K\. Kavukcuoglu, and O\. VinyalsCompetition\-level code generation with alphacode\.Science378\(6624\),pp\. 1092–1097\.External Links:[Document](https://dx.doi.org/10.1126/science.abq1158),[Link](https://www.science.org/doi/abs/10.1126/science.abq1158),https://www\.science\.org/doi/pdf/10\.1126/science\.abq1158Cited by:[Limitations](https://arxiv.org/html/2609.13997#Sx1.p2.1)\.
- Liet al\.\(2025\)Z\. Li, Z\. Sun, J\. Zhao, E\. Min, Y\. Zeng, H\. Wu, H\. Cai, S\. Wang, D\. Yin, X\. Chen, and Z\. DengStaying in the sweet spot: responsive reasoning evolution via capability\-adaptive hint scaffolding\.External Links:2509\.06923,[Link](https://arxiv.org/abs/2509.06923)Cited by:[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px2.p1.1)\.
- Lianget al\.\(2025\)X\. Liang, Z\. Li, Y\. Gong, Y\. Wang, H\. Zhang, Y\. Shen, Y\. N\. Wu, and W\. ChenSwS: self\-aware weakness\-driven problem synthesis in reinforcement learning for LLM reasoning\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=0jQUNQsZra)Cited by:[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px1.p1.1)\.
- Lightmanet al\.\(2024\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=v8L0pN6EOi)Cited by:[§4\.1](https://arxiv.org/html/2609.13997#S4.SS1.SSS0.Px4.p1.1)\.
- Liuet al\.\(2025a\)M\. Liu, S\. Diao, X\. Lu, J\. Hu, X\. Dong, Y\. Choi, J\. Kautz, and Y\. DongProRL: prolonged reinforcement learning expands reasoning boundaries in large language models\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=YPsJha5HXQ)Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2025b\)Z\. Liu, C\. Chen, W\. Li, P\. Qi, T\. Pang, C\. Du, W\. S\. Lee, and M\. LinUnderstanding r1\-zero\-like training: a critical perspective\.arXiv preprint arXiv:2503\.20783\.Cited by:[§1](https://arxiv.org/html/2609.13997#S1.p1.1)\.
- Luoet al\.\(2024\)L\. Luo, Y\. Liu, R\. Liu, S\. Phatale, M\. Guo, H\. Lara, Y\. Li, L\. Shu, Y\. Zhu, L\. Meng, J\. Sun, and A\. RastogiImprove mathematical reasoning in language models by automated process supervision\.External Links:2406\.06592,[Link](https://arxiv.org/abs/2406.06592)Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px3.p1.1)\.
- Minet al\.\(2024\)Y\. Min, Z\. Chen, J\. Jiang, J\. Chen, J\. Deng, Y\. Hu, Y\. Tang, J\. Wang, X\. Cheng, H\. Song, W\. X\. Zhao, Z\. Liu, Z\. Wang, and J\. WenImitate, explore, and self\-improve: a reproduction report on slow\-thinking reasoning systems\.External Links:2412\.09413,[Link](https://arxiv.org/abs/2412.09413)Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px3.p1.1)\.
- OpenAI \(2024\)OpenAIOpenAI o1 system card\.External Links:2412\.16720,[Link](https://arxiv.org/abs/2412.16720)Cited by:[§1](https://arxiv.org/html/2609.13997#S1.p1.1)\.
- Ouyanget al\.\(2022\)L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. Christiano, J\. Leike, and R\. LoweTraining language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,A\. H\. Oh, A\. Agarwal, D\. Belgrave, and K\. Cho \(Eds\.\),External Links:[Link](https://openreview.net/forum?id=TG8KACxEON)Cited by:[§1](https://arxiv.org/html/2609.13997#S1.p1.1)\.
- Pikuset al\.\(2025\)B\. Pikus, P\. R\. Tiwari, and B\. YeHard examples are all you need: maximizing grpo post\-training under annotation budgets\.arXiv preprint arXiv:2508\.14094\.Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.13997#S1.p1.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px3.p1.1)\.
- Reinet al\.\(2024\)D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. BowmanGPQA: a graduate\-level google\-proof q&a benchmark\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=Ti67584b98)Cited by:[§4\.1](https://arxiv.org/html/2609.13997#S4.SS1.SSS0.Px4.p1.1)\.
- Rosset al\.\(2011\)S\. Ross, G\. Gordon, and D\. BagnellA reduction of imitation learning and structured prediction to no\-regret online learning\.InProceedings of the fourteenth international conference on artificial intelligence and statistics,pp\. 627–635\.Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px3.p1.1)\.
- Salimans and Chen \(2018\)T\. Salimans and R\. ChenLearning montezuma’s revenge from a single demonstration\.arXiv preprint arXiv:1812\.03381\.Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2609.13997#S3.SS1.SSS0.Px1.p1.1)\.
- Schulmanet al\.\(2017\)J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. KlimovProximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§1](https://arxiv.org/html/2609.13997#S1.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[§1](https://arxiv.org/html/2609.13997#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.13997#S2.SS1.p1.1)\.
- Shenget al\.\(2025\)G\. Sheng, C\. Zhang, Z\. Ye, X\. Wu, W\. Zhang, R\. Zhang, Y\. Peng, H\. Lin, and C\. WuHybridflow: a flexible and efficient rlhf framework\.InProceedings of the Twentieth European Conference on Computer Systems,pp\. 1279–1297\.Cited by:[Appendix B](https://arxiv.org/html/2609.13997#A2.p1.1)\.
- Tanget al\.\(2026\)X\. Tang, Z\. Zhang, Y\. Liu, W\. X\. Zhao, Z\. Wen, Z\. Zhang, and J\. ZhouTowards high data efficiency in reinforcement learning with verifiable reward\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=sruA4AZmZI)Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px1.p1.1)\.
- Taoet al\.\(2024\)S\. Tao, A\. Shukla, T\. Chan, and H\. SuReverse forward curriculum learning for extreme sample and demo efficiency\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=w4rODxXsmM)Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px2.p1.1)\.
- Uesatoet al\.\(2022\)J\. Uesato, N\. Kushman, R\. Kumar, F\. Song, N\. Siegel, L\. Wang, A\. Creswell, G\. Irving, and I\. HigginsSolving math word problems with process\- and outcome\-based feedback\.arXiv preprint arXiv:2211\.14275\.Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2024a\)P\. Wang, L\. Li, Z\. Shao, R\. Xu, D\. Dai, Y\. Li, D\. Chen, Y\. Wu, and Z\. SuiMath\-shepherd: verify and reinforce llms step\-by\-step without human annotations\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 9426–9439\.Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2024b\)X\. Wang, Z\. Hu, P\. Lu, Y\. Zhu, J\. Zhang, S\. Subramaniam, A\. R\. Loomba, S\. Zhang, Y\. Sun, and W\. WangSciBench: evaluating college\-level scientific problem\-solving abilities of large language models\.InForty\-first International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=bq1JEgioLr)Cited by:[§4\.1](https://arxiv.org/html/2609.13997#S4.SS1.SSS0.Px4.p1.1)\.
- Wanget al\.\(2025\)Y\. Wang, Q\. Yang, Z\. Zeng, L\. Ren, L\. Liu, B\. Peng, H\. Cheng, X\. He, K\. Wang, J\. Gao, W\. Chen, S\. Wang, S\. S\. Du, and Y\. ShenReinforcement learning for reasoning in large language models with one training example\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=IBrRNLr6JA)Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.13997#S1.p1.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px3.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, brian ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. ZhouChain of thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,A\. H\. Oh, A\. Agarwal, D\. Belgrave, and K\. Cho \(Eds\.\),External Links:[Link](https://openreview.net/forum?id=_VjQlMeSB_J)Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px3.p1.1)\.
- Xiet al\.\(2024\)Z\. Xi, W\. Chen, B\. Hong, S\. Jin, R\. Zheng, W\. He, Y\. Ding, S\. Liu, X\. Guo, J\. Wang, H\. Guo, W\. Shen, X\. Fan, Y\. Zhou, S\. Dou, X\. Wang, X\. Zhang, peng sun, T\. Gui, Q\. Zhang, and X\. HuangTraining large language models for reasoning through reverse curriculum reinforcement learning\.InForty\-first International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=t82Y3fmRtk)Cited by:[§3\.1](https://arxiv.org/html/2609.13997#S3.SS1.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2609.13997#S3.SS1.SSS0.Px2.p1.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px2.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. QiuQwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§4\.1](https://arxiv.org/html/2609.13997#S4.SS1.SSS0.Px1.p1.1)\.
- Yanget al\.\(2024\)A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. QiuQwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[§4\.1](https://arxiv.org/html/2609.13997#S4.SS1.SSS0.Px1.p1.1)\.
- Yaoet al\.\(2025\)X\. Yao, L\. Yu, X\. Hu, F\. Teng, Q\. Cui, J\. Zhou, and Y\. LiuThe debate on rlvr reasoning capability boundary: shrinkage, expansion, or both? a two\-stage dynamic view\.arXiv preprint arXiv:2510\.04028\.Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.13997#S1.p2.1)\.
- Yuet al\.\(2025\)Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, W\. Dai, T\. Fan, G\. Liu, J\. Liu, L\. Liu, X\. Liu, H\. Lin, Z\. Lin, B\. Ma, G\. Sheng, Y\. Tong, C\. Zhang, M\. Zhang, R\. Zhang, W\. Zhang, H\. Zhu, J\. Zhu, J\. Chen, J\. Chen, C\. Wang, H\. Yu, Y\. Song, X\. Wei, H\. Zhou, J\. Liu, W\. Ma, Y\. Zhang, L\. Yan, Y\. Wu, and M\. WangDAPO: an open\-source LLM reinforcement learning system at scale\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=2a36EMSSTp)Cited by:[§B\.1](https://arxiv.org/html/2609.13997#A2.SS1.p1.1),[§1](https://arxiv.org/html/2609.13997#S1.p1.1),[§4\.1](https://arxiv.org/html/2609.13997#S4.SS1.SSS0.Px3.p1.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px1.p1.1)\.
- Yueet al\.\(2025\)Y\. Yue, Z\. Chen, R\. Lu, A\. Zhao, Z\. Wang, Y\. Yue, S\. Song, and G\. HuangDoes reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=4OsgYD7em5)Cited by:[2nd item](https://arxiv.org/html/2609.13997#S1.I1.i2.p1.1),[§1](https://arxiv.org/html/2609.13997#S1.p2.1),[§4\.3](https://arxiv.org/html/2609.13997#S4.SS3.p2.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px1.p1.1),[footnote 1](https://arxiv.org/html/2609.13997#footnote1)\.
- Zelikmanet al\.\(2022\)E\. Zelikman, Y\. Wu, J\. Mu, and N\. GoodmanStar: bootstrapping reasoning with reasoning\.Advances in Neural Information Processing Systems35,pp\. 15476–15488\.Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px3.p1.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px3.p1.1)\.
- Zhanget al\.\(2024\)D\. Zhang, S\. Zhoubian, Z\. Hu, Y\. Yue, Y\. Dong, and J\. TangRest\-mcts\*: llm self\-training via process reward guided tree search\.Advances in Neural Information Processing Systems37,pp\. 64735–64772\.Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px3.p1.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px3.p1.1)\.
- Zhanget al\.\(2025\)E\. Zhang, X\. Yan, W\. Lin, T\. Zhang, and Q\. LuLearning like humans: advancing LLM reasoning capabilities via adaptive difficulty curriculum learning and expert\-guided self\-reformulation\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 6619–6633\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.336/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.336),ISBN 979\-8\-89176\-332\-6Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px2.p1.1),[§6](https://arxiv.org/html/2609.13997#S6.SS0.SSS0.Px1.p1.1)\.
- Zhaoet al\.\(2023\)Y\. Zhao, A\. Gu, R\. Varma, L\. Luo, C\. Huang, M\. Xu, L\. Wright, H\. Shojanazeri, M\. Ott, S\. Shleifer, A\. Desmaison, C\. Balioglu, P\. Damania, B\. Nguyen, G\. Chauhan, Y\. Hao, A\. Mathews, and S\. LiPyTorch fsdp: experiences on scaling fully sharded data parallel\.External Links:2304\.11277,[Link](https://arxiv.org/abs/2304.11277)Cited by:[§B\.3](https://arxiv.org/html/2609.13997#A2.SS3.p1.1)\.
- Zhouet al\.\(2024\)S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. NeubigWebArena: a realistic web environment for building autonomous agents\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=oKn9c6ytLx)Cited by:[Limitations](https://arxiv.org/html/2609.13997#Sx1.p2.1)\.
- Zhuet al\.\(2026\)E\. Zhu, D\. Jiang, Y\. Wang, X\. Li, J\. Cheng, Y\. Gu, Y\. Niu, A\. Zeng, J\. Tang, M\. Huang, and H\. WangData efficient RLVR via off\-policy influence guidance\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 46167–46192\.External Links:[Link](https://aclanthology.org/2026.acl-long.2141/),[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.2141),ISBN 979\-8\-89176\-390\-6Cited by:[Appendix A](https://arxiv.org/html/2609.13997#A1.SS0.SSS0.Px1.p1.1)\.

## Appendix AExtended Related Work

#### Data\-centric RLVR\.

The choice of training data profoundly impacts RLVR outcomes\.[Pikus et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib8)demonstrate that filtering to the hardest problems yields disproportionate gains over uniform sampling, while[Foster et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib9)formalize the “learnability” criterion showing that maximizing reward variance is equivalent to optimizing expected improvement\.[Wang et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib10)push this further, showing a single well\-chosen problem can unlock significant reasoning capability\.[Tang et al\. \(2026\)](https://arxiv.org/html/2609.13997#bib.bib11)study data efficiency for RLVR through offline subset selection based on diversity, influence, and moderate difficulty, together with online explorability\-based rollout pruning, and[Zhu et al\. \(2026\)](https://arxiv.org/html/2609.13997#bib.bib13)propose influence\-guided data selection to accelerate training\. Complementary work on reward signal design—process reward[Uesato et al\. \(2022\)](https://arxiv.org/html/2609.13997#bib.bib45);[Wang et al\. \(2024a\)](https://arxiv.org/html/2609.13997#bib.bib47)versus outcome reward—also shapes effective data utilization\. Our work identifies the extreme end of this spectrum—problems with*zero*feasibility—and shows they can be made productive through teacher guidance rather than filtering them away\. The treatment of problems beyond the sampled frontier is discussed in §[6](https://arxiv.org/html/2609.13997#S6)\.

#### Curriculum learning beyond partial traces\.

Curriculum learning orders or samples training experiences according to difficulty[Bengio et al\. \(2009\)](https://arxiv.org/html/2609.13997#bib.bib24)\. Reverse curricula in reinforcement learning start near successful states and move the initial state backward as the policy improves[Salimans and Chen \(2018\)](https://arxiv.org/html/2609.13997#bib.bib22);[Tao et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib23)\. Other curricula act at the batch level by selecting or reordering problems, including schedules that track changes in model capability[Zhang et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib20)\. Our guidance lattice acts within each problem by controlling the portion of a trace supplied as context\. Related studies of reasoning\-boundary dynamics examine how prolonged RLVR interacts with exploration and coverage[Liu et al\. \(2025a\)](https://arxiv.org/html/2609.13997#bib.bib12);[Yao et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib16)\.

#### Teacher traces as targets and as context\.

Knowledge distillation[Hinton et al\. \(2015\)](https://arxiv.org/html/2609.13997#bib.bib25)and imitation learning[Ross et al\. \(2011\)](https://arxiv.org/html/2609.13997#bib.bib26)train students against outputs or actions supplied by a teacher\. Chain\-of\-thought prompting[Wei et al\. \(2022\)](https://arxiv.org/html/2609.13997#bib.bib44)shows that explicit reasoning steps can improve generation, while STaR[Zelikman et al\. \(2022\)](https://arxiv.org/html/2609.13997#bib.bib29)and process reward methods[Luo et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib46)use intermediate reasoning during training\.[Guo et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib2)demonstrate that supervised distillation from reasoning traces can produce capable models\.[Gulcehre et al\. \(2023\)](https://arxiv.org/html/2609.13997#bib.bib27)and[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib28)use generated solutions to bootstrap further training\. STILL\-2 distills long reasoning traces and then bootstraps from successful rollouts[Min et al\. \(2024\)](https://arxiv.org/html/2609.13997#bib.bib19)\. Our SFT baseline in §[5\.2](https://arxiv.org/html/2609.13997#S5.SS2)compares target\-based training with the use of teacher traces as prompt context under curriculum RL\.

## Appendix BImplementation Details

This appendix specifies the training configuration shared by all RLVR runs in §[4](https://arxiv.org/html/2609.13997#S4), the per\-method curriculum hyperparameters, and the hardware setup\. All training and rollout infrastructure is built on top of verl[Sheng et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib38); our curriculum algorithms \(Mixture, AdaBack, MFC\) are implemented as additional curriculum managers within the same verl\-based stack, so that the four trained methods share an identical training and inference pipeline\.

### B\.1Shared GRPO Training Core

All four trained algorithms \(GRPO, Mixture, AdaBack, MFC\) on both base models share an identical GRPO training core \(Table[4](https://arxiv.org/html/2609.13997#A2.T4)\)\. Two modifications inherited from DAPO[Yu et al\. \(2025\)](https://arxiv.org/html/2609.13997#bib.bib4)are layered on top: an asymmetric PPO clip ratio \(ϵlow=0\.2\\epsilon\_\{\\mathrm\{low\}\}=0\.2,ϵhigh=0\.28\\epsilon\_\{\\mathrm\{high\}\}=0\.28\), and a length\-based*overlong\-response penalty*that linearly penalizes incorrect responses approaching the8,1928\{,\}192\-token cap \(buffer length10241024, penalty factor1\.01\.0\)\. The penalty extends the verifiable reward from\{0,1\}\\\{0,1\\\}to\[−1,1\]\[\-1,1\]but does not alter the group\-relative advantage structure of GRPO\. KL regularization is applied as an auxiliary loss term using a low\-variance estimator \(kl\_loss\_type=low\_var\_kl\) rather than mixed into the reward\.

Table 4:Shared GRPO training core, identical across all four trained algorithms and both base models in §[4](https://arxiv.org/html/2609.13997#S4)\.
### B\.2Per\-Method Curriculum Hyperparameters

Method\-specific curriculum hyperparameters are listed in Table[5](https://arxiv.org/html/2609.13997#A2.T5)\. Mixture has no learnable per\-sample state; its only curriculum choice is the discretizedρ\\rho\-level grid\. AdaBack follows the public\-recipe defaults; two parameters bear brief explanation\. The*on\-target valve*pzerop\_\{\\mathrm\{zero\}\}is the small fixed probability of overriding the sampledρ\\rhowithρ=0\\rho=0, introduced in the original AdaBack paper[Amani et al\. \(2026\)](https://arxiv.org/html/2609.13997#bib.bib18)to close the train\-test distribution mismatch \(we usepzero=0\.1p\_\{\\mathrm\{zero\}\}=0\.1\)\. The*minimum step delta*governs the smallest discrete change to the per\-sample hint length per update\. MFC reduces to a single hyperparameter, the success\-commit thresholdτ\\tau\(Algorithm[1](https://arxiv.org/html/2609.13997#alg1)\)\.

#### Discretization of AdaBack\.

We keep AdaBack’s hyperparameters in the original continuous\-ρ\\rhoform for fidelity with the public recipe \(Table[5](https://arxiv.org/html/2609.13997#A2.T5)\), but the actual rollout samples on the integer hint lattice so that AdaBack and MFC share an identical sampling space \(§[3\.1](https://arxiv.org/html/2609.13997#S3.SS1)\)\. At each rollout, the continuous bounds\[ρmin​\(s\),ρmax​\(s\)\]\[\\rho\_\{\\mathrm\{min\}\}\(s\),\\rho\_\{\\mathrm\{max\}\}\(s\)\]are mapped to integer bounds viag=round⁡\(ρ⋅nsteps​\(s\)\)g=\\mathrm\{round\}\(\\rho\\cdot n\_\{\\mathrm\{steps\}\}\(s\)\), an integergused∼Uniform⁡\{gmin,…,gmax\}g\_\{\\mathrm\{used\}\}\\sim\\mathrm\{Uniform\}\\\{g\_\{\\mathrm\{min\}\},\\ldots,g\_\{\\mathrm\{max\}\}\\\}is drawn, and the bidirectional update is applied directly on the integer interval \(the minimum step delta of11ensures each successful update strictly contracts it\)\. This rules out the round\-induced sampling bias atg=0g=0that would otherwise systematically penalize AdaBack’s measuredM0\(t\)M\_\{0\}^\{\(t\)\}relative to MFC’s discrete\-uniform draw\.

Table 5:Method\-specific curriculum hyperparameters\. The three teacher\-guided methods all use hint\-mode guidance with the discretization of §[3\.1](https://arxiv.org/html/2609.13997#S3.SS1)\.

### B\.3Hardware and Wall\-Clock

All RLVR runs use8×8\\timesNVIDIA RTX PRO 6000 \(96 GB\) GPUs; FSDP[Zhao et al\. \(2023\)](https://arxiv.org/html/2609.13997#bib.bib39)with parameter and optimizer sharding is used throughout \(offload disabled\)\. Rollouts are served by vLLM[Kwon et al\. \(2023\)](https://arxiv.org/html/2609.13997#bib.bib40)at70%70\\%GPU memory utilization with prefix caching enabled\. Dynamic batching usesppo\_max\_token\_len\_per\_gpu=48,000=48\{,\}000andppo\_micro\_batch\_size\_per\_gpu=4=4\. A single1,2001\{,\}200\-step training run takes approximately9090wall\-clock hours for Qwen3\-1\.7B and approximately7070hours for Qwen3\-0\.6B on the same8×8\\timesRTX PRO 6000 setup\. Including all diagnostic and development runs, our total compute usage amounted to approximately7,000\\mathbf\{7\{,\}000\}RTX PRO 6000 GPU hours\.

## Appendix CTraining\-Data Construction

This appendix details the construction pipeline for the four corpora used in §[4\.1](https://arxiv.org/html/2609.13997#S4.SS1): the mixed\-difficulty training setOR1\-2k, the held\-out test setOR1\-200, the128128\-problem unsolvable training setUns\-128, and the held\-out unsolvable subsetUns\-22\. All four are derived from the*train*split of OpenR1\-Math\-220k[Hugging Face \(2025\)](https://arxiv.org/html/2609.13997#bib.bib32)\(93\.793\.7k problems\) under a single, deterministic protocol with random seed4242\.

#### Filtering and train/test split \(→\\toOR1\-2k, OR1\-200\)\.

Starting from the93\.793\.7k\-problem train split, we apply a chain of quality filters to keep the corpus narrowly scoped to verifiable, single\-answer competition\-level mathematical reasoning problems \(substantially harder than elementary benchmarks such as GSM8K[Cobbe et al\. \(2021\)](https://arxiv.org/html/2609.13997#bib.bib37)\):

- •
- •drop multiple\-choice problems \(where the verifiable reward degenerates to chance\);
- •drop problems whose statement is too short or too long, and problems whose released reference solution is too long, to keep prompt and rollout budgets bounded;
- •drop problems whose ground\-truth answer string contains a comma \(a strong indicator of multi\-answer or list\-valued problems that defeat exact\-match scoring\)\.

From the surviving pool we randomly draw2,2002\{,\}200problems under seed4242and split them into2,0002\{,\}000training problems \(OR1\-2k\) and200200held\-out test problems \(OR1\-200\), keeping the splits disjoint\. Difficulty composition is preserved by construction \(random sampling\), and the resulting train set spans the fullpass​@​64\\mathrm\{pass\}@64spectrum from trivial \(pass​@​64≈1\\mathrm\{pass\}@64\\approx 1\) to unsolvable \(pass​@​64=0\\mathrm\{pass\}@64=0\)\.

#### Unsolvable filtering \(→\\toUns\-128\)\.

The unsolvable pool is derived only from OR1\-2k:

1. 1\.Re\-evaluate every problem in OR1\-2k with Qwen3\-1\.7B at the response\-length cap used at training time \(8,1928\{,\}192tokens\) and retain those withpass​@​64=0\\mathrm\{pass\}@64=0\(147147problems,∼\\sim7% of the train set\)\.
2. 2\.Generate a structured reasoning trace for every retained problem with the DeepSeek\-V3\.2[DeepSeek\-AI et al\. \(2025b\)](https://arxiv.org/html/2609.13997#bib.bib31)teacher \(the prompt template, structured\-output schema, and step\-extraction rule are described in Appendix[G](https://arxiv.org/html/2609.13997#A7)\)\. Drop any problem on which the teacher itself fails the Math\-Verify check, leaving136136raw rows\.
3. 3\.Sort the surviving problems by combined question\-plus\-trace character length and take the128128shortest\. This length\-controlled selection bounds the maximum prompt length seen at training time and removes a small fraction of pathologically long traces; it is otherwise content\-agnostic\.

The resulting128128\-problem set isUns\-128, the single unsolvable training corpus shared by Mixture, AdaBack, and MFC\.

#### In\-domain unsolvable test set \(→\\toUns\-22\)\.

For evaluation purposes, we additionally extract the2222\-problem subset of OR1\-200 withpass​@​64=0\\mathrm\{pass\}@64=0on Qwen3\-1\.7B, using the same8,1928\{,\}192\-token response cap as the unsolvable filter above\. This subset,Uns\-22, is a strict held\-out evaluation set: it shares no problems with Uns\-128 and is never used for training or hyperparameter selection\.

#### Reuse for Qwen3\-0\.6B\.

Uns\-128 was filtered using Qwen3\-1\.7B’spass​@​64\\mathrm\{pass\}@64but is reused unchanged when training Qwen3\-0\.6B\. Because Qwen3\-0\.6B is a strictly weaker sibling from the same model family and training generation, every problem unsolvable for Qwen3\-1\.7B is at least as hard for Qwen3\-0\.6B\. Sharing the unsolvable corpus across base models keeps the comparison apples\-to\-apples and avoids a confound where the smaller model would otherwise see an easier curriculum\.

#### Data release\.

We release complete records for OR1\-2k, OR1\-200, Uns\-128, and Uns\-22 as a public Hugging Face dataset repository\.999[https://huggingface\.co/datasets/yukangzhu/unlocking\-the\-unsolvable](https://huggingface.co/datasets/yukangzhu/unlocking-the-unsolvable)The four configsor1\_2k,uns128,or1\_200, anduns22can be loaded and used directly\. Teacher traces appear only in the Uns\-128 config\. These files can be used for training or evaluation without mapping their identifiers back to OpenR1\-Math\-220k\.

## Appendix DAdditional Experimental Details

### D\.1Curriculum Metric Definitions

Throughout §[5\.1](https://arxiv.org/html/2609.13997#S5.SS1)we report two curriculum\-level metrics, computed per training step:

- •Unguided training massM0\(t\)M\_\{0\}^\{\(t\)\}\(Eq\. \([4](https://arxiv.org/html/2609.13997#S3.E4)\); logged asfrac\_effective\_zeroorfrac\_at\_zero\): the fraction of in\-batch prompt visits whose discretized hint lengthg⁡\(s,ρ\)=0g\(s,\\rho\)=0\. Higher values indicate that a larger share of training compute is spent on prompts matching the test\-time distribution\.
- •Mean guidance levelmean​\_​rho​\_​used\\mathrm\{mean\\\_rho\\\_used\}: the batch\-averaged value of the guidance levelρ\\rhoactually sampled at each step, i\.e\.1\|ℬ\|​∑\(s,ρ\)∈ℬρ\\frac\{1\}\{\|\\mathcal\{B\}\|\}\\sum\_\{\(s,\\rho\)\\in\\mathcal\{B\}\}\\rho\. Lower values indicate the curriculum is closer to the unguided regime on average\.

### D\.2Qwen3\-0\.6B Curriculum Dynamics

Figure[6](https://arxiv.org/html/2609.13997#A4.F6)replicates the curriculum\-dynamics comparison of Figure[5](https://arxiv.org/html/2609.13997#S5.F5)\(main text, Qwen3\-1\.7B\) on Qwen3\-0\.6B\. The qualitative pattern is identical: MFC’sM0\(t\)M\_\{0\}^\{\(t\)\}continues to climb throughout training while AdaBack plateaus, and MFC’smean​\_​rho​\_​used\\mathrm\{mean\\\_rho\\\_used\}descends well below AdaBack’s\. This confirms that MFC’s structural advantage is not specific to the 1\.7B scale\.

Figure 6:Curriculum dynamics of MFC vs\. AdaBack on Uns\-128 \(Qwen3\-0\.6B\)\. Left: unguided training massM0\(t\)M\_\{0\}^\{\(t\)\}; right: average guidance level\. The qualitative separation is identical to the 1\.7B results \(Figure[5](https://arxiv.org/html/2609.13997#S5.F5)\)\.
### D\.3Training and Validation Reward Dynamics

Figures[7](https://arxiv.org/html/2609.13997#A4.F7)and[8](https://arxiv.org/html/2609.13997#A4.F8)show the training reward and validation reward across all four methods on both base models\.

#### Training reward \(Figure[7](https://arxiv.org/html/2609.13997#A4.F7)\)\.

An important caveat: training reward is*not*directly comparable across methods\. The curriculum mechanisms \(Mixture, AdaBack, MFC\) present each sample at a guidance level selected by their respective scheduling rules; higherρ\\rhoyields easier effective prompts and thus higher expected reward\. GRPO, by contrast, trains on mixed\-difficulty data without guidance\. Consequently, the absolute level and the trajectory of training reward reflect the effective difficulty of the curriculum at each step, not the model’s test\-time capability\.

#### Validation reward \(Figure[8](https://arxiv.org/html/2609.13997#A4.F8)\)\.

Validation is always performed on OR1\-200 without any teacher guidance \(ρ=0\\rho=0\), making it a fair apples\-to\-apples comparison of unguided capability at each checkpoint\. All four methods converge to broadly similar validation reward by step1,2001\{,\}200, but the curriculum methods reach this level faster \(fewer wasted early steps\) and MFC shows the steepest early rise, consistent with its higherM0\(t\)M\_\{0\}^\{\(t\)\}allocating more on\-target gradient compute from the start\.

Figure 7:Training reward \(mean per batch\) across1,2001\{,\}200steps for all four methods on both base models\. Note that training reward is not directly comparable across methods because curriculum mechanisms present samples at different effective difficulty levels; see text for interpretation\.Figure 8:Validation reward on OR1\-200 \(no guidance,ρ=0\\rho=0\) evaluated every1010steps\. This is the fair comparison of unguided capability across methods\. All methods converge by step1,2001\{,\}200; the curriculum methods \(especially MFC\) rise faster in the first∼\\sim400 steps\.

## Appendix ESFT Baseline Details

This appendix provides full details for the SFT baseline of §[5\.2](https://arxiv.org/html/2609.13997#S5.SS2)\.

#### Data construction\.

Each of the128128Uns\-128 samples is formatted as a single\-turn model conversation: the system and user messages are identical to theρ=0\\rho=0prompt template used during curriculum RL evaluation, with no hints or guidance of any kind; the assistant message contains the full teacher reasoning wrapped in<think\>…\\ldots</think\>tags followed by\\boxed\{teacher\_answer\}, exactly matching the generation format trained by curriculum RL\. Theteacher\_answerfield is recovered from the original training\-pool metadata to ensure complete fidelity to the teacher model’s own output\.

#### Training configuration\.

We train Qwen3\-1\.7B \(base\) via full\-parameter SFT \(no LoRA\) using FSDP on2×2\\timesNVIDIA RTX PRO 6000 GPUs\. Key hyperparameters: global batch size128128\(entire dataset in one step\), learning rate1​e−51\\mathrm\{e\}\{\-5\}with cosine decay and10%10\\%warmup, weight decay0\.010\.01, gradient clipping at1\.01\.0, bf16 mixed precision, max sequence length2,0482\{,\}048tokens,5050epochs \(=50=50gradient steps\)\. The2,0482\{,\}048\-token cap is shorter than the8,1928\{,\}192\-token response cap used by curriculum RL, but exceeds the full prompt\-plus\-trace length of every problem in Uns\-128, so no truncation occurs in the SFT data\. Total wall\-clock time is approximately1515minutes\.

#### Checkpoint selection\.

Figure[9](https://arxiv.org/html/2609.13997#A5.F9)shows the training loss curve\. Since the validation set equals the training set for this128128\-sample corpus, we show only training NLL\. Loss levels off by step∼35\{\\sim\}35–4040\. We select step4040as the evaluation checkpoint\.

Figure 9:SFT training NLL loss over5050epochs on Uns\-128 \(Qwen3\-1\.7B\)\. The dashed line marks the selected checkpoint at step4040where loss has effectively converged\.
#### Per\-benchmark results\.

Table[6](https://arxiv.org/html/2609.13997#A5.T6)reports the full per\-benchmarkpass​@​1\\mathrm\{pass\}@1for the SFT baseline \(Qwen3\-1\.7B, step4040\)\.

Table 6:Per\-benchmarkpass​@​1\\mathrm\{pass\}@1\(%\) for the SFT baseline on Qwen3\-1\.7B\. Compare with Base \(36\.836\.8avg\) and MFC \(47\.347\.3avg\) in Table[2](https://arxiv.org/html/2609.13997#S4.T2)\.

## Appendix FFormal View of the Unguided Training Mass Surrogate

This appendix formalizes the on\-target / augmented gradient decomposition that motivates the unguided training massM0\(t\)M\_\{0\}^\{\(t\)\}of Eq\. \([4](https://arxiv.org/html/2609.13997#S3.E4)\) as the surrogate of choice in §[3\.2](https://arxiv.org/html/2609.13997#S3.SS2), and gives a careful comparison of MFC and AdaBack onM0\(t\)M\_\{0\}^\{\(t\)\}\.

#### First\-order Taylor decomposition\.

Letθ\\thetadenote the current parameters and consider one gradient step against the curriculum objectiveJ𝒞\(t\)J\_\{\\mathcal\{C\}\}^\{\(t\)\}of Eq\. \([2](https://arxiv.org/html/2609.13997#S3.E2)\),θ′=θ\+η∇J𝒞\(t\)\(θ\)\\theta^\{\\prime\}=\\theta\+\\eta\\nabla J\_\{\\mathcal\{C\}\}^\{\(t\)\}\(\\theta\)for a small step sizeη\>0\\eta\>0\. The first\-order Taylor expansion of the test\-time objectiveJ0J\_\{0\}of Eq\. \([3](https://arxiv.org/html/2609.13997#S3.E3)\) gives

J0​\(θ′\)−J0​\(θ\)=\\displaystyle J\_\{0\}\(\\theta^\{\\prime\}\)\-J\_\{0\}\(\\theta\)\\;=η⁡⟨∇J0​\(θ\),∇J𝒞\(t\)​\(θ\)⟩\\displaystyle\\eta\\,\\bigl\\langle\\nabla J\_\{0\}\(\\theta\),\\,\\nabla J\_\{\\mathcal\{C\}\}^\{\(t\)\}\(\\theta\)\\bigr\\rangle\(5\)\+O⁡\(η2\)\.\\displaystyle\+\\;O\(\\eta^\{2\}\)\.Decompose the curriculum gradient on the on\-target \(g=0g=0\) / augmented \(g\>0g\>0\) partition,

∇J𝒞\(t\)​\(θ\)=\\displaystyle\\nabla J\_\{\\mathcal\{C\}\}^\{\(t\)\}\(\\theta\)\\;=M0\(t\)∇Jzero\(θ\)\\displaystyle M\_\{0\}^\{\(t\)\}\\,\\nabla J\_\{\\mathrm\{zero\}\}\(\\theta\)\(6\)\+\(1−M0\(t\)\)∇Jaug\(θ\),\\displaystyle\+\\,\\bigl\(1\-M\_\{0\}^\{\(t\)\}\\bigr\)\\,\\nabla J\_\{\\mathrm\{aug\}\}\(\\theta\),where∇Jzero\\nabla J\_\{\\mathrm\{zero\}\}is the gradient of the conditional curriculum objective restricted to visits withg⁡\(s,ρ\)=0g\(s,\\rho\)=0,∇Jaug\\nabla J\_\{\\mathrm\{aug\}\}the analogous quantity restricted tog\>0g\>0, andM0\(t\)M\_\{0\}^\{\(t\)\}as in Eq\. \([4](https://arxiv.org/html/2609.13997#S3.E4)\)\. Substituting Eq\. \([6](https://arxiv.org/html/2609.13997#A6.E6)\) into Eq\. \([5](https://arxiv.org/html/2609.13997#A6.E5)\),

Δ​J0=\\displaystyle\\Delta J\_\{0\}\\;=η​M0\(t\)​⟨∇J0,∇Jzero⟩\\displaystyle\\eta\\,M\_\{0\}^\{\(t\)\}\\,\\bigl\\langle\\nabla J\_\{0\},\\,\\nabla J\_\{\\mathrm\{zero\}\}\\bigr\\rangle\(7\)\+η⁡\(1−M0\(t\)\)​⟨∇J0,∇Jaug⟩\\displaystyle\+\\,\\eta\\,\\bigl\(1\-M\_\{0\}^\{\(t\)\}\\bigr\)\\,\\bigl\\langle\\nabla J\_\{0\},\\,\\nabla J\_\{\\mathrm\{aug\}\}\\bigr\\rangle\+O⁡\(η2\)\.\\displaystyle\+\\,O\(\\eta^\{2\}\)\.

#### The on\-target term under unbiased sampling\.

Under the assumption that the conditional distribution ofssgiveng⁡\(s,ρ\)=0g\(s,\\rho\)=0matches𝒟\\mathcal\{D\},∇Jzero\\nabla J\_\{\\mathrm\{zero\}\}is an unbiased estimator of∇J0\\nabla J\_\{0\}in expectation, and the first term of Eq\. \([7](https://arxiv.org/html/2609.13997#A6.E7)\) satisfies

𝔼⁡\[⟨∇J0,∇Jzero⟩\]=‖∇J0‖2≥0,\\mathbb\{E\}\\bigl\[\\bigl\\langle\\nabla J\_\{0\},\\,\\nabla J\_\{\\mathrm\{zero\}\}\\bigr\\rangle\\bigr\]\\;=\\;\\\|\\nabla J\_\{0\}\\\|^\{2\}\\;\\geq\\;0,contributing a definite\-sign improvement toJ0J\_\{0\}at an expected rate proportional toM0\(t\)M\_\{0\}^\{\(t\)\}\. This condition holds for Mixture’sρ=0\\rho=0row and for the portion of AdaBack visits induced by its fixedpzerop\_\{\\mathrm\{zero\}\}valve\. It does not hold in general for MFC, since its lattice\-floor probability1/\(gcurr​\(s\)\+1\)1/\(g\_\{\\mathrm\{curr\}\}\(s\)\+1\)depends on the per\-sample frontier\. For MFC,M0\(t\)M\_\{0\}^\{\(t\)\}remains the weight on unguided visits with the test\-time input format, but those visits follow a frontier\-reweighted distribution over problems rather than𝒟\\mathcal\{D\}\.

#### The augmented term has unconstrained sign\.

The second term of Eq\. \([7](https://arxiv.org/html/2609.13997#A6.E7)\) is mediated entirely through parameter sharing across distinct prompts\. Its sign and magnitude depend on whether the gradient on augmented promptssρs\_\{\\rho\}is well\-aligned with the gradient on the unguided promptss, which is not in general guaranteed and can be:

- •positive \(helpful generalization, when reasoning learned under partial guidance transfers to unaided solving\);
- •zero \(neutral, when the model learns hint\-conditioned behaviors that do not transfer\);
- •negative \(negative transfer, when the model overfits to teacher\-style scaffolding or prompt artifacts that hurt unguided generation\)\.

Cauchy–Schwarz bounds the magnitude by‖∇Jaug‖​‖∇J0‖\\\|\\nabla J\_\{\\mathrm\{aug\}\}\\\|\\,\\\|\\nabla J\_\{0\}\\\|, but does not pin the sign\.

#### Implication for surrogate design\.

Eq\. \([7](https://arxiv.org/html/2609.13997#A6.E7)\) suggests that a curriculum cannot expect to improveJ0J\_\{0\}purely through augmented visits, but neither can it improveJ0J\_\{0\}by visiting onlyρ=0\\rho=0when those visits produce uniform failure \(in which case‖∇Jzero‖\\\|\\nabla J\_\{\\mathrm\{zero\}\}\\\|itself collapses by the same advantage\-normalization mechanism that makes unsolvable problems inert under standard GRPO\)\. Both factors of the on\-target term,M0\(t\)M\_\{0\}^\{\(t\)\}and‖∇Jzero‖\\\|\\nabla J\_\{\\mathrm\{zero\}\}\\\|, must be advanced jointly\. MFC’s per\-sample frontier monotonically raises the probability of an unguided visit after a successful visit at a lowergg\(Algorithm[1](https://arxiv.org/html/2609.13997#alg1)\)\. The valueg=0g=0remains in the sampling support from the first visit, although early unguided groups may still yield vanishing gradients before unaided success becomes likely\.

#### Comparison with AdaBack onM0\(t\)M\_\{0\}^\{\(t\)\}\.

Conditional on the per\-sample state, MFC’s on\-target visit probability is exactly1/\(gcurr​\(s\)\+1\)1/\(g\_\{\\mathrm\{curr\}\}\(s\)\+1\)by the discrete\-uniform draw on\{0,1,…,gcurr​\(s\)\}\\\{0,1,\\ldots,g\_\{\\mathrm\{curr\}\}\(s\)\\\}; sincegcurrg\_\{\\mathrm\{curr\}\}is monotone non\-increasing under MFC’s success\-only update, this conditional probability is monotone non\-decreasing across visits toss\. AdaBack admits no such monotone bound: under the same discrete\-uniform sampling \(§[3\.1](https://arxiv.org/html/2609.13997#S3.SS1)\), its per\-sample on\-target visit probability ispzero\+\(1−pzero\)1\[gmin\(s\)=0\]/\(gmax\(s\)−gmin\(s\)\+1\)p\_\{\\mathrm\{zero\}\}\+\(1\-p\_\{\\mathrm\{zero\}\}\)\\,\\mathbf\{1\}\[g\_\{\\mathrm\{min\}\}\(s\)=0\]\\,/\\,\(g\_\{\\mathrm\{max\}\}\(s\)\-g\_\{\\mathrm\{min\}\}\(s\)\+1\)because the valve overrides the uniform draw\. Whengmin​\(s\)\>0g\_\{\\mathrm\{min\}\}\(s\)\>0, the uniform\-draw contribution vanishes and the on\-target probability equalspzerop\_\{\\mathrm\{zero\}\}\. A subsequent success at anyggresetsgmin​\(s\)g\_\{\\mathrm\{min\}\}\(s\)to00, restoring uniform mass atg=0g=0\. In our discrete unsolvable\-only runs this mass is intermittent, and AdaBack’s aggregateM0\(t\)M\_\{0\}^\{\(t\)\}rises before plateauing around0\.400\.40\(Figure[3](https://arxiv.org/html/2609.13997#S3.F3)\)\.

## Appendix GPrompt Templates

This section provides the prompt templates used for teacher trace generation and hint\-based guidance throughout the paper\.

### G\.1Teacher Trace Generation

To obtain structured reasoning traces for each unsolvable problem, we prompt DeepSeek\-V3\.2 with the following system prompt\. The output is required to consist of discrete<step\>tags wrapped inside a<solution\>block, followed by a single<answer\>tag\. We parse the output by extracting the content of each<step\>tag into an ordered list of reasoning steps\.

System Prompt for Teacher Trace Generation\# \[Persona Definition\]You are an expert mathematics teacher preparing educational material for students\. Your goal is to explain the solution to a complex math problem in a way that is clear, logical, and easy to follow\. Imagine your student is intelligent but may miss intermediate steps if you are not explicit\.\# \[Strict Formatting Rules & Few\-Shot Example\]Your entire response MUST be contained within a single<solution\>tag\. You MUST reason step by step\. You MUST wrap each logical reasoning step in a<step\>\.\.\.</step\>tag\. You MUST wrap ONLY the final answer in an<answer\>\.\.\.</answer\>tag\. Do not include any additional explanations, comments, or text inside the<answer\>tag\. Do not output any text outside these tags\.Example:User:The real roots of the equationsx5\+x\+1=0x^\{5\}\+x\+1=0andx\+x5\+1=0x\+\\sqrt\[5\]\{x\}\+1=0areα,β\\alpha,\\betarespectively, thenα\+β\\alpha\+\\betaequals?Assistant: <solution\> <step\>Let’s define a function based on the first equation:f⁡\(x\)=x5\+x\+1f\(x\)=x^\{5\}\+x\+1\. Its derivative isf′​\(x\)=5​x4\+1f^\{\\prime\}\(x\)=5x^\{4\}\+1, which is always positive\.</step\> <step\>The first equation has real rootα\\alpha, sof⁡\(α\)=0f\(\\alpha\)=0\.</step\> <step\>From the second equation,β5=−β−1\\sqrt\[5\]\{\\beta\}=\-\\beta\-1\.</step\> <step\>Evaluatingf⁡\(−1−β\)=\(−1−β\)5\+\(−1−β\)\+1=β−1−β\+1=0f\(\-1\-\\beta\)=\(\-1\-\\beta\)^\{5\}\+\(\-1\-\\beta\)\+1=\\beta\-1\-\\beta\+1=0\.</step\> <step\>Sinceffis strictly increasing,α=−1−β\\alpha=\-1\-\\beta, soα\+β=−1\\alpha\+\\beta=\-1\.</step\> <answer\>−1\-1</answer\> </solution\>

### G\.2Hint\-Based Guidance Prompt

During training, a guidance levelρ\\rhodetermines how many of the teacher’s extracted steps are revealed to the student \(Eq\.[1](https://arxiv.org/html/2609.13997#S2.E1)\)\. The selected steps are injected into the user message as a hint\. The full prompt seen by the student model consists of a system message and a user message constructed as follows:

System Prompt \(Student Model\)You are an expert mathematician with strong problem\-solving skills\. Think step by step\.

User Message Template \(with guidance\)\{question\}Below are some initial reasoning steps that may help you:\{guidance\_steps\}Please solve the problem step by step\. You should use the provided steps as a reference, but do NOT just copy them\. Instead, reconstruct the complete reasoning process in your own words, starting from the beginning, and continue the reasoning to find the final answer\.Use this format: <think\> \[Your reasoning process here\] </think\> \\boxed\{answer\}

Whenρ=0\\rho=0\(no guidance\), the hint block is omitted and the user message reduces to the question followed by the standard solving instruction\.

## Appendix HBroader Impacts

This work advances methods for improving mathematical reasoning in LLMs via reinforcement learning\. The potential negative societal impacts align with those generally associated with LLM reasoning technologies: stronger reasoning capabilities could in principle be misused to generate more convincing misinformation or to automate harmful planning tasks\. However, our contribution is methodological \(a curriculum scheduling algorithm\) and operates on publicly available base models; it does not introduce new capabilities beyond what larger\-scale training on public data already enables\. On the positive side, our data\-efficiency finding \(∼16×\{\\sim\}16\\timesreduction in required training data\) may help democratize access to reasoning improvements for resource\-constrained research groups\.

## Appendix ILicenses of Artifacts Used

#### Models\.

Qwen3\-1\.7B and Qwen3\-0\.6B \(base models\) are downloaded from their official HuggingFace repositories, where they are released under the Apache 2\.0 license\. DeepSeek\-V3\.2 \(teacher model\) was accessed exclusively through the OpenRouter API for the purpose of generating teacher reasoning traces; use of the API and the resulting outputs is governed by the provider’s terms of service\. We do not redistribute the model weights or any portion of them\.

#### Training data\.

OpenR1\-Math\-220k is released under the Apache 2\.0 license\.

#### Evaluation benchmarks\.

MATH\-500 is a subset of the MATH dataset; the source dataset and its splits are released under the MIT license\. SciBench is released under the MIT license\. GPQA\-Diamond is a subset of GPQA, whose source dataset is released under CC\-BY 4\.0; the specific redistribution we use does not specify a separate license\. AMC23 and AIME24/25/26 are mathematics competition problems copyrighted by the Mathematical Association of America \(MAA\); we access them through community\-redistributed datasets on HuggingFace and use them solely for non\-commercial research evaluation, consistent with standard practice in the LLM reasoning literature\.

#### Infrastructure\.

verl \(training framework\), vLLM \(inference engine\), and Math\-Verify \(answer verification\) are all released under the Apache 2\.0 license\.

All artifacts are used in accordance with their stated terms\. We release the training and in\-domain evaluation datasets used in this work, together with the Uns\-128 teacher traces, in a public Hugging Face dataset repository\.101010[https://huggingface\.co/datasets/yukangzhu/unlocking\-the\-unsolvable](https://huggingface.co/datasets/yukangzhu/unlocking-the-unsolvable)The repository documents the source of each record, the construction procedure, and the model used to generate the traces\. The released bundle is distributed under the Apache 2\.0 license, matching the Apache 2\.0 license of its OpenR1\-Math\-220k source; we apply the same license to our own rights, if any, in the DeepSeek\-V3\.2 teacher traces, whose generation remains governed by the provider’s terms of service\. Because the upstream problems are drawn from olympiad, competition, and examination sources, a dataset\-level license does not by itself clear rights in every underlying problem, and users remain responsible for those rights\. We do not release model weights\.

## Appendix JUse of AI Assistants

Large language models \(Claude, ChatGPT\) were used to assist with grammar checking, paraphrasing, and improving clarity of the authors’ original content\.