MagicBeanStalk：通过递归数据合成摆脱 RLVR 零优势陷阱

# Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis
Source: [https://arxiv.org/html/2610.12659](https://arxiv.org/html/2610.12659)
Ximing Lu††thanks:Core contributors\.Aviral KumarAffiliation:Carnegie Mellon UniversityAffiliation:NVIDIAEmail:[dacunamarrer@nvidia\.com](mailto:[email protected])Jan KautzAffiliation:NVIDIAYejin ChoiAffiliation:Stanford UniversityDavid AcunaAffiliation:NVIDIA

###### Abstract

Data\-centric scaling of RLVR has largely focused on collecting or generating verifiable training problems\. However, corpus size alone does not determine whether the dataset provides a useful learning signal for a target policy\. To guide the synthesis of effective RLVR data, we analyze the RLVR objective and experimentally characterize dataset effectiveness as a function of its problems, the target policy, and the training rollout budget\. Under this characterization, we show that a large open\-source math corpus pooled from widely used RLVR datasets falls almost entirely into a*zero\-advantage trap*: for a fixed target policy and rollout budget, roughly88%of its problems are solved in every rollout or failed in every rollout\. Filtering concentrates what remains, but it can only select, discarding the hard problems\. Effective synthetic problems should therefore require the reasoning operations and compositions required to solve those hard problems\. Yet their difficulty must be calibrated, so that some target\-policy rollouts succeed, some fail, and at least one complete correct solution fits within the training budget\. Prompting a teacher to vary hard problems directly does not achieve this\. Guided by these empirical design principles, we introduceMagicBeanStalk, a top\-down decomposition and bottom\-up recursive data\-synthesis algorithm that grows seed problems into families of substantively new, policy\- and budget\-calibrated training problems while preserving the seeds’ reasoning blueprints\. Across 8 math reasoning benchmarks, continual RLVR on MagicBeanStalk problems improves gpt\-oss\-120b from 61\.2 to69\.8\(\+8\.6\) and DeepSeek v4 Flash from 72\.4 to77\.9\(\+5\.4\) on a 16K\-token budget\. Training on the full 78K source corpus improves gpt\-oss\-120b by only \+1\.1 and its best filtered subset by \+5\.8\. In finance, we synthesize over 14K verifiable training problems from only 340 solutions and improve Qwen3 by\+6\.9and\+2\.4points on SecQue and FinanceBench, respectively\.

Figure 1:MagicBeanStalk synthesizes RLVR data from hard seeds to escape the zero\-advantage trap\.\(a\)For a fixed policyπ\\piand budgetLL, all\-failure and all\-success rollout groups yield zero advantage;≈88%\\approx 88\\%of a7878K math corpus falls in this trap forgpt\-oss\-120bat 32K\.\(b\)Three empirical principles: hard seeds specify what to synthesize; synthetic problems must be substantively new yet preserve the seed’s reasoning operations and their composition; and usefulness is relative toπ\\piandLL\.\(c\)MBS applies them in order: the seed’s verified teacher trace is decomposed top\-down into subproblems that help a student solve their parent \(1\); harder new leaves are synthesized and recursively composed bottom\-up into parallel treesp′p^\{\\prime\},p′′p^\{\\prime\\prime\}\(2\); synthetic root problems with mixed target\-policy rollouts and a complete correct response withinLLare kept \(3\)\.## 1Introduction

Reinforcement learning with verifiable rewards \(RLVR\) has become key for improving reasoning models, but as policies become stronger, scaling RLVR increasingly becomes a data problem requiring a continual supply of verifiable data\. Existing efforts have expanded this supply by collecting human\-authored problems for math, coding and other domains\([Chen et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib28);[Luo et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib29);[Albalak et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib30);[Cui et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib31);[Lu et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib32)\), as well as by developing procedural environments that can generate multiple problems per environment\([Stojanovski et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib27);[Lacombe et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib26);[Zeng et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib34)\)\. However, sourcing high\-quality human problems become increasingly hard as we run out of internet data\. And procedural environments are typically hand\-engineered, limited to a set of domains, and often produce problems that vary surface details within a fixed reasoning structure\. As a result, recent approaches synthesize surrogate tasks from Internet data\([Lu et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib22)\), yet more data does not necessarily ensure the problems provide useful learning signal for a target policy and training budget\.

In this work, we return to the binary\-reward RLVR objective and characterize dataset effectiveness as a function of three interacting components: the problems it contains, the target policy, and the training rollout budget\. Using this characterization and a set of controlled experiments, we design key principles to guide the synthesis of effective RLVR data\. We demonstrate that more data does not necessarily lead to better downstream performance, and a large corpus can provide little learning signal when most problems fall into what we call thezero\-advantage trap\(i\.e\. problems are either too easy or too hard for the target policy under the training budget\)\. Specifically, in a corpus of approximately78​K78Kopen\-source math problems, roughly88%88\\%fall into this trap forgpt\-oss\-120bat a 32K rollout budget\. Filtering alone is limited to selection, and disregards the hard problems that sits at the frontier of the target policy and expose the reasoning capabilities the policy lacks\.

Our key insight is to repurpose the hard problems as seeds from which to grow substantively new problems whose solutions require the same reasoning operations, but whose difficulty is calibrated to produce mixed rollout outcomes and admit at least one complete correct solution within the training budget\. However, the obvious baseline—directly prompting a strong teacher to produce variations of the hard seed— performs no better than training on the best problems already present in the corpus\.

This motivates a more structured construction,MagicBeanStalk\(MBS\)\. MBS is a top\-down decomposition and bottom\-up recursive synthesis method that grows a small collection of seeds into families of policy\- and budget\-calibrated RLVR problems\. MBS decomposes each seed into a validated tree of simpler subproblems, synthesizes substantively new leaf problems, and recombines them while preserving the seed’s reasoning blueprint \(Figure[1](https://arxiv.org/html/2610.12659#S0.F1)\)\. MBS escapes the trap where direct synthesis does not: from hard seeds on whichgpt\-oss\-120bscores0/80/8, it produces in\-budget Goldilocks problems for74\.7%\\mathbf\{74\.7\\%\}of seeds versus43\.8%43\.8\\%for direct synthesis\. In a matched training control on a shared seed population, MBS reaches 69\.2, versus 63\.9 for direct synthesis and 65\.3 for training on the seeds themselves\. Ongpt\-oss\-120b, continual RLVR raises the avg across 8 math reasoning benchmarks \(AIME 24/25, AMC, HMMT\-F/N, Minerva, Olympiad, Omega\-500\) from61\.261\.2to69\.8\\mathbf\{69\.8\}\(\+8\.6\\mathbf\{\+8\.6\}points\) at a 16K\-token budget\. The same data applied to DeepSeek v4 Flash raises its average from72\.472\.4to77\.9\\mathbf\{77\.9\}\(\+5\.4\\mathbf\{\+5\.4\}points\) under the same budget\. Further, MBS gains persist at 24K and 32K and on MathArenaApex\([Dekoninck et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib17)\)they exceed the best baseline’s gain at every budget\. Beyond math, we also apply MBS to finance, a specialized domain where annotation is expensive and verifiable RL data is scarce\. Starting from only340340expert\-curated seed questions, MBS synthesizes over14,00014\{,\}000grounded financial reasoning problems\. Continual RLVR on this data improves Qwen3\-4B by\+6\.9\\mathbf\{\+6\.9\}pass@1 on SecQue and\+2\.4\\mathbf\{\+2\.4\}on FinanceBench\.

![Refer to caption](https://arxiv.org/html/2610.12659v1/sections/figures/fig_insight_1_difficulty_only_v3.png)

![Refer to caption](https://arxiv.org/html/2610.12659v1/sections/figures/fig_insight_2_relative_difficulty_v3.png)

Figure 2:Effective RLVR data synthesis must account for the problem, target policy, and response budget\.Difficulty distributions \(left\):Empirical success\-rate distributions from 8gpt\-oss\-120brollouts per problem in math and finance\. We denote problems with0/80/8,11–7/87/8, and8/88/8correct responses as*hard*,*Goldilocks*, and*stale/mastered*, respectively; each bar height is the fraction of problems in that row\. Roughly𝟖𝟖%\\mathbf\{88\\%\}of the corpus falls in the zero\-advantage trapproviding no learning signal, whereas MagicBeanStalk places58\.2%58\.2\\%of its math problems and56\.7%56\.7\\%of its finance problems in the Goldilocks band\.Policy and budget dependence \(right\):The two panels on \(b\) use the problem subset that GPT\-5\-nano does not solve in every rollout excluding most of the stale mass on the left\. Holding the problems and 32K response limit fixed, changing the target policy reclassifies32\.3%32\.3\\%of problems\. Holdinggpt\-oss\-120bfixed, increasing the response limit from 16K to 64K reduces the fraction of hard problems from33\.1%33\.1\\%to10\.6%10\.6\\%\. Black curve reports avg accuracy\. Difficulty depends on the problem, policy, and rollout budget\.
## 2Design Principles for Effective RLVR Data Synthesis

To derive design principles for useful RLVR data synthesis, we first characterize when a problem provides learning signal for a target policy under a fixed training rollout budget\. Consider a verified problem\(p,a∗\)\(p,a^\{\*\}\), a target policyπ\\mathcal\{\\pi\}, and a rollout budgetLL\. We sampleNNresponses\(τi,ai\)∼π\(⋅∣p;L\)\(\\tau\_\{i\},a\_\{i\}\)\\sim\\mathcal\{\\pi\}\(\\cdot\\mid p;L\), whereτi\\tau\_\{i\}is the reasoning trace andaia\_\{i\}is the final answer\. We score each answer with a binary verifier,ri=R⁡\(p,ai\)∈\{0,1\}r\_\{i\}=R\(p,a\_\{i\}\)\\in\\\{0,1\\\}, using the reference answera∗a^\{\*\}\. The empirical success rate and the corresponding group\-relative advantage estimate are:

r¯π,L​\(p\)=1N​∑j=1Nrj,A^i​\(p\)=ri​\(p\)−r¯π,L​\(p\)\.\\vskip\-8\.53581pt\\bar\{r\}\_\{\\pi,L\}\(p\)=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}r\_\{j\},\\qquad\\hat\{A\}\_\{i\}\(p\)=r\_\{i\}\(p\)\-\\bar\{r\}\_\{\\pi,L\}\(p\)\.\\vskip\-8\.53581pt\(1\)
In a mixed group,A^i\\hat\{A\}\_\{i\}is positive for a successful response and negative for an unsuccessful response\. When all responses receive the same reward,A^i=0\\hat\{A\}\_\{i\}=0for every response, and the sampled group provides no within\-problem advantage signal\. We refer to this failure mode as thezero\-advantage trap\. These are empirical finite\-sample classifications depending on rollout counts, they do not establish absolute inability/mastery\. Further, we call problems satisfying0<r¯π,L​\(p\)<10<\\bar\{r\}\_\{\\pi,L\}\(p\)<1the policy’s*Goldilocks band*\. This implies some rollouts succeed and others fail, yielding a nonzero group\-relative learning signal\. The definition is explicitly relative to the target policyπ\\piand the rollout budgetLLand counts\.

The zero advantage trap and the goldilock bands\. More data does not always means better performance\.We characterize the learning signal available in a large corpus of open\-source mathematical reasoning problems using the success rate\. The corpus combines problem from DAPO\-Math\-17K\([Yu et al\., 2025b](https://arxiv.org/html/2610.12659#bib.bib36)\), Skywork\-OR1\-RL\-Data\([He et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib5)\), Turing Open\-RL\([Patil et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib21)\), and a subset of StackMathQA[Zhang \(2024\)](https://arxiv.org/html/2610.12659#bib.bib20), and contains approximately78,00078\{,\}000problems\. For each problem, we sampleN=8N=8responses fromgpt\-oss\-120busing rollout budgetLL\. Despite the size of the corpus, only12\.2%12\.2\\%of its problems fall within the observed Goldilocks band \(L=32​KL=32K\)\. As shown in Figure[2](https://arxiv.org/html/2610.12659#S1.F2), most problems are solved in all rollouts and therefore have empirical success rater¯π,L​\(p\)=1\\bar\{r\}\_\{\\pi,L\}\(p\)=1while a6\.3%6\.3\\%are not solved in any rollout\. Therefore, roughly𝟖𝟖%\\mathbf\{88\\%\}of the corpus falls in the zero\-advantage trap\. They contribute no group\-relative advantage\. A matched downstream control experiment further shows that the Goldilocks subset and a stricter subset that additionally demonstrates in\-budget solvability outperform a proxy preserving the full corpus distribution, despite using an order of magnitude fewer problems\. \(Figure[3](https://arxiv.org/html/2610.12659#S2.F3)\(b\)\) Thus, more RLVR data does not necessarily means better downstream performance\. \(Appendix[B\.1](https://arxiv.org/html/2610.12659#A2.SS1)for more\)

![Refer to caption](https://arxiv.org/html/2610.12659v1/sections/figures/three_runs_avg4_std.png)

\(a\)Downstream training dynamics

![Refer to caption](https://arxiv.org/html/2610.12659v1/sections/figures/size_efficiency_scatter_bars.png)

\(b\)A large corpus can provide little signal

Figure 3:MBS improves beyond seed training and direct synthesis\. Filtering shows that more data alone is not enough but discards the hard problems the target policy is not able to solve yet\.\(a\)Starting from the same base policy, we compare one epoch of training on fixed seed problems and the same amount of qualified seed\-synthesis and MBS problems\. All derived from the same fixed seed population\. Seed synthesis does not surpass seed\-only training, whereas MBS provides a clear additional gain\. In this control experiment, the seed population mixes goldilocks and hard problems, so that seed\-only training is a meaningful baseline\.\(b\)Holding the base policy, training recipe, and evaluation fixed, we train on a6,9226\{,\}922\-problem proxy that preserves the source mixture’s difficulty distribution, on its862862\-problem Goldilocks subset, and on the696696\-problem subset that additionally demonstrates in\-budget solvability\. Both filtered subsets outperform the full proxy while using fewer problems\.Difficulty is relative to the policy and training budget\. Crucially, whether a problem provides useful signal depends on both the target policy and the rollout budget\. Holding the problems and a 32K response limit fixed, changing the target policy reclassifies32\.3%32\.3\\%of problems among the zero\-success, Goldilocks, and fully solved regions\. Holding the policy fixed, increasing the response limit from 16K to 64K reduces the fraction of apparently hard problems from33\.1%33\.1\\%to10\.6%10\.6\\%\(Figure[2](https://arxiv.org/html/2610.12659#S1.F2)\)\. A truncated response therefore does not establish that the policy lacks the required reasoning capability\. It establishes only that no complete solution was produced within budget\. We say that a problem has demonstrated*in\-budget solvability*if at least one correct response terminates normally \(i\.e\. without reaching the rollout limit\)\. Difficulty is consequently a property of the tuple\. Therefore, generated problems should be calibrated to the target policy and training rollout budget, producing both successful and unsuccessful rollouts while admitting at least one complete correct response within budget\. Filtering alone, however, can only select from problems a corpus already contains\. Filtering cannot create new problems targeting capabilities absent from the filtered subset\.

Hard problems are specifications for what data to synthesize\.Hard problems play a paradoxical role in RLVR\. Problems with zero observed success provide no group\-relative advantage in the sampled group, yet their verified solutions identify reasoning operations and compositions to target through synthesis\. We refer to the essential reasoning operations exposed by a verified solution and how those operations are composed as*reasoning blueprints*\. We define a*verified hard problem*as a solvable problem for whichr¯π,L​\(p\)=0\\bar\{r\}\_\{\\mathcal\{\\pi\},L\}\(p\)=0and the sampled responses terminate normally\. The termination requirement ensures that the observed failures are not artifact of the rollout budget\. To test whether these blueprints provide actionable specifications, we study9898hard math seeds and obtain verified traces from a pool of 5 frontier models\. For each seed, we generate one simpler subproblem instantiating some of its blueprint operations and provide its verified solution as a single in\-context example\. We verify that it does not reveal the original solution\. This intervention enables the target policy to solve at least one rollout for78/9878/98seeds \(79\.6%79\.6\\%\) and raises mean accuracy on the original problems from0%0\\%to46\.9%46\.9\\%\. Thus, hard problems are not merely training examples\. Their verified solutions provide specifications of what skill a synthetic problem should target\. Moreover, among the4343seeds with at least two verified blueprints obtained with different teachers,90\.7%90\.7\\%produce the same blueprint or a compatible variant\. This further suggests that the blueprint primarily reflects the problem’s structure rather than a particular teacher style\. \(See Appendix[B\.3](https://arxiv.org/html/2610.12659#A2.SS3)for more\)

![Refer to caption](https://arxiv.org/html/2610.12659v1/sections/figures/vs_seed_synthesis.png)

\(a\)Blueprint preservation and novelty

![Refer to caption](https://arxiv.org/html/2610.12659v1/sections/figures/fig_insight_factory_yield_curve_math_finance_combined_v4.png)

\(b\)Qualified problems per source seed

Figure 4:Direct prompting a teacher to generate variants is limited by both the problems it constructs and the number of useful candidates it yields\.Left:We evaluate300300same\-seed pairs in math and100100in finance for each approach\. A judge scores each generated problem for reasoning\-blueprint preservation and problem novelty\. Blueprint preservation measures if the solutions to the synthetic problem requires the seed’s essential reasoning operations and dependencies\. Novelty measures if the synthetic problem is new rather than a paraphrase, contextual rewrite, or numerical substitution\. Higher is better\.Right:Cumulative qualified problems as source seeds are added, \(both methods evaluated on the same seeds\)\. Qualification requires11–7/87/8correct and at least one correct response that terminates normally \. In finance, we use same qualification\. Finance curves are observed through 340 seeds and linearly projected thereafter\. MBS produces more qualified descendants per seed while more often preserving the blueprint in a new problem\.Direct prompting a teacher to generate variations is not enough\.Filtering can only select from problems a corpus already contains while hard seeds specify the reasoning the policy is missing and left out during filtering\. A simple way to synthesize problem starting with hard seeds is to provide the problem, reasoning trajectory, and its verified solution to a strong teacher \. Then perform rejection sampling on quality control, goldilocks and in\-budget solvability\. This baseline is straightforward and we experimentally observed it often retains the mathematical techniques used by the seed\. In a controlled set of experiments, however, we observedseed synthesis provides no clear gain beyond the seeds\.Figure[3](https://arxiv.org/html/2610.12659#S2.F3)\(a\) shows the training curves\. We experimentally found two main reasons for this\.\(1\) seed synthesis preserves techniques but often makes shallow variants\.We study this behavior with a blinded judge study over300300generated problems in math and100100in finance\. Figure[4](https://arxiv.org/html/2610.12659#S2.F4)\(left\) shows that seed synthesis often retains recognizable reasoning techniques, particularly in math, but frequently does so through a shallow variation of the original problem rather than a substantively new instantiation\. And\(2\) post\-hoc filtering calibrates difficulty but cannot repair the candidate pool\.Filtering is subtractive and cannot turn a shallow variant into a substantively new problem, nor produce a candidate for a seed that yielded none\. Seed synthesis consequently returns below one qualified problem per seed \(Figure[4](https://arxiv.org/html/2610.12659#S2.F4), right\)\. We study the full yield and seed\-coverage comparison in Section[4\.1](https://arxiv.org/html/2610.12659#S4.SS1)\. Also, see appendix[C](https://arxiv.org/html/2610.12659#A3)for more details\.

Takeaway\.Three principles follow\. First, hard problems are specifications\. Their verified solutions name the reasoning operations and skills synthesis should target\. Second, synthesis must do what filtering cannot and should be complementary to dynamic sampling\. Synthetic problems should be substantively new and preserve the hard seed’s reasoning operations and their composition\. Third, usefulness is relative\. A problem carries signal for a given target policy and rollout budget, so generated problems must be calibrated to both\. They should yield mixed target\-policy rollouts with a complete correct response within the training budget\.

## 3MagicBeanStalk: Effective Data Synthesis from Hard Seeds

We now introduce MagicBeanStalk \(MBS\), a data synthesis algorithm motivated by the findings in in[2](https://arxiv.org/html/2610.12659#S2)\. Each stage in the following subsections is inspired by those principles\.

### 3\.1Preliminaries and Notation

We assume access to a collection ofteachermodels𝒯:=\{T1,T2,…,TM\}\\mathcal\{T\}:=\\\{T\_\{1\},T\_\{2\},\\ldots,T\_\{M\}\\\}\. In our setting, these are strong reasoning LLMs used to solve, decompose, synthesize, and verify problems\. Additionally, we define an ordered pool ofstudentmodels𝒮:=\{S1,S2,…,SJ\}\\mathcal\{S\}:=\\\{S\_\{1\},S\_\{2\},\\ldots,S\_\{J\}\\\}, ranked from strongest \(S1S\_\{1\}\) to weakest \(SJS\_\{J\}\), and a target policyπ\\mathcal\{\\pi\}\. In our setting,π\\mathcal\{\\pi\}is part of the student pool\. A problem is a tuple\(p,a∗\)\(p,a^\{\*\}\)whereppis a question anda∗∈𝒜a^\{\*\}\\in\\mathcal\{A\}is its ground\-truth answer\. In document\-grounded settings \(e\.g\. finance\), we let thep=\(q,d\)p=\(q,d\)whereddis a document snippet that contains the information required to answerqq\.Decomposition tree\.Adecomposition tree𝒢p:=⟨p,τ,a∗,𝒞⟩\\mathcal\{G\}\_\{p\}:=\\langle p,\\tau,a^\{\*\},\\mathcal\{C\}\\rangleis rooted at\(p,a∗\)\(p,a^\{\*\}\)with its verified teacher traceτ\\tau, and recursively contains a set ofKKchildren𝒞=\{𝒢p1,…,𝒢pK\}\\mathcal\{C\}=\\\{\\mathcal\{G\}\_\{p\_\{1\}\},\\ldots,\\mathcal\{G\}\_\{p\_\{K\}\}\\\}, each itself a tree on an easier subproblem\. A nodennwith𝒞n=∅\\mathcal\{C\}\_\{n\}=\\emptysetis aleaf\.

### 3\.2Rejection\-Sample Teacher Traces

Given a set of verified problems\(p,a∗\)\(p,a^\{\*\}\), we use a collection of teachers to sample reasoning traces\. Formally, for each problempp, we iterate through teachersT∈𝒯T\\in\\mathcal\{T\}in order and, from each, draw up toNtN\_\{t\}reasoning traces and answers\(τ\(i\),a\)∼T⁡\(p\)\(\\tau^\{\(i\)\},a\)\\sim T\(p\)withi∈\{1,…,Nt\}i\\in\\\{1,\\ldots,N\_\{t\}\\\}\. A trace isverifiedwhen its final answer matches the GT\. We accept the first trace satisfyingverified⁡\(τ,a∗\)=1\\mathrm\{verified\}\(\\tau,a^\{\*\}\)=1and store it as the root’s verified trace in𝒢p\\mathcal\{G\}\_\{p\}\. Here,verified\\mathrm\{verified\}represents a verification function either answer matching or an LLM\-as\-a\-judge\. Problems for which no trace verifies after\|𝒯\|⋅Nt\|\\mathcal\{T\}\|\\cdot N\_\{t\}attempts are discarded\. A collection of teacher reduces the number of discarded problems due to solvability\. Section[B\.3](https://arxiv.org/html/2610.12659#A2.SS3)motivates why multiples teacher are beneficial for solution coverage\.

### 3\.3Top\-Down Problem Decomposition

We treat a verified\(p,τ,a∗\)\(p,\\tau,a^\{\*\}\)as the root of a decomposition tree𝒢p\\mathcal\{G\}\_\{p\}\. The teacher is then asked to identify the key skills and knowledge required to solvepp, and emitKKeasier self\-contained subproblems that collectively teach those skills, each with its own answer and reasoning trace:\{\(pk,τk,ak∗\)\}k=1K∼T⁡\(p,τ,a∗\)\.\\\{\(p\_\{k\},\\tau\_\{k\},a\_\{k\}^\{\*\}\)\\\}\_\{k=1\}^\{K\}\\;\\sim\\;T\(p,\\tau,a^\{\*\}\)\.Each subproblem is itself rejection\-sampled and verified as in the previous stage\. Each verified subproblem becomes a child node𝒢pk\\mathcal\{G\}\_\{p\_\{k\}\}that is in turn used as a parent in the recursive decomposition\. In our experimentsK≤3K\\leq 3\.

Does the subproblem help a student solve the harder problem?At each node, we evaluate the student pool from weakest to strongest and select the strongest evaluated student with zero empirical accuracy as the evaluated student\. We then run an*effectivity test*\. Specifically, each evaluated student is given theKKsubproblems and their traces as in\-context worked examples and re\-tested on the parent\. Let𝒮n⊆𝒮\\mathcal\{S\}\_\{n\}\\subseteq\\mathcal\{S\}denote the students evaluated at nodenn\. A decomposition iseffectivewhen conditioning raises accuracy for at least one student:∃S∈𝒮n:acccond​\(S,p∣\{pk,τk,ak∗\}k\)\>acc⁡\(S,p\)\.\\exists S\\in\\mathcal\{S\}\_\{n\}:\\quad\\mathrm\{acc\}^\{\\mathrm\{cond\}\}\(S,p\\mid\\\{p\_\{k\},\\tau\_\{k\},a^\{\*\}\_\{k\}\\\}\_\{k\}\)\\;\>\\;\\mathrm\{acc\}\(S,p\)\.Hereacc⁡\(S,p\)\\mathrm\{acc\}\(S,p\)is the fraction ofNNsampled solutions whose answer matchesa∗a^\{\*\}\. We keep effective decompositions and recurse on each child\. If the student pool does not show an improvement, we re\-prompt the teacher for a fresh decomposition with the previous failed attempts as negative context, up toNdN\_\{d\}retries\. Nodes whose decompositions remain ineffective are dropped\. In our experimentsN=8N=8andNd=3N\_\{d\}=3, and we recurse up to depthD=3D=3\. The effectivity test is motivated by the second design principle \(See[2](https://arxiv.org/html/2610.12659#S2)and[B\.3](https://arxiv.org/html/2610.12659#A2.SS3)\)\.

### 3\.4Bottom\-up synthesis of calibrated problems

At this stage, every retained seed has a decomposition tree𝒢p\\mathcal\{G\}\_\{p\}rooted at\(p,τ,a∗\)\(p,\\tau,a^\{\*\}\), with verified subproblems at its descendant nodes\. We now walk this tree bottom\-up to synthesize a parallel*synthetic*tree of new problems that share the original’s reasoning structure\. At each leaf, we explicitly ask the teacher to generate harder variants of the leaf problem, and recursively generate a synthetic parent problem by combining the retained synthetic children\. Formally, a synthetic parent\(p~,τ~,a~∗\)\(\\widetilde\{p\},\\widetilde\{\\tau\},\\widetilde\{a\}^\{\*\}\)is obtained recursively from itsKKsynthetic child problems:\(p~,τ~,a~∗\)∼T⁡\(\{\(p~k,τ~k,a~k∗\)\}k=1K,ℬ⁡\(𝒢n\)\)\.\(\\widetilde\{p\},\\widetilde\{\\tau\},\\widetilde\{a\}^\{\*\}\)\\;\\sim\\;T\\bigl\(\\\{\(\\widetilde\{p\}\_\{k\},\\widetilde\{\\tau\}\_\{k\},\\widetilde\{a\}\_\{k\}^\{\*\}\)\\\}\_\{k=1\}^\{K\}\\,;\\,\\mathcal\{B\}\(\\mathcal\{G\}\_\{n\}\)\\bigr\)\.whereℬ⁡\(𝒢n\)\\mathcal\{B\}\(\\mathcal\{G\}\_\{n\}\)denotes the original parent–children configuration at the current node\. The original\(parent,children\)\(\\text\{parent\},\\text\{children\}\)tree is provided to the teacher as a demonstration pattern that the combination should mirror in logical structure and difficulty\. Each candidate synthetic parent is itself rejection\-sampled and verified, and is discarded when its accuracy on the weakest studentSJS\_\{J\}exceedsαparent\\alpha\_\{\\text\{parent\}\}\. Propagating upward, every node ends up with synthetic versions, yielding a complete synthetic tree that parallels the original\. In our experimentsαparent=0\.625\\alpha\_\{\\text\{parent\}\}=0\.625and up toNc=10N\_\{c\}=10child\-combination trials are attempted per parent\.

Leaf synthesis\.For each leaf\(pℓ,τℓ,aℓ∗\)\(p\_\{\\ell\},\\tau\_\{\\ell\},a\_\{\\ell\}^\{\*\}\), the teacher samplesNℓN\_\{\\ell\}harder variants of the question:\{\(p~ℓ\(s\),τ~ℓ\(s\),a~ℓ∗\(s\)\)\}s=1Nℓ∼T⁡\(pℓ,τℓ,aℓ∗\)\.\\\{\(\\widetilde\{p\}\_\{\\ell\}^\{\(s\)\},\\widetilde\{\\tau\}\_\{\\ell\}^\{\(s\)\},\\widetilde\{a\}\_\{\\ell\}^\{\*\(s\)\}\)\\\}\_\{s=1\}^\{N\_\{\\ell\}\}\\;\\sim\\;T\(p\_\{\\ell\},\\tau\_\{\\ell\},a\_\{\\ell\}^\{\*\}\)\.We experimentally noted that explicitly requiring*harder*variants at the leaves was beneficial\. Without this constraint, the teacher tends to default to easy paraphrases that lose both difficulty and diversity of the synthetic tree\. Variants whose accuracy on the weakest studentSJS\_\{J\}exceedsαleaf\\alpha\_\{\\text\{leaf\}\}are discarded, and we retry up toRRrounds with the remaining teachers\. In our experiments,R=3R=3, andαleaf=0\.8\\alpha\_\{\\text\{leaf\}\}=0\.8\.

Grounding on new documents\.In document\-grounded scenarios \(e\.g\., finance\), each synthetic must remain answerable from a document\. We retrieve top\-KdK\_\{d\}unseen documents from a corpus by embedding similarity to the seed’s document summary, then ask the teacher to extract from each a relevant snippetd′d^\{\\prime\}together with an*entity of interest*e′e^\{\\prime\}\. The synthetic tree is then re\-grounded ond′d^\{\\prime\}with every node threaded arounde′e^\{\\prime\}\. In our experimentsKd∈\{35,75\}K\_\{d\}\\in\\\{35,75\\\}documents per seed\. Finally, we retain only synthetic problems that fall in the Goldilocks band of the target policy and that satisfy in\-budget solvability\.

Table 1:Main math results at 16K output tokens\.Pass@1 accuracy \(%\) across AIME 24/25, AMC, HMMT\-F/N, Minerva, Olympiad, and Omega\-500\. Reference API models use 4 samples per problem and are included as context;gpt\-oss\-120buses 16 and DeepSeek v4 Flash uses one\. AVG is the mean across all 8 benchmarks andΔ\\Deltais the within\-base\-model improvement\.ModelAIME24AIME25AMCHMMT\-FHMMT\-NMinervaOlympiadOmegaAVGΔ\\Delta*Reference API models*Claude Opus 4\.698\.368\.397\.052\.561\.760\.988\.645\.271\.6—Gemini 3\.1 Pro82\.563\.393\.459\.269\.260\.876\.448\.269\.1—Gemini 3 Flash66\.748\.386\.150\.055\.059\.472\.635\.559\.2—GPT\-5\.262\.550\.079\.228\.345\.853\.367\.936\.553\.0—MiniMax M2\.756\.760\.084\.945\.048\.354\.164\.739\.056\.6—*Base model:gpt\-oss\-120b*gpt\-oss\-120b\(base\)70\.063\.389\.040\.050\.055\.171\.450\.861\.2—\+ RL \(full source mixture\)80\.056\.793\.943\.346\.752\.972\.352\.462\.3\+1\.1\+ RL \(Golden Goose\([Lu et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib22)\)\)80\.060\.091\.543\.350\.054\.872\.053\.263\.1\+1\.9\+ RL \(in\-budget Goldilocks source subset\)79\.274\.693\.749\.257\.754\.573\.653\.567\.0\+5\.8\+ RL \(Seed synthesis Goldilocks, synthetic\)73\.373\.192\.951\.260\.054\.776\.153\.266\.8\+5\.6\+ RL \(MagicBeanStalk, synthetic\)80\.278\.893\.457\.559\.055\.578\.255\.969\.8\+8\.6*Base model: DeepSeek v4 Flash*DeepSeek v4 Flash \(base\)76\.780\.092\.773\.376\.757\.075\.447\.672\.4—\+ RL \(MagicBeanStalk, same data\)93\.383\.392\.783\.383\.359\.277\.550\.277\.9\+5\.4

## 4Experimental Results

In this section, we first quantify qualified yield, blueprint preservation, data quality, and downstream performance relative to the source seeds and seed synthesis in a control experiment\. We then evaluate continual RLVR across 8 math benchmarks, transfer to a different target policy, performance beyond the training response budget, and document\-grounded finance\. Unless specifically mentioned, for most experiments involving training, we use as 16K rollout budget due to computational constrains\.

### 4\.1From Zero\-Advantage Seeds to Effective RLVR Data

Qualified synthesis yield from zero\-signal seeds\.We randomly sample approximately1,2501\{,\}250source seeds on whichgpt\-oss\-120bobtained0/80/8correct rollouts at 16K\. From the same seeds, MBS produces3,3703\{,\}370descendants and seed synthesis produces2,4942\{,\}494\. A descendant is qualified if it lies in the Goldilocks band and has at least one correct rollout that terminates normally within 16K\. MBS yields qualified descendants from74\.7%74\.7\\%of the seeds, compared with43\.8%43\.8\\%under seed synthesis; its candidate\-level qualified rate is also higher \(42\.8%42\.8\\%versus31\.7%31\.7\\%; Table[9](https://arxiv.org/html/2610.12659#A3.T9)\)\. Our algorithm notably converts more zero\-signal hard seeds into usable training data\.

Blueprint preservation and novelty\.For each method, GPT\-5\.6 scores300300math and100100finance seed–candidate pairs from 1 to 5 for blueprint preservation and substantive novelty\. In math, our method preserves the blueprint comparably to seed synthesis while producing more novel problems\. Concretely,19\.7%19\.7\\%of MBS pairs score at least 4 on both criteria, compared with3\.0%3\.0\\%for seed synthesis\. In finance, the corresponding rates are86\.0%86\.0\\%and17\.0%17\.0\\%\(Figure[4](https://arxiv.org/html/2610.12659#S2.F4), left\)\.

Data quality analysis\.Three frontier models \(GPT\-5\.6, Opus 4\.8 and Gemini 3\.1 Pro\) assess whether each of150150math and5050finance randomly sampled problems are well\-posed and its GT answer is correct\. For finance,we also include the source document\. All three judges reject only8\.0%8\.0\\%of the math answers and none of the finance answers \(Table[4](https://arxiv.org/html/2610.12659#S4.T4)\)\. We report all agreement metrics but emphasize the usage of unanimous rejection\. Failure to verify an answer may also reflect the judge’s limitations on difficult problems\. Our hard\-seed control experiment \([B\.3](https://arxiv.org/html/2610.12659#A2.SS3)\) shows this\.

Downstream performance analysis\.Starting from the samegpt\-oss\-120bcheckpoint, we analyze the training dynamics for one epoch on data derived from the same fixed seed population, and report the average over the 4 core benchmarks \(AIME 24/25, HMMT\-F/N\)\. The average is65\.365\.3for source\-seed training,63\.963\.9for Goldilocks\-filtered seed synthesis, and69\.269\.2for MBS \(Figure[3](https://arxiv.org/html/2610.12659#S2.F3), left\)\. Seed synthesis does not improve over the source seeds, whereas MBS improves over both\. This is a separate control run and is no the full\-budget results in Table[1](https://arxiv.org/html/2610.12659#S3.T1)\.

Performance on the original hard seeds after synthetic training\.We reevaluate approximately1,2501\{,\}250source seeds on which the initialgpt\-oss\-120bsample obtained0/80/8at 16K\. A fresh base sample recovers9\.5%9\.5\\%of these seeds, compared with23\.2%23\.2\\%after seed synthesis training and25\.0%25\.0\\%after MagicBeanStalk training\. MagicBeanStalk also yields840840correct rollouts and masters1414seeds, compared with694694and66under seed synthesis \(Table[9](https://arxiv.org/html/2610.12659#A3.T9)\)\. Both methods transfer learning back to their source seeds, while MBS produces more correct rollouts and more fully mastered seeds\.

Table 2:Inference\-budget evaluation of 16K\-trained checkpoints\.Parentheses report the difference from the base policy at the same inference budget\.*Top:*average pass@1 across the 8 math benchmarks, average over 3 samples\.*Bottom:*pass@1 on MathArena Apex\-Shortlist \(4747problems\), average over 16 samples per problem, evaluated from the 16K training budget upward\.
### 4\.2Continual RLVR on Math

Training data and baselines\.We draw math seeds from an approximately7878K\-problem open\-source mixture of DAPO\-Math\-17K\([Yu et al\., 2025b](https://arxiv.org/html/2610.12659#bib.bib36)\), Skywork\-OR1\-RL\-Data\([He et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib5)\), Turing Open\-RL\([Patil et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib21)\), and a subset of StackMathQA\([Zhang, 2024](https://arxiv.org/html/2610.12659#bib.bib20)\)\. We compare MBS with the full source mixture, its in\-budget Goldilocks subset, Goldilocks\-filtered seed synthesis, and Golden Goose\([Lu et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib22)\)\. Our work focus is on the effectiveness of the synthesized data rather than to minimize synthesis cost or algorithm hyperparams\. Therefore, training set sizes are each subset Goldilocks counts\(Fig[2](https://arxiv.org/html/2610.12659#S1.F2)a\) refiltered to match 16K training rollouts \. RL implementation and hypers are fixed\. We stop when a decline is not recovered within 10 subsequent iterations on the same held\-out synthetic subset\.

Training setup\.We traingpt\-oss\-120bwith GRPO on 2 nodes \(16 H100 GPUs\), using LoRA \(rank 32\), a constant lr of4×10−54\\times 10^\{\-5\}, and a 16K response limit\. Each step samples 16 responses for each of 32 prompts , with an optimization batch size of 32\. DeepSeek v4 Flash uses full\-model GRPO on 10 nodes \(80 H100 GPUs\), with a10−610^\{\-6\}lr and 8 responses under the same 16K limit\. We use miles with Megatron\-LM training and SGLang inference engine\. To reject sample correct solutions, we use a pool of teachers including: Gemini 3 Flash, Claude Opus 4\.6, DeepSeek V4 Pro, GPT 5\.2 and Kimi K2\.6\. We use the same pool for both seed synthesis and MBS\.

Evaluation\.We report pass@1 with a 16K limit on the complete 8\-benchmark suite: AIME 2024/2025, AMC, HMMT February and November, Minerva, Olympiad, and Omega\-500\.gpt\-oss\-120buses 16 rollouts per problem\. API rows use 4\. Answers are checked using normalized string matching , numeric equivalence and GPT\-5\.2 as a judge\.

Main result\.Continual RLVR on MBS data raises thegpt\-oss\-120baverage from61\.261\.2to69\.869\.8\(\+8\.6\+8\.6points\) and improves most benchmarks \(Table[1](https://arxiv.org/html/2610.12659#S3.T1)\)\. Training on the full source mixture reaches62\.362\.3and on Golden Goose63\.163\.1, while the in\-budget Goldilocks source subset and seed synthesis reach67\.067\.0and66\.866\.8\. We use the released Golden Goose corpus as\-is, without regenerating or re\-filtering it forgpt\-oss\-120b, so this row is a published\-data reference rather than a re\-run of that method under our target policy\.Transfer to a different target policy\.We train DeepSeek v4 Flash on the same MBS data, without regeneration or target\-specific re\-filtering, and evaluate with 1 rollout per problem at 16K\. Its 8\-benchmark average rises from72\.472\.4to77\.977\.9\(\+5\.4\+5\.4points\. Table[1](https://arxiv.org/html/2610.12659#S3.T1)\)\.

Table 3:Continual RLVR on MagicBeanStalk financial data\.Pass@1 and pass@5 \(%\) on SecQue and FinanceBench using 8 rollouts and an 8K maximum response length\. MBS improves on both benchmarks\.
Table 4:Judge\-assessed quality of our data\.Three frontier judges classify synthetic problem–answer as correct, incorrect, or uncertain\.

Does the improvement persist beyond the training response budget?In this experiment, we hold each checkpoint and the evaluation protocol fixed and vary only the inference response length\. All 3 trained policies improve over the base policy at 16K\. At 24K and 32K, however, source\-seed training falls below the base policy and seed synthesis converges to it, whereas MBS remains above base by1\.01\.0and1\.51\.5points, respectively \(Table[2](https://arxiv.org/html/2610.12659#S4.T2), top\)\. Next, we evaluate the same checkpoints at the same budgets on a substantially harder set known to benefit from larger responses\. MathArena Apex\-Shortlist\([Dekoninck et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib17)\)contains4747competition problems\. MBS improves over base by11\.011\.0,13\.313\.3, and10\.910\.9points at 16K, 24K, and 32K \(Table[2](https://arxiv.org/html/2610.12659#S4.T2), bottom\)\.

Table 5:Qualified yield from hard seeds\.We synthesize descendants from approximately1,2501\{,\}250seeds on whichgpt\-oss\-120bobtained0/80/8correct rollouts at 16K\. A qualified descendant is Goldilocks and admits a normally terminated correct rollout; seed conversion measures whether a seed yields at least one such descendant\. MBS improves both candidate\-level yield and seed\-level conversion, turning substantially more hard seeds into usable RLVR data\.
Table 6:Recovery on the original hard seeds\.Evaluation of the models trained with seed synthesis and MBS on approx1,2501\{,\}250source seeds that received0/80/8at 16K\. Recovery requires at lest one correct rollout; correct rollouts aggregate all attempts\. Both strategies transfer to the original hard seeds\. This is expected for seed synthesis as variations effectively augments the seed problem neighborhood\. MBS preserves transfer, master more source seeds, and generalizes better\.

### 4\.3MagicBeanStalk for Finance Data

Training data\.Starting from340340expert\-collected questions grounded in 17 corporate credit agreements, MBS generates approximately2525K document\-grounded candidates from the same seeds\. MBS yields14\.214\.2K Goldilocks problems \(Figures[2](https://arxiv.org/html/2610.12659#S1.F2)\.

Training setup\.We use GRPO in veRL to train Qwen3\-4B\-Instruct on the MBS corpus with an 8K response limit\. We use a batch size of 512 prompts, 16 responses per prompt, and a lr of10−610^\{\-6\}\. Two additional nodes host Qwen3\-30B\-A3B\-Instruct\-2507, which assigns binary rewards by comparing each extracted final answer with the reference under a finance\-aware equivalence rubric\.Evaluation\.We evaluate the Qwen3\-4B\-Instruct , and MBS\-trained policy on SecQue\([BenYoash et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib18)\)and FinanceBench\([Islam et al\., 2023](https://arxiv.org/html/2610.12659#bib.bib19)\)\. We follow each benchmarks eval protocol, sample 8 responses per problem at temperature1\.01\.0with an 8K response limit and report pass@1 and pass@5 and use GPT\-5\.2 as a judge\.

Main result\.Table[4](https://arxiv.org/html/2610.12659#S4.T4)reports continual RLVR of Qwen3\-4B\-Instruct on the MBS finance corpus\. Training on MBS improves pass@1 by6\.96\.9points on SecQue and2\.42\.4points on FinanceBench\.

## 5Related work

Scaling Training Data for RLVR\.Collected corpora supply verified math and code problems\([Albalak et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib30);[Gao et al\., 2024](https://arxiv.org/html/2610.12659#bib.bib33);[Chen et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib28);[Luo et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib29);[Cui et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib31);[Lu et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib32);[Jain et al\., 2024](https://arxiv.org/html/2610.12659#bib.bib24);[Liu et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib25);[Jung et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib4);[Lin et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib3);[Lu et al\.,](https://arxiv.org/html/2610.12659#bib.bib2)\)\. Others generate tasks in hand\-designed environments\([Lacombe et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib26);[Stojanovski et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib27)\), adapt difficulty to the policy\([Zeng et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib34);[Xu et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib23)\), or mine tasks from Internet text\([Lu et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib22)\)\. Closest to us, a teacher writes new problems from seeds or concepts for SFT\([Toshniwal et al\., 2024](https://arxiv.org/html/2610.12659#bib.bib9);[Zhao et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib16)\), VLMs\([Liao et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib6);[Acuna et al\., 2025b](https://arxiv.org/html/2610.12659#bib.bib7);[Jung et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib8);[Acuna et al\., 2025a](https://arxiv.org/html/2610.12659#bib.bib1)\), and RLVR\([Yu et al\., 2025a](https://arxiv.org/html/2610.12659#bib.bib15);[Liang et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib10);[Liang et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib11)\)\. Rather than collecting problems, hand\-engineering environments, scaling quantity, or asking a teacher to synthesize based on seeds, we analyze what makes a synthetic problem useful for RLVR, then design a top\-down, bottom\-up recursive algorithm that turns hard seeds into new, policy\- and budget\-calibrated problems\.

Algorithmic RL for Hard Problems\.Dynamic sampling discards zero\-advantage groups\([Yu et al\., 2025b](https://arxiv.org/html/2610.12659#bib.bib36)\), BroRL scales rollouts per prompt\([Hu et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib35)\)\. Recent work guide exploration with oracle prefixes, hints, or off\-policy traces\([Qu et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib12);[Li et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib13);[Zhang et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib14)\)\. These methods modify the training algorithm or optimizer\. Our work is complementary and focuses on the data rather than the modifying the algorithm with the same problems\.

## 6Conclusion

We show the value of RLVR data is relative to the problem, target policy, and rollout budget\. We also show hard problems should guide the type of problems to synthesize\. Based on these findings, we design key principles to guide effective data synthesis algorithms\. MBS instantiates these principles by decomposing hard seeds into validated reasoning trees and regrowing them into substantively new, policy\- and budget\-calibrated problems\. Our data improvesgpt\-oss\-120bby\+8\.6\+8\.6and DeepSeek v4 Flash by\+5\.4\+5\.4points across 8 math benchmarks\. In finance,340340expert questions yield over14,00014\{,\}000verifiable training problems and improve Qwen3 by \+6\.9 and \+2\.4 points on SecQue and FinanceBench, respectively\.

### AI use statement

We used generative AI tools to help with language polishing and improve clarity\. We additionally use LLM to generate synthetic datasets as this is the core of our work\. The AI tools did not contribute to the intellectual content\. We reviewed all AI\-assisted edits and take responsibility for the final content of this work\.

### Reproducibility statement

We use publicly available models and benchmarks with dataset splits and training hyperparameters, implementation details, and checkpoint selection disclosed in Section[4](https://arxiv.org/html/2610.12659#S4)\. Prompt templates for synthesis and evaluation appears in Section[E](https://arxiv.org/html/2610.12659#A5)and[F](https://arxiv.org/html/2610.12659#A6)\. Other reproducibility details are in Section[D](https://arxiv.org/html/2610.12659#A4)\. Reported results are averaged over multiple rounds as disclosed in in Section[4](https://arxiv.org/html/2610.12659#S4)\. One caveat may limit exact reproduction of the synthesis algorithm from scratch\. Some of teachers used to reject sample solutions are proprietary models served behind APIs whose updates we do not control, so solutions and problems synthesized using them may differ if APIs change\. We will the release the generated MagicBeanStalk datasets upon acceptance\.

### 6\.1Ethics Statement

We use LLM to generate training problems, so the resulting data may contain errors\. We audit a sample with three additional models and report their agreement and rejection rates, while recognizing that these judgments may also contain mistakes\. Our finance experiments evaluate benchmark performance and are intended to showcase the strength of the proposed algorithms\. Further testing would be needed before relying on synthetic data to train models for financial decisions\.

## References

- Acunaet al\.\(2025a\)D\. Acuna, X\. Lu, J\. Jung, H\. Kim, A\. Kar, S\. Fidler, and Y\. ChoiSocratic\-mcts: test\-time visual reasoning by asking the right questions\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 24158–24171\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Acunaet al\.\(2025b\)D\. Acuna, C\. H\. Yang, Y\. Deng, J\. Jung, X\. Lu, P\. Ammanabrolu, H\. Kim, Y\. Liao, and Y\. ChoiLong grounded thoughts: synthesizing visual problems and reasoning chains at scale\.arXiv preprint arXiv:2511\.05705\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Albalaket al\.\(2025\)A\. Albalak, D\. Phung, N\. Lile, R\. Rafailov, K\. Gandhi, L\. Castricato, A\. Singh, C\. Blagden, V\. Xiang, D\. Mahan, and N\. HaberBig\-math: a large\-scale, high\-quality math dataset for reinforcement learning in language models\.External Links:2502\.17387,[Link](https://arxiv.org/abs/2502.17387)Cited by:[§1](https://arxiv.org/html/2610.12659#S1.p1.1),[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- BenYoashet al\.\(2025\)N\. BenYoash, M\. Brief, O\. Ovadia, G\. Shenderovitz, M\. Mishaeli, R\. Lemberg, and E\. SheetritSecque: a benchmark for evaluating real\-world financial analysis capabilities\.InProceedings of the Fourth Workshop on Generation, Evaluation and Metrics \(GEM2\{\}^\{2\}\),pp\. 212–230\.Cited by:[§4\.3](https://arxiv.org/html/2610.12659#S4.SS3.p2.1)\.
- Chenet al\.\(2025\)Y\. Chen, Z\. Yang, Z\. Liu, C\. Lee, P\. Xu, M\. Shoeybi, B\. Catanzaro, and W\. PingAceReason\-nemotron: advancing math and code reasoning through reinforcement learning\.arXiv preprint arXiv:2505\.16400\.Cited by:[§1](https://arxiv.org/html/2610.12659#S1.p1.1),[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Cuiet al\.\(2025\)G\. Cui, L\. Yuan, Z\. Wang, H\. Wang, W\. Li, B\. He, Y\. Fan, T\. Yu, Q\. Xu, W\. Chen,et al\.Process reinforcement through implicit rewards\.arXiv preprint arXiv:2502\.01456\.Cited by:[§1](https://arxiv.org/html/2610.12659#S1.p1.1),[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Dekonincket al\.\(2026\)J\. Dekoninck, N\. Jovanović, T\. Gehrunger, K\. Rögnvaldsson, I\. Petrov, C\. Sun, and M\. VechevBeyond benchmarks: matharena as an evaluation platform for mathematics with llms\.External Links:2605\.00674,[Link](https://arxiv.org/abs/2605.00674)Cited by:[§1](https://arxiv.org/html/2610.12659#S1.p4.1),[§4\.2](https://arxiv.org/html/2610.12659#S4.SS2.p5.1)\.
- Gaoet al\.\(2024\)B\. Gao, F\. Song, Z\. Yang, Z\. Cai, Y\. Miao, Q\. Dong, L\. Li, C\. Ma, L\. Chen, R\. Xu, Z\. Tang, B\. Wang, D\. Zan, S\. Quan, G\. Zhang, L\. Sha, Y\. Zhang, X\. Ren, T\. Liu, and B\. ChangOmni\-math: a universal olympiad level mathematic benchmark for large language models\.External Links:2410\.07985,[Link](https://arxiv.org/abs/2410.07985)Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Heet al\.\(2025\)J\. He, J\. Liu, C\. Y\. Liu, R\. Yan, C\. Wang, P\. Cheng, X\. Zhang, F\. Zhang, J\. Xu, W\. Shen, S\. Li, L\. Zeng, T\. Wei, C\. Cheng, B\. An, Y\. Liu, and Y\. ZhouSkywork open reasoner 1 technical report\.arXiv preprint arXiv:2505\.22312\.Cited by:[§B\.1](https://arxiv.org/html/2610.12659#A2.SS1.p1.1),[§2](https://arxiv.org/html/2610.12659#S2.p3.1),[§4\.2](https://arxiv.org/html/2610.12659#S4.SS2.p1.1)\.
- Huet al\.\(2025\)J\. Hu, M\. Liu, X\. Lu, F\. Wu, Z\. Harchaoui, S\. Diao, Y\. Choi, P\. Molchanov, J\. Yang, J\. Kautz, and Y\. DongBroRL: scaling reinforcement learning via broadened exploration\.External Links:2510\.01180,[Link](https://arxiv.org/abs/2510.01180)Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p2.1)\.
- Islamet al\.\(2023\)P\. Islam, A\. Kannappan, D\. Kiela, R\. Qian, N\. Scherrer, and B\. VidgenFinancebench: a new benchmark for financial question answering\.arXiv preprint arXiv:2311\.11944\.Cited by:[§4\.3](https://arxiv.org/html/2610.12659#S4.SS3.p2.1)\.
- Jainet al\.\(2024\)N\. Jain, K\. Han, A\. Gu, W\. Li, F\. Yan, T\. Zhang, S\. Wang, A\. Solar\-Lezama, K\. Sen, and I\. StoicaLiveCodeBench: holistic and contamination free evaluation of large language models for code\.External Links:2403\.07974,[Link](https://arxiv.org/abs/2403.07974)Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Junget al\.\(2025\)J\. Jung, S\. Han, X\. Lu, S\. Hallinan, D\. Acuna, S\. Prabhumoye, M\. Patwary, M\. Shoeybi, B\. Catanzaro, and Y\. ChoiPrismatic synthesis: gradient\-based data diversification boosts generalization in llm reasoning\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38, Main Conference,pp\. 90649–90685\.External Links:[Document](https://dx.doi.org/10.52202/085713-3032),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/82a34882f560db982d1257b5af461605-Paper-Conference.pdf)Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Junget al\.\(2026\)J\. Jung, H\. Kim, B\. Cui, X\. Lu, D\. Acuna, P\. Ammanabrolu, and Y\. ChoiDeltaPrompts: escaping the zero\-delta trap in multimodal distillation\.arXiv preprint arXiv:2605\.15532\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Lacombeet al\.\(2025\)V\. Lacombe, V\. Quesnel, and D\. SileoReasoning core: a scalable rl environment for llm symbolic reasoning\.arXiv preprint arXiv:2509\.18083\.Cited by:[§1](https://arxiv.org/html/2610.12659#S1.p1.1),[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Liet al\.\(2026\)J\. Li, H\. Lin, H\. Lu, K\. Wen, Z\. Yang, J\. Gao, Y\. Wu, and J\. ZhangQuesta: expanding reasoning capacity in llms via question augmentation\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 144191–144215\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p2.1)\.
- Lianget al\.\(2025\)X\. Liang, Z\. Li, Y\. Gong, Y\. Wang, H\. Zhang, Y\. Shen, Y\. N\. Wu, and W\. ChenSwS: self\-aware weakness\-driven problem synthesis in reinforcement learning for llm reasoning\.External Links:2506\.08989,[Link](https://arxiv.org/abs/2506.08989)Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Lianget al\.\(2026\)X\. Liang, Z\. Li, Y\. Gong, Y\. Wu, Z\. Guo, W\. Chen,et al\.Beyond pass@ 1: self\-play with variational problem synthesis sustains rlvr\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 79443–79491\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Liaoet al\.\(2025\)Y\. Liao, S\. Elflein, L\. He, L\. Leal\-Taixé, Y\. Choi, S\. Fidler, and D\. AcunaLongperceptualthoughts: distilling system\-2 reasoning for system\-1 perception\.arXiv preprint arXiv:2504\.15362\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Linet al\.\(2026\)Y\. Lin, S\. Tang, B\. Lyu, Z\. Yang, J\. Chung, H\. Zhao, L\. Jiang, Y\. Geng, J\. Ge, J\. Sun,et al\.Goedel\-prover\-v2: scaling formal theorem proving with scaffolded data synthesis and self\-correction\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 11793–11818\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Liuet al\.\(2025\)Y\. Liu, L\. L\. Zhang, Y\. Zhu, B\. Dong, X\. Zhou, N\. Shang, F\. Yang, and M\. YangRStar\-coder: scaling competitive code reasoning with a large\-scale verified dataset\.External Links:2505\.21297,[Link](https://arxiv.org/abs/2505.21297)Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Luet al\.\(2025\)D\. Lu, X\. Tan, R\. Xu, T\. Yao, C\. Qu, W\. Chu, Y\. Xu, and Y\. QiSCP\-116k: a high\-quality problem\-solution dataset and a generalized pipeline for automated extraction in the higher education science domain\.External Links:2501\.15587,[Link](https://arxiv.org/abs/2501.15587)Cited by:[§1](https://arxiv.org/html/2610.12659#S1.p1.1),[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Luet al\.\(2026\)X\. Lu, D\. Acuna, J\. Jung, J\. Hu, D\. Zhang, S\. Diao, Y\. Zou, S\. Zhang, B\. Cui, M\. Liu,et al\.Golden goose: a simple trick to synthesize unlimited rlvr tasks from unverifiable internet text\.arXiv preprint arXiv:2601\.22975\.Cited by:[§1](https://arxiv.org/html/2610.12659#S1.p1.1),[Table 1](https://arxiv.org/html/2610.12659#S3.T1.6.1.1.1.1.1.1.11.1),[§4\.2](https://arxiv.org/html/2610.12659#S4.SS2.p1.1),[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- \[24\]X\. Lu, S\. Han, D\. Acuna, H\. Kim, J\. Jung, S\. Prabhumoye, N\. Muennighoff, M\. Patwary, M\. Shoeybi, B\. Catanzaro,et al\.Retro\-search: exploring untaken paths for deeper and efficient reasoning \(2025\)\.arXiv preprint arXiv:2504\.04383\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Luoet al\.\(2025\)M\. Luo, S\. Tan, J\. Wong, X\. Shi, W\. Y\. Tang, M\. Roongta, C\. Cai, J\. Luo, L\. E\. Li, R\. A\. Popa, and I\. StoicaDeepScaleR: surpassing o1\-preview with a 1\.5b model by scaling rl\.Note:[https://pretty\-radio\-b75\.notion\.site/DeepScaleR\-Surpassing\-O1\-Preview\-with\-a\-1\-5B\-Model\-by\-Scaling\-RL\-19681902c1468005bed8ca303013a4e2](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2)Notion BlogCited by:[§1](https://arxiv.org/html/2610.12659#S1.p1.1),[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Patilet al\.\(2026\)S\. Patil, A\. Lall, M\. Pavlovic, C\. Shukla, S\. Pande, T\. M\. Ukarde, A\. G\. Bertollo, M\. Joshi, and K\. HanOpen\-rl\.External Links:[Link](https://huggingface.co/datasets/TuringEnterprises/Open-RL)Cited by:[§B\.1](https://arxiv.org/html/2610.12659#A2.SS1.p1.1),[§2](https://arxiv.org/html/2610.12659#S2.p3.1),[§4\.2](https://arxiv.org/html/2610.12659#S4.SS2.p1.1)\.
- Quet al\.\(2026\)Y\. Qu, A\. Setlur, V\. Smith, R\. Salakhutdinov, and A\. KumarPope: learning to reason on hard problems via privileged on\-policy exploration\.arXiv preprint arXiv:2601\.18779\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p2.1)\.
- Stojanovskiet al\.\(2025\)Z\. Stojanovski, O\. Stanley, J\. Sharratt, R\. Jones, A\. Adefioye, J\. Kaddour, and A\. KöpfREASONING gym: reasoning environments for reinforcement learning with verifiable rewards\.External Links:2505\.24760,[Link](https://arxiv.org/abs/2505.24760)Cited by:[§1](https://arxiv.org/html/2610.12659#S1.p1.1),[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Toshniwalet al\.\(2024\)S\. Toshniwal, W\. Du, I\. Moshkov, B\. Kisacanin, A\. Ayrapetyan, and I\. GitmanOpenMathInstruct\-2: accelerating ai for math with massive open\-source instruction data\.External Links:2410\.01560,[Link](https://arxiv.org/abs/2410.01560)Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Xuet al\.\(2026\)C\. Xu, C\. Xiao, Z\. Peng, X\. Wang, and Y\. CaoSCALER: synthetic scalable adaptive learning environment for reasoning\.arXiv preprint arXiv:2601\.04809\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Yuet al\.\(2025a\)P\. Yu, J\. Lanchantin, T\. Wang, W\. Yuan, O\. Golovneva, I\. Kulikov, S\. Sukhbaatar, J\. Weston, and J\. XuCot\-self\-instruct: building high\-quality synthetic prompts for reasoning and non\-reasoning tasks\.arXiv preprint arXiv:2507\.23751\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Yuet al\.\(2025b\)Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, W\. Dai, T\. Fan, G\. Liu, L\. Liu, X\. Liu, H\. Lin, Z\. Lin, B\. Ma, G\. Sheng, Y\. Tong, C\. Zhang, M\. Zhang, W\. Zhang, H\. Zhu, J\. Zhu, J\. Chen, J\. Chen, C\. Wang, H\. Yu, Y\. Song, X\. Wei, H\. Zhou, J\. Liu, W\. Ma, Y\. Zhang, L\. Yan, M\. Qiao, Y\. Wu, and M\. WangDAPO: an open\-source llm reinforcement learning system at scale\.arXiv preprint arXiv:2503\.14476\.External Links:[Link](https://arxiv.org/abs/2503.14476)Cited by:[§B\.1](https://arxiv.org/html/2610.12659#A2.SS1.p1.1),[§2](https://arxiv.org/html/2610.12659#S2.p3.1),[§4\.2](https://arxiv.org/html/2610.12659#S4.SS2.p1.1),[§5](https://arxiv.org/html/2610.12659#S5.p2.1)\.
- Zenget al\.\(2025\)Z\. Zeng, H\. Ivison, Y\. Wang, L\. Yuan, S\. S\. Li, Z\. Ye, S\. Li, J\. He, R\. Zhou, T\. Chen, C\. Zhao, Y\. Tsvetkov, S\. S\. Du, N\. Jaques, H\. Peng, P\. W\. Koh, and H\. HajishirziRLVE: scaling up reinforcement learning for language models with adaptive verifiable environments\.ArXivabs/2511\.07317\.External Links:[Link](https://api.semanticscholar.org/CorpusID:282911886)Cited by:[§1](https://arxiv.org/html/2610.12659#S1.p1.1),[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.
- Zhanget al\.\(2026\)X\. Zhang, S\. Wu, Y\. Zhu, H\. Tan, S\. Yu, Z\. He, and J\. JiaScaf\-grpo: scaffolded group relative policy optimization for enhancing llm reasoning\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 131946–131974\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p2.1)\.
- Zhang \(2024\)Y\. ZhangStackMathQA: A Curated Collection of 2 Million Mathematical Questions and Answers Sourced from Stack Exchange\.Technical reportMath\-AI\.Note:[https://stackmathqa\.github\.io/StackMathQA\.pdf](https://stackmathqa.github.io/StackMathQA.pdf)Cited by:[§B\.1](https://arxiv.org/html/2610.12659#A2.SS1.p1.1),[§2](https://arxiv.org/html/2610.12659#S2.p3.1),[§4\.2](https://arxiv.org/html/2610.12659#S4.SS2.p1.1)\.
- Zhaoet al\.\(2025\)X\. Zhao, W\. Wu, J\. Guan, and L\. KongPromptcot: synthesizing olympiad\-level problems for mathematical reasoning in large language models\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 18167–18188\.Cited by:[§5](https://arxiv.org/html/2610.12659#S5.p1.1)\.

## Appendix AAppendix

## Appendix BWhat Makes a Problem Useful for RLVR?

### B\.1A large corpus can provide little learning signal

We next use the empirical success rate to characterize the learning signal available in a large corpus of open\-source mathematical reasoning problems\. The corpus combines problems drawn from DAPO\-Math\-17K\([Yu et al\., 2025b](https://arxiv.org/html/2610.12659#bib.bib36)\), Skywork\-OR1\-RL\-Data\([He et al\., 2025](https://arxiv.org/html/2610.12659#bib.bib5)\), Turing Open\-RL\([Patil et al\., 2026](https://arxiv.org/html/2610.12659#bib.bib21)\), and a subset of StackMathQA[Zhang \(2024\)](https://arxiv.org/html/2610.12659#bib.bib20), and contains approximately78,00078\{,\}000problems\. For each problem, we sampleN=8N=8responses fromgpt\-oss\-120busing rollout budgetLL\. Despite the size of the corpus, only12\.2%12\.2\\%of its problems fall within the observed Goldilocks band \(L=32​KL=32K\)\. As shown in Figure[2](https://arxiv.org/html/2610.12659#S1.F2), most problems are solved in all rollouts and therefore have empirical success rater¯π,L​\(p\)=1\\bar\{r\}\_\{\\pi,L\}\(p\)=1while a6\.3%6\.3\\%are not solved in any rollout\. Therefore, roughly𝟖𝟖%\\mathbf\{88\\%\}of the corpus falls in the zero\-advantage trap\. They contribute no group\-relative advantage\.

We then test whether the empirical Goldilocks criterion identifies problems that are more useful for downstream RLVR\. Starting from the same initialgpt\-oss\-120bpolicy, we compare three computationally tractable training sets\. We first construct a∼\\sim7K\-problem proxy whose proportions of hard, Goldilocks, and stale problems match those of the full open\-source corpus\. From this proxy, we extract the∼\\sim862 Goldilocks problems and a stricter 696\-problem subset that also demonstrates in\-budget solvability under the 16K training rollout limit\. All conditions use the same training recipe and are evaluated after approximately one epoch\. As shown in Figure[3](https://arxiv.org/html/2610.12659#S2.F3)\(b\), both the Goldilocks\-only and the Goldilocks plus in\-budget subsets reach a higher average accuracy than the full\-mixture proxy, while training on an order of magnitude fewer problems\. Thus, Goldilocks filtering concentrates useful RLVR signal\. Filtering, however, can only select\. It retains the small fraction of a corpus that already yields mixed outcomes and discards the rest, including the hard problems on which the policy never succeeds\. Most importantly, filtering cannot produce a problem for a capability the corpus does not already cover\.

### B\.2Difficulty is relative to the policy and training budget

Problem difficulty in RLVR depends on both the target policy and the rollout budget used during training\. We demonstrate each dependence through a controlled experiment\. Letfi∈\{stop,length\}f\_\{i\}\\in\\\{\\texttt\{stop\},\\texttt\{length\}\\\}indicate whether response terminates normally or reaches the rollout limitLL\. We say that a problem has demonstrated*in\-budget solvability*if at least one correct response terminates:

Qπ,L\(p\)=\[∃i:ri=1∧fi=stop\]\.Q\_\{\\pi,L\}\(p\)=\\mathbf\{1\}\\\!\\left\[\\exists i:\\ r\_\{i\}=1\\land f\_\{i\}=\\texttt\{stop\}\\right\]\.\(2\)
#### Dependence on the target policy\.

We first hold the problem set and rollout budget fixed and vary only the policy\. We use the15,47015\{,\}470problems that GPT\-5\-nano does not solve in every rollout, discarding those it has already mastered, and evaluate them also withgpt\-oss\-120busing 8 rollouts per problem and 32K response limit\. Changing only the policy causes nearly one third of the problems \(32\.3%32\.3\\%\) to move between the zero\-success, Goldilocks, and fully solved regions\. Thus relative to a target policy, the same problem can provide different learning signal\.

#### Dependence on the rollout budget\.

We next hold the policy and evaluation set fixed and vary only the maximum response length\. Increasing the rollout budget ofgpt\-oss\-120bfrom 16K to 64K reduces the fraction of hard problems from33\.1%33\.1\\%to10\.6%10\.6\\%and raises the average accuracy on the 8 math benchmarks from61\.2%61\.2\\%to78\.9%78\.9\\%\(Figure[2](https://arxiv.org/html/2610.12659#S1.F2)b\)\. Thus, the same problems appear substantially harder under a smaller rollout budget, even though the policy is unchanged\. Therefore, a truncated response does not establish that the policy lacks the reasoning capability required by the problem\. It only establishes that no complete solution was produced within budget\. We therefore treat a problem as providing a clean in\-budget group\-relative learning signal only when:

0<r¯π,L​\(p\)<1andQπ,L​\(p\)=1\.0<\\bar\{r\}\_\{\\pi,L\}\(p\)<1\\qquad\\text\{and\}\\qquad Q\_\{\\pi,L\}\(p\)=1\.\(3\)
Difficulty is consequently a property of the tuple\(p,π,L\)\(p,\\pi,L\), rather than of the problem alone\.

### B\.3Verified Hard problems specify frontier capabilities

Hard problems play a paradoxical role in RLVR\. Problems with empirical success rate zero provide no direct group\-relative advantage signal\. Yet, these problems may still reveal reasoning capabilities that the target policy lacks\. We argue that a verified solution trace exposing the essential reasoning operations together with how those operations are composed reveals the capability the target policy lack\. We refer to this structure as the problem’s*reasoning blueprint*\. It provides a specification of capabilities at the frontier of the current policy\. We define a*verified hard problem*as a solvable/correct problem for whichr¯π,L​\(p\)=0\\bar\{r\}\_\{\\pi,L\}\(p\)=0and all sampled responses terminate normally:

r¯π,L​\(p\)=0andfi=stop∀i∈\{1,…,N\}\.\\bar\{r\}\_\{\\pi,L\}\(p\)=0\\qquad\\text\{and\}\\qquad f\_\{i\}=\\texttt\{stop\}\\quad\\forall i\\in\\\{1,\\ldots,N\\\}\.\(4\)This termination requirement ensures that the observed failures reflect a reasoning limitation rather than an artifact of the rollout budget\.

#### Extracting reasoning blueprints\.

We study9898math hard seed problems randomly selected from the open corpus\. For each seed, we obtain a verified solution and reasoning trace from a pool of frontier models\. We use a pool of teacher \( Gemini 3 Flash, Claude Opus 4\.6, DeepSeek V4 Pro, GPT 5\.2 and Kimi K2\.6\) as we found with one attempt per model, the strongest individual teacher solved only \(59/98\) seeds\. From each trace, we then extract the essential reasoning operations and how they are composed \(i\.e\. we extract their*reasoning blueprint*\)\.

#### Controlled in\-context experiment\.

For each seed, we generate a simpler subproblem that instantiates some of the operations in the reasoning blueprint and provide its verified solution as a single in\-context example to the target policy\. We use an LLM as a judge to verify the subproblem does not explicitly reveal the solution of the original problem\. For simplicity, and to avoid best\-of\-KKselection effects, we use only the first generated subproblem\. After this intervention, the policy solves at least one rollout for78/9878/98seeds \(79\.6%79\.6\\%\), while mean accuracy on the original hard problems increases from0%0\\%to46\.9%46\.9\\%\. These results suggest that hard problems provide no direct RLVR signal while still serving as useful specifications of missing reasoning capabilities\.

#### Are blueprints consistent across frontier models?

Among the4343seeds with at least two verified blueprints, we find90\.7%90\.7\\%produce the same reasoning blueprint or a compatible variant\. This suggests that the blueprint primarily reflects the structure of the problem rather than the idiosyncrasies of a particular frontier model\.

#### Observation\.

Hard problems are not merely training examples; they can be specifications of capabilities at the frontier of the current policy\. We call a verified problem a*capability\-frontier seed*when the target policy cannot solve it reliably and its solution specifies a reasoning capability, or composition of capabilities, beyond the policy’s current competence\. The in\-context experiment shows one of the many actionable insights: a simpler, subproblem requiring a capability observed in the blueprint of the original problems solution can substantially improves the target policy’s success on the hard problem\.

## Appendix CSeed Synthesis: Why Direct Seed Synthesis Is Not Enough

Filtering can only select from problems a corpus already contains , while hard seeds specify the reasoning the policy is missing and left out during filtering\. The natural next step is therefore to generate new \(easier\) problems directly from those hard seeds\. A simple way to synthesize problem starting with hard seeds is to provide the problem, reasoning trajectory, and its verified solution to a strong teacher , and perform rejection sampling on quality control, goldilocks and in\-budget solvability\. If\(p,τ,a∗\)\(p,\\tau,a^\{\*\}\)denotes the seed problem, reasoning trace, and verified answer, seed synthesis samples:

\(p~,τ~,a~∗\)∼T⁡\(p,τ,a∗\)\.\(\\widetilde\{p\},\\widetilde\{\\tau\},\\widetilde\{a\}^\{\*\}\)\\sim T\(p,\\tau,a^\{\*\}\)\.\(5\)This baseline is straightforward and we experimentally observed it often retains the mathematical techniques used by the seed\. In a controlled set of experiments, however, we observedseed synthesis provides no clear gain beyond the seeds\.Starting from the same base policy and the same fixed seed population, we compare training on the2,1952\{,\}195source seeds with training on approximately the same number of Goldilocks\-filtered synthetic problems derived from those seeds\. Figure[3](https://arxiv.org/html/2610.12659#S2.F3)\(a\) shows the training curves\. The best average accuracy across the 4 core math benchmarks \(AIME 24/25, HMMT\-F/N\) is65\.365\.3for seed\-only training and63\.963\.9for seed synthesis\. Therefore, this simple method produces additional problem instances, but does not provide a clear downstream improvement in unseen benchmarks\. We then investigate why:

#### Seed synthesis preserves techniques but often makes shallow variants\.

We study this behavior with a blinded judge study over300300generated problems in math and100100in finance\. The judge takes in the seed, the reasoning blueprint and one generated problem with its reference solution, but not the synthesis method\. It evaluates whether the generated solution requires the operations and dependencies in the seed blueprint, and whether the problem is substantively new rather than a paraphrase, contextual rewrite, or numerical substitution\. Figure[4](https://arxiv.org/html/2610.12659#S2.F4)shows that seed synthesis often retains recognizable reasoning techniques, particularly in math, but frequently does so through a shallow variation of the original problem rather than a substantively new instantiation \.

#### Post\-hoc filtering calibrates difficulty but cannot repair the candidate pool\.

Filtering is subtractive\. It can calibrate the difficulty and response length of the candidates it retains, but it cannot turn a shallow variant into a substantively new problem, nor produce a candidate for a seed that yielded none\. Seed synthesis consequently returns below one qualified problem per seed \(Figure[4](https://arxiv.org/html/2610.12659#S2.F4), right\)\. We study the full yield and seed\-coverage comparison in Section[4\.1](https://arxiv.org/html/2610.12659#S4.SS1)\.

What changes when direct synthesis receives the MBS inference budget?We compare direct synthesis and MBS on the same 64 seeds\. For the matched control, we repeatedly generate and verify direct\-synthesis candidates from each seed until reaching or exceeding MBS’s per\-seed construction budget\. The resulting control receives 16\.0M estimated construction tokens, compared with MBS’s 14\.3M reference budget\. We evaluate blueprint preservation and novelty using the same protocol as above, and estimate Goldilocks difficulty fromN=8N=8gpt\-oss\-120brollouts with a 16K limit\. As shown in Table[7](https://arxiv.org/html/2610.12659#A3.T7), additional sampling improves direct synthesis’s Goldilocks coverage, but does not close its novelty gap\. MBS yields an all\-three candidate for 28 seeds, compared with 13 under the matched direct\-synthesis control\.

Table 7:Direct synthesis under the MBS construction budget\. Samples is the number of evaluated candidates\. Blueprint and novelty require judge scores of at least three, and Goldilocks denotes11–77correct responses amongN=8N=8gpt\-oss\-120brollouts at 16K\.
#### Observation\.

With todays frontier models, a single call from the hard seed does not produce enough valid, new, blueprint\-preserving problems for subsequent policy\- and budget\-based filtering\.

Together with the preceding analysis of policy\- and budget\-relative learning signal, this result suggests a simple separation of roles\. Hard problems specify the reasoning to target; the target policy and training rollout budget determine which generated problems are useful for RLVR\. Ideally, we should aim to generate substantively new problems that preserve the hard seed’s essential reasoning operations and their composition, then retain those that yield both successful and unsuccessful target\-policy rollouts and at least one correct target\-policy rollout within the training budget\.

Table 8:Qualified yield from hard seeds\.We synthesize descendants from approximately1,2501\{,\}250seeds on whichgpt\-oss\-120bobtained0/80/8correct rollouts at 16K\. A qualified descendant is Goldilocks and admits a normally terminated correct rollout; seed conversion measures whether a seed yields at least one such descendant\. MBS improves both candidate\-level yield and seed\-level conversion, turning substantially more hard seeds into usable RLVR data\.
Table 9:Recovery on the original hard seeds\.Evaluation of the base policy and models trained with seed synthesis and MBS on approx1,2501\{,\}250source seeds that received0/80/8at 16K\. Recovery requires at lest one correct rollout; correct rollouts aggregate all attempts\. Both strategies transfer to the original hard seeds\. This is expected for seed synthesis as variations effectively augments the seed problem neighborhood\. MBS preserves transfer, master more source seeds, and generalizes better to math benchmarks \(see main table\)\.

## Appendix DSupplementary Reproducibility Details

#### Teacher, student, and target\-policy roles\.

The teacher pool solves seed and synthetic problems, proposes decompositions, generates new problems, and supplies candidates for verification\. The ordered student pool𝒮\\mathcal\{S\}is used for the decomposition*effectivity*test\. For an evaluated studentSS, we compare rollouts on a parent problem with rollouts obtained after conditioning on the verified child problems and their worked solutions; a decomposition is effective when this conditioning raises the empirical accuracy of at least one evaluated student\. This test decides whether the proposed children are useful scaffolds\. It does not decide whether a final synthetic problem is in the Goldilocks band\.

The target policyπ\\mathcal\{\\pi\}has a separate role\. Final Goldilocks membership and in\-budget solvability are determined only from target\-policy rollouts at the training response limit\. In the reported setup, a retained problem has11–77correct responses among88target\-policy rollouts and at least one correct response that terminates normally within the response budget\. The target policy can also be a member of𝒮\\mathcal\{S\}, but a positive effectivity test from any student never substitutes for this final target\-policy screen\.

#### Document\-grounded finance\.

For finance, the document is part of the problem:p=\(q,d\)p=\(q,d\)\. It is not auxiliary metadata that may be dropped after synthesis\. Decomposition and verification use the supplied snippet, and a re\-grounded tree uses a new snippet throughout\. Every synthesized question and its answer must be supported by that snippet alone; facts from the seed document cannot be carried into the new problem unless they also appear in the new document\.

## Appendix ECore Prompt Templates

The following are condensed task\-defining templates\. Bracketed text denotes runtime fields\. Production wrappers also enforce serialization and answer formatting, while generic solving, retry, solution\-rewriting, and parsing instructions are omitted here\. A “verified solution” is teacher\-side construction context, not an RLVR training target\.

Problem decomposition\.

*Input:*\[PARENT PROBLEM\],\[VERIFIED TEACHER SOLUTION\],\[GROUND\-TRUTH ANSWER\], and, when applicable,\[DOCUMENT\]\.Identify the key skills and knowledge required by the parent\. Produce two or three easier, independent, self\-contained subproblems that collectively teach those skills\. Give each subproblem a correct worked solution and one concise final answer\. If a document is supplied, every subproblem must be answerable using only that document\.

Conditioned solving for the effectivity test\.

*Input:*\[VERIFIED CHILD PROBLEMS AND WORKED SOLUTIONS\],\[PARENT PROBLEM\], and, when applicable,\[DOCUMENT\]\.Study the solved child problems\. Then solve the parent problem using the skills and techniques demonstrated by those examples\. Return a complete solution and final answer\.

Harder leaf synthesis\.

*Input:*\[LEAF PROBLEM\],\[VERIFIED SOLUTION\], and, when applicable,\[DOCUMENT\]\.Generate diverse variants that test the same core skills but are harder than the leaf and require more reasoning steps\. Each variant must be self\-contained and have exactly one verifiable final answer\. Supply a correct worked solution\. If a document is supplied, use no facts outside it\.

Bottom\-up parent composition\.

*Input:*\[ORIGINAL PARENT\-\-CHILD CONFIGURATION\],\[SYNTHETIC CHILD PROBLEMS, SOLUTIONS, AND ANSWERS\], and, when applicable,\[DOCUMENT\]\.Combine the synthetic children into one new parent problem\. Require the skills from all children and mirror the logical dependency pattern by which the original children form the original parent\. Match the original parent’s difficulty and level of abstraction; do not simplify\. Return one problem, a worked solution that composes the child results, and one final answer\. A document\-grounded parent must be answerable solely from the supplied document\.

Finance document re\-grounding\.

*Input:*\[SEED SUMMARY OR SNIPPET\],\[SEED ENTITY\], and\[NEW DOCUMENT\]\.Locate an analogous entity in the new document and copy a sufficiently complete supporting passage, preserving its numbers, tables, definitions, and surrounding conditions\. Re\-ground the synthetic tree on this new passage and entity\. Preserve the reasoning roles of the seed tree, but import no seed\-document facts\. Every resulting node must be answerable from the copied passage alone\.

Direct seed synthesis\.

*Input:*\[SEED PROBLEM\],\[VERIFIED REASONING TRACE\], and\[VERIFIED SOLUTION AND FINAL ANSWER\]\.Generate diverse, self\-contained problems that test the same underlying skills identified in the original problem and its verified solution\. The problem must be novel and not a simple rewrite\. For each problem, provide a correct worked solution and one final answer\.

## Appendix FJudge\-Based Analysis Protocols

### F\.1Blueprint Preservation and Problem Novelty

The reported study uses same\-seed candidate pairs: one bottom\-up MagicBeanStalk candidate and one direct seed\-synthesis candidate\. The two candidates are selected in the observed Goldilocks band and matched on the recorded target\-policy correct\-rollout count\. The reported samples contain300300math pairs and100100finance pairs\. A fixed reasoning blueprint is first extracted from the seed problem, its verified solution, and its reasoning trace\. Candidate scoring by GPT\-5\.6 is then symmetric and method\-label blinded\. The judge receives the seed, the fixed blueprint, and one generated problem with its reference solution and answer, but no method name\. The two methods are scored in separate calls under the same rubric\. The synthetic MBS tree is not exposed during candidate scoring\.

#### Reasoning preservation \(1–5\)\.

1. 1\.The required reasoning is different; any overlap is only topical or surface\-level\.
2. 2\.One minor or peripheral component overlaps, but the main reasoning is not preserved\.
3. 3\.A meaningful subset of the core operations is required, with an important operation or dependency omitted or changed\.
4. 4\.Most core operations and their principal dependencies are required; additional reasoning is allowed\.
5. 5\.The same core operations and their composition are clearly required by the generated problem\.

#### Problem novelty \(1–5\)\.

1. 1\.The candidate is effectively a duplicate or paraphrase of the seed\.
2. 2\.The candidate mainly substitutes numbers, names, entities, documents, line items, or superficial context\.
3. 3\.The candidate is a meaningful variation of substantially the same setup\.
4. 4\.The candidate is a substantively new setup that still instantiates the seed reasoning\.
5. 5\.The candidate is a clearly new construction or composition that cannot be recovered through superficial substitutions\.

The scores are assigned independently\. Novelty without blueprint preservation does not count as successful transfer\. The joint criterion reported in the paper is met only when both reasoning preservation and problem novelty are at least44for the same candidate\.

## Appendix GMBS Additional Ablation

Which MBS components matter?Table[10](https://arxiv.org/html/2610.12659#A7.T10)evaluates four design choices at their corresponding construction stage\. With the same four teacher calls per seed, the heterogeneous teacher pool covers more seeds than the strongest single teacher\. Teacher verification alone retains all 22 valid decompositions, whereas the effectivity test removes the 16 whose solutions do not improve parent solving\. Explicitly requesting harder leaf variants increases rollout\-qualified coverage in the controlled construction audit, and recursive composition adds qualified problems from 116 source seeds not covered by one\-level composition\.

Table 10:Component analyses\. Teacher coverage uses 100 hard seeds under an equal four\-call budget\. The leaf comparison uses 10 fixed seeds\. Qualified problems produce mixed outcomes overN=8N=8gpt\-oss\-120brollouts and at least one complete correct response within 16K\.