Planned Test-Time Scaling with Coordinated Reasoning Paths

# Planned Test-Time Scaling with Coordinated Reasoning Paths
Source: [https://arxiv.org/html/2609.27374](https://arxiv.org/html/2609.27374)
Langxing BaiAffiliation:University of California, Los AngelesHritik BansalAffiliation:University of California, Los AngelesPo\-Nien KungAffiliation:University of California, Los AngelesShuo LiAffiliation:Amazon[https://github\.com/shirley\-wu/planned\-test\-time\-scaling](https://github.com/shirley-wu/planned-test-time-scaling)Hao LiuAffiliation:Amazon[https://github\.com/shirley\-wu/planned\-test\-time\-scaling](https://github.com/shirley-wu/planned-test-time-scaling)Nanyun PengAffiliation:University of California, Los AngelesKai\-Wei ChangAffiliation:University of California, Los Angeles

###### Abstract

Test\-time scaling with parallel branches is widely adopted to improve performance on challenging reasoning tasks\. The predominant approach, repeated sampling, draws branches independently from a single policy, which can produce redundant attempts and thereby limit the gains from additional inference compute\. To address this limitation, we proposePlanned Test\-Time Scaling\(PTTS\), which replaces independent sampling with a coordinated joint policy: aplannergenerates a solution outline for each branch, steering the branches toward distinct reasoning paths, and anexecutorproduces a full solution conditioned on each outline\. Formally, we show that PTTS strictly generalizes repeated sampling and, in a stylized setting, provably promotes coverage of complementary reasoning modes and yields betterpass​@​k\\mathrm\{pass@\}\{k\}scaling\. We instantiate PTTS on top of strong reasoning models, keeping them fixed as executors while replacing repeated sampling with PTTS inference to further enhance test\-time scaling\. Concretely, we develop two variants:PTTS\-ZSprompts a model to jointly generate outlines for all branches in a single autoregressive pass, whilePTTS\-RLdirectly optimizes the planner against thepass​@​k\\mathrm\{pass@\}\{k\}reward using truncated execution rollouts for efficient training and a sharper reward signal\. Across five mathematical reasoning benchmarks with Qwen3\-1\.7B and 4B, PTTS\-ZS improvespass​@​64\\mathrm\{pass@\}\{64\}over repeated sampling by up to 6\.7 points, while PTTS\-RL further increases the gain to up to 13\.4 points\. Further analysis indicates that broader coverage of distinct reasoning paths contributes to these gains\. Overall, PTTS provides a general framework for improving test\-time scaling by coordinating reasoning branches, with zero\-shot and trainable instantiations that yield substantial performance gains\.

![Refer to caption](https://arxiv.org/html/2609.27374v1/figures/teaser2.png)Figure 1:Left: overview of planned test\-time scaling \(PTTS\)\.Unlike repeated sampling \(RS\) that drawskkindependent solutions from a frozen executor and often repeats the same errors, PTTS uses a planner to generatekkdiverse outlines and guide the executor, where the planner can be further trained via reinforcement learning to directly optimizepass​@​k\\mathrm\{pass@\}\{k\}\.
Right: PTTS performancewith Qwen3\-1\.7B on AIME\-2024–2026\. PTTS matches thepass​@​64\\mathrm\{pass@\}\{64\}of RS with 67% less computation \(efficiency\) and exceeds it by \+8\.8 points atk=64k\\\!=\\\!64\(effectiveness\)\.## 1Introduction

As large language models \(LLMs\) are applied to increasingly challenging reasoning tasks, even strong models often fail to solve a problem reliably in a single attempt\. Test\-time scaling addresses this limitation by allocating additional inference compute\([Wu et al\., 2024](https://arxiv.org/html/2609.27374#bib.bib23);[Snell et al\., 2024](https://arxiv.org/html/2609.27374#bib.bib17);[Brown et al\., 2024](https://arxiv.org/html/2609.27374#bib.bib2)\), most commonly throughrepeated sampling\([Brown et al\., 2024](https://arxiv.org/html/2609.27374#bib.bib2)\): given a budget ofkkattempts, the model independently sampleskkcandidate solutions from a single policy, and performance is measured bypass​@​k\\mathrm\{pass@\}\{k\}, i\.e\., whether at least one attempt is correct\. The benefit of this additional compute therefore depends critically oncoverage: additional attempts are useful only insofar as they explore complementary reasoning paths rather than repeat existing ones\. Under repeated sampling, however, independent samples from the same policy tend to concentrate around high\-probability reasoning modes, producing redundant attempts\. Reinforcement learning \(RL\) can further amplify this effect, improvingpass​@​1\\mathrm\{pass@\}\{1\}while narrowing the policy distribution and degradingpass​@​k\\mathrm\{pass@\}\{k\}\([Yue et al\.,](https://arxiv.org/html/2609.27374#bib.bib28);[Cui et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib4);[Wu et al\., 2026](https://arxiv.org/html/2609.27374#bib.bib22)\)\. Unlocking the full value of parallel compute therefore requires explicit coordination that actively steers branches toward distinct reasoning paths rather than relying on stochasticity to provide coverage\.

Recent work explicitly coordinates parallel attempts by first constructing diverse high\-level reasoning plans through handcrafted pipelines—for example, iterative concept elicitation from an LLM\([Handa et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib7)\)or over\-generation followed by clustering of candidate plans\([Yang et al\., 2025b](https://arxiv.org/html/2609.27374#bib.bib25)\)—and then using these plans to steer different reasoning branches\. Their empirical gains suggest that explicit coordination via a planning stage is a promising direction\. We formalize this shared structure by treating planning as a joint policy across branches, enabling both theoretical analysis and direct optimization\.

We introducePlanned Test\-Time Scaling\(PTTS\), which factorizes the joint policy overkkreasoning branches into a planner\-executor decomposition\. As shown in Figure[1](https://arxiv.org/html/2609.27374#S0.F1), aplannerjointly produceskksolution outlines, one for each branch, and anexecutorthen generates a full solution for each branch conditioned on its outline\. By construction, PTTS strictly generalizes repeated sampling and thus guarantees no worse achievablepass​@​k\\mathrm\{pass@\}\{k\}\. In a stylized setting of reasoning\-mode selection, we further show that a marginal policy optimized forpass​@​1\\mathrm\{pass@\}\{1\}collapses onto a single dominant mode, causing repeated sampling to replay similar failures and bottlenecking test\-time scaling on problems that benefit from complementary reasoning modes\. In contrast, PTTS can be optimized directly forpass​@​k\\mathrm\{pass@\}\{k\}, incentivizing the planner to coordinate branches across complementary reasoning modes and yielding strictly betterpass​@​k\\mathrm\{pass@\}\{k\}scaling\.

We instantiate PTTS in two complementary forms, PTTS\-ZS and PTTS\-RL\. The zero\-shot variant,PTTS\-ZS, uses a strong reasoning model as the executor and abaseLLM as the planner, leveraging the base model’s broader generation diversity\. Unlike prior approaches that rely on multi\-stage pipelines to construct diverse plans, PTTS\-ZS generates allkkoutlines jointly in a single autoregressive pass, yielding a simple yet effective design that naturally enables end\-to\-end optimization\. Building on this formulation,PTTS\-RLfreezes the executor and directly optimizes the planner with thepass​@​k\\mathrm\{pass@\}\{k\}of thekkexecutor solutions as the reward\. To reduce rollout cost and sharpen credit assignment to the planner, we truncate the executor’s reasoning budget during training, preventing the executor from rescuing poor outlines through extended rethinking\.

Evaluation on five mathematical reasoning benchmarks using Qwen3 models\([Yang et al\., 2025a](https://arxiv.org/html/2609.27374#bib.bib24)\)demonstrates substantial gains in both effectiveness and efficiency\. PTTS\-ZS consistently improves over repeated sampling, yielding up to a 6\.7\-point gain inpass​@​64\\mathrm\{pass@\}\{64\}, while optimizing the planner with PTTS\-RL further increases this gain to as much as 13\.4 points\. Notably, PTTS\-RL surpasses repeated sampling using only half the sampling budget, making it more than 2×\\timesas compute\-efficient\. Further analysis points to improved coverage of the reasoning space: PTTS\-ZS substantially increases overall reasoning diversity, while PTTS\-RL further enhances diversity among branches that reach correct answers\. Finally, PTTS\-RL planners can serve as flexible add\-ons that transfer to executors unseen during training, including larger reasoning models \(\+4\.4pass​@​64\\mathrm\{pass@\}\{64\}\) and executors trained with diversity\-oriented RL \(\+4\.4pass​@​64\\mathrm\{pass@\}\{64\}\), demonstrating that PTTS is complementary to executor\-side improvements\.

To summarize, our contributions are as follows: \(1\) We introduce Planned Test\-Time Scaling \(PTTS\), a planner\-executor framework that replaces independent repeated sampling with a coordinated joint policy over multiple reasoning branches\. \(2\) We analyze PTTS theoretically, showing that it strictly generalizes repeated sampling and that directly optimizing forpass​@​k\\mathrm\{pass@\}\{k\}promotes coverage of complementary reasoning modes under fixed inference budgets\. \(3\) We instantiate PTTS in zero\-shot and RL\-based variants, achieving gains of up to 13\.4 points over repeated sampling across five mathematical reasoning benchmarks\.

## 2Related Work

Mode collapse in test\-time scaling\.Recent work increasingly uses test\-time scaling to enhance off\-the\-shelf models by allocating additional inference compute\([Snell et al\., 2024](https://arxiv.org/html/2609.27374#bib.bib17);[Wu et al\., 2024](https://arxiv.org/html/2609.27374#bib.bib23);[Zhang et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib29);[Brown et al\., 2024](https://arxiv.org/html/2609.27374#bib.bib2)\)\. A particularly simple and effective approach is repeated sampling, which independently draws multiple solutions from the same policy\([Brown et al\., 2024](https://arxiv.org/html/2609.27374#bib.bib2)\)\. However, repeated sampling relies on independently sampled branches from the same policy, so reduced diversity makes additional samples redundant, limiting coverage and performance\. This issue is especially pronounced after reinforcement learning with verifiable rewards \(RLVR\)\([Shao et al\., 2024](https://arxiv.org/html/2609.27374#bib.bib15);[Guo et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib6)\), which can concentrate the model on a narrow distribution\([Yue et al\.,](https://arxiv.org/html/2609.27374#bib.bib28);[Cui et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib4);[Wu et al\., 2026](https://arxiv.org/html/2609.27374#bib.bib22)\), thereby harmingpass​@​k\\mathrm\{pass@\}\{k\}\.

Diversity\-aware test\-time scaling\.One line of work addresses this at inference time by coordinating thekkbranches with a planning stage prior to problem solving\. As our primary baseline, Guided Sampling\([Handa et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib7)\)iteratively generates distinct solution concepts to guide the final generation; others sample and cluster high\-level plans\([Yang et al\., 2025b](https://arxiv.org/html/2609.27374#bib.bib25)\)or perturb the query before parallel attempts\([Wang et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib21)\)\. While effective, these methods lack a unified formal framework that can be optimized end\-to\-end, which we aim to provide\. Closely related to our approach,[Kang & Zhang \(2025\)](https://arxiv.org/html/2609.27374#bib.bib10)and[Li et al\. \(2026\)](https://arxiv.org/html/2609.27374#bib.bib12)train planners to propose multiple strategies before solving, but mostly focus on synthetic or coding tasks and lack a joint formulation or theoretical perspective\.

Diversity\-aware RL training\.Another line of work redesigns RL training to promote exploration behavior and diversity in the marginal policy itself\. Prior work directly optimizes estimates ofpass​@​k\\mathrm\{pass@\}\{k\}for each branch\([Tang et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib18);[Chen et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib3);[Walder & Karkhanis, 2025](https://arxiv.org/html/2609.27374#bib.bib19)\), maintains entropy to prevent mode collapse\([Cui et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib4);[Wang et al\., 2026](https://arxiv.org/html/2609.27374#bib.bib20)\), promotes semantic diversity\([Li et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib11);[Yao et al\., 2026](https://arxiv.org/html/2609.27374#bib.bib26)\), adopts risk\-aware training objectives\([Ren et al\.,](https://arxiv.org/html/2609.27374#bib.bib13);[Jiang et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib9)\), or designs structured curricula to encourage exploration\([Setlur et al\., 2026](https://arxiv.org/html/2609.27374#bib.bib14)\)\. These methods focus on improving the marginal policy and are therefore complementary to our proposed test\-time scaling algorithm\. As shown in §[4\.3](https://arxiv.org/html/2609.27374#S4.SS3), our proposed PTTS can be combined with this line of work to further improve performance\.

## 3Planned Test\-Time Scaling

This section presents our planned test\-time scaling \(PTTS\) framework and instantiations\. We begin with the problem formulation \(§[3\.1](https://arxiv.org/html/2609.27374#S3.SS1)\), then provide two theoretical perspectives on why PTTS improvespass​@​k\\mathrm\{pass@\}\{k\}\(§[3\.2](https://arxiv.org/html/2609.27374#S3.SS2)\), and finally describe the concrete algorithm \(§[3\.3](https://arxiv.org/html/2609.27374#S3.SS3)\)\.

### 3\.1From Repeated Sampling to PTTS

Consider a verifiable reasoning problem𝐱\\mathbf\{x\}\{\}, and letr⁡\(𝐲,𝐱\)∈\{0,1\}r\(\\mathbf\{y\}\{\};\\mathbf\{x\}\{\}\)\\in\\\{0,1\\\}indicate whether a candidate solution𝐲\\mathbf\{y\}\{\}is correct\. Given a test\-time budget ofkkattempts, we measure success usingpass​@​k\\mathrm\{pass@\}\{k\}:

pass@k\(𝐲,1…,𝐲;k𝐱\)≜maxi∈\{1,…,k\}r\(𝐲i;𝐱\)\.\\displaystyle\\mathrm\{pass@\}\{k\}\\left\(\\mathbf\{y\}\{\}\_\{1\},\\ldots,\\mathbf\{y\}\{\}\_\{k\}\{\};\\mathbf\{x\}\{\}\\right\)\\triangleq\\max\_\{i\\in\\\{1,\\ldots,k\\\}\}r\(\\mathbf\{y\}\_\{i\};\\mathbf\{x\}\{\}\)\.
![Refer to caption](https://arxiv.org/html/2609.27374v1/figures/mode_collapse_visualization.png)\(a\)Comparing thepass​@​k\\mathrm\{pass@\}\{k\}of PTTS\-RL against repeated sampling based on GRPO \(GRPO\-RS\)\.
![Refer to caption](https://arxiv.org/html/2609.27374v1/figures/diversity.png)\(b\)Inference budgets \(out ofkk\) allocated to the non\-dominant modez2z\_\{2\}, with larger value representing higher diversity\.

Figure 2:Pass​@​k\\mathrm\{Pass@\}\{k\}performance and policy diversity in a synthetic setupwith two data categories and two reasoning modes \(z1z\_\{1\}being dominant andz2z\_\{2\}being non\-dominant\), as detailed in Appendix[A\.1](https://arxiv.org/html/2609.27374#A1.SS1)\.Repeated sampling \(RS\)\.As the predominant test\-time scaling approach, RS draws thekkattempts independently from a single base policyπ\\pi, inducing the joint distribution:

\(𝐲,1…,𝐲\)k∼πrs\(k\)\(⋅∣𝐱\),πrs\(k\)\(𝐲,1…,𝐲∣k𝐱\)≜∏i=1kπ\(𝐲i∣𝐱\)\.\\displaystyle\(\\mathbf\{y\}\{\}\_\{1\},\\ldots,\\mathbf\{y\}\{\}\_\{k\}\{\}\)\\sim\\pi\_\{\\mathrm\{rs\}\}^\{\(k\)\}\(\\cdot\\mid\\mathbf\{x\}\{\}\),\\qquad\\pi\_\{\\mathrm\{rs\}\}^\{\(k\)\}\(\\mathbf\{y\}\{\}\_\{1\},\\ldots,\\mathbf\{y\}\{\}\_\{k\}\{\}\\mid\\mathbf\{x\}\{\}\)\\triangleq\\prod\_\{i=1\}^\{k\}\\pi\(\\mathbf\{y\}\_\{i\}\\mid\\mathbf\{x\}\{\}\)\.RS is simple and broadly applicable, but it does not coordinate branches or allocate the budget across complementary reasoning paths\. As a result, additional samples may cluster around similar high\-probability solutions and repeat the same mistakes\.

Planned test\-time scaling \(PTTS\)\.PTTS instead treats thekkattempts as a structured test\-time computation rather than independent samples\. Ideally, one would model a joint policy directly over\(𝐲,1…,𝐲\)k∼πptts\(k\)\(⋅∣𝐱\)\(\\mathbf\{y\}\{\}\_\{1\},\\ldots,\\mathbf\{y\}\{\}\_\{k\}\{\}\)\\sim\\pi\_\{\\mathrm\{ptts\}\}^\{\(k\)\}\(\\cdot\\mid\\mathbf\{x\}\{\}\)\. However, directly modeling an unrestricted joint distribution over\(𝐲,1…,𝐲\)k\(\\mathbf\{y\}\{\}\_\{1\},\\ldots,\\mathbf\{y\}\{\}\_\{k\}\{\}\)is difficult: the space of solution tuples grows rapidly withkk, and the policy must capture both solution quality and how attempts should differ\.

PTTS therefore uses aplanner\-executorfactorization\. Theplannerfirst produces a shared plan consisting ofkksolution outlines,𝐨=\(𝐨,1…,𝐨\)k∼πp\(⋅∣𝐱\)\\mathbf\{o\}\{\}=\(\\mathbf\{o\}\{\}\_\{1\},\\ldots,\\mathbf\{o\}\{\}\_\{k\}\{\}\)\\sim\\pi\_\{p\}\(\\cdot\\mid\\mathbf\{x\}\{\}\), where each outline steers one attempt toward a distinct reasoning path\. Theexecutorthen generates a full solution conditioned on the problem and the corresponding outline:𝐲i∼πe\(⋅∣𝐱,𝐨\)i\\mathbf\{y\}\_\{i\}\\sim\\pi\_\{e\}\(\\cdot\\mid\\mathbf\{x\}\{\},\\mathbf\{o\}\{\}\_\{i\}\)\. Equivalently, PTTS defines the following joint distribution over the generated plan and solutions:

πptts\(k\)\(𝐨,𝐲,1…,𝐲∣k𝐱\)≜πp\(𝐨∣𝐱\)∏i=1kπe\(𝐲i∣𝐱,𝐨\)i\.\\displaystyle\\pi\_\{\\mathrm\{ptts\}\}^\{\(k\)\}\(\\mathbf\{o\}\{\},\\mathbf\{y\}\{\}\_\{1\},\\ldots,\\mathbf\{y\}\{\}\_\{k\}\{\}\\mid\\mathbf\{x\}\{\}\)\\triangleq\\pi\_\{p\}\(\\mathbf\{o\}\{\}\\mid\\mathbf\{x\}\{\}\)\\prod\_\{i=1\}^\{k\}\\pi\_\{e\}\(\\mathbf\{y\}\_\{i\}\\mid\\mathbf\{x\}\{\},\\mathbf\{o\}\{\}\_\{i\}\)\.This factorization makes PTTS tractable while preserving branch\-level coordination: the planner chooses complementary high\-level reasoning paths, and the executor realizes each path independently\. In §[3\.2](https://arxiv.org/html/2609.27374#S3.SS2), we analyze why this structure can improvepass​@​k\\mathrm\{pass@\}\{k\}over RS\.

### 3\.2Why PTTS Improves Pass@kk

Expressiveness\.From a policy\-class perspective,πptts\(k\)\\pi\_\{\\mathrm\{ptts\}\}^\{\(k\)\}strictly generalizesπrs\(k\)\\pi\_\{\\mathrm\{rs\}\}^\{\(k\)\}: when the planner emits an empty or constant outline for every branch, PTTS reduces to RS\. Therefore, the optimalpass​@​k\\mathrm\{pass@\}\{k\}attainable within PTTS is no lower than that attainable within RS\.

This gap can be madestrictunder a per\-attempt success constraint\. Suppose each branch has fixed marginal success probabilityp≜𝔼⁡\[r⁡\(𝐲i,𝐱\)\]p\\triangleq\\mathbb\{E\}\\left\[r\(\\mathbf\{y\}\_\{i\};\\mathbf\{x\}\{\}\)\\right\]\. For RS, independence givespass​@​k=1−\(1−p\)k\\mathrm\{pass@\}\{k\}=1\-\(1\-p\)^\{k\}\. In contrast, PTTS can coordinate branches so that their success events are nearly disjoint, for example by assigning complementary strategies\. In the ideal disjoint case,pass​@​k=min⁡\(k​p,1\)\\mathrm\{pass@\}\{k\}=\\min\(kp,1\), which isstrictly largerthan RS forp∈\(0,1\)p\\in\(0,1\)andk\>1k\>1\. Thus, PTTS converts the same average per\-attempt competence into a higher joint pass rate by reducing redundancy across attempts\.

![Refer to caption](https://arxiv.org/html/2609.27374v1/figures/example_output.png)Figure 3:Example output from PTTS\-ZS\.Left:a markdown list of four distinct outlines generated by the plannerπp\\pi\_\{p\}in a single autoregressive pass\.Right:solutions generated by the executorπe\\pi\_\{e\}conditioned on each outline, with the outline\-following portionshighlighted\. Most solutions faithfully follow their outlines, steering the reasoning toward different modes and boosting the diversity\.Mode coverage analysis\.We then take a dataset\-level optimization perspective to illustrate why standard paradigms such as GRPO suffer frommode collapse, and how PTTS\-RL naturally mitigates this issue\. Optimizing dataset\-levelpass​@​1\\mathrm\{pass@\}\{1\}encourages the policy to concentrate on adominant reasoning mode\. While this maximizes average single\-shot reward, it inherently sacrifices diversity and can ultimately degradepass​@​k\\mathrm\{pass@\}\{k\}\([Yue et al\.,](https://arxiv.org/html/2609.27374#bib.bib28)\)\.

To formalize this intuition, we study a simplified setting in which the executor is fixed and imperfect, reducing the reasoning process to a mode\-selection problem\. Assume each example belongs to a latent categoryc∈𝒞c\\in\\mathcal\{C\}, and the policy selects a reasoning modez∈𝒵z\\in\\mathcal\{Z\}, with success rateRz​cR\_\{zc\}determined only by the selected modezzand data categorycc\. We consider three realistic assumptions, with the full derivation deferred to Appendix[A](https://arxiv.org/html/2609.27374#A1):

1. 1\.Imperfect policy:The policy cannot observe the true categoryccperfectly\. Instead, it acts based on a noisy observed categoryc′c^\{\\prime\}, making perfect deterministic mode\-matching impossible\.
2. 2\.Mode dominance:For a certain observed categoryc′c^\{\\prime\}, there exists a single dominant modez∗​\(c′\)z^\{\*\}\(c^\{\\prime\}\)that achieves a strictly higher expected success rate than any other mode\.
3. 3\.Complementary modes:The dominant modez∗z^\{\*\}struggles on a “worst\-case” categoryc†c^\{\\dagger\}by a strict margin\. Crucially, this worst\-case category benefits more from a non\-dominant mode\.

Under these conditions, optimizing the marginal policy forpass​@​1\\mathrm\{pass@\}\{1\}via GRPO drives fundamentally different behavior than optimizing the joint policy againstpass​@​k\\mathrm\{pass@\}\{k\}via PTTS\-RL\. Since GRPO maximizes expected average return, it assigns all probability mass to the dominant modez∗​\(c′\)z^\{\*\}\(c^\{\\prime\}\), leading directly to mode collapse\. In contrast, PTTS\-RL optimizes directly forpass​@​k\\mathrm\{pass@\}\{k\}, naturally accounting for thediminishing marginal returnsof repeatedly samplingz∗z^\{\*\}\. We show that beyond a finite thresholdk∗k^\{\*\},PTTS\-RL strategically allocates reasoning slots to non\-dominant modes, thereby rescuing performance on the worst\-case categoryc†c^\{\\dagger\}\. Consequently, these policies exhibit distinct scaling dynamics: PTTS\-RL drives the residual errorϵ=1−pass​@​k\\epsilon=1\-\\mathrm\{pass@\}\{k\}to zero exponentially ask→∞k\\to\\infty, whereas GRPO’spass​@​k\\mathrm\{pass@\}\{k\}plateaus whenever the dominant mode has zero accuracy on the worst\-case category\. Even when GRPO does not plateau,PTTS\-RL shrinks the residual error at a strictly faster exponential rateby actively covering blind spots rather than repeatedly resampling the dominant mode\.

To illustrate these theoretical mechanics, Figure[2](https://arxiv.org/html/2609.27374#S3.F2)instantiates this model in a minimal two\-category, two\-mode environment wherez1z\_\{1\}is dominant\. The numerical results confirm our intuition: PTTS\-RL actively breaks mode collapse, substantially improving coverage of the non\-dominant class \(Figure[2\(b\)](https://arxiv.org/html/2609.27374#S3.F2.sf2)\) and acceleratingpass​@​k\\mathrm\{pass@\}\{k\}improvements after thek∗k^\{\*\}threshold is crossed and before test\-time scaling saturates \(Figure[2\(a\)](https://arxiv.org/html/2609.27374#S3.F2.sf1)\)\.

### 3\.3Instantiation

PTTS\-ZS\.In the zero\-shot setting, we instantiate both the plannerπp\\pi\_\{p\}and the executorπe\\pi\_\{e\}with LLMs\. As shown in Figure[3](https://arxiv.org/html/2609.27374#S3.F3), given a problem𝐱\\mathbf\{x\}\{\}, the planner generates a markdown list ofkksolution outlines in a single autoregressive pass, which is then deterministically parsed intokkoutlines\(𝐨,1…,𝐨\)k\(\\mathbf\{o\}\{\}\_\{1\},\\ldots,\\mathbf\{o\}\{\}\_\{k\}\{\}\)\. The executor then independently generates a full solution𝐲i∼πe\(⋅∣𝐱,𝐨i\)\\mathbf\{y\}\_\{i\}\\sim\\pi\_\{e\}\(\\cdot\\mid\\mathbf\{x\}\{\},\\mathbf\{o\}\_\{i\}\)for each outline\. Concretely, we use strong reasoning models such as Qwen3\([Yang et al\., 2025a](https://arxiv.org/html/2609.27374#bib.bib24)\)as executors, making PTTS\-ZS an inference\-only add\-on to further enhance the test\-time scaling\. However, we find that reasoning models are poorly suited as planners: they tend to commit early to a single strategy and start solving the problem, rather than generating multiple diverse, high\-level solution outlines\. Therefore, we use the corresponding base models as planners\. Despite its simplicity, PTTS\-ZS yields surprisingly large improvements overpass​@​k\\mathrm\{pass@\}\{k\}\.

PTTS\-RL\.We then present PTTS\-RL, which trains the planner end\-to\-end with GRPO\([Shao et al\., 2024](https://arxiv.org/html/2609.27374#bib.bib15);[Yu et al\., 2026](https://arxiv.org/html/2609.27374#bib.bib27)\)to directly optimizepass​@​k\\mathrm\{pass@\}\{k\}\.

In GRPO, for each input𝐱\\mathbf\{x\}, we sampleGGoutputs\{𝐲\}\(i\)i=1G\\\{\\mathbf\{y\}\{\}^\{\(i\)\}\\\}\_\{i=1\}^\{G\}from the model and obtain a rewardRiR\_\{i\}for each output\. We then compute the advantageAiA\_\{i\}by normalizing the rewards within the group:Ai=\(Ri−mean​\(\{Rj\}j=1G\)\)/std​\(\{Rj\}j=1G\)A\_\{i\}=\\Big\(R\_\{i\}\-\\text\{mean\}\(\\\{R\_\{j\}\\\}\_\{j=1\}^\{G\}\)\\Big\)~/~\\text\{std\}\(\\\{R\_\{j\}\\\}\_\{j=1\}^\{G\}\)\. This same advantage is assigned to every token in the corresponding output𝐲\(i\)\\mathbf\{y\}\{\}^\{\(i\)\}, i\.e\.,Ai,t=AiA\_\{i,t\}=A\_\{i\}\. The resulting training objective is:

𝒥GRPO=𝔼\[1G∑i=1G1\|𝐲\(i\)\|∑t=1\|𝐲\(i\)\|min\(ρi,tAi,t,clip\(ρi,t,1−ϵL,1\+ϵH\)Ai,t\)\],\\displaystyle\\mathcal\{J\}\_\{\\text\{GRPO\}\}=\\mathbb\{E\}\\left\[\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\frac\{1\}\{\|\\mathbf\{y\}\{\}^\{\(i\)\}\|\}\\sum\_\{t=1\}^\{\|\\mathbf\{y\}\{\}^\{\(i\)\}\|\}\\min\\Big\(\\rho\_\{i,t\}A\_\{i,t\},~\\text\{clip\}\(\\rho\_\{i,t\},1\-\\epsilon\_\{L\},1\+\\epsilon\_\{H\}\)A\_\{i,t\}\\Big\)\\right\],whereρi,t=πθ\(𝐲\|t\(i\)𝐱,𝐲<t\(i\)\)/πθo​l​d\(𝐲\|t\(i\)𝐱,𝐲<t\(i\)\)\\rho\_\{i,t\}=\\pi\_\{\\theta\}\\left\(\\mathbf\{y\}\{\}^\{\(i\)\}\_\{t\}\|~\\mathbf\{x\}\{\},\{\\mathbf\{y\}\{\}\}^\{\(i\)\}\_\{<t\}\\right\)~/~\\pi\_\{\\theta\_\{old\}\}\\left\(\\mathbf\{y\}\{\}^\{\(i\)\}\_\{t\}\|~\\mathbf\{x\}\{\},\{\\mathbf\{y\}\{\}\}^\{\(i\)\}\_\{<t\}\\right\)is the importance sampling term\.

PTTS\-RL applies GRPO to the planner modelπp\\pi\_\{p\}, where each sampled output is a list of outlines𝐨=\(𝐨,1…,𝐨\)k\\mathbf\{o\}=\(\\mathbf\{o\}\{\}\_\{1\},\\ldots,\\mathbf\{o\}\{\}\_\{k\}\{\}\)\. The reward for a sampled output is computed using a fixed executorπe\\pi\_\{e\}, which generates a solution𝐲i\\mathbf\{y\}\_\{i\}for each of thekkoutlines,i=1,…,ki=1,\\ldots,k\. Each generated solution is evaluated for accuracy asr⁡\(𝐲i∣𝐱\)∈\{0,1\}r\(\\mathbf\{y\}\_\{i\}\\mid\\mathbf\{x\}\{\}\)\\in\\\{0,1\\\}, yielding thepass​@​k\\mathrm\{pass@\}\{k\}reward:

R\(𝐨\)=maxi=1,…,kr\(𝐲i∣𝐱\),𝐲i∼πe\(⋅∣𝐱,𝐨i\)\.\\displaystyle R\\left\(\\mathbf\{o\}\{\}\\right\)=\\max\_\{i=1,\\ldots,k\}r\(\\mathbf\{y\}\_\{i\}\\mid\\mathbf\{x\}\{\}\),\\quad\\mathbf\{y\}\_\{i\}\\sim\\pi\_\{e\}\(\\cdot\\mid\\mathbf\{x\}\{\},\\mathbf\{o\}\_\{i\}\)\.However, generating full\-length solutions for allkkbranches makes training computationally expensive\. Moreover, long execution traces allow the executor to drift away from the outline and achieve the correct answer through extensive rethinking, assigning positive reward to a poor outline and weakening the training signal\. We therefore adopt atruncated executionstrategy during training, limiting executor outputs to fewer tokens than at inference \(4​k4kvs\.10​k10ktokens, as detailed in §[4\.1](https://arxiv.org/html/2609.27374#S4.SS1)\) to improve efficiency and sharpen the reward signal\.

## 4Experiments

We empirically study three questions: \(1\) How much does the PTTS framework improvepass​@​k\\mathrm\{pass@\}\{k\}? \(2\) Which output properties drive these gains, and how do PTTS\-ZS and PTTS\-RL shape them? \(3\) Which PTTS design choices and hyperparameters most affect PTTSpass​@​k\\mathrm\{pass@\}\{k\}performance? In this section, we describe the experimental setup in §[4\.1](https://arxiv.org/html/2609.27374#S4.SS1), address \(1\)–\(2\) through the results and analyses in §[4\.2](https://arxiv.org/html/2609.27374#S4.SS2), and study \(3\) through ablations in §[4\.3](https://arxiv.org/html/2609.27374#S4.SS3)\.

### 4\.1Setup

Evaluation settings\.We evaluate on five benchmarks: MATH\-500\([Hendrycks et al\.,](https://arxiv.org/html/2609.27374#bib.bib8)\), AIME 2024, 2025, and 2026\([Art of Problem Solving, n\.d\.](https://arxiv.org/html/2609.27374#bib.bib1)\), and HMMT\-Feb26\([Dekoninck et al\., 2026](https://arxiv.org/html/2609.27374#bib.bib5)\)\. We reportpass​@​k\\mathrm\{pass@\}\{k\}forkkup to 64\. For each problem, we generaten=64n=64solutions, with a budget of10​k10ktokens per solution\. We then analytically computepass​@​k\\mathrm\{pass@\}\{k\}as the expected maximum reward amongkkresponses sampled from the set ofnngenerated responses:

pass@k=𝔼\[1−\(n−ck\)/\(nk\)\],c=∑i=1n𝟏\(r\(𝐲∣i𝐱\)=1\)\.\\displaystyle\\mathrm\{pass@\}\{k\}=\\mathbb\{E\}\\left\[1\-\\binom\{n\-c\}\{k\}~/~\\binom\{n\}\{k\}\\right\],\\quad c=\\sum\_\{i=1\}^\{n\}\\mathbf\{1\}\\left\(r\(\\mathbf\{y\}\{\}\_\{i\}\\mid\\mathbf\{x\}\{\}\)=1\\right\)\.
Models and baselines\.We compare PTTS\-ZS and PTTS\-RL against two baselines: \(1\) repeated sampling, and \(2\) Guided Sampling\([Handa et al\., 2025](https://arxiv.org/html/2609.27374#bib.bib7)\), an inference\-time method similar to PTTS\-ZS that first generates diverseconceptsand then uses them to guide reasoning\. We conduct experiments with Qwen3 models at 1\.7B and 4B scales\. For all methods, we use the reasoning modelQwen3\-1\.7B/4Bas the executor\. For zero\-shot methods that involve planning \(PTTS\-ZS and Guided Sampling\), we use the corresponding base model,Qwen3\-1\.7B/4B\-Base, as the planner\. For Guided Sampling, we follow the recommended setting of generating 5 ideas\. For PTTS\-RL, we train the planner starting from the base model, as discussed below\.

![Refer to caption](https://arxiv.org/html/2609.27374v1/figures/main_1.7b.png)Figure 4:Main results for Qwen3\-1\.7B\.PTTS\-RL improvespass​@​64\\mathrm\{pass@\}\{64\}over repeated sampling by up to 13\.4 \(6\.4 on average\), while itspass​@​32\\mathrm\{pass@\}\{32\}already exceeds repeated sampling’spass​@​64\\mathrm\{pass@\}\{64\}\(58\.3 vs\. 55\.8\)\.Training settings\.We train the planner from the base models using the DAPO training set\([Yu et al\., 2026](https://arxiv.org/html/2609.27374#bib.bib27)\), optimizing for thepass​@​k\\mathrm\{pass@\}\{k\}\(k=4k=4\) reward\. As in §[3\.3](https://arxiv.org/html/2609.27374#S3.SS3), we use a truncated execution budget of4​k4ktokens for each solution\. We train theQwen3\-1\.7B/4B\-Basemodels with a learning rate of 1e\-6 for 240 steps and use the final checkpoint; detailed hyperparameter settings are provided in Appendix[B](https://arxiv.org/html/2609.27374#A2)\.

Scaling PTTS to larger test\-time budgets\.In both inference \(PTTS\-ZS and PTTS\-RL\) and training \(PTTS\-RL\), we use a branching factor ofk=4k=4\. To evaluate larger total budgetsK\>kK\>k, we independently repeat the PTTS procedure until we collectKKexecutor solutions in total, and then computepass​@​K\\mathrm\{pass@\}\{K\}over the aggregated set\. For PTTS\-RL, this allows us to keep the same planner while seamlessly scaling to larger inference budgets\. As discussed in §[4\.3](https://arxiv.org/html/2609.27374#S4.SS3), further increasingkkyields diminishing returns; therefore, we usek=4k=4as a computationally efficient setting to demonstrate our method\.

### 4\.2Main Results and Discussions

Main results\.Figures[4](https://arxiv.org/html/2609.27374#S4.F4)and[5](https://arxiv.org/html/2609.27374#S4.F5)present our main results for the 1\.7B and 4B models, respectively\. Despite its simple design, PTTS\-ZS consistently outperforms repeated sampling, withpass​@​64\\mathrm\{pass@\}\{64\}gains of up to 6\.7 for 1\.7B models and 3\.4 for 4B models\. PTTS\-RL further improves upon PTTS\-ZS and consistently outperforms both repeated sampling and Guided Sampling\. Specifically, compared to repeated sampling, PTTS\-RL improvespass​@​64\\mathrm\{pass@\}\{64\}by up to 13\.4 for 1\.7B models and 6\.7 for 4B models\. Beyond performance gains, improved compute efficiency is a broader benefit of PTTS\. Most notably, at both model scales, PTTS\-RL’spass​@​32\\mathrm\{pass@\}\{32\}exceeds repeated sampling’spass​@​64\\mathrm\{pass@\}\{64\}, achieving better performance with only half the sampling budget\. Overall, these results demonstrate the benefit of coordinating test\-time compute across complementary reasoning paths\.

Diversity as an indicator of𝐩𝐚𝐬𝐬​@​𝐤\\mathbf\{pass@k\}\.To better understand what drives these gains, we examine whether broader coverage of reasoning strategies is associated with betterpass​@​k\\mathrm\{pass@\}\{k\}performance\. Concretely, we prompt GPT\-5\-mini to group outlines and solutions into clusters and report the number of unique clusters as the diversity metric, as detailed in Appendix[C\.2](https://arxiv.org/html/2609.27374#A3.SS2)\. We then use Qwen3\-1\.7B’s PTTS\-ZS outputs on AIME\-2024–2026 to analyze the rank\-averaged correlations among outline diversity, solution diversity, andpass​@​4\\mathrm\{pass@\}\{4\}across groups ofk=4k=4reasoning branches\. Results show that greater outline diversity is associated with higher solution diversity \(Spearman’sρ=0\.57\\rho=0\.57\) and better resultingpass​@​4\\mathrm\{pass@\}\{4\}\(ρ=0\.59\\rho=0\.59\), while solution diversity is also positively correlated withpass​@​4\\mathrm\{pass@\}\{4\}\(ρ=0\.32\\rho=0\.32\)\. Overall, these results support diversity as a meaningful indicator ofpass​@​k\\mathrm\{pass@\}\{k\}performance across both the outline and solution levels\.

![Refer to caption](https://arxiv.org/html/2609.27374v1/figures/main_4b.png)Figure 5:Main results for Qwen3\-4B\.PTTS\-RL improvespass​@​64\\mathrm\{pass@\}\{64\}over repeated sampling by up to 6\.7 \(3\.6 on average\), while itspass​@​32\\mathrm\{pass@\}\{32\}already exceeds repeated sampling’spass​@​64\\mathrm\{pass@\}\{64\}\(65\.1 vs\. 63\.8\)\.![Refer to caption](https://arxiv.org/html/2609.27374v1/figures/diversity_stats.png)Figure 6:Solution diversity for repeated sampling \(RS\), PTTS\-ZS, and PTTS\-RL, measured on Qwen3\-1\.7B outputs for AIME 2024–2026\.Left:Overall diversity, measured by the number of distinct solution clusters amongk=64k\\\!=\\\!64branches \(Appendix[C\.2](https://arxiv.org/html/2609.27374#A3.SS2)\)\. Bars show the fraction of problems in each diversity range, with parentheses indicating the mean number of distinct clusters\.Right:Useful diversity, measured as the fraction of correct solutions belonging to distinct solution clusters \(Appendix[C\.3](https://arxiv.org/html/2609.27374#A3.SS3)\)\.Diversity induced by PTTS\.We next investigate whether PTTS effectively enhances diversity\. Using the same diversity measure discussed above, we compare the diversity of repeated sampling, PTTS\-ZS, and PTTS\-RL on Qwen3\-1\.7B outputs on the AIME\-2024–2026 datasets\. As shown in Figure[6](https://arxiv.org/html/2609.27374#S4.F6), both PTTS variants increase overall response diversity over repeated sampling, with PTTS\-ZS achieving the highest diversity across generated solutions\. Optimized for thepass​@​k\\mathrm\{pass@\}\{k\}objective, PTTS\-RL does not further increase overall diversity over PTTS\-ZS, but instead steers diversity toward branches that reach correct answers\. Measuring thisuseful diversityas the normalized count of distinct clusters among correct branches \(detailed in Appendix[C\.3](https://arxiv.org/html/2609.27374#A3.SS3)\), PTTS\-RL achieves the highestuseful diversityof 15\.9%, outperforming both repeated sampling and PTTS\-ZS\. Figure[10](https://arxiv.org/html/2609.27374#A4.F10)further shows a qualitative example of the increased diversity under PTTS\. Overall, PTTS not only increases reasoning diversity but also steers it toward successful reasoning paths, thereby improvingpass​@​k\\mathrm\{pass@\}\{k\}\.

Outline adherence of the executor\.Beyond the planner, PTTS also relies on the executor to faithfully develop each proposed outline into a full solution\. Reasoning models, however, are not explicitly trained for outline\-guided generation, making it unclear how closely their solutions adhere to the provided outlines\. We evaluate outline adherence via LLM\-as\-a\-judge using GPT\-5\-mini, as detailed in Appendix[C\.1](https://arxiv.org/html/2609.27374#A3.SS1)\. Analysis of PTTS\-ZS and PTTS\-RL outputs on the AIME\-2024–2026 datasets shows that Qwen3\-1\.7B and Qwen3\-4B follow outlines reasonably well, achieving adherence rates of 70% and 71%, respectively\. Interestingly, qualitative observations show that while executors typically begin by either strictly following or reiterating the outline verbatim, they may pivot to alternative strategies if the initial approach fails, with an example shown in Figure[11](https://arxiv.org/html/2609.27374#A4.F11)\. This may dilute the outline’s influence on final performance as reasoning traces become longer, thereby affecting the training signal, as discussed in §[4\.3](https://arxiv.org/html/2609.27374#S4.SS3)\.

### 4\.3Ablation Studies

Ablations on the branching factorkk\.We study the effect of the PTTS branching factorkkwhile fixing the total inference budget atK=64K=64, following the scaling procedure in §[4\.1](https://arxiv.org/html/2609.27374#S4.SS1)\. As shown in Figure[8](https://arxiv.org/html/2609.27374#S4.F8), moving from repeated sampling \(k=1k=1\) to modest branching factors substantially improves PTTS\-ZS, with performance peaking atk=4k=4for Qwen3\-1\.7B andk=8k=8for Qwen3\-4B\. Beyond these points, increasingkkyields no consistentpass​@​64\\mathrm\{pass@\}\{64\}gains, with performance fluctuating at larger values\. Since largerkkprovides no reliable benefit while substantially increasing PTTS\-RL training cost, we usek=4k=4throughout our main experiments\.

![Refer to caption](https://arxiv.org/html/2609.27374v1/figures/qwen3_k_ablation_pass64.png)Figure 7:Pass​@​64\\mathrm\{Pass@\}\{64\}performance of PTTS\-ZS across branching factorskkat a fixed inference budget ofK=64K=64, averaged across AIME 2024–2026;k=1k=1denotes repeated sampling\.
![Refer to caption](https://arxiv.org/html/2609.27374v1/figures/ablation_trunc_exec_pass64.png)Figure 8:Pass​@​64\\mathrm\{Pass@\}\{64\}performance of PTTS\-RL using Qwen3\-1\.7B trained with varying execution budgets, averaged across AIME 2024–2026\.

Ablations on truncated execution\.To evaluate the impact of training\-time execution limits, we train Qwen3\-1\.7B with PTTS\-RL using execution budgets from2​k2kto10​k10ktokens, while fixing the inference budget at10​k10k\. As shown in Figure[8](https://arxiv.org/html/2609.27374#S4.F8), performance peaks at4​k4kand degrades with larger budgets: matching the10​k10kinference budget lowers averagepass​@​64\\mathrm\{pass@\}\{64\}by 7\.8 points and throughput by 27%\. Longer executions allow the executor to recover from poor outlines through extended rethinking, causing these outlines to receive positive reward and weakening the training signal\. Conversely, an overly strict2​k2kbudget prematurely truncates valid reasoning, making the reward

Table 1:Pass​@​k\\mathrm\{Pass@\}\{k\}performance of repeated sampling \(RS\), PTTS\-ZS, and PTTS\-RL under different planner and executor configurations\.Highlighted rowsindicate configurations where the planner and executor differ in size\.uninformative and substantially hurting performance\. Overall, a4​k4kbudget yields the optimal balance, providing enough length to reliably execute outlines and actively truncating unguided rethinking\.

Ablations on planner and executor sizes\.While our main experiments use planners and executors of the same sizes, decoupling them reveals distinct scaling behaviors\. As shown in Table[1](https://arxiv.org/html/2609.27374#S4.T1), 1\.7B planners effectively guide a larger Qwen3\-4B executor, yielding significant gains over repeated sampling and matching the performance of 4B planners, despite a train\-test mismatch: the 1\.7B PTTS\-RL planner is trained with the smaller Qwen3\-1\.7B executor\. In contrast, using 4B planners to guide a Qwen3\-1\.7B executor provides no additional benefit over 1\.7B planners\. These results suggest a promising strategy of training small but capable planners alongside small executors, then scaling only the executor at inference time\.

Figure 9:Pass​@​k\\mathrm\{Pass@\}\{k\}of repeated sampling, PTTS\-ZS, and PTTS\-RL withe3\-1\.7Bas the executor\.Combination with diversity\-aware RL\.As discussed in §[2](https://arxiv.org/html/2609.27374#S2), PTTS operates at inference time and is complementary to training\-side diversity\-aware improvements\. We demonstrate this by using e3\([Setlur et al\., 2026](https://arxiv.org/html/2609.27374#bib.bib14)\)as the executor, which is trained to improve exploration through a structured curriculum\. Withe3\-1\.7Bas the executor, both the zero\-shot 1\.7B planner and the RL planner trained with a Qwen3\-1\.7B executor outperform repeated sampling by 4\.4 and 3\.3 points, respectively\. These gains show that PTTS remains effective on top of a diversity\-aware RL executor, providing an additional and complementary source of improvement\.

## 5Conclusion

We introduce Planned Test\-Time Scaling \(PTTS\), a framework that addresses the redundant reasoning and repeated errors of standard repeated sampling by explicitly coordinating attempts acrosskkdifferent branches\. PTTS decomposes test\-time scaling under a budget ofkkreasoning branches into a planner stage that produces diverse solution outlines and an executor stage that follows each outline\. We instantiate PTTS with a zero\-shot variant, PTTS\-ZS, which plans using an off\-the\-shelf model, and a trained variant, PTTS\-RL, which directly optimizes the planner for the pass@kkreward\. We show theoretically and empirically that PTTS outperforms independent repeated sampling: across five mathematical datasets, both PTTS\-ZS and PTTS\-RL consistently yield gains of up to 13\.4 points\. Overall, PTTS provides a principled paradigm for test\-time scaling, highlighting explicit coordination and budget allocation as key to better unlocking its potential\.

## References

- Art of Problem Solving \(n\.d\.\)Art of Problem Solving\.AIME Problems and Solutions\.[https://artofproblemsolving\.com/wiki/index\.php/AIME\_Problems\_and\_Solutions](https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions), n\.d\.AoPS Wiki\. Accessed: 2026\-05\-14\.
- Brown et al\. \(2024\)Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini\.Large language monkeys: Scaling inference compute with repeated sampling\.*arXiv preprint arXiv:2407\.21787*, 2024\.
- Chen et al\. \(2025\)Zhipeng Chen, Xiaobo Qin, Youbin Wu, Yue Ling, Qinghao Ye, Wayne Xin Zhao, and Guang Shi\.Pass@ k training for adaptively balancing exploration and exploitation of large reasoning models\.*arXiv preprint arXiv:2508\.10751*, 2025\.
- Cui et al\. \(2025\)Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al\.The entropy mechanism of reinforcement learning for reasoning language models\.*arXiv preprint arXiv:2505\.22617*, 2025\.
- Dekoninck et al\. \(2026\)Jasper Dekoninck, Nikola Jovanović, Tim Gehrunger, Kári Rögnvaldsson, Ivo Petrov, Chenhao Sun, and Martin Vechev\.Beyond benchmarks: Matharena as an evaluation platform for mathematics with llms\.2026\.URL[https://arxiv\.org/abs/2605\.00674](https://arxiv.org/abs/2605.00674)\.
- Guo et al\. \(2025\)Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al\.Deepseek\-r1: Incentivizing reasoning capability in llms via reinforcement learning\.*arXiv preprint arXiv:2501\.12948*, 2025\.
- Handa et al\. \(2025\)Divij Handa, Mihir Parmar, Aswin RRV, Md Nayem Uddin, Hamid Palangi, and Chitta Baral\.Guidedsampling: Steering llms towards diverse candidate solutions at inference\-time\.*arXiv preprint arXiv:2510\.03777*, 2025\.
- \(8\)Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt\.Measuring mathematical problem solving with the math dataset\.In*Thirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 2\)*\.
- Jiang et al\. \(2025\)Yuhua Jiang, Jiawei Huang, Yufeng Yuan, Xin Mao, Yu Yue, Qianchuan Zhao, and Lin Yan\.Risk\-sensitive rl for alleviating exploration dilemmas in large language models\.*arXiv preprint arXiv:2509\.24261*, 2025\.
- Kang & Zhang \(2025\)Shijia Kang and Muhan Zhang\.The road less traveled: Enhancing exploration in llms via sequential sampling\.*arXiv preprint arXiv:2510\.15502*, 2025\.
- Li et al\. \(2025\)Tianjian Li, Yiming Zhang, Ping Yu, Swarnadeep Saha, Daniel Khashabi, Jason Weston, Jack Lanchantin, and Tianlu Wang\.Jointly reinforcing diversity and quality in language model generations\.*arXiv preprint arXiv:2509\.02534*, 2025\.
- Li et al\. \(2026\)Yilong Li, Suman Banerjee, and Tong Che\.Cast a wider net: Coordinated pass@ k policy optimization for code reasoning\.*arXiv preprint arXiv:2605\.27000*, 2026\.
- \(13\)Tao Ren, Jinyang Jiang, Hui Yang, Wan Tian, and Yijie Peng\.Riskpo: Risk\-based policy optimization with verifiable reward for llm post\-training\.In*NeurIPS 2025 Workshop MLxOR: Mathematical Foundations and Operational Integration of Machine Learning for Uncertainty\-Aware Decision\-Making*\.
- Setlur et al\. \(2026\)Amrith Setlur, Matthew Yang, Charlie Snell, Jeremiah Greer, Ian Wu, Virginia Smith, Max Simchowitz, and Aviral Kumar\.e3: Learning to explore enables extrapolation of test\-time compute for llms\.In*International Conference on Learning Representations*, volume 2026, pp\. 127323–127361, 2026\.
- Shao et al\. \(2024\)Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al\.Deepseekmath: Pushing the limits of mathematical reasoning in open language models\.*arXiv preprint arXiv:2402\.03300*, 2024\.
- Sheng et al\. \(2024\)Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu\.Hybridflow: A flexible and efficient rlhf framework\.*arXiv preprint arXiv: 2409\.19256*, 2024\.
- Snell et al\. \(2024\)Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar\.Scaling llm test\-time compute optimally can be more effective than scaling model parameters\.*arXiv preprint arXiv:2408\.03314*, 2024\.
- Tang et al\. \(2025\)Yunhao Tang, Kunhao Zheng, Gabriel Synnaeve, and Rémi Munos\.Optimizing language models for inference time objectives using reinforcement learning\.*arXiv preprint arXiv:2503\.19595*, 2025\.
- Walder & Karkhanis \(2025\)Christian Walder and Deep Karkhanis\.Pass@ k policy optimization: Solving harder reinforcement learning problems\.*arXiv preprint arXiv:2505\.15201*, 2025\.
- Wang et al\. \(2026\)Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xiong\-Hui Chen, Jianxin Yang, Zhenru Zhang, et al\.Beyond the 80/20 rule: High\-entropy minority tokens drive effective reinforcement learning for llm reasoning\.*Advances in Neural Information Processing Systems*, 38:115452–115486, 2026\.
- Wang et al\. \(2025\)Tianchun Wang, Zichuan Liu, Yuanzhou Chen, Jonathan Light, Weiyang Liu, Haifeng Chen, Xiang Zhang, and Wei Cheng\.On the effect of sampling diversity in scaling llm inference, 2025\.URL[https://arxiv\.org/abs/2502\.11027](https://arxiv.org/abs/2502.11027)\.
- Wu et al\. \(2026\)Fang Wu, Weihao Xuan, Ximing Lu, Mingjie Liu, Yi Dong, Zaid Harchaoui, and Yejin Choi\.The invisible leash: Why rlvr may or may not escape its origin, 2026\.URL[https://arxiv\.org/abs/2507\.14843](https://arxiv.org/abs/2507.14843)\.
- Wu et al\. \(2024\)Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang\.Inference scaling laws: An empirical analysis of compute\-optimal inference for problem\-solving with language models\.*arXiv preprint arXiv:2408\.00724*, 2024\.
- Yang et al\. \(2025a\)An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu\.Qwen3 technical report, 2025a\.URL[https://arxiv\.org/abs/2505\.09388](https://arxiv.org/abs/2505.09388)\.
- Yang et al\. \(2025b\)Kaisen Yang, Lixuan He, Rushi Shah, Kaicheng Yang, Qinwei Ma, Dianbo Liu, and Alex Lamb\.Explore\-execute chain: Towards an efficient structured reasoning paradigm\.*arXiv preprint arXiv:2509\.23946*, 2025b\.
- Yao et al\. \(2026\)Jian Yao, Ran Cheng, Xingyu Wu, Jibin Wu, and Kay Chen Tan\.Diversity\-aware policy optimization for large language model reasoning\.*Advances in Neural Information Processing Systems*, 38:94801–94826, 2026\.
- Yu et al\. \(2026\)Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al\.Dapo: An open\-source llm reinforcement learning system at scale\.*Advances in Neural Information Processing Systems*, 38:113222–113244, 2026\.
- \(28\)Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang\.Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?In*The Thirty\-ninth Annual Conference on Neural Information Processing Systems*\.
- Zhang et al\. \(2025\)Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, et al\.A survey on test\-time scaling in large language models: What, how, where, and how well?*arXiv preprint arXiv:2503\.24235*, 2025\.

## Appendix AProofs of Propositions

As discussed in §[3\.2](https://arxiv.org/html/2609.27374#S3.SS2), we consider a finite space for data category𝒞\\mathcal\{C\}and a finite space for reasoning mode𝒵\\mathcal\{Z\}\. For a problem𝐱\\mathbf\{x\}\{\}with data categoryc⁡\(𝐱\)=c∈𝒞c\(\\mathbf\{x\}\{\}\)=c\\in\\mathcal\{C\}, a policyπ\(⋅∣𝐱\)\\pi\(\\cdot\\mid\\mathbf\{x\}\{\}\)chooses a reasoning modez∼π\(⋅∣𝐱\),z∈𝒵z\\sim\\pi\(\\cdot\\mid\\mathbf\{x\}\{\}\),~z\\in\\mathcal\{Z\}for solving this task, and the resulting success rateRz​cR\_\{zc\}is determined only by the selected modezzand data categorycc; when multiple attempts are issued on the same problem, their outcomes are independent conditional on the data category, each succeeding with probabilityRz​cR\_\{zc\}under its selected mode\.

Critically, we assume animperfect policythat cannot perfectly observe the true data categoryc⁡\(𝐱\)c\(\\mathbf\{x\}\{\}\)and perform optimal mode selection accordingly\. Instead, we assume the policy that chooses mode based on observed data categoryc′​\(𝐱\)c^\{\\prime\}\(\\mathbf\{x\}\{\}\)with imperfect data categorization,

π\(⋅∣𝐱\)=π\(⋅∣c′\(𝐱\)\),\\displaystyle\\pi\(\\cdot\\mid\\mathbf\{x\}\{\}\)=\\pi\(\\cdot\\mid c^\{\\prime\}\(\\mathbf\{x\}\{\}\)\),with some emission probability conditioned on the true data categoryc⁡\(𝐱\)c\(\\mathbf\{x\}\{\}\), i\.e\.

Pr⁡\[c′​\(𝐱\)=c′∣c⁡\(𝐱\)=c\]\.\\displaystyle\\Pr\\left\[c^\{\\prime\}\(\\mathbf\{x\}\{\}\)=c^\{\\prime\}\\mid c\(\\mathbf\{x\}\{\}\)=c\\right\]\.For notation convenience, we assume this probability is strictly positive for any\(c,c′\)\(c,c^\{\\prime\}\)pair; similarly, we assume the marginal probabilityPr\[c\(𝐱\)=c\]\\Pr\[c\(\\mathbf\{x\}\{\}\)=c\]is strictly positive for allcc\. Then we have a posterior distribution of

ω⁡\(c∣c′\)≜Pr⁡\[c⁡\(𝐱\)=c∣c′​\(𝐱\)=c′\]\\displaystyle\\omega\(c\\mid c^\{\\prime\}\)\\triangleq\\Pr\\left\[c\(\\mathbf\{x\}\{\}\)=c\\mid c^\{\\prime\}\(\\mathbf\{x\}\{\}\)=c^\{\\prime\}\\right\]that is also non\-zero for any\(c,c′\)\(c,c^\{\\prime\}\)\. Then, for a certain observed data categoryc′c^\{\\prime\}, we have theposterior\-averaged success rateof each modez∈𝒵z\\in\\mathcal\{Z\}under the observationc′c^\{\\prime\},

R¯z​c′≜𝔼c∼ω\(⋅∣c′\)\[Rz​c\]=∑c∈𝒞ω\(c∣c′\)Rz​c,\\displaystyle\\bar\{R\}\_\{zc^\{\\prime\}\}\\;\\triangleq\\;\\mathbb\{E\}\_\{c\\sim\\omega\(\\cdot\\mid c^\{\\prime\}\)\}\\left\[R\_\{zc\}\\right\]\\;=\\;\\sum\_\{c\\in\\mathcal\{C\}\}\\omega\(c\\mid c^\{\\prime\}\)\\,R\_\{zc\},i\.e\. the success rate that modezzeffectively attains through the imperfect observation: since the true category is unobserved, a single attempt with modezzsucceeds with probability exactlyR¯z​c′\\bar\{R\}\_\{zc^\{\\prime\}\}givenc′​\(𝐱\)=c′c^\{\\prime\}\(\\mathbf\{x\}\{\}\)=c^\{\\prime\}by the tower rule\. Based on this formulation, we enforce two critical assumptions that will drive the rest of the proof:

###### Assumption 1\(Mode dominance\)

The observed categoryc′∈𝒞c^\{\\prime\}\\in\\mathcal\{C\}is dominated by a single modez∗∈𝒵z^\{\*\}\\in\\mathcal\{Z\}:

∀z∈𝒵,z≠z∗,R¯z​c′<R¯z∗​c′\.\\displaystyle\\forall z\\in\\mathcal\{Z\}\{\},z\\neq z^\{\*\},~\\bar\{R\}\_\{zc^\{\\prime\}\}<\\bar\{R\}\_\{z^\{\*\}c^\{\\prime\}\}\.

###### Assumption 2\(Complementary modes\)

There exists a single worst categoryc†∈𝒞c^\{\\dagger\}\\in\\mathcal\{C\},

∃c†∈𝒞,∀c∈𝒞,c≠c†,Rz∗​c\>Rz∗​c†,\\displaystyle\\exists~c^\{\\dagger\}\\in\\mathcal\{C\},~\\forall c\\in\\mathcal\{C\},c\\neq c^\{\\dagger\},~R\_\{z^\{\*\}c\}\>R\_\{z^\{\*\}c^\{\\dagger\}\},and the category benefits more from a mode other thanz∗z^\{\*\}:

∃z∈𝒵,Rz​c†\>Rz∗​c†\.\\displaystyle\\exists z\\in\\mathcal\{Z\}\{\},~R\_\{zc^\{\\dagger\}\}\>R\_\{z^\{\*\}c^\{\\dagger\}\}\.

We now consider two types of policies, aGRPO\-RSpolicy and aPTTS\-RLpolicy, both operating on the observed categoryc′​\(𝐱\)=c′c^\{\\prime\}\(\\mathbf\{x\}\{\}\)=c^\{\\prime\}and issuingkkattempts under inference\-time budgetkk\. Each attempt will adopt a reasoning modezz, resulting in a solution𝐲\\mathbf\{y\}with a binary outcomer⁡\(𝐲,𝐱\)∈\{0,1\}r\(\\mathbf\{y\}\{\};\\mathbf\{x\}\{\}\)\\in\\\{0,1\\\}, wherePr\[r\(𝐲;𝐱\)=1\]=Rz​c\\Pr\[r\(\\mathbf\{y\}\{\};\\mathbf\{x\}\{\}\)=1\]=R\_\{zc\}\.

GRPO\-RS policy, as the standard approach, optimizes a marginal policy againstpass​@​1\\mathrm\{pass@\}\{1\}, and utilizes repeated sampling \(RS\) to scale to a larger inference\-time budgetkk\. With slight abuse of notation, we writepass​@​1​\(π\)\\mathrm\{pass@\}\{1\}\(\\pi\)for the expectation ofpass​@​1\\mathrm\{pass@\}\{1\}under policyπ\\pi\. Formally, the marginal policy is trained to maximizepass​@​1\\mathrm\{pass@\}\{1\}, i\.e\. the single\-attempt success rate,

πrs\(⋅∣c′\)∈arg​maxπpass@1\(π\),\\displaystyle\\pi\_\{\\mathrm\{rs\}\}\(\\cdot\\mid c^\{\\prime\}\)\\;\\in\\;\\argmax\_\{\\pi\}\\;\\mathrm\{pass@\}\{1\}\(\\pi\),where

pass​@​1​\(π\)\\displaystyle\\mathrm\{pass@\}\{1\}\(\\pi\)≜𝔼c∼ω\(⋅∣c′\),z∼π\(⋅∣c′\)\[𝟏\(r\(𝐲;𝐱\)=1\)\]=𝔼c∼ω\(⋅∣c′\)\[∑z∈𝒵π\(z∣c′\)Rz​c\]\\displaystyle\\triangleq\\mathbb\{E\}\_\{c\\sim\\omega\(\\cdot\\mid c^\{\\prime\}\),\\,z\\sim\\pi\(\\cdot\\mid c^\{\\prime\}\)\}\\left\[\\mathbf\{1\}\\left\(r\(\\mathbf\{y\}\{\};\\mathbf\{x\}\{\}\)=1\\right\)\\right\]=\\mathbb\{E\}\_\{c\\sim\\omega\(\\cdot\\mid c^\{\\prime\}\)\}\\left\[\\sum\_\{z\\in\\mathcal\{Z\}\}\\pi\(z\\mid c^\{\\prime\}\)\\,R\_\{zc\}\\right\]=∑z∈𝒵π⁡\(z∣c′\)​R¯z​c′\.\\displaystyle=\\sum\_\{z\\in\\mathcal\{Z\}\}\\pi\(z\\mid c^\{\\prime\}\)\\,\\bar\{R\}\_\{zc^\{\\prime\}\}\.At inference budgetkk, RS draws modes i\.i\.d\. from this*fixed*marginal,z1,…,zk∼i\.i\.d\.πrs\(⋅∣c′\)z\_\{1\},\\dots,z\_\{k\}\\overset\{\\text\{i\.i\.d\.\}\}\{\\sim\}\\pi\_\{\\mathrm\{rs\}\}\(\\cdot\\mid c^\{\\prime\}\), and runs one attempt per draw\. The resultingpass​@​k\\mathrm\{pass@\}\{k\}has expectation

pass@k\(πrs\(k\)\)=1−𝔼c∼ω\(⋅∣c′\)\[\(∑z∈𝒵πrs\(z∣c′\)\(1−Rz​c\)\)k\],\\displaystyle\\mathrm\{pass@\}\{k\}\\left\(\\pi\_\{\\mathrm\{rs\}\}^\{\(k\)\}\\right\)=1\-\\mathbb\{E\}\_\{c\\sim\\omega\(\\cdot\\mid c^\{\\prime\}\)\}\\left\[\\Big\(\\sum\_\{z\\in\\mathcal\{Z\}\}\\pi\_\{\\mathrm\{rs\}\}\(z\\mid c^\{\\prime\}\)\\,\(1\-R\_\{zc\}\)\\Big\)^\{k\}\\right\],with residual errorϵkrs≜1−pass​@​k​\(πrs\(k\)\)\\epsilon^\{\\mathrm\{rs\}\}\_\{k\}\\;\\triangleq\\;1\-\\mathrm\{pass@\}\{k\}\\left\(\\pi\_\{\\mathrm\{rs\}\}^\{\(k\)\}\\right\)\.

PTTS\-RL policy, on the other hand, optimizes the use of the entire inference budget directly againstpass​@​k\\mathrm\{pass@\}\{k\}\. At inference budgetkk, the policy selects an allocation𝐤=\(kz\)z∈𝒵\\mathbf\{k\}=\(k\_\{z\}\)\_\{z\\in\\mathcal\{Z\}\}that assigns thekkattempt slots across reasoning modes, withkz∈ℤ≥0k\_\{z\}\\in\\mathbb\{Z\}\_\{\\geq 0\}and∑z∈𝒵kz=k\\sum\_\{z\\in\\mathcal\{Z\}\}k\_\{z\}=k\. An allocation𝐤\\mathbf\{k\}then achieves apass​@​k\\mathrm\{pass@\}\{k\}of:

pass@k\(𝐤\)=1−𝔼c∼ω\(⋅∣c′\)\[∏z∈𝒵\(1−Rz​c\)kz\],\\displaystyle\\mathrm\{pass@\}\{k\}\(\\mathbf\{k\}\)=1\-\\mathbb\{E\}\_\{c\\sim\\omega\(\\cdot\\mid c^\{\\prime\}\)\}\\left\[\\prod\_\{z\\in\\mathcal\{Z\}\}\(1\-R\_\{zc\}\)^\{k\_\{z\}\}\\right\],and the PTTS\-RL policy at budgetkkselects the optimal allocation

𝐤\(k\)∈arg​max𝐤:∑zkz=kpass@k\(𝐤\),\\displaystyle\\mathbf\{k\}^\{\(k\)\}\\;\\in\\;\\argmax\_\{\\mathbf\{k\}:\\,\\sum\_\{z\}k\_\{z\}=k\}\\;\\mathrm\{pass@\}\{k\}\(\\mathbf\{k\}\),with residual errorϵkptts≜1−pass​@​k​\(𝐤\(k\)\)\\epsilon^\{\\mathrm\{ptts\}\}\_\{k\}\\;\\triangleq\\;1\-\\mathrm\{pass@\}\{k\}\\bigl\(\\mathbf\{k\}^\{\(k\)\}\\bigr\)and induced mode\-selection distributionπptts\(k\)​\(z∣c′\)≜kz\(k\)/k\\pi^\{\(k\)\}\_\{\\mathrm\{ptts\}\}\(z\\mid c^\{\\prime\}\)\\triangleq k^\{\(k\)\}\_\{z\}/k\. In contrast to GRPO\-RS whose marginal distribution is optimized once againstpass​@​1\\mathrm\{pass@\}\{1\}, PTTS\-RL is optimized againstpass​@​k\\mathrm\{pass@\}\{k\}specifically for the budgetkk, and different attempts within a single budget may use different modes\.

Result 1: GRPO\-RS collapses onto the dominant mode\.Optimizing againstpass​@​1\\mathrm\{pass@\}\{1\}alone leaves no reason to preserve mode diversity: under mode dominance, the optimal marginal policy is unique and deterministic\.

###### Proposition 1\(GRPO\-RS selects onlyz∗z^\{\*\}\)

Under Assumption[1](https://arxiv.org/html/2609.27374#Thmassumption1), the uniquepass​@​1\\mathrm\{pass@\}\{1\}\-optimal marginal policy for the observed categoryc′c^\{\\prime\}is the point mass on the dominant mode,πrs\(⋅∣c′\)=δz∗\\pi\_\{\\mathrm\{rs\}\}\(\\cdot\\mid c^\{\\prime\}\)=\\delta\_\{z^\{\*\}\}\. Consequently, at every inference budgetkk, GRPO\-RS issues allkkattempts with modez∗z^\{\*\}, and its residual error is

ϵkrs=∑c∈𝒞ω⁡\(c∣c′\)​\(1−Rz∗​c\)k\.\\displaystyle\\epsilon^\{\\mathrm\{rs\}\}\_\{k\}=\\sum\_\{c\\in\\mathcal\{C\}\}\\omega\(c\\mid c^\{\\prime\}\)\\,\(1\-R\_\{z^\{\*\}c\}\)^\{k\}\.

Proof\.Sincepass​@​1​\(π\)=∑zπ⁡\(z∣c′\)​R¯z​c′\\mathrm\{pass@\}\{1\}\(\\pi\)=\\sum\_\{z\}\\pi\(z\\mid c^\{\\prime\}\)\\,\\bar\{R\}\_\{zc^\{\\prime\}\}is linear inπ\\piand∑zπ⁡\(z∣c′\)=1\\sum\_\{z\}\\pi\(z\\mid c^\{\\prime\}\)=1,

pass​@​1​\(δz∗\)−pass​@​1​\(π\)=∑z≠z∗π⁡\(z∣c′\)​\(R¯z∗​c′−R¯z​c′\),\\displaystyle\\mathrm\{pass@\}\{1\}\(\\delta\_\{z^\{\*\}\}\)\-\\mathrm\{pass@\}\{1\}\(\\pi\)=\\sum\_\{z\\neq z^\{\*\}\}\\pi\(z\\mid c^\{\\prime\}\)\\left\(\\bar\{R\}\_\{z^\{\*\}c^\{\\prime\}\}\-\\bar\{R\}\_\{zc^\{\\prime\}\}\\right\),which is non\-negative by Assumption[1](https://arxiv.org/html/2609.27374#Thmassumption1)and zero if and only ifπ⁡\(z∣c′\)=0\\pi\(z\\mid c^\{\\prime\}\)=0for everyz≠z∗z\\neq z^\{\*\}; henceδz∗\\delta\_\{z^\{\*\}\}is the unique maximizer\. GRPO\-RS therefore issues every attempt withz∗z^\{\*\}, and substitutingπrs=δz∗\\pi\_\{\\mathrm\{rs\}\}=\\delta\_\{z^\{\*\}\}intopass​@​k​\(πrs\(k\)\)\\mathrm\{pass@\}\{k\}\\left\(\\pi\_\{\\mathrm\{rs\}\}^\{\(k\)\}\\right\)yields the closed form\.□\\square

Result 2: PTTS\-RL diversifies beyond a finite budget\.Optimizing againstpass​@​k\\mathrm\{pass@\}\{k\}directly leads to a qualitatively different behavior: beyond an explicit finite budget, the optimal allocation never concentrates onz∗z^\{\*\}alone\. Letz†∈𝒵z^\{\\dagger\}\\in\\mathcal\{Z\}denote a complementary mode furnished by Assumption[2](https://arxiv.org/html/2609.27374#Thmassumption2), i\.e\.Rz†​c†\>Rz∗​c†R\_\{z^\{\\dagger\}c^\{\\dagger\}\}\>R\_\{z^\{\*\}c^\{\\dagger\}\}\(note this forcesz†≠z∗z^\{\\dagger\}\\neq z^\{\*\}\), and define the complementarity gap on the worst category, the failure rate ofz∗z^\{\*\}on its worst category, and its largest failure rate on any other category:

Δ≜Rz†​c†−Rz∗​c†\>0,q†≜1−Rz∗​c†,ρ≜maxc≠c†⁡\(1−Rz∗​c\)\.\\displaystyle\\Delta\\;\\triangleq\\;R\_\{z^\{\\dagger\}c^\{\\dagger\}\}\-R\_\{z^\{\*\}c^\{\\dagger\}\}\\;\>\\;0,\\qquad q^\{\\dagger\}\\;\\triangleq\\;1\-R\_\{z^\{\*\}c^\{\\dagger\}\},\\qquad\\rho\\;\\triangleq\\;\\max\_\{c\\neq c^\{\\dagger\}\}\\,\(1\-R\_\{z^\{\*\}c\}\)\.
###### Proposition 2\(PTTS\-RL assigns mass outsidez∗z^\{\*\}\)

Under Assumptions[1](https://arxiv.org/html/2609.27374#Thmassumption1)and[2](https://arxiv.org/html/2609.27374#Thmassumption2), define the finite threshold

kptts∗≜2\+⌊ln⁡\(ω⁡\(c†∣c′\)​Δ\)ln⁡\(ρ/q†\)⌋,\\displaystyle k^\{\*\}\_\{\\mathrm\{ptts\}\}\\;\\triangleq\\;2\+\\left\\lfloor\\frac\{\\ln\\left\(\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\\,\\Delta\\right\)\}\{\\ln\\left\(\\rho/q^\{\\dagger\}\\right\)\}\\right\\rfloor,with the conventionkptts∗≜2k^\{\*\}\_\{\\mathrm\{ptts\}\}\\triangleq 2whenρ=0\\rho=0\. Then for every budgetk≥kptts∗k\\geq k^\{\*\}\_\{\\mathrm\{ptts\}\}, every optimal allocation𝐤\(k\)\\mathbf\{k\}^\{\(k\)\}satisfies∑z≠z∗kz\(k\)≥1\\sum\_\{z\\neq z^\{\*\}\}k^\{\(k\)\}\_\{z\}\\geq 1; equivalently, the induced policyπptts\(k\)\\pi^\{\(k\)\}\_\{\\mathrm\{ptts\}\}assigns non\-zero probability mass to modes other thanz∗z^\{\*\}\.

Proof\.The threshold is well\-defined and finite\. First,\|𝒞\|≥2\|\\mathcal\{C\}\|\\geq 2: if𝒞=\{c†\}\\mathcal\{C\}=\\\{c^\{\\dagger\}\\\}, mode dominance would giveRz†​c†=R¯z†​c′<R¯z∗​c′=Rz∗​c†R\_\{z^\{\\dagger\}c^\{\\dagger\}\}=\\bar\{R\}\_\{z^\{\\dagger\}c^\{\\prime\}\}<\\bar\{R\}\_\{z^\{\*\}c^\{\\prime\}\}=R\_\{z^\{\*\}c^\{\\dagger\}\}, contradicting Assumption[2](https://arxiv.org/html/2609.27374#Thmassumption2); henceρ\\rhois a maximum over a non\-empty finite set\. Next,q†≥Δ\>0q^\{\\dagger\}\\geq\\Delta\>0\(asRz†​c†≤1R\_\{z^\{\\dagger\}c^\{\\dagger\}\}\\leq 1\), and0≤ρ<q†0\\leq\\rho<q^\{\\dagger\}sincec†c^\{\\dagger\}is the single worst category ofz∗z^\{\*\}\. Finally,ω⁡\(c†∣c′\)∈\(0,1\)\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\\in\(0,1\)because the posterior is strictly positive on at least two categories, soω⁡\(c†∣c′\)​Δ∈\(0,1\)\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\\,\\Delta\\in\(0,1\)\. Forρ\>0\\rho\>0, both logarithms are therefore negative andkptts∗≥2k^\{\*\}\_\{\\mathrm\{ptts\}\}\\geq 2is finite\.

Now fix anyk≥2k\\geq 2and compare the pure allocation𝐤all\\mathbf\{k\}^\{\\mathrm\{all\}\}, placing allkkslots onz∗z^\{\*\}, against the swapped allocation𝐤swap\\mathbf\{k\}^\{\\mathrm\{swap\}\}, placingk−1k\-1slots onz∗z^\{\*\}and one onz†z^\{\\dagger\}\. Since the corresponding failure products differ only in one factor,

pass​@​k​\(𝐤swap\)−pass​@​k​\(𝐤all\)\\displaystyle\\mathrm\{pass@\}\{k\}\(\\mathbf\{k\}^\{\\mathrm\{swap\}\}\)\-\\mathrm\{pass@\}\{k\}\(\\mathbf\{k\}^\{\\mathrm\{all\}\}\)=∑c∈𝒞ω⁡\(c∣c′\)​\(1−Rz∗​c\)k−1​\(Rz†​c−Rz∗​c\)\\displaystyle=\\sum\_\{c\\in\\mathcal\{C\}\}\\omega\(c\\mid c^\{\\prime\}\)\\,\(1\-R\_\{z^\{\*\}c\}\)^\{k\-1\}\\left\(R\_\{z^\{\\dagger\}c\}\-R\_\{z^\{\*\}c\}\\right\)≥ω⁡\(c†∣c′\)​Δ​\(q†\)k−1−ρk−1,\\displaystyle\\geq\\;\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\\,\\Delta\\,\(q^\{\\dagger\}\)^\{k\-1\}\-\\rho^\{k\-1\},where the last step keeps thec†c^\{\\dagger\}term exactly and bounds every other term via\|Rz†​c−Rz∗​c\|≤1\|R\_\{z^\{\\dagger\}c\}\-R\_\{z^\{\*\}c\}\|\\leq 1,1−Rz∗​c≤ρ1\-R\_\{z^\{\*\}c\}\\leq\\rho, and∑c≠c†ω⁡\(c∣c′\)≤1\\sum\_\{c\\neq c^\{\\dagger\}\}\\omega\(c\\mid c^\{\\prime\}\)\\leq 1\. Ifρ=0\\rho=0, the right\-hand side is positive for everyk≥2=kptts∗k\\geq 2=k^\{\*\}\_\{\\mathrm\{ptts\}\}\. Otherwise it is positive iff\(ρ/q†\)k−1<ω⁡\(c†∣c′\)​Δ\(\\rho/q^\{\\dagger\}\)^\{k\-1\}<\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\\,\\Delta, i\.e\. iffk−1\>ln⁡\(ω⁡\(c†∣c′\)​Δ\)/ln⁡\(ρ/q†\)k\-1\>\\ln\(\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\\Delta\)/\\ln\(\\rho/q^\{\\dagger\}\), since dividing byln⁡\(ρ/q†\)<0\\ln\(\\rho/q^\{\\dagger\}\)<0flips the inequality; this holds for everyk≥kptts∗k\\geq k^\{\*\}\_\{\\mathrm\{ptts\}\}becausekptts∗−1=1\+⌊⋅⌋k^\{\*\}\_\{\\mathrm\{ptts\}\}\-1=1\+\\lfloor\\cdot\\rfloorstrictly exceeds the ratio\. Hence the pure allocation is strictly suboptimal for everyk≥kptts∗k\\geq k^\{\*\}\_\{\\mathrm\{ptts\}\}, and since only finitely many allocations of sizekkexist, every optimal allocation places at least one slot on a mode other thanz∗z^\{\*\}\.□\\square

Note thatkptts∗k^\{\*\}\_\{\\mathrm\{ptts\}\}is a sufficient budget rather than an exact transition point—the proof discards every favorable term outsidec†c^\{\\dagger\}—and it decreases as the worst category becomes more probable \(largerω⁡\(c†∣c′\)\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\) or more fixable \(largerΔ\\Delta\): hedging pays off sooner when the failure mode ofz∗z^\{\*\}is both likely and repairable\.

Result 3: PTTS\-RL reduces error exponentially and strictly faster; GRPO\-RS may not\.Having characterized how each policy allocates its attempts, we now compare their residual errors and the exponential rates at which these decay\. The two policies part ways on both counts: PTTS\-RL drives its error to zero exponentially fast under Assumptions[1](https://arxiv.org/html/2609.27374#Thmassumption1)and[2](https://arxiv.org/html/2609.27374#Thmassumption2)alone, and with a strictly better error exponent, whereas GRPO\-RS converges only when its dominant mode retains some success probability on the worst category\. Throughout, let

r≜1−Rz†​c†q†∈\[0,1\),\\displaystyle r\\;\\triangleq\\;\\frac\{1\-R\_\{z^\{\\dagger\}c^\{\\dagger\}\}\}\{q^\{\\dagger\}\}\\;\\in\\;\[0,1\),which is well\-defined sinceq†\>0q^\{\\dagger\}\>0and below one sinceRz†​c†\>Rz∗​c†R\_\{z^\{\\dagger\}c^\{\\dagger\}\}\>R\_\{z^\{\*\}c^\{\\dagger\}\}by Assumption[2](https://arxiv.org/html/2609.27374#Thmassumption2)\.

###### Proposition 3\(Error rates and strict separation\)

Under Assumptions[1](https://arxiv.org/html/2609.27374#Thmassumption1)and[2](https://arxiv.org/html/2609.27374#Thmassumption2):

1. \(i\)ϵkptts≤ϵkrs\\epsilon^\{\\mathrm\{ptts\}\}\_\{k\}\\leq\\epsilon^\{\\mathrm\{rs\}\}\_\{k\}for every budgetk≥1k\\geq 1\.
2. \(ii\)ω⁡\(c†∣c′\)​\(q†\)k≤ϵkrs≤\(q†\)k\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\\,\(q^\{\\dagger\}\)^\{k\}\\leq\\epsilon^\{\\mathrm\{rs\}\}\_\{k\}\\leq\(q^\{\\dagger\}\)^\{k\}for everyk≥1k\\geq 1, and hence λrs≜limk→∞\(ϵkrs\)1/k=q†\.\\displaystyle\\lambda\_\{\\mathrm\{rs\}\}\\;\\triangleq\\;\\lim\_\{k\\to\\infty\}\\bigl\(\\epsilon^\{\\mathrm\{rs\}\}\_\{k\}\\bigr\)^\{1/k\}\\;=\\;q^\{\\dagger\}\.In particular,ϵkrs→0\\epsilon^\{\\mathrm\{rs\}\}\_\{k\}\\to 0if and only ifRz∗​c†\>0R\_\{z^\{\*\}c^\{\\dagger\}\}\>0; otherwiseϵkrs\\epsilon^\{\\mathrm\{rs\}\}\_\{k\}converges to the plateauω⁡\(c†∣c′\)\>0\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\>0: GRPO\-RS never solves problems from the worst category\.
3. \(iii\)For everyα∈\(0,1\)\\alpha\\in\(0,1\)satisfyingρ1−α<q†\\rho^\{1\-\\alpha\}<q^\{\\dagger\}, which exists sinceρ<q†\\rho<q^\{\\dagger\}, lim supk→∞\(ϵkptts\)1/k≤max⁡\{q†​rα,ρ1−α\}<q†=λrs\.\\displaystyle\\limsup\_\{k\\to\\infty\}\\,\\bigl\(\\epsilon^\{\\mathrm\{ptts\}\}\_\{k\}\\bigr\)^\{1/k\}\\;\\leq\\;\\max\\left\\\{q^\{\\dagger\}r^\{\\alpha\},\\;\\;\\rho^\{\\,1\-\\alpha\}\\right\\\}\\;<\\;q^\{\\dagger\}\\;=\\;\\lambda\_\{\\mathrm\{rs\}\}\.

In particular,ϵkptts→0\\epsilon^\{\\mathrm\{ptts\}\}\_\{k\}\\to 0at least exponentially fast even whenϵkrs\\epsilon^\{\\mathrm\{rs\}\}\_\{k\}does not vanish, and PTTS\-RL strictly improves the error exponent over GRPO\-RS\.

Proof\.*\(i\)*The pure allocation placing allkkslots onz∗z^\{\*\}is feasible for the optimization defining𝐤\(k\)\\mathbf\{k\}^\{\(k\)\}, and itspass​@​k\\mathrm\{pass@\}\{k\}coincides with that of GRPO\-RS by Proposition[1](https://arxiv.org/html/2609.27374#Thmproposition1); optimality of𝐤\(k\)\\mathbf\{k\}^\{\(k\)\}yields the claim\.

*\(ii\)*Start from the closed form of Proposition[1](https://arxiv.org/html/2609.27374#Thmproposition1)\. Sincec†c^\{\\dagger\}is the single worst category ofz∗z^\{\*\}by Assumption[2](https://arxiv.org/html/2609.27374#Thmassumption2), every category satisfies1−Rz∗​c≤q†1\-R\_\{z^\{\*\}c\}\\leq q^\{\\dagger\}; bounding every term accordingly gives the upper bound, and retaining only thec†c^\{\\dagger\}term gives the lower bound, whereω⁡\(c†∣c′\)\>0\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\>0since the posterior is non\-zero for every pair\. Both bounds hold for every value ofRz∗​c†R\_\{z^\{\*\}c^\{\\dagger\}\}, includingq†=1q^\{\\dagger\}=1; takingkk\-th roots and usingω​\(c†∣c′\)1/k→1\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)^\{1/k\}\\to 1givesλrs=q†\\lambda\_\{\\mathrm\{rs\}\}=q^\{\\dagger\}\. IfRz∗​c†\>0R\_\{z^\{\*\}c^\{\\dagger\}\}\>0, thenq†<1q^\{\\dagger\}<1and the upper bound vanishes\. If insteadRz∗​c†=0R\_\{z^\{\*\}c^\{\\dagger\}\}=0, thec†c^\{\\dagger\}term is frozen atω⁡\(c†∣c′\)\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\), while every remaining term vanishes ask→∞k\\to\\inftysince1−Rz∗​c≤ρ<q†=11\-R\_\{z^\{\*\}c\}\\leq\\rho<q^\{\\dagger\}=1; henceϵkrs\\epsilon^\{\\mathrm\{rs\}\}\_\{k\}converges toω⁡\(c†∣c′\)\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\.

*\(iii\)*Fixα∈\(0,1\)\\alpha\\in\(0,1\)withρ1−α<q†\\rho^\{1\-\\alpha\}<q^\{\\dagger\}; suchα\\alphaexists becauseρ1−α→ρ<q†\\rho^\{1\-\\alpha\}\\to\\rho<q^\{\\dagger\}asα→0\+\\alpha\\to 0^\{\+\}\. For everyk\>11−αk\>\\frac\{1\}\{1\-\\alpha\}, so that⌈α​k⌉<k\\lceil\\alpha k\\rceil<k, consider the feasible allocation placing⌈α​k⌉\\lceil\\alpha k\\rceilslots onz†z^\{\\dagger\}and the remainingk−⌈α​k⌉k\-\\lceil\\alpha k\\rceilslots onz∗z^\{\*\}\. Itsc†c^\{\\dagger\}term satisfies, usingr<1r<1and⌈α​k⌉≥α​k\\lceil\\alpha k\\rceil\\geq\\alpha k,

ω⁡\(c†∣c′\)​\(q†\)k−⌈α​k⌉​\(1−Rz†​c†\)⌈α​k⌉≤\(q†\)k​r⌈α​k⌉≤\(q†​rα\)k\.\\displaystyle\\omega\(c^\{\\dagger\}\\mid c^\{\\prime\}\)\\,\(q^\{\\dagger\}\)^\{\\,k\-\\lceil\\alpha k\\rceil\}\\,\(1\-R\_\{z^\{\\dagger\}c^\{\\dagger\}\}\)^\{\\lceil\\alpha k\\rceil\}\\;\\leq\\;\(q^\{\\dagger\}\)^\{k\}\\,r^\{\\lceil\\alpha k\\rceil\}\\;\\leq\\;\\left\(q^\{\\dagger\}r^\{\\alpha\}\\right\)^\{k\}\.Every other term vanishes ifρ=0\\rho=0; otherwise, using1−Rz†​c≤11\-R\_\{z^\{\\dagger\}c\}\\leq 1,1−Rz∗​c≤ρ<11\-R\_\{z^\{\*\}c\}\\leq\\rho<1,⌈α​k⌉≤α​k\+1\\lceil\\alpha k\\rceil\\leq\\alpha k\+1, and∑c≠c†ω⁡\(c∣c′\)≤1\\sum\_\{c\\neq c^\{\\dagger\}\}\\omega\(c\\mid c^\{\\prime\}\)\\leq 1, these terms contribute at most

ρk−⌈α​k⌉≤ρ\(1−α\)​k−1=ρ−1​\(ρ1−α\)k\.\\displaystyle\\rho^\{\\,k\-\\lceil\\alpha k\\rceil\}\\;\\leq\\;\\rho^\{\\,\(1\-\\alpha\)k\-1\}\\;=\\;\\rho^\{\-1\}\\left\(\\rho^\{\\,1\-\\alpha\}\\right\)^\{k\}\.Writingμ≜max⁡\{q†​rα,ρ1−α\}\\mu\\triangleq\\max\\\{q^\{\\dagger\}r^\{\\alpha\},\\,\\rho^\{1\-\\alpha\}\\\}, optimality of𝐤\(k\)\\mathbf\{k\}^\{\(k\)\}over feasible allocations therefore certifies

ϵkptts≤\(1\+ρ−1\)​μkwhen​ρ\>0,ϵkptts≤μkwhen​ρ=0\.\\displaystyle\\epsilon^\{\\mathrm\{ptts\}\}\_\{k\}\\;\\leq\\;\\left\(1\+\\rho^\{\-1\}\\right\)\\mu^\{k\}\\quad\\text\{when \}\\rho\>0,\\qquad\\epsilon^\{\\mathrm\{ptts\}\}\_\{k\}\\;\\leq\\;\\mu^\{k\}\\quad\\text\{when \}\\rho=0\.Takingkk\-th roots giveslim supk\(ϵkptts\)1/k≤μ\\limsup\_\{k\}\(\\epsilon^\{\\mathrm\{ptts\}\}\_\{k\}\)^\{1/k\}\\leq\\mu\. It remains to checkμ<q†\\mu<q^\{\\dagger\}: the first branch satisfiesq†​rα<q†q^\{\\dagger\}r^\{\\alpha\}<q^\{\\dagger\}sincer<1r<1andα\>0\\alpha\>0, and the second satisfiesρ1−α<q†\\rho^\{1\-\\alpha\}<q^\{\\dagger\}by the choice ofα\\alpha\. Sinceμ<q†≤1\\mu<q^\{\\dagger\}\\leq 1, the certificate also showsϵkptts→0\\epsilon^\{\\mathrm\{ptts\}\}\_\{k\}\\to 0at least exponentially fast\.□\\square

The separation quantifies the value of hedging: GRPO\-RS decays exactly at the rate at which its single mode clears the worst category, and wheneverz∗z^\{\*\}is hopeless on that category, repeated sampling replays the same failure and the error plateaus at exactly its posterior mass\. Devoting even a constant fraction of the budget to a complementary mode strictly improves the error exponent, with the residual error dominated by whichever category the mixed allocation covers worst\.

### A\.1Closed\-Form Analysis of a Synthetic Example

Finally, we apply our framework to a two\-category, two\-mode example to derive analytical expressions for bothpass​@​k\\mathrm\{pass@\}\{k\}and mode coverage\. These results are visualized in Figure[2](https://arxiv.org/html/2609.27374#S3.F2)of the main text\.

Instantiation\.Let𝒞=\{c1,c2\}\\mathcal\{C\}=\\\{c\_\{1\},c\_\{2\}\\\}and𝒵=\{z1,z2\}\\mathcal\{Z\}=\\\{z\_\{1\},z\_\{2\}\\\}, with a prior probability ofPr\[c\(𝐱\)=c1\]=p=0\.75\\Pr\[c\(\\mathbf\{x\}\{\}\)=c\_\{1\}\]=p=0\.75\. We consider a “blind” policy as an extreme case of an imperfect policy, where the observed categoryc′​\(𝐱\)c^\{\\prime\}\(\\mathbf\{x\}\{\}\)is independent of the true category\. Consequently, the posterior reduces to the prior:ω⁡\(c1∣c′\)=p=0\.75\\omega\(c\_\{1\}\\mid c^\{\\prime\}\)=p=0\.75andω⁡\(c2∣c′\)=1−p=0\.25\\omega\(c\_\{2\}\\mid c^\{\\prime\}\)=1\-p=0\.25\. We define the success rates as follows:

Rz1​c1=Rz2​c2=a=0\.40,Rz1​c2=Rz2​c1=b=0\.02\.\\displaystyle R\_\{z\_\{1\}c\_\{1\}\}=R\_\{z\_\{2\}c\_\{2\}\}=a=0\.40,\\quad R\_\{z\_\{1\}c\_\{2\}\}=R\_\{z\_\{2\}c\_\{1\}\}=b=0\.02\.Thus, each mode is highly effective on exactly one category\.

This instantiation satisfies the two critical assumptions\. The posterior success rates areR¯z1​c′=p​a\+\(1−p\)​b=0\.305\\bar\{R\}\_\{z\_\{1\}c^\{\\prime\}\}=pa\+\(1\-p\)b=0\.305forz1z\_\{1\}, andR¯z2​c′=p​b\+\(1−p\)​a=0\.115\\bar\{R\}\_\{z\_\{2\}c^\{\\prime\}\}=pb\+\(1\-p\)a=0\.115forz2z\_\{2\}; asR¯z1​c′\>R¯z2​c′\\bar\{R\}\_\{z\_\{1\}c^\{\\prime\}\}\>\\bar\{R\}\_\{z\_\{2\}c^\{\\prime\}\}, Assumption[1](https://arxiv.org/html/2609.27374#Thmassumption1)holds with the dominant modez∗=z1z^\{\*\}=z\_\{1\}\. Forz1z\_\{1\}, then,c†=c2c^\{\\dagger\}=c\_\{2\}is the single worst category that is better served byz†=z2≠z1z^\{\\dagger\}=z\_\{2\}\\neq z\_\{1\}\. Thus, Assumption[2](https://arxiv.org/html/2609.27374#Thmassumption2)also holds\.

GRPO\-RS in closed form\.By Proposition[1](https://arxiv.org/html/2609.27374#Thmproposition1), GRPO\-RS always selects the dominant modez1z\_\{1\}\(see Figure[2\(b\)](https://arxiv.org/html/2609.27374#S3.F2.sf2)\), yielding:

pass​@​k​\(πrs\(k\)\)\\displaystyle\\mathrm\{pass@\}\{k\}\\left\(\\pi\_\{\\mathrm\{rs\}\}^\{\(k\)\}\\right\)=1−p⋅\(1−a\)k−\(1−p\)⋅\(1−b\)k\\displaystyle=1\-p\\cdot\(1\-a\)^\{k\}\-\(1\-p\)\\cdot\(1\-b\)^\{k\}=1−0\.75×0\.6k−0\.25×0\.98k\.\\displaystyle=1\-0\.75\\times 0\.6^\{k\}\-0\.25\\times 0\.98^\{k\}\.As visualized in Figure[2\(a\)](https://arxiv.org/html/2609.27374#S3.F2.sf1), the performance curve quickly saturates onc1c\_\{1\}problems \(as the0\.6k0\.6^\{k\}term decays rapidly to near zero\) and then only slowly improves on the remainingc2c\_\{2\}problems \(due to the slow decay of the0\.98k0\.98^\{k\}term\)\.

PTTS\-RL in closed form\.By Proposition[2](https://arxiv.org/html/2609.27374#Thmproposition2), PTTS\-RL splits the budget across both modes oncekkis large enough\. In this symmetric instance, the optimal allocation placesm⁡\(k\)≈k/2−1\.12m\(k\)\\approx k/2\-1\.12\(rounded to an integer\) slots onz2z\_\{2\}, which becomes non\-zero and breaks the mode collapse atk=4k=4, as visualized in Figure[2\(b\)](https://arxiv.org/html/2609.27374#S3.F2.sf2)\. Thepass​@​k\\mathrm\{pass@\}\{k\}is then derived as:

pass​@​k​\(𝐤\(k\)\)\\displaystyle\\mathrm\{pass@\}\{k\}\\bigl\(\\mathbf\{k\}^\{\(k\)\}\\bigr\)=1−p​\(1−a\)\(k−m⁡\(k\)\)​\(1−b\)m⁡\(k\)−\(1−p\)​\(1−b\)\(k−m⁡\(k\)\)​\(1−a\)m⁡\(k\)\\displaystyle=1\-p\\,\(1\-a\)^\{\\left\(k\-m\(k\)\\right\)\}\(1\-b\)^\{m\(k\)\}\-\(1\-p\)\\,\(1\-b\)^\{\\left\(k\-m\(k\)\\right\)\}\(1\-a\)^\{m\(k\)\}=1−0\.75×0\.6\(k−m⁡\(k\)\)×0\.98m⁡\(k\)−0\.25×0\.98\(k−m⁡\(k\)\)×0\.6m⁡\(k\),\\displaystyle=1\-0\.75\\times 0\.6^\{\\left\(k\-m\(k\)\\right\)\}\\times 0\.98^\{m\(k\)\}\-0\.25\\times 0\.98^\{\\left\(k\-m\(k\)\\right\)\}\\times 0\.6^\{m\(k\)\},which strictly exceedspass​@​k​\(πrs\(k\)\)\\mathrm\{pass@\}\{k\}\\left\(\\pi\_\{\\mathrm\{rs\}\}^\{\(k\)\}\\right\)oncem⁡\(k\)\>0m\(k\)\>0\. Moreover, since PTTS\-RL splits the inference budget approximately evenly \(m⁡\(k\)∼k/2m\(k\)\\sim k/2\), the residual error decays at a rate of approximately\(1−a\)​\(1−b\)=0\.6×0\.98≈0\.77\\sqrt\{\(1\-a\)\(1\-b\)\}=\\sqrt\{0\.6\\times 0\.98\}\\approx 0\.77per attempt, significantly faster than the0\.98k0\.98^\{k\}term of GRPO\-RS, leading to the wide gap in Figure[2\(a\)](https://arxiv.org/html/2609.27374#S3.F2.sf1)\.

## Appendix BTraining Details

For the PTTS\-RL training, we initialize from the Qwen3\-1\.7B/4B\-Base models, using the corresponding Qwen3\-1\.7B or Qwen3\-4B models as executors\. The execution budget is fixed as4​k4ktokens, and the branching factorkkis set as 4\. Our implementation is based onverl\([Sheng et al\., 2024](https://arxiv.org/html/2609.27374#bib.bib16)\)\. We train on the DAPO\-Math\-17K dataset\([Yu et al\., 2026](https://arxiv.org/html/2609.27374#bib.bib27)\)using 8x 80GB NVIDIA A100 GPUs\. Each model is trained for 240 optimization steps, and we report results from the final checkpoint\. Additional hyperparameters are shown in Table[2](https://arxiv.org/html/2609.27374#A2.T2)\.

Table 2:Training hyperparameters\.
## Appendix CAnalysis Details

### C\.1Outline Adherence Analysis

We use GPT\-5\-mini to evaluate how faithfully the executor follows the outline generated by the planner\. Specifically, for each instance, we provide the judge with the problem statement, the planner\-generated outline, and the corresponding executor\-generated response, and ask it to assess whether the executor’s primary reasoning path implements the strategy, decomposition, or perspective specified by the outline\. The detailed prompt is shown in Prompt\. The model assigns a score on a five\-point scale, which we normalize to \[0, 1\] for ease of interpretation and comparison across settings\.

### C\.2Diversity Analysis by Clustering

We quantify the diversity of both outlines and solutions by clustering them into distinct conceptual groups and computing the metrics reported in Appendix[C\.3](https://arxiv.org/html/2609.27374#A3.SS3)\. We apply the same procedure separately to planner outlines and executor solutions\.

Concretely, the clustering procedure consists of two steps\. \(1\)Concept extraction: we prompt GPT\-5\-mini to identify the single most important mathematical or logical concept underlying each planner outlineoio\_\{i\}and each executor response𝐲i\\mathbf\{y\}\_\{i\}\. The extracted concept may be a theorem, formula, invariant, transformation, or canonical solution strategy\. The full prompt is shown in Prompt\. \(2\)Concept clustering: after extracting the concepts independently, we provide GPT\-5\-mini with all extracted concepts for each problem and ask it to group concepts that represent the same underlying mathematical mechanism\. The full prompt is shown in Prompt\.

### C\.3Diversity Metrics

For a given problem, considerkkresponses, eitherkkoutlines sampled from the planner orkksolutions generated by repeated sampling or PTTS\. We use the procedure described above in Appendix[C\.2](https://arxiv.org/html/2609.27374#A3.SS2)to assign each responseiito a clustercic\_\{i\}, and use the following metrics to quantify their diversity:

Overall diversity\.We measure overall diversity by the number of distinct clusters represented among thekkresponses, formally defined as:

𝒞=\{ci∣i=1,…,k\},Overall diversity=\|𝒞\|\.\\displaystyle\\mathcal\{C\}=\\\{c\_\{i\}\\mid i=1,\\ldots,k\\\},\\qquad\\textmd\{Overall diversity\}=\|\\mathcal\{C\}\|\.Intuitively, a larger value indicates that the responses cover a broader range of distinct reasoning approaches\.

Useful diversity\.Overall diversity can be easily hacked by producing responses that are superficially diverse but ultimately unhelpful for solving the problem\. We therefore also measure diversity among responses that eventually lead to a correct answer\. Letri∈\{0,1\}r\_\{i\}\\in\\\{0,1\\\}indicate whether responseiileads to a correct solution, and define

𝒞\+=\{ci∣ri=1\}\.\\displaystyle\\mathcal\{C\}^\{\+\}=\\\{c\_\{i\}\\mid r\_\{i\}=1\\\}\.We normalize the number of distinct clusters among correct responses by the total number of correct responses:

Useful diversity=\|𝒞\+\|/∑i=1kri\.\\displaystyle\\textmd\{Useful diversity\}=\|\\mathcal\{C\}^\{\+\}\|~/~\\sum\_\{i=1\}^\{k\}r\_\{i\}\.Thus, useful diversity is high when correct responses span distinct reasoning approaches, and low when many correct responses concentrate on the same approach\.

### C\.4Ranked\-Averaged Correlation Test

When reporting an aggregated correlation between two metrics, such as diversity andpass​@​k\\mathrm\{pass@\}\{k\}, we haveMMgroups of responses for each problem \(M=16M=16in §[4\.2](https://arxiv.org/html/2609.27374#S4.SS2)\), each with a diversity score and apass​@​k\\mathrm\{pass@\}\{k\}value\. Typically, each problem provides too few points to reliably estimate a correlation on its own\. We therefore leverage data across multiple problems to identify a consistent trend\. However, points from different problems cannot be pooled directly: simply concatenating them introduces problem difficulty as a confounding factor, while directly averaging across problems is not meaningful because there is no natural correspondence between groups from different problems\.

We therefore use a rank\-averaging approach that aligns points across problems by theirrelative positionrather than their raw values\. For a variableaa\(e\.g\., outline diversity\), letaq,ia\_\{q,i\},i=1,…,Mi=1,\\ldots,M, denote its values for problemqq, and letbq,ib\_\{q,i\}denote the corresponding values of a target variablebb\(e\.g\.,pass​@​k\\mathrm\{pass@\}\{k\}\)\. For each problem, we sort theMMgroups byaa\. Letaq,\(j\)a\_\{q,\(j\)\}denote thejj\-th smallest value ofaa, and letbq,\(j\)b\_\{q,\(j\)\}denote thebb\-value of the same group\. We then average both variables across problems at each rankjj:

a¯j=1Q​∑q=1Qaq,\(j\),b¯j=1Q​∑q=1Qbq,\(j\)\.\\displaystyle\\bar\{a\}\_\{j\}=\\frac\{1\}\{Q\}\\sum\_\{q=1\}^\{Q\}a\_\{q,\(j\)\},\\qquad\\bar\{b\}\_\{j\}=\\frac\{1\}\{Q\}\\sum\_\{q=1\}^\{Q\}b\_\{q,\(j\)\}\.Finally, we compute the Spearman correlation between\{a¯j\}j=1M\\\{\\bar\{a\}\_\{j\}\\\}\_\{j=1\}^\{M\}and\{b¯j\}j=1M\\\{\\bar\{b\}\_\{j\}\\\}\_\{j=1\}^\{M\}\.

We apply this procedure to three pairs in §[4\.2](https://arxiv.org/html/2609.27374#S4.SS2): outline diversity vs\. response diversity, outline diversity vs\.pass​@​k\\mathrm\{pass@\}\{k\}, and response diversity vs\.pass​@​k\\mathrm\{pass@\}\{k\}\.

## Appendix DQualitative Examples

This section presents additional qualitative examples, including an example of diversity induced by PTTS in Figure[10](https://arxiv.org/html/2609.27374#A4.F10)and an example of outline adherence and drift in Figure[11](https://arxiv.org/html/2609.27374#A4.F11)\.

Problem:Let△​A​B​C\\triangle ABCbe a right triangle with∠​A=90∘\\angle A=90^\{\\circ\}andB​C=38\.BC=38\.There exist pointsKKandLLinside the triangle such thatA​K=A​L=B​K=C​L=K​L=14\.AK=AL=BK=CL=KL=14\.The area of the quadrilateralB​K​L​CBKLCcan be expressed asn​3n\\sqrt\{3\}for some positive integern\.n\.Findn\.n\.Answer:104104

Repeated Sampling:All four branches develop the same line of reasoning: they placeAAat the origin withBBandCCon the axes, deriveA​B2\+A​C2=1444AB^\{2\}\+AC^\{2\}=1444, and locateKKandLLas intersections of equal\-radius circles\. None resolves the resulting system\. Across the full pool of6464branches, only two distinct solution clusters appear\.𝐏𝐚𝐬𝐬​@​𝟒=𝟎\\mathbf\{Pass@4=0\}

PTTS\-RLBranch 1Outline:Symmetry and Equilateral Triangles: SinceA​K=A​L=B​K=C​L=14AK=AL=BK=CL=14, we can consider that pointsKKandLLform two equilateral triangles with sides of length 14\. … we can infer that the quadrilateralB​K​L​CBKLCis composed of two congruent triangles …Reasoning:First, the problem mentions thatA​K=A​L=B​K=C​L=14AK=AL=BK=CL=14\. So, pointsKKandLLare each 14 units away fromAAand fromBBandCCrespectively\. Also,K​LKLis 14\.The solution outline suggests symmetry and equilateral triangles\.Let me try to visualize this\. …But the outline says that quadrilateralB​K​L​CBKLCis composed of two congruent triangles\.Maybe trianglesB​K​CBKCandL​K​CLKC? Wait, not sure\. Alternatively, maybe trianglesB​K​LBKLandL​K​CLKC? Hmm\. Alternatively,sinceA​K​LAKLis equilateral, andB​K=C​L=14BK=CL=14, maybe there’s some symmetry here\.…Final answer:none\.Correct:×\\timesBranch 2Outline:Coordinate Geometry: We can place the triangle△​A​B​C\\triangle ABCin a coordinate plane, with pointAAat the origin\(0,0\)\(0,0\), pointBBon the x\-axis, and pointCCon the y\-axis\. … calculate the area of quadrilateralB​K​L​CBKLCusing the Shoelace formula or the determinant method …Reasoning:First, let me recall the solution outline\. The problem suggests using coordinate geometry\.So I should place triangleA​B​CABCon a coordinate plane\. Let me try to visualize that\.Since angleAAis90∘90^\{\\circ\}, I can put pointAAat\(0,0\)\(0,0\), pointBBon the x\-axis, soBBis\(b,0\)\(b,0\), and pointCCon the y\-axis, soCCis\(0,c\)\(0,c\)\.Then,B​CBCis the hypotenuse, so the distance betweenBBandCCisb2\+c2=38\\sqrt\{b^\{2\}\+c^\{2\}\}=38\. So that gives me the equationb2\+c2=382=1444b^\{2\}\+c^\{2\}=38^\{2\}=1444\. …So now, coordinates ofKKare\(b/2,196−b2/4\)\(b/2,\\sqrt\{196\-b^\{2\}/4\}\)and coordinates ofLLare\(196−c2/4,c/2\)\(\\sqrt\{196\-c^\{2\}/4\},c/2\)\.Now, we need to use theK​LKLdistance equation …Final answer:none\.Correct:×\\timesBranch 3Outline:Trigonometric Relationships and Angles: … if we denote the angle∠​B​A​K\\angle BAKasθ\\theta, we can use trigonometric identities to express the lengths of the segmentsB​KBKandC​LCLin terms ofθ\\theta\. …Reasoning:Alternatively, maybe I can consider trianglesA​B​KABKandA​L​CALC\. SinceA​K=B​K=14AK=BK=14, triangleA​B​KABKis isoceles withA​K=B​KAK=BK\. …Therefore, angle atAAis angleB​A​KBAK, which I denoted earlier asθ\\theta\. Then, using the law of cosines:… Simplify:196=c2\+196−28​c​cos⁡\(θ\)⇒0=c2−28​c​cos⁡\(θ\)⇒c2=28​c​cos⁡\(θ\)⇒c=28​cos⁡\(θ\)196=c^\{2\}\+196\-28c\\cos\(\\theta\)\\Rightarrow 0=c^\{2\}\-28c\\cos\(\\theta\)\\Rightarrow c^\{2\}=28c\\cos\(\\theta\)\\Rightarrow c=28\\cos\(\\theta\)\. Therefore,cos⁡\(θ\)=c/28\\cos\(\\theta\)=c/28\. …Therefore, coordinates ofKKare\(c/2,14​sin⁡\(θ\)\)=\(14​cos⁡\(θ\),14​sin⁡\(θ\)\)\(c/2,14\\sin\(\\theta\)\)=\(14\\cos\(\\theta\),14\\sin\(\\theta\)\)…Final answer:none\.Correct:×\\timesBranch 4Outline:Geometric Transformations and Similarity: … if we rotate pointKKaround pointAAby120∘120^\{\\circ\}\(a characteristic angle in equilateral triangles\), we can create a new point that might help in understanding the relationship betweenKKandLL\. …Reasoning:The solution outline suggests using geometric transformations, like rotations\. Maybe a rotation by 120 degrees?Since 120 degrees is related to equilateral triangles, which have angles of 60 degrees, and maybe that could help with the distances here\. Let me think\. …Alternatively, maybe there’s a rotation that mapsKKtoLLor something like that\. … Suppose we rotate pointKKaroundAAby 60 degrees\. If that rotation mapsKKtoLL, thenA​K=A​L=14AK=AL=14, and angle betweenA​KAKandA​LALis 60 degrees\. Then triangleA​K​LAKLwould be equilateral, which matches the givenK​L=14KL=14\.So that seems possible\. … But regardless, the area ofB​K​L​CBKLCcame out to be104​3104\\sqrt\{3\}\. Given that the steps are correct and the logic holds, I think that is the answer\. Therefore, the value ofnnis 104\.Final answer:104104\.Correct:✓𝐏𝐚𝐬𝐬​@​𝟒=𝟏\\mathbf\{Pass@4=1\}

Figure 10:Qualitative example of diversity induced by PTTS, generated by PTTS\-RL with Qwen3\-4B\. Reasoning parts that explicitly follow the given outline arehighlighted\. The four reasoning traces cover distinct directions, with one reaching the correct final answer\.Problem:Find the sum of all real numbersrrsuch that there is at least one point where the circle with radiusrrcentered at\(4,39\)\(4,39\)is tangent to the parabola with equation2​y=x2−8​x\+12\.2y=x^\{2\}\-8x\+12\.Answer:5050

Outline:Geometric Distance Approach: Begin by considering the geometric interpretation of tangency between a circle and a parabola\.The distance from the center of the circle\(4,39\)\(4,39\)to any point on the parabola must equal the radiusrrof the circle at the point of tangency\.Define the distance formulabetween the center of the circle and a general point\(x,y\)\(x,y\)on the parabola\. This distance will be expressed in terms ofxxandyy, withyybeing determined by the parabola’s equation\. Since the circle and parabola are tangent,the distance equation will have exactly one solution forxx\. This condition will provide a relationship betweenxx,yy, andrr\.Use the discriminant of the resulting quadratic equation inxxto ensure that there is exactly one solution, which implies thatthe discriminant must be zero\. This will give us a condition onrrwithout needing to solve forxxexplicitly\. Solve the condition onrrobtained from the discriminant to find the possible values ofrr\. The sum of these values will be the answer\.Reasoning:First, I need to recall what it means for a circle and a parabola to be tangent\. Tangent means they touch at exactly one point\. So, the distance from the center of the circle to the parabola must be equal to the radiusrr, and there should be exactly one such point\.The solution outline suggests using the geometric distance approach\. So, I should start by considering a general point\(x,y\)\(x,y\)on the parabola\.Since the parabola’s equation is given as2​y=x2−8​x\+122y=x^\{2\}\-8x\+12, I can expressyyin terms ofxx\. … Dividing both sides by 2:y=\(x2−8​x\+12\)/2y=\(x^\{2\}\-8x\+12\)/2\. So, any point on the parabola can be represented as\(x,\(x2−8​x\+12\)/2\)\(x,\(x^\{2\}\-8x\+12\)/2\)\.Now, the distance from the center of the circle\(4,39\)\(4,39\)to this point\(x,y\)\(x,y\)should be equal to the radiusrr\. The distance formula is\(x−4\)2\+\(y−39\)2=r\\sqrt\{\(x\-4\)^\{2\}\+\(y\-39\)^\{2\}\}=r\.But sinceyyis expressed in terms ofxx, I can substitute that into the distance formula\. … To eliminate the square root, I can square both sides:\(x−4\)2\+\[\(\(x2−8​x\+12\)/2−39\)\]2=r2\(x\-4\)^\{2\}\+\\left\[\\left\(\(x^\{2\}\-8x\+12\)/2\-39\\right\)\\right\]^\{2\}=r^\{2\}\.Now, this equation should have exactly one solution forxxbecause the circle and parabola are tangent at that point\. Therefore, the equation must have exactly one real root\. To find the condition for this, I need to simplify the equation and then find the discriminant of the resulting quadratic \(or higher degree\) equation inxx\.… So, rearranged:x4−16​x3−64​x2\+1024​x\+\(4420−4​r2\)=0x^\{4\}\-16x^\{3\}\-64x^\{2\}\+1024x\+\(4420\-4r^\{2\}\)=0\.Now, this is a quartic equation inxx\. …However, the solution outline says that the equation must have exactly one solution forxx, which would correspond to the case where they are tangent\.However, perhaps the equation is a quartic, which can have multiple roots\. But for tangency, maybe there is a multiple root? So, the quartic equation should have a multiple root, which would mean that the equation and its derivative both equal zero at that root\.But the solution outline says that the discriminant of the resulting quadratic equation inxxmust be zero\. Wait, but the equation we have is quartic\. Hmm, maybe I made a mistake here\. Let me check again\.… But that seems correct\. However, the problem mentions that the discriminant of the resulting quadratic equation inxxmust be zero\. But here, we have a quartic equation\. So maybe there’s a miscalculation here\.Alternatively, perhaps there is a different approach\.… But how do I find the values ofrrfor which the quartic equation has at least one multiple root? … Alternatively, maybe the equation we derived \(the quartic\) can be factored\. Let me check if that’s possible\. … But this seems complicated\. Maybe there’s a better way\. Let me think again\.Alternatively, perhaps I can use calculus\. If the circle and parabola are tangent, then they have a common point and their tangent lines at that point are the same\. So, maybe I can set up the system of equations for the circle and the parabola, and then impose that their derivatives are equal at the point of intersection\. Let me try that approach\.…d​y/d​x=\[2​x−8\]/2=x−4dy/dx=\[2x\-8\]/2=x\-4\. Now, find the derivative of the circle implicitly\. …dy/dx=−\(x−4\)/\(y−39\)dy/dx=\-\(x\-4\)/\(y\-39\)\. At the point of tangency, the derivatives must be equal, so:x−4=−\(x−4\)/\(y−39\)x\-4=\-\(x\-4\)/\(y\-39\)\. Assuming thatx≠4x\\neq 4\(ifx=4x=4, then the derivative of the parabola is 0, and we can check separately\)\. So, ifx≠4x\\neq 4, then we can divide both sides by\(x−4\)\(x\-4\):1=−1/\(y−39\)1=\-1/\(y\-39\)\. Therefore,y−39=−1⇒y=38y\-39=\-1\\Rightarrow y=38\. … Butxxis4±4​54\\pm 4\\sqrt\{5\}, so\(x−4\)2\(x\-4\)^\{2\}is\(±4​5\)2=16⋅5=80\(\\pm 4\\sqrt\{5\}\)^\{2\}=16\\cdot 5=80\. Therefore, the distance is80\+1=81=9\\sqrt\{80\+1\}=\\sqrt\{81\}=9\. Therefore,r=9r=9\. … Ifx=4x=4, then from the parabola equation,y=\(16−32\+12\)/2=\(−4\)/2=−2y=\(16\-32\+12\)/2=\(\-4\)/2=\-2\. So, the point is\(4,−2\)\(4,\-2\)\. The distance from\(4,39\)\(4,39\)to\(4,−2\)\(4,\-2\)is0\+\(39−\(−2\)\)2=412=41\\sqrt\{0\+\(39\-\(\-2\)\)^\{2\}\}=\\sqrt\{41^\{2\}\}=41\. Therefore,r=41r=41\. … Therefore, there are two values ofrr: 9 and 41\. Their sum is9\+41=509\+41=50\. So, the answer should be 50\.But wait, why did the earlier approach with the quartic equation not lead me to this? Because when I derived the quartic equation, I considered that for tangency, the equation has exactly one solution\. However, in reality, there are two different radii: one corresponding to the point\(4,−2\)\(4,\-2\)and another corresponding to the points\(4±4​5,38\)\(4\\pm 4\\sqrt\{5\},38\)\.…Final answer:5050\.Correct:✓

Figure 11:Qualitative example of outline adherence and drift, from PTTS\-RL with Qwen3\-4B\. Reasoning parts that explicitly follow the outline are marked inblue, while parts that deviate are marked inorange\. The executor initially follows the outline but deviates when the outlined approach fails, eventually reaching the correct answer\. Such recovery can assign positive reward to a poor outline, weakening the training signal, as discussed in §[3\.3](https://arxiv.org/html/2609.27374#S3.SS3)\.
## Appendix EPrompts

This section presents the prompts used for planning, execution, and analysis, including the PTTS planner prompt in Prompt, the executor prompt in Prompt, the outline adherence evaluation prompt in Prompt, and the concept extraction and clustering prompts in Promptsand\.

\[SYSTEM\]

Youareanannotatortaskedwithgeneratingmultiplehigh\-levelsolutionoutlinesforamathproblem\.

Yourgoalistoexploredifferentperspectives,strategies,orconceptualapproachesthatcouldbeusedtosolvetheproblem\.Basedonthis,produce\{num\_suboutlines\}distinctoutlinesthatcouldindependentlyguideasolverfromstarttofinish\.

\*YoumustNOTsolvetheproblem\.

\*YoumustNOTcomputevalues,simplifyexpressions,orusealgebra\.

\*Anycalculationmakestheoutputinvalid\.

Formatoutputasanumberedlist\(1\.\-\{num\_suboutlines\}\.\),whereeachitemisanoutline\.

\[USER\]

\{question\}

Prompt 1:Prompt used by the PTTS planner\.\[SYSTEM\]

Youareamathproblemsolver\.Youaregivenamathproblemandasolutionoutline\.Followtheoutlinecarefullytosolvetheproblemstepbystep\.Showyourworkandputyourfinalanswerin\\boxed\{\}\.

\[USER\]

Problem:

\{question\}

SolutionOutline:

\{outline\}

Nowsolvetheproblembyfollowingthisoutline:

Prompt 2:Prompt used by the PTTS executor\.\[SYSTEM\]

Youareanexpertevaluatorformathematicalreasoningand

outline\-conditionedgeneration\.

Youwillbegiven:

1\.Amathproblem\.

2\.Anoutlineprovidedtoamodel\.

3\.Asolutionresponsegeneratedbythemodelconditionedonthatoutline\.

Determinewhetherthesolutionresponsefollowstheprovidedoutline\.

Evaluationrules:

\-Donotassignahighfollowingscoremerelybecausethesolutioniscorrect\.

\-Donotassignalowfollowingscoremerelybecausethesolutionisincorrect\.

\-Aresponsefollowsanoutlinewhenitsprimaryreasoningpathusesthestrategy,decomposition,orperspectivedescribedbytheoutline\.

\-Aresponsemayintroduceadditionaldetailswhilestillfollowingtheoutline,providedthatthosedetailsareconsistentwiththeoutlinedapproach\.

\-Aresponsethatusesasubstantiallydifferentmethodshouldreceivealowfollowingscore,evenifitreachesthecorrectanswer\.

\-Iftheoutlineorresponseistoounclear,malformed,orirrelevanttoassess,useCANNOT\_JUDGE\.

Outlinefollowingrubric:

5=FOLLOWS\_EXACTLY:Clearlyfollowstheintendedstrategy\.

4=MOSTLY\_FOLLOWS:Mainlyfollowstheoutlinebutadds,skips,orchangesminorsteps\.

3=PARTIALLY\_FOLLOWS:Usessomeideasfromtheoutlinebutsubstantiallydeviates\.

2=DOES\_NOT\_FOLLOW:Usesadifferentstrategyorlargelyignorestheoutline\.

1=CANNOT\_JUDGE:Theoutlineorresponseistoounclear,malformed,orirrelevanttoassess\.

ReturnonlyvalidJSONusingthefollowingschema:

\{

"outline\_following\_score":<integerfrom1to5\>,

"outline\_following\_label":

"<FOLLOWS\_EXACTLY\|MOSTLY\_FOLLOWS\|PARTIALLY\_FOLLOWS\|

DOES\_NOT\_FOLLOW\|CANNOT\_JUDGE\>",

"outline\_following\_note":"<briefexplanation\>",

\}

\[USER\]

\[QUESTION\]

\{question\}

\[/QUESTION\]

\[OUTLINE\]

\{outline\}

\[/OUTLINE\]

\[OUTLINE\_CONDITIONED\_SOLUTION\]

\{response\}

\[/OUTLINE\_CONDITIONED\_SOLUTION\]

Evaluatetheoutline\-followingbehavioraccordingtothesysteminstructions\.ReturnonlyvalidJSON\.

Prompt 3:Prompt used to evaluate outline following\.Youaregivenamathproblemandafullsolutionresponseoroutlinegeneratedbyamodel\.

Theresponsemaybecorrectorincorrect\.

YourtaskistoidentifytheSINGLEmostimportantmathematicalorlogicalconcept,theorem,canonicalformula,invariant,transformation,orstrategythatmakestheattemptedsolutionoroutlinepossible\.

Important:

\-Summarizetheconceptactuallyusedorattemptedintheresponse\.

\-DoNOTrepairtheresponseintoabetterconcept\.

\-DoNOTintroduceaconceptthatisnotsupportedbytheresponse\.

\-Iftheresponseisflawed,stillidentifythekeyconcepttheresponseattemptedtouse\.

\-Iftheresponseusesmultipleconcepts,choosetheonewithoutwhichtheattemptedsolutionwouldnotwork,usuallythefirstpivotalstep\.

\-Choosethenarrowestconceptthatstillcoverstheattemptedsolution\.

\-Good:"PythagoreanTheorem","Vieta’sFormulas","ModularInvariant","ComplementCounting"\.

\-Bad:"Geometry","Algebra","NumberTheory","Counting"\.

Definitionofavalidkeyconcept:

\-Itshouldbeaspecificmathematicalconcept,theorem,formula,invariant,transformation,orcanonicalstrategy\.

\-Itshouldexplainthecoremechanismoftheattemptedsolution\.

\-Itshouldnotbeafulloutline\.

\-Itshouldnotincludethefinalanswer\.

\-Itshouldnotincludedetailedcomputations,equationchains,ornumericalderivations\.

\-Itshouldbespecificenoughtodistinguishthissolutionpathfromotherpossibleapproaches\.

Formattingrules:

\-TheconceptnameshoulduseTitleCaseandsingularformwhenpossible\.

\-Preferstandardmathematicalnameswhenavailable\.

\-Ifnostandardtheoremapplies,useaconcisestrategylabel,suchas"CaseworkonRemainders","SymmetryReduction","ComplementCounting",or"BoundingArgument"\.

\-Donotuseoverlybroadlabelssuchas"Algebra","Geometry","Combinatorics",or"NumberTheory"unlesstheresponseistoouncleartoidentifyanarrowerconcept\.

ReturnONLYvalidJSONinthefollowingformat:

\{\{

"concept":"<singlemostimportantconceptname\>",

"evidence":"<oneshortsentenceexplainingwhythisconceptispivotalintheattemptedresponse\>",

"confidence":<integerfrom1to5\>

\}\}

Confidencescore:

5=clearlyidentifiableandspecificconcept\.

4=mostlyclearconcept,withminorambiguity\.

3=plausibleconcept,butresponseusesseveralcompetingideasorispartiallyunclear\.

2=weakguessbecausetheresponseisflawed,vague,orinconsistent\.

1=cannotmeaningfullyidentifyaconcept\.

\[QUESTION\]

\{question\}

\[/QUESTION\]

\[FULL\_SOLUTION\_RESPONSE\]

\{response\}

\[/FULL\_SOLUTION\_RESPONSE\]

Prompt 4:Prompt used to extract the pivotal concept from each executor response\.Youaregivenamathproblemandalistofcandidatekeyconceptsextractedfrommodel\-generatedsolutionresponsesoroutlines\.

Somesourceresponsesmaybecorrectandsomemaybeincorrect\.

Yourtaskistoclusterthecandidateconceptsbytheunderlyingmathematicalorlogicalconceptusedintheattemptedsolution\.

Clusteringrules:

\-Groupconceptstogetheriftheyrefertoessentiallythesamepivotalmathematicalidea,theorem,invariant,transformation,orcanonicalstrategy\.

\-Mergesynonymousornear\-synonymousnames\.

\-Example:"ModuloInvariant","InvariantModulo5",and"ResidueClassInvariant"maybelongtogetheriftheydescribethesameattemptedidea\.

\-Separateconceptsiftheyrepresentmeaningfullydifferentsolutionmechanisms,eveniftheyarefromthesamebroadfield\.

\-Example:"Vieta’sFormulas"and"DiscriminantCondition"shouldusuallybeseparated\.

\-Example:"ComplementCounting"and"Inclusion\-ExclusionPrinciple"shouldbeseparatedunlesstheevidenceshowstheyrefertothesamepivotalstep\.

\-Clusterbyattemptedconcept,notbywhetherthesourceresponsewascorrect\.

\-Aclustermaycontainconceptsextractedfrombothcorrectandincorrectsourceresponses\.

\-DoNOTrepairanincorrectorvagueconceptintoabetterone\.

\-DoNOTintroduceanewconceptthatisnotrepresentedbytheclustermembers\.

\-Preferthenarrowestcanonicalconceptnamethatcoversthecluster\.

Foreachcluster,write:

\-"canonical\_concept":thestandardorclearestconceptname\.

\-"member\_indices":theindicesofcandidateconceptsinthiscluster\.

\-"canonical\_description":oneshortsentencedescribingthesharedconcept\.

Canonicalconceptrules:

\-UseTitleCaseandsingularformwhenpossible\.

\-Preferstandardmathematicalnameswhenavailable\.

\-Ifnostandardtheoremapplies,useaconcisestrategylabel,suchas"CaseworkonRemainders","SymmetryReduction","ComplementCounting",or"BoundingArgument"\.

\-Donotuseoverlybroadlabelssuchas"Algebra","Geometry","Combinatorics",or"NumberTheory"unlesstheclusterisgenuinelytoovaguetoidentifyanarrowerconcept\.

\-Donotincludefinalanswers,computations,equationchains,orproblem\-specificnumericalvalues\.

ReturnONLYvalidJSONinthisexactschema:

\{\{

"clusters":\[

\{\{

"cluster\_id":1,

"canonical\_concept":"<TitleCaseconceptname\>",

"member\_indices":\[1,3\],

"canonical\_description":"<oneshortsentencedescribingthesharedconcept\>"

\}\}

\]

\}\}

Example:

\[EXAMPLE\_QUESTION\]

Findthenumberofpositiveintegersnsatisfyingacertaindivisibilitycondition\.

\[/EXAMPLE\_QUESTION\]

\[EXAMPLE\_CANDIDATE\_CONCEPTS\]

1\.Concept:ModularArithmeticFiltering

Evidence:Thesolutionrestrictspossiblevaluesofnusingcongruencesmodulosmallintegers\.

2\.Concept:CaseworkonParity

Evidence:Thesolutionsplitsnintooddandevencasesandeliminatesimpossiblecases\.

3\.Concept:CongruenceClassFiltering

Evidence:Thesolutionusesresiduesmodulosmallprimestonarrowthecandidatevalues\.

4\.Concept:ExhaustiveEnumeration

Evidence:Thesolutionchecksallfeasiblecandidatesdirectlyagainstthecondition\.

\[/EXAMPLE\_CANDIDATE\_CONCEPTS\]

\[EXAMPLE\_EXPECTED\_OUTPUT\]

\{\{

"clusters":\[

\{\{

"cluster\_id":1,

"canonical\_concept":"ModularArithmeticFiltering",

"member\_indices":\[1,3\],

"canonical\_description":"Usecongruenceconstraintstonarrowthepossiblevaluesbeforecheckingtheoriginalcondition\."

\}\},

\{\{

"cluster\_id":2,

"canonical\_concept":"CaseworkOnParity",

"member\_indices":\[2\],

"canonical\_description":"Splitthevariableintoparitycasesandanalyzeeachcaseseparately\."

\}\},

\{\{

"cluster\_id":3,

"canonical\_concept":"ExhaustiveEnumeration",

"member\_indices":\[4\],

"canonical\_description":"Checkallfeasiblecandidatesdirectlyagainsttherequiredcondition\."

\}\}

\]

\}\}

\[/EXAMPLE\_EXPECTED\_OUTPUT\]

Nowclustertheactualcandidateconceptsbelow\.

\[QUESTION\]

\{question\}

\[/QUESTION\]

\[CANDIDATE\_CONCEPTS\]

\{candidate\_concepts\}

\[/CANDIDATE\_CONCEPTS\]

Prompt 5:Prompt used to cluster response concepts for each problem\.
## Appendix FAI Assistants In Research Or Writing

We used AI assistants solely for stylistic improvements in writing, such as improving clarity, grammar, and phrasing\. We did not use AI assistants for coding, brainstorming, research design, data analysis, interpretation of results, or any other critical intellectual contribution\.