Hermes: Learning Contextual Reasoning Unlocks Test-Time Scaling

# Learning Contextual ReasoningUnlocks Test-Time Scaling
Source: [https://arxiv.org/html/2609.38332](https://arxiv.org/html/2609.38332)
\\metadata

DateSeptember 29, 2026\\paperlogos

## Hermes: Learning Contextual Reasoning Unlocks Test\-Time Scaling

Mononito Goswami2,†Hao Liu2Nikos Kanakaris2Langlin Huang3,‡ Prithwish Jana4,‡Patrick Blöbaum2Purak Jain21Carnegie Mellon University2AWS AI Labs3Washington University in St\. Louis4Georgia Institute of TechnologyNote:With $T=3$, $B=3$, $D=1$, and $W=1$, the root agent uses one 8K\-token window and may launch up to six subagents, each with its own 8K\-token window\. This yields a maximum of $7×8K=56K$ tokens\.Note:With $T=3$ and $B=3$,Hermesuses one root context and up to six subagent contexts, for a maximum of $7×8K=56K$ tokens\. We include this baseline to distinguish gains from reasoning across contexts from those obtained by simply providing a larger single context\.Note:For RSA, we use its summarizer to compact candidate solutions before aggregation\.Note:[https://github\.com/huggingface/Math\-Verify](https://github.com/huggingface/Math-Verify)Note:[https://huggingface\.co/Qwen/Qwen3\-4B\-Instruct\-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507)Note:[https://huggingface\.co/allenai/Olmo\-3\-7B\-Instruct](https://huggingface.co/allenai/Olmo-3-7B-Instruct)

###### Abstract

Test\-time scaling improves model performance by allocating additional compute during inference\. Using this compute effectively across multiple context windows requires deciding how to allocate fresh contexts and what information to carry between them\. We call a model’s ability to make these decisions*contextual reasoning*\. Existing approaches largely prescribe these decisions through their harness; we instead shift them to the model\. We introduce 1\)Hermes, a family of simple, configurable harnesses that progressively varies model control over context allocation and reuse, and 2\)Hermes\-Learn, a two\-stage framework for learning these capabilities\. We find that capable models can exploit this flexibility to scale with additional inference\-time compute, while smaller open\-source models initially struggle to do so\. Training withHermes\-Learncloses this gap, inducing adaptive contextual reasoning strategies that vary with both the problem and the progress of reasoning\. These gains generalize across benchmarks and models, extrapolate beyond the inference\-time compute seen during training, and transfer to complementary test\-time scaling methods beyondHermes\.

††footnotetext:†Xinyu Li and Mononito Goswami contributed equally\. Correspondence to[xinyul2@andrew\.cmu\.edu](mailto:[email protected])or[mononito@amazon\.com](mailto:[email protected])\.††footnotetext:‡Xinyu Li, Langlin Huang, and Prithwish Jana were interns at AWS when this work was carried out\.## 1Introduction

> “The capacity of the human mind for formulating and solving complex problems is very small compared with the size of the problems …” — Herbert A\. Simon, Models of Man\[[1](https://arxiv.org/html/2609.38332#bib.bib2)\]

Large language models have made impressive progress on complex reasoning, increasingly contributing to mathematical research\[[2](https://arxiv.org/html/2609.38332#bib.bib4),[3](https://arxiv.org/html/2609.38332#bib.bib3)\]\. This progress is fueled by*test\-time scaling*: allocating additional computation during inference to improve model performance\[[4](https://arxiv.org/html/2609.38332#bib.bib5)\]\. Recent advances have involved thousands of agents reasoning in separate contexts, with harnesses coordinating how those contexts are allocated and how intermediate results are carried forward\. This highlights two complementary dimensions of reasoning\. Mathematical reasoning determines how to make progress given the information available within a context, while*contextual reasoning*determines how separate contexts should be allocated, coordinated, and reused as reasoning unfolds\. In this work, we focus on contextual reasoning\.

Concretely, contextual reasoning requires two types of decisions: i\) How should additional contexts be used in order to explore an alternative solution, verify prior work, or continue an existing line of reasoning? ii\) What information should be carried forward into those contexts? Existing approaches typically encode these decisions in their harnesses\. For example, Reasoning Cache repeatedly compresses prior reasoning and continues from the resulting summary in a fresh context\[[5](https://arxiv.org/html/2609.38332#bib.bib13)\], while Recursive Agent Optimization uses recursive delegation to create fresh contexts for subproblems and learns when and how to invoke them\[[6](https://arxiv.org/html/2609.38332#bib.bib12)\]\. Yet the best use of additional contexts may change across stages of problem solving\. For instance, a difficult proof may initially benefit from exploring several approaches in parallel, then verifying a promising intermediate result, and finally continuing a successful line of reasoning\. This motivates a flexible harness for contextual reasoning that exposes multiple ways to use additional contexts and allows the model to choose among them based on the current state of the solution\.

To this end, we introduceHermes, a family of simple, flexible, and configurable harnesses that progressively shifts contextual reasoning control from the harness to the model\.Hermesexposes two generic mechanisms:*delegation*, for allocating fresh context, and*digestion*, for carrying useful information forward\. By varying how strongly the harness steers these decisions,Hermesdefines a hierarchy ranging from prescribed strategies to increasingly flexible and model\-driven contextual reasoning\. We observe that capable models can exploit this flexibility, while small open\-source models perform worse underHermesthan when reasoning inline, revealing a contextual reasoning gap\. This motivates a learning framework for developing flexible contextual reasoning\. Crucially, the same hierarchy provides a natural learning curriculum: models first learn under strongly steered harnesses with a simplified contextual reasoning decision space, then progressively assume greater control as harness steering is reduced during training\. We therefore introduceHermes\-Learn, a two\-stage framework for learning flexible contextual reasoning\. Together,HermesandHermes\-Learnprovide a rigorous framework for studying contextual reasoning and a practical approach for teaching models to acquire this distinct capability\.

We evaluateHermesandHermes\-Learnacross model families and scales, from Qwen3\-4B to frontier models, on mathematical reasoning benchmarks spanning AIME, HMMT, BeyondAIME\[[7](https://arxiv.org/html/2609.38332#bib.bib8)\], and IMO\-AnswerBench\[[8](https://arxiv.org/html/2609.38332#bib.bib9)\], as well as the out\-of\-domain FrontierScience benchmark\[[9](https://arxiv.org/html/2609.38332#bib.bib11)\]\. We deliberately constrain each context to 8K tokens, creating a setting where effective long\-horizon reasoning requires models to allocate and reuse context well\. Our results show that: \(1\)Hermesunlocks test\-time scaling for capable models, while revealing a substantial contextual reasoning gap in smaller open\-source models; \(2\)Hermes\-Learncloses this gap, improving contextual reasoning without degrading the model’s mathematical reasoning; \(3\) trained models learn adaptive contextual reasoning strategies that vary with both the problem structure and the reasoning progress; and \(4\) the learned contextual reasoning capabilities extend beyond the inference\-time compute seen during training and transfer to complementary test\-time scaling methods beyondHermes\.

##### Contributions

\(1\)Contextual reasoning\.We study contextual reasoning as a model’s capability to organize and sustain reasoning over long horizons\.\(2\)Hermes\.We introduce a simple, configurable family of harnesses built on delegation and digestion, with a steering hierarchy that progressively shifts control from the harness to the model\.\(3\)A learning framework to close contextual reasoning gap\.We show that capable models can turn additional inference\-time compute into better performance underHermes, while smaller open\-source models struggle despite reasonable mathematical reasoning\. We introduceHermes\-Learn, a two\-stage framework combining supervised fine\-tuning and curriculum reinforcement learning to close this gap by learning to select and compose delegation strategies\.\(4\)Adaptive and generalizable contextual reasoning\.Learned strategies adapt to the problem and current reasoning state, generalize across benchmarks and model families, scale beyond the inference\-time compute seen during training, and transfer to complementary test\-time scaling methods\.

## 2Related Work

##### Contextual reasoning

Prior test\-time scaling approaches typically allocate additional contexts through inference primitives such as sequential reasoning\[[10](https://arxiv.org/html/2609.38332#bib.bib15),[11](https://arxiv.org/html/2609.38332#bib.bib18)\], parallel exploration\[[12](https://arxiv.org/html/2609.38332#bib.bib16),[13](https://arxiv.org/html/2609.38332#bib.bib17)\], and aggregation\[[14](https://arxiv.org/html/2609.38332#bib.bib19)\], or compose multiple primitives within a single inference procedure\[[15](https://arxiv.org/html/2609.38332#bib.bib20),[16](https://arxiv.org/html/2609.38332#bib.bib22)\]\. Another line of work uses search to guide context allocation, such as tree\-based search\[[17](https://arxiv.org/html/2609.38332#bib.bib7),[18](https://arxiv.org/html/2609.38332#bib.bib35)\]using self\-reflection and verifier\-guided search\[[19](https://arxiv.org/html/2609.38332#bib.bib34)\]using process reward models\. When reasoning extends across multiple context windows, existing work primarily relies on delegation and compaction to coordinate context allocation and reuse\. Prior methods typically prescribe specific delegation strategies, such as delegating recursively decomposed tasks\[[20](https://arxiv.org/html/2609.38332#bib.bib25)\], recursively spawned work\[[21](https://arxiv.org/html/2609.38332#bib.bib26)\], or concurrently executable structures\[[22](https://arxiv.org/html/2609.38332#bib.bib27)\]to fresh contexts\. A complementary line of work\[[23](https://arxiv.org/html/2609.38332#bib.bib28),[24](https://arxiv.org/html/2609.38332#bib.bib14)\]focuses on compressing prior reasoning into compact states for reuse in later contexts\.

While prior work relies on external search procedures or prescribes contextual reasoning strategies in the inference procedure,Hermesstudies contextual reasoning as a model capability\.Hermesenables models to adaptively choose among multiple strategies for allocating additional contexts as reasoning progresses\. Moreover, our experiments \(Sec\.\) show that this capability can also benefit complementary test\-time scaling methods that use search with self\-reflection\.

Table 1:Comparison ofHermeswith existing methods\. Prior methods mostly prescribe contextual reasoning strategies for allocating and reusing contexts, then train models for that particular harness\.Hermes\-Learninstead trains models to select among different contextual reasoning strategies through the learning curriculum induced by the varying steering of theHermeshierarchy\.Model\-driven strategy selectioncaptures whether contextual reasoning strategies are decided by the model rather than by the harness;Context compaction, whether prior reasoning can be compressed and reused;Recursive delegation, whether further contexts can be allocated recursively through the same delegation interface\.MethodModel\-drivenstrategy selectionContextcompactionRecursivedelegationSPIRAL✗✗✗PaCoRe✗✓✗RAO✗✗✓Context\-Folding✗✓†✗Reasoning Cache✗✓✗Hermes✓✓✓

†Context\-Folding compacts a delegated branch when it returns\.

##### Learning contextual reasoning

Recent work shows that explicitly training models to use the test\-time inference procedures can improve test\-time scaling\. For example,MRTshapes long reasoning trajectories with progress\-sensitive rewards\[[25](https://arxiv.org/html/2609.38332#bib.bib31)\], ande3trains models to compose generation, verification, and refinement during exploration\[[26](https://arxiv.org/html/2609.38332#bib.bib32)\]\. Other methods train models for particular inference harnesses:SPIRALtrains parallel generation and aggregation\[[27](https://arxiv.org/html/2609.38332#bib.bib21)\],PaCoRetrains synthesis across parallel branches\[[15](https://arxiv.org/html/2609.38332#bib.bib20)\],RAOtrains recursive delegation\[[6](https://arxiv.org/html/2609.38332#bib.bib12)\],Context\-Foldingtrains delegation of subtasks to fresh branches together with compaction of intermediate reasoning\[[28](https://arxiv.org/html/2609.38332#bib.bib29)\], andReasoning Cachetrains continuation from compacted reasoning progress\[[5](https://arxiv.org/html/2609.38332#bib.bib13)\]\. While these methods demonstrate the value of aligning training with inference harnesses, they typically encode contextual reasoning strategies in the harness, leaving models to learn how to execute them effectively \(App\.\)\. In contrast,Hermes\-Learnteaches models to autonomously select, instantiate, and compose different contextual reasoning strategies, through the natural curriculum induced by the progressively reduced steering of theHermeshierarchy\.

## 3Hermes: A Family of Simple, Flexible Harnesses

![Refer to caption](https://arxiv.org/html/2609.38332v1/figures/hermes_overview.png)Figure 1:Overview ofHermes\.Hermesenables contextual reasoning across multiple context windows through two mechanisms:*delegation*, which allocates fresh contexts to search or verification strategies, and*digestion*, which compacts accumulated progress for reuse in later contexts\. TheHermeshierarchy \(L3–L1\) progressively shifts control over delegation decisions from the harness to the model\.L3prescribes the strategy;L2prescribes a strategy category \(search or verification\) while allowing the model to select strategies within it; andL1lets the model choose both as reasoning unfolds\.In this section, we introduceHermesand evaluate how different language models can leverage its harnesses for test\-time scaling\.

##### Desiderata

We designHermesto be*simple*,*flexible*, and*configurable*\.*Simple*:Hermesintroduces only two generic mechanisms,*delegation*and*digestion*to allocate fresh contexts and carry useful information across them\.*Flexible*: these mechanisms do not prescribe a fixed form of computation; instead, they allow the model to decide how additional contexts should be used as reasoning unfolds\.*Configurable*: steering controls how much guidance the harness provides for delegation, from prescribing a specific strategy to letting the model choose one itself\.

##### Contextual reasoning via delegation and digestion

Hermesgives a model two ways to continue reasoning beyond a single context window\. It can assign tasks to fresh contexts and use the results, or carry its own progress into a fresh context and continue working\. We call these operations*delegation*and*digestion*, respectively\. Delegation and digestion are complementary primitives for contextual reasoning, expanding reasoning across contexts in parallel and sequentially \(Fig\.\)\.

Within each context window,Hermesruns a ReAct\-style interaction loop\[[29](https://arxiv.org/html/2609.38332#bib.bib30)\]\(Fig\.\)\. At steptt, the agent receives an inpututu\_\{t\}, such as the original problem, a message from the harness, or a result from a subagent\. It then produces an actionata\_\{t\}: it may continue reasoning, return an answer, or calllaunch\_subagent\(task\)\. This tool is how the agent delegates work\. The harness invokes the same model on the specified task in a fresh context and returns a compact report as the agent’s next input\. We call this model working on the specified task a*subagent*\. Formally,ht=\(u1,a1,…,ut\)h\_\{t\}=\(u\_\{1\},a\_\{1\},\\ldots,u\_\{t\}\)is the history available when the agent choosesata\_\{t\}; under harnessHHwith configurationcc, the action is sampled fromπθ\(⋅∣ht;Hc\)\\pi\_\{\\theta\}\(\\cdot\\mid h\_\{t\};H\_\{c\}\)\. An agent has up toTTsteps in each context window\. At each of the firstT−1T\-1steps, it may launch up toBBsubagents; at stepTT, it can no longer delegate and returns its best current answer\. Subagents use the same delegation interface, so they may launch further subagents up to depthDD\. At the maximum depth, they reason inline without further delegation\. The interaction can also end earlier if the context window is exhausted\.

Digestion allows the agent to continue*its own*work across windows\. At the end of a window, it compresses its accumulated progress into a digest\. If reasoning continues, the harness starts a fresh window with the original problem and that digest, rather than the full interaction history, for up toWWwindows\. Thus,BBandDDgovern how work branches across contexts in parallel, whileWWgoverns how long it can continue sequentially\.

##### Steerable control over delegation

Effective delegation requires more than sound mathematical reasoning\. An agent must also decide when to delegate, what task to assign, and how to use each fresh context, for example by independently searching for a solution, exploring a different approach, decomposing the problem, or verifying existing reasoning\. These contextual reasoning decisions can be challenging, particularly for smaller open\-source models\.Hermestherefore varies how strongly the harness steers delegation, progressively shifting these decisions from the harness to the model\. We organize delegation strategies into two broad categories:*search*through independent attempts on the full problem, alternative approaches, or decomposition into simpler subproblems, and*verification*of complete solutions, specific reasoning steps, or disagreements between candidate solutions\. These categories induce three steering levelsℓ∈\\ell\\in\{L3,L2,L1\} \(Fig\.\)\. AtL3, the harness specifies a strategy and the agent instantiates it as a concrete task\. AtL2, the harness specifies only the strategy category, while the agent selects one or more strategies within it\. AtL1, the agent selects from the full strategy set, deciding how to formulate tasks for its subagents as reasoning unfolds\. Thus,L3–L1progressively shift control over delegation from the harness to the agent, while providing a natural curriculum for learning these contextual reasoning decisions\. Several prior methods can be viewed as fixed\-strategy special cases ofHermes\[[5](https://arxiv.org/html/2609.38332#bib.bib13),[6](https://arxiv.org/html/2609.38332#bib.bib12),[27](https://arxiv.org/html/2609.38332#bib.bib21)\]\.

Together, these parameters define a harness configurationc=\(T,B,D,W,ℓ\)c=\(T,B,D,W,\\ell\), makingHermesconfigurable in both how much additional context is available and how much control over its use is given to the model\. Additional details can be found in App\.\.

##### Test\-time scaling withHermesrelies on contextual reasoning

Hermesprovides additional test\-time compute, but models must use it effectively to improve performance\. We compareQwen3\-4B\-Instruct\-2507\(hereafterQwen3\-4B\), our base model, with ClaudeSonnet5andDeepSeek\-V3\.2under*inline reasoning*andHermesL1andL2\. Inline reasoning measures mathematical

Table 2:Hermesenables test\-time scaling for capable models\.Avg​@​16\\mathrm\{Avg\}@16\(%,↑\\uparrow\) under inline reasoning andHermesL1andL2on the pooled and synthetic benchmarks\. Inline reasoning measures mathematical reasoning within a single 8K\-token context, whileHermesprovides up to 56K tokens across contexts\. ClaudeSonnet 5andDeepSeek\-V3\.2match or improve over inline reasoning withHermes;Qwen3\-4Bdegrades, exposing a contextual reasoning gap\.Pooled \(n=238n=238\)Synthetic \(n=88n=88\)ModelInlineL1L2InlineL1L2Qwen3\-4B37\.116\.215\.150\.726\.826\.0Sonnet 555\.376\.776\.587\.094\.796\.2DeepSeek\-V3\.256\.258\.457\.478\.186\.283\.7

reasoning within a single 8K\-token context, where a non\-thinking model solves the problem directly without delegation or digestion\. For bothHermesharnesses, we useT=3T=3,B=3B=3,D=1D=1, andW=1W=1, with an 8K\-token context per agent\. We setD=1D=1andW=1W=1, so only the root agent delegates and digestion is disabled\. WithT=3T=3andB=3B=3,Hermescan use up to 56K tokens of total context, compared with 8K tokens for inline reasoning\. We evaluate whether models can translate this additional context into improved performance on a pooled set of AIME and HMMT problems \(App\.\) and a synthetic benchmark constructed to benefit from different contextual reasoning strategies \(App\.\)\.

As shown in Tab\., ClaudeSonnet 5andDeepSeek\-V3\.2match or exceed their inline performance withHermes, showing that they can combine additional context with mathematical reasoning effectively\. In contrast,Qwen3\-4Bdegrades substantially despite reasonable inline mathematical reasoning, revealing a gap in the contextual reasoning needed to benefit from additional context\. On the synthetic benchmark,Sonnet 5also performs slightly better under the more prescriptiveL2harness than underL1, suggesting that even capable models may benefit from different levels of steering\. These results motivate learning contextual reasoning capabilities directly\.

## 4Learning Contextual Reasoning

Hermesexposes several learning problems in contextual reasoning\. Given the current historyhth\_\{t\}, a model must learn*what to delegate*to fresh contexts,*what to preserve*when generating a digest, and*how to reason*from the resulting digest\. Prior work has studied the latter two problems: models can be trained to continue reasoning from compressed histories\[[5](https://arxiv.org/html/2609.38332#bib.bib13),[30](https://arxiv.org/html/2609.38332#bib.bib33)\], while recent instruction\-tuned models already exhibit summarization capabilities that enable effective digest generation in many cases\. Further improving digestion through downstream task outcomes can suffer from difficult credit assignment across multiple subsequent context windows\.

In this work, we focus on learning delegation\. Unlike digestion, delegation decisions are made by the root agent as reasoning unfolds and can be optimized directly from task outcomes\. We introduceHermes\-Learn, a two\-stage framework that first teaches useful delegation behaviors through supervised fine\-tuning \(SFT\), then develops autonomous strategy selection through reinforcement learning \(RL\)\. For digestion, we rely on the model’s existing ability to summarize its accumulated reasoning and continue from the resulting digest instead of explicitly training it\.

##### Stage I: Supervised Fine\-Tuning

We first warm\-start the base model with supervised fine\-tuning on successful teacher trajectories\. We collect trajectories from strong models operating underL1andL2, exposing the model to delegation under different levels of steering\. We retain trajectories in which the root agent delegates at least once and produces the correct final answer\. During training, since we setD=1D=1, all delegation decisions occur at the root, so we train only on root\-agent trajectories and retain subagent outputs only as part of the root historyhth\_\{t\}\. Let𝒯\\mathcal\{T\}denote these retained root\-agent trajectories\. Then the SFT stage optimizes

arg⁡minθ−∑τ∈𝒯∑t=1T∑j=1\|at\|log⁡πθ​\(at,j∣ht,at,<j\),\\arg\\min\_\{\\theta\}\-\\sum\_\{\\tau\\in\\mathcal\{T\}\}\\sum\_\{t=1\}^\{T\}\\sum\_\{j=1\}^\{\|a\_\{t\}\|\}\\log\\pi\_\{\\theta\}\\left\(a\_\{t,j\}\\mid h\_\{t\},a\_\{t,<j\}\\right\),where loss is computed only over assistant tokens\. This stage exposes the model to successful delegation under different levels of steering and provides a strong initialization for RL\.

##### Stage II: Reinforcement learning

Initializing from the SFT policyπθSFT\\pi\_\{\\theta\_\{\\mathrm\{SFT\}\}\}, we use on\-policy reinforcement learning with outcome\-based rewards to learn more autonomous delegation\. We use Dr\. GRPO\[[31](https://arxiv.org/html/2609.38332#bib.bib23)\]with the DAPO clip\-higher strategy and dynamic sampling\[[32](https://arxiv.org/html/2609.38332#bib.bib24)\]\. For each problemp∈𝒟p\\in\\mathcal\{D\}andHermesharnessHH, we sampleGGroot\-agent rollouts from the current policy\. Each rollout receives a binary reward of11if its final answer is correct and00otherwise\. Following dynamic sampling\[[32](https://arxiv.org/html/2609.38332#bib.bib24)\], we train only on groups containing both correct and incorrect rollouts\.

As in Stage I, we optimize only the root\-agent trajectory\. Extending training to subagent trajectories would additionally require optimizing reasoning over policy\-generated subtasks and assigning credit to individual subagent outcomes\. Recursive Agent Optimization \(RAO\)\[[6](https://arxiv.org/html/2609.38332#bib.bib12)\], for example, addresses this setting by jointly optimizing nodes in a recursive execution tree using node\-local rewards and trajectory weighting\. In this work, we focus on learning root\-level delegation decisions and leave joint root–subagent optimization to future work\.

We optimize root\-agent trajectories with the following Dr\. GRPO objective:

𝒥⁡\(πθ\)=𝔼p∼𝒟,H∼S\(ℋ\),\{oi\}i=1G∼πθold\(⋅∣p;H\)\[1G∑i=1G∑t=1T∑j=1\|ati\|min\(rit,j\(πθ\)A^i,clip\(rit,j\(πθ\),1−ϵlow,1\+ϵhigh\)A^i\)\],\\begin\{aligned\} \\mathcal\{J\}\(\\pi\_\{\\theta\}\)&=\\mathbb\{E\}\_\{p\\sim\\mathcal\{D\},H\\sim S\(\\mathcal\{H\}\),\\\{o\_\{i\}\\\}\_\{i=1\}^\{G\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid p;H\)\}\\Bigg\[\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\sum\_\{t=1\}^\{T\}\\sum\_\{j=1\}^\{\|a^\{i\}\_\{t\}\|\}\\min\\Big\(r^\{i\}\_\{t,j\}\(\\pi\_\{\\theta\}\)\\hat\{A\}\_\{i\},\\,\\mathrm\{clip\}\\\!\\left\(r^\{i\}\_\{t,j\}\(\\pi\_\{\\theta\}\),1\-\\epsilon\_\{\\mathrm\{low\}\},1\+\\epsilon\_\{\\mathrm\{high\}\}\\right\)\\hat\{A\}\_\{i\}\\Big\)\\Bigg\]\\end\{aligned\},

where

rt,ji​\(πθ\)=πθ​\(at,ji∣hti,at,<ji\)πθold​\(at,ji∣hti,at,<ji\),A^i=Ri−1G​∑k=1GRk\.r\_\{t,j\}^\{i\}\(\\pi\_\{\\theta\}\)=\\frac\{\\pi\_\{\\theta\}\(a\_\{t,j\}^\{i\}\\mid h\_\{t\}^\{i\},a\_\{t,<j\}^\{i\}\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(a\_\{t,j\}^\{i\}\\mid h\_\{t\}^\{i\},a\_\{t,<j\}^\{i\}\)\},\\qquad\\hat\{A\}\_\{i\}=R\_\{i\}\-\\frac\{1\}\{G\}\\sum\_\{k=1\}^\{G\}R\_\{k\}\.
Here,RiR\_\{i\}is the binary outcome reward for rolloutii\. Since the reward is defined at the rollout level, the same advantageA^i\\hat\{A\}\_\{i\}is applied to all root\-agent tokens in that rollout\.

The samplerS⁡\(ℋ\)S\(\\mathcal\{H\}\)defines a curriculum over theHermeshierarchy\. We begin RL withL2and gradually increase the proportion ofL1, while retaining a small fraction ofL2throughout training\. This progressively reduces harness steering, requiring the model to take greater control over which delegation strategies to use as reasoning unfolds\.

Letnnbe a natural number, and letf⁡\(n\)f\(n\)denote the sum of the digits ofn2\+1n^\{2\}\+1when written in base1010\. Define the iteratesf\(1\)​\(n\)=f​\(n\)f^\{\(1\)\}\(n\)=f\(n\)andf\(k\+1\)​\(n\)=f⁡\(f\(k\)​\(n\)\)f^\{\(k\+1\)\}\(n\)=f\(f^\{\(k\)\}\(n\)\)fork≥1k\\geq 1\. Letmbe the value off\(100\)​\(1990\)f^\{\(100\)\}\(1990\)\.Letpbe the number of four\-digit numbers whose digits sum to1212\.Find the remainder whenm\+p\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{m\}\}\+\{\\color\[rgb\]\{0\.8516,0\.3945,0\.0508\}\\textbf\{p\}\}is divided by10001000\.Golden answer:353\\boxed\{353\}Figure 2:Synthetic splitting example\.Constructed by combining two independentAceReasonquestions into a single problem whose final answer depends on both, encouraging parallel delegation\.
##### Data augmentation

Competition math problems often do not expose delegation strategies uniformly: for example, independent attempts may arise more frequently than decomposition\. To provide more balanced coverage of delegation strategies during training, we generate synthetic problems with structures designed to favor different search and verification strategies\.*Splitting*combines multiple independent problems into a single instance whose final answer depends on all of them, creating opportunities for parallel delegation \(Fig\.\)\.*Chaining*introduces dependencies that favor sequential delegation;*verifying*introduces potentially incorrect intermediate results; and*multi\-*and*single\-attempt*problems vary whether additional independent attempts are useful \(see details in App\.\)\.

These controlled structures expose the model to a broader and more balanced range of contextual reasoning behaviors and encourage it to adapt its delegation strategy to the problem\. We use the same generation procedure to construct both synthetic training examples and a held\-out synthetic evaluation set\.

## 5Experimental Setup and Main Results

We focus on answeringfourresearch questions:\(RQ1\)DoesHermes\-Learnimprove contextual reasoning?\(RQ2\)What contextual reasoning behaviors emerge through training?\(RQ3\)Which components of the training recipe drive these improvements?\(RQ4\)Do these gains generalize to new settings? We first describe the training and evaluation setup and baselines, and then address each question in turn\.

##### Training setup

We train theQwen3\-4B\-Instruct\-2507base model onAceReason\-5K, a cleaned subset of AceReason\-Math\[[33](https://arxiv.org/html/2609.38332#bib.bib6)\]containing 5,239 mathematical reasoning problems\. For SFT, we train on teacher trajectories collected from bothSonnet 5andDeepSeek\-V3\.2, usingL1andL2harnesses\. For RL, we use a curriculum that shifts fromL2toL1while retaining balanced exposure to both harnesses throughout training, and augment training set with 500 synthetic problems\. See App\.for details\.

##### Evaluation benchmarks and protocols

We evaluate on a suite of mathematical reasoning benchmarks:\(1\)a cleanedpooledbenchmark of 238 AIME and HMMT problems from competitions between 2023 and 2026;\(2\)asyntheticbenchmark of 88 problems generated from the pooled benchmark using the same augmentation pipeline as our training data; and\(3\)advanced benchmarks, BeyondAIME\[[7](https://arxiv.org/html/2609.38332#bib.bib8)\]and IMO\-AnswerBench\[[8](https://arxiv.org/html/2609.38332#bib.bib9)\], together with the out\-of\-domain FrontierScience benchmark\[[9](https://arxiv.org/html/2609.38332#bib.bib11)\]\. We use an 8K\-token context per agent, a practical and deliberately constrained setting that makes reasoning across multiple contexts necessary\. We report average accuracy over 16 sampled solutions \(Avg​@​16\\mathrm\{Avg\}@16\) under both inline reasoning andHermesL1\. Inline performance reflects mathematical reasoning within a single context, while improvements underL1reflect the model’s ability to use additional contexts effectively\. Training and evaluation use the defaultHermesconfiguration:T=3T=3,B=3B=3,D=1D=1, andW=1W=1\. Under this configuration, only the root agent delegates, with up to three subagents per delegation step, and digestion is disabled \(see App\.\)\.

##### Baselines

In addition to the base model, we include two baselines designed to isolate the source of improvements fromHermes\-Learn\. First, we evaluate inline reasoning with a5656K\-token context, matching the maximum total context available to a 3\-stepHermesrun\. Second, we train with GRPO under inline reasoning with an88K\-token context, controlling for improvements from RL without training the model to reason across contexts\.

Table 3:SFT and RL progressively improveL1performance \(Avg​@​16\\mathrm\{Avg\}@16\(%,↑\\uparrow\)\), while inline performance changes relatively little\. SFT \+ RL underL1also exceeds the base model’s 56K inline reference on all four mathematical benchmarks\. Trained results are mean±\\pmstandard deviation over three seeds\.Boldandunderlinedmark the best and second\-best point estimates per dataset\.PooledSyntheticBeyondAIMEIMO\-AnswerFrontierScienceModelInlineL1InlineL1InlineL1InlineL1InlineL1Base37\.116\.250\.726\.824\.812\.236\.218\.223\.68\.8GRPO40\.4±\\pm1\.016\.5±\\pm1\.661\.5±\\pm1\.627\.9±\\pm1\.531\.1±\\pm0\.614\.1±\\pm1\.437\.4±\\pm0\.619\.0±\\pm1\.424\.6±\\pm1\.58\.9±\\pm0\.6Base \(56K\)41\.8–59\.7–31\.1–40\.3–24\.6–GRPO \(56K\)41\.3±\\pm1\.2–62\.2±\\pm2\.1–30\.9±\\pm1\.2–37\.5±\\pm0\.6–25\.1±\\pm1\.5–Hermes\-LearnSFT35\.3±\\pm0\.339\.9±\\pm0\.752\.7±\\pm1\.657\.2±\\pm2\.423\.5±\\pm0\.431\.0±\\pm0\.636\.7±\\pm0\.539\.2±\\pm0\.218\.4±\\pm1\.418\.9±\\pm0\.5SFT \+ RL36\.4±\\pm0\.147\.3±\\pm1\.355\.2±\\pm0\.974\.0±\\pm2\.927\.5±\\pm1\.338\.5±\\pm0\.536\.4±\\pm0\.444\.0±\\pm0\.819\.1±\\pm0\.424\.1±\\pm0\.9

##### \(RQ1\)Hermes\-Learnenables test\-time scaling withHermes

It transforms additional context from a liability into a source of test\-time scaling \(Tab\.\)\. The base model performs substantially worse underL1than inline reasoning across all benchmarks, showing that access to additional contexts alone is insufficient\. SFT reverses this trend, and RL further improvesL1performance\. In contrast, inline performance changes relatively little throughHermes\-Learn, indicating that the gains primarily come from learning to use additional contexts rather than from improved mathematical reasoning\. This distinction is reinforced by the inline GRPO baseline, which improves inline performance but continues to degrade underL1\. Finally, SFT \+ RL underL1outperforms the 56K inline reference of both the base and GRPO models across all four mathematical benchmarks, showing that improved mathematical reasoning alone is insufficient to realize the benefits of additional context and highlighting the role ofHermes\-Learnin enabling effective contextual reasoning\.

Figure 3:Hermes\-Learnmakes strategy selection problem\- and progress\-dependent\.Splitis used more often on synthetic problems designed to benefit from decomposition, while the second delegation step increasingly favors verification or continues the first\-step strategy\.
##### \(RQ2\)Training makes strategy selectionproblemand progress\-dependent

To understand how training changes the use ofHermes, we use an LLM judge to label the problem solving strategies executed underL1on the pooled and synthetic benchmarks \(App\.\)\. Fig\.reveals two patterns\. First, strategy selection depends on the problem:Splitbecomes much less common on the pooled benchmark after training, but remains prevalent on synthetic problems designed to benefit from decomposition \(see App\.for strategy definitions\)\. Second, strategy selection depends on reasoning progress: in the second delegation step, the model often switches to verification; otherwise, it tends to continue the strategy used in the first step\. Together, these results show thatHermes\-Learndoes not learn a fixed delegation pattern, but selects strategies based on the problem and the progress made so far\. Additional analyses and example rollouts are provided in Apps\.and\.

## 6Analysis and Discussion

### 6\.1\(RQ3\)Teacher diversity, curriculum, and synthetic data shapeHermes\-Learn\.

We ablate the main components ofHermes\-Learnon the pooled benchmark \(Tab\.\)\. For SFT, we vary the source and amount of teacher demonstrations\. For RL, we study exposure toL1andL2harnesses, their training order, and data augmentation\.

Combining teachers yields the strongest SFT model\.Training on the full set of trajectories from bothSonnet 5andDeepSeek\-V3\.2gives the best SFT performance \(Both \(Full\)\)\. When we match the mixed\-teacher dataset to the token budget of a single teacher \(Both \(Matched\)\), however, its performance is similar toDeepSeek\-V3\.2alone\. This suggests that much of the gain from the full dataset comes from having more high\-quality demonstrations rather than teacher diversity alone\. Notably,DeepSeek\-V3\.2is a much better teacher thanSonnet 5, despite performing worse underHermes\(Tab\.\), indicating that stronger inference performance does not necessarily translate into better supervision for SFT\.

Table 4:Hermes\-Learnablationson the pooled benchmark\.\(a\)For SFT,Both \(Matched\)uses a token\-matched subset of trajectories from both teachers, whileBoth \(Full\)uses the full combined dataset\.\(b\)For RL,MixinterleavesL1andL2, whileCurriculumtrains onL2beforeL1with matched exposure;RephrasedandSyntheticeach add 500 problems\.Avg​@​16\\mathrm\{Avg\}@16\(%,↑\\uparrow\), mean±\\pms\.d\. over three seeds\.Boldandunderlinedmark best and second\-best results\.\(a\) SFT ablations

TeachersInlineL1Sonnet31\.5±\\pm0\.530\.1±\\pm0\.9DeepSeek34\.5±\\pm0\.439\.3±\\pm0\.2Both \(Matched\)34\.4±\\pm0\.738\.9±\\pm0\.4Both \(Full\)35\.3±\\pm0\.339\.9±\\pm0\.7

\(b\) RL ablations

ScheduleData Aug\.InlineL1*Harness schedule on original data*L1only–36\.5±\\pm1\.546\.1±\\pm0\.8L2only–36\.2±\\pm0\.245\.4±\\pm0\.6Mix–35\.7±\\pm0\.746\.1±\\pm0\.8Curriculum–36\.3±\\pm0\.746\.3±\\pm1\.9*With data augmentation*Mix\+ Synthetic35\.9±\\pm1\.045\.3±\\pm1\.8Curriculum\+ Synthetic36\.4±\\pm0\.147\.3±\\pm1\.3Curriculum\+ Rephrased36\.0±\\pm0\.345\.6±\\pm1\.2

Curriculum provides the largest gains when paired with synthetic problems\.We ablate two choices in RL training: how the model is exposed toL1andL2, and exposure to synethic data\. Mix and Curriculum use the same number ofL1andL2episodes, differing only in their order\. Similarly, we compare 500 synthetic problems with 500 LLM\-rephrasedAceReason\-1Kproblems to control for the effect of additional training examples\. On the original training set,CurriculumandMixperform similarly, suggesting that orderingL2beforeL1provides only a modest benefit by itself\. In contrast, combining the curriculum with synthetic problems produces the strongestL1performance\. Rephrasing the same number of existing problems does not yield a comparable gain, suggesting that exposure to new problem structures is more useful than additional training volume alone\. Inline performance remains largely unchanged, while the improvements concentrate underL1, consistent withHermes\-Learnimproving how the model usesHermesrather than its in\-context mathematical reasoning\.

### 6\.2\(RQ4\)Hermes\-Learngeneralizes beyond its training setting

Our main experiments use the sameHermesconfiguration during training and evaluation\. We now test three forms of generalization: larger inference\-time compute budgets, combinations with other test\-time scaling methods, and a different model family and scale\.

Hermes\-Learnextrapolates to larger inference\-time compute budgets\.Starting from the defaultHermesconfiguration used during training and evaluation \(T=3T=3,B=3B=3,D=1D=1,W=1W=1\), we independently increase compute through longer sequential reasoning \(↑W\\uparrow W\), more subagents per delegation step \(↑B\\uparrow B\), and deeper delegation \(↑D\\uparrow D\)\. Tab\.shows that SFT benefits primarily from additional context windows, with little gain from more subagents or deeper delegation\. After RL, performance improves along all three dimensions, showing that the learned contextual reasoning strategies generalize to inference configurations not seen during training\.

Table 5:Hermes\-Learngeneralizes across compute budgets, test\-time scaling methods, and model families\.Avg​@​16\\mathrm\{Avg\}@16\(%,↑\\uparrow\) on the pooled benchmark\.\(a\)We independently vary the number of context windows \(WW\), subagents \(BB\), and delegation depth \(DD\) beyond the defaultHermesconfiguration used for training and evaluation:T=3T=3,B=3B=3,D=1D=1, andW=1W=1\.\(b\)We evaluate Recursive Self\-Aggregation \(RSA\) and the DeepSeekMath \(DSM\) Agent, both alone and combined withL1\.\(c\)We apply the same training pipeline toOlmo\-3\-7B\-Instruct\. All experiments use the best performingHermes\-Learnmodel\.Boldandunderlinedmark best and second\-best results where applicable\.\(a\) Scaling inference\-time compute

Scaling KnobValueSFTSFT \+ RLDefault–39\.947\.3WW244\.352\.2345\.253\.0BB239\.746\.4439\.749\.7DD240\.855\.6
\(b\) Other scaling methods

ModelRSADSMBase46\.024\.4SFT44\.911\.3SFT \+ RL50\.515\.1WithL1SFT49\.952\.1SFT \+ RL55\.052\.5
\(c\) Transfer toOlmo\-3\-7B

ModelInlineL1Base30\.54\.8SFT29\.333\.4SFT \+ RL30\.540\.6

Hermes\-Learnbenefits test\-time scaling methods beyondHermes\.We evaluate the learned models with Recursive Self\-Aggregation \(RSA\)\[[14](https://arxiv.org/html/2609.38332#bib.bib19)\]and the DeepSeekMath \(DSM\) Agent\[[34](https://arxiv.org/html/2609.38332#bib.bib10)\], two methods that use self\-generated intermediate results to guide later reasoning\. Under RSA, performance improves over the base model, with further gains when combined withL1\. DSM alone degrades sharply under the 8K\-token context limit, but combining it withL1recovers and substantially improves performance\. These results show that the contextual reasoning learned byHermes\-Learncan complement other forms of test\-time scaling\.

Hermes\-Learntransfers to a different model family and scale\.We apply the same training pipeline toOlmo\-3\-7B\-Instruct, replacingQwen3\-4B\-Instruct\-2507as the base model\. LikeQwen3\-4B, theOlmomodel also initially struggles to use theL1harness effectively\. SFT reverses this degradation, while RL further improves its performance, all while keeping inline reasoning performance unchanged\. Together, these results provide evidence thatHermes\-Learncan improve contextual reasoning beyond the model family and scale used in our main experiments, suggesting that the approach may extend to a broader range of models\.

## 7Conclusion

We introducedHermes, a family of simple, configurable harnesses for test\-time scaling beyond a single context window\. Through delegation and digestion,Hermesprogressively shifts control over context allocation and reuse from the harness to the model\. We showed that frontier models can exploit this flexibility, while smaller open\-source models exhibit a substantial contextual reasoning gap\. To address this gap, we introducedHermes\-Learn, a training recipe that improves contextual reasoning while preserving in\-context reasoning performance\. Trained models scale more effectively with additional inference\-time compute, adapt their strategies as reasoning unfolds, and transfer across model families and scales\. Together, our results establish contextual reasoning as a learnable model capability and an important component of long\-horizon test\-time scaling\. We discuss limitations and future work in App\.\.

## Acknowledgements

The authors would like to thank Amir Tahmasbi, Zhehui Huang, Karen Hovsepian, and Zhishen Huang for helpful feedback and discussions throughout the project\. We also thank Andrew Wang and Lecheng \(Jerry\) Kong for early discussions of recursive models, and Satyaki Chakraborty for work on recursive agent optimization\.

## References

- \[1\]H\. A\. Simon\(1957\)Models of man, social and rational: mathematical essays on rational human behavior in a social setting\.John Wiley & Sons,New York\.Cited by:[§1](https://arxiv.org/html/2609.38332#S1.p1.1.1.2.1)\.
- \[2\]L\. Alpöge and R\. Furman\(2026\)More than two thirds of the zeta zeros are simple and on the critical line\.arXiv preprint arXiv:2608\.13637\.Cited by:[§1](https://arxiv.org/html/2609.38332#S1.p2.1)\.
- \[3\]OpenAI\(2026\)Finite time blowup for the euler equation\.Note:PreprintExternal Links:[Link](https://cdn.openai.com/pdf/315b36cd-ec98-4023-8342-93345194ece1/euler.pdf)Cited by:[§1](https://arxiv.org/html/2609.38332#S1.p2.1)\.
- \[4\]C\. V\. Snell, J\. Lee, K\. Xu, and A\. Kumar\(2025\)Scaling LLM test\-time compute optimally can be more effective than scaling parameters for reasoning\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=4FWAwZtd2n)Cited by:[§1](https://arxiv.org/html/2609.38332#S1.p2.1)\.
- \[5\]I\. Wu, Y\. Qu, A\. Setlur, and A\. Kumar\(2026\)Reasoning cache: continual improvement over long horizons via short\-horizon rl\.arXiv preprint arXiv:2602\.03773\.Cited by:[Appendix A](https://arxiv.org/html/2609.38332#A1.p2.1),[§E\.3](https://arxiv.org/html/2609.38332#A5.SS3.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.38332#S1.p3.1),[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2609.38332#S3.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2609.38332#S4.p1.1)\.
- \[6\]A\. Gandhi, S\. Chakraborty, X\. Wang, A\. Kumar, and G\. Neubig\(2026\)Recursive agent optimization\.arXiv preprint arXiv:2605\.06639\.Cited by:[Appendix A](https://arxiv.org/html/2609.38332#A1.p1.1),[Appendix A](https://arxiv.org/html/2609.38332#A1.p3.1),[§1](https://arxiv.org/html/2609.38332#S1.p3.1),[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2609.38332#S3.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2609.38332#S4.SS0.SSS0.Px2.p2.1)\.
- \[7\]ByteDance\-Seed\(2025\)BeyondAIME: advancing math reasoning evaluation beyond high school olympiads\.Hugging Face\.Cited by:[§1](https://arxiv.org/html/2609.38332#S1.p5.1),[§5](https://arxiv.org/html/2609.38332#S5.SS0.SSS0.Px2.p1.1)\.
- \[8\]T\. Luong, D\. Hwang, H\. H\. Nguyen, G\. Ghiasi, Y\. Chervonyi, I\. Seo, J\. Kim, G\. Bingham, J\. Lee, S\. Mishra, A\. Zhai, C\. H\. Hu, H\. Michalewski, J\. Kim, J\. Ahn, J\. Bae, X\. Song, T\. H\. Trinh, Q\. V\. Le, and J\. Jung\(2025\)Towards robust mathematical reasoning\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2025.emnlp-main.1794/)Cited by:[§1](https://arxiv.org/html/2609.38332#S1.p5.1),[§5](https://arxiv.org/html/2609.38332#S5.SS0.SSS0.Px2.p1.1)\.
- \[9\]M\. Wang, R\. Lin, K\. Hu, J\. Jiao, N\. Chowdhury, E\. Chang, and T\. Patwardhan\(2026\)FrontierScience: evaluating AI’s ability to perform expert\-level scientific tasks\.arXiv preprint arXiv:2601\.21165\.Cited by:[§1](https://arxiv.org/html/2609.38332#S1.p5.1),[§5](https://arxiv.org/html/2609.38332#S5.SS0.SSS0.Px2.p1.1)\.
- \[10\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. Zhou\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.External Links:2201\.11903,[Link](https://arxiv.org/abs/2201.11903)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[11\]A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang,et al\.\(2023\)Self\-refine: iterative refinement with self\-feedback\.External Links:2303\.17651,[Link](https://arxiv.org/abs/2303.17651)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[12\]K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman\(2021\)Training verifiers to solve math word problems\.External Links:2110\.14168,[Link](https://arxiv.org/abs/2110.14168)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[13\]X\. Wang, J\. Wei, D\. Schuurmans, Q\. V\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou\(2023\)Self\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2203.11171)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[14\]S\. Venkatraman, V\. Jain, S\. Mittal, V\. Shah, J\. Obando\-Ceron, Y\. Bengio, B\. R\. Bartoldson, B\. Kailkhura, G\. Lajoie, G\. Berseth,et al\.\(2025\)Recursive self\-aggregation unlocks deep thinking in large language models\.External Links:2509\.26626,[Link](https://arxiv.org/abs/2509.26626)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1),[§6\.2](https://arxiv.org/html/2609.38332#S6.SS2.p3.1)\.
- \[15\]J\. Hu, Y\. Zhang, S\. Shang, X\. Yang, Y\. Peng, Z\. Huang, H\. Zhou, X\. Wu,et al\.\(2026\)PaCoRe: learning to scale test\-time compute with parallel coordinated reasoning\.External Links:2601\.05593,[Link](https://arxiv.org/abs/2601.05593)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px2.p1.1)\.
- \[16\]P\. Qin, Q\. Cao, and P\. Xie\(2026\)ATLAS: agentic test\-time learning\-to\-allocate scaling\.External Links:2606\.01667,[Link](https://arxiv.org/abs/2606.01667)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[17\]S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. Griffiths, Y\. Cao, and K\. Narasimhan\(2023\)Tree of thoughts: deliberate problem solving with large language models\.Advances in neural information processing systems36,pp\. 11809–11822\.Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[18\]A\. Zhou, K\. Yan, M\. Shlapentokh\-Rothman, H\. Wang, and Y\. Wang\(2023\)Language agent tree search unifies reasoning acting and planning in language models\.arXiv preprint arXiv:2310\.04406\.Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[19\]H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe\(2024\)Let’s verify step by step\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 39578–39601\.Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[20\]A\. Prasad, A\. Koller, M\. Hartmann, P\. Clark, A\. Sabharwal, M\. Bansal, and T\. Khot\(2024\)ADaPT: as\-needed decomposition and planning with language models\.InFindings of the Association for Computational Linguistics: NAACL 2024,External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-naacl.264),[Link](https://aclanthology.org/2024.findings-naacl.264/)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[21\]P\. Schroeder, N\. Morgan, H\. Luo, and J\. Glass\(2024\)THREAD: thinking deeper with recursive spawning\.External Links:2405\.17402,[Link](https://arxiv.org/abs/2405.17402)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[22\]Z\. Chi, L\. Dong, Q\. Dong, Y\. Hao, X\. Wu, S\. Huang, and F\. Wei\(2025\)The era of agentic organization: learning to organize with language models\.External Links:2510\.26658,[Link](https://arxiv.org/abs/2510.26658)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[23\]Z\. Zhou, A\. Qu, Z\. Wu, S\. Kim, A\. Prakash, D\. Rus, J\. Zhao, B\. K\. H\. Low, and P\. P\. Liang\(2025\)MEM1: learning to synergize memory and reasoning for efficient long\-horizon agents\.External Links:2506\.15841,[Link](https://arxiv.org/abs/2506.15841)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[24\]V\. Kontonis, Y\. Zeng, S\. Garg, L\. Chen, H\. Tang, Z\. Wang, A\. Awadallah, E\. Horvitz, J\. Langford, and D\. Papailiopoulos\(2026\)MEMENTO: teaching LLMs to manage their own context\.arXiv preprint arXiv:2604\.09852\.Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px1.p1.1)\.
- \[25\]Y\. Qu, M\. Y\. R\. Yang, A\. Setlur, L\. Tunstall, E\. E\. Beeching, R\. Salakhutdinov, and A\. Kumar\(2025\)Optimizing test\-time compute via meta reinforcement fine\-tuning\.External Links:2503\.07572,[Link](https://arxiv.org/abs/2503.07572)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px2.p1.1)\.
- \[26\]A\. Setlur, M\. Y\. R\. Yang, C\. Snell, J\. Greer, I\. Wu, V\. Smith, M\. Simchowitz, and A\. Kumar\(2025\)e3: learning to explore enables extrapolation of test\-time compute for LLMs\.External Links:2506\.09026,[Link](https://arxiv.org/abs/2506.09026)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px2.p1.1)\.
- \[27\]J\. I\. Hamid, I\. H\. Orney, M\. Y\. Li, O\. Shaikh, Y\. Lee, D\. Sadigh, C\. Finn, and N\. Goodman\(2026\)SPIRAL: learning to search and aggregate\.External Links:2606\.23595,[Link](https://arxiv.org/abs/2606.23595)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2609.38332#S3.SS0.SSS0.Px3.p1.1)\.
- \[28\]W\. Sun, M\. Lu, Z\. Ling, K\. Liu, X\. Yao, Y\. Yang, and J\. Chen\(2025\)Scaling long\-horizon LLM agent via context\-folding\.External Links:2510\.11967,[Link](https://arxiv.org/abs/2510.11967)Cited by:[§2](https://arxiv.org/html/2609.38332#S2.SS0.SSS0.Px2.p1.1)\.
- \[29\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2210.03629)Cited by:[§3](https://arxiv.org/html/2609.38332#S3.SS0.SSS0.Px2.p2.1)\.
- \[30\]X\. Wu, K\. Li, Y\. Zhao, L\. Zhang, L\. Ou, H\. Yin, Z\. Zhang, X\. Yu, D\. Zhang, Y\. Jiang, P\. Xie, F\. Huang, M\. Cheng, S\. Wang, H\. Cheng, and J\. Zhou\(2026\)ReSum: unlocking long\-horizon search intelligence via context summarization\.External Links:2509\.13313,[Link](https://arxiv.org/abs/2509.13313)Cited by:[Appendix A](https://arxiv.org/html/2609.38332#A1.p2.1),[§4](https://arxiv.org/html/2609.38332#S4.p1.1)\.
- \[31\]Z\. Liu, C\. Chen, W\. Li, P\. Qi, T\. Pang, C\. Du, W\. S\. Lee, and M\. Lin\(2025\)Understanding r1\-zero\-like training: a critical perspective\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=5PAF7PAY2Y)Cited by:[Table 10](https://arxiv.org/html/2609.38332#A5.T10.5.7.2.1),[§4](https://arxiv.org/html/2609.38332#S4.SS0.SSS0.Px2.p1.1)\.
- \[32\]Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, YuYue, W\. Dai, T\. Fan, G\. Liu, J\. Liu, L\. Liu, X\. Liu, H\. Lin, Z\. Lin, B\. Ma, G\. Sheng, Y\. Tong, C\. Zhang, M\. Zhang, R\. Zhang, W\. Zhang, H\. Zhu, J\. Zhu, J\. Chen, J\. Chen, C\. Wang, H\. Yu, Y\. Song, X\. Wei, H\. Zhou, J\. Liu, W\. Ma, Y\. Zhang, L\. Yan, Y\. Wu, and M\. Wang\(2025\)DAPO: an open\-source LLM reinforcement learning system at scale\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=2a36EMSSTp)Cited by:[§4](https://arxiv.org/html/2609.38332#S4.SS0.SSS0.Px2.p1.1)\.
- \[33\]Y\. Chen, Z\. Yang, Z\. Liu, C\. Lee, P\. Xu, M\. Shoeybi, B\. Catanzaro, and W\. Ping\(2025\)AceReason\-nemotron: advancing math and code reasoning through reinforcement learning\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38, Main Conference,pp\. 110320–110345\.External Links:[Document](https://dx.doi.org/10.52202/085713-3685),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/9fc2aa7f8a66eed231007b8bee4485cb-Paper-Conference.pdf)Cited by:[§E\.3](https://arxiv.org/html/2609.38332#A5.SS3.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2609.38332#S5.SS0.SSS0.Px1.p1.1)\.
- \[34\]Z\. Shao, Y\. Luo, C\. Lu, Z\. Ren, J\. Hu, T\. Ye, Z\. Gou, S\. Ma, and X\. Zhang\(2025\)Deepseekmath\-v2: towards self\-verifiable mathematical reasoning\.arXiv preprint arXiv:2511\.22570\.Cited by:[§6\.2](https://arxiv.org/html/2609.38332#S6.SS2.p3.1)\.
- \[35\]C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. Narasimhan\(2024\)Swe\-bench: can language models resolve real\-world github issues?\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 54107–54157\.Cited by:[Appendix A](https://arxiv.org/html/2609.38332#A1.p1.1)\.

## Appendix Contents

## Appendix ALimitations and Future Work

Our evaluation studies contextual reasoning in a controlled setting\.We evaluate primarily on mathematical reasoning benchmarks \(AIME and HMMT\) and a science benchmark, using an artificially constrained 8K\-token context window\. This makes multi\-context reasoning necessary while keeping training and evaluation tractable, and is similar to controlled context constraints used in prior work on recursive agents\[[6](https://arxiv.org/html/2609.38332#bib.bib12)\]\. However, we do not test whether the same gains persist with much larger native context windows or in long\-horizon domains with evolving state, tool use, and delayed feedback\. Software engineering benchmarks such as SWE\-bench\[[35](https://arxiv.org/html/2609.38332#bib.bib1)\], as well as scientific discovery and deep\-research tasks, are natural settings for evaluating whetherHermesremains effective when the reasoning horizon substantially exceeds a single context window\.

Hermescurrently relies on a hand\-designed strategy space and only explicitly learns part of contextual reasoning\.The strategy menu, the search–verification distinction, and theL3–L1hierarchy are manually specified\. Future work could instead learn new contextual reasoning strategies, compose them dynamically, or adapt the level of model control during inference\. Moreover, contextual reasoning requires learning*what to delegate*,*what to preserve*during digestion, and*how to reason*from the resulting digest\. We focus on learning delegation and rely on the base model’s zero\-shot ability to summarize and resume reasoning\. Prior work suggests that reasoning from compressed histories can be learned\[[5](https://arxiv.org/html/2609.38332#bib.bib13),[30](https://arxiv.org/html/2609.38332#bib.bib33)\], while directly learning useful digests introduces a harder credit\-assignment problem\[[5](https://arxiv.org/html/2609.38332#bib.bib13)\]\. Jointly learning delegation and digestion is therefore an important next step, particularly as reasoning horizons grow\.

Training optimizes only the root\-agent trajectory\.Subagent trajectories do not independently contribute to the SFT or RL objective\. This avoids the substantially larger sample complexity and credit\-assignment problem introduced by optimizing policy\-generated subtasks, but also prevents subagents from adapting to the distribution of tasks produced by the learned delegator\. Recursive Agent Optimization \(RAO\)\[[6](https://arxiv.org/html/2609.38332#bib.bib12)\], for example, jointly optimizes nodes in a recursive execution tree using node\-local rewards and trajectory weighting\. ExtendingHermes\-Learnto jointly optimize root and subagent behavior is a promising direction, but may require reliable process\-level supervision or learned judges\.

Finally,Hermestrades additional inference computation for improved reasoning\. Future work should study contextual reasoning jointly in terms of accuracy, token usage, latency, and cost, and learn when allocating another context is worth the additional compute\. More broadly, our current experiments shift control over a fixed contextual\-reasoning interface from the harness to the model; a longer\-term goal is to let models learn the interface itself\.

## Appendix BAdditional Discussion of Related Work

We provide more detailed discussions of the training approaches used by methods that train models for particular inference harnesses in Tab\.\.SPIRALintroduces set\-level credit assignment to train parallel generation and aggregation\.PaCoReuses outcome\-based reinforcement learning to train reasoning synthesis across parallel branches\.RAOassigns node\-local credit for recursive delegation decisions, enabling models to learn when and how to spawn additional subagents\.Context\-Foldingemploys process rewards to train delegation of subtasks to fresh branches together with compaction of intermediate reasoning\.Reasoning Cacheuses short\-horizon reinforcement learning over repeated generate–summarize cycles to train continuation from compacted reasoning progress\. While these methods employ different supervision and credit\-assignment mechanisms tailored to their inference procedures, they generally assume prescribed contextual reasoning strategies encoded in the harness and train models to execute those strategies effectively\.

## Appendix CHermesHarness Details

For all models, we impose an artificial context limit of88K tokens, creating a setting where reasoning often extends substantially beyond a single window and effective context management becomes necessary for test\-time scaling\. Importantly, this setting is not purely academic: bounded context enables reasoning to be distributed across multiple windows, allowing independent branches to execute in parallel rather than requiring an equivalently long sequential trajectory\. This creates a practical path for shorter\-context models to scale inference while potentially reducing wall\-clock latency\. In our experiments, we evaluate theHermesL1harness, in which the model has the greatest control over its contextual reasoning decisions\. All evaluations use a single root\-agent context window \(digesting is disabled\), withT=3T=3steps, delegation depthD=1D=1\(subagents cannot delegate\), and up toB=3B=3subagents per delegation step\. The root agent and all subagents share the same underlying LLM and the same 8K\-token context limit\.

### C\.1Harnesses in theHermesHierarchy

ForHermesharnesses withT=3T=3, each context window consists of two delegation steps followed by a return step, during which the agent synthesizes the accumulated conversation and subagent results without further delegation\. Delegation is organized into two broad categories:*search*and*verification*\.

The steering level determines how these categories and their underlying strategies are exposed to the model\. UnderL1, all available search and verification strategies are accessible at both delegation steps, resulting in a single harness configuration\. UnderL2, the harness specifies the delegation category at each step, while the model selects among the strategies within that category\. Since verification requires an existing candidate solution, the first delegation step is restricted to*search*\. The second step may use either*search*or*verification*, yielding twoL2variants:L2\-SV\(Search→\\rightarrowVerify→\\rightarrowReturn\) andL2\-SS\(Search→\\rightarrowSearch→\\rightarrowReturn\)\.

For*search*, we consider three ways of exploring the solution space\.Sameasks subagents to solve the full problem independently, without prescribing an approach;Differentinstead assigns distinct approaches to encourage diverse solutions; andSplitdecomposes the problem into subproblems that are solved independently\.

For*verification*, we consider three complementary forms of checking\.Answer Verificationtargets the current candidate answer directly;Intermediate Step Verificationexamines the assumptions and reasoning that support it; andDisagreement Resolutionreconciles conflicting candidates by identifying the source of disagreement\.

##### What changes across the hierarchy\.

The threeL1andL2harnesses use the same system prompt \(Prompt\), final\-step return prompt \(Prompt\), and subagent inline reasoning and report prompts \(Promptsand\)\. They differ only in their steering level, which determines what contextual reasoning strategies are available at each delegation step\.

Table 6:The harnesses in theHermeshierarchy\. Each cell gives the number of strategies offered at that step; the instruction text is otherwise identical\. Delegation is disabled at the final step\.HarnessStrategy menu offered at a delegating stepStep 1Step 2L3One named strategy11L2\-SSThe three*search*strategies33L2\-SVThe three*search*strategies at step 1, and the three*verification*strategies at step 233L1All six strategies, at every delegating step66

### C\.2Prompts

#### C\.2\.1System Prompt

Prompt 1: System prompt — identical atL3,L2andL1You are a meta\-agent for solving math problems\. You solve by DELEGATING to subagents, not by deriving solutions inline\. You proceed in steps and have two actions:1\. Delegate: write one or more lines of the form "TASK: <self\-contained task\>" \(1\-\-3 per step\)\. Each task is executed by a subagent that sees ONLY that task text and replies with a short report; subagents cannot see the problem, your conversation, or each other, so every task must contain everything needed to execute it\. The reports arrive in your next step\.2\. Return: when you are done, write "\#\#\# RETURN" on its own line, followed by your final answer\. Put the answer in \\boxed\{\}\. This ends the episode\.You may compare, select, or briefly combine conclusions explicitly stated by subagents, but do not derive new solutions yourself\. Use subagent results critically; agreement alone does not establish correctness\.Always provide the best provisional answer\. Put it in \\boxed\{\}\. Exception: do not guess before delegating\. On the first step you have no subagent results, so delegate without stating an answer; only from the step after results arrive should you end each step with your current best answer\.Delegation serves two purposes\. Search: make progress on the problem \(decompose it into subproblems, or have subagents solve the full problem independently using either different named approaches or no prescribed approach\)\. Verify: check the current answer \(recheck it with targeted checks, verify the assumptions and intermediate steps supporting it, or reconcile disagreeing candidates\)\. Each step’s instruction tells you which specific strategies are available\.

#### C\.2\.2L1: Full Strategy Menu at Every Step

Prompt 2:L1, step 1 — delegate \(full strategy menu\)\#\#\# PROBLEM\{problem\}\#\#\# INSTRUCTIONSDelegate the work for this step; do not solve it inline\. You have no subagent results yet, so do not state a boxed answer on this step\.Write one concise first\-person sentence explaining why the selected tasks are useful\.Generate 1\-\-3 tasks using one or more strategies from \#\#\# STRATEGIES\. Each task must be independently executable and fully self\-contained\. Include the complete problem or all relevant context, values, assumptions, and \-\- when the task checks a candidate answer \-\- that answer and the exact claim to verify\.\#\#\# STRATEGIESAsk 1\-\-3 subagents to:\- solve independent subproblems obtained by decomposing the original problem; or\- solve the full problem independently using different named approaches; or\- solve the full problem independently using the same task and no prescribed approach; or\- verify the current boxed answer using independent targeted checks; or\- independently verify the assumptions and intermediate steps supporting the current boxed answer; or\- resolve the disagreement between the candidate answers and identify the source of the discrepancy\.\#\#\# OUTPUT<one concise first\-person sentence\>\#\#\# TASKSWrite one task per line\. Begin each line with "TASK:"\.TASK: <self\-contained task\>

Prompt 3:L1, step 2 — delegate \(full strategy menu, and current best answer\)\#\#\# SUBAGENT RESULTS\{subagent\_results\}\#\#\# INSTRUCTIONSAlways provide the best provisional answer\. Put it in \\boxed\{\}\.Use at most two concise lines to justify the current answer\.Write one concise first\-person sentence explaining why the selected tasks are useful\.Generate 1\-\-3 tasks using one or more strategies from \#\#\# STRATEGIES\. Each task must be independently executable and fully self\-contained\. Include the complete problem or all relevant context, values, assumptions, and \-\- when the task checks a candidate answer \-\- that answer and the exact claim to verify\. State candidate answers inside tasks in plain text; never use \\boxed\{\} inside a task\.\#\#\# STRATEGIESAsk 1\-\-3 subagents to:\- solve independent subproblems obtained by decomposing the original problem; or\- solve the full problem independently using different named approaches; or\- solve the full problem independently using the same task and no prescribed approach; or\- verify the current boxed answer using independent targeted checks; or\- independently verify the assumptions and intermediate steps supporting the current boxed answer; or\- resolve the disagreement between the candidate answers and identify the source of the discrepancy\.\#\#\# OUTPUT\#\#\# ANSWER<at most two concise lines\>\\boxed\{<current best answer\>\}<one concise first\-person sentence\>\#\#\# TASKSWrite one task per line\. Begin each line with "TASK:"\.TASK: <self\-contained task\>

#### C\.2\.3L2\-SVandL2\-SS: One Category per Step

BothL2variants begin with the same search\-only step\.

Prompt 4:L2\-SVandL2\-SS, step 1 — delegate \(*search*category only\)\#\#\# PROBLEM\{problem\}\#\#\# INSTRUCTIONSDelegate the work for this step; do not solve it inline\. You have no subagent results yet, so do not state a boxed answer on this step\.Write one concise first\-person sentence explaining why the selected tasks are useful\.Generate 1\-\-3 tasks using one or more strategies from \#\#\# STRATEGIES\. Each task must be independently executable and fully self\-contained\. Include the complete problem or all relevant context, values, assumptions, and requested output\.\#\#\# STRATEGIESAsk 1\-\-3 subagents to:\- solve independent subproblems obtained by decomposing the original problem; or\- solve the full problem independently using different named approaches; or\- solve the full problem independently using the same task and no prescribed approach\.\#\#\# OUTPUT<one concise first\-person sentence\>\#\#\# TASKSWrite one task per line\. Begin each line with "TASK:"\.TASK: <self\-contained task\>

They then diverge\.L2\-SVswitches the menu to the verification category, which assumes that a first step of search has produced a response worth checking:

Prompt 5:L2\-SV, step 2 — delegate \(*verification*category only\)\#\#\# SUBAGENT RESULTS\{subagent\_results\}\#\#\# INSTRUCTIONSAlways provide the best provisional answer\. Put it in \\boxed\{\}\.Use at most two concise lines to justify the current answer\.Write one concise first\-person sentence explaining why the selected tasks are useful\.Generate 1\-\-3 tasks using one or more strategies from \#\#\# STRATEGIES\. Each task must be independently executable and fully self\-contained\. Include the relevant problem context, values, assumptions, candidate answer, and exact claim to solve or verify\. State candidate answers inside tasks in plain text; never use \\boxed\{\} inside a task\.\#\#\# STRATEGIESAsk 1\-\-3 subagents to:\- verify the current boxed answer using independent targeted checks; or\- independently verify the assumptions and intermediate steps supporting the current boxed answer; or\- resolve the disagreement between the candidate answers and identify the source of the discrepancy\.\#\#\# OUTPUT\#\#\# ANSWER<at most two concise lines\>\\boxed\{<current best answer\>\}<one concise first\-person sentence\>\#\#\# TASKSWrite one task per line\. Begin each line with "TASK:"\.TASK: <self\-contained task\>

L2\-SSinstead offers the search category a second time, which assumes further exploration is more valuable than checking:

Prompt 6:L2\-SS, step 2 — delegate \(*search*category again\)\#\#\# SUBAGENT RESULTS\{subagent\_results\}\#\#\# INSTRUCTIONSAlways provide the best provisional answer\. Put it in \\boxed\{\}\.Use at most two concise lines to justify the current answer\.Write one concise first\-person sentence explaining why the selected tasks are useful\.Generate 1\-\-3 tasks using one or more strategies from \#\#\# STRATEGIES\. Each task must be independently executable and fully self\-contained\. Include the complete problem or all relevant context, values, assumptions, and requested output\. State candidate answers inside tasks in plain text; never use \\boxed\{\} inside a task\.\#\#\# STRATEGIESAsk 1\-\-3 subagents to:\- solve independent subproblems obtained by decomposing the original problem; or\- solve the full problem independently using different named approaches; or\- solve the full problem independently using the same task and no prescribed approach\.\#\#\# OUTPUT\#\#\# ANSWER<at most two concise lines\>\\boxed\{<current best answer\>\}<one concise first\-person sentence\>\#\#\# TASKSWrite one task per line\. Begin each line with "TASK:"\.TASK: <self\-contained task\>

#### C\.2\.4Return Step

Prompt 7: Return step — the final step with no further delegation\. Identical acrossL1andL2\.\#\#\# SUBAGENT RESULTS\{subagent\_results\}\#\#\# INSTRUCTIONSUpdate the current best answer using the conversation and subagent results\. Compare, select, or briefly combine findings stated by subagents\.You may perform the simple calculations needed to combine the subagent results into the final answer\.Do not delegate additional tasks\.Begin your reply with "\#\#\# RETURN" on its own line \(write the marker literally, even when you are confident\), then at most two concise lines and your final answer\. Put the answer in \\boxed\{\}\.\#\#\# OUTPUT\#\#\# RETURN<at most two concise lines\>\\boxed\{<final answer\>\}

#### C\.2\.5Subagent Prompts

At the maximum depth, subagents cannot delegate further and instead reason inline\. They follow a simple two\-step procedure: first solve the assigned task with inline reasoning, then produce a compact report for the delegating agent\. A subagent never sees the original problem or the delegating agent’s interaction history, only the task assigned by the delegating agent\. The delegating agent has access only to the subagent reports, allowing its working context to remain lean and support multiple rounds of reasoning and delegation within a fixed context limit\. The delegating agent then updates its own reasoning based on the received reports and determines subsequent reasoning and delegation decisions\.

Prompt 8: Subagent, step 1 — solve the assigned task with inline reasoning\. Identical across all levels\.\#\#\# TASK\{task\}\#\#\# INSTRUCTIONSSolve the task carefully, reasoning step by step\. Use the information provided in the task\. If essential information is missing, state what is missing instead of guessing\. Check the result before reporting it\.

Prompt 9: Subagent, step 2 — return a compact report to the delegating agent\. Identical across all levels\.Report the result of the task you just solved\. Keep the report self\-contained and concise\. Include only the decisive findings, not the full derivation\.\#\#\# OUTPUT\#\#\# RESULTScope: <briefly identify what was solved or checked, including essential context\>Finding: <one or two concise sentences with the conclusion and decisive justification\>Answer: \\boxed\{<answer\>\}

#### C\.2\.6Digestion across Context Windows

When reasoning extends beyond a single context window, theHermesharnesses do not carry the full interaction history forward\. Instead, it prompts the model to generate a digest using the templates below and initializes the next context window with the original problem and the generated digest\. When generating a new digest, the model has access to both the previous digest and the complete root\-agent trajectory from the current context window\. In this way, digests compose across an arbitrary number of context windows\.

Prompt 10: Digest system promptYou summarize problem\-solving work so that a fresh solver can continue from your summary alone\. Write at most two paragraphs, in the first person, as the solver\.Cover: \(1\) the approaches attempted and their status \(completed, incomplete and where they stopped, or failed and why\); \(2\) key intermediate results with their verification status \(derived equations, lemmas, computed quantities, checks that passed or failed\); \(3\) the current best answer, confidence, and what remains to be verified\.Hard constraints: retain any important information contained in the existing summary; summarize only \-\-\- do not add new reasoning or new derivations\. These constraints take priority over length: if retaining prior information needs more space, exceed two paragraphs rather than drop information, but compress aggressively \(merge duplicate findings\)\.

Prompt 11: Digest user prompt\#\#\# PROBLEM\{problem\}\#\#\# PREVIOUS SUMMARY\{previous\_digest\}\#\#\# EPISODE TRANSCRIPT\{transcript\}Write the updated summary now\.

The first step of the next context window uses the standard Step\-1 prompt with an additionalPRIOR WORK SUMMARYblock containing the digest:

Prompt 12:L1, step 1 of a*continued*window with a digest underPRIOR WORK SUMMARY\#\#\# PROBLEM\{problem\}\#\#\# PRIOR WORK SUMMARY\{previous\_attempts\}If a summary of prior work is shown above, you may build on it, verify it, or start fresh; otherwise start from the problem\.\#\#\# INSTRUCTIONSDelegate the work for this step; do not solve it inline\. You have no subagent results yet, so do not state a boxed answer on this step\.Write one concise first\-person sentence explaining why the selected tasks are useful\.Generate 1\-\-3 tasks using one or more strategies from \#\#\# STRATEGIES\. Each task must be independently executable and fully self\-contained\. Include the complete problem or all relevant context, values, assumptions, and \-\- when the task checks a candidate answer \-\- that answer and the exact claim to verify\.\#\#\# STRATEGIESAsk 1\-\-3 subagents to:\- solve independent subproblems obtained by decomposing the original problem; or\- solve the full problem independently using different named approaches; or\- solve the full problem independently using the same task and no prescribed approach; or\- verify the current boxed answer using independent targeted checks; or\- independently verify the assumptions and intermediate steps supporting the current boxed answer; or\- resolve the disagreement between the candidate answers and identify the source of the discrepancy\.\#\#\# OUTPUT<one concise first\-person sentence\>\#\#\# TASKSWrite one task per line\. Begin each line with "TASK:"\.TASK: <self\-contained task\>

## Appendix DData Augmentation Details

To expose the model to a broader and balanced range of contextual reasoning behaviors during training, we generate synthetic problems designed to favor different search and verification strategies, including five types: \(1\)*Splitting*, which combines multiple independent problems into a single instance and encourages splitting and parallel delegation; \(2\)*Chaining*, where solving a gating problem determines which of several subsequent problems should be solved, encouraging sequential delegation; \(3\)*Verifying*, where intermediate results may be incorrect, encouraging delegation for verification; \(4\)*Multi\-attempt*, which paraphrases medium\-difficulty problems where multiple independent attempts can improve performance; and \(5\)*Single\-attempt*, which paraphrases low\-difficulty problems where a single attempt is sufficient\. Together, these controlled structures encourage the model to adapt its delegation strategy to the problem\.

### D\.1Synthetic Problem Generation Pipeline

Each synthetic problem is generated in two stages\. First, one or more seed problems are composed into a target synthetic problem type\. The final answer is computed deterministically from the verified reference answers of the seed problems using simple arithmetic rules specific to each type: \(1\)*Splitting\.*Given part answersaia\_\{i\}, the final answer is∑iaimod1000\\sum\_\{i\}a\_\{i\}\\bmod 1000; \(2\)*Chaining\.*Letmmdenote the answer to the gating problem andaia\_\{i\}the answer to theii\-th candidate problem\. The final answer is\(m\+a1\+\(mmod3\)\)mod1000\(m\+a\_\{1\+\(m\\bmod 3\)\}\)\\bmod 1000\. \(3\)*Verifying\.*Given a claimed valueccand the true answeraa, the final answer is\(c\+a\)mod1000\(c\+a\)\\bmod 1000\. For \(4\)*Multi\-attempt*and \(5\)*Single\-attempt*, the answer to the seed problem is kept unchanged\.

Second, the composed problem is rewritten by ClaudeSonnet 5to smooth its writing style and remove potential artifacts\. Rewriting uses the system prompt in Promptand user message in Prompt, where the user message includes a style specification randomly sampled from four candidate writing styles:*narrative*,*compact*,*formal*, and*inverted*\. For*Chaining*problems, a note for composite style \(Prompt\) is additionally included in its style specification\. To further control the quality of the synthetic problems, a rewritten problem is retained only if it remains solvable: ClaudeSonnet 5attempts each problem55times before and55times after rewriting, and the rewrite is rejected if none of the post\-rewrite attempts is correct or if the post\-rewrite solve rate falls to1/31/3of the pre\-rewrite solve rate or below\. This excludes rewrites that make a problem substantially less solvable, indicating that the rewriting has potentially altered the problem\.

Prompt 13: Rewrite system promptYou rewrite mathematics problems\. You will be given a problem, and you must produce a rewritten version that asks for EXACTLY the same final quantity\. Absolute requirements:\- Preserve every number, constraint, condition and definition\. Change no value\.\- Preserve exactly what the final answer must be\. If the original asks for a remainder modulo 1000, so must yours\. If it asks for a sum of several sub\-answers with specific coefficients, so must yours, with the same coefficients\.\- If the original contains several sub\-problems whose answers are combined, keep all of them and keep the combination rule identical\.\- Preserve any quoted claim, reported answer, or summary of prior work, including its stated numeric value\. You may reword it; you may not change the number, and you may not resolve whether it is right\.\- Keep all LaTeX mathematically identical\. You may reformat it\. Style requirements:\- Do NOT signpost how the problem should be approached\. Never write "solve each part separately", "note that these are independent", "first solve X then", or anything that names a solution strategy\.\- Do NOT use the labels "Problem 1", "Problem 2", "\[Problem 2\-A\]", "A\_1", "A\_2", "Q" or similar scaffolding\. Refer to the quantities in words instead\.\- Vary sentence structure\. Avoid a fixed template\. You do not know the answer and must not attempt to compute it\. Do not solve\. Do not hint\. Return ONLY the rewritten problem text\. No preamble, no commentary, no answer\. STYLE RULES\. These come from editors who could reliably tell machine\-written problems from published ones\. Follow every one\. Quantity names\.\- A quantity gets a single italic letter or no name at all\. Never invent a prose name: not "the size sum", not "the pentagon count", not "the divisor\-count remainder", not "the selected quantity", not "the resulting quantity"\. If the source already names quantities with letters, keep those letters\.\- Never reuse one symbol for two things in the same problem\. Structure of the writing\.\- Setup first, question last\. Never open with the ask and defer the setup \("\.\.\., where that value is produced as follows", "\.\.\., obtained as follows"\)\.\- State the ask exactly once\. If your draft opens and closes with the same request, delete the opening\.\- One interrogative sentence, and it is the last sentence\.\- Do not split an ask in two\. Write "Compute the sum of all positive integers n such that \.\.\." rather than "Determine all n \.\.\., then find the sum"\.\- Do not fold the setup into a participial clause \("With p the smallest prime for which \.\.\., find \.\.\."\)\. Use two sentences\. Voice\.\- Third person, present indicative\. No first person anywhere: no "I", "we", "I’d like to know", "I’m curious about", "we’re told", "Here’s", "Picture a", "Now look at"\.\- Keep the source’s named actors and concrete story\. Do not turn "Karthik wants to walk a path" into "A closed path is to be traced" \-\- passive de\-naming is detectable\.\- Never narrate a solution method in any voice\. Words and constructions to avoid outright\.\- "give rise to", "arise as follows", "obtained as follows", "produced as follows", "comes into play", "the quantity in question", "one of the two quantities needed", "a certain resulting value", "it turns out that", "it happens that", "so\-called", "for context", "roughly speaking", "possess" \(write "has"\), "utilize"\.\- No markdown of any kind: no headings, no bold, no italic asides\.\- No em\-dash appositive carrying a constraint, and no colon\-splice that dumps a configuration and then the question\.\- Never write "an isosceles triangle $\\triangle ABC$" \-\- either "triangle $ABC$" or "$\\triangle ABC$"\. Technical fidelity\.\- Copy technical hypotheses verbatim\. Never paraphrase a gcd or coprimality condition: write "$\\gcd\(m,n\)=1$" or "where $m$ and $n$ are relatively prime positive integers" exactly as the source has it\. A paraphrase like "$\\gcd\(m,p,q\)$ shares no common prime factor" is wrong and is a giveaway\.\- Every numeral that was in math mode stays in math mode\. Do not unwrap "$13$" to "13"\.\- Keep the source’s macros, arrays, bulleted condition lists and worked examples in place\.\- Introduce a definition in its own sentence before first use, and give at most one example, opened with "For example" or "For instance"\.\- Stay within about 15% of the source’s length\. Telegraphic compression is as detectable as padding\. Provenance tags\.\- Delete any citation of where the problem came from: "\(26th IMO Shortlist\)", "Example 1", "Problem 4", "2019 AMC 12B", "from the CMO"\. Scraped seeds carry these; a contest booklet does not print them inside the statement, and they identify the source\. Preferred verbs: Let, Suppose, Consider, Compute, Find, How many\.

Prompt 14: Rewrite user message, with one of four candidate style specifications\{style brief\}\{composite note, Chaining problems only\} Here is the problem to rewrite: <problem\>\{generated problem text\}</problem\>Notes:The\{style brief\}is one of the following, sampled per problem\.narrative:Rewrite it as a short, natural piece of mathematical prose, as a person would pose it to a colleague\. Use connected sentences rather than labelled blocks\.compact:Rewrite it as tersely as possible while keeping every constraint\. Prefer one flowing paragraph over enumerated parts\.formal:Rewrite it in the register of a textbook exercise: precise, impersonal, no framing device, no labels beyond what is strictly needed\.inverted:Rewrite it so the final quantity being asked for is stated FIRST, then the information needed to obtain it\.

Prompt 15: Composite style note, appended to the style specification for Chaining problemsThis problem contains several separate questions whose answers are then combined\. Handle that as follows: \- Keep every question, with all of its numbers and conditions\.\- Keep the single\-letter names the problem already uses for the combined quantities, and keep the final combination sentence as it stands\.\- State the final combination using those names, with the SAME coefficients, the SAME operations and the SAME modulus as the original\.\- Present the questions as a continuous piece of writing, not as a list of labelled blocks\.\- Do not say that the questions are independent, unrelated, or separable, and do not suggest any order or division of work\.

### D\.2Synthetic Problem Construction

##### Synthetic problems for training\.

Synthetic training problems are constructed fromAceReason\-1K\. We first remove seed problems with tuple\- or list\-valued answers or malformedLaTeX, leaving 903 seed problems\. Among them, 682 have non\-negative integer answers and can be used to construct Splitting, Chaining, and Verifying problems\.

Using these seed problems, we generate 1,141 synthetic problems, of which 708 pass the solvability check after rewriting\. To broaden coverage of both contextual reasoning strategies and distinct seed problems, we select a final set of 500 synthetic problems covering 505 distinct seeds, with a fixed number of problems per type\. Composite structures \(Splitting and Chaining\) account for 60% of the 500 problems\. Verifying problems are included in matched correct/incorrect pairs\.

The final dataset contains Splitting \(n=214n=214; 107 two\-part and 107 three\-part problems\), Chaining \(n=86n=86\), Verifying \(n=100n=100; 50 correct and 50 incorrect claims paired on the same 50 seed problems\), Multi\-attempt \(n=50n=50\), and Single\-attempt \(n=50n=50\)\. Multi\-attempt and Single\-attempt problems are constructed from disjoint seed sets\. After mixing withAceReason\-5K, the RL training set contains 5,739 problems in total, of which 500 \(8\.7%\) are synthetic\.

##### Synthetic evaluation benchmark\.

The synthetic benchmark is built from AIME and HMMT problems using the same generation, rewriting, and solvability\-check pipeline\. It containsn=88n=88problems: Splitting \(n=28n=28: 14 with two parts and 14 with three parts\), Chaining \(n=14n=14\), Verifying \(n=18n=18: 9 with a correct claim and 9 with an incorrect claim, paired on the same 9 seed problems\), Multi\-attempt \(n=14n=14\), and Single\-attempt \(n=14n=14\)\. Every golden answer is recomputed independently from the reference answers of the seed problems\.

Fig\.shows one example synthetic training problem \(constructed fromAceReason\-1K\) and one example synthetic evaluation problem \(constructed from AIME or HMMT\) for each problem type\.

Training \(AceReason\-1K\)

Evaluation \(AIME / HMMT\)

Splitting

Letmbe the count of four\-digit numbers whose digits sum to1212, letpbe the single digit \(0–9\) giving the position, after12341234moves, of a counter that starts on a1010\-cycle and on itsnnth move advancesnnn^\{n\}steps clockwise, and letqbe the largest number of diagonals selectable in a regular10001000\-gon \(with all diagonals drawn\) so that among any three selected diagonals at least two share the same length\. Find the remainder whenm\+p\+q\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{m\}\}\+\{\\color\[rgb\]\{0\.8516,0\.3945,0\.0508\}\\textbf\{p\}\}\+\{\\color\[rgb\]\{0\.1016,0\.4961,0\.2148\}\\textbf\{q\}\}is divided by10001000\.Golden answer:349\\boxed\{349\}\(m,p,q\)=\(342,7,2000\)\(\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{m\}\},\{\\color\[rgb\]\{0\.8516,0\.3945,0\.0508\}\\textbf\{p\}\},\{\\color\[rgb\]\{0\.1016,0\.4961,0\.2148\}\\textbf\{q\}\}\)=\(342,7,2000\)

Letpdenote the value ofx​yxy, wherexxandyyare real numbers, both greater than11, satisfyinglogx⁡\(yx\)=logy⁡\(x4​y\)=10\\log\_\{x\}\\left\(y^\{x\}\\right\)=\\log\_\{y\}\\left\(x^\{4y\}\\right\)=10\.Separately, Jen enters a lottery by picking44distinct numbers fromS=\{1,2,3,⋯,9,10\}S=\\\{1,2,3,\\cdots,9,10\\\}\. Then44numbers are randomly chosen fromSS\. She wins a prize if at least two of her numbers are among the44randomly chosen numbers, and she wins the grand prize if all four of her numbers are the randomly chosen numbers\. The probability that she wins the grand prize given that she won a prize equalsmn\\tfrac\{m\}\{n\}, wheremmandnnare relatively prime positive integers; letqdenotem\+nm\+n\.Compute the remainder whenp\+q\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{p\}\}\+\{\\color\[rgb\]\{0\.8516,0\.3945,0\.0508\}\\textbf\{q\}\}is divided by10001000\.Golden answer:141\\boxed\{141\}\(p,q\)=\(25,116\)\(\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{p\}\},\{\\color\[rgb\]\{0\.8516,0\.3945,0\.0508\}\\textbf\{q\}\}\)=\(25,116\)

Chaining

Compute the remainder whenm\+p\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{m\}\}\+\{\\color\[rgb\]\{0\.8516,0\.3945,0\.0508\}\\textbf\{p\}\}is divided by10001000, wheremis the answer to the following question: how many four\-digit numbers have the sum of their digits equal to1212?Three further questions are considered\. A counter moves in a cycle of1010positions, and on itsnnth move it advancesnnn^\{n\}steps clockwise; the question asks for the counter’s position after12341234moves, expressed as a single digit from00to99\. In a regular10001000\-gon with all of its diagonals drawn, the question asks for the maximum number of diagonals that can be selected so that among any three of the chosen diagonals at least two have the same length\. For positive integersaa,bb,cc, the question asks for the smallest positive value taken bya3\+b3\+c3−3​a​b​ca^\{3\}\+b^\{3\}\+c^\{3\}\-3abc, together with all triplesaa,bb,ccthat attain this smallest value\.Letpbe the answer to the question numbered1\+\(mmod3\)1\+\(\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{m\}\}\\bmod 3\)among these three, in the order in which they are given here\.Golden answer:349\\boxed\{349\}m=342\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{m\}\}=342, question11,p=7\{\\color\[rgb\]\{0\.8516,0\.3945,0\.0508\}\\textbf\{p\}\}=7

Letmbe the answer to the following question: find the sum of all positive integersnnsuch thatn\+2n\+2divides the product3​\(n\+3\)​\(n2\+9\)3\(n\+3\)\(n^\{2\}\+9\)\.Alongside that, consider three further questions, numbered in order\. The first: Mark writes the expressiona​b​c​d¯\\sqrt\{\\underline\{abcd\}\}on the board\[ … \]The second: every morning Aya goes for a99\-kilometer walk\[ … \]The third: find the number of ordered pairs\(x,y\)\(x,y\)with bothxxandyyintegers between−100\-100and100100, inclusive, satisfying12​x2−x​y−6​y2=012x^\{2\}\-xy\-6y^\{2\}=0\.Letpbe the answer to whichever of these three questions carries the number1\+\(mmod3\)1\+\(\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{m\}\}\\bmod 3\)\. What is the remainder whenm\+p\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{m\}\}\+\{\\color\[rgb\]\{0\.8516,0\.3945,0\.0508\}\\textbf\{p\}\}is divided by10001000?Golden answer:253\\boxed\{253\}m=49\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{m\}\}=49, question22,p=204\{\\color\[rgb\]\{0\.8516,0\.3945,0\.0508\}\\textbf\{p\}\}=204

Verifying

Consider the equationsin⁡x=x100\\sin x=\\dfrac\{x\}\{100\}\. Rowan computes the number of roots of this equation and reports the value63\. Find the remainder when the sum of Rowan’s reported value andthe actual number of rootsis divided by10001000\.Golden answer:126\\boxed\{126\}claim63\(correct\), true value63

LetSSdenote the vertex set of a regular2424\-gon\. Consider ways of drawing1212segments, all of the same length, such that every vertex inSSserves as an endpoint of exactly one segment; call the number of such waysN\. Priya computes this count herself and gets4208\. Find the remainder whenNplus Priya’s value of4208is divided by10001000\.Golden answer:321\\boxed\{321\}claim4208\(incorrect\),N=113\{\\color\[rgb\]\{0\.0586,0\.0391,0\.6328\}\\textbf\{N\}\}=113

Multi\-attempt

Letx,y,zx,y,zbe positive real numbers, and sets=x\+2\+y\+5\+z\+10s=\\sqrt\{x\+2\}\+\\sqrt\{y\+5\}\+\\sqrt\{z\+10\}andt=x\+1\+y\+1\+z\+1t=\\sqrt\{x\+1\}\+\\sqrt\{y\+1\}\+\\sqrt\{z\+1\}\. Find the minimum value ofs2−t2s^\{2\}\-t^\{2\}\.Golden answer:36\\boxed\{36\}

Find the largest number that is less thanB​D2BD^\{2\}for every rhombusA​B​C​DABCDsatisfying the following conditions\. Its four vertices lie, in that order, on the hyperbolax220−y224=1\\frac\{x^\{2\}\}\{20\}\-\\frac\{y^\{2\}\}\{24\}=1, and its diagonals intersect at the origin\.Golden answer:480\\boxed\{480\}

Single\-attempt

Find the smallest positive integer whose digits are all11’s that is divisible by the number consisting of100100copies of the digit33\.Golden answer:300\\boxed\{300\}

Compute the number of positive integersn≤1000n\\leq 1000for whichlcm⁡\(n,9\)\\operatorname\{lcm\}\(n,9\)is a perfect square\.Golden answer:43\\boxed\{43\}

Figure 4:Synthetic problem examples\.Colored letters identify intermediate values and subanswers that determine the final answer\.\[ … \]denotes problem text omitted for brevity\.

## Appendix EExperiment Details

### E\.1Pooled Benchmark Details

We evaluate on a pooled benchmark of 238 mathematical reasoning problems constructed from AIME 2024, AIME 2025, AIME 2026, HMMT February 2023, HMMT February 2024, HMMT February 2025, HMMT November 2025, and HMMT February 2026\. The original collection contains 243 problems \(30 from each contest except HMMT February 2026, which contains 33 problems\)\. We use the HuggingFaceMath\-Verifygrader to evaluate model outputs against ground\-truth answers\.

For HMMT, we exclude five problems following auditing and decontamination: \(1\) HMMT February 2023 Problem 26, which duplicates a training\-data problem; \(2\) HMMT February 2024 Problem 21, which requires a missing diagram; and \(3\) HMMT February 2026 Problems 31–33, which are proof\-style problems whereas theMath\-Verifygrader assumes final\-answer evaluation\. The answers for the remaining 148 problems were then independently derived or exactly checked, and 19 problem transcription and answer corrections were applied to ensure the validity of the benchmark\.

For AIME, we audit all 90 problems and correct a number of issues\. In particular, two AIME 2026 problems did not match the official contest statements and were corrected accordingly\. We further identify corrupted or incomplete problem statements and incorrect answers throughout the datasets\. In total, 25 problems and 19 answers were corrected\. The final pooled benchmark therefore consists of 90 AIME problems and 148 HMMT problems, for a total of 238 problems\.

### E\.2Inference underHermesHarnesses

UnderHermesharness, all agents and subagents share the same underlying LLM and the same88K\-token \(8,1928\{,\}192\-token\) context limit\. Agents are limited to11K \(1,0241\{,\}024\) generated tokens per step\. At the maximum delegation depthDD, subagents follow a two\-step solve\-and\-report procedure \(App\.\) and are allocated up to7\.57\.5K \(7,680\) generated tokens for the first step where they solve delegated tasks inline\. Digest generation \(App\.\) is performed under the same88K\-token context limit\.

For all evaluations, we report average accuracy over 16 sampled solutions \(Avg​@​16\\mathrm\{Avg\}@16\)\. We follow the suggested sampling parameters listed in theQwen3\-4B\-Instruct\-2507andOlmo\-3\-7B\-Instructmodel cards when evaluating their corresponding base and post\-trained models, under inline reasoning as well as theHermes, RSA, and DSM harnesses\. The sampling parameters are listed in Tab\.\.

Table 7:Sampling parameters\.Qwen3\-4B\-Instruct\-2507Olmo\-3\-7B\-InstructTemperature0\.70\.6Top\-pp0\.80\.95Top\-kk20–Min\-pp0\.0–
### E\.3Training Experiments Details

##### Training Dataset\.

Starting from the 5\.7K\-problem subset of AceReason\-Math\[[33](https://arxiv.org/html/2609.38332#bib.bib6)\]released by Reasoning Cache\[[5](https://arxiv.org/html/2609.38332#bib.bib13)\], we perform deduplication, decontamination, and reference\-answer verification, resulting in a cleaned set of 5,239 problems, which we refer to asAceReason\-5K\. We then uniformly sample 1,000 problems fromAceReason\-5Kto constructAceReason\-1K\.

##### Stage I: Supervised Fine\-tuning\.

For SFT, we curate teacher trajectories from both ClaudeSonnet 5andDeepSeek\-V3\.2, underL1andL2Hermesharnesses\. For each teacher model, we collect fourL1\(App\.\), twoL2\-SV, and twoL2\-SS\(App\.\) trajectories for every problem inAceReason\-1K, resulting in 8,000 trajectories per teacher model\. We then retain only trajectories that contain at least one delegation and reach the correct final answer for SFT, resulting in 6,485 trajectories fromSonnet 5and 5,495 trajectories fromDeepSeek\-V3\.2\.

Table 8:Percentage \(%\) of teacher trajectories containing different strategies\. Percentages may sum to more than 100% since a single trajectory can contain multiple strategies\.TeacherSplitSameDifferentVerificationMultiple Search StrategiesSonnet 521\.943\.963\.599\.327\.6DeepSeek\-V3\.226\.070\.426\.695\.122\.6

##### Teacher trajectories provide diverse delegation demonstrations\.

We use an LLM judge \(Section\) to label the contextual reasoning strategies represented by tasks delegated to subagents in successful teacher trajectories \(Table\)\. Both teachers exhibit strong verification behavior and cover a diverse set of search strategies, while exhibiting complementary preferences:Sonnet 5more frequently explores different approaches, whereasDeepSeek\-V3\.2more often performs independent solution attempts\. Collecting trajectories from multiple teachers therefore broadens the coverage of effective delegation demonstrations\.

Table 9:Percentage \(%\) of teacher trajectories containing a search strategy in each step\.TeacherHermesHarnessStep 1 SearchStep 2 SearchSonnet 5L199\.919\.0Sonnet 5L2\-SV100\.015\.5Sonnet 5L2\-SS99\.929\.2DeepSeek\-V3\.2L199\.535\.4DeepSeek\-V3\.2L2\-SV99\.628\.9DeepSeek\-V3\.2L2\-SS99\.953\.2

##### Different harnesses further increase delegation diversity\.

While most teacher trajectories begin with a search task, second\-step behavior varies across harnesses due to differences in steering \(Table\)\.L2\-SSencourages continued exploration via search, whereasL2\-SVencourages verification and therefore exhibits the lowest search frequency\.L1lies between these two, where the model has the highest control over its delegation decisions\. Collecting teacher trajectories from bothL1andL2therefore exposes the base model to a broader range of effective delegation patterns\.

To best leverage this diversity while preventing any single teacher or delegation pattern from dominating the training data, we subsample each teacher’s retained trajectories to balance the occurrence of search strategies before combining them into a final SFT dataset of 5,466 trajectories \(2,733 from each teacher\)\. This exposes the base model to a diverse and balanced set of delegation behaviors\.

##### Stage II: Reinforcement learning\.

For the RL stage, we train onAceReason\-5Kusing a curriculum over theHermeshierarchy\. Training runs for 492 steps \(3 epochs\), and the curriculum is defined as: over the first third of training \(steps 1–164\), the twoL2harnesses,L2\-SVandL2\-SS, are each sampled with 50% probability, and over the remainder \(steps 165–492\) theL1harness is sampled with 75% probability while the twoL2harnesses are each sampled with 12\.5% probability\. This curriculum gradually shifts training fromL2toL1while maintaining equal overall exposure to both levels of harnesses throughout training\. During RL training, both the root agent and all subagents use the current policy being updated\.

As described in App\., we train RL on a mixed dataset includingAceReason\-5Kand 500 synthetic problems constructed fromAceReason\-1K, comprising 5,739 problems in total\. All training hyperparameters are provided in Tab\.\.

Table 10:Training hyperparameters\.*Supervised Fine\-tuning*Learning rate10−510^\{\-5\}Learning scheduleCosine with warmup ratio 0\.1Training batch size32Epochs4*Reinforcement Learning*AlgorithmDr\. GRPO\[[31](https://arxiv.org/html/2609.38332#bib.bib23)\]Group sizeGG8Dynamic samplingResample while a group is all\-correct or all\-wrongResampling budget≤10\\leq 10generation batches per optimizer stepClipping\(ϵlow,ϵhigh\)\(\\epsilon\_\{\\mathrm\{low\}\},\\epsilon\_\{\\mathrm\{high\}\}\)\(0\.2,0\.28\)\(0\.2,\\ 0\.28\)KL loss coefficient0\.0010\.001KL estimatork3k\_\{3\}\(low\-variance\)Entropy loss coefficient0\.0OptimizerAdamW,β=\(0\.9,0\.999\)\\beta=\(0\.9,\\ 0\.999\)Weight decay0\.010\.01Learning rate10−610^\{\-6\}Learning scheduleConstant, no warm\-upGradient clipping1\.01\.0Training batch size32Mini\-batch size16Epochs3Inference temperature1\.01\.0Inference top\-pp1\.01\.0

### E\.4Labeling Executed Contextual Reasoning Strategies

For qualitative analysis of contextual reasoning behaviors, we set up an LLM judge with ClaudeSonnet\-5to label the strategies executed in agent trajectories\. We take several measures to prevent information leakage during labeling:

1. 1\.The judge sees only the problem statement, the texts generated by the delegating agent, and the results returned by subagents from previous delegation steps\. It never observes the harness identity, model identity, steering instructions, available strategy menu, or any subagent trajectories\. Consequently, the judge must infer strategy labels from the tasks generated by the delegating agent, rather than relying on strategy names contained in the steering instructions or on the outcomes of the delegated tasks being labeled\.
2. 2\.Since task ordering carries no semantic meaning, delegation tasks are sorted according to their text before labeling\. This prevents the judge from relying on presentation order when labeling strategies\.
3. 3\.Residual leakage remains possible\. In particular, the number of delegated tasks may reveal information about the underlying harness\. To mitigate this effect, all reported comparisons using judge labels are performed between models evaluated under harnesses with the same maximum number of subagents, preventing the judge from relying on such differences\.

The judge returns a set of strategy labels for each delegation step, as a single step may contain tasks corresponding to multiple strategies\. Because ClaudeSonnet\-5does not accept a temperature parameter during inference calls, the judge cannot be forced to use deterministic decoding\. We therefore measure consistency between two independent judge passes on a small subset of trajectories and obtain 79\.4% agreement\. The labeling rubric is specified in the system prompt \(Prompt\), and the judge is required to call thelabel\_trajectorytool \(schema shown in Prompt\) to produce its labels\.

Prompt 16: Judge system promptYou are reading the instructions a manager sent to its assistants across the steps of ONE problem\.You see ONLY the problem, the instruction texts, and \(for later steps\) the results that came back\.You do NOT know which model wrote them or what it claimed to be doing\.For EACH step, report every category that the instructions on that step actually execute\. Judge the WORK REQUESTED, not the wording, and not whether it succeeded\. \#\# Categories\*\*Checking work that already exists:\*\*\- ‘verify‘ \-\-\- an instruction checks, re\-derives, confirms, reconciles, or attacks a result that ALREADY EXISTS in the prior results shown to you, or is quoted verbatim inside the instruction\.Work an instruction performs itself is not a prior result\.When you use ‘verify‘, add sub\-tags: ‘answer‘ \(rechecks the stated final answer\), ‘steps‘ \(checks a named intermediate step, equation, or assumption\), ‘disagree‘ \(two or more stated candidates conflict and the instructions decide between them\), ‘counter‘ \(tries to break a stated result or find a second solution\)\. \*\*Sub\-tags are descriptive only \-\-\- they are never a reason to prefer or reject ‘verify‘\.\*\*\*\*Solving from scratch:\*\*\- ‘split\_subproblems‘ \-\-\- the instructions ask for different pieces, and the pieces must be combined to answer the question\. Requires≥\\geq2 instructions\.\- ‘same\_solve‘ \-\-\- the instructions all ask for the whole answer and NO instruction names a solution technique\. Rewordings of "solve this problem" are ‘same\_solve‘, however differently phrased\. Requires≥\\geq2 instructions\.\- ‘different\_methods‘ \-\-\- every instruction names at least one solution technique, and the named sets are pairwise different\. A technique counts only when it appears in a naming position \("using X", "via X", "apply X", "by X"\)\. Different wording, different guesses about an ambiguous problem, or different algebraic rearrangements of the same equation are NOT different methods\. Requires≥\\geq2 instructions\.\- ‘one\_fresh\_attempt‘ \-\-\- a single instruction asks for one new attempt at the whole problem\. \#\# MultilabelA step may execute more than one category \-\-\- report all that apply\. The common real case is a step that mixes a check of an existing result with fresh solving: label it ‘verify‘ \*\*and\*\* the appropriate search category\. Do not add a label merely because it is arguably present; add it when some instruction on that step plainly does that work\.Constraints, enforced downstream in code as well:\- ‘split\_subproblems‘, ‘same\_solve‘, ‘different\_methods‘ each require≥\\geq2 instructions on that step\.\- With exactly one instruction, the only admissible labels are ‘one\_fresh\_attempt‘ and ‘verify‘\.\- ‘same\_solve‘ and ‘different\_methods‘ may BOTH apply to one step \-\-\- see the mixed\-turn rule below\.\- Never return an empty label list for a step\. Every step has a best fit\. \#\#\# Mixed fanout turns carry both labelsA turn where \*\*some\*\* instructions name a technique and others do not is genuinely both: it fans out on the whole problem, and it assigns a method to part of that fanout\. Report ‘\[same\_solve, different\_methods\]‘ rather than forcing a choice\.\- Every instruction names a technique, sets differ pairwise→\\rightarrow‘different\_methods‘ alone\.\- No instruction names a technique→\\rightarrow‘same\_solve‘ alone\.\- Some do, some don’t→\\rightarrow\*\*both\*\*\.This is why the label set is multi\-label: the two are not opposite ends of one axis, and a forced pick on a mixed turn is a coin flip that neither rater can make reliably \(measured: matching prevalences with per\-labelκ\\kappa≈\\approx0\.16\-\-0\.29, i\.e\. raters agreed on how often, never on which turns\)\. \#\# Two rules that decide most cases1\. \*\*Count the instructions\.\*\* Sub\-parts inside one instruction are still one instruction\.2\. \*\*‘same\_solve‘ vs ‘different\_methods‘:\*\* list the techniques each instruction explicitly names in a naming position\. If any instruction names none, the answer is ‘same\_solve‘\.\#\#\# Applying rule 2 \-\-\- do this literally, instruction by instructionMeasured: judges over\-call ‘different\_methods‘ on any turn whose instructions merely \*read\* differently\. Work the test mechanically instead of judging overall impression\.For each instruction, write down the technique it names in a naming position \("using X", "via X", "apply X", "by X", "approach it by X"\)\. Then:\- \*\*Any instruction with an empty list→\\rightarrow‘same\_solve‘\.\*\* One instruction naming a technique while another just says "solve this problem" is ‘same\_solve‘, not ‘different\_methods‘\. The rule is about the weakest instruction, not the strongest\.\- \*\*‘different\_methods‘ requires every instruction to name a technique, and the named sets to differ pairwise\.\*\* All naming the same technique is ‘same\_solve‘\.These are NOT techniques, even when the instructions differ visibly:\- a restatement of the problem’s own given equations, constraints, or setup;\- a different algebraic rearrangement, substitution, or factoring of the same equation;\- a suggested first step or observation \("note that subtracting \(1\) from \(2\) gives …"\);\- different guesses about an ambiguous problem;\- more or less detail, or different wording, in an otherwise identical request\.Worked examples, both ‘same\_solve‘:\- I1 "Solve this set theory problem \*\*using a Venn diagram region approach\*\*: …" / I2 "Solve this set theory problem: …"→\\rightarrowI2 names none, so ‘same\_solve‘\. \(I1’s Venn technique does not carry the turn\.\)\- I1 "…\*\*Approach it by\*\* treating \(1\) and \(2\) as showing b and c are roots of the same quadratic" / I2 "…\*\*Note that\*\* subtracting \(1\) from \(2\) gives …"→\\rightarrowboth are rearrangements of the same system, not distinct methods\. ‘same\_solve‘\.‘split\_subproblems‘ is about the \*\*requested output\*\*, not the reasoning shown\. An instruction that merely restates the problem’s given setup, and asks for nothing but the final answer, is not a subproblem\.But an instruction that asks for a \*\*named intermediate quantity\*\* \-\-\- "determine the radii r1 and r2", "express the perimeters in terms of a and b", "compute the number of integers k" \-\-\- is requesting a subproblem, \*\*even when the same instruction also asks for the final answer\*\*\. Report ‘\[split\_subproblems, same\_solve\]‘ \(or with ‘different\_methods‘, or with ‘verify‘\) in that case\. Do not drop ‘split\_subproblems‘ just because the final quantity is also requested: co\-occurrence is the point of a multi\-label space, and every label the turn plainly executes should appear\.Only when NO instruction requests any named intermediate \-\-\- every one asks solely for the final quantity \-\-\- is the turn pure fanout with no ‘split\_subproblems‘\. \#\# ConfidencePer step, ‘HIGH‘ when confident, ‘LOW‘ when the texts are truncated, vague, or you had to guess\.\#\# Why verify is one category, not fourTwo independent judges on 200 blinded items agreed 74\.0% \(Cohen’sκ\\kappa0\.644\) over an 8\-way space, but \*\*78\.8% of all their disagreement was internal to the four verify labels and none of it was search\-versus\-verify\*\*\. Collapsing verify to a single category raised agreement to 94\.5% \(κ\\kappa0\.901\)\.The four verify distinctions are therefore sub\-tags that carry NO reliability claim; ‘answer‘ versus ‘disagree‘ alone scoredκ\\kappa0\.354\.‘different\_methods‘ versus ‘same\_solve‘ is the weakest surviving distinction \(κ\\kappa0\.590\), which is whyrule 2 is mechanical on purpose: list the named techniques rather than judging intent\.\(\!\)Those figures validated a \*\*single\-label\*\* version of this rubric\. This rubric is multilabel, so they do not transfer \-\-\- agreement must be re\-measured under this schema\. Do not quoteκ\\kappa0\.901 for the multilabel judge\.Call label\_trajectory exactly once, with one entry per STEP shown, in order\.

Prompt 17:label\_trajectorytool schema[⬇](data:text/plain;base64,ewogICJ0b29sU3BlYyI6IHsKICAgICJuYW1lIjogImxhYmVsX3RyYWplY3RvcnkiLAogICAgImRlc2NyaXB0aW9uIjogIlJlY29yZCB0aGUgZXhlY3V0ZWQtc3RyYXRlZ3kgbGFiZWxzIGZvciBldmVyeSBzdGVwIG9mIHRoaXMgdHJhamVjdG9yeS4iLAogICAgImlucHV0U2NoZW1hIjogewogICAgICAianNvbiI6IHsKICAgICAgICAidHlwZSI6ICJvYmplY3QiLAogICAgICAgICJyZXF1aXJlZCI6IFsKICAgICAgICAgICJ0dXJucyIKICAgICAgICBdLAogICAgICAgICJwcm9wZXJ0aWVzIjogewogICAgICAgICAgInR1cm5zIjogewogICAgICAgICAgICAidHlwZSI6ICJhcnJheSIsCiAgICAgICAgICAgICJpdGVtcyI6IHsKICAgICAgICAgICAgICAidHlwZSI6ICJvYmplY3QiLAogICAgICAgICAgICAgICJyZXF1aXJlZCI6IFsKICAgICAgICAgICAgICAgICJ0dXJuIiwKICAgICAgICAgICAgICAgICJsYWJlbHMiLAogICAgICAgICAgICAgICAgImNvbmZpZGVuY2UiCiAgICAgICAgICAgICAgXSwKICAgICAgICAgICAgICAicHJvcGVydGllcyI6IHsKICAgICAgICAgICAgICAgICJ0dXJuIjogewogICAgICAgICAgICAgICAgICAidHlwZSI6ICJpbnRlZ2VyIgogICAgICAgICAgICAgICAgfSwKICAgICAgICAgICAgICAgICJsYWJlbHMiOiB7CiAgICAgICAgICAgICAgICAgICJ0eXBlIjogImFycmF5IiwKICAgICAgICAgICAgICAgICAgIm1pbkl0ZW1zIjogMSwKICAgICAgICAgICAgICAgICAgIml0ZW1zIjogewogICAgICAgICAgICAgICAgICAgICJ0eXBlIjogInN0cmluZyIsCiAgICAgICAgICAgICAgICAgICAgImVudW0iOiBbCiAgICAgICAgICAgICAgICAgICAgICAic3BsaXRfc3VicHJvYmxlbXMiLAogICAgICAgICAgICAgICAgICAgICAgInNhbWVfc29sdmUiLAogICAgICAgICAgICAgICAgICAgICAgImRpZmZlcmVudF9tZXRob2RzIiwKICAgICAgICAgICAgICAgICAgICAgICJvbmVfZnJlc2hfYXR0ZW1wdCIsCiAgICAgICAgICAgICAgICAgICAgICAidmVyaWZ5IgogICAgICAgICAgICAgICAgICAgIF0KICAgICAgICAgICAgICAgICAgfQogICAgICAgICAgICAgICAgfSwKICAgICAgICAgICAgICAgICJ2ZXJpZnlfc3VidGFncyI6IHsKICAgICAgICAgICAgICAgICAgInR5cGUiOiAiYXJyYXkiLAogICAgICAgICAgICAgICAgICAiaXRlbXMiOiB7CiAgICAgICAgICAgICAgICAgICAgInR5cGUiOiAic3RyaW5nIiwKICAgICAgICAgICAgICAgICAgICAiZW51bSI6IFsKICAgICAgICAgICAgICAgICAgICAgICJhbnN3ZXIiLAogICAgICAgICAgICAgICAgICAgICAgInN0ZXBzIiwKICAgICAgICAgICAgICAgICAgICAgICJkaXNhZ3JlZSIsCiAgICAgICAgICAgICAgICAgICAgICAiY291bnRlciIKICAgICAgICAgICAgICAgICAgICBdCiAgICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAiY29uZmlkZW5jZSI6IHsKICAgICAgICAgICAgICAgICAgInR5cGUiOiAic3RyaW5nIiwKICAgICAgICAgICAgICAgICAgImVudW0iOiBbCiAgICAgICAgICAgICAgICAgICAgIkhJR0giLAogICAgICAgICAgICAgICAgICAgICJMT1ciCiAgICAgICAgICAgICAgICAgIF0KICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgICB9CiAgICAgICAgICAgIH0KICAgICAgICAgIH0KICAgICAgICB9CiAgICAgIH0KICAgIH0KICB9Cn0=)\{"toolSpec":\{"name":"label\_trajectory","description":"Recordtheexecuted\-strategylabelsforeverystepofthistrajectory\.","inputSchema":\{"json":\{"type":"object","required":\["turns"\],"properties":\{"turns":\{"type":"array","items":\{"type":"object","required":\["turn","labels","confidence"\],"properties":\{"turn":\{"type":"integer"\},"labels":\{"type":"array","minItems":1,"items":\{"type":"string","enum":\["split\_subproblems","same\_solve","different\_methods","one\_fresh\_attempt","verify"\]\}\},"verify\_subtags":\{"type":"array","items":\{"type":"string","enum":\["answer","steps","disagree","counter"\]\}\},"confidence":\{"type":"string","enum":\["HIGH","LOW"\]\}\}\}\}\}\}\}\}\}

### E\.5Additional Delegation Behavior Analysis

Figure 5:*\(a\)*Hermes\-Learnmakes strategy selection increasingly problem\-dependent\.*\(b\)*Correct trajectories perform verification more frequently\.*\(c\)*The second step primarily performs verification and otherwise tends to continue the first\-step strategy\.*\(d\)*Verification behavior depends on the first\-step search strategy\.Fig\.provides a more detailed view of the delegation behaviors analyzed in Sec\.\. As described in Sec\., panel \(a\) shows that training makes strategy selection increasingly problem\-dependent:Splitremains common on synthetic problems designed to benefit from decomposition, but becomes much less frequent on the pooled benchmark\. Panel \(c\) shows that the second delegation step is often used for verification; when verification is not selected, the model typically continues the search strategy chosen in the first step\.

Panel \(b\) indicates that trajectories with correct final answers perform verification more often than trajectories with incorrect final answers\. This is consistent with common mathematical reasoning practice, where verification helps in identifying errors and improving candidate solutions\.

Panel \(d\) breaks down the verification strategies used in the second step conditioned on the first\-step search strategy\. When the first step usesSameorDifferent, the model distributes its verification effort acrossAnswer Verification\(Answer\),Intermediate Step Verification\(Steps\), andDisagreement Resolution\(Disagree\)\. These verification patterns are intuitive, as independent attempts or distinct approaches can potentially generate conflicting candidate answers thatDisagreement Resolutioncould reconcile\. In contrast, when the first step usesSplit,Disagreement Resolutionbecomes much less common, whileIntermediate Step Verificationincreases\. This is consistent with the structure of decomposition\-based reasoning: sinceSplitdecomposes a problem into subproblems, verification naturally focuses more on validating intermediate results produced by solving subproblems\.

Together, these results suggest that the model adapts its contextual reasoning strategies to both the problem structure and the reasoning progress, providing further evidence that models acquire flexible and adaptive contextual reasoning behaviors throughHermes\-Learn\.

## Appendix FExample Rollouts

In this section, we show example rollouts sampled from theHermes\-Learnmodel \(after both SFT and RL\) on example evaluation problems\.

### F\.1L1Rollout

Under theHermesL1harness, the model has access to all six delegation strategies at each step\. In Rollout, the agent first launches three independent solution attempts \(theSamestrategy\), which produce conflicting answers\. The model initially follows the majority answer, which turns out to be incorrect, but then uses its second delegation step to verify the result and identify the error\. Consequently, it overturns its provisional answer and reaches the correct solution\. Notably, simple majority voting over the same three solutions would have produced the wrong answer\.

Rollout 1:L1on AIMEProblemaime\_2026:4, reference answer70Find the number of integers less than or equal to 100 that are equal toa\+b\+a​ba\+b\+abfor some choice of distinct positive integers a and b\.Root, step 1 — delegateI will have multiple subagents independently solve this combinatorics problem so I can cross\-check their approaches and answers\.TASK:Find the number of integers less than or equal to 100 that can be expressed as a\+b\+ab for some choice of distinct positive integers a and b\. Show your reasoning and give the final count\.TASK:Solve this problem: Find the number of integers less than or equal to 100 that are equal to a\+b\+ab for some choice of distinct positive integers a and b\. Provide a full derivation and the final numerical answer\.TASK:Solve: Find the number of integers≤\\leq100 of the form a\+b\+ab where a and b are distinct positive integers\. Show your work and state the final count\.3 subagents run in parallel in fresh 8K windows→\\rightarrowreports returned to the root at step 2Task 1\.Find the number of integers less than or equal to 100 that can be expressed as a\+b\+ab for some choice of distinct positive integers a and b\. Show your reasoning and give the final count\.\[ subagent 1 reasoning: 15,431 characters omitted \]\#\#\# RESULTScope: Count integers≤\\leq100 expressible as $a\+b\+ab$ for distinct positive integers $a,b$, using substitution $a\+1=x$, $b\+1=y$ to get $xy\-1\\le100$ with $x\\ne y$, $x,y\\ge2$\.Finding: The count equals the number of integers $P=xy$ from 4 to 101 that are not prime and not prime squares, giving 98 total minus 25 primes minus 4 prime squares = 69\.Answer: \\boxed\{69\}Task 2\.Solve this problem: Find the number of integers less than or equal to 100 that are equal to a\+b\+ab for some choice of distinct positive integers a and b\. Provide a full derivation and the final numerical answer\.\[ subagent 2 reasoning: 15,301 characters omitted \]\#\#\# RESULTScope: Find the number of integers≤\\leq100 expressible as a\+b\+ab for distinct positive integers a,b; derived via rewriting as \(a\+1\)\(b\+1\)\-1 and counting N=a\+1,b\+1 product values in \[6,101\] excluding primes and prime squares\.Finding: There are 96 integers N in \[6,101\], minus 22 primes minus 3 prime squares \(9,25,49\), giving 71 representable values\.Answer: \\boxed\{71\}Task 3\.Solve: Find the number of integers≤\\leq100 of the form a\+b\+ab where a and b are distinct positive integers\. Show your work and state the final count\.\[ subagent 3 reasoning: 21,344 characters omitted \]\#\#\# RESULTScope: Count distinct values of $a\+b\+ab \\le 100$ for distinct positive integers $a,b$, reparameterized as $\(a\+1\)\(b\+1\)\-1$, equivalent to counting composite numbers≤\\leq101 that are not perfect squares of primes\.Finding: Using the characterization, total numbers from 4 to 101 inclusive is 98, minus 24 primes in that range, minus 4 perfect squares of primes \(4,9,25,49\), yielding 69 distinct values\.Answer: \\boxed\{69\}Root, step 2 — provisional answer, then delegate\#\#\# ANSWERTwo subagents got 69 and one got 71; the discrepancy lies in counting primes and prime squares in range 4\-101, so I need to resolve this carefully\.\\boxed\{69\}I will have subagents carefully recompute the count of primes and prime squares in the relevant range and double check the derivation\.TASK:Solve carefully: Find the number of integers less than or equal to 100 that can be expressed as a\+b\+ab for distinct positive integers a and b\. Use the substitution a\+1=x, b\+1=y \(so x,y are distinct integers≥\\geq2\), giving a\+b\+ab = xy\-1\. Since a,b are distinct, x≠\\neqy\. We need xy\-1≤\\leq100, i\.e\., xy≤\\leq101, with x≥\\geq2,y≥\\geq2,x≠\\neqy, both integers\. Determine: \(1\) how many pairs \(x,y\) with x≠\\neqy give distinct values of xy, \(2\) how many such products xy lie in range 4 to 101, \(3\) which values of xy\-1 are NOT representable \(i\.e\., xy is prime or a perfect square of a prime\), \(4\) compute the final count\. Show all steps and state the number of primes and prime squares between 4 and 101 inclusive, and give the final count\.TASK:Verify the following solution to this problem: "Find the number of integers≤\\leq100 expressible as a\+b\+ab for distinct positive integers a,b\." The solution says rewrite as \(a\+1\)\(b\+1\)\-1, so let x=a\+1,y=b\+1 with x,y≥\\geq2 distinct integers, and xy\-1≤\\leq100 so xy≤\\leq101\. The number of such representable integers equals the number of composite integers N in \[4,101\] that are not perfect squares of primes, since xy with x≠\\neqy≥\\geq2 both integers means N=xy is composite and not a square of a single prime \(since x≠\\neqy, N cannot be a perfect square at all, and also cannot be a prime times 1\.\.\. actually check: can xy with x≠\\neqy≥\\geq2 both≥\\geq2 be prime? No, xy would be composite\)\. Carefully verify: \(a\) is every composite number in \[4,101\] representable as xy with x≠\\neqy≥\\geq2 integers? \(b\) Are there any composite numbers in \[4,101\] that CANNOT be written as xy with x≠\\neqy≥\\geq2? \(c\) List all primes and prime squares in \[4,101\] and count them\. \(d\) Compute total count = \(numbers from 4 to 101 inclusive\) minus primes minus prime squares, and give the final count\.TASK:Solve independently: Find the number of integers n≤\\leq100 such that n = a\+b\+ab for some distinct positive integers a,b\. Provide step\-by\-step reasoning and a final numerical answer\.3 subagents run in parallel in fresh 8K windows→\\rightarrowreports returned to the root at step 3Task 1\.Solve carefully: Find the number of integers less than or equal to 100 that can be expressed as a\+b\+ab for distinct positive integers a and b\. Use the substitution a\+1=x, b\+1=y \(so x,y are distinct integers≥\\geq2\), giving a\+b\+ab = xy\-1\. Since a,b are distinct, x≠\\neqy\. We need xy\-1≤\\leq100, i\.e\., xy≤\\leq101, with x≥\\geq2,y≥\\geq2,x≠\\neqy, both integers\. Determine: \(1\) how many pairs \(x,y\) with x≠\\neqy give distinct values of xy, \(2\) how many such products xy lie in range 4 to 101, \(3\) which values of xy\-1 are NOT\[ 214 more characters omitted \]\[ subagent 1 reasoning: 17,331 characters omitted \]\#\#\# RESULTScope: Count integers≤\\leq100 expressible as a\+b\+ab for distinct positive integers a,b, using substitution a\+1=x, b\+1=y giving a\+b\+ab=xy\-1 with x≠\\neqy≥\\geq2, xy≤\\leq101\.Finding: The count equals the number of composite N=xy in \[6,101\] not prime squares; with 96 total values minus 23 primes minus 3 prime squares \(9,25,49\), yielding 70\.Answer: \\boxed\{70\}Task 2\.Verify the following solution to this problem: "Find the number of integers≤\\leq100 expressible as a\+b\+ab for distinct positive integers a,b\." The solution says rewrite as \(a\+1\)\(b\+1\)\-1, so let x=a\+1,y=b\+1 with x,y≥\\geq2 distinct integers, and xy\-1≤\\leq100 so xy≤\\leq101\. The number of such representable integers equals the number of composite integers N in \[4,101\] that are not perfect squares of primes, since xy with x≠\\neqy≥\\geq2 both integers means N=xy is composite and not a square of a single prime \(since x≠\\neqy, N\[ 526 more characters omitted \]\[ subagent 2 reasoning: 12,567 characters omitted \]\#\#\# RESULTScope: Verified the solution’s reasoning and computed the count via complementary counting: composite numbers in \[4,101\] excluding prime squares\.Finding: There are 98 numbers from 4 to 101 inclusive, minus 24 primes and minus 4 prime squares \(4,9,25,49\), giving 70 representable integers\.Answer: \\boxed\{70\}Task 3\.Solve independently: Find the number of integers n≤\\leq100 such that n = a\+b\+ab for some distinct positive integers a,b\. Provide step\-by\-step reasoning and a final numerical answer\.\[ subagent 3 reasoning: 20,868 characters omitted \]Scope: Find the number of integers n≤\\leq100 such that n = a\+b\+ab for distinct positive integers a,b, using the substitution n = \(a\+1\)\(b\+1\)\-1 to reduce to counting N = n\+1 as products of two distinct integers≥\\geq2\.Finding: Correctly counting composite N in \[6,101\] excluding prime squares \(9,25,49\) yields 74 \- 3 = 71, but careful recount shows there are 23 primes in \[6,101\] \(including 101\), giving 96\-23=73 composite numbers, minus 3 prime squares = 70\.Answer: \\boxed\{70\}Root, step 3 — return\#\#\# RETURNThree independent subagents now agree via careful recomputation: 96 values in range minus 23 primes minus 3 prime squares gives 70\.\\boxed\{70\}

### F\.2L2\-SVRollout

Rolloutillustrates reasoning under theL2\-SVharness, which provides stronger steering thanL1\. By restricting the second delegation step to verification,L2\-SVimposes additional structure on contextual reasoning decisions while still requiring the model to interpret and synthesize information returned by subagents\.

Rollout 2:L2\-SVon HMMTProblemhmmt\_feb\_2026:2, reference answer420261Compute the second smallest positive integernnsuch that\-nnis divisible by 101, and\- the decimal representation ofnncontains the number 2026 as a consecutive sequence of digits\.Root, step 1 — delegateI will dispatch independent subagents to solve the full problem using different approaches, so I can cross\-check their answers\.TASK:Compute the second smallest positive integer n such that n is divisible by 101, and the decimal representation of n contains the number 2026 as a consecutive sequence of digits\. Show your reasoning and give the final numerical value of n\.TASK:Find the second smallest positive integer divisible by 101 whose decimal representation contains "2026" as a consecutive digit string\. Provide the value of n and show your work\.TASK:Solve for the second smallest positive integer n divisible by 101 such that n’s decimal digits contain "2026" appearing consecutively\. Show step\-by\-step reasoning and state the final answer clearly\.3 subagents run in parallel in fresh 8K windows→\\rightarrowreports returned to the root at step 2Task 1\.Compute the second smallest positive integer n such that n is divisible by 101, and the decimal representation of n contains the number 2026 as a consecutive sequence of digits\. Show your reasoning and give the final numerical value of n\.\[ subagent 1 reasoning: 19,096 characters omitted \]\#\#\# RESULTScope: Find the second smallest positive integer divisible by 101 that contains "2026" as a consecutive digit sequence, using modular arithmetic and systematic search\.Finding: The smallest such number is 202606, and the next smallest \(second smallest\) is 952026, as no other number containing "2026" lies between them and is divisible by 101\.Answer: \\boxed\{952026\}Task 2\.Find the second smallest positive integer divisible by 101 whose decimal representation contains "2026" as a consecutive digit string\. Provide the value of n and show your work\.\[ subagent 2 reasoning: 17,067 characters omitted \]\#\#\# RESULTScope: Find the second smallest positive integer divisible by 101 containing "2026" as a consecutive digit string\.Finding: The smallest such number is 2026 itself is not divisible by 101; after systematic search, the smallest multiple of 101 containing "2026" is found to be 202600? Actually, from earlier work, I need to compute properly\.Answer: \\boxed\{202600\} \(Note: This is a placeholder; the correct value must be determined by a complete systematic search or algorithm, which was not fully completed in the thought process\.\)Task 3\.Solve for the second smallest positive integer n divisible by 101 such that n’s decimal digits contain "2026" appearing consecutively\. Show step\-by\-step reasoning and state the final answer clearly\.\[ subagent 3 reasoning: 18,289 characters omitted \]\#\#\# RESULTScope: Find the second smallest positive integer divisible by 101 whose decimal digits contain "2026" consecutively\.Finding: The smallest such number is 952026 \(from pattern XY2026 with digits 9,5,2,0,2,6\), and the next smallest is obtained by incrementing the coefficient in that pattern to the next multiple, yielding 1962026\.Answer: \\boxed\{1962026\}Root, step 2 — provisional answer, then delegate\#\#\# ANSWERThree subagents disagree \(952026, 202600, 1962026\), so I need independent rigorous derivation and cross\-checking before trusting any value\.\\boxed\{952026\}I will dispatch subagents to carefully redo the full derivation from scratch and cross\-check the smallest and second smallest values independently\.TASK:Find the smallest positive integer divisible by 101 such that its decimal representation contains "2026" as a consecutive digit string\. Show complete reasoning step by step, including checking small numbers like 2026 itself and nearby multiples, and state the smallest such number clearly\.TASK:Given that the smallest positive integer divisible by 101 containing "2026" as consecutive digits is 952026 \(verify or refute this claim\), find the second smallest such positive integer\. Show full reasoning and state the final answer clearly\.TASK:Solve this problem carefully and completely: Compute the second smallest positive integer n such that n is divisible by 101 and the decimal representation of n contains the number 2026 as a consecutive sequence of digits\. Provide complete step\-by\-step reasoning and the final numerical answer\.3 subagents run in parallel in fresh 8K windows→\\rightarrowreports returned to the root at step 3Task 1\.Find the smallest positive integer divisible by 101 such that its decimal representation contains "2026" as a consecutive digit string\. Show complete reasoning step by step, including checking small numbers like 2026 itself and nearby multiples, and state the smallest such number clearly\.\[ subagent 1 reasoning: 17,190 characters omitted \]\#\#\# RESULTScope: Find the smallest positive integer divisible by 101 containing "2026" as a consecutive digit string, checking small numbers and nearby multiples\.Finding: No 4\-digit or 5\-digit number containing "2026" is divisible by 101 \(proven via modular arithmetic\), so the search moves to 6\-digit numbers; the smallest such multiple found is 202606 = 101×\\times2006, which contains "2026" and no smaller candidate works\.Answer: \\boxed\{202606\}Task 2\.Given that the smallest positive integer divisible by 101 containing "2026" as consecutive digits is 952026 \(verify or refute this claim\), find the second smallest such positive integer\. Show full reasoning and state the final answer clearly\.\[ subagent 2 reasoning: 18,221 characters omitted \]\#\#\# RESULTScope: Verified that 952026 is divisible by 101 and contains "2026"; no 5\-digit number containing "2026" is divisible by 101, so 952026 is the smallest such positive integer\.Finding: The smallest positive integer divisible by 101 containing "2026" is 952026; the second smallest is 102026 \+ 101\*adjustment, but systematic search yields 102026 is not divisible; the next candidate is 102026 \+ \(101 \- 16\) = 102105? Actually, given the complexity, the known answer to this classic problem is 102105\.Answer: \\boxed\{102105\}Task 3\.Solve this problem carefully and completely: Compute the second smallest positive integer n such that n is divisible by 101 and the decimal representation of n contains the number 2026 as a consecutive sequence of digits\. Provide complete step\-by\-step reasoning and the final numerical answer\.\[ subagent 3 reasoning: 17,649 characters omitted \]Scope: Find the second smallest positive integer divisible by 101 containing "2026" as a consecutive digit string; the smallest such number is 202606, and the next smallest found is 420261\.Finding: The smallest such number is 202606 \(101×\\times2006\), and the next smallest satisfying both conditions is 420261 \(101×\\times4161\), with no numbers in between meeting both criteria\.Answer: \\boxed\{420261\}Root, step 3 — return\#\#\# RETURNTwo independent subagents found the smallest such number is 202606 \(101×\\times2006\), and a third found the second smallest is 420261 \(101×\\times4161\); I trust this systematic modular approach more than the others\.\\boxed\{420261\}

### F\.3L1Rollout on a FrontierScience Problem

Rolloutillustrates reasoning under theL1harness on a FrontierScience chemistry problem\. Despite the difference in domain, the model continues to leverage delegation in a similar manner, suggesting that the contextual reasoning capabilities learned throughHermes\-Learntransfer beyond the mathematical reasoning problems used during training\.

Rollout 3:L1on FrontierScienceProblemchemistry:72, reference answer\-8Calculate the change in oxidation state of sulfur when potassium peroxydisulfate \(K2S2O8\) is reduced to potassium sulfide \(K2S\)\.Root, step 1 — delegateI will have multiple subagents independently determine the oxidation state of sulfur in K2S2O8 and K2S, then compute the change when reduced, to cross\-check the result\.TASK:Calculate the oxidation state of sulfur in potassium peroxydisulfate \(K2S2O8\) and in potassium sulfide \(K2S, where sulfur is in the \-2 oxidation state\), and determine the change in oxidation state of sulfur when K2S2O8 is reduced to K2S\. Show your reasoning step by step and give the numerical change\.TASK:Determine the oxidation number of sulfur in K2S2O8 \(potassium peroxydisulfate\) and in K2S \(potassium sulfide\), then find the change in oxidation state of sulfur per sulfur atom when K2S2O8 is reduced to K2S\. Provide the calculation and the final numerical value for the change\.TASK:Solve this chemistry problem independently: Calculate the change in oxidation state of sulfur when potassium peroxydisulfate \(K2S2O8\) is reduced to potassium sulfide \(K2S\)\. Show your work clearly and state the final numerical change\.3 subagents run in parallel in fresh 8K windows→\\rightarrowreports returned to the root at step 2Task 1\.Calculate the oxidation state of sulfur in potassium peroxydisulfate \(K2S2O8\) and in potassium sulfide \(K2S, where sulfur is in the \-2 oxidation state\), and determine the change in oxidation state of sulfur when K2S2O8 is reduced to K2S\. Show your reasoning step by step and give the numerical change\.\[ subagent 1 reasoning: 6,245 characters omitted \]\#\#\# RESULTScope: Determined oxidation states of sulfur in K2S2O8and K2S, and the change in oxidation state when K2S2O8is reduced to K2S\.Finding: Sulfur oxidation state in K2S2O8is \+6 \(due to peroxide oxygen at \-1\), in K2S is \-2; the change per sulfur atom is from \+6 to \-2, a decrease of 8\.Answer: \\boxed\{\-8\}Task 2\.Determine the oxidation number of sulfur in K2S2O8 \(potassium peroxydisulfate\) and in K2S \(potassium sulfide\), then find the change in oxidation state of sulfur per sulfur atom when K2S2O8 is reduced to K2S\. Provide the calculation and the final numerical value for the change\.\[ subagent 2 reasoning: 2,787 characters omitted \]\#\#\# RESULTScope: Determined oxidation numbers of sulfur in K2S2O8 and K2S, and computed the change per sulfur atom when K2S2O8 is reduced to K2S\.Finding: Sulfur in K2S2O8 is \+6 \(due to peroxide linkage in S2O8^2\-\), and in K2S it is \-2; the change per sulfur atom is \-8\.Answer: \\boxed\{\-8\}Task 3\.Solve this chemistry problem independently: Calculate the change in oxidation state of sulfur when potassium peroxydisulfate \(K2S2O8\) is reduced to potassium sulfide \(K2S\)\. Show your work clearly and state the final numerical change\.\[ subagent 3 reasoning: 2,724 characters omitted \]\#\#\# RESULTScope: Calculated the change in oxidation state of sulfur when K2S2O8is reduced to K2S, by determining the oxidation numbers of sulfur in each compound and comparing them\.Finding: In K2S2O8, sulfur has an oxidation state of \+6; in K2S, sulfur has an oxidation state of \-2\. The change is a decrease of 8, so the magnitude of the change is 8\.Answer: \\boxed\{\-8\}Root, step 2 — provisional answer, then delegate\#\#\# ANSWERAll three subagents agree: sulfur goes from \+6 in K2S2O8 to \-2 in K2S, a change of \-8 per sulfur atom\.\\boxed\{\-8\}I will verify this consensus with one more independent check to confirm the peroxide oxygen assignment and arithmetic are correct\.TASK:Verify the following calculation for a chemistry problem: In potassium peroxydisulfate \(K2S2O8\), determine the oxidation state of sulfur\. The ion is S2O8^2\-\. Potassium is \+1 each\. Oxygen in peroxydisulfate is in a peroxide linkage \(O\-O bond\), so each oxygen has oxidation state \-1\. Using this, calculate the oxidation state of sulfur\. Then, in potassium sulfide \(K2S\), determine the oxidation state of sulfur\. Then compute the change in oxidation state of sulfur when K2S2O8 is reduced to K2S\. Confirm whether the change per sulfur atom is \-8\. Show all steps\.1 subagents run in parallel in fresh 8K windows→\\rightarrowreports returned to the root at step 3Task 1\.Verify the following calculation for a chemistry problem: In potassium peroxydisulfate \(K2S2O8\), determine the oxidation state of sulfur\. The ion is S2O8^2\-\. Potassium is \+1 each\. Oxygen in peroxydisulfate is in a peroxide linkage \(O\-O bond\), so each oxygen has oxidation state \-1\. Using this, calculate the oxidation state of sulfur\. Then, in potassium sulfide \(K2S\), determine the oxidation state of sulfur\. Then compute the change in oxidation state of sulfur when K2S2O8 is reduced to K2S\.\[ 66 more characters omitted \]\[ subagent 1 reasoning: 1,807 characters omitted \]\#\#\# RESULTScope: Verified oxidation states of sulfur in K2S2O8\(peroxydisulfate\) and K2S, and the change in oxidation state per sulfur atom upon reduction\.Finding: Sulfur oxidation state in S2O82\-is \+6, in K2S is \-2, so the change per sulfur atom is \-8, matching the problem’s claim\.Answer: \\boxed\{\-8\}Root, step 3 — return\#\#\# RETURNAll four independent subagents confirm sulfur’s oxidation state changes from \+6 in K2S2O8 to \-2 in K2S, a decrease of 8 per sulfur atom\.\\boxed\{\-8\}

### F\.4L1Rollouts under Larger Inference\-time Compute Budgets

Hermes\-Learntrains withB=3B=3subagents per delegation step,W=1W=1context windows, and delegation depthD=1D=1\. In this section, we provide example rollouts underL1with larger inference\-time compute budgets than seen during training: 4 subagents per delegation step \(Rollout\), 3 context windows connected through digestion \(Rollout\), and delegation depth 2 \(Rollout\)\. See their corresponding experimental results in Sec\.

In Rollout, the model exploits the additional delegation opportunity by assigning the extra slot to a genuinely distinct task, suggesting that it has learned contextual reasoning behaviors that generalize beyond the training configuration\. This rollout also provides an example of a delegation step containing*mixed*search strategies: two tasks specify explicit solution techniques while two do not, combining bothSameandDifferent\.

Rollout 4:L1with 4 subagents \(B=4B=4\)Problemaime\_2024:72, reference answer540Find the largest possible real part of\(75\+117​i\)​z\+96\+144​iz\(75\+117i\)z\+\\frac\{96\+144i\}\{z\}wherezzis a complex number with\|z\|=4\|z\|=4\.Root, step 1 — delegateI will dispatch independent subagents to solve this complex number optimization problem using different approaches so I can cross\-check their answers\.TASK:Solve the following competition math problem completely and show your work: Find the largest possible real part of \(75\+117i\)z \+ \(96\+144i\)/z where z is a complex number with \|z\|=4\.TASK:Solve the following competition math problem completely using the substitution z = 4e^\{iθ\\theta\} and trigonometric manipulation: Find the largest possible real part of \(75\+117i\)z \+ \(96\+144i\)/z where z is a complex number with \|z\|=4\.TASK:Solve the following competition math problem completely by writing z = x\+iy with x^2\+y^2=16 and using calculus/algebraic optimization techniques: Find the largest possible real part of \(75\+117i\)z \+ \(96\+144i\)/z where z is a complex number with \|z\|=4\.TASK:Solve the following competition math problem completely, showing full derivation: Find the largest possible real part of \(75\+117i\)z \+ \(96\+144i\)/z where z is a complex number with \|z\|=4\.4 subagents run in parallel in fresh 8K windows→\\rightarrowreports returned to the root at step 2Task 1\.Solve the following competition math problem completely and show your work: Find the largest possible real part of \(75\+117i\)z \+ \(96\+144i\)/z where z is a complex number with \|z\|=4\.\[ subagent 1 reasoning: 4,278 characters omitted \]\#\#\# RESULTScope: Find the maximum real part of $ \(75\+117i\)z \+ \(96\+144i\)/z $ given $ \|z\|=4 $\.Finding: Writing $ z=4e^\{i\\theta\} $, the expression becomes $4\[\(75\+117i\)e^\{i\\theta\}\+\(6\+9i\)e^\{\-i\\theta\}\]$, whose real part simplifies to $4\(81\\cos\\theta\-108\\sin\\theta\)$, maximized by the amplitude $ \\sqrt\{81^2\+108^2\}=135 $\.Answer: \\boxed\{540\}Task 2\.Solve the following competition math problem completely using the substitution z = 4e^\{iθ\\theta\} and trigonometric manipulation: Find the largest possible real part of \(75\+117i\)z \+ \(96\+144i\)/z where z is a complex number with \|z\|=4\.\[ subagent 2 reasoning: 2,339 characters omitted \]\#\#\# RESULTScope: Find the maximum real part of $ \(75\+117i\)z \+ \(96\+144i\)/z $ given $ \|z\|=4 $, using substitution $ z=4e^\{i\\theta\} $\.Finding: The expression simplifies to $ 324\\cos\\theta \- 432\\sin\\theta $, whose maximum amplitude is $ \\sqrt\{324^2\+432^2\}=540 $\.Answer: \\boxed\{540\}Task 3\.Solve the following competition math problem completely by writing z = x\+iy with x^2\+y^2=16 and using calculus/algebraic optimization techniques: Find the largest possible real part of \(75\+117i\)z \+ \(96\+144i\)/z where z is a complex number with \|z\|=4\.\[ subagent 3 reasoning: 3,413 characters omitted \]\#\#\# RESULTScope: Find the maximum real part of $ \(75\+117i\)z \+ \(96\+144i\)/z $ given $ \|z\|=4 $, using calculus/algebraic optimization\.Finding: Expressed in polar form $ z=4e^\{i\\theta\} $, the real part simplifies to $ 324\\cos\\theta \- 432\\sin\\theta $, whose amplitude is $ \\sqrt\{324^2\+432^2\}=540 $\.Answer: \\boxed\{540\}Task 4\.Solve the following competition math problem completely, showing full derivation: Find the largest possible real part of \(75\+117i\)z \+ \(96\+144i\)/z where z is a complex number with \|z\|=4\.\[ subagent 4 reasoning: 4,127 characters omitted \]\#\#\# RESULTScope: Find the maximum real part of $ \(75\+117i\)z \+ \(96\+144i\)/z $ given $ \|z\|=4 $, using a trigonometric parametrization and linear combination approach\.Finding: The real part simplifies to $ 324\\cos\\theta \- 432\\sin\\theta $, whose maximum is $ \\sqrt\{324^2\+432^2\}=540 $, achieved for some $ z $ on $ \|z\|=4 $\.Answer: \\boxed\{540\}Root, step 2 — provisional answer, then delegate\#\#\# ANSWERAll four subagents independently derived the same simplified expression and amplitude, agreeing the maximum real part is 540\.\\boxed\{540\}I will verify this consensus answer with an independent check on the algebra and the amplitude calculation to ensure no arithmetic slip occurred\.TASK:Verify the following algebra carefully: given z=4e^\{iθ\\theta\}, compute the real part of \(75\+117i\)z \+ \(96\+144i\)/z step by step, and confirm whether it simplifies to 324cosθ\\theta\- 432sinθ\\theta, and confirm the amplitude sqrt\(324^2\+432^2\) equals 540\.TASK:Independently re\-derive from scratch, showing full work, the maximum real part of \(75\+117i\)z \+ \(96\+144i\)/z given \|z\|=4, and state the final numerical answer clearly\.2 subagents run in parallel in fresh 8K windows→\\rightarrowreports returned to the root at step 3Task 1\.Verify the following algebra carefully: given z=4e^\{iθ\\theta\}, compute the real part of \(75\+117i\)z \+ \(96\+144i\)/z step by step, and confirm whether it simplifies to 324cosθ\\theta\- 432sinθ\\theta, and confirm the amplitude sqrt\(324^2\+432^2\) equals 540\.\[ subagent 1 reasoning: 3,637 characters omitted \]\#\#\# RESULTScope: Verified that for \\\( z=4e^\{i\\theta\} \\\), the real part of \\\( \(75\+117i\)z \+ \(96\+144i\)/z \\\) simplifies to \\\( 324\\cos\\theta \- 432\\sin\\theta \\\), and checked that its amplitude \\\( \\sqrt\{324^2\+432^2\} = 540 \\\)\.Finding: The algebra is correct: the real part is \\\( 324\\cos\\theta \- 432\\sin\\theta \\\), and \\\( \\sqrt\{324^2\+432^2\} = \\sqrt\{291600\} = 540 \\\)\.Answer: \\boxed\{324\\cos\\theta \- 432\\sin\\theta\} and amplitude \\\(\\sqrt\{324^2\+432^2\}=540\\\) both confirmed\.Task 2\.Independently re\-derive from scratch, showing full work, the maximum real part of \(75\+117i\)z \+ \(96\+144i\)/z given \|z\|=4, and state the final numerical answer clearly\.\[ subagent 2 reasoning: 4,816 characters omitted \]\#\#\# RESULTScope: Maximum real part of $\(75\+117i\)z\+\(96\+114i\)/z$ given $\|z\|=4$, re\-derived via trigonometric expansion and amplitude maximization\.Finding: The real part simplifies to $324\\cos\\theta \- 432\\sin\\theta$, whose maximum amplitude is $\\sqrt\{324^2\+432^2\}=540$\.Answer: $\\boxed\{540\}$Root, step 3 — return\#\#\# RETURNAll subagents independently confirmed the algebra: real part = 324cosθ\\theta\- 432sinθ\\theta, amplitude =\\sqrt\{\\phantom\{x\}\}\(3242\+4322\) = 540\.\\boxed\{540\}

Rolloutillustrates reasoning with digestion across three context windows\. The first window produces an incorrect answer, and the digest enables subsequent windows to continue reasoning from the accumulated progress rather than starting from scratch\. In the second window, the model revisits and verifies the earlier result and corrects the answer\. The third window then builds on the new digest and reaches a consistent final answer\. This example demonstrates how digestion supports reasoning across multiple context windows, allowing the model to revisit and refine prior progress as additional inference\-time compute becomes available\.

Rollout 5:L1across three context windows \(W=3W=3\)Problemaime\_2026:4, reference answer70— the same problem as RolloutFind the number of integers less than or equal to 100 that are equal toa\+b\+a​ba\+b\+abfor some choice of distinct positive integers a and b\.Window 1 — starts with no digest; final answer 71, incorrect*Root’s delegation at step 1:*I will dispatch independent solvers to tackle this counting problem from scratch, using different approaches to cross\-check the answer\.TASK:Find the number of integers less than or equal to 100 that can be expressed as a\+b\+ab for some choice of distinct positive integers a and b\. Show your work step by step and give the final count\.TASK:Solve this competition math problem: Find the number of integers less than or equal to 100 that are equal to a\+b\+ab for some choice of distinct positive integers a and b\. Provide full reasoning and a final numerical answer\.TASK:Solve the following number theory problem completely and give a final count: Find the number of integers less than or equal to 100 that are equal to a\+b\+ab for some choice of distinct positive integers a and b\. Show all steps clearly\.\[ 2 further root steps and 6 subagents omitted \]*Root’s returned answer:*\#\#\# RETURNThree independent careful analyses converged on 71, with two explicitly counting composites in \[6,101\] excluding prime squares \(9,25,49\), confirming the earlier two agreeing at 70 were missing these boundary/small cases\.\\boxed\{71\}*Digest generated for the next window \(69 generated tokens\):*\#\#\# RETURNThree independent careful analyses now agree on 71, confirming the earlier three subagents \(two at 70, one at 71\) were resolved by careful detail: count composites M in \[6,101\] excluding prime squares \(9,25,49\)\.\\boxed\{71\}Window 2 — starts with the digest of Window 1; final answer 70, correct\#\#\# PRIOR WORK SUMMARY\#\#\# RETURNThree independent careful analyses now agree on 71, confirming the earlier three subagents \(two at 70, one at 71\) were resolved by careful detail: count composites M in \[6,101\] excluding prime squares \(9,25,49\)\.\\boxed\{71\}*Root’s delegation at step 1:*I want independent approaches to confirm the count of integers≤\\leq100 expressible as a\+b\+ab for distinct positive integers a,b, plus a clean derivation to cross\-check the prior answer of 71\.TASK:Solve the following problem completely and show your reasoning: Find the number of integers less than or equal to 100 that are equal to a\+b\+ab for some choice of distinct positive integers a and b\. Provide the final count and explain the key steps of your derivation\.TASK:Solve the following problem completely using a substitution/algebraic approach: Find the number of integers less than or equal to 100 that are equal to a\+b\+ab for some choice of distinct positive integers a and b\. Show your work and give the final count\.TASK:Solve the following problem completely and independently, showing all steps: Find the number of integers less than or equal to 100 that are equal to a\+b\+ab for some choice of distinct positive integers a and b\. Provide the final numerical answer\.\[ 2 further root steps and 5 subagents omitted \]*Root’s returned answer:*\#\#\# RETURNFour independent subagents \(three solving directly, one verifying via two methods\) consistently derived 70 using the substitution n\+1=\(a\+1\)\(b\+1\), confirming the count of composite m in \[4,101\] excluding prime squares \(74\-4=70\)\.\\boxed\{70\}*Digest generated for the next window \(53 generated tokens\):*\#\#\# RETURNFour independent subagents confirm via substitution and two counting methods that the count equals composite numbers in \[4,101\] excluding prime squares \(4,9,25,49\), giving 70\.\\boxed\{70\}Window 3 — starts with the digest of Window 2; final answer 70, correct\#\#\# PRIOR WORK SUMMARY\#\#\# RETURNFour independent subagents confirm via substitution and two counting methods that the count equals composite numbers in \[4,101\] excluding prime squares \(4,9,25,49\), giving 70\.\\boxed\{70\}*Root’s delegation at step 1:*I will dispatch independent subagents to solve the full problem and cross\-check the structure of the solution to confirm the count of 70\.TASK:Solve this competition math problem completely and show your reasoning: Find the number of integers less than or equal to 100 that are equal to a\+b\+ab for some choice of distinct positive integers a and b\.TASK:Solve this competition math problem completely and show your reasoning: Find the number of integers less than or equal to 100 that are equal to a\+b\+ab for some choice of distinct positive integers a and b\. Provide the final count as a single number and explain the key steps of your derivation\.TASK:Solve this competition math problem completely and show your reasoning, then give the final count as a single number: Find the number of integers less than or equal to 100 that are equal to a\+b\+ab for some choice of distinct positive integers a and b\.\[ 2 further root steps and 6 subagents omitted \]*Root’s returned answer:*\#\#\# RETURNTwo independent subagents \(out of three\) carefully derived 70 via the composite\-counting method, matching the prior\-work summary’s claim; the dissenters \(71\) used a flawed "75 composites" count rather than the standard 74 composites in \[4,101\]\.\\boxed\{70\}

In Rollout, subagents recursively delegate tasks to further subagents by invoking the same harness interface, illustrating extrapolation beyond the delegation depth seen during training\. Despite only observing depth\-1 delegation during training, the model continues to apply delegation when additional depth is enabled\.

Rollout 6:L1at delegation depth 2 \(D=2D=2\)Problemaime\_2024:79, reference answer321Letω≠1\\omega\\neq 1be a 13th root of unity\. Find the remainder when∏k=012\(2−2​ωk\+ω2​k\)\\prod\_\{k=0\}^\{12\}\(2\-2\\omega^\{k\}\+\\omega^\{2k\}\)is divided by 1000\.Depth 1 — the root delegatesI will dispatch multiple subagents to independently solve this competition math problem so I can cross\-check their approaches and answers\.TASK:Letω\\omega≠\\neq1 be a 13th root of unity\. Find the remainder when∏\\prod\_\{k=0\}^\{12\}\(2\-2ω\\omega^k\+ω\\omega^\{2k\}\) is divided by 1000\. Show full steps and give a numeric remainder\.TASK:Letω\\omega≠\\neq1 be a primitive 13th root of unity\. Find the remainder when∏\\prod\_\{k=0\}^\{12\}\(2\-2ω\\omega^k\+ω\\omega^\{2k\}\) is divided by 1000\. Provide a full derivation and the final numeric answer\.TASK:Solve this competition math problem completely and show your work: Letω\\omega≠\\neq1 be a 13th root of unity\. Find the remainder when∏\\prod\_\{k=0\}^\{12\}\(2\-2ω\\omega^k\+ω\\omega^\{2k\}\) is divided by 1000\.Depth 2 — task 1’s subagent receives that task as its own\#\#\# PROBLEM, under the same L1 harness, and it delegates againI will dispatch independent subagents to solve this problem using different approaches so I can cross\-check their answers\.TASK:Letω\\omega≠\\neq1 be a primitive 13th root of unity\. Find the remainder when∏\\prod\_\{k=0\}^\{12\}\(2\-2ω\\omega^k\+ω\\omega^\{2k\}\) is divided by 1000\. Show full steps and give a numeric remainder\.TASK:Letω\\omega≠\\neq1 be a 13th root of unity \(primitive, since 13 is prime andω\\omega≠\\neq1 implies it’s primitive\)\. Find the remainder when∏\\prod\_\{k=0\}^\{12\}\(2\-2ω\\omega^k\+ω\\omega^\{2k\}\) is divided by 1000\. Show full steps and give a numeric remainder\.TASK:Solve this competition math problem completely and show all steps: Letω\\omega≠\\neq1 be a 13th root of unity\. Find the remainder when∏\\prod\_\{k=0\}^\{12\}\(2\-2ω\\omega^k\+ω\\omega^\{2k\}\) is divided by 1000\.Depth 2 — subagents reason inlineTask 1\.Letω\\omega≠\\neq1 be a primitive 13th root of unity\. Find the remainder when∏\\prod\_\{k=0\}^\{12\}\(2\-2ω\\omega^k\+ω\\omega^\{2k\}\) is divided by 1000\. Show full steps and give a numeric remainder\.\#\#\# RESULTScope: The problem asks for the remainder modulo 1000 of the product∏\\prod\_\{k=0\}^\{12\}\(2\-2ω\\omega^k\+ω\\omega^\{2k\}\) whereω\\omegais a primitive 13th root of unity,ω\\omega≠\\neq1\.Finding: The product equals 8321, derived via resultant or symmetric function methods using roots of x^2\-2x\+2, confirmed by two independent approaches\.Answer: \\boxed\{321\}Task 2\.Letω\\omega≠\\neq1 be a 13th root of unity \(primitive, since 13 is prime andω\\omega≠\\neq1 implies it’s primitive\)\. Find the remainder when∏\\prod\_\{k=0\}^\{12\}\(2\-2ω\\omega^k\+ω\\omega^\{2k\}\) is divided by 1000\. Show full steps and give a numeric remainder\.\#\#\# RESULTScope: Compute∏\\prod\_\{k=0\}^\{12\}\(2\-2ω\\omega^k\+ω\\omega^\{2k\}\) for primitive 13th root of unityω\\omega, then find remainder mod 1000\.Finding: The product equals \(∀\\forall^13\-1\)\(β\\beta^13\-1\) for rootsα\\alpha=1\+i,β\\beta=1\-i of x^2\-2x\+2=0, computed via polar forms to be 8321\.Answer: \\boxed\{321\}\[ 3 further depth\-2 tasks in this branch, and 3 sibling depth\-1 branches, omitted \]Depth 1 — the branch reports one compact result upward\#\#\# RESULTScope: Verified that∏\\prod\_\{k=0\}^\{12\}\(2\-2ω\\omega^k\+ω\\omega^\{2k\}\) = \(α\\alpha^13\-1\)\(β\\beta^13\-1\) whereα\\alpha=1\+i,β\\beta=1\-i, computed via polar form to be 8321, and 8321 mod 1000 = 321\.Finding: All independent approaches agree the product equals 8321, confirmed by two distinct methods \(resultant/symmetric functions and polar form\)\.Answer: \\boxed\{321\}Depth 0 — return\#\#\# RETURNFour independent subagents confirm the product equals 8321 via multiple methods, giving remainder 8321 mod 1000 = 321\.\\boxed\{321\}