SynAct：面向大语言模型智能体的协同主动反馈学习

# SynAct: Synergistic Active Feedback Learning for LLM Agents
Source: [https://arxiv.org/html/2610.12651](https://arxiv.org/html/2610.12651)
###### Abstract

LLM agents executing multi\-step workflows fail in two fundamentally different ways: recoverable slips, which further attempts can repair, and knowledge gaps, which retrying cannot resolve\. Expert feedback can address the latter, but expert attention is scarce\. The key challenge is therefore not whether to ask for feedback, but*where feedback can change the outcome*\. Existing approaches either avoid expert feedback, allowing agents to reinforce their own errors, or allocate feedback at the task level without identifying consequential decisions\. We presentSynAct, an active learning framework that treats expert attention as the binding constraint and allocates feedback at the level of individual decisions\. SynAct ranks decisions by the mutual information between the selected action and the terminal outcome across repeated rollouts, queries an expert at the highest\-value decisions, and distills the resulting corrections into transferable rules that the agent can reuse without expert access at deployment\. We theoretically characterize what this signal identifies under finite repeated sampling: withkkrollouts and at leastmmobservations per action, outcome\-constant decisions have zero mutual information, while outcome\-separating decisions have mutual information at leastH2​\(m/k\)H\_\{2\}\(m/k\)bits, yielding exact separation for any thresholdt∈\(0,H2​\(m/k\)\)t\\in\(0,H\_\{2\}\(m/k\)\)\. This also explains why success\-based acquisition can be blind when all rollouts fail: success is constant even when failed runs reach different outcomes\. Across four agent benchmarks and four models spanning three families,SynActimproves held\-out accuracy in all 16 evaluated settings, by\+11\.7\+11\.7pp on average\. The same signal transfers to inference\-time routing, whereSynActoutperforms budget\-matched cascade, uncertainty, and expert\-judge routers in 15 of 16 settings, while a single targeted correction outperforms seven additional unaimed retries\.

## 1Introduction

An LLM agent can fail a multi\-step task for two fundamentally different reasons: it can*slip*despite knowing how to succeed, or it can hit a*knowledge gap*that no amount of retrying will close\. Additional test\-time compute, through self\-reflection\([Shinn et al\., 2023](https://arxiv.org/html/2610.12651#bib.bib9);[Madaan et al\., 2023](https://arxiv.org/html/2610.12651#bib.bib10)\)or trajectory search\([Yao et al\., 2023a](https://arxiv.org/html/2610.12651#bib.bib13);[Zhou et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib14)\), can often recover the first but not the second: an agent checking its own work has little basis for distinguishing a correct interpretation from a confidently wrong one\([Huang et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib11)\)\. An external expert can resolve such gaps, but expert attention is scarce\. Agents that sidestep the expense by distilling context from their own traces\([Zhang et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib1);[Ouyang et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib2)\)return judgment to the same self\-assessment that fails on knowledge gaps\. In real agentic systems, the missing knowledge often resides in domain\-specific procedures and human expertise\([Liu et al\., 2026a](https://arxiv.org/html/2610.12651#bib.bib47)\), making expert attention the one resource in the loop that does not scale\. This creates a fundamental allocation problem:*when expert feedback is limited, where should it be spent?*A failing trajectory is not an atomic example: a 17\-step episode may contain one or two decisions that determine its outcome and many that do not\. Spending expert attention on the entire trajectory therefore wastes effort on irrelevant decisions\. The right unit of feedback is not the trajectory, but the*decision point*\.

We cast expert feedback as active learning over decision points\. Rather than asking which trajectories are wrong, SynAct asks a more targeted question:*which action differences explain where the agent ends up?*We sample repeated rollouts of a task, group actions taken from the same environment state, and ask whether different actions lead to different terminal outcomes \(Figure[1](https://arxiv.org/html/2610.12651#S1.F1)\)\. We formalize this with the mutual informationI⁡\(A,O\)I\(A;O\)between the actionAAtaken at a decision point and the run’s terminal outcome classOO\. Both are observable from the agent’s own rollouts\. Acquisition therefore requires neither a gold answer nor a success label\. This gives the signal two properties that are particularly important for feedback allocation\. First, it measures*consequential*disagreement rather than behavioral uncertainty: action entropy is high whenever runs disagree, whereasI⁡\(A,O\)I\(A;O\)is high when those action differences explain different outcomes\. Second, it remains informative when every rollout fails\. Different actions can still lead to different wrong end states, whereas a success\-based signal collapses to zero when success is constant\. SynAct can therefore surface decisions that matter precisely on tasks that are beyond the agent’s current capability\.

![Refer to caption](https://arxiv.org/html/2610.12651v1/synact_method_cropped.png)Figure 1:SynAct allocates a fixed expert budget over decision points rather than episodes\.Acquisition scores each state\-keyed decision node byI⁡\(A,O\)I\(A;O\), computed from the run itself with no gold answer or success check\. Authoring sends the top\-scoring nodes to the expert in one call, which abstracts them into a convention plus an antipattern\. Application injects the convention only into failed attempts exhibiting that antipattern, iteration re\-rolls to surface residual failures, and the same score routes consultation per task at inference time\.This raises a second question: once a consequential decision is found, how should its correction be turned into reusable knowledge? SynAct sends the selected decision points to the expert together and asks for a compact rule: a*convention*describing what to do and an*antipattern*describing the failure it prevents\. At deployment, an error gate applies the rule only when a failed trajectory exhibits that antipattern\. This separates three decisions that are often conflated: acquisition determines*where*expert attention is spent, authoring determines*what*is retained, and the gate determines*when*it is applied\. The resulting rules outlive the trajectories that produced them, while the agent remains frozen and the expert stays out of the execution loop\. We evaluate SynAct on four agent benchmarks spanning text\-to\-SQL, compositional crafting, shell scripting, and tool\-use dialogue, using four agent models across three model families\. Our contributions are:

- •We introduce*budgeted active feedback acquisition over decision points*and a label\-free signal,I⁡\(A,O\)I\(A;O\), that ranks decisions by how strongly their action choices explain downstream outcomes\. It remains informative when all rollouts fail and outperforms random, entropy, and success\-based selection at matched budgets\.
- •We characterize the*finite\-sample identifiability*of this signal\. Withkkrepeated rollouts and a minimum ofmmobservations per action, outcome\-constant and outcome\-separating decisions are exactly distinguished with any thresholdt<H2​\(m/k\)t<H\_\{2\}\(m/k\), while between these extremes the score measures how much behavioral variation the outcome resolves\. The same analysis proves success\-based acquisition blind when all rollouts fail\.
- •We make expert feedback*non\-regressive*through selective application: an error gate withholds a learned rule from already\-solved attempts and activates it only when its antipattern is present, eliminating win\-to\-lose flips even when always\-on context costs a weak agent up to17\.617\.6pp\.
- •Across44benchmarks×\\times44agent models with an equal expert budget, SynAct improves accuracy in all1616settings and leads four experience\-distillation baselines in1515\. The same signal transfers to inference\-time routing, where it beats budget\-matched cascade, uncertainty, and expert\-judge routers in1515of1616settings\.

## 2Related Work

#### Self\-improving agents\.

A large body of work builds reusable context from an agent’s own traces, including playbooks\([Zhang et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib1)\), reasoning and causal memories\([Ouyang et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib2);[Majumder et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib6);[Zhao et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib7)\), induced workflows and tools\([Wang et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib3);[Kujanpää et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib46)\), and self\-written guidelines and manuals\([Fu et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib4);[Chen et al\., 2024b](https://arxiv.org/html/2610.12651#bib.bib5)\)\. These form our primary baselines, and they differ from SynAct on two axes\. Their supervision reduces to the agent’s own judgment of its traces, which is least reliable where the agent is confidently wrong\([Huang et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib11)\), and several bootstrap only from successes\([Wang et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib3);[Zhao et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib37)\), leaving no signal on tasks the agent fails throughout\. They also inject context unconditionally, a regime known to introduce regressions\([Shao et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib38)\)\. Reflective prompt search\([Agrawal et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib8)\)is complementary, optimizing prompt space rather than allocating supervision\. SynAct instead spends a small metered expert budget and gates what it learns\.

#### Test\-time correction and search\.

Self\-generated feedback\([Shinn et al\., 2023](https://arxiv.org/html/2610.12651#bib.bib9);[Madaan et al\., 2023](https://arxiv.org/html/2610.12651#bib.bib10)\), search over trajectories\([Yao et al\., 2023a](https://arxiv.org/html/2610.12651#bib.bib13);[Zhou et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib14)\), and compute\-optimal allocation\([Snell et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib25)\)scale the agent’s own computation, which can recover slips yet cannot supply knowledge the agent lacks\([Huang et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib11)\)\. Relevant to our mechanism,[Chen et al\. \(2026\)](https://arxiv.org/html/2610.12651#bib.bib45)distinguish syntactic misunderstanding of an environment’s conventions from semantic misunderstanding of the task, and the former is what an authored convention repairs\. Our premise is that one class of failures resists additional agent compute, and the contribution lies in where to spend a different resource\.

#### Budgeted acquisition and expert intervention\.

We inherit from pool\-based active learning\([Settles, 2009](https://arxiv.org/html/2610.12651#bib.bib17)\), where query\-by\-committee measures disagreement across hypotheses\([Seung et al\., 1992](https://arxiv.org/html/2610.12651#bib.bib18)\)and information\-theoretic criteria select maximally informative queries\([Houlsby et al\., 2011](https://arxiv.org/html/2610.12651#bib.bib19)\)\. Ourkkrollouts act as such a committee, but the unit of acquisition is a decision inside an episode rather than an instance, and the learner updates context rather than weights\. On the intervention side, DAgger queries an expert at visited states\([Ross et al\., 2011](https://arxiv.org/html/2610.12651#bib.bib20)\), preference learning extracts training signal from scarce human judgments\([Christiano et al\., 2017](https://arxiv.org/html/2610.12651#bib.bib21);[Liu et al\., 2026b](https://arxiv.org/html/2610.12651#bib.bib49)\), and targeted feedback recovers full\-annotation quality from a fraction of labels\([Xu et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib39)\), and all three economize how much supervision is needed while leaving open where within an episode it is most informative\. LEAP is the closest prior work, teaching an agent from a privileged AI expert\([Choudhury and Sodhi, 2025](https://arxiv.org/html/2610.12651#bib.bib43)\), though it consults the teacher per episode and fine\-tunes, whereas we make one expert call per iteration and leave the model frozen\.[Deng et al\. \(2026\)](https://arxiv.org/html/2610.12651#bib.bib44)come nearest to our score, timing a clarification request by information gain, though they query the user about intent rather than an expert about a decision\.

#### Credit assignment and routing\.

Localizing the step responsible for a failure has been studied through automated failure attribution, which requires a failure label our score does not\([Zhang et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib40)\), and critic\-free RL methods that group repeated environment states across rollouts to sharpen step\-level advantages\([Feng et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib41);[Cheng et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib42)\)\. The grouping instinct is shared, and the consumers differ: they convert the grouped statistics into gradient updates for an unfrozen policy, whereas we convert them into a ranking that routes a metered expert to a frozen one\. Process supervision localizes credit by training a step\-level verifier\([Lightman et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib26);[Wang et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib27)\), whereas we spend a few expert calls to author context, and the two could compose, with a verifier refining the outcome projection or breaking ties among equally ranked decisions\. Uncertainty estimates over*answers*\([Wang et al\., 2023](https://arxiv.org/html/2610.12651#bib.bib15);[Kadavath et al\., 2022](https://arxiv.org/html/2610.12651#bib.bib16)\)and cost\-aware model selection\([Chen et al\., 2024a](https://arxiv.org/html/2610.12651#bib.bib22);[Ong et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib23);[Yue et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib24)\)both operate on whole queries\. We instead allocate expert attention within a query, at the decision whose alternatives separate the outcomes\.

## 3SynAct: Synergistic Active Feedback Learning

The central challenge is deciding*where expert attention is worth spending*\. Given a fixed*acquisition budget*BB, weacquiretheBBdecision points that appear most consequential, have the expertauthora reusable rule distilling what those decisions reveal, andapplythat rule only when the current failure matches the learned error pattern\. We make only one expert call per iteration\. The objective is therefore to maximize the value obtained from each expert query rather than accuracy alone\. We optionally repeat this process as the learned rule changes the agent’s remaining failures \(§[3\.5](https://arxiv.org/html/2610.12651#S3.SS5)\)\.

#### Problem setup\.

An*agent*is a small or medium open\-weights modelπ\\pithat interacts with an environment through a ReAct\-style loop\([Yao et al\., 2023b](https://arxiv.org/html/2610.12651#bib.bib12)\)\. An*expert*provides knowledge the agent does not have: a subject\-matter expert in deployment, instantiated in our experiments by a strong model \(Claude Opus\([Anthropic, 2026](https://arxiv.org/html/2610.12651#bib.bib36)\)\)\. The expert is available during training only, leaving the agent to operate alone at test time\. A rollout is a sequence of state–action pairsτ=\(s0,a0,…,sT\)\\tau=\(s\_\{0\},a\_\{0\},\\dots,s\_\{T\}\), withsucc⁡\(τ\)∈\{0,1\}\\mathrm\{succ\}\(\\tau\)\\in\\\{0,1\\\}denoting its outcome under the benchmark checker\.

### 3\.1Acquisition: Finding Consequential Decisions

The ideal acquisition score would tell us how much an expert correction at each decision would improve the agent\. That quantity is counterfactual: estimating it requires intervening on the decision and observing what would have happened, often before we know the task’s answer\. We therefore extract a proxy for consequence from the agent’s own behavior\. We run the same task multiple times, find decisions where different runs make different choices*in the same state*, and ask whether those choices lead to different terminal outcomes\. If all alternatives converge to the same outcome, the decision provides little evidence that changing it would matter\. If different alternatives lead to different outcomes, the decision becomes a natural target for intervention\. We capture this contrast with the mutual informationI⁡\(A,O\)I\(A;O\)between the action at a decision and the rollout’s terminal outcome class\. Ranking by this signal concentrates useful decisions near the top, with the leading decile capturing most of the benefit \(§[5\.3](https://arxiv.org/html/2610.12651#S5.SS3)\)\.

#### State\-keyed grouping\.

For each training task, we independently samplekkrollouts fromπ\\piat temperature\>0\>0\(k=8k\{=\}8by default; Appendix[B\.3](https://arxiv.org/html/2610.12651#A2.SS3)\) and group steps across rollouts by the environment state\. Letϕ⁡\(s\)\\phi\(s\)denote a state key: a normalized hash of the observation atss, canonicalized per benchmark by lower\-casing, collapsing whitespace, and removing volatile fields such as step counters\. A*decision node*vvis a maximal set of \(run, step\) pairs sharing a state key, withR⁡\(v\)R\(v\)the runs reachingvv, partitioned into*arms*by the next action\. We score only arms supported by at leastm=2m\{=\}2runs, since an arm observed once cannot provide reliable within\-state evidence\. We key on state rather than trajectory position because two runs may reach the same state through different action sequences while facing exactly the same decision, a correspondence position\-based grouping discards\. We additionally verify the key against an order\-invariant alternative: the acquired set changes on5%5\\%of tasks, and an expert judge deems the additional merges largely illegitimate \(Appendix[C](https://arxiv.org/html/2610.12651#A3)\)\.

###### Assumption 1\(Decision identifiability\)\.

The state keyϕ\\phiis*decision\-sufficient*: ifϕ⁡\(s\)=ϕ⁡\(s′\)\\phi\(s\)=\\phi\(s^\{\\prime\}\)then the two states admit the same action set and induce the same conditional outcome distribution for each action\. The outcome projectionω\\omegadepends only on the terminal state\. Conditioned on that state,OOis therefore independent ofAvA\_\{v\}\.

Assumption[1](https://arxiv.org/html/2610.12651#Thmassumption1)makes the comparison meaningful\. The first clause ensures that states sharing a key represent the same decision, while the second ensures that the outcome projection measures what happens*after*the decision rather than encoding the decision itself\. Without the latter, anω\\omegathat reads the branching action would make every decision consequential by construction\. Deterministic environments sharpen the conditional outcome distributions but guarantee neither clause: both depend on the sufficiency ofϕ\\phiandω\\omega\(Appendix[A\.7](https://arxiv.org/html/2610.12651#A1.SS7)\)\.

#### Outcome projection\.

The outcome variable should capture a decision’s consequence without requiring a gold answer\. LetAvA\_\{v\}be the action taken atvvandO=ω⁡\(τ\)O=\\omega\(\\tau\)the rollout’s*terminal outcome class*, computed from its own trajectory\. We assign the same class to runs that end in the same state\. In practice, we deriveOOfrom the normalized finalSELECTon SQL, sortedcraft/gettargets on TextCraft, the final non\-navigation command on Bash, and the write\-tool multiset onτ\\tau\-bench\. Coarsening or refining these projections shifts the ranking but leaves the authored rule unchanged or improves it \(Appendix[D](https://arxiv.org/html/2610.12651#A4)\)\.

#### Acquisition score\.

We now measure how strongly the terminal outcome distinguishes the alternatives at a decision\. Let𝒜m​\(v\)=\{a:\|Ra​\(v\)\|≥m\}\\mathcal\{A\}\_\{m\}\(v\)=\\\{a:\|R\_\{a\}\(v\)\|\\geq m\\\}denote the supported arms andRm​\(v\)=⋃a∈𝒜m​\(v\)Ra​\(v\)R\_\{m\}\(v\)=\\bigcup\_\{a\\in\\mathcal\{A\}\_\{m\}\(v\)\}R\_\{a\}\(v\)the runs they contain\. Estimating both entropies by plug\-in counts overRm​\(v\)R\_\{m\}\(v\)gives

ℐ⁡\(v\)=H^​\(O\)−∑a∈𝒜m​\(v\)\|Ra​\(v\)\|\|Rm​\(v\)\|​H^​\(O∣A=a\),\\mathcal\{I\}\(v\)\\;=\\;\\hat\{H\}\(O\)\\;\-\\;\\sum\_\{a\\in\\mathcal\{A\}\_\{m\}\(v\)\}\\frac\{\|R\_\{a\}\(v\)\|\}\{\|R\_\{m\}\(v\)\|\}\\,\\hat\{H\}\(O\\mid A\{=\}a\),\(1\)the standard decompositionI⁡\(A,O\)=H⁡\(O\)−H⁡\(O∣A\)I\(A;O\)=H\(O\)\-H\(O\\mid A\)evaluated on the supported population\. The restriction to supported arms keeps the arm weights normalized and ensuresℐ⁡\(v\)≥0\\mathcal\{I\}\(v\)\\geq 0\. We retain the highest\-scoring node from each task, rank tasks by that score, and send the topBBto the expert\. Acquisition therefore requires neither a gold label nor a success check: the agent’s own repeated behavior determines where expert attention is most likely to matter\.

### 3\.2Identifiability Under Repeated Sampling

The acquisition score comes from onlykkrollouts\. The natural question is what it can reliably identify at finite sample sizes\. We begin with two extremes the score should distinguish: decisions whose alternatives converge to one outcome and decisions whose alternatives lead to disjoint outcomes\. Once a node is scorable, this distinction is exact for every realization\.

###### Definition 1\(Outcome\-constant and outcome\-separating nodes\)\.

A scorable nodevvis*outcome\-constant*ifω\\omegais constant onRm​\(v\)R\_\{m\}\(v\), so its runs reach one outcome class whichever arm they take\. It is*outcome\-separating*if the arms of𝒜m​\(v\)\\mathcal\{A\}\_\{m\}\(v\)have disjoint outcome supports, that isω⁡\(Ra​\(v\)\)∩ω⁡\(Ra′​\(v\)\)=∅\\omega\(R\_\{a\}\(v\)\)\\cap\\omega\(R\_\{a^\{\\prime\}\}\(v\)\)=\\emptysetfora≠a′a\\neq a^\{\\prime\}, so the outcome determines which arm was taken\.

###### Proposition 1\(Exact separation at finitekk\)\.

Fixm≥2m\\geq 2and letvvbe scorable, meaning\|𝒜m​\(v\)\|≥2\|\\mathcal\{A\}\_\{m\}\(v\)\|\\geq 2\. Ifvvis outcome\-constant thenℐ⁡\(v\)=0\\mathcal\{I\}\(v\)=0for each realization of the rollouts, and ifvvis outcome\-separating thenℐ⁡\(v\)=H^​\(Av\)≥H2​\(m/k\)\\mathcal\{I\}\(v\)=\\hat\{H\}\(A\_\{v\}\)\\geq H\_\{2\}\(m/k\), withH2H\_\{2\}the binary entropy in bits\. Hence any thresholdt∈\(0,H2​\(m/k\)\)t\\in\\bigl\(0,H\_\{2\}\(m/k\)\\bigr\)classifies both types with zero error for every realization, and at the defaultk=8k\{=\}8,m=2m\{=\}2the admissible range ist∈\(0,0\.811\)t\\in\(0,0\.811\)\.

We prove both clauses in Appendix[A\.2](https://arxiv.org/html/2610.12651#A1.SS2), and we can read the result directly off Equation[1](https://arxiv.org/html/2610.12651#S3.E1): an outcome\-constant node hasH^​\(O\)=0\\hat\{H\}\(O\)=0, an outcome\-separating node hasH^​\(Av∣O\)=0\\hat\{H\}\(A\_\{v\}\\mid O\)=0so its mutual information reachesH^​\(Av\)\\hat\{H\}\(A\_\{v\}\), and the support floor supplies the finite\-sample gap between them, since any arm appearing in at leastmmof thekkrollouts forcesH^​\(Av\)≥H2​\(m/k\)\\hat\{H\}\(A\_\{v\}\)\\geq H\_\{2\}\(m/k\)\. Sampling noise therefore cannot collapse the two extremes\. What sampling does control is*scorability*: sinceNv,a∼Binomial⁡\(k,pv,a\)N\_\{v,a\}\\sim\\mathrm\{Binomial\}\(k,p\_\{v,a\}\), we retain an arm taken half the time with probability0\.9650\.965atk=8k\{=\}8, but one taken a tenth of the time only with probability0\.1870\.187\(Proposition[3](https://arxiv.org/html/2610.12651#Thmproposition3)\)\. The support floor thus trades coverage for reproducibility, keeping the alternatives a written rule can address\.

#### Partial identifiability\.

Most decisions lie between the two extremes\. In this regime,ℐ⁡\(v\)=H^​\(Av\)−H^​\(Av∣O\)\\mathcal\{I\}\(v\)=\\hat\{H\}\(A\_\{v\}\)\-\\hat\{H\}\(A\_\{v\}\\mid O\)\(Appendix[A\.1](https://arxiv.org/html/2610.12651#A1.SS1)\): the score measures the fraction of behavioral variability atvvthat the outcome resolves, identifying a decision to the extent that its alternatives produce distinguishable outcomes \(Proposition[2](https://arxiv.org/html/2610.12651#Thmproposition2), Appendix[A\.3](https://arxiv.org/html/2610.12651#A1.SS3)\)\. The plug\-in estimator also carries a positive null bias,𝔼⁡\[ℐ⁡\(v\)\]\>0\\mathbb\{E\}\[\\mathcal\{I\}\(v\)\]\>0even for outcomes independent of the action\. We useℐ⁡\(v\)\\mathcal\{I\}\(v\)to*rank*decisions rather than as a hypothesis test\. We compute the estimator’s exact null distribution, its growth with the number of outcome classes, and its ranking stability at the default budget in Appendix[A\.6](https://arxiv.org/html/2610.12651#A1.SS6)\.

#### Why outcomes rather than actions or success?

Action entropyH^​\(Av\)\\hat\{H\}\(A\_\{v\}\)detects behavioral variation but cannot tell whether it matters: an outcome\-constant and an outcome\-separating node can share the same action entropy, whileℐ⁡\(v\)\\mathcal\{I\}\(v\)is00for the former andH^​\(Av\)\\hat\{H\}\(A\_\{v\}\)for the latter\. Success creates the opposite problem: replacingOOwithsucc⁡\(τ\)\\mathrm\{succ\}\(\\tau\)makes an all\-fail task outcome\-constant, hiding exactly the tasks for which the agent most needs external knowledge \(Proposition[4](https://arxiv.org/html/2610.12651#Thmproposition4), Appendix[A\.5](https://arxiv.org/html/2610.12651#A1.SS5)\)\. Our outcome projection instead preserves distinctions among*wrong*end states, keeping the signal informative even when no rollout succeeds\. Empirically,I⁡\(A,O\)I\(A;O\)routes1\.9×1\.9\\timesmore decisions than a success\-based signal \(274274against146146on one SQL model\), with2020of4949acquired Qwen TextCraft tasks coming from all\-fail cases \(Appendix[G](https://arxiv.org/html/2610.12651#A7)\)\.

### 3\.3Authoring: From Episodic Evidence to Semantic Rule

Acquisition identifies*where*expert attention is valuable, and authoring determines*what*reusable knowledge that evidence supports\. For each acquired node, we construct an evidence block containing the task goal and two arms with different terminal outcomes, each with its diverging action, the trajectory that followed, and acorrect/wrongtag from its success flag\. When no rollout succeeds, we instead mark the blockALL\-FAILand provide two distinct wrong end states\. Acquisition thus remains label\-free, while authoring may use success annotations where they exist\.

A single decision can expose a mistake without revealing whether that mistake recurs across tasks\. We therefore aggregate all acquired blocks and give the expert the full digest inone call\. We ask the expert to*diagnose*recurring failure modes,*count*the blocks each mode explains, mark those a fixed convention could repair, and*write*one convention of 2–4 imperative lines covering the dominant repairable modes, with individual bullets free to address distinct modes, plus one*antipattern*summarizing the family of mistakes the convention prevents\. This aggregation turns isolated episodes into a transferable rule, and the single call decouples expert cost from the acquisition budget: more evidence costs no additional expert invocations\.

### 3\.4Application: Selective Injection Behind an Error Gate

A useful convention can still hurt when applied to the wrong failure\. SynAct lets the current attempt determine whether the learned rule should be used on the next one\. Each task receives at most two attempts\. We first run the agent with no convention at temperature00and stop if it succeeds\. Otherwise, an LLM gate inspects the failing trajectory for the learned antipattern\. Only when the gate fires do we prepend the convention and retry once\. The gate sees neither the success bit nor the reference solution\. We use the agent model for gating rather than the expert, so this step adds no expert cost \(Appendix[B\.4](https://arxiv.org/html/2610.12651#A2.SS4)\)\.

This design gives a pointwise non\-regression guarantee: since a base success never sees the convention, the rule cannot convert it into a failure, and any improvement must come from repairing failures\. We report accuracy as base successes plus gate\-repaired failures and tracknfiredn\_\{\\text\{fired\}\}, the tasks reaching the gated retry\. In Appendix[J](https://arxiv.org/html/2610.12651#A10)we evaluate the gate against unconditional injection, validate its firing decisions against an expert judge, redeploy the baselines’ knowledge behind the same gate, and substitute a weaker authoring expert\.

### 3\.5Iteration and Inference\-Time Reuse

#### Iterative acquisition\.

Learning a rule changes the failure distribution\. Once the first convention repairs some errors, previously hidden failure modes become more prominent, and the original rollouts cannot reveal this shift because they were generated before the rule existed\. SynAct therefore re\-collects after each update\. At iteration 1, we acquire from base rollouts\. At iterationi\>1i\>1, we generatekkfresh rollouts with the current convention active, score the residual failures, and ask the expert to*revise*the convention: retaining bullets supported by the new evidence, adding bullets for newly surfaced modes, and removing bullets the evidence contradicts\. We rewrite the antipattern alongside the convention so that the gate always evaluates the current rule, and each iteration thus spends its single expert call on what remains unrepaired\. We save and evaluate every iteration’s rule independently, turning iteration into a budget–performance curve rather than a fixed schedule \(§[5\.3](https://arxiv.org/html/2610.12651#S5.SS3)\)\.

#### Inference\-time variant\.

The same signal can allocate expert attention directly when the expert remains available at deployment, as in an LLM cascade\([Chen et al\., 2024a](https://arxiv.org/html/2610.12651#bib.bib22);[Yue et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib24)\)\. Becauseℐ⁡\(v\)\\mathcal\{I\}\(v\)is computed fromkkrollouts of the task at hand, the separation result applies under the same conditions with no training set\. Instead of amortizing the feedback into a rule, SynAct spends the signal immediately\. We first drawkkrollouts, and if the task is*behaviorally consistent*, meaning its modal outcome class covers at least a fractionθ\\thetaof runs, we return the modal answer at no further cost\. Otherwise, under a budget permitting expert consultation on a fractionBBof tasks, we send the topBBinconsistent tasks byℐ⁡\(v⋆\)\\mathcal\{I\}\(v^\{\\star\}\)to the expert, which supplies a correction atv⋆v^\{\\star\}for the agent to apply on retry, and the remainder receive additional self\-reflection\([Shinn et al\., 2023](https://arxiv.org/html/2610.12651#bib.bib9)\)at a matched attempt budget, equivalently a per\-benchmark threshold onℐ⁡\(v⋆\)\\mathcal\{I\}\(v^\{\\star\}\)\.

Consistency thus determines*whether*to intervene, whileℐ⁡\(v⋆\)\\mathcal\{I\}\(v^\{\\star\}\)determines*where*expert attention is worth spending: SynAct reserves the expert for cases where behavioral disagreement points to a consequential decision, and resolves the remainder with cheaper self\-reflection\. Offline, the expert teaches once and the resulting rule deploys without the expert, while online the expert is consulted per difficult task and needs no training set \(§[5\.4](https://arxiv.org/html/2610.12651#S5.SS4)\)\.

## 4Experimental Setup

#### Benchmarks\.

We evaluate on four agent benchmarks with binary success checks: InterCode\-SQL on Spider \(ntest=150n\_\{\\text\{test\}\}\{=\}150\([Yang et al\., 2023](https://arxiv.org/html/2610.12651#bib.bib28);[Yu et al\., 2018](https://arxiv.org/html/2610.12651#bib.bib31)\)\), TextCraft at recipe depth 3 \(n=58n\{=\}58\([Prasad et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib29)\)\), InterCode\-Bash on nl2bash \(n=40n\{=\}40\([Yang et al\., 2023](https://arxiv.org/html/2610.12651#bib.bib28);[Lin et al\., 2018](https://arxiv.org/html/2610.12651#bib.bib32)\)\), andτ\\tau\-bench retail \(n=115n\{=\}115\([Yao et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib30)\)\), training on a disjoint split of up to 50 tasks each\. We select these four because each exposes an observable state that recurs across rollouts, which is what makes a state key well defined under Assumption[1](https://arxiv.org/html/2610.12651#Thmassumption1), and because they span text\-to\-SQL, compositional crafting, shell scripting, and tool\-use dialogue \(Appendix[B\.1](https://arxiv.org/html/2610.12651#A2.SS1)\)\.

#### Agent models and expert\.

We run four models from three families, Qwen3\-32B\([Yang et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib33)\), Gemma\-3\-12B and Gemma\-3\-4B\([Gemma Team, 2025](https://arxiv.org/html/2610.12651#bib.bib34)\), and Ministral\-3\-3B\([Mistral AI, 2025](https://arxiv.org/html/2610.12651#bib.bib35)\), and substitute Claude Opus\([Anthropic, 2026](https://arxiv.org/html/2610.12651#bib.bib36)\)for the subject\-matter expert, held fixed across all1616settings and iterations\. The substitution is conservative: the expert sees only the acquired decision points and returns a short convention, an interface a human could equally serve, though whether one would author the same convention is untested\. We span weak\-to\-mid agents, where expert feedback has the most headroom, and Qwen3\-32B is the strong control \(Appendix[B\.2](https://arxiv.org/html/2610.12651#A2.SS2)\)\.

#### Baselines\.

We compare against four experience\-distillation baselines and give each the same expert SynAct receives\. The comparison thus isolates*where*expert attention is pointed from*whether*it is available:ACE\([Zhang et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib1)\)grows an evolving playbook of tactics prepended to the prompt,ReasoningBank\([Ouyang et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib2)\)retrieves distilled strategies from a memory of past episodes,AWM\([Wang et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib3)\)induces workflow snippets from successful trajectories, andAWM\-iterruns AWM for ten iterative expert induction rounds, matching our iteration count\. We tune setup parameters alone and leave the methods unmodified \(Appendix[B\.5](https://arxiv.org/html/2610.12651#A2.SS5)\)\. In §[5\.2](https://arxiv.org/html/2610.12651#S5.SS2)we add*acquisition\-function*baselines holding our pipeline fixed and varying only the selection rule: SynAct\-Succ \(I⁡\(A,succ\)I\(A;\\mathrm\{succ\}\)\), SynAct\-Ent \(action entropy\), and SynAct\-Rand \(uniform\)\.

#### Protocol\.

We report each setting as the mean of 3 repeats on the full test set, holding the rule, gate, and test set fixed except for the ablated component\. The gate re\-measures the unaided agent at each iteration, letting us estimate measurement noise directly: base accuracy varies by3\.43\.4–8\.58\.5pp across iterations\. We pool base where possible and treat differences of this size between adjacent iterations as indistinguishable \(Appendix[B\.6](https://arxiv.org/html/2610.12651#A2.SS6)\)\. Agent and expert tokens are reported separately because expert tokens cost roughly1010–30×30\\timesmore; turns are exact and tokens estimated \(Appendix[N](https://arxiv.org/html/2610.12651#A14)\)\.

## 5Results

### 5\.1Routed Expert Feedback Against Experience Distillation

#### Accuracy against experience distillation\.

Distilling experience into context is the prevailing way to improve a frozen agent\. We first ask what routing the same expert to decision points buys over distilling with it\. We give ACE, ReasoningBank, AWM, and ten\-round AWM\-iter the identical expert and hold the training split, test set, and agent fixed \(Figure[2](https://arxiv.org/html/2610.12651#S5.F2), Table[5](https://arxiv.org/html/2610.12651#A12.T5)\)\. SynAct improves on base in all 16 settings, by\+8\.3\+8\.3pp on average after a single iteration and\+11\.7\+11\.7pp at the best iteration\. Taking the strongest baseline per setting, an oracle over the four, yields\+4\.4\+4\.4pp on average, while no fixed baseline exceeds\+1\.0\+1\.0pp\. SynAct at one iteration therefore nearly doubles the gain of even the per\-setting oracle\. Exact sign tests over the 16 settings confirm the advantage against base, each baseline, and the oracle atp≤2\.1×10−3p\\leq 2\.1\\times 10^\{\-3\}\(Appendix[L](https://arxiv.org/html/2610.12651#A12)\)\. At its best iteration, SynAct leads all four baselines in 15 of 16 settings, the exception being Ministral TextCraft, where AWM reaches20\.720\.7against our19\.519\.5\. The largest gains occur where the base agent is weakest,\+24\.8\+24\.8pp for Gemma\-4B on Bash and\+18\.0\+18\.0pp for Ministral on SQL, which identifies SynAct’s most favorable regime: agents that can reach informative end states yet lack the knowledge to reach the correct one\. The controlled comparison points to*where*attention is spent: distillation summarizes whole trajectories, whereas SynAct routes the expert to the decisions determining outcomes \(Appendix[I](https://arxiv.org/html/2610.12651#A9)\)\.

Figure 2:Training\-time methods with an equal expert budget\(agent Qwen3\-32B, one panel per benchmark\): ACE, ReasoningBank, AWM, AWM\-iter, and SynAct at its best iteration, all given the same expert\. Table[5](https://arxiv.org/html/2610.12651#A12.T5)reports all four agent models\.
#### Downside risk\.

Mean accuracy conceals whether a method repairs failures or exchanges them for new errors\. To separate the two, we compare each method’s worst setting to its base\. SynAct stays at or above base within noise in every setting, whereas always\-on injection can substantially reduce accuracy: ReasoningBank costs Gemma\-4B17\.617\.6pp on SQL \(47\.6→30\.047\.6\\rightarrow 30\.0\), and ACE costs Ministral3\.63\.6pp on the same benchmark\. The damage concentrates on the weakest agents, the population with the most headroom, and the distinction is selective application\. Holding the rule fixed and varying only its application, unconditional injection yields 32 to 36 win→\\rightarrowlose flips on SQL, compared with00when gated\. We therefore deploy the gate without an additional validation step, and we further validate its firing decisions against an expert judge, which the gate rarely misses \(Appendix[J](https://arxiv.org/html/2610.12651#A10)\)\.

### 5\.2Comparison of Acquisition Functions

Consulting a strong model helps on its own\. We therefore ask whether the choice of decisions drives the gain\. We hold the rollouts, authoring call, and consistency gate fixed, varying only the selection rule\. Outcome\-MI is the best rule in all 8 swept settings and all 23 pairwise comparisons, gaining\+12\.5\+12\.5pp over base, compared with\+8\.2\+8\.2for random selection,\+6\.9\+6\.9for success\-MI, and\+6\.8\+6\.8for entropy\. The entropy comparison reveals why outcome information matters: when two actions differ frequently while converging on the same outcome, ranking by disagreement alone spends the budget on variation without consequence, precisely the distinction Proposition[1](https://arxiv.org/html/2610.12651#Thmproposition1)captures\. Success\-MI fails in the complementary regime, remaining silent on all\-fail tasks where the success label is constant \(Appendix[G](https://arxiv.org/html/2610.12651#A7)\)\. Under the final protocol, the four rules yield indistinguishable one\-iteration gains, suggesting that outcome information matters where the theory predicts: all\-fail coverage and per\-task routing, while the training loop is largely insensitive to the ranking \(Appendices[F](https://arxiv.org/html/2610.12651#A6)and[P](https://arxiv.org/html/2610.12651#A16)\)\.

### 5\.3Expert\-Token and Sampling Budgets

#### Cost of a routed expert budget\.

Figure 3:Gain vs\. cumulative expert tokens\(TextCraft, Qwen3\-32B, logxx\): SynAct over ten iterations, distillers at their measured budgets\.With the expert held fixed, we next measure how much accuracy each method buys per expert token\. Figure[3](https://arxiv.org/html/2610.12651#S5.F3)plots SynAct’s gain over base across ten iterations against the cumulative expert tokens spent authoring the rule, with the one\-shot distillers at their own measured budgets\. The distillers cluster near zero gain at comparable spend, whereas SynAct’s gain grows through the first several iterations before flattening: routed expert tokens keep buying accuracy after the distillers have saturated\. Expert turns diverge further, one call per iteration against one or more per task \(Appendix[M](https://arxiv.org/html/2610.12651#A13)\)\.

#### Sensitivity to sampling budget and iteration count\.

Since rollouts dominate the cost, we turn both knobs down\. All six swept settings beat base atk=2k\{=\}2, a4×4\\timescheaper collection, and the top decile of MI\-ranked decisions already delivers most of the full\-budget gain \(Table[4](https://arxiv.org/html/2610.12651#A6.T4), Appendices[F](https://arxiv.org/html/2610.12651#A6)and[H](https://arxiv.org/html/2610.12651#A8)\)\. Iteration then compounds what a single pass leaves: the first iteration captures70%70\\%of the median setting’s best\-iteration gain, and later passes lift Qwen TextCraft from\+11\.4\+11\.4pp to\+20\.6\+20\.6pp\. Across all160160iteration evaluations, exactly one dips below base, by0\.20\.2pp within noise \(Table[7](https://arxiv.org/html/2610.12651#A12.T7), Appendix[K](https://arxiv.org/html/2610.12651#A11)\)\.

### 5\.4Inference\-Time Routing

Figure 4:Inference\-time routing at a matched expert budget\(θ=0\.6\\theta\{=\}0\.6,R=3R\{=\}3, one panel per benchmark\): self\-reflection with ten retries, a token\-matched cascade, SynAct, and SynAct with the iteration\-10 rule active\. Table[9](https://arxiv.org/html/2610.12651#A15.T9)adds the action\-entropy and expert\-judge routers\.We finally test whether the allocation principle transfers to inference, matching each router to SynAct’s budget \(§[3\.5](https://arxiv.org/html/2610.12651#S3.SS5)\) and varying only its aim: a token\-matched cascade\([Chen et al\., 2024a](https://arxiv.org/html/2610.12651#bib.bib22);[Yue et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib24)\), an action\-entropy router, and an expert judge routing tasks itself \(Figure[4](https://arxiv.org/html/2610.12651#S5.F4), Table[9](https://arxiv.org/html/2610.12651#A15.T9)\)\. SynAct or its rule\-augmented variant is the best router in 15 of 16 settings, and on tasks both routers admit, the outcome\-MI branch rescues up to2\.7×2\.7\\timesmore \(Appendix[P](https://arxiv.org/html/2610.12651#A16)\)\. Against self\-reflection with ten retries\([Shinn et al\., 2023](https://arxiv.org/html/2610.12651#bib.bib9)\), three attempts plus routed consultation win 14 of 16 settings and add\+11\.6\+11\.6pp on average: one aimed correction outperforms seven unaimed retries\. The token\-matched cascade never exceeds the retry baseline, consistent with whole\-answer escalation wasting budget on localized failures\.

The learned rule and online routing also compose: activating the iteration\-10 convention alongside routing lifts accuracy in 14 of 16 settings over routing alone, by\+5\.1\+5\.1pp on average and\+12\.1\+12\.1pp for Gemma\-12B TextCraft, as the standing rule repairs the recurring mode and frees the routed call\. Raising the retry budget fromR=3R\{=\}3to1010never hurts and helps 11 of 16 settings \(Appendix[O](https://arxiv.org/html/2610.12651#A15)\)\.

## 6Conclusion

We recast improving an LLM agent as active learning over decision points: under scarce expert attention, the question is not which trajectories to review, but which decisions are worth explaining\. SynAct answers with the mutual information between an action at a state and the terminal outcome class, which under decision\-sufficient projections exactly separates outcome\-separating from outcome\-constant decisions at finitekkwhile remaining informative where success\-based scores are blind\. One expert call per iteration turns this signal into a gated convention improving all 16 settings, while four distillers using the same expert budget average under\+1\+1pp\. The same signal also routes consultation at deployment, suggesting a natural next step: closing the loop with human experts, where turns rather than tokens are the scarce resource\.

#### Limitations\.

Because Assumption[1](https://arxiv.org/html/2610.12651#Thmassumption1)constrains the projections rather than the dynamics, SynAct applies to stochastic environments, though we do not measure that degradation\. The label\-free claim covers acquisition alone, and the consistency gate can miss confidently wrong but behaviorally consistent failures\. Our expert is a proxy for a strong human, leaving the cost of human authoring untested, and the support floor can miss rare modes \(Appendices[J](https://arxiv.org/html/2610.12651#A10),[A\.4](https://arxiv.org/html/2610.12651#A1.SS4)\)\.

### AI use statement

In this work, we used generative AI tools for designing or providing feedback on research methodology or experiments, implementing methods, and interpreting results\. We have not used generative AI tools for generating synthetic data sets, developing theoretical models or conceptual frameworks, formulating mathematical claims, providing critical ingredients for proving mathematical claims, assisting in the writing of proofs, or proposing or refining hypotheses\. Assisting with translation, cleaning and reformatting datasets, and supporting qualitative and thematic data analysis are not applicable to this work\. Additionally, we used generative AI tools for editing the paper to improve readability and for identifying relevant literature\. We have reviewed all AI\-assisted work\. LLM\-generated code was verified and tested for correctness, AI\-suggested interpretations were checked against outputs, and AI\-identified references were verified against their source records\. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI\.

### Reproducibility statement

The method is specified completely in §[3](https://arxiv.org/html/2610.12651#S3), covering the state key and grouping \(§[3\.1](https://arxiv.org/html/2610.12651#S3.SS1), Appendix[C](https://arxiv.org/html/2610.12651#A3)\), the per\-benchmark outcome projectionsω\\omega\(Appendix[D](https://arxiv.org/html/2610.12651#A4)\), the acquisition score with itsm≥2m\{\\geq\}2arm floor \(Eq\.[1](https://arxiv.org/html/2610.12651#S3.E1)\), the verbatim authoring prompt and output schema \(Appendix[E](https://arxiv.org/html/2610.12651#A5)\), and the two\-attempt gated application protocol \(§[3\.4](https://arxiv.org/html/2610.12651#S3.SS4)\)\. Appendix[B](https://arxiv.org/html/2610.12651#A2)specifies splits, step limits, models, hyperparameters, the gate, baseline configurations, and the measurement protocol\. All benchmarks are public\([Yang et al\., 2023](https://arxiv.org/html/2610.12651#bib.bib28);[Prasad et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib29);[Yao et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib30)\)and agent models are served through a public API\. Each number in the iteration and budget tables comes from a script that refuses to print any entry lacking all 3 repeats: partially\-evaluated results keep moving, and no number is transcribed by hand\. The harness also distinguishes a task the agent failed from a call that never completed, retrying transport and throttling errors rather than recording them as failures\.

## References

- Agrawalet al\.\(2026\)L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang, C\. Potts, K\. Sen, A\. G\. Dimakis, I\. Stoica, D\. Klein, M\. Zaharia, and O\. KhattabGEPA: reflective prompt evolution can outperform reinforcement learning\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2507\.19457Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1)\.
- Anthropic \(2026\)AnthropicClaude Opus 4\.6 System Card\.Note:[https://www\.anthropic\.com/claude\-opus\-4\-6\-system\-card](https://www.anthropic.com/claude-opus-4-6-system-card)Accessed 2026\-08\-17Cited by:[§B\.2](https://arxiv.org/html/2610.12651#A2.SS2.p1.1),[§3](https://arxiv.org/html/2610.12651#S3.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2026\)A\. Chen, Z\. Liu, J\. Zhang, A\. Prabhakar, Z\. Liu, S\. Heinecke, S\. Savarese, V\. Zhong, and C\. XiongTest\-time adaptation for LLM agents via environment interaction\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2511\.04847Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2024a\)L\. Chen, M\. Zaharia, and J\. ZouFrugalGPT: how to use large language models while reducing cost and improving performance\.Transactions on Machine Learning Research\.Note:Featured Certification\. arXiv:2305\.05176External Links:ISSN 2835\-8856Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px4.p1.1),[§3\.5](https://arxiv.org/html/2610.12651#S3.SS5.SSS0.Px2.p1.1),[§5\.4](https://arxiv.org/html/2610.12651#S5.SS4.p1.1)\.
- Chenet al\.\(2024b\)M\. Chen, Y\. Li, Y\. Yang, S\. Yu, B\. Lin, and X\. HeAutoManual: constructing instruction manuals by LLM agents via interactive environmental learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2405\.16247Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1)\.
- Chenget al\.\(2026\)X\. Cheng, S\. He, L\. Feng, H\. Xu, M\. Yan, L\. Feng, and B\. AnBeyond trajectory\-level attribution: graph\-based credit assignment for agentic reinforcement learning\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:2605\.26684Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px4.p1.1)\.
- Choudhury and Sodhi \(2025\)S\. Choudhury and P\. SodhiBetter than your teacher: LLM agents that learn from privileged AI feedback\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2410\.05434Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px3.p1.1)\.
- Christianoet al\.\(2017\)P\. F\. Christiano, J\. Leike, T\. Brown, M\. Martic, S\. Legg, and D\. AmodeiDeep reinforcement learning from human preferences\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:1706\.03741Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px3.p1.1)\.
- Denget al\.\(2026\)M\. Deng, Z\. Li, X\. Li, T\. Zhu, Y\. Zhao, Z\. Guo, and W\. WangUncertainty\-aware clarification in LLM agents with information gain\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:2606\.03135Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px3.p1.1)\.
- Fenget al\.\(2025\)L\. Feng, Z\. Xue, T\. Liu, and B\. AnGroup\-in\-group policy optimization for LLM agent training\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2505\.10978Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px4.p1.1)\.
- Fuet al\.\(2024\)Y\. Fu, D\. Kim, J\. Kim, S\. Sohn, L\. Logeswaran, K\. Bae, and H\. LeeAutoGuide: automated generation and selection of context\-aware guidelines for large language model agents\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2403\.08978Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1)\.
- Gemma Team \(2025\)Gemma TeamGemma 3 technical report\.arXiv preprint arXiv:2503\.19786\.Cited by:[§B\.2](https://arxiv.org/html/2610.12651#A2.SS2.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px2.p1.1)\.
- Houlsbyet al\.\(2011\)N\. Houlsby, F\. Huszár, Z\. Ghahramani, and M\. LengyelBayesian active learning for classification and preference learning\.arXiv preprint arXiv:1112\.5745\.Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px3.p1.1)\.
- Huanget al\.\(2024\)J\. Huang, X\. Chen, S\. Mishra, H\. S\. Zheng, A\. W\. Yu, X\. Song, and D\. ZhouLarge language models cannot self\-correct reasoning yet\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.01798Cited by:[§1](https://arxiv.org/html/2610.12651#S1.p1.1),[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px2.p1.1)\.
- Kadavathet al\.\(2022\)S\. Kadavath, T\. Conerly, A\. Askell, T\. Henighan, D\. Drain, E\. Perez, N\. Schiefer, Z\. Hatfield\-Dodds, N\. DasSarma, E\. Tran\-Johnson, S\. Johnston, S\. El\-Showk, A\. Jones, N\. Elhage, T\. Hume, A\. Chen, Y\. Bai, S\. Bowman, S\. Fort, D\. Ganguli, D\. Hernandez, J\. Jacobson, J\. Kernion, S\. Kravec, L\. Lovitt, K\. Ndousse, C\. Olsson, S\. Ringer, D\. Amodei, T\. Brown, J\. Clark, N\. Joseph, B\. Mann, S\. McCandlish, C\. Olah, and J\. KaplanLanguage models \(mostly\) know what they know\.arXiv preprint arXiv:2207\.05221\.Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px4.p1.1)\.
- Kujanpääet al\.\(2026\)K\. Kujanpää, N\. Liu, S\. Alam, Y\. R\. Sura, T\. Yang, K\. Klinkner, and S\. MalmasiTool\-making and self\-evolving LLM agents in low\-latency systems\.arXiv preprint arXiv:2607\.08010\.Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1)\.
- Lightmanet al\.\(2024\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2305\.20050Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px4.p1.1)\.
- Linet al\.\(2018\)X\. V\. Lin, C\. Wang, L\. Zettlemoyer, and M\. D\. ErnstNL2Bash: a corpus and semantic parser for natural language interface to the Linux operating system\.InProceedings of the Eleventh International Conference on Language Resources and Evaluation \(LREC\),Note:arXiv:1802\.08979Cited by:[§B\.1](https://arxiv.org/html/2610.12651#A2.SS1.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2026a\)N\. Liu, P\. A\. Sreekar, K\. Kujanpää, Z\. Zhu, K\. Liu, C\. Sun, J\. M\. Menendez, M\. Bales, T\. Yang, S\. Alam,et al\.Eluna: an agentic LLM system for automating warehouse operations with reasoning and task execution\.arXiv preprint arXiv:2607\.08960\.Cited by:[§1](https://arxiv.org/html/2610.12651#S1.p1.1)\.
- Liuet al\.\(2026b\)N\. Liu, C\. Sun, K\. Klinkner, and S\. MalmasiBeyond pairs: your language model is secretly optimizing a preference graph\.arXiv preprint arXiv:2605\.08037\.Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px3.p1.1)\.
- Liuet al\.\(2026c\)N\. Liu, L\. Zhang, T\. Gao, and Y\. YuDisentangled representation learning for parametric partial differential equations\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2410\.02136Cited by:[§A\.7](https://arxiv.org/html/2610.12651#A1.SS7.p1.1)\.
- Madaanet al\.\(2023\)A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. ClarkSelf\-refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2303\.17651Cited by:[§1](https://arxiv.org/html/2610.12651#S1.p1.1),[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px2.p1.1)\.
- Majumderet al\.\(2024\)B\. P\. Majumder, B\. Dalvi Mishra, P\. Jansen, O\. Tafjord, N\. Tandon, L\. Zhang, C\. Callison\-Burch, and P\. ClarkCLIN: a continually learning language agent for rapid task adaptation and generalization\.InConference on Language Modeling \(COLM\),Note:arXiv:2310\.10134Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1)\.
- Mistral AI \(2025\)Mistral AIMinistral 3\.Note:[https://docs\.mistral\.ai/models](https://docs.mistral.ai/models)Ministral 3 3B \(v25\.12\), Apache 2\.0\. Accessed 2026\-09\-21Cited by:[§B\.2](https://arxiv.org/html/2610.12651#A2.SS2.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px2.p1.1)\.
- Onget al\.\(2025\)I\. Ong, A\. Almahairi, V\. Wu, W\. Chiang, T\. Wu, J\. E\. Gonzalez, M\. W\. Kadous, and I\. StoicaRouteLLM: learning to route LLMs from preference data\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2406\.18665Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px4.p1.1)\.
- Ouyanget al\.\(2026\)S\. Ouyang, J\. Yan, I\. Hsu, Y\. Chen, K\. Jiang, Z\. Wang, R\. Han, L\. T\. Le, S\. Daruki, X\. Tang, V\. Tirumalashetty, G\. Lee, M\. Rofouei, H\. Lin, J\. Han, C\. Lee, and T\. PfisterReasoningBank: scaling agent self\-evolving with reasoning memory\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2509\.25140Cited by:[§B\.5](https://arxiv.org/html/2610.12651#A2.SS5.p1.1),[§1](https://arxiv.org/html/2610.12651#S1.p1.1),[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px3.p1.1)\.
- Prasadet al\.\(2024\)A\. Prasad, A\. Koller, M\. Hartmann, P\. Clark, A\. Sabharwal, M\. Bansal, and T\. KhotADaPT: as\-needed decomposition and planning with language models\.InFindings of the Association for Computational Linguistics: NAACL 2024,pp\. 4226–4252\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-naacl.264)Cited by:[§B\.1](https://arxiv.org/html/2610.12651#A2.SS1.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2610.12651#S6.SSx2.p1.1)\.
- Rosset al\.\(2011\)S\. Ross, G\. Gordon, and D\. BagnellA reduction of imitation learning and structured prediction to no\-regret online learning\.InProceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics \(AISTATS\),Proceedings of Machine Learning Research, Vol\.15,pp\. 627–635\.Note:arXiv:1011\.0686Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px3.p1.1)\.
- Settles \(2009\)B\. SettlesActive learning literature survey\.Computer Sciences Technical ReportTechnical ReportTR1648,University of Wisconsin–Madison Department of Computer Sciences\.Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px3.p1.1)\.
- Seunget al\.\(1992\)H\. S\. Seung, M\. Opper, and H\. SompolinskyQuery by committee\.InProceedings of the Fifth Annual Workshop on Computational Learning Theory \(COLT\),pp\. 287–294\.External Links:[Document](https://dx.doi.org/10.1145/130385.130417)Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px3.p1.1)\.
- Shaoet al\.\(2026\)S\. Shao, Q\. Ren, D\. Liu, C\. Qian, B\. Wei, D\. Guo, J\. Yang, X\. Song, L\. Zhang, W\. Zhang, and J\. ShaoYour agent may misevolve: emergent risks in self\-evolving LLM agents\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2509\.26354Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2610.12651#S1.p1.1),[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px2.p1.1),[§3\.5](https://arxiv.org/html/2610.12651#S3.SS5.SSS0.Px2.p1.1),[§5\.4](https://arxiv.org/html/2610.12651#S5.SS4.p1.1)\.
- Snellet al\.\(2025\)C\. Snell, J\. Lee, K\. Xu, and A\. KumarScaling LLM test\-time compute optimally can be more effective than scaling parameters for reasoning\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2408\.03314Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2024\)P\. Wang, L\. Li, Z\. Shao, R\. Xu, D\. Dai, Y\. Li, D\. Chen, Y\. Wu, and Z\. SuiMath\-Shepherd: verify and reinforce LLMs step\-by\-step without human annotations\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 9426–9439\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.510)Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px4.p1.1)\.
- Wanget al\.\(2023\)X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. ZhouSelf\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2203\.11171Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px4.p1.1)\.
- Wanget al\.\(2025\)Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. NeubigAgent workflow memory\.InInternational Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.267,pp\. 63897–63911\.Note:arXiv:2409\.07429Cited by:[§B\.5](https://arxiv.org/html/2610.12651#A2.SS5.p1.1),[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px3.p1.1)\.
- Xuet al\.\(2025\)Y\. Xu, T\. Chakraborty, E\. Kiciman, B\. Aryal, S\. Sharma, S\. Lu, and R\. ChandraRLTHF: targeted human feedback for LLM alignment\.InInternational Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.267,pp\. 69096–69115\.Note:arXiv:2502\.13417Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px3.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. QiuQwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§B\.2](https://arxiv.org/html/2610.12651#A2.SS2.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px2.p1.1)\.
- Yanget al\.\(2023\)J\. Yang, A\. Prabhakar, K\. Narasimhan, and S\. YaoInterCode: standardizing and benchmarking interactive coding with execution feedback\.InAdvances in Neural Information Processing Systems \(NeurIPS\), Datasets and Benchmarks Track,Note:arXiv:2306\.14898Cited by:[§B\.1](https://arxiv.org/html/2610.12651#A2.SS1.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2610.12651#S6.SSx2.p1.1)\.
- Yaoet al\.\(2024\)S\. Yao, N\. Shinn, P\. Razavi, and K\. Narasimhanτ\\tau\-Bench: a benchmark for tool\-agent\-user interaction in real\-world domains\.arXiv preprint arXiv:2406\.12045\.Cited by:[§B\.1](https://arxiv.org/html/2610.12651#A2.SS1.SSS0.Px4.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2610.12651#S6.SSx2.p1.1)\.
- Yaoet al\.\(2023a\)S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. NarasimhanTree of thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2305\.10601Cited by:[§1](https://arxiv.org/html/2610.12651#S1.p1.1),[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px2.p1.1)\.
- Yaoet al\.\(2023b\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2210\.03629Cited by:[§3](https://arxiv.org/html/2610.12651#S3.SS0.SSS0.Px1.p1.1)\.
- Yuet al\.\(2018\)T\. Yu, R\. Zhang, K\. Yang, M\. Yasunaga, D\. Wang, Z\. Li, J\. Ma, I\. Li, Q\. Yao, S\. Roman, Z\. Zhang, and D\. RadevSpider: a large\-scale human\-labeled dataset for complex and cross\-domain semantic parsing and text\-to\-SQL task\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 3911–3921\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1425)Cited by:[§B\.1](https://arxiv.org/html/2610.12651#A2.SS1.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px1.p1.1)\.
- Yueet al\.\(2024\)M\. Yue, J\. Zhao, M\. Zhang, L\. Du, and Z\. YaoLarge language model cascades with mixture of thought representations for cost\-efficient reasoning\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2310\.03094Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px4.p1.1),[§3\.5](https://arxiv.org/html/2610.12651#S3.SS5.SSS0.Px2.p1.1),[§5\.4](https://arxiv.org/html/2610.12651#S5.SS4.p1.1)\.
- Zhanget al\.\(2026\)Q\. Zhang, C\. Hu, S\. Upasani, B\. Ma, F\. Hong, V\. Kamanuru, J\. Rainton, C\. Wu, M\. Ji, H\. Li, U\. Thakker, J\. Zou, and K\. OlukotunAgentic context engineering: evolving contexts for self\-improving language models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2510\.04618Cited by:[§B\.5](https://arxiv.org/html/2610.12651#A2.SS5.p1.1),[§1](https://arxiv.org/html/2610.12651#S1.p1.1),[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2610.12651#S4.SS0.SSS0.Px3.p1.1)\.
- Zhanget al\.\(2025\)S\. Zhang, M\. Yin, J\. Zhang, J\. Liu, Z\. Han, J\. Zhang, B\. Li, C\. Wang, H\. Wang, Y\. Chen, and Q\. WuWhich agent causes task failures and when? On automated failure attribution of LLM multi\-agent systems\.InInternational Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.267,pp\. 76583–76599\.Note:arXiv:2505\.00212Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px4.p1.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpeL: LLM agents are experiential learners\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19632–19642\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v38i17.29936)Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1)\.
- Zhaoet al\.\(2025\)W\. Zhao, M\. Yuksekgonul, S\. Wu, and J\. ZouSiriuS: self\-improving multi\-agent systems via bootstrapped reasoning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv:2502\.04780Cited by:[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px1.p1.1)\.
- Zhouet al\.\(2024\)A\. Zhou, K\. Yan, M\. Shlapentokh\-Rothman, H\. Wang, and Y\. WangLanguage agent tree search unifies reasoning, acting, and planning in language models\.InInternational Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.235,pp\. 62138–62160\.Note:arXiv:2310\.04406Cited by:[§1](https://arxiv.org/html/2610.12651#S1.p1.1),[§2](https://arxiv.org/html/2610.12651#S2.SS0.SSS0.Px2.p1.1)\.

## Appendix AIdentifiability of Consequential Decisions

We characterize whatℐ⁡\(v\)\\mathcal\{I\}\(v\)can identify from onlykkrollouts, proceeding from its empirical mutual\-information representation to finite\-sample separation, partial identifiability, sampling recovery, ranking behavior, and the assumptions these guarantees require\. For a decision nodevv, let𝒜m​\(v\)=\{a:\|Ra​\(v\)\|≥m\}\\mathcal\{A\}\_\{m\}\(v\)=\\\{a:\|R\_\{a\}\(v\)\|\\geq m\\\}be the arms clearing the support floor,Rm​\(v\)=⋃a∈𝒜m​\(v\)Ra​\(v\)R\_\{m\}\(v\)=\\bigcup\_\{a\\in\\mathcal\{A\}\_\{m\}\(v\)\}R\_\{a\}\(v\),N=\|Rm​\(v\)\|N=\|R\_\{m\}\(v\)\|, andna=\|Ra​\(v\)\|n\_\{a\}=\|R\_\{a\}\(v\)\|\. We compute entropies as plug\-in estimates in bits onRm​\(v\)R\_\{m\}\(v\), callvv*scorable*when\|𝒜m​\(v\)\|≥2\|\\mathcal\{A\}\_\{m\}\(v\)\|\\geq 2, and writeH2​\(x\)=−x​log2​x−\(1−x\)​log2⁡\(1−x\)H\_\{2\}\(x\)=\-x\\log\_\{2\}x\-\(1\-x\)\\log\_\{2\}\(1\-x\)\. Restating Eq\.[1](https://arxiv.org/html/2610.12651#S3.E1),

ℐ⁡\(v\)=H^​\(O\)−∑a∈𝒜m​\(v\)naN​H^​\(O∣A=a\)\.\\mathcal\{I\}\(v\)=\\hat\{H\}\(O\)\-\\sum\_\{a\\in\\mathcal\{A\}\_\{m\}\(v\)\}\\frac\{n\_\{a\}\}\{N\}\\hat\{H\}\(O\\mid A\{=\}a\)\.\(2\)
### A\.1The Mutual Information Representation

We first identify whatℐ⁡\(v\)\\mathcal\{I\}\(v\)measures, and the support restriction makes the answer exact\. Because both entropy terms are computed on the same populationRm​\(v\)R\_\{m\}\(v\), the arm weights sum to one, and Eq\.[2](https://arxiv.org/html/2610.12651#A1.E2)is exactly mutual information under the empirical joint distribution of actions and outcomes\.

###### Lemma 1\(Empirical joint representation\)\.

LetP^\\hat\{P\}be the empirical joint distribution of\(Av,O\)\(A\_\{v\},O\)onRm​\(v\)R\_\{m\}\(v\), that isP^​\(a,o\)=\|\{τ∈Ra​\(v\):ω⁡\(τ\)=o\}\|/N\\hat\{P\}\(a,o\)=\|\\\{\\tau\\in R\_\{a\}\(v\):\\omega\(\\tau\)=o\\\}\|/Nfora∈𝒜m​\(v\)a\\in\\mathcal\{A\}\_\{m\}\(v\)\. Thenℐ⁡\(v\)=IP^​\(Av,O\)\\mathcal\{I\}\(v\)=I\_\{\\hat\{P\}\}\(A\_\{v\};O\), the mutual information ofP^\\hat\{P\}\. Consequently

ℐ⁡\(v\)=H^​\(O\)−H^​\(O∣Av\)=H^​\(Av\)−H^​\(Av∣O\),\\mathcal\{I\}\(v\)=\\hat\{H\}\(O\)\-\\hat\{H\}\(O\\mid A\_\{v\}\)=\\hat\{H\}\(A\_\{v\}\)\-\\hat\{H\}\(A\_\{v\}\\mid O\),\(3\)and0≤ℐ⁡\(v\)≤min⁡\{H^​\(Av\),H^​\(O\)\}0\\leq\\mathcal\{I\}\(v\)\\leq\\min\\\{\\hat\{H\}\(A\_\{v\}\),\\hat\{H\}\(O\)\\\}\.

###### Proof\.

The marginal ofP^\\hat\{P\}onAvA\_\{v\}isP^​\(a\)=na/N\\hat\{P\}\(a\)=n\_\{a\}/N, and its conditional distribution givenAv=aA\_\{v\}\{=\}ais the empirical distribution ofω\\omegaonRa​\(v\)R\_\{a\}\(v\)\. Substituting these distributions into Eq\.[2](https://arxiv.org/html/2610.12651#A1.E2)givesℐ⁡\(v\)=IP^​\(Av,O\)\\mathcal\{I\}\(v\)=I\_\{\\hat\{P\}\}\(A\_\{v\};O\)\. The identities and bounds then follow from standard properties of mutual information\. BecauseP^\\hat\{P\}is fixed for each realization, no assumption about the sampling process is required\. ∎

We can therefore reason directly about the two variables that matter: the action taken atvvand the outcome reached after it\. The key question is whether finite sampling gives us enough separation to identify consequential decisions\.

### A\.2Finite\-Sample Separation

We next show that the support floor creates such a separation, and it is the only place where the rollout budgetkkenters the bound\.

###### Lemma 2\(Arm\-entropy floor\)\.

Ifvvis scorable thenH^​\(Av\)≥H2​\(m/N\)≥H2​\(m/k\)\\hat\{H\}\(A\_\{v\}\)\\geq H\_\{2\}\(m/N\)\\geq H\_\{2\}\(m/k\), with equality in the first bound when\|𝒜m​\(v\)\|=2\|\\mathcal\{A\}\_\{m\}\(v\)\|=2and the arm counts are\(m,N−m\)\(m,N\-m\)\.

###### Proof\.

Writeqa=na/Nq\_\{a\}=n\_\{a\}/N\. Scorability gives at least two arms withqa≥m/Nq\_\{a\}\\geq m/N\. Leta⋆a^\{\\star\}be an arm of minimum count, soqa⋆∈\[m/N,1/2\]q\_\{a^\{\\star\}\}\\in\[m/N,1/2\]\. By the grouping property of entropy,

H^​\(Av\)=H2​\(qa⋆\)\+\(1−qa⋆\)​H^​\(qa1−qa⋆\|a≠a⋆\)≥H2​\(qa⋆\)≥H2​\(m/N\),\\hat\{H\}\(A\_\{v\}\)=H\_\{2\}\(q\_\{a^\{\\star\}\}\)\+\(1\-q\_\{a^\{\\star\}\}\)\\,\\hat\{H\}\\\!\\left\(\\tfrac\{q\_\{a\}\}\{1\-q\_\{a^\{\\star\}\}\}\\Big\|\_\{a\\neq a^\{\\star\}\}\\right\)\\;\\geq\\;H\_\{2\}\(q\_\{a^\{\\star\}\}\)\\;\\geq\\;H\_\{2\}\(m/N\),using thatH2H\_\{2\}is non\-decreasing on\[0,1/2\]\[0,1/2\]\. Equality requires the residual term to vanish withqa⋆=m/Nq\_\{a^\{\\star\}\}=m/N, that is exactly two arms with counts\(m,N−m\)\(m,N\-m\)\.

Scorability forcesN≥2​mN\\geq 2m, andN≤kN\\leq k, som/k≤m/N≤1/2m/k\\leq m/N\\leq 1/2and monotonicity ofH2H\_\{2\}on\[0,1/2\]\[0,1/2\]givesH2​\(m/N\)≥H2​\(m/k\)H\_\{2\}\(m/N\)\\geq H\_\{2\}\(m/k\)\. ∎

We can now ask whether the score exactly separates the two decision types that matter most\. It does, provided the relevant arms are observed often enough\.

###### Proof of Proposition[1](https://arxiv.org/html/2610.12651#Thmproposition1)\.

*Clause 1, outcome\-constant nodes\.*Ifω\\omegais constant onRm​\(v\)R\_\{m\}\(v\), thenH^​\(O\)=0\\hat\{H\}\(O\)=0, and Lemma[1](https://arxiv.org/html/2610.12651#Thmlemma1)gives0≤ℐ⁡\(v\)≤H^​\(O\)=00\\leq\\mathcal\{I\}\(v\)\\leq\\hat\{H\}\(O\)=0\.

*Clause 2, outcome\-separating nodes\.*If the arms have disjoint outcome supports, then each observed outcome identifies a unique arm, soH^​\(Av∣O\)=0\\hat\{H\}\(A\_\{v\}\\mid O\)=0\. By Eq\.[3](https://arxiv.org/html/2610.12651#A1.E3),

ℐ⁡\(v\)=H^​\(Av\)−H^​\(Av∣O\)=H^​\(Av\)≥H2​\(m/k\),\\mathcal\{I\}\(v\)=\\hat\{H\}\(A\_\{v\}\)\-\\hat\{H\}\(A\_\{v\}\\mid O\)=\\hat\{H\}\(A\_\{v\}\)\\geq H\_\{2\}\(m/k\),where the last inequality is Lemma[2](https://arxiv.org/html/2610.12651#Thmlemma2)\.

*Separation\.*For anyt∈\(0,H2​\(m/k\)\)t\\in\\bigl\(0,H\_\{2\}\(m/k\)\\bigr\), outcome\-constant nodes haveℐ⁡\(v\)=0<t\\mathcal\{I\}\(v\)=0<tand outcome\-separating nodes haveℐ⁡\(v\)≥H2​\(m/k\)\>t\\mathcal\{I\}\(v\)\\geq H\_\{2\}\(m/k\)\>t, so thresholding atttseparates the two classes without error\. Atm=2m\{=\}2andk=8k\{=\}8,H2​\(2/8\)=H2​\(0\.25\)=2−34​log2​3≈0\.8113H\_\{2\}\(2/8\)=H\_\{2\}\(0\.25\)=2\-\\tfrac\{3\}\{4\}\\log\_\{2\}3\\approx 0\.8113, giving the admissible interval\(0,0\.811\)\(0,0\.811\)quoted in the proposition\. ∎

The result requires no asymptotics\. We obtain zero information when the outcome is constant, we obtain the full arm entropy when the outcome identifies the arm, and the support floor creates a finite\-sample gap between the two\. We also do not require each arm to produce a single outcome: an arm may produce several outcomes as long as no outcome is shared across arms\. What remains is the ground between the extremes, where supports overlap\.

### A\.3Partial Identifiability

We rarely expect decisions to be perfectly constant or perfectly separating: arms share some outcomes while differing on others\. We can characterize this intermediate regime directly through mutual information\.

###### Proposition 2\(Component\-wise partial identifiability\)\.

For a scorable nodevv, define the*resolved fraction*ρ⁡\(v\)=ℐ⁡\(v\)/H^​\(Av\)\\rho\(v\)=\\mathcal\{I\}\(v\)/\\hat\{H\}\(A\_\{v\}\)whenH^​\(Av\)\>0\\hat\{H\}\(A\_\{v\}\)\>0\. Then

ρ⁡\(v\)∈\[0,1\],ρ⁡\(v\)=1−H^​\(Av∣O\)H^​\(Av\),\\rho\(v\)\\in\[0,1\],\\qquad\\rho\(v\)=1\-\\frac\{\\hat\{H\}\(A\_\{v\}\\mid O\)\}\{\\hat\{H\}\(A\_\{v\}\)\},\(4\)soρ⁡\(v\)=1\\rho\(v\)=1exactly whenOOdetermines the arm, recovering the outcome\-separating case, andρ⁡\(v\)=0\\rho\(v\)=0exactly whenOOcarries no information about the arm, which includes the outcome\-constant case\. Furthermore

H^​\(Av\)−ℐ⁡\(v\)=H^​\(Av∣O\)≤log2⁡\|𝒜m​\(v\)\|,\\hat\{H\}\(A\_\{v\}\)\-\\mathcal\{I\}\(v\)=\\hat\{H\}\(A\_\{v\}\\mid O\)\\leq\\log\_\{2\}\\bigl\|\\mathcal\{A\}\_\{m\}\(v\)\\bigr\|,\(5\)and the shortfall ofℐ⁡\(v\)\\mathcal\{I\}\(v\)belowH^​\(O\)\\hat\{H\}\(O\)is the average within\-arm outcome entropy,H^​\(O\)−ℐ⁡\(v\)=∑a\(na/N\)​H^​\(O∣A=a\)\\hat\{H\}\(O\)\-\\mathcal\{I\}\(v\)=\\sum\_\{a\}\(n\_\{a\}/N\)\\hat\{H\}\(O\\mid A\{=\}a\)\.

###### Proof\.

By Lemma[1](https://arxiv.org/html/2610.12651#Thmlemma1),0≤ℐ⁡\(v\)≤H^​\(Av\)0\\leq\\mathcal\{I\}\(v\)\\leq\\hat\{H\}\(A\_\{v\}\), givingρ⁡\(v\)∈\[0,1\]\\rho\(v\)\\in\[0,1\], and Eq\.[4](https://arxiv.org/html/2610.12651#A1.E4)follows by dividingℐ⁡\(v\)=H^​\(Av\)−H^​\(Av∣O\)\\mathcal\{I\}\(v\)=\\hat\{H\}\(A\_\{v\}\)\-\\hat\{H\}\(A\_\{v\}\\mid O\)byH^​\(Av\)\\hat\{H\}\(A\_\{v\}\)\. Thusρ⁡\(v\)=1\\rho\(v\)=1iffH^​\(Av∣O\)=0\\hat\{H\}\(A\_\{v\}\\mid O\)=0, meaning each observed outcome co\-occurs with exactly one arm, whileρ⁡\(v\)=0\\rho\(v\)=0iffAvA\_\{v\}andOOare independent underP^\\hat\{P\}, which an outcome\-constant node satisfies becauseOOis degenerate\. Equation[5](https://arxiv.org/html/2610.12651#A1.E5)follows from Eq\.[3](https://arxiv.org/html/2610.12651#A1.E3)andH^​\(Av∣O\)≤H^​\(Av\)≤log2⁡\|𝒜m​\(v\)\|\\hat\{H\}\(A\_\{v\}\\mid O\)\\leq\\hat\{H\}\(A\_\{v\}\)\\leq\\log\_\{2\}\|\\mathcal\{A\}\_\{m\}\(v\)\|, and the final claim is Eq\.[2](https://arxiv.org/html/2610.12651#A1.E2)rearranged\. ∎

Equation[4](https://arxiv.org/html/2610.12651#A1.E4)gives the interpretation:H^​\(Av\)\\hat\{H\}\(A\_\{v\}\)measures how much the agent’s decisions vary, andℐ⁡\(v\)\\mathcal\{I\}\(v\)measures the portion of that variation the observed outcome resolves\. Arms reaching\{x,y\}\\\{x,y\\\}and\{z,w\}\\\{z,w\\\}at balanced counts giveℐ⁡\(v\)=H^​\(Av\)=1\\mathcal\{I\}\(v\)=\\hat\{H\}\(A\_\{v\}\)=1bit, whereas arms reaching\{x,y\}\\\{x,y\\\}and\{y,z\}\\\{y,z\\\}share the classyyand giveℐ⁡\(v\)=0\.5\\mathcal\{I\}\(v\)=0\.5\. SynAct therefore spends its acquisition budget on the component of decision variation the outcome actually resolves, and the digest exposes the corresponding divergent end states to the expert\.

### A\.4Recovery Under Repeated Sampling

The separation result is conditional on scorability\. The natural question is how often finite sampling recovers the relevant arms\. A consequential arm that appears only once is invisible to the score however different its outcome, and we quantify this coverage directly\.

###### Proposition 3\(Arm recovery under repeated sampling\)\.

Fix a nodevvand suppose each rollout is independent and reachesvvand takes armaawith probabilitypv,ap\_\{v,a\}\. ThenNv,a∼Binomial⁡\(k,pv,a\)N\_\{v,a\}\\sim\\mathrm\{Binomial\}\(k,p\_\{v,a\}\)and

Pr\[Nv,a≥m\]=1−∑j=0m−1\(kj\)pv,aj\(1−pv,a\)k−j,\\Pr\[N\_\{v,a\}\\geq m\]=1\-\\sum\_\{j=0\}^\{m\-1\}\\binom\{k\}\{j\}p\_\{v,a\}^\{\\,j\}\(1\-p\_\{v,a\}\)^\{k\-j\},\(6\)which atm=2m\{=\}2is1−\(1−pv,a\)k−k​pv,a​\(1−pv,a\)k−11\-\(1\-p\_\{v,a\}\)^\{k\}\-kp\_\{v,a\}\(1\-p\_\{v,a\}\)^\{k\-1\}\. Ifrrarms each satisfypv,a≥pminp\_\{v,a\}\\geq p\_\{\\min\}, a union bound givesPr⁡\[all​r​arms clear the floor\]≥1−r⁡\[\(1−pmin\)k\+k​pmin​\(1−pmin\)k−1\]\\Pr\[\\text\{all \}r\\text\{ arms clear the floor\}\]\\geq 1\-r\\bigl\[\(1\-p\_\{\\min\}\)^\{k\}\+kp\_\{\\min\}\(1\-p\_\{\\min\}\)^\{k\-1\}\\bigr\], andvvis scorable once any two arms clear it\.

###### Proof\.

Each rollout contributes an independent Bernoulli\(pv,a\)\(p\_\{v,a\}\)trial, givingNv,a∼Binomial⁡\(k,pv,a\)N\_\{v,a\}\\sim\\mathrm\{Binomial\}\(k,p\_\{v,a\}\)and Eq\.[6](https://arxiv.org/html/2610.12651#A1.E6)\. Atm=2m\{=\}2the failure probability\(1−pv,a\)k\+k​pv,a​\(1−pv,a\)k−1\(1\-p\_\{v,a\}\)^\{k\}\+kp\_\{v,a\}\(1\-p\_\{v,a\}\)^\{k\-1\}is non\-increasing inpv,ap\_\{v,a\}, and a union bound over therrarms atpv,a≥pminp\_\{v,a\}\\geq p\_\{\\min\}gives the stated bound\. Any two retained arms makevvscorable\. ∎

Table[1](https://arxiv.org/html/2610.12651#A1.T1)evaluates Eq\.[6](https://arxiv.org/html/2610.12651#A1.E6)at the defaultk=8k\{=\}8,m=2m\{=\}2\.

Table 1:Arm recovery at the default budgetk=8k\{=\}8,m=2m\{=\}2, from Eq\.[6](https://arxiv.org/html/2610.12651#A1.E6)\. An arm the agent takes in half its rollouts is retained with probability0\.9650\.965, whereas one taken a tenth of the time is retained with probability0\.1870\.187\. The support floor therefore selects reproducible decision components and discards singleton behavior\.The support floor therefore trades coverage for reproducibility\. We do not claimk=8k\{=\}8recovers every consequential decision: it recovers the portion of behavior supported by repeated evidence, which is also the portion a written convention can address\. Raisingkkwidens this coverage at linear rollout cost \(Appendix[F](https://arxiv.org/html/2610.12651#A6)\)\.

### A\.5Visibility Beyond Success

We also ask whether the signal survives when success supplies no information\. A natural alternative is to defineOOby success, and this works when some attempts succeed, with a sharp blind spot: when every attempt fails, every decision looks identical under the success projection\. Outcome information carries no such limitation\.

###### Proposition 4\(Beyond\-capability visibility\)\.

Letvvbe scorable on a task where no rollout succeeds, sosucc⁡\(τ\)=0\\mathrm\{succ\}\(\\tau\)=0for everyτ∈Rm​\(v\)\\tau\\in R\_\{m\}\(v\)\. Then the success\-based score is identically zero,ℐsucc​\(v\)=0\\mathcal\{I\}\_\{\\mathrm\{succ\}\}\(v\)=0\. If in addition two armsa,a′∈𝒜m​\(v\)a,a^\{\\prime\}\\in\\mathcal\{A\}\_\{m\}\(v\)have differing empirical outcome distributions,P^​\(O∣A=a\)≠P^​\(O∣A=a′\)\\hat\{P\}\(O\\mid A\{=\}a\)\\neq\\hat\{P\}\(O\\mid A\{=\}a^\{\\prime\}\), thenℐ⁡\(v\)\>0\\mathcal\{I\}\(v\)\>0\.

###### Proof\.

For the first claim,succ\\mathrm\{succ\}is constant onRm​\(v\)R\_\{m\}\(v\), so Clause 1 of Proposition[1](https://arxiv.org/html/2610.12651#Thmproposition1)givesℐsucc​\(v\)=0\\mathcal\{I\}\_\{\\mathrm\{succ\}\}\(v\)=0for every realization\. For the second, Lemma[1](https://arxiv.org/html/2610.12651#Thmlemma1)identifiesℐ⁡\(v\)\\mathcal\{I\}\(v\)withIP^​\(Av,O\)I\_\{\\hat\{P\}\}\(A\_\{v\};O\), which vanishes iffAvA\_\{v\}andOOare independent underP^\\hat\{P\}\. Independence would forceP^​\(O∣A=a\)=P^​\(O∣A=a′\)\\hat\{P\}\(O\\mid A\{=\}a\)=\\hat\{P\}\(O\\mid A\{=\}a^\{\\prime\}\), contradicting the hypothesis, soℐ⁡\(v\)\>0\\mathcal\{I\}\(v\)\>0\. ∎

The distinction is structural rather than empirical: success\-based acquisition is blind on all\-fail tasks by construction, whereas outcome information stays positive whenever different arms lead to distinguishable end states\. This is the mechanism behind the coverage gap measured in §[5\.2](https://arxiv.org/html/2610.12651#S5.SS2), including the Gemma\-12B Bash setting where success\-MI authors no rule at all\.

### A\.6Finite\-Sample Ranking

The two extremes admit exact separation, and SynAct does not threshold decisions into classes\. We rank them\. We therefore ask what finite\-sample estimation does to that ranking, and we answer under an exact null: take two arms with each run assigned uniformly, andω\\omegadrawn uniformly overccclasses independently of the arm, soI⁡\(Av,O\)=0I\(A\_\{v\};O\)=0for the underlying law\. Enumerating all2k​ck2^\{k\}c^\{k\}realizations and conditioning on scorability gives Table[2](https://arxiv.org/html/2610.12651#A1.T2)\.

Table 2:Exact null behavior of the plug\-in scorewhenOOis independent ofAvA\_\{v\}, so the true mutual information is00\. Computed by complete enumeration over all realizations, conditional on the node being scorable\.PscP\_\{\\text\{sc\}\}is the probability of scorability\. The estimator is upward\-biased, and the bias grows with the number of outcome classescc, which is why we rank byℐ⁡\(v\)\\mathcal\{I\}\(v\)rather than threshold it at00\.The plug\-in score is therefore not a zero\-centered statistic at finitekk\. The bias decays roughly as1/k1/kbut remains non\-negligible atk=8k\{=\}8: a node with no true dependence scores0\.1240\.124bits on average\. The bias also grows with the number of outcome classes, reaching0\.3460\.346bits atc=4c\{=\}4, motivating the coarse per\-benchmark projections of Appendix[D](https://arxiv.org/html/2610.12651#A4)\. Finally, the null reaches the maximum1\.01\.0bits with probability0\.0020\.002: no threshold strictly below the maximum eliminates false positives in this regime, and Proposition[1](https://arxiv.org/html/2610.12651#Thmproposition1)is correspondingly restricted to the two extremes\.

Ranking is more useful than thresholding here\. Model the arm\-outcome dependence as a binary symmetric channel in which the outcome class matches the arm except with flip probabilityε\\varepsilon, so the true mutual information is1−H2​\(ε\)1\-H\_\{2\}\(\\varepsilon\)bits, and compare an independent draw of a dependent node against an independent draw of a null node, each conditioned on its own scorability atk=8k\{=\}8,m=2m\{=\}2\. The dependent node receives the strictly higher score with probability0\.9950\.995atε=0\\varepsilon\{=\}0,0\.8820\.882atε=0\.125\\varepsilon\{=\}0\.125,0\.6920\.692atε=0\.25\\varepsilon\{=\}0\.25, and0\.5170\.517atε=0\.375\\varepsilon\{=\}0\.375, where the true information is0\.0460\.046bits and the comparison approaches chance\. The ranking therefore reliably lifts strongly separating nodes above the null at the default budget and degrades gracefully toward chance as the true dependence vanishes\.

These results motivate the way SynAct consumes the statistic, as an acquisition score and never as a hypothesis test: selecting the topBBnodes compares scores against one another, whereas thresholding at zero would admit the entire null\. The bias does vary with the support sizes, at the order of the Miller–Madow correction\(\|𝒜m​\(v\)\|−1\)​\(\|𝒪\|−1\)/\(2​N​ln⁡2\)\(\|\\mathcal\{A\}\_\{m\}\(v\)\|\-1\)\(\|\\mathcal\{O\}\|\-1\)/\(2N\\ln 2\)\. A general ranking\-recovery guarantee would therefore need a separation condition accounting for this cross\-node spread, and we do not claim one\. The practical mitigation is a coarse per\-benchmarkω\\omega, which holds\|𝒪\|\|\\mathcal\{O\}\|comparable across nodes by construction\. The scorability row also explains the default budget: atk=8k\{=\}8roughly93%93\\%of two\-arm nodes are scorable, against38%38\\%atk=4k\{=\}4\(Appendix[F](https://arxiv.org/html/2610.12651#A6)\)\.

### A\.7Assumptions and Scope

We have so far established statistical identifiability, and neither causal effect nor repairability\. The score measures association under the agent’s sampling distribution: a hidden variable can induce a high score without the action being responsible, and an outcome\-separating decision need not be repairable by a written convention\. We use “identifiability” narrowly, for whether finite\-sample rollouts separate outcome\-constant from outcome\-separating decisions, a weaker sense than the recovery of latent factors up to ambiguity in representation learning\([Liu et al\., 2026c](https://arxiv.org/html/2610.12651#bib.bib48)\), with the intermediate regime characterized by Proposition[2](https://arxiv.org/html/2610.12651#Thmproposition2)and Appendix[A\.6](https://arxiv.org/html/2610.12651#A1.SS6)\.

Two assumptions are load\-bearing, the clauses of Assumption[1](https://arxiv.org/html/2610.12651#Thmassumption1)\. Determinism is not one of them\.

#### State identifiability\.

We requireϕ\\phito distinguish distinct situations without making recurring ones unnecessarily unique\. Ifϕ\\phialiases distinct situations, as under partial observability, the arms pool incomparable decisions andℐ⁡\(v\)\\mathcal\{I\}\(v\)measures a mixture\. Ifϕ\\phiis over\-discriminative, reading timestamps or continuous values, groups become singletons and fail the support floor\. We therefore need sufficient recurrence without aliasing, and a learned representation could relax the exact\-match form we use\.

#### Outcome identifiability\.

We requireω\\omegato carry no encoding of the branching action: otherwise distinct arms are outcome\-disjoint by construction andℐ​\(v\)=H^​\(Av\)\\mathcal\{I\}\(v\)=\\hat\{H\}\(A\_\{v\}\), reducing the score to action entropy\. We therefore restrictω\\omegato the terminal state, ensuring that any dependence ofOOonAvA\_\{v\}is mediated by where the run ended\. A projection reading the actions a run performed violates this requirement, and one reading the resulting environment state satisfies it\.

#### Determinism\.

We require determinism for neither condition and use it only to simplify implementation: equivalent situations produce identical observations, which letsϕ\\phiuse exact matching, and retrying without the injected rule reproduces the original failure, giving the gated protocol of §[3\.4](https://arxiv.org/html/2610.12651#S3.SS4)its control arm for free\. Under stochastic dynamics we would instead use a resample\-only control to separate repair from environmental randomness, and the stochasticity makingℐ⁡\(v\)\\mathcal\{I\}\(v\)informative in our experiments already comes from policy sampling at temperature0\.70\.7rather than from the environment\.

For a new benchmark, we therefore need to redesign onlyϕ\\phiandω\\omega:ϕ\\phimust induce recurring decision contexts, andω\\omegamust preserve outcome distinctions without encoding the branching action\.

## Appendix BDetailed Experimental Setup

We provide the configuration needed to reproduce Table[5](https://arxiv.org/html/2610.12651#A12.T5), complementing the summary in §[4](https://arxiv.org/html/2610.12651#S4)\.

### B\.1Benchmarks, Splits, and Step Limits

We summarize the per\-benchmark configuration in Table[3](https://arxiv.org/html/2610.12651#A2.T3)\. We use exact\-match state keys because repeating an action sequence reproduces the observation sequence closely enough to group equivalent situations without a similarity threshold, as Assumption[1](https://arxiv.org/html/2610.12651#Thmassumption1)requires\. Theτ\\tau\-bench user simulator is the least controlled case because its replies are model\-generated, and Appendix[C](https://arxiv.org/html/2610.12651#A3)reports the resulting grouping statistics\. We fix the split seed to 42 so that train and test stay disjoint and reproducible across runs\.

Table 3:Per\-benchmark configuration\.*Steps*is the hard cap on agent actions per rollout\.*Workers*is the concurrency cap used during evaluation, which is 1 for the Docker\-backed shell environment because that environment is not thread\-safe\.#### InterCode\-SQL\.

We evaluate text\-to\-SQL on the Spider development set\([Yu et al\., 2018](https://arxiv.org/html/2610.12651#bib.bib31)\)through the InterCode harness\([Yang et al\., 2023](https://arxiv.org/html/2610.12651#bib.bib28)\)against a local MySQL instance\. The agent issues SQL statements, observes result rows or errors, and submits a final query\. We count an episode as successful when its result set matches the gold query’s\. The agent sees the natural\-language question plus the table\-to\-column schema\.

#### TextCraft\.

We evaluate compositional crafting on TextCraft\([Prasad et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib29)\)at recipe depth 3, meaning the target item requires a three\-level crafting tree\. The agent issuescraftandgetcommands and observes inventory changes\. We count an episode as successful when the target item is present at the end\. The prompt lists the full recipe menu and states that only raw leaf items may be obtained withget\.

#### InterCode\-Bash\.

We evaluate shell scripting on nl2bash\([Lin et al\., 2018](https://arxiv.org/html/2610.12651#bib.bib32)\)through InterCode, running each task in a Docker container\. We count an episode as successful when the submitted command’s stdout matches the reference command’s output exactly, which makes this the most format\-sensitive of the four\. Because the container environment is not thread\-safe, we run all Bash evaluation single\-worker\.

#### τ\\tau\-bench retail\.

We evaluate tool\-agent\-user interaction in the retail domain ofτ\\tau\-bench\([Yao et al\., 2024](https://arxiv.org/html/2610.12651#bib.bib30)\)\. The agent calls typed tools against a mock database while conversing with a simulated user, and we count an episode as successful when the final database state is correct and the domain policy wiki is respected\. We run the user simulator at temperature 0 so that episodes are reproducible\. This is the only benchmark of the four where an action is a structured tool call rather than a text command, and the only one requiring a native tool\-calling interface, which is why we do not evaluate the two Gemma models here\.

### B\.2Models and Serving

We serve all agent models through Amazon Bedrock: Qwen3\-32B\([Yang et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib33)\), Gemma\-3\-12B and Gemma\-3\-4B\([Gemma Team, 2025](https://arxiv.org/html/2610.12651#bib.bib34)\), and Ministral\-3\-3B\([Mistral AI, 2025](https://arxiv.org/html/2610.12651#bib.bib35)\)\. We use Claude Opus 4\.6\([Anthropic, 2026](https://arxiv.org/html/2610.12651#bib.bib36)\)as the expert, called with temperature00, a40004000\-token output cap, and extended reasoning enabled\. We sample acquisition rollouts at temperature0\.70\.7and evaluate at temperature00\(greedy\), which makes the two attempts of the gated protocol differ only in whether the rule is present\. We cap per\-step output at10241024tokens\.

#### Reasoning mode\.

Qwen3\-32B runs in thinking mode, and the trajectory text passed to the gate excludes the reasoning trace so that the gate judges actions rather than deliberation\. The other three models have no separate reasoning channel\.

#### Tool routing\.

All four models run on all four benchmarks, withτ\\tau\-bench tool calls routed natively through each model’s tool\-calling interface\.

### B\.3Acquisition Hyperparameters

We usek=8k\{=\}8rollouts per training task, a minimum arm size ofm=2m\{=\}2, and acquire one decision node per task, the highest\-scoring one\. We varyk∈\{2,4,6\}k\\in\\\{2,4,6\\\}and the expert budgetB∈\{10%,25%,50%\}B\\in\\\{10\\%,25\\%,50\\%\\\}of MI\-ranked decisions in the corresponding ablations, holding everything else fixed\. For the online variant, we use a consistency thresholdθ=0\.6\\theta\{=\}0\.6to decide whether a task needs intervention at all, and we thresholdℐ⁡\(v⋆\)\\mathcal\{I\}\(v^\{\\star\}\)on the remainder, sweeping its operating range so that the expert\-consultation rate spans the reported curve\.

We cache rollouts to disk per \(benchmark, model, task\), letting us resume sweeps without re\-sampling and recompute acquisition from a fixed rollout set\. Each ablation re\-uses the same cached rollouts wherever the setting permits, isolating the acquisition function from sampling variation\.

### B\.4Error Gate

Because we implement the gate as a single call to the*agent*model rather than to the expert, gating adds no expert cost, though it does add one agent call of latency to each failed attempt\. We pass it the task description, the first15001500characters of the failing trajectory, the antipattern sentence, and the rule, and it answersFIRESorSKIP\. We default malformed or failed responses toSKIP: an infrastructure error can only withhold the rule rather than inject it spuriously\. We recordnfiredn\_\{\\text\{fired\}\}, the number of tasks reaching the gated retry, alongside each accuracy number\.

### B\.5Baseline Configurations

We give every baseline the same expert model SynAct uses \(§[4](https://arxiv.org/html/2610.12651#S4)\), with the expert performing the method’s own distillation step\. Table[5](https://arxiv.org/html/2610.12651#A12.T5)thus compares where expert attention is allocated while holding expert strength fixed\. We configure each method by its own protocol: ACE\([Zhang et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib1)\)runs one epoch of reflect\-and\-curate over the training tasks and injects the resulting playbook always\-on, ReasoningBank\([Ouyang et al\., 2026](https://arxiv.org/html/2610.12651#bib.bib2)\)distils one memory item per training task and retrieves the topk=3k\{=\}3at test time, AWM\([Wang et al\., 2025](https://arxiv.org/html/2610.12651#bib.bib3)\)induces workflows from successful training trajectories and injects them always\-on, and AWM\-iter repeats the induction for ten rounds, matching our iteration count\. Consequently the expert\-turn counts of Table[8](https://arxiv.org/html/2610.12651#A14.T8)scale with the training set for the baselines and with the iteration count for SynAct\. We tune only setup parameters, namely training\-set size, rollout count, epoch count, retrieval depth, and token budget, and leave each method otherwise unmodified\. The self\-supervised variants of Appendix[I](https://arxiv.org/html/2610.12651#A9), ACE \(val\-select\) and ReasoningBank \(top\-1\), follow the earlier protocol of Appendix[H](https://arxiv.org/html/2610.12651#A8)and use no expert\.

### B\.6Measurement Protocol

We report each setting as the mean of 3 independent repeats over the full test set\. Each repeat re\-runs both attempts of the gated protocol, including the unaided first attempt, re\-measuring base accuracy within every repeat and at every iteration\. This lets us estimate measurement noise from the experiment itself instead of assuming it: base accuracy moves3\.43\.4–8\.58\.5pp across iterations of the same setting\. We therefore pool the base over completed iterations when reporting a base column and treat adjacent\-iteration differences of that magnitude as indistinguishable at 3 repeats\.

We generate per\-setting numbers with a script that reads the run directory and refuses to print an entry lacking all 3 repeats: a partially\-evaluated result cannot enter a table, and no number is transcribed by hand\. We also record paired flip counts \(win→\\rightarrowlose and lose→\\rightarrowwin\) per setting, because aggregate accuracy hides churn in which a method breaks as many tasks as it fixes\.

## Appendix CState\-Keyed Grouping

Acquisition compares arms only when they represent the same decision, and the grouping key determines which actions SynAct treats as alternatives from a shared state\. We describe the key used in our experiments and the approximation required by our cached rollout format\.

#### The key\.

We define a decision node as a maximal set of \(run, step\) pairs sharing a state key\. The intended key is a normalized hash of the observation atss, canonicalized per benchmark by lower\-casing, collapsing whitespace, and dropping volatile fields such as step counters, as constrained by Assumption[1](https://arxiv.org/html/2610.12651#Thmassumption1)\. Our cached rollouts, however, record actions rather than full observations at every step, and the implementation keys each depth by the ordered action prefix of the run\. An order\-invariant alternative keys on the prefix multiset\!a0,…,ad−1\!\{\!\{a\_\{0\},\\dots,a\_\{d\-1\}\}\!\}, additionally merging runs that reach one state through different action orders\. We audit this alternative below and find that its extra merges are usually illegitimate, precisely where they occur most often\.

#### Why not index by position\.

A position index treats stepddof every run as the same decision even when runs have reached different states, which happens routinely once a trajectory takes an extra exploratory action\. Recovering decision context from position alone therefore requires the action history to determine the state, whereas state keying makes the context explicit\. The tradeoff runs in the other direction as well: an over\-discriminativeϕ\\phifragments a node into singletons, which the support floor then discards\. The key thus trades coverage against the risk of aliasing distinct decisions \(Appendix[A\.7](https://arxiv.org/html/2610.12651#A1.SS7)\)\.

#### Collision audit\.

Does the ordered\-prefix key alias distinct decisions, or split decisions it should merge? We re\-group the cachedk=8k\{=\}8training rollouts from all 16 settings under the multiset key and compare the resulting groups\. Merges are structurally common in the flexible\-order benchmarks:1,9431\{,\}943of6,5366\{,\}536depth groups merge more than one ordered prefix\. Yet the acquired top\-1 set barely changes: 28 of 565 routed tasks change \(5\.0%5\.0\\%\), and the per\-setting overlap is at least0\.920\.92in 15 of 16 settings\. To determine whether these extra merges are genuine, we replay the merged branches to their reached states and ask Claude Opus 4\.6 whether they face the same decision\. On SQL, every sampled merge is legitimate \(0 of 19 judged distinct\)\. On TextCraft, however,90%90\\%conflate genuinely distinct decisions, different crafting sub\-goals reached through different action multisets, withτ\\tau\-bench between the two at17\.5%17\.5\\%\. The ordered\-prefix key is therefore the correct default: where action order carries state, the order\-invariant merge creates exactly the aliasing that Assumption[1](https://arxiv.org/html/2610.12651#Thmassumption1)rules out\.

## Appendix DOutcome Projectionsω\\omega

The acquisition signal needs an outcome representation that distinguishes the consequences of different actions without consulting a gold answer\. We therefore project each run’s terminal state into a benchmark\-specific outcome class using only its own trajectory; two runs share a class when they end in the same projected state\.

- •InterCode\-SQL: the last executedSELECT/WITHstatement, lower\-cased with whitespace collapsed\. Two runs submitting the same query share an outcome\.
- •TextCraft: the sorted set ofcraft/gettargets the run ended with, representing what it actually made, computed order\-invariantly\.
- •InterCode\-Bash: the final submitted command, normalized and excluding pure navigation \(cd,ls\)\. Bash success is determined by the command’s output, making the command the closest no\-gold proxy for the terminal state\.
- •τ\\tau\-bench retail: the multiset of write tool calls made by the run, computed order\-invariantly, sinceτ\\tau\-bench success is determined by the database end state\.

#### How sensitive is acquisition to the choice ofω\\omega?

We coarsen and refine each projection and rescore the cached rollouts\. Refining is nearly rank\-preserving on SQL \(Spearman0\.980\.98to1\.01\.0against the default\) and mildly destabilizing on TextCraft, whereas coarsening destabilizes both directions of the pipeline: on SQL, it collapses distinct queries into one class and shrinks the routed set from4343to1818tasks for Qwen; on TextCraft, it scrambles the ranking \(Spearman0\.460\.46to0\.520\.52\)\. Rank movement does not translate into end\-to\-end damage, however\. Authoring one iteration under the coarse projection is competitive\-to\-better on all four re\-run settings, and on TextCraft the coarse variant lifts more than the default \(\+17\.2\+17\.2vs\.\+13\.8\+13\.8pp for Qwen,\+12\.1\+12\.1vs\.\+6\.9\+6\.9pp for Gemma\-12B\) at matched coverage\. Grouping more divergent endings per mode lets the expert surface an additional valid convention\. The default projection thus sits in a usable middle, and the authored rule tolerates substantial variation in granularity\.

#### Sensitivity to projection granularity\.

We further examine how the granularity ofω\\omegaaffects acquisition by rescoring the cached rollouts under a coarser projection \(the table set touched on SQL, the count of craft actions on TextCraft\) and a finer one \(the raw un\-normalized terminal query, the full ordered action sequence\)\. Refining is nearly rank\-preserving on SQL \(Spearmanρ≥0\.98\\rho\\geq 0\.98against the default\) and mildly destabilizing on TextCraft \(ρ\\rhoof0\.620\.62to0\.840\.84\), while coarsening moves rankings more, shrinking the routed set on SQL from 43 to 18 tasks and droppingρ\\rhoto0\.460\.46on TextCraft\. Yet these ranking changes do not translate into end\-to\-end damage: authoring and evaluating under the coarseω\\omegamatches the default on SQL and improves TextCraft for both swept models \(\+17\.2\+17\.2vs\.\+13\.8\+13\.8pp for Qwen,\+12\.1\+12\.1vs\.\+6\.9\+6\.9pp for Gemma\-12B at one iteration\)\. Grouping more divergent endings into the same class gives the expert more evidence per mode\. The defaultω\\omegatherefore lies in a forgiving region, while coarser projections remain a defensible, cheaper setting for free\-form action tasks\.

## Appendix EWorked Example

What does the expert see after SynAct identifies a consequential decision? We trace one iteration on TextCraft with Qwen3\-32B\. We drawk=8k\{=\}8rollouts on each of the 50 training tasks, group their steps by state key, and score each node by Eq\.[1](https://arxiv.org/html/2610.12651#S3.E1)\. Keeping the highest\-scoring node per task yields 49 scorable tasks, including 20 from tasks where no rollout succeeded and which a success\-based score would have discarded \(Proposition[4](https://arxiv.org/html/2610.12651#Thmproposition4)\)\.

#### The evidence\.

We compress each acquired node into one digest block holding the task goal, two arms whose end states differ, the diverging action and the trajectory that followed for each, and acorrect/wrongtag from that arm’s success flag\. Acquisition itself reads no labels, and the tags enter only here\. The digest contains qualitatively different kinds of evidence\.

- •A task where both arms succeed by different routes, one reachingcraft 2 lime dyethroughgreen dyeand the other throughbone meal\. The expert sees two valid behaviors rather than a single canonical route\.
- •A task where both arms fail differently: one issuescraft 1 tripwire hookbefore its inputs exist, and the other loopscraft 4 stick using 2 plankssix times\.
- •AnALL\-FAILtask carrying two distinct wrong end states, one repeatingcraft 1 granite using 1 diorite, 1 quartzwithout acquiring the inputs\. No arm is labeled correct, and the divergent failures are the only evidence available\. This is precisely the case in which the success bit is silent\.

#### The rule the expert returns\.

From all 49 blocks in one call the expert produces the following convention, reproduced verbatim as the agent receives it\.

Convention authored by the expert\- Before issuing any ‘craft’ command, verify that ALL required input materials are already in your inventory; if not, first ‘get’ or craft the missing prerequisites\. \- At the start of a task, trace the full recipe tree to leaf \(raw\) materials and compute the TOTAL quantity of each needed; acquire all base materials before beginning the crafting chain\. \- Never repeat the same craft or get command more than twice in a row; if it fails or yields no progress, re\-check inventory and revise your plan\. \- Work bottom\-up: craft intermediate ingredients first, then combine them into the final product\.

Antipattern that gates itAttempting to craft items without first ensuring all input materials are present in inventory, leading to failed crafts, wasted steps, and infinite retry loops\.

#### What the expert abstracts\.

The four bullets cover distinct failure modes in the acquired evidence\. The first addresses crafting before acquiring ingredients, the third addresses the repeatedcraft 4 stickloop, and the second and fourth generalize the two successful routes into a plan\-then\-execute ordering\. Crucially, the convention names none of the concrete items that appeared in training and therefore transfers to recipe trees the training tasks never contained\. A per\-episode repair cannot produce this abstraction, since counting modes across blocks is what identifies which failures are systematic\. Prepending these four bullets behind the gate lifts accuracy in this setting by14\.214\.2pp over base\.

#### Application\.

The resulting convention is not applied blindly\. At test time the task first runs without it at temperature00, and a successful attempt terminates immediately\. After a failure, the gate reads the trajectory against the antipattern, and only when it fires do we prepend the convention and retry once\.

## Appendix FRollout and Expert Budget Sensitivity

SynAct spends two distinct budgets: rollouts determine how much behavioral variation we observe, and the expert budget determines how much of that variation we ask the expert to explain\. We vary each independently to measure how quickly the signal saturates\.

Table 4:The two budget knobs\.Accuracy \(%, 3\-repeat mean\) when cutting rollout countkk\(k=8k\{=\}8is the full method\) and expert budgetBB, the topB%B\\%of MI\-ranked decisions \(B=100%B\{=\}100\\%is full\)\. Each panel carries its own*base*, pooled over that panel’s settings because the gate re\-measures the unaided agent in every run, and bold marks the best setting per row\. In the mean\-gain row, — marks the base columns, which have no gain by definition\. All 36 results are complete at 3 repeats\.#### How many rollouts do we need?

Equation[1](https://arxiv.org/html/2610.12651#S3.E1)requiresm=2m\{=\}2runs per arm before a node becomes scorable, but additional rollouts can expose arms that a smaller budget misses and make the estimate more stable\. We therefore sweepk∈\{2,4,6\}k\\in\\\{2,4,6\\\}against thek=8k\{=\}8default \(Table[4](https://arxiv.org/html/2610.12651#A6.T4), left\)\. All six settings already beat their base atk=2k\{=\}2, a4×4\\timescheaper collection, and this small budget often captures most of the gain available atk=6k\{=\}6: Ministral SQL gains\+12\.0\+12\.0pp atk=2k\{=\}2against\+14\.4\+14\.4pp atk=6k\{=\}6over a pooled base of51\.651\.6\. Accuracy rises fromk=2k\{=\}2tok=6k\{=\}6in 5 of 6 settings, and the exception, Ministral TextCraft \(16\.9/18\.6/11\.916\.9/18\.6/11\.9on a12\.212\.2base\), is the noisiest setting in the grid\. Sincekkmultiplies the dominant collection cost, these results favor a small rollout budget: coarse behavioral divergence carries most of the useful signal without requiring a precise probability estimate\.

#### How much expert attention is enough?

The ranking lets us truncate the expert budget without changing the acquisition rule: we expose only the topB%B\\%of MI\-ranked decisions\. The top decile already gains\+7\.1\+7\.1pp over base on average and the quarter\-budget reaches\+9\.2\+9\.2pp, which on Qwen SQL gives71\.171\.1against the full\-budget68\.968\.9of Table[5](https://arxiv.org/html/2610.12651#A12.T5)\. The leading decisions therefore contain most of the dominant failure mode, making much of the remaining budget redundant\.

More budget does not always help\. AtB=50%B\{=\}50\\%the average gain falls to\+6\.8\+6\.8pp, below the quarter\-budget on 5 of 6 settings\. A larger digest contains more heterogeneous failure modes while the expert still compresses the evidence into a single convention, and beyond a point the rule becomes less focused rather than more complete\. The exception, Gemma\-12B TextCraft at31\.631\.6against31\.131\.1, is a0\.50\.5pp difference on a 59\-task set and does not separate at 3 repeats\.

The two sweeps share a pattern\. We do not need exhaustive behavioral coverage to obtain useful feedback: a handful of rollouts exposes enough divergent behavior to rank decisions, and the highest\-ranked decisions carry enough shared structure for the expert to form a useful convention\. SynAct therefore spends compute on discovering where behavior diverges and spends expert attention on the most concentrated part of that divergence\.

#### Where does the rollout budget actually bind?

A full ten\-iteration training sweep on TextCraft with Qwen sharpens the picture:k=2k\{=\}2degenerates completely, exposing no within\-state divergence, surfacing zero branches, and authoring no rule at any iteration\. In contrast,k∈4,8,16k\\in\{4,8,16\}all learn, with iteration\-10 lifts of\+20\.7\+20\.7,\+27\.6\+27\.6, and\+12\.1\+12\.1pp, respectively, within single\-repeat noise\. The sharp dependence is therefore thek=2k\{=\}2cliff, not the exact budget above it, consistent with the recovery probabilities in Proposition[3](https://arxiv.org/html/2610.12651#Thmproposition3): the signal needs enough rollouts to observe self\-disagreement at all, then saturates quickly\. Throttling the per\-iteration expert budget to the top25%25\\%or50%50\\%of routed tasks over the same sweep retains most of the full\-budget lift \(\+20\.7\+20\.7and\+17\.2\+17\.2vs\.\+27\.6\+27\.6pp\)\.

#### Sensitivity to the support floor\.

Rescoring the cached rollouts atm=1m\{=\}1andm=3m\{=\}3shows that the defaultm=2m\{=\}2sits at a knee\. Dropping tom=1m\{=\}1adds acquisitions in only 4 of 16 settings, almost all on the weakest model, whereas raising tom=3m\{=\}3costs coverage in 10 of 16, collapsing Ministral’s TextCraft set from 37 to 9 tasks\. Runningm=1m\{=\}1authoring end to end on the two most affected settings never hurts and helps TextCraft with Ministral \(\+5\.2\+5\.2vs\.\+1\.7\+1\.7pp at the same firing set\), because several singleton arms carry verifiably correct recipe facts that the floor would otherwise discard\. The floor is thus a safe default with a real coverage cost for weak agents, matching the coverage tradeoff in Proposition[3](https://arxiv.org/html/2610.12651#Thmproposition3)\.

#### Does the acquisition ranking itself carry the gain?

At one iteration under the final protocol, swapping the selection rule while holding the digest format, authoring call, and gate fixed yields statistically indistinguishable full\-set gains across all 16 settings \(\+6\.6\+6\.6to\+7\.6\+7\.6pp\), with per\-setting winners spread across all four signals\. The explanation is visible in the authored rules: when one failure mode dominates a benchmark, every selector over branching points surfaces it, and on SQL all four signals converge on the same convention\. The value of outcome information therefore concentrates where the theory predicts: all\-fail tasks that success\-based selection cannot see \(§[5\.2](https://arxiv.org/html/2610.12651#S5.SS2)\) and per\-task routing at inference time \(Appendix[P](https://arxiv.org/html/2610.12651#A16)\)\. The end\-to\-end training loop is nevertheless robust to the precise ranking, lowering the cost of adopting the method\.

## Appendix GCoverage of Acquisition Signals

How much of the gain comes from knowing*where*to ask the expert? The clean test holds the rollouts, authoring call, and gate fixed and varies only the selection rule, comparing outcome information against random selection over decision nodes, action entropy, and a success\-based score \(Table[6](https://arxiv.org/html/2610.12651#A12.T6), under the single\-iteration protocol of Appendix[H](https://arxiv.org/html/2610.12651#A8)\)\. Outcome\-MI is best in all 8 settings and all 23 pairwise comparisons, improving over base by\+12\.5\+12\.5pp on average against\+8\.2\+8\.2pp for random,\+6\.9\+6\.9pp for success\-MI, and\+6\.8\+6\.8pp for entropy\. Random selection is therefore a strong baseline, since a strong expert helps wherever it is pointed, and the additional margin is what pointing it at the right decisions buys\. More strikingly, entropy trails random: disagreement alone spends the budget on cosmetic variation that does not separate outcomes, precisely the distinction Proposition[1](https://arxiv.org/html/2610.12651#Thmproposition1)draws\. Success\-MI has a different failure mode: where no task contains both a winning and a losing rollout, it has nothing to acquire\.

The gap lies in coverage as well as ranking quality\. Accuracy in Table[6](https://arxiv.org/html/2610.12651#A12.T6)is defined only where a signal produces evidence, and Clause 1 of Proposition[1](https://arxiv.org/html/2610.12651#Thmproposition1)predicts where each signal goes silent\. Success\-MI requires*both*a winning and a losing rollout, and on Gemma\-12B Bash zero mixed contrasts occur, no rule is authored, and accuracy stays at the19\.219\.2base\. Outcome\-MI carries no such requirement: accuracy in the same setting reaches31\.731\.7\(\+12\.5\+12\.5pp\), and the signal acquires1\.9×1\.9\\timesmore decisions overall \(274274against146146on one SQL model\)\. This separates two properties of an acquisition signal:*quality*, how well it ranks the evidence available to it, and*coverage*, how often it produces evidence at all\. Under a fixed expert budget the latter binds\. A signal that goes silent on difficult tasks does not merely rank poorly: it leaves its budget unused exactly where feedback is most needed\.

## Appendix HScope of the Acquisition\-Function Comparison

Table[6](https://arxiv.org/html/2610.12651#A12.T6)gives the full acquisition\-function sweep on InterCode\-SQL and TextCraft\. We restrict the comparison to these two benchmarks because holding the rollouts, the authoring call, and the gate fixed while varying only the selection rule costs one complete training run per rule per setting\. The resulting comparison is broad across models and narrower across environments: the claim that outcome information beats random, entropy, and success\-based selection rests on 8 settings across two benchmarks\.

This scope also explains why the base rows differ from Table[5](https://arxiv.org/html/2610.12651#A12.T5)\. The sweep predates the equal\-expert protocol of §[5\.1](https://arxiv.org/html/2610.12651#S5.SS1), leaving its absolute values incomparable across tables\. Within each setting, however, the four selection rules share the same rollouts, expert, and gate, which isolates the acquisition function\. The same provenance applies to the rollout and budget sensitivity grid of Table[4](https://arxiv.org/html/2610.12651#A6.T4)\. The headline training results, the iteration curves of Table[7](https://arxiv.org/html/2610.12651#A12.T7), and the routing results of Table[9](https://arxiv.org/html/2610.12651#A15.T9)all come from the final protocol\.

## Appendix IWhy Self\-Supervised Curation Saturates

Can stronger curation close the gap without changing the source of supervision? We compare each baseline against a more selective variant under the single\-iteration protocol of Appendix[H](https://arxiv.org/html/2610.12651#A8):ACE \(val\-select\)deploys its playbook only when it beats the empty playbook on a held\-out split, andReasoningBank \(top\-1\)retrieves at most one memory\. Neither changes the basic picture\. ACE and ReasoningBank land within noise of base about as often as they improve on it, with a negative aggregate effect on the two weakest models on TextCraft, and the protective variants mostly trade failure for stagnation: ACE \(val\-select\) often declines to deploy, and ReasoningBank \(top\-1\) stays flat\. The limiting factor is therefore deeper than curation, since a weak agent’s traces record what it did without recording why it was wrong\.

Only two settings favor a baseline, both on Gemma\-3\-4B: ACE on Bash \(23\.323\.3against20\.020\.0, a margin of1\.31\.3tasks\) and ReasoningBank \(top\-1\) on TextCraft \(4\.04\.0against2\.82\.8\), both near the floor where under one task separates the methods\. The result points to a limitation of self\-supervised experience distillation: selecting better memories cannot recover information the generating agent never possessed\.

## Appendix JError Gate Ablation

#### Gating against unconditional injection\.

Does the non\-regression come from the authored rule or from the gate that decides when to apply it? Holding the rule fixed and changing only its application answers this directly\. Under the single\-iteration protocol of Appendix[H](https://arxiv.org/html/2610.12651#A8), unconditional injection produces 32 to 36 win→\\rightarrowlose flips on SQL against00when gated, and Table[5](https://arxiv.org/html/2610.12651#A12.T5)shows the same pattern in aggregate, with always\-on baselines regressing on several settings while SynAct regresses on none\. Flip counts expose the effect more clearly than accuracy alone, since equal numbers of repairs and regressions cancel in the aggregate\. The gate thus provides deployment\-time protection without a held\-out validation split: when the unaided attempt succeeds, the rule is withheld and cannot undo that success\.

#### Is the served gate faithful?

The gate is a single call to the agent model, and we measure how well it approximates expert judgment\. On sampled failing trajectories, we compare the served gate’s firing decision against an independent Claude Opus 4\.6 judgment of whether the failure exhibits the antipattern\. On TextCraft, Bash, andτ\\tau\-bench, the gate reaches recall1\.01\.0, firing on every failure flagged by the expert judge, at precision0\.580\.58to0\.860\.86\. SQL is the conservative outlier, with recall0\.220\.22at precision0\.500\.50, because its failures are heterogeneous and the per\-item selector often abstains\. Abstention is cheap, however: force\-injecting the convention into gate\-declined failures repairs only1515to21%21\\%of them: most skipped failures would not have been fixed by the rule anyway\. Firings on unrepairable failures waste a retry,4848to84%84\\%of firings depending on the benchmark, but because the gate is consulted only after a failure, this waste costs compute rather than accuracy: win→\\rightarrowlose churn is00in every setting\.

#### Does gating alone explain the gap?

A sharper decomposition redeploys each baseline’s own Opus\-authored library behind SynAct’s gate, changing only the deployment protocol\. Gating the ACE and AWM libraries recovers roughly half of SynAct’s full\-set gain \(\+4\.6\+4\.6and\+5\.0\+5\.0pp vs\.\+8\.2\+8\.2over the 12 swept settings\), showing that selective application is a substantial transferable source of benefit\. The remainder comes from the knowledge itself: SynAct’s rule repairs18%18\\%of the failures it fires on, versus11%11\\%for either gated baseline library\. On InterCode\-Bash, where the baseline libraries rarely trigger the gate, targeted acquisition dominates \(\+11\.7\+11\.7vs\.\+3\.1\+3\.1and\+2\.5\+2\.5pp\)\. Both components matter, but their relative contribution varies by benchmark\.

#### Does the author’s strength matter?

Swapping the authoring expert from Claude Opus 4\.6 to Claude Sonnet 4\.6, while holding the digest, gate, and evaluation fixed, lowers the mean firing\-set lift from\+8\.8\+8\.8to\+4\.9\+4\.9pp over 8 settings, with Opus ahead or tied in 7\. The gap concentrates on the hardest benchmark: on Bash with Qwen, Sonnet authored a convention prescribing exactly the buggy aggregation that the evidence warned against, and the gate never matched its antipattern to a failure, and the rule silently deactivated\. Author quality therefore matters, and a bad rule fails safely through inaction rather than causing damage\.

## Appendix KAccuracy Across Acquisition Iterations

#### Does more feedback keep helping?

A procedure that rewrites its own context on each pass can improve in two ways: later iterations may uncover new failure modes, but they may also compound mistakes from earlier ones\. We therefore run ten iterations per setting, re\-rolling the training set under the active rule, and evaluate each iteration’s rule independently \(Table[7](https://arxiv.org/html/2610.12651#A12.T7), Figure[5](https://arxiv.org/html/2610.12651#A11.F5)\)\. The resulting curves are not uniformly front\-loaded\. The first iteration already gives\+11\.4\+11\.4pp on Qwen TextCraft,\+15\.8\+15\.8pp on Ministral SQL, and\+15\.4\+15\.4pp on Qwen Bash, and it captures at least70%70\\%of the best\-iteration gain in half the settings\. Others continue to improve substantially: Qwen TextCraft rises from\+11\.4\+11\.4pp at iteration 1 to\+20\.6\+20\.6pp at iteration 6, and Qwenτ\\tau\-bench from\+10\.3\+10\.3pp to\+15\.2\+15\.2pp\. The distinction is intuitive: one convention can repair a dominant failure mode, whereas several modes require several passes\. Since each iteration costs one expert call, iteration count is therefore the more attractive knob for increasing expert compute\.

#### Does iteration introduce drift?

The curves show little evidence that successive rules accumulate damage\. Across the160160iteration evaluations in Table[7](https://arxiv.org/html/2610.12651#A12.T7), exactly one falls below its own base, Gemma\-12B Bash at iteration 3 \(23\.323\.3against a23\.523\.5base\), a0\.20\.2pp dip within noise on a 40\-task set\. The gate provides a simple explanation for this stability: when a task does not exhibit the antipattern, the rule is withheld, and an unhelpful convention cannot touch a solved attempt\.

Figure 5:Gain over the re\-measured greedy base per training iteration, one panel per benchmark and one line per agent model\. Solid lines are 3\-iteration moving averages and faded dots are raw 3\-repetition means\.

## Appendix LFull Per\-Iteration and Per\-Acquisition Tables

#### Statistical reporting\.

The test sets are small enough that a single task visibly moves a setting’s accuracy: one task changes accuracy by0\.70\.7pp on InterCode\-SQL \(n=150n\{=\}150\),1\.71\.7pp on TextCraft \(n=58n\{=\}58\),2\.52\.5pp on InterCode\-Bash \(n=40n\{=\}40\), and0\.90\.9pp onτ\\tau\-bench \(n=115n\{=\}115\)\. We therefore flag near\-floor comparisons where fewer than one task separates methods, treat each model–benchmark setting as the paired unit, and use exact one\-sided sign tests across the 16 settings\. SynAct at one iteration beats base in all 16 settings \(p=1\.5×10−5p=1\.5\\times 10^\{\-5\}\), each individual baseline in 14–15 settings \(p≤2\.1×10−3p\\leq 2\.1\\times 10^\{\-3\}\), and the per\-setting oracle in 14 settings \(p=2\.1×10−3p=2\.1\\times 10^\{\-3\}\)\. Routing shows the same pattern: SynAct\-rules wins all 16 settings against base and cascade \(p=1\.5×10−5p=1\.5\\times 10^\{\-5\}\), 15 against SR\(10\) \(p=2\.6×10−4p=2\.6\\times 10^\{\-4\}\), 14 against the judge \(p=2\.1×10−3p=2\.1\\times 10^\{\-3\}\), and 12 against action entropy \(p=1\.8×10−2p=1\.8\\times 10^\{\-2\}\)\. Because sign tests use only win counts, they are conservative and avoid conflating the different scales of accuracy changes across benchmarks\. Within each setting, exact McNemar tests on paired per\-task outcomes provide the same picture\. Across 63 single\-realization comparisons of SynAct’s best iteration against base and the three baselines, 31 are significant atp<0\.05p<0\.05and 37 have bootstrap95%95\\%confidence intervals strictly above zero\. Mean advantages range from\+9\.8\+9\.8to\+12\.6\+12\.6pp, with only three negative point estimates, none significant\. The largest gains are likewise robust: Bash with Gemma\-4B reaches\+35\.0\+35\.0pp against ACE, with 14 discordant wins versus 0 losses, while the strongest\-base settings appropriately straddle zero\.

Table 5:Training\-time methods with an equal expert, test accuracy \(%, 3\-repeat mean\), referenced from §[5\.1](https://arxiv.org/html/2610.12651#S5.SS1)\. The shaded rows are ours\.*base*for SynAct is pooled over its complete iterations, and each baseline reports its own paired greedy base\. SynAct appears after one training iteration and at its best iteration, with the best iteration in parentheses\. Bold marks the best learning method per column and underline the second best\.This appendix provides the two full grids summarized in the body: the acquisition\-function comparison of §[5\.2](https://arxiv.org/html/2610.12651#S5.SS2)\(Table[6](https://arxiv.org/html/2610.12651#A12.T6)\) and the per\-iteration curve of Appendix[K](https://arxiv.org/html/2610.12651#A11)\(Table[7](https://arxiv.org/html/2610.12651#A12.T7)\)\.

#### What makes the acquisition signal useful?

Table[6](https://arxiv.org/html/2610.12651#A12.T6)holds everything but the selection rule fixed, isolating the ranking from the rest of the pipeline\. Outcome\-MI wins every column, and the alternatives are informative in different ways\. Random selection reaches\+8\.2\+8\.2pp over base because the expert is useful wherever it is pointed: a randomly chosen decision still lands in a real failing trajectory, and the digest still describes something worth fixing\. Action entropy reaches only\+6\.8\+6\.8pp despite deliberately preferring variable decisions, and Proposition[1](https://arxiv.org/html/2610.12651#Thmproposition1)explains the gap: entropy cannot distinguish a cosmetic decision from an outcome\-separating one when their arm counts agree\. Part of the budget therefore buys blocks whose divergent arms reach the same end state, and the expert must then explain variation that the outcome does not resolve\. Ranking on the wrong quantity can be worse than not ranking at all\.

Success\-MI reaches\+6\.9\+6\.9pp and exposes a different limitation\. It fails outright on Gemma\-3\-4B TextCraft, where*n/a*means no task contains both a winning and a losing rollout: underω=succ\\omega\{=\}\\mathrm\{succ\}those nodes are outcome\-constant, the score is identically zero by Clause 1, and the budget goes unspent\. Outcome\-MI retains information whenever different actions lead to distinguishable outcomes, including when every rollout fails\.

#### What do successive iterations learn?

Table[7](https://arxiv.org/html/2610.12651#A12.T7)evaluates each iteration’s rule independently rather than committing to a schedule in advance, which lets a row be read as a curve\. Two regimes recur across the sixteen rows\. When one convention captures the dominant failure mode, performance peaks early and plateaus, as on Qwen Bash, which gains\+15\.4\+15\.4pp at iteration 1 and holds within a point of that level\. When training tasks contain several distinct modes, later iterations continue to add value: Qwen TextCraft climbs from58\.058\.0at iteration 1 to67\.267\.2at iteration 6\. Only one row peaks at iteration 10: a fixed schedule would either stop before the peak or spend expert calls after it, which motivates evaluating each iteration’s rule separately\. The*base*column moves across iterations because the gate re\-measures the unaided agent each time, and we pool those measurements and treat differences of a few points as inseparable at 3 repeats\.

Table 6:Acquisition\-function comparison\(accuracy %, 3\-repeat mean\), referenced from §[5\.2](https://arxiv.org/html/2610.12651#S5.SS2)\. Everything but the selection rule is held fixed, using the same rollouts, the same authoring call, and the same gate\.*n/a*means the signal acquired too little usable evidence to author a rule at all, a failure mode unique to the success\-based score, which requires a task to contain both a winning and a losing rollout\. The shaded row is ours, and bold marks the best rule per column\.Table 7:SynAct test accuracy \(%\) at each training iteration\(3\-repeat mean\), grouped by benchmark with one row per agent model, referenced from §[5\.3](https://arxiv.org/html/2610.12651#S5.SS3)and Appendix[K](https://arxiv.org/html/2610.12651#A11)\. The*base*column pools the unaided measurement over the completed iterations, because the gate re\-measures the base each iteration\.

## Appendix MCost of a Routed Expert Budget

#### Agent tokens against expert tokens\.

What does the budget look like when agent and expert computation are priced separately? SynAct spends*more total*tokens, running the agentk=8k\{=\}8times per training task, and roughly15×15\\times*fewer expert*tokens,0\.13%0\.13\\%of its own budget, calling the expert once per iteration rather than once or twice per training task\. In turns the separation widens to one expert turn per iteration against 50 to 100 per run \(Appendix[N](https://arxiv.org/html/2610.12651#A14)\)\. The two resources move in opposite directions, and total token count obscures the relevant trade\.

#### Substituting a human expert\.

The asymmetry matters most when the expert is a person rather than a model \(§[4](https://arxiv.org/html/2610.12651#S4)\)\. Human throughput is bound by the interventions that can be read and authored rather than by token price\. A method whose expert cost scales with the training set therefore becomes increasingly expensive to operate, whereas one intervention per iteration stays feasible under either instantiation\. We measure the per\-call load directly: the digest for the worked example of Appendix[E](https://arxiv.org/html/2610.12651#A5)spans4949blocks and roughly2727k characters, a reading task of minutes rather than hours, and the returned convention is22–44lines\. Expert turns are consequently the more meaningful unit for human deployment, which is why we report them alongside tokens\.

## Appendix NCost Accounting Details

Table 8:Training cost per \(model×\\timesdataset\) runatntrain=50n\_\{\\text\{train\}\}\{=\}50andk=8k\{=\}8\. Expert tokens are the expensive ones, costing1010–30×30\\timesthe agent’s per\-token rate\. Token counts are estimates and turn counts are exact\.TokensExpertSettingAgentExpertturnsSynAct, 1 iter\.7\.6M0\.01M1SynAct, 3 iter\.22\.9M0\.03M3SynAct, 10 iter\.76\.2M0\.10M10ACE \(1 epoch\)0\.95M0\.20M100ReasoningBank0\.95M0\.15M50Table[8](https://arxiv.org/html/2610.12651#A14.T8)separates agent and expert tokens because they price differently\. At one iteration SynAct spends0\.010\.01M expert tokens against ReasoningBank’s0\.150\.15M and ACE’s0\.200\.20M, a1515to20×20\\timesreduction\. The ratio narrows to55to6\.7×6\.7\\timesat three iterations, since each additional SynAct iteration adds one expert call while the baselines pay their expert cost once\. The largest savings therefore occur in the same regime where most of the accuracy gain is obtained, since iteration 1 captures most of the available improvement \(Appendix[K](https://arxiv.org/html/2610.12651#A11)\), and the headline15×15\\timesshould consequently be read as a single\-iteration figure rather than an iteration\-independent property\.

The token ratios also hide a second asymmetry\. Expert tokens account for0\.13%0\.13\\%of SynAct’s own budget at every iteration count, against17%17\\%for ACE and14%14\\%for ReasoningBank: a rise in the expert’s price per token barely moves SynAct’s total and moves the baselines’ substantially\. Expert turns diverge even more sharply, one call per iteration against 50 to 100 per baseline run, and for a human expert the turn count is the binding resource\. This is why we report both units\.

The remaining cost is agent\-side\. SynAct spends8×8\\timesmore agent tokens at one iteration because it runsk=8k\{=\}8rollouts per training task to obtain the acquisition scores\. This cost is reducible: Appendix[F](https://arxiv.org/html/2610.12651#A6)showskkcan be cut to22while still beating base on all six settings, whereas the baselines’ expert cost is not reducible without changing the methods\. Turn counts are exact, taken from cached traces, and token counts are estimates formed from exact call counts and per\-call anchors measured live\.

## Appendix OInference\-Time Routing Details

#### Where should the expert intervene?

At inference time the question shifts: instead of deciding which training decisions deserve explanation, we decide which test tasks deserve intervention\. For each task we drawk=8k\{=\}8rollouts at temperature0\.70\.7and group them by state key as in §[3\.1](https://arxiv.org/html/2610.12651#S3.SS1)\. A consistency gate decides whether intervention is warranted: we extract an outcome key per rollout, take the modal key, and route only when the modal fraction falls belowθ=0\.6\\theta\{=\}0\.6\. A task that agrees with itself is answered by majority vote at no cost, and a task already solved by any pooled rollout is kept as is\. The expert thus sees only tasks where the agent both disagrees with itself and has not already succeeded\. On a routed task,ℐ⁡\(v⋆\)\\mathcal\{I\}\(v^\{\\star\}\)selects the decision node, the expert returns a correction anchored at that node, and the agent retries with the correction as the first ofRRattempts\. Attempts2,…,R2,\\dots,Rare ordinary self\-reflection\.

#### Can other routers spend the same budget as effectively?

We cap every expert\-using alternative in Table[9](https://arxiv.org/html/2610.12651#A15.T9)to SynAct’s expert\-call countBBat this operating point, making the comparison like for like\. The cascade escalates whole tasks to the expert under the same expert\-token spend\. The action\-entropy router keeps the consistency gate but replacesℐ⁡\(v⋆\)\\mathcal\{I\}\(v^\{\\star\}\)witharg⁡maxv​δ​H​\(v\)\\arg\\max\_\{v\}\\delta H\(v\), spending itsBBexpert plans on the most uncertain tasks\. The judge hands the routing decision to the expert itself, capped atBBjudge calls and averaged over task\-arrival orders, since the judge cannot rank tasks it has not yet seen\. SynAct\-rules additionally activates the iteration\-10 convention from §[5\.3](https://arxiv.org/html/2610.12651#S5.SS3)during rollouts and retries\.

Table 9:Inference\-time routing at a matched expert budget\(θ=0\.6\\theta\{=\}0\.6,R=3R\{=\}3\), held\-out accuracy \(%\), referenced from §[5\.4](https://arxiv.org/html/2610.12651#S5.SS4)\. SR\(10\) is self\-reflection with ten retries and no expert\. Cascade, action\-entropy, and the expert judge are each throttled to SynAct’s expert budget at this operating point\. Bold marks the best routing method per row\.Figure 6:Accuracy against retry budgetRR, mean over the four agent models\. The no\-expert pool with self\-reflection against SynAct and SynAct\-rules atθ=0\.6\\theta\{=\}0\.6\.
#### How sensitive is routing to the consistency threshold?

Sweepingθ\\thetafrom0\.50\.5to0\.80\.8atR=3R\{=\}3expands the routed pool at every step, sincek=8k\{=\}8restricts the modal fraction to multiples of1/81/8\. Across the two SQL settings, accuracy changes by at most1\.31\.3pp while expert calls roughly triple, leaving little for the threshold to decide once the pool is near saturation\. TextCraft with Qwen is the one swept setting where the knob matters: accuracy rises\+6\.9\+6\.9pp fromθ=0\.5\\theta\{=\}0\.5to0\.70\.7before the routable pool saturates\. TextCraft with Ministral is flat because every threshold selects the same capability\-bound pool\. We therefore useθ=0\.6\\theta\{=\}0\.6for every benchmark and model without per\-setting tuning: where routing helps, a broad range of thresholds captures the gain\.

#### What if the agent gets more chances after intervention?

Table[10](https://arxiv.org/html/2610.12651#A15.T10)raises the retry budget fromR=3R\{=\}3toR=10R\{=\}10at the same expert budget and operating point, matching the retry allowance of SR\(10\)\. Accuracy never falls and rises in 11 of 16 settings, with the largest gain\+11\.3\+11\.3pp for Ministral onτ\\tau\-bench\. The correction and self\-reflection therefore compose rather than compete: the expert supplies a targeted change, and additional attempts continue the search from there\. Figure[6](https://arxiv.org/html/2610.12651#A15.F6)makes the comparison across the full retry range, and the routed variants dominate the no\-expert pool at matchedRRthroughout: the benefit of expert feedback does not vanish once the agent is allowed more retries\.

Table 10:SynAct atR=10R\{=\}10againstR=3R\{=\}3, same expert budget and operating point \(θ=0\.6\\theta\{=\}0\.6\), accuracy \(%\)\.

## Appendix PRouting Signal at Inference: Outcome\-MI Against Action Entropy

The training loop is robust to the acquisition ranking \(Appendix[F](https://arxiv.org/html/2610.12651#A6)\)\. We therefore ask where the choice of signal does matter\. Inference\-time routing is the natural place to look: the signal is spent per task rather than amortized into a reusable rule, and a misplaced branch point cannot be recovered through aggregation across tasks\. We compare SynAct’s outcome\-MI branch point with the action\-entropy alternative \(arg⁡maxv​δ​H​\(v\)\\arg\\max\_\{v\}\\delta H\(v\)\) onτ\\tau\-bench retail from two views\. First, we run both routers on the full benchmark at the shared operating point \(θ=0\.6\\theta\{=\}0\.6,R=3R\{=\}3\)\. Full\-benchmark accuracy mixes the routing decision with the consistency gate and the unrouted pool\. The second view therefore isolates the signal itself: both routers admit exactly the sameBBtasks through the consistency gate, differing only in*where*in the trajectory the expert plan is requested\. We report the resulting rescue rate, solved over routed, on this shared task set\. Gemma\-3\-4B is excluded because its near\-floor accuracy leaves too few rescues for a meaningful comparison\.

Table 11:Outcome\-MI against action entropy as the routing signal\(τ\\tau\-bench retail,θ=0\.6\\theta\{=\}0\.6,R=3R\{=\}3,n=115n\{=\}115\)\. Left: full\-benchmark accuracy \(%\)\. Right: expert rescues among theBBtasks both signals route, solved/routed \(%\)\. Both signals route the sameBBtasks per model, and the rescue columns isolate*where*the expert plan is anchored\. Bold marks the better signal per view\.On the full benchmark, outcome\-MI leads action entropy by1\.81\.8–4\.44\.4pp across the three models, although this gap reflects both the routing signal and the rest of the routing pipeline\. The controlled comparison is more revealing: on the shared routed set, where the only difference is the branch presented to the expert, outcome\-MI rescues1313–21%21\\%of routed tasks, compared with55–15%15\\%for action entropy, a1\.4×1\.4\\times–2\.7×2\.7\\timesadvantage that widens on the weakest model\. This is the regime captured by Proposition[1](https://arxiv.org/html/2610.12651#Thmproposition1): entropy identifies the step where the agent is most uncertain, which need not be the step that determines the outcome\. Anchoring the expert’s single correction at a cosmetic branch therefore spends the consultation without changing where the run ends\.

The contrast with training\-time robustness is instructive\. Aggregating evidence into a digest lets the expert recover from imperfect selection, whereas a per\-task consultation has no such slack\. The value of a consequence\-aware signal therefore concentrates where feedback cannot be amortized: at inference, choosing*where*to intervene is itself the bottleneck\.