MOCHA: Multi-Objective Chebyshev Annealing for Agent Skill Optimization

# Multi-Objective Chebyshev Annealing for Agent Skill Optimization
Source: [https://arxiv.org/html/2605.19330](https://arxiv.org/html/2605.19330)
Md Mehrab Tanjim, Jayakumar Subramanian, Xiang Chen, Branislav Kveton, Subhojyoti Mukherjee, Anlan Zhang, Sungchul Kim, Somdeb Sarkhel, Sunav Choudhury Adobe Research \{tanjim, jasubram, xiangche, kveton, subhomuk, anlanz, sukim, sarkhel, schoudha\}@adobe\.com

###### Abstract

LLM agents organize behavior through*skills*—structured natural\-language specifications governing how an agent reasons, retrieves, and responds\. Unlike monolithic prompts, skills are multi\-field artifacts subject to hard platform constraints: description fields are truncated for routing, instruction bodies are compacted via progressive disclosure, and co\-resident skills compete for limited context windows\. These constraints make skill optimization*inherently*multi\-objective: a skill must simultaneously maximize task performance and satisfy platform limits\. Yet existing prompt optimizers either ignore these trade\-offs or collapse them into a weighted sum, missing Pareto\-optimal variants in non\-convex objective regions\. We introduceMOCHA\(Multi\-ObjectiveCHebyshevAnnealing\), which replaces single\-objective selection with Chebyshev scalarization— covering the full Pareto front, including non\-convex regions—combined with exponential annealing that transitions from exploration to exploitation\. In our experiments across six diverse agent skills—where all methods share the same multi\-objective mutation operator and baselines receive identical per\-objective textual feedback—existing optimizers fail to improve the seed skill on 4 of 6 tasks: 1000 rollouts yield zero progress\. MOCHA breaks through on every task, achieving7\.5%relative improvement in mean correctness over the strongest baseline \(up to14\.9%on FEVER and10\.4%on TheoremQA\) while discoveringtwiceas many Pareto\-optimal skill variants\.

## 1Introduction

The dominant abstraction in early LLM applications was the*prompt*—a monolithic natural\-language string optimized end\-to\-end for a single task\[[4](https://arxiv.org/html/2605.19330#bib.bib1),[27](https://arxiv.org/html/2605.19330#bib.bib2)\]\. As LLM\-powered agents have grown more capable, a richer abstraction has emerged: the*skill*\. A skill is a structured behavioral specification—comprising a description field \(used for routing and retrieval\), an instruction body \(governing reasoning and response\), and metadata \(preconditions, output schema\)—that encapsulates a reusable unit of agent behavior\[[25](https://arxiv.org/html/2605.19330#bib.bib22),[28](https://arxiv.org/html/2605.19330#bib.bib21)\]\. Modern agent frameworks organize their entire behavioral repertoire as skill/plugin libraries: a coding agent selects among debugging, refactoring, and explanation skills; a customer\-facing agent routes between product suggestion, policy lookup, and escalation skills\.

Because skills are ultimately expressed in natural language, automated prompt optimization\[[32](https://arxiv.org/html/2605.19330#bib.bib4),[21](https://arxiv.org/html/2605.19330#bib.bib3),[14](https://arxiv.org/html/2605.19330#bib.bib8),[20](https://arxiv.org/html/2605.19330#bib.bib9)\]can be applied to refine them \(illustrated in[Figure1](https://arxiv.org/html/2605.19330#S1.F1)a\)\. But prompt optimizers treat their target as a single text blob optimized for a single metric\. Skills are not single\-objective artifacts\. They are*multi\-field*specifications subject to*hard platform constraints*: description fields are truncated at 1,024 characters in routing indexes; instruction bodies exceeding a certain limit of characters are truncated at deployment; and co\-resident skills share a finite context budget, so one verbose skill reduces the token budget available to its neighbors\[[2](https://arxiv.org/html/2605.19330#bib.bib27),[28](https://arxiv.org/html/2605.19330#bib.bib21)\]\. Conversely, a skill compressed to fit within limits may sacrifice the reasoning structure that drives performance\.*Every author of a deployed skill faces this tension*—yet no existing optimizer acknowledges it\.

![Refer to caption](https://arxiv.org/html/2605.19330v1/x1.png)Figure 1:\(a\) Skill optimization produces a correctness–compliance trade\-off: the optimized skillp∗p^\{\*\}gains correctness but may violate compliance limits\. \(b\) MOCHA navigates this trade\-off via two phases: exploration \(green\) expands the Pareto front, then exploitation \(purple\) refines the extremes\.The natural adaptation is to employ reflection\-based prompt optimization techniques\[[21](https://arxiv.org/html/2605.19330#bib.bib3),[31](https://arxiv.org/html/2605.19330#bib.bib10),[1](https://arxiv.org/html/2605.19330#bib.bib34)\], which refine text through iterative textual feedback\. One could extend these methods by incorporating per\-objective textual feedback into their mutation step—and indeed our experiments do exactly this—yet that alone is insufficient, as our results demonstrate\. The root cause lies in candidate selection: all three methods ultimately collapse multiple objectives into a single scalar, whether by greedy pick or bandit score— missing Pareto\-optimal solutions in non\-convex regions\.

Key insight: skill optimization is a structured multi\-objective problem that requires principled Pareto front navigation\.The Pareto front of a skill—the set of non\-dominated variants trading accuracy against platform compliance across multiple fields—can be*non\-convex*, meaning linear methods cannot reach all optimal points, where Chebyshev scalarization provably covers the full front\[[18](https://arxiv.org/html/2605.19330#bib.bib15)\]\. However, under limited budget, Chebyshev alone converges to a narrow front with limited diversity, as our experiments confirm\. This motivates two modes:*exploration*, which uses HVC\-gated acceptance early on to push the front broadly by discovering diverse trade\-off points; and*exploitation*, which anneals to Chebyshev\-consistent acceptance as the front matures to refine the weakest objective directly \(shown in[Figure1](https://arxiv.org/html/2605.19330#S1.F1)b\)\. The crux is transitioning between these modes as the budget is consumed\.

Contributions\.We present MOCHA \(Multi\-ObjectiveCHebyshevAnnealing\), a framework for multi\-objective skill optimization in LLM agents:

- •Problem formulation: We formalize skill optimization as a structured multi\-objective problem over multi\-field natural\-language artifacts subject to hard platform constraints \(SKILL\.md field limits\), identifying competing objectives—task correctness and platform compliance—that existing single\-objective optimizers collapse or ignore\.
- •Multi\-objective optimization \(MOO\) in discrete NL: While Chebyshev scalarization and hypervolume\-based optimization are well\-studied in continuous spaces\[[18](https://arxiv.org/html/2605.19330#bib.bib15),[16](https://arxiv.org/html/2605.19330#bib.bib44),[19](https://arxiv.org/html/2605.19330#bib.bib46)\], their efficacy in the*discrete, sample\-expensive*setting of natural\-language skill search remains largely underexplored\. MOCHA integrates these mechanisms within a unified SKILL\.md\-aware mutation framework, showing that principled MOO machinery yields consistent gains over other heuristic approaches in this setting\.
- •Comprehensive evaluation: We evaluate across six diverse agent skills and reflection\-based optimization baselines \(TextGrad, ProTeGi, GEPA\) on Claude Haiku 4\.5 as the skill\-execution backbone, measuring correctness, compliance, and hypervolume\. All methods share the sameSKILL\.md\-aware mutation interface with identical per\-objective textual feedback; the sole independent variable is the candidate selection strategy\. Across six agent skills, existing optimizers get stuck: on 4 of 6 tasks, all three baselines return the seed skill unchanged after 1000 rollouts\. MOCHA breaks through on every task, achieving7\.5%relative improvement in mean correctness over the strongest baseline—with gains up to14\.9%on FEVER and10\.4%on TheoremQA—while discoveringtwicemore Pareto\-optimal skill variants\.

## 2Related Work

MOCHA sits at the intersection of three research threads: prompt/instruction optimization, agent skill libraries, and multi\-objective optimization\. We discuss each in turn, highlighting the specific gap that MOCHA fills\.

Prompt and instruction optimization\.Automated prompt optimization methods fall into two broad categories\.*Gradient\-dependent*methods—including trace\-based optimization\[[6](https://arxiv.org/html/2605.19330#bib.bib33)\]and RL\-based search\[[9](https://arxiv.org/html/2605.19330#bib.bib20)\]—require differentiable computation traces and policy\-gradient reward signals, making them inapplicable to black\-box optimization of skill definitions\.*Gradient\-free*methods operate solely through LLM calls and divide further into: \(1\)*propose\-and\-rank*approaches\[[32](https://arxiv.org/html/2605.19330#bib.bib4),[29](https://arxiv.org/html/2605.19330#bib.bib32),[12](https://arxiv.org/html/2605.19330#bib.bib12)\]that propose a batch of candidates, score them, and select the best—without iterative textual feedback between rounds; and \(2\)*reflection\-based iterative refinement*\[[21](https://arxiv.org/html/2605.19330#bib.bib3),[31](https://arxiv.org/html/2605.19330#bib.bib10),[1](https://arxiv.org/html/2605.19330#bib.bib34)\]that refine candidates through an iterative loop of execution, textual critique, and mutation\. MOCHA belongs to the second family—reflection\-based methods are the natural fit for multi\-field skill optimization, where compliance violations and correctness failures require qualitatively different corrective signals that only iterative textual feedback can deliver\. Among reflection\-based methods, the key differentiator is candidate selection: ProTeGi\[[21](https://arxiv.org/html/2605.19330#bib.bib3)\]uses UCB\-based beam search, TextGrad\[[31](https://arxiv.org/html/2605.19330#bib.bib10)\]uses greedy selection, and GEPA\[[1](https://arxiv.org/html/2605.19330#bib.bib34)\]introduces Pareto\-aware filtering but defines the Pareto front over validation datapoints rather than objectives themselves\. Critically, all prior methods treat the optimization target as a*monolithic prompt*optimized for a single metric—none account for the structured, multi\-field, constraint\-governed nature of agent skill definitions\.

Agent skill discovery and refinement\.Learning from feedback in LLM\-based agentic systems can proceed along two axes: updating the underlying model’s weights, or updating the skills that govern its behavior\[[28](https://arxiv.org/html/2605.19330#bib.bib21),[25](https://arxiv.org/html/2605.19330#bib.bib22)\]\. We restrict attention to the latter—specifically, to*refining*existing skill definitions rather than discovering new ones\. Skill discovery methods \(SkillRL\[[28](https://arxiv.org/html/2605.19330#bib.bib21)\], Voyager\[[25](https://arxiv.org/html/2605.19330#bib.bib22)\], EUREKA\[[17](https://arxiv.org/html/2605.19330#bib.bib23)\]\) are sample\-expensive, requiring many trajectories to extract a single reusable skill; moreover, skills are tightly coupled to underlying tools, which are finite and costly to develop\. The practical solution is therefore refining how existing tool\-backed skills are described and invoked\. More importantly, when the underlying agent is a closed\-source API model, fine\-tuning\-based approaches are inapplicable; prompt optimization is the only available lever\. MOCHA addresses this setting: given a skill \(whether hand\-authored or discovered\), refine its natural\-language definition across multiple competing objectives without requiring model access\.

Multi\-objective optimization\.Classical MOO methods such as NSGA\-II\[[8](https://arxiv.org/html/2605.19330#bib.bib16)\]and hypervolume\-based algorithms\[[11](https://arxiv.org/html/2605.19330#bib.bib50)\]assume continuous decision spaces with cheap evaluations—neither assumption holds for skill optimization, where the search space is discrete natural language and each evaluation requires an expensive LLM call\. Concurrent work applies multi\-objective preference optimization to LLM alignment\[[33](https://arxiv.org/html/2605.19330#bib.bib35)\], directional scalarization with multi\-objective rewards\[[26](https://arxiv.org/html/2605.19330#bib.bib37)\], and differentiable expected hypervolume improvement to parallel Bayesian optimization\[[7](https://arxiv.org/html/2605.19330#bib.bib36)\]; these methods are orthogonal to MOCHA, as they operate on continuous parameter spaces with gradient access\. Linear scalarization\[[18](https://arxiv.org/html/2605.19330#bib.bib15)\]is the most common multi\-objective reduction but provably misses Pareto\-optimal points in non\-convex regions\. Chebyshev \(ℓ∞\\ell\_\{\\infty\}\) scalarization guarantees access to the full Pareto front\[[18](https://arxiv.org/html/2605.19330#bib.bib15)\], but has not been applied to the*gradient\-free, discrete*setting of natural\-language skill optimization\. MOCHA demonstrates that these principles extend effectively to this challenging regime, combining Chebyshev scalarization with hypervolume\-based exploration and annealed mode switching for structured, sample\-expensive skill refinement\.

## 3Method

Notation\.Letp∈𝒫p\\in\\mathcal\{P\}denote a skill definition,𝒫\\mathcal\{P\}the set of all candidate skill definitions,MMthe number of metrics, andmi​\(p\)∈\[0,1\]m\_\{i\}\(p\)\\in\[0,1\]the value of skillppon metricii, with𝐦​\(p\)=\(m1​\(p\),…,mM​\(p\)\)\\mathbf\{m\}\(p\)=\(m\_\{1\}\(p\),\\ldots,m\_\{M\}\(p\)\)\.

### 3\.1Problem Formulation

Given a task dataset𝒟=\{\(xi,yi\)\}i=1N\\mathcal\{D\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}, a backbone LLMfθf\_\{\\theta\}\(serving all LLM calls in the evaluation pipeline, including the optimizer’s mutation and reflection\), andMMperformance metrics, we seek the set of Pareto\-optimal skill definitions:

𝒫∗=\{p∈𝒫:∄​p′​s\.t\.​𝐦​\(p′\)≻𝐦​\(p\)\}\\mathcal\{P\}^\{\*\}=\\\{p\\in\\mathcal\{P\}:\\nexists p^\{\\prime\}\\text\{ s\.t\. \}\\mathbf\{m\}\(p^\{\\prime\}\)\\succ\\mathbf\{m\}\(p\)\\\}\(1\)where Pareto dominance is defined as\[[10](https://arxiv.org/html/2605.19330#bib.bib49)\]:

𝐦​\(p′\)≻𝐦​\(p\)⇔mj​\(p′\)≥mj​\(p\)​∀j∈\[M\]​and​∃j∈\[M\]:mj​\(p′\)\>mj​\(p\)\\mathbf\{m\}\(p^\{\\prime\}\)\\succ\\mathbf\{m\}\(p\)\\iff m\_\{j\}\(p^\{\\prime\}\)\\geq m\_\{j\}\(p\)\\ \\forall j\\in\[M\]\\text\{ and \}\\exists j\\in\[M\]:m\_\{j\}\(p^\{\\prime\}\)\>m\_\{j\}\(p\)\(2\)Rather than committing to a single optimal skill, we adopt an*a\-posteriori*MOO approach\[[18](https://arxiv.org/html/2605.19330#bib.bib15)\]: the full Pareto front𝒫∗\\mathcal\{P\}^\{\*\}is returned to a human decision maker, who selects the variant that best fits their deployment preferences—prioritizing correctness, compliance, or a balance of both\.

### 3\.2Overview of MOCHA

MOCHA structures each iteration around two stages \([Algorithm1](https://arxiv.org/html/2605.19330#alg1), illustrated in[Figure1](https://arxiv.org/html/2605.19330#S1.F1)b\)\.Stage 1\(lines[5](https://arxiv.org/html/2605.19330#alg1.l5)–[6](https://arxiv.org/html/2605.19330#alg1.l6)\): select a parent via randomized Chebyshev scalarization \([Section3\.2\.1](https://arxiv.org/html/2605.19330#S3.SS2.SSS1)\)—a random weight vector𝐰∼Dirichlet​\(𝟏\)\\mathbf\{w\}\\sim\\mathrm\{Dirichlet\}\(\\mathbf\{1\}\)is drawn and the skill minimizings𝐰s\_\{\\mathbf\{w\}\}is chosen, covering all Pareto front regions including non\-convex pockets\.Stage 2\(lines[8](https://arxiv.org/html/2605.19330#alg1.l8)–[14](https://arxiv.org/html/2605.19330#alg1.l14)\): improve the front via mutation, with the acceptance criterion adapting as optimization progresses\. We define two acceptance modes:

- •*Exploration*\(HVC gating,[Section3\.2\.2](https://arxiv.org/html/2605.19330#S3.SS2.SSS2)\): accept a candidate if it improves the Pareto front inanydirection, irrespective of the𝐰\\mathbf\{w\}used to choose the parent\.
- •*Exploitation*\(Chebyshev acceptance, line[13](https://arxiv.org/html/2605.19330#alg1.l13)\): accept a candidate only if it improves the front in thesame directionas𝐰\\mathbf\{w\}—the direction that selected the parent\.

In theory, Chebyshev acceptance alone suffices given unlimited budget—[Proposition3\.1](https://arxiv.org/html/2605.19330#S3.Thmproposition1)guarantees full Pareto front recovery\[[18](https://arxiv.org/html/2605.19330#bib.bib15)\]\. However, under limited budget only finitely many weight vectors are drawn, so some front regions receive no optimization pressure\. Moreover, the front is initially a single point \(the seed skill\); we want to expand it as quickly as possible in any direction\. HVC measures front improvement directly without relying on fortuitous weight draws\. Once exploration has established multiple points on the front, we want to push it uniformly in all directions\. Chebyshev parent selection \(which targets the weakest region under the drawn𝐰\\mathbf\{w\}\) followed by Chebyshev acceptance \(which requires improvement in that same direction\) provides a coherent “push” that refines the front where it is weakest\. The scheduleτ​\(b\)→0\\tau\(b\)\\to 0\([Section3\.2\.3](https://arxiv.org/html/2605.19330#S3.SS2.SSS3), line[9](https://arxiv.org/html/2605.19330#alg1.l9)\) transitions smoothly between these modes\. Our ablation \([Section4\.3](https://arxiv.org/html/2605.19330#S4.SS3)\) confirms the design: HVC\-only \(exploration\) maximizes front diversity, Chebyshev\-only \(exploitation\) maximizes correctness, and the annealed combination balances both\.

#### 3\.2\.1Chebyshev Scalarization

MOCHA uses Chebyshev scalarization forStage 1: selecting which skill to mutate at each iteration\. Given weight vector𝐰∈ΔM−1\\mathbf\{w\}\\in\\Delta^\{M\-1\}and ideal point𝐳∗=\(1,…,1\)\\mathbf\{z\}^\{\*\}=\(1,\\ldots,1\), Chebyshev scalarization minimizes the worst\-case weighted deviation from the ideal:

s𝐰​\(p\)=maxj∈\[M\]⁡\[wj⋅\|mj​\(p\)−zj∗\|\]s\_\{\\mathbf\{w\}\}\(p\)=\\max\_\{j\\in\[M\]\}\\left\[w\_\{j\}\\cdot\|m\_\{j\}\(p\)\-z\_\{j\}^\{\*\}\|\\right\]\(3\)In words,s𝐰​\(p\)s\_\{\\mathbf\{w\}\}\(p\)is the maximum weighted gap between skillppand the ideal point—the worst\-case cost across objectives\.Lower is better: minimizing this cost focuses optimization on the*weakest*metric, encouraging balanced skill definitions that perform well across all objectives\.

###### Proposition 3\.1\(Chebyshev Completeness\[[18](https://arxiv.org/html/2605.19330#bib.bib15)\]\)\.

For any Pareto\-optimalp∗∈𝒫∗p^\{\*\}\\in\\mathcal\{P\}^\{\*\}, there exists𝐰∗\\mathbf\{w\}^\{\*\}such thatp∗p^\{\*\}minimizess𝐰∗s\_\{\\mathbf\{w\}^\{\*\}\}\. This guarantees access to all Pareto\-optimal solutions—including those in non\-convex regions that linear scalarization \(∑iwi​mi\\sum\_\{i\}w\_\{i\}m\_\{i\}\) cannot reach\.

Parent Selection\.Since MOCHA generates new candidates by mutating an existing skill followingAgrawalet al\.\[[1](https://arxiv.org/html/2605.19330#bib.bib34)\]\(an evolutionary metaphor: the selected skill is the*parent*, its mutation or rewritten prompt by the optimizer is the*offspring*\), we must choose which skill to mutate at each iteration\. We draw𝐰\\mathbf\{w\}uniformly from the weight simplexΔM−1\\Delta^\{M\-1\}\(i\.e\.,𝐰∼Dirichlet​\(𝟏\)\\mathbf\{w\}\\sim\\mathrm\{Dirichlet\}\(\\mathbf\{1\}\)\) and select the parent aspparent=arg⁡minp∈𝒫⁡s𝐰​\(p\)p\_\{\\mathrm\{parent\}\}=\\arg\\min\_\{p\\in\\mathcal\{P\}\}s\_\{\\mathbf\{w\}\}\(p\), i\.e\., the pool member whose worst\-case weighted gap is smallest \(ties are broken randomly\)\. This is the simplest parameter\-free choice: it treats all objectives symmetrically and covers all Pareto front regions with equal probability over time\.

#### 3\.2\.2Hypervolume Contribution for Exploration

As described above, exploration accepts candidates that improve the front in*any*direction—irrespective of the weight𝐰\\mathbf\{w\}used for parent selection\. We need a direction\-agnostic quality measure for this purpose\. We adopt theHypervolume Contribution\(HVC\)\[[34](https://arxiv.org/html/2605.19330#bib.bib47),[11](https://arxiv.org/html/2605.19330#bib.bib50)\]—the only unary quality indicator strictly monotone with Pareto dominance\[[34](https://arxiv.org/html/2605.19330#bib.bib47)\]: if𝒫′\\mathcal\{P\}^\{\\prime\}dominates𝒫\\mathcal\{P\}, thenHV​\(𝒫′\)\>HV​\(𝒫\)\\mathrm\{HV\}\(\\mathcal\{P\}^\{\\prime\}\)\>\\mathrm\{HV\}\(\\mathcal\{P\}\), making it a principled, weight\-free measure of front improvement\. The*hypervolume*of a solution set𝒫\\mathcal\{P\}is the Lebesgue measure \(volume\) of objective space jointly dominated by𝒫\\mathcal\{P\}:

HV​\(𝒫\)=λ​\(⋃p∈𝒫×i=1M\[0,mi​\(p\)\]\)\\mathrm\{HV\}\(\\mathcal\{P\}\)=\\lambda\\\!\\left\(\\bigcup\_\{p\\in\\mathcal\{P\}\}\\bigtimes\_\{i=1\}^\{M\}\[0,m\_\{i\}\(p\)\]\\right\)\(4\)whereλ​\(⋅\)\\lambda\(\\cdot\)denotes the Lebesgue measure and each×i=1M\[0,mi​\(p\)\]\\bigtimes\_\{i=1\}^\{M\}\[0,m\_\{i\}\(p\)\]is the axis\-aligned box from the origin \(reference point\) to the objective vector ofpp\. Intuitively, a larger HV means the set covers more of the achievable trade\-off surface\. The*contribution*of a new candidateppis the exclusive volume it adds—the region it dominates that no existing solution covers:

HVC​\(p,𝒫\)=HV​\(𝒫∪\{p\}\)−HV​\(𝒫\)\\mathrm\{HVC\}\(p,\\mathcal\{P\}\)=\\mathrm\{HV\}\(\\mathcal\{P\}\\cup\\\{p\\\}\)\-\\mathrm\{HV\}\(\\mathcal\{P\}\)\(5\)HVC​\(p,𝒫\)\>0\\mathrm\{HVC\}\(p,\\mathcal\{P\}\)\>0iffppis non\-dominated by any point in𝒫\\mathcal\{P\}, providing a direct signal for Pareto front expansion independent of scalarization weights\. WithM=3M\{=\}3objectives, exact computation is tractable inO​\(n2​log⁡n\)O\(n^\{2\}\\log n\)\[[11](https://arxiv.org/html/2605.19330#bib.bib50)\]\(see[SectionA\.3](https://arxiv.org/html/2605.19330#A1.SS3)for more details\)\.

#### 3\.2\.3Threshold Annealing

MOCHA transitions between the exploration and exploitation modes of Stage 2 via*threshold annealing*:

Accept​\(p\)=\{HVC​\(p,𝒫\)\>τ​\(b\)if​τ​\(b\)\>0\(exploration\)s𝐰​\(p\)<s𝐰​\(pparent\)if​τ​\(b\)≈0\(exploitation\)\\mathrm\{Accept\}\(p\)=\\begin\{cases\}\\mathrm\{HVC\}\(p,\\mathcal\{P\}\)\>\\tau\(b\)&\\text\{if \}\\tau\(b\)\>0\\quad\\text\{\(exploration\)\}\\\\ s\_\{\\mathbf\{w\}\}\(p\)<s\_\{\\mathbf\{w\}\}\(p\_\{\\mathrm\{parent\}\}\)&\\text\{if \}\\tau\(b\)\\approx 0\\quad\\text\{\(exploitation\)\}\\end\{cases\}\(6\)
The thresholdτ​\(b\)\\tau\(b\)decays exponentially with consumed budget:

τ​\(b\)=τend\+\(τ0−τend\)⋅exp⁡\(−λ⋅b/B\)\\tau\(b\)=\\tau\_\{\\mathrm\{end\}\}\+\(\\tau\_\{0\}\-\\tau\_\{\\mathrm\{end\}\}\)\\cdot\\exp\\left\(\-\\lambda\\cdot b/B\\right\)\(7\)wherebbis consumed budget andBBis total budget, andλ\\lambdacontrols the decay rate\. We setλ\\lambdaso thatτ\\taureaches near\-zero around the midpoint of the budget, transitioning the optimizer from exploration to exploitation in the second half \(exact values in[AppendixB](https://arxiv.org/html/2605.19330#A2)\)\. Early in optimization, highτ\\tauactivates HVC\-based acceptance, encouraging diverse Pareto front exploration\. Asτ​\(b\)→0\\tau\(b\)\\to 0, Chebyshev\-based acceptance takes over, refining near\-optimal skill variants\.

Algorithm 1MOCHA: Multi\-Objective Skill Optimization1:Initial skill

p0p\_\{0\}, budget

BB, metrics

𝐦\\mathbf\{m\}, minibatch size

nn, validation set

𝒟val\\mathcal\{D\}\_\{\\mathrm\{val\}\}
2:Initialize pool

𝒫←\{p0\}\\mathcal\{P\}\\leftarrow\\\{p\_\{0\}\\\}, buffer

ℬ←∅\\mathcal\{B\}\\leftarrow\\emptyset\(capacity

KK\), budget

b←0b\\leftarrow 0
3:Evaluate

p0p\_\{0\}on

𝒟val\\mathcal\{D\}\_\{\\mathrm\{val\}\}:

b←b\+\|𝒟val\|b\\leftarrow b\+\|\\mathcal\{D\}\_\{\\mathrm\{val\}\}\|
4:while

b<Bb<Bdo

5:Sample

𝐰\\mathbf\{w\}uniformly from simplex

ΔM−1\\Delta^\{M\-1\}
6:Select parent:

pparent←arg⁡minp∈𝒫⁡s𝐰​\(p\)p\_\{\\mathrm\{parent\}\}\\leftarrow\\arg\\min\_\{p\\in\\mathcal\{P\}\}s\_\{\\mathbf\{w\}\}\(p\)⊳\\trianglerightChebyshev selection

7:Sample minibatch

𝒟mini⊂𝒟train\\mathcal\{D\}\_\{\\mathrm\{mini\}\}\\subset\\mathcal\{D\}\_\{\\mathrm\{train\}\},

\|𝒟mini\|=n\|\\mathcal\{D\}\_\{\\mathrm\{mini\}\}\|=n
8:Evaluate

pparentp\_\{\\mathrm\{parent\}\}, generate candidate

p′p^\{\\prime\}via LLM mutation, evaluate

p′p^\{\\prime\}:

b←b\+2​nb\\leftarrow b\+2n
9:Compute

τ​\(b\)\\tau\(b\)via[Equation7](https://arxiv.org/html/2605.19330#S3.E7)⊳\\trianglerightAnnealed mode switching

10:

⊳\\trianglerightExplore\(

τ​\(b\)\>0\\tau\(b\)\\\!\>\\\!0\):

11:if

HVC​\(p′,𝒫∪ℬ\)\>0\\mathrm\{HVC\}\(p^\{\\prime\},\\mathcal\{P\}\\\!\\cup\\\!\\mathcal\{B\}\)\>0: add

p′p^\{\\prime\}to

ℬ\\mathcal\{B\}\(ranked by HVC, capacityKK\)

12:if

HVC​\(p′,𝒫\)\>τ​\(b\)\\mathrm\{HVC\}\(p^\{\\prime\},\\mathcal\{P\}\)\>\\tau\(b\):

p∗←pop​\_​best​\(ℬ\)p^\{\*\}\\\!\\leftarrow\\\!\\mathrm\{pop\\\_best\}\(\\mathcal\{B\}\); elsecontinue

13:

⊳\\trianglerightExploit\(

τ​\(b\)≈0\\tau\(b\)\\\!\\approx\\\!0\):

p∗←p′p^\{\*\}\\\!\\leftarrow\\\!p^\{\\prime\}if

s𝐰​\(p′\)<s𝐰​\(pparent\)s\_\{\\mathbf\{w\}\}\(p^\{\\prime\}\)<s\_\{\\mathbf\{w\}\}\(p\_\{\\mathrm\{parent\}\}\); elsecontinue

14:Evaluate

p∗p^\{\*\}on

𝒟val\\mathcal\{D\}\_\{\\mathrm\{val\}\}:

b←b\+\|𝒟val\|b\\leftarrow b\+\|\\mathcal\{D\}\_\{\\mathrm\{val\}\}\|;

𝒫←𝒫∪\{p∗\}\\mathcal\{P\}\\leftarrow\\mathcal\{P\}\\cup\\\{p^\{\*\}\\\}
15:endwhile

16:return

𝒫\\mathcal\{P\}

During exploration, we keep a simple priority queueℬ\\mathcal\{B\}of sizeK=5K\{=\}5, ranked by HVC\. Candidates with*any*positive hypervolume contribution enter the queue, but a full validation commit is triggered only when a candidate exceeds the annealing thresholdτ​\(b\)\\tau\(b\)\. At that point, the best candidate fromℬ\\mathcal\{B\}is popped and committed to the pool, ensuring the most promising candidate receives the expensive validation evaluation\. See[AppendixB](https://arxiv.org/html/2605.19330#A2)for details\.

Final Skill Selection\.Over the course of optimization, the skill pool𝒫\\mathcal\{P\}grows from the initial seed\{p0\}\\\{p\_\{0\}\\\}as each accepted candidate is committed \(line[14](https://arxiv.org/html/2605.19330#alg1.l14)\): it is the accumulated set of all validated skill variants, each a distinct point in the objective space \(correctness×\\timesdescription compliance×\\timesbody compliance\)\. After optimization, MOCHA returns this full pool to the practitioner, who selects a deployment variant based on their priorities \(e\.g\., correctness, compliance or balance of both\)\. Additional implementation details \(two\-stage evaluation, HVC computation\) are in[AppendixB](https://arxiv.org/html/2605.19330#A2)\.

#### 3\.2\.4Structured Mutation for Multi\-Field Skills

Skills are multi\-field artifacts; mutations must respect this structure\. We introduce two skill\-aware mutation strategies used within the LLM\-based mutation step \(line 8 of[Algorithm1](https://arxiv.org/html/2605.19330#alg1)\):

Compliance\-aware mutation\.The LLM mutator receives the current SKILL\.md alongside explicit format constraints \(description≤\\leq1,024 chars, body≤\\leq5,000 chars\) and a per\-field compliance status report \(e\.g\.,body: FAIL \(6,412/5,000 chars\)\)\. This biases candidate generation toward the feasible region without altering the selection or acceptance mechanisms\. All methods—TextGrad, ProTeGi, GEPA, and MOCHA—receive this identical mutation prompt; MOCHA’s gains come purely from the candidate*selection*strategy \(full prompt template in[SectionC\.1](https://arxiv.org/html/2605.19330#A3.SS1)\)\.

#### 3\.2\.5Metric Normalization

All objectives are mapped to\[0,1\]\[0,1\]with*higher = better*\. Correctness is the task\-specific metric \(accuracy or F1\) naturally in\[0,1\]\[0,1\]\. Description and body compliance use a linear scoring function:compliance​\(l\)=max⁡\(0,1−l/L\)\\mathrm\{compliance\}\(l\)=\\max\(0,\\;1\-l/L\)wherellis the field length andLLis the limit \(1,0241\{,\}024characters for description,5,0005\{,\}000characters for body\)\. An empty field scores11; a field at the limit \(l=Ll\{=\}L\) scores0; fields exceeding the limit are clamped to0\. The hypervolume reference point is the origin\(0,0,0\)\(0,0,0\)\.

## 4Experiments

### 4\.1Setup

Skill structure\.Each skill follows the SKILL\.md specification adopted by modern agent frameworks\[[2](https://arxiv.org/html/2605.19330#bib.bib27),[28](https://arxiv.org/html/2605.19330#bib.bib21)\]: YAML frontmatter withname\(routing\),description\(skill discovery and documentation\),compatibility\(environment requirements\),metadata, andallowed\-tools, followed by a Markdown instruction body that governs execution\. We initialize each skill with required metadata and optimize the two fields that matter most: thedescription\(≤\\leq1,024 chars\), which co\-resides with other skills in a shared retrieval index and must be concise to compete for limited context; and the instructionbody\(≤\\leq5,000 chars\), which the harness may truncate if verbose\. These two constraints—discovery conciseness and execution brevity—create the multi\-objective tension\.

Skill types\.We evaluate six skills grouped by category\.*Reasoning*: GPQA\[[22](https://arxiv.org/html/2605.19330#bib.bib39)\]\(graduate STEM QA, accuracy\) and TheoremQA\[[5](https://arxiv.org/html/2605.19330#bib.bib40)\]\(mathematical reasoning, accuracy\)\.*Multi\-hop*: HoVer\[[13](https://arxiv.org/html/2605.19330#bib.bib41)\]\(claim verification, accuracy\), HotpotQA\[[30](https://arxiv.org/html/2605.19330#bib.bib42)\]\(question answering, F1\), and FEVER\[[23](https://arxiv.org/html/2605.19330#bib.bib43)\]\(fact verification, accuracy\)\.*Code*: DebugBench\[[24](https://arxiv.org/html/2605.19330#bib.bib29)\]\(code debugging, pass@1\)\. We sample 100 train / 100 val / 100 test examples per benchmark\.

Metrics\.We optimize and report three objectives:Correctness\(↑\\uparrow\): task\-specific accuracy on the held\-out test set\.Description Compliance\(↑\\uparrow\): whether the optimized skill’sdescriptionfield satisfies the≤\\leq1,024 character platform limit\.Body Compliance\(↑\\uparrow\): whether the instruction body satisfies the≤\\leq5,000 character limit\. We additionally reportHypervolume\(HV,↑\\uparrow\): the dominated volume of the discovered Pareto front in the 3D space \(correctness×\\timesdescription compliance×\\timesbody compliance\)\[[11](https://arxiv.org/html/2605.19330#bib.bib50)\]—higher HV indicates both more accurate*and*more diverse skill variants\.

Configuration and budget\.All methods are run with 5 random seeds \(mean±\\pmstd, data is resampled and shuffled across seeds\) for 1000 rollouts \(one rollout = one skill execution \+ metric evaluation\) following the fair\-comparison protocol ofAgrawalet al\.\[[1](https://arxiv.org/html/2605.19330#bib.bib34)\]under a matched budget\. Number of iterations needed for optimization depends on this given budget: per iteration, the budget cost is2​n2nrollouts for the minibatch \(parent \+ candidate\) plus\|𝒟val\|\|\\mathcal\{D\}\_\{\\mathrm\{val\}\}\|rollouts if the candidate is accepted for validation\. We useClaude Haiku 4\.5for skill execution, following the harness–skill evaluation protocol ofLeeet al\.\[[15](https://arxiv.org/html/2605.19330#bib.bib38)\], andClaude Opus 4\.6as the shared reflection and mutation model across all optimizers\.

Baselines\.As discussed in[Section2](https://arxiv.org/html/2605.19330#S2), fine\-tuning is inapplicable for our scope: our setting operates on the*skill definition*axis rather than model weights, and our evaluation backbone \(Claude Haiku 4\.5\) is a closed\-source API with no gradient access, makinggradient\-freeprompt optimization techniques the sole choice\. Among these, propose\-and\-rank methods \(APE\[[32](https://arxiv.org/html/2605.19330#bib.bib4)\], MIPROv2\[[20](https://arxiv.org/html/2605.19330#bib.bib9)\]\) lack an iterative feedback loop—they propose, score, and select without per\-iteration textual critique—and therefore cannot receive multi\-objective compliance feedback\. We therefore compare against the reflection\-based optimizers that iterate via textual feedback: \(1\)ProTeGi\[[21](https://arxiv.org/html/2605.19330#bib.bib3)\]—UCB\-based beam search \(beam width 3,c=2c=\\sqrt\{2\}\) balancing exploration and exploitation across candidate trajectories; \(2\)TextGrad\[[31](https://arxiv.org/html/2605.19330#bib.bib10)\]—greedy selection accepting a candidate only if it improves over the current best; and \(3\)GEPA\[[1](https://arxiv.org/html/2605.19330#bib.bib34)\]—stochastic Pareto\-aware selection over validation datapoints\. All methods share the same SkillMdProposer mutation interface \([SectionC\.1](https://arxiv.org/html/2605.19330#A3.SS1)\), which provides the reflection LM with: \(i\) the current SKILL\.md, \(ii\) a compliance status report, and \(iii\) per\-example correctness feedback\.

### 4\.2Main Results

[Table1](https://arxiv.org/html/2605.19330#S4.T1)reports correctness across all six skills\. The “Seed Skill” column shows performance of the unoptimized initial prompt;shaded cellsindicate methods that failed to improve over this baseline\.

Table 1:Main results: Correctness \(↑\\uparrow\) on Claude Haiku 4\.5 under matched 1000\-rollout budget \(mean±\\pmstd, 5 seeds\)\. Bestbold, secondunderlined\.Shaded: no improvement over seed skill\.![Refer to caption](https://arxiv.org/html/2605.19330v1/x2.png)Figure 2:Optimization dynamicsacross six skills\. Correctness vs\. iteration \(mean±\\pm1 std, 5 seeds\)\. MOCHA \(blue\) consistently improves beyond the initial prompt, while baselines plateau early or remain stuck at the seed skill\. Dashed grey: seed skill performance\.Key findings\.\(1\)Baselines get stuck\.On 4 of 6 tasks \(GPQA, HoVer, FEVER, DebugBench\),*all three baselines*return exactly the seed skill—the red\-shaded cells in[Table1](https://arxiv.org/html/2605.19330#S4.T1)—meaning 1000 rollouts of optimization produced zero improvement \([Figure2](https://arxiv.org/html/2605.19330#S4.F2)\)\. These methods receive the same multi\-objective feedback during mutation \([SectionC\.1](https://arxiv.org/html/2605.19330#A3.SS1)\), yet their single\-objective selection strategies cannot leverage it to escape the initial prompt\. \(2\)MOCHA breaks through\.MOCHA improves on every task, achieving7\.5%relative improvement in mean correctness over the strongest baseline \(ProTeGi\) and 21\.8% over the unoptimized seed\. Gains are largest where baselines are completely stuck:14\.9%on FEVER and8\.3%on DebugBench over the unchanged seed\.[Figure4](https://arxiv.org/html/2605.19330#S4.F4)shows the qualitative difference: MOCHA discovers structured classification rules and step\-by\-step reasoning while baselines return the one\-line seed template unchanged \(full per\-task comparisons in[SectionC\.6](https://arxiv.org/html/2605.19330#A3.SS6)\)\. \(3\)Exploration helps\.On TheoremQA and HotpotQA, baselines do improve over the seed, with ProTeGi’s UCB beam search performing best among them—suggesting that structured exploration is valuable even without multi\-objective selection\. Yet MOCHA still leads on TheoremQA by10\.4%over ProTeGi, demonstrating that principled Pareto exploration compounds on top of single\-objective gains\. \(4\)Low\-conflict tasks reduce selection pressure\.HotpotQA is the only task where a baseline leads \(ProTeGi\.622\.622vs\. MOCHA\.600\.600\), a gap within one standard deviation\. Its seed scores just\.336\.336: our experimental logs show simple formatting instructions \(e\.g\., “answer with just the entity name”, “strip surrounding prose”\) yield a\+66%\+66\\%relative gain in a single iteration \(\.336→\.560\.336\\to\.560\)\. When correctness improves without conflicting with compliance, any selection strategy suffices\. On the four stuck tasks, by contrast, the seed already sits near a local optimum where compliance and correctness are tightly coupled, and only MOCHA’s principled Pareto exploration breaks free\. \(5\)Pareto diversity\.MOCHA discovers2×2\\timesmore Pareto\-optimal skill variants \(3\.63\.6vs\.1\.61\.6\) with\+3\.1%\+3\.1\\%higher 3D HV \([Table2](https://arxiv.org/html/2605.19330#S4.T2)\)\.[Figure3](https://arxiv.org/html/2605.19330#S4.F3)illustrates why: baselines cluster at a single operating point while MOCHA variants span the correctness–compliance frontier\. This pattern is consistent across body, description, and overall compliance views \([SectionC\.3](https://arxiv.org/html/2605.19330#A3.SS3)\)\.

Table 2:Multi\-objective exploration: 3D HV and Pareto front diversity \(mean across 6 skills, 5 seeds\)\. \#PF = Pareto front size\. Full 6\-task visualization in[SectionC\.3](https://arxiv.org/html/2605.19330#A3.SS3)\.![[Uncaptioned image]](https://arxiv.org/html/2605.19330v1/x3.png)

Figure 3:2D Pareto front\(correctness×\\timesbody compliance\): MOCHA \(blue, HV=\.563\) sits balanced between w/o HVC \(exploitation, purple\) and w/o Annealing \(exploration, green\)\. Baselines cluster at a single operating point\. HV values in legend\.

### 4\.3Ablation Study

[Table3](https://arxiv.org/html/2605.19330#S4.T3)reveals an*exploration–exploitation spectrum*across MOCHA variants\. Each row removes one component; the three columns—correctness, hypervolume, Pareto size—quantify the resulting shift along this spectrum\.

Table 3:Ablation: the exploration–exploitation spectrum\.Removing HVC gating pushes toward exploitation \(higher correctness, lower diversity\); removing annealing pushes toward exploration \(more diversity, lower correctness\)\. MOCHA \(full system\) sits at the balanced midpoint\. All values are mean across 6 skills and 5 seeds\.Δ\\Deltacolumns show change vs\. best external baseline \(ProTeGi\)\.Analysis\.\(1\)A clean spectrum emerges\.Removing HVC gating \(w/o HVC\) eliminates the exploration signal, yielding the highest correctness \(\.687\.687,\+5\.9\+5\.9pp over ProTeGi\) but the lowest diversity \(3\.4 Pareto points,\.530\.530HV\)\. Removing annealing \(w/o Annealing\) sustains the HVC exploration signal indefinitely, producing the richest Pareto fronts \(3\.8 points,\.533\.533HV\) but sacrificing correctness \(\.671\.671\)\. MOCHA balances these forces: its annealed threshold transitions from HVC\-driven exploration to Chebyshev\-driven exploitation, achieving strong correctness \(\.675\.675,\+4\.7\+4\.7pp\) with diverse Pareto fronts \(3\.6 points,\.531\.531HV\)\. \(2\)Every MOCHA variant dominates every baseline\.Even the weakest ablation \(w/o Annealing,\.671\.671\) outperforms the strongest baseline \(ProTeGi,\.628\.628\) by\+4\.3\+4\.3pp—a gap5×5\\timeslarger than the spread among baselines themselves \(\.619\.619–\.628\.628\)\. This confirms that the multi\-objective selection framework, not any single component, drives the improvement \(3\)Practitioners choose their operating point\.The modular design means users who prioritize raw accuracy can disable HVC gating; those who need diverse operating points for downstream selection can disable annealing\. MOCHA provides the recommended default for balanced operation\.

FEVER Claim Verification — SKILL\.md\-\-\-
name:fever\_verificationcompatibility:dspy\>=2\.5, python\>=3\.10
metadata:task: fever\_singlecall, modules: 1allowed\-tools:\[retriever\]
\-\-\-
description:Claim verification with retrieval tool✓38/1,024 chars→\\todesc = 0\.96
\# execute\.predict
Given the fields ‘claim’, ‘evidence’, produce the fields ‘verdict’\.✓67/5,000 chars→\\tobody = 0\.99
Baselines: Corr\. = 0\.63 — desc = 0\.96 — body = 0\.99→\\tocomposite = 0\.88\. No mutation improves all three; single\-objective selectors reject every candidate\.
description:FEVER\-style 3\-class claim verification: classify as SUPPORTS,REFUTES, or NOT ENOUGH INFO based on evidence passages\.✓137/1,024 chars→\\todesc = 0\.77
\# execute\.predict
Given ‘claim’ and ‘evidence’, produce ‘verdict’\. Must be: SUPPORTS, REFUTES, orNOT ENOUGH INFO\.Rule 1: Strict evidence\-only reasoning\. No outside knowledge\.Rule 2: ALL claim parts must be supported for SUPPORTS\.Rule 3: REFUTES requires direct contradiction only\.Rule 4: NOT ENOUGH INFO is default when evidence is partial or about related entities\.Rule 5: Never invent labels\.Step\-by\-step: \(1\) Decompose claim→\\to\(2\) Check each assertion→\\to\(3\) Check contradiction→\\to\(4\) Aggregate✓2,426/5,000 chars→\\tobody = 0\.38
MOCHA: Corr\. = 0\.70 — desc = 0\.77 — body = 0\.38\. MOCHA accepts the \+11% correctness gain despite compliance trade\-off\. \(Body truncated for display; full skill is 39 lines\.\)Figure 4:FEVER qualitative comparison\.Grey: shared YAML fields\.Red: baseline skill \(all three baselines returned the seed template unchanged\)\.Green: MOCHA\-optimized skill with structured rules and explicit reasoning\. Per\-task comparisons in[SectionC\.6](https://arxiv.org/html/2605.19330#A3.SS6)\.

## 5Discussion and Conclusion

When does MOCHA help?MOCHA’s gains scale with*objective conflict*\. On FEVER \(14\.9% relative gain\) and TheoremQA \(10\.4%\), improving correctness requires longer instructions that push against body token limits— MOCHA navigates this non\-convex trade\-off\. The core finding is stark: on 4 of 6 tasks, all three baselines return the seed skill unchanged after 1000 rollouts—single\-objective selection strategies simply cannot escape the initial prompt, even when given the same multi\-objective feedback during mutation\. MOCHA breaks through by exploring the trade\-off surface, discovering2×2\\timesmore non\-dominated skill variants \(3\.63\.6Pareto points vs\.1\.61\.6\)\. The ablation \([Table3](https://arxiv.org/html/2605.19330#S4.T3)\) further confirms this: removing HVC gating favors exploitation \(highest correctness\); removing annealing favors exploration \(richest Pareto fronts\); full MOCHA balances both\.

From skills to harnesses\.Our design fixes the execution pipeline and varies only the skill specification, isolating each optimizer’s selection strategy\. The multi\-objective machinery is not skill\-specific: applying MOCHA to*meta\-harness*optimization\[[15](https://arxiv.org/html/2605.19330#bib.bib38)\], where the pipeline structure itself is the search target, is a natural extension\.

Limitations\.\(1\)*Low\-conflict tasks\.*When objectives do not conflict \(e\.g\., HotpotQA\), MOCHA reduces to an expensive alternative to single\-objective methods—detecting such cases automatically remains open\. \(2\)*Fixed annealing schedule\.*The exponential decay is a hyperparameter; adaptive schedules that respond to optimization progress could improve robustness\. \(3\)*Platform\-specific compliance\.*Compliance metrics are tied to one platform’s SKILL\.md spec \(i\.e\., Anthropic’s SKILL\.md specification\); different constraint schemas may shift the trade\-off landscape\.

Conclusion\.Skill optimization is inherently multi\-objective: SKILL\.md constraints on description size and body tokens create trade\-offs invisible to single\-objective optimizers\. Our central finding is that*existing optimizers fail to make any progress*on 4 of 6 tasks—1000 rollouts yield zero improvement over the seed skill\. MOCHA’s Chebyshev scalarization breaks through this barrier, achieving7\.5%relative improvement in mean correctness over the strongest baseline, with gains up to14\.9%on FEVER and10\.4%on TheoremQA where objective conflict is strongest\. Looking ahead, adaptive annealing, meta\-harness optimization\[[15](https://arxiv.org/html/2605.19330#bib.bib38)\], and integration with skill discovery\[[28](https://arxiv.org/html/2605.19330#bib.bib21),[25](https://arxiv.org/html/2605.19330#bib.bib22)\]form a path toward end\-to\-end agent skill evolution\.

## References

- \[1\]L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang, C\. Potts, K\. Sen, A\. G\. Dimakis, D\. Klein, I\. Stoica, M\. Zaharia, and O\. Khattab\(2026\)GEPA: reflective prompt evolution can outperform reinforcement learning\.InICLR,Cited by:[§A\.4](https://arxiv.org/html/2605.19330#A1.SS4.p1.1),[§1](https://arxiv.org/html/2605.19330#S1.p3.1),[§2](https://arxiv.org/html/2605.19330#S2.p2.1),[§3\.2\.1](https://arxiv.org/html/2605.19330#S3.SS2.SSS1.p2.4),[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p4.3),[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p5.1)\.
- \[2\]Extend claude with skills\.Note:[https://code\.claude\.com/docs/en/skills](https://code.claude.com/docs/en/skills)Accessed: 2026\-04\-25Cited by:[§1](https://arxiv.org/html/2605.19330#S1.p2.1),[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p1.2)\.
- \[3\]K\. Bringmann and T\. Friedrich\(2013\)Approximation quality of the hypervolume indicator\.Artificial Intelligence195,pp\. 265–290\.Cited by:[2nd item](https://arxiv.org/html/2605.19330#A1.I1.i2.p1.4)\.
- \[4\]T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.\(2020\)Language models are few\-shot learners\.InNeurIPS,Vol\.33,pp\. 1877–1901\.Cited by:[§1](https://arxiv.org/html/2605.19330#S1.p1.1)\.
- \[5\]W\. Chen, M\. Yin, M\. Ku, P\. Lu, Y\. Wan, X\. Ma, J\. Xu, X\. Wang, and T\. Xia\(2023\)TheoremQA: a theorem\-driven question answering dataset\.InEMNLP,pp\. 7889–7901\.Cited by:[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p2.1)\.
- \[6\]C\. Cheng, A\. Nie, and A\. Swaminathan\(2024\)Trace is the next autodiff: generative optimization with rich feedback, execution traces, and LLMs\.arXiv preprint arXiv:2406\.16218\.Cited by:[§2](https://arxiv.org/html/2605.19330#S2.p2.1)\.
- \[7\]S\. Daulton, M\. Balandat, and E\. Bakshy\(2020\)Differentiable expected hypervolume improvement for parallel multi\-objective Bayesian optimization\.InNeurIPS,Vol\.33,pp\. 9851–9864\.Cited by:[§2](https://arxiv.org/html/2605.19330#S2.p4.1)\.
- \[8\]K\. Deb, A\. Pratap, S\. Agarwal, and T\. Meyarivan\(2002\)A fast and elitist multiobjective genetic algorithm: nsga\-ii\.IEEE Transactions on Evolutionary Computation6\(2\),pp\. 182–197\.Cited by:[§2](https://arxiv.org/html/2605.19330#S2.p4.1)\.
- \[9\]M\. Deng, J\. Wang, C\. Hsieh, Y\. Wang, H\. Guo, T\. Shu, M\. Song, E\. P\. Xing, and Z\. Hu\(2022\)RLPrompt: optimizing discrete text prompts with reinforcement learning\.InEMNLP,Cited by:[§2](https://arxiv.org/html/2605.19330#S2.p2.1)\.
- \[10\]M\. T\. M\. Emmerich and A\. H\. Deutz\(2018\)A tutorial on multiobjective optimization: fundamentals and evolutionary methods\.Natural Computing17\(3\),pp\. 585–609\.Cited by:[§A\.1](https://arxiv.org/html/2605.19330#A1.SS1.p1.4),[§3\.1](https://arxiv.org/html/2605.19330#S3.SS1.p1.5)\.
- \[11\]A\. P\. Guerreiro, C\. M\. Fonseca, and L\. Paquete\(2021\)The hypervolume indicator: problems and algorithms\.ACM Computing Surveys54\(6\),pp\. 1–42\.Cited by:[2nd item](https://arxiv.org/html/2605.19330#A1.I1.i2.p1.4),[§A\.3](https://arxiv.org/html/2605.19330#A1.SS3.p1.1),[§A\.3](https://arxiv.org/html/2605.19330#A1.SS3.p3.2),[Appendix B](https://arxiv.org/html/2605.19330#A2.p3.2),[§2](https://arxiv.org/html/2605.19330#S2.p4.1),[§3\.2\.2](https://arxiv.org/html/2605.19330#S3.SS2.SSS2.p1.15),[§3\.2\.2](https://arxiv.org/html/2605.19330#S3.SS2.SSS2.p1.6),[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p3.8)\.
- \[12\]Q\. Guo, R\. Wang, J\. Guo, B\. Li, K\. Song, X\. Tan, G\. Liu, J\. Bian, and Y\. Yang\(2024\)Connecting large language models with evolutionary algorithms yields powerful prompt optimizers\.arXiv preprint arXiv:2309\.08532\.Cited by:[§2](https://arxiv.org/html/2605.19330#S2.p2.1)\.
- \[13\]Y\. Jiang, S\. Bordia, Z\. Zhong, C\. Dognin, M\. Singh, and M\. Bansal\(2020\)HoVer: a dataset for many\-hop fact extraction and claim verification\.InFindings of EMNLP,Cited by:[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p2.1)\.
- \[14\]O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. Vardhamanan, S\. Haq, A\. Sharma, T\. T\. Joshi, H\. Moazam, H\. Miller, M\. Zaharia, and C\. Potts\(2024\)DSPy: compiling declarative language model calls into self\-improving pipelines\.InICLR,Cited by:[§1](https://arxiv.org/html/2605.19330#S1.p2.1)\.
- \[15\]Y\. Lee, R\. Nair, Q\. Zhang, K\. Lee, O\. Khattab, and C\. Finn\(2026\)Meta\-harness: end\-to\-end optimization of model harnesses\.arXiv preprint arXiv:2603\.28052\.Cited by:[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p4.3),[§5](https://arxiv.org/html/2605.19330#S5.p2.1),[§5](https://arxiv.org/html/2605.19330#S5.p4.1)\.
- \[16\]X\. Lin, X\. Zhang, Z\. Yang, F\. Liu, Z\. Wang, and Q\. Zhang\(2024\)Smooth tchebycheff scalarization for multi\-objective optimization\.InICML,Cited by:[2nd item](https://arxiv.org/html/2605.19330#S1.I1.i2.p1.1)\.
- \[17\]Y\. J\. Ma, W\. Liang, G\. Wang, D\. Huang, O\. Bastani, D\. Jayaraman, Y\. Zhu, L\. Fan, and A\. Anandkumar\(2024\)Eureka: human\-level reward design via coding large language models\.InICLR,Cited by:[§2](https://arxiv.org/html/2605.19330#S2.p3.1)\.
- \[18\]K\. Miettinen\(1999\)Nonlinear multiobjective optimization\.Springer,Boston, MA\.Cited by:[§A\.2](https://arxiv.org/html/2605.19330#A1.SS2.p1.2),[2nd item](https://arxiv.org/html/2605.19330#S1.I1.i2.p1.1),[§1](https://arxiv.org/html/2605.19330#S1.p4.1),[§2](https://arxiv.org/html/2605.19330#S2.p4.1),[§3\.1](https://arxiv.org/html/2605.19330#S3.SS1.p1.4),[§3\.2](https://arxiv.org/html/2605.19330#S3.SS2.p1.4),[Proposition 3\.1](https://arxiv.org/html/2605.19330#S3.Thmproposition1)\.
- \[19\]S\. Mukherjee, A\. Lalitha, S\. Sengupta, A\. Deshmukh, and B\. Kveton\(2024\)Multi\-objective alignment of large language models through hypervolume maximization\.arXiv preprint arXiv:2412\.05469\.Cited by:[2nd item](https://arxiv.org/html/2605.19330#S1.I1.i2.p1.1)\.
- \[20\]K\. Opsahl\-Ong, M\. J\. Ryan, J\. Purtell, D\. Broman, C\. Potts, M\. Zaharia, and O\. Khattab\(2024\)Optimizing instructions and demonstrations for multi\-stage language model programs\.InEMNLP,Cited by:[§1](https://arxiv.org/html/2605.19330#S1.p2.1),[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p5.1)\.
- \[21\]R\. Pryzant, D\. Iter, J\. Li, Y\. T\. Lee, C\. Zhu, and M\. Zeng\(2023\)Automatic prompt optimization with “gradient descent” and beam search\.InEMNLP,Cited by:[§1](https://arxiv.org/html/2605.19330#S1.p2.1),[§1](https://arxiv.org/html/2605.19330#S1.p3.1),[§2](https://arxiv.org/html/2605.19330#S2.p2.1),[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p5.1)\.
- \[22\]D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. Bowman\(2023\)GPQA: a graduate\-level google\-proof q&a benchmark\.arXiv preprint arXiv:2311\.12022\.Cited by:[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p2.1)\.
- \[23\]J\. Thorne, A\. Vlachos, C\. Christodoulopoulos, and A\. Mittal\(2018\)FEVER: a large\-scale dataset for fact extraction and VERification\.InNAACL\-HLT,pp\. 809–819\.Cited by:[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p2.1)\.
- \[24\]R\. Tian, Y\. Ye, Y\. Qin, X\. Cong, Y\. Lin, Y\. Pan, Y\. Wu, H\. Hui, W\. Liu, Z\. Liu, and M\. Sun\(2024\)DebugBench: evaluating debugging capability of large language models\.InFindings of ACL,pp\. 4173–4198\.Cited by:[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p2.1)\.
- \[25\]G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. Anandkumar\(2023\)Voyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§1](https://arxiv.org/html/2605.19330#S1.p1.1),[§2](https://arxiv.org/html/2605.19330#S2.p3.1),[§5](https://arxiv.org/html/2605.19330#S5.p4.1)\.
- \[26\]H\. Wang, Y\. Lin, W\. Xiong, R\. Yang, S\. Diao, S\. Qiu, H\. Zhao, and T\. Zhang\(2024\)Arithmetic control of LLMs for diverse user preferences: directional preference alignment with multi\-objective rewards\.arXiv preprint arXiv:2402\.18571\.Cited by:[§2](https://arxiv.org/html/2605.19330#S2.p4.1)\.
- \[27\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InNeurIPS,Vol\.35,pp\. 24824–24837\.Cited by:[§1](https://arxiv.org/html/2605.19330#S1.p1.1)\.
- \[28\]P\. Xia, J\. Chen, H\. Wang, J\. Liu, K\. Zeng, Y\. Wang, S\. Han, Y\. Zhou, X\. Zhao, H\. Chen, Z\. Zheng, C\. Xie, and H\. Yao\(2026\)SkillRL: evolving agents via recursive skill\-augmented reinforcement learning\.arXiv preprint arXiv:2602\.08234\.Cited by:[§1](https://arxiv.org/html/2605.19330#S1.p1.1),[§1](https://arxiv.org/html/2605.19330#S1.p2.1),[§2](https://arxiv.org/html/2605.19330#S2.p3.1),[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p1.2),[§5](https://arxiv.org/html/2605.19330#S5.p4.1)\.
- \[29\]C\. Yang, X\. Wang, Y\. Lu, H\. Liu, Q\. V\. Le, D\. Zhou, and X\. Chen\(2024\)Large language models as optimizers\.InICLR,Cited by:[§2](https://arxiv.org/html/2605.19330#S2.p2.1)\.
- \[30\]Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning\(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InEMNLP,pp\. 2369–2380\.Cited by:[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p2.1)\.
- \[31\]M\. Yüksekgönül, F\. Bianchi, J\. Boen, S\. Liu, Z\. Huang, C\. Guestrin, and J\. Zou\(2024\)TextGrad: automatic “differentiation” via text\.arXiv preprint arXiv:2406\.07496\.Cited by:[§1](https://arxiv.org/html/2605.19330#S1.p3.1),[§2](https://arxiv.org/html/2605.19330#S2.p2.1),[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p5.1)\.
- \[32\]Y\. Zhou, A\. I\. Muresanu, Z\. Han, K\. Paster, S\. Pitis, H\. Chan, and J\. Ba\(2023\)Large language models are human\-level prompt engineers\.InICLR,Cited by:[§1](https://arxiv.org/html/2605.19330#S1.p2.1),[§2](https://arxiv.org/html/2605.19330#S2.p2.1),[§4\.1](https://arxiv.org/html/2605.19330#S4.SS1.p5.1)\.
- \[33\]Z\. Zhou, J\. Liu, J\. Shao, X\. Yue, C\. Yang, W\. Ouyang, and Y\. Qiao\(2024\)Beyond one\-preference\-fits\-all alignment: multi\-objective direct preference optimization\.InFindings of ACL,Cited by:[§2](https://arxiv.org/html/2605.19330#S2.p4.1)\.
- \[34\]E\. Zitzler, L\. Thiele, M\. Laumanns, C\. M\. Fonseca, and V\. G\. Da Fonseca\(2003\)Performance assessment of multiobjective optimizers: an analysis and review\.IEEE Transactions on Evolutionary Computation7\(2\),pp\. 117–132\.Cited by:[1st item](https://arxiv.org/html/2605.19330#A1.I1.i1.p1.3),[§3\.2\.2](https://arxiv.org/html/2605.19330#S3.SS2.SSS2.p1.6)\.
- \[35\]E\. Zitzler and L\. Thiele\(1999\)Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach\.IEEE Transactions on Evolutionary Computation3\(4\),pp\. 257–271\.Cited by:[§A\.3](https://arxiv.org/html/2605.19330#A1.SS3.p1.1)\.

## Appendix ABackground: Scalarization and Hypervolume Theory

We provide extended background on the theoretical foundations underlying MOCHA\.

### A\.1Multi\-Objective Optimization

The fundamental challenge: no single solutionp∗∈𝒫p\_\{\\ast\}\\in\\mathcal\{P\}satisfiesmi​\(p∗\)≥mi​\(p\)m\_\{i\}\(p\_\{\\ast\}\)\\geq m\_\{i\}\(p\)for allp∈𝒫p\\in\\mathcal\{P\}and all metricsi∈\[M\]i\\in\[M\]simultaneously\. Two paradigms exist\[[10](https://arxiv.org/html/2605.19330#bib.bib49)\]:*a\-priori*methods, where the decision maker’s utility is known in advance, and*a\-posteriori*methods, which learn the full Pareto front for post\-hoc selection\. Ours fall in*a\-posteriori*methods\.

### A\.2Linear vs\. Chebyshev Scalarization

*Scalarization*reduces multi\-objective optimization to single\-objective via a weight vector𝐰\\mathbf\{w\}:

Linear:s𝐰lin​\(p\)\\displaystyle\\text\{Linear:\}\\quad s\_\{\\mathbf\{w\}\}^\{\\mathrm\{lin\}\}\(p\)=∑i=1Mwi​mi​\(p\)\\displaystyle=\\textstyle\\sum\_\{i=1\}^\{M\}w\_\{i\}\\,m\_\{i\}\(p\)\(8\)Chebyshev:s𝐰​\(p\)\\displaystyle\\text\{Chebyshev:\}\\quad s\_\{\\mathbf\{w\}\}\(p\)=maxj∈\[M\]⁡\[wj⋅\|mj​\(p\)−zj∗\|\]\\displaystyle=\\max\_\{j\\in\[M\]\}\\left\[w\_\{j\}\\cdot\|m\_\{j\}\(p\)\-z\_\{j\}^\{\*\}\|\\right\]\(9\)Linear scalarization finds the skill maximizing the weighted sum\. Chebyshev finds the skill with the best worst\-case weighted deviation from the ideal point𝐳∗=\(1,…,1\)\\mathbf\{z\}^\{\*\}=\(1,\\ldots,1\)\. The critical distinction: linear scalarization provably misses Pareto\-optimal points in non\-convex regions of the objective space, while Chebyshev can reach*every*Pareto\-optimal solution\[[18](https://arxiv.org/html/2605.19330#bib.bib15)\]\.

For a\-posteriori exploration, we sample𝐰\\mathbf\{w\}uniformly from the weight simplexΔM−1\\Delta^\{M\-1\}\(i\.e\.,𝐰∼Dirichlet​\(𝟏\)\\mathbf\{w\}\\sim\\mathrm\{Dirichlet\}\(\\mathbf\{1\}\), all concentration parameters equal to 1\)\. This parameter\-free choice treats all objectives symmetrically and visits every Pareto front region with equal probability across iterations\.

### A\.3Hypervolume Indicator

The hypervolume indicator\[[11](https://arxiv.org/html/2605.19330#bib.bib50)\]\(also called theSS\-metric\[[35](https://arxiv.org/html/2605.19330#bib.bib17)\]\) is formally defined as follows\.

Definition\.Given a solution set𝒫⊂ℝM\\mathcal\{P\}\\subset\\mathbb\{R\}^\{M\}and a reference pointr∈ℝMr\\in\\mathbb\{R\}^\{M\}, the hypervolume indicator is the Lebesgue measure of the region in objective space weakly dominated by𝒫\\mathcal\{P\}and bounded byrr:

HV\(𝒫\)=λ\(\{q∈ℝM\|∃p∈𝒫:ri≤qi≤mi\(p\),∀i\}\)\\displaystyle\\mathrm\{HV\}\(\\mathcal\{P\}\)=\\lambda\\\!\\left\(\\left\\\{q\\in\\mathbb\{R\}^\{M\}\\;\\middle\|\\;\\exists\\,p\\in\\mathcal\{P\}:r\_\{i\}\\leq q\_\{i\}\\leq m\_\{i\}\(p\),\\;\\forall\\,i\\right\\\}\\right\)\(10\)whereλ​\(⋅\)\\lambda\(\\cdot\)is the Lebesgue measure andmi​\(p\)m\_\{i\}\(p\)is theii\-th objective value of solutionpp\. Equivalently, this is the volume of the union of axis\-aligned boxes fromrrto each solution:

HV​\(𝒫\)=λ​\(⋃p∈𝒫×i=1M\[ri,mi​\(p\)\]\)\\displaystyle\\mathrm\{HV\}\(\\mathcal\{P\}\)=\\lambda\\\!\\left\(\\bigcup\_\{p\\in\\mathcal\{P\}\}\\bigtimes\_\{i=1\}^\{M\}\[r\_\{i\},\\;m\_\{i\}\(p\)\]\\right\)\(11\)In MOCHA, all objectives \(correctness, description compliance, body compliance\) are non\-negative and maximized, with the reference point at the originr=𝟎r=\\mathbf\{0\}\. This yields the compact form used in the main text \([Equation4](https://arxiv.org/html/2605.19330#S3.E4)\)\.

Hypervolume Contribution\.The contribution of a pointppto a set𝒫\\mathcal\{P\}\[[11](https://arxiv.org/html/2605.19330#bib.bib50)\]is:

HVC​\(p,𝒫\)=HV​\(𝒫∪\{p\}\)−HV​\(𝒫\)\\displaystyle\\mathrm\{HVC\}\(p,\\mathcal\{P\}\)=\\mathrm\{HV\}\(\\mathcal\{P\}\\cup\\\{p\\\}\)\-\\mathrm\{HV\}\(\\mathcal\{P\}\)\(12\)HVC​\(p,𝒫\)\>0\\mathrm\{HVC\}\(p,\\mathcal\{P\}\)\>0if and only ifppis non\-dominated by any member of𝒫\\mathcal\{P\}\.

Key properties\.

- •*Strict monotonicity*: If𝒫′\\mathcal\{P\}^\{\\prime\}Pareto\-dominates𝒫\\mathcal\{P\}, thenHV​\(𝒫′\)\>HV​\(𝒫\)\\mathrm\{HV\}\(\\mathcal\{P\}^\{\\prime\}\)\>\\mathrm\{HV\}\(\\mathcal\{P\}\)\. Hypervolume is the only known unary indicator with this property\[[34](https://arxiv.org/html/2605.19330#bib.bib47)\]\.
- •*Computation*: Exact HV is NP\-hard for generalMM\[[3](https://arxiv.org/html/2605.19330#bib.bib48)\], but tractable for smallMM\. WithM=3M\{=\}3in our setting, we use the exactO​\(n2​log⁡n\)O\(n^\{2\}\\log n\)algorithm\[[11](https://arxiv.org/html/2605.19330#bib.bib50)\]\.
- •*Complementarity with Chebyshev scalarization*: Chebyshev scalarization targets a specific Pareto\-optimal point for a given weight vector𝐰\\mathbf\{w\}, while HVC measures the total new volume a candidate contributes regardless of direction\. The two mechanisms are complementary: Chebyshev exploitation refines the worst\-case objective along a chosen direction, while HVC exploration rewards candidates that expand the front in*any*under\-covered region\. This motivates their combination in MOCHA’s annealed two\-phase strategy\.

### A\.4GEPA Framework

GEPA\[[1](https://arxiv.org/html/2605.19330#bib.bib34)\]optimizes skill definitions through iterative evolution: evaluate candidates on a validation subset, estimate gradients via LLM feedback, select promising candidates, and generate mutations\. MOCHA replaces GEPA’s heuristic candidate selection with principled multi\-objective mechanisms while retaining its mutation and evaluation infrastructure\.

## Appendix BImplementation Details

Two\-Stage Evaluation\.MOCHA uses a two\-stage strategy: \(1\)*Minibatch gating*: parent and candidate are evaluated on a small training minibatch \(nnsamples\); acceptance criteria \(HVC or Chebyshev\) is applied to minibatch scores, filtering poor candidates cheaply\. \(2\)*Validation scoring*: accepted candidates are evaluated on the full validation set and unconditionally committed to𝒫\\mathcal\{P\}\. Validation scores are used for parent selection in subsequent iterations, providing reliable signal for Pareto front navigation\.

Budget Accounting\.BudgetBBcounts individual evaluations\. Per iteration:Δ​b=2​n\+\|𝒟val\|⋅𝟏​\[commit\]\\Delta b=2n\+\|\\mathcal\{D\}\_\{\\mathrm\{val\}\}\|\\cdot\\mathbf\{1\}\[\\text\{commit\}\]\.

HVC Computation\.We useM=3M=3metrics throughout \(correctness, description compliance, body compliance\)\. HVC is computed via the exact HSO algorithm \(O​\(n2​log⁡n\)O\(n^\{2\}\\log n\)\)\[[11](https://arxiv.org/html/2605.19330#bib.bib50)\]\.

Speculative Buffer\.During exploration \(τ​\(b\)\>0\\tau\(b\)\>0\), a priority queueℬ\\mathcal\{B\}\(capacity 5\) stores non\-dominated candidates ranked by HVC\. When a candidate’s HVC exceedsτ​\(b\)\\tau\(b\), the best candidate fromℬ\\mathcal\{B\}is committed\. This prevents premature commitment to marginal candidates while ensuring the most impactful discovery is selected\.

Annealing Hyperparameters\.We setτ0=0\.1\\tau\_\{0\}=0\.1,τend=0\.0\\tau\_\{\\mathrm\{end\}\}=0\.0, andλ=10\\lambda=10in[Equation7](https://arxiv.org/html/2605.19330#S3.E7)\. With these values,τ​\(B/2\)=0\.1⋅e−5≈0\.0007\\tau\(B/2\)=0\.1\\cdot e^\{\-5\}\\approx 0\.0007, so the threshold is effectively zero by mid\-budget\.

Unified Optimization Framework\.We reimplement TextGrad and ProTeGi within our unified optimization framework, ensuring identical meta prompts, evaluation harness, and rollout budget across all methods\. This unified implementation isolates the effect of selection strategy as the sole independent variable: TextGrad uses greedy acceptance, ProTeGi uses UCB beam search, GEPA uses stochastic Pareto selection, and MOCHA uses Chebyshev scalarization with threshold annealing\. Using the original codebases would introduce confounds \(different prompt templates, evaluation code, rollout accounting\); the unified framework makes the comparison*more*fair—a reviewer cannot attribute differences to implementation artifacts\.

## Appendix CAdditional Experimental Results

### C\.1Shared Mutation Interface

A critical design decision is that*all methods share the same mutation interface*\. The completeSkillMdProposerprompt, used identically by TextGrad, ProTeGi, GEPA, and MOCHA, is shown below\.

Complete SkillMdProposer Prompt Template\(identical for all methods\)You are optimizing a SKILL\.md specification for a language model skill\. \#\# SKILL\.md Format Constraints \(all fields are optimization targets\) \- description:≤\\leq1,024 characters \- body:≤\\leq5,000 characters \- All YAML frontmatter fields must remain valid YAML \#\# Current SKILL\.md \`\`\` \{current\_skill\_md\} \`\`\` \#\# Compliance Status \{compliance\_report\} \#\# Task Examples with Feedback \{feedback\_text\} \#\# Sections to Improve:\{components\_to\_update\} Rewrite the SKILL\.md: TOP priority is to improve task accuracy by adding as many steps or instructions as necessary while still respecting all field constraints\. You may modify ANY field \(description, body sections\)\. Maintain valid YAML frontmatter and \`\# section\_name\` headers in the body\. Return the complete SKILL\.md within \`\`\` blocks\.

Template variables\.\{compliance\_report\}is a per\-field PASS/FAIL status with current and allowed lengths \(e\.g\.,body: FAIL \(6,412/5,000 chars\)\)\.\{feedback\_text\}contains task\-specific per\-example correctness feedback, e\.g\.,"Correct\! Verdict is \{expected\}\."or"Incorrect\. Expected ’\{expected\}’, got ’\{predicted\}’\."for fact verification tasks\.\{components\_to\_update\}lists the SKILL\.md sections that the optimizer has flagged for revision\.

Baselines receive the same multi\-objective feedback during mutation: compliance constraints and per\-example correctness signals are available to*every*method\. MOCHA’s gains come purely from the candidate*selection*strategy, not from privileged mutation feedback\.

### C\.2Per\-Task Compliance Analysis

[Table4](https://arxiv.org/html/2605.19330#A3.T4)reports description and body compliance for all methods across the six skills\.

Table 4:Compliance\(mean across 5 seeds\)\. Desc\. = description≤\\leq1,024 chars; Body = instruction body≤\\leq5,000 chars\.Baselines maintain high compliance because their selection strategies rarely accept candidates that deviate from the initial SKILL\.md template\. MOCHA trades some compliance for correctness—the multi\-objective machinery makes this trade\-off explicit and navigable rather than hidden\.

### C\.3Full Pareto Front Visualization

[Figures5](https://arxiv.org/html/2605.19330#A3.F5),[6](https://arxiv.org/html/2605.19330#A3.F6)and[7](https://arxiv.org/html/2605.19330#A3.F7)extend the 2\-task visualization in[Figure3](https://arxiv.org/html/2605.19330#S4.F3)to all six skills across three compliance views: body compliance, description compliance, and overall \(average\) compliance\. The exploration–exploitation spectrum observed: w/o Annealing \(always\-on HVC\) produces the most candidates, w/o HVC \(Chebyshev only\) pushes furthest on correctness, and MOCHA \(full\) balances both\. Baselines in most cases cluster at a single operating point\.

![Refer to caption](https://arxiv.org/html/2605.19330v1/x4.png)Figure 5:2D Pareto fronts\(correctness×\\timesbody compliance\) for all six skills\. Three baselines \(TextGrad, ProTeGi, GEPA\) and three MOCHA variants are shown\. Shaded regions indicate dominated hypervolume\. MOCHA variants consistently explore multiple non\-dominated operating points while baselines remain near the initial prompt\.![Refer to caption](https://arxiv.org/html/2605.19330v1/x5.png)Figure 6:2D Pareto fronts\(correctness×\\timesdescription compliance\) for all six skills\. The same pattern holds: MOCHA discovers diverse non\-dominated skill variants spanning the correctness–description compliance frontier, while baselines cluster at a single operating point\.![Refer to caption](https://arxiv.org/html/2605.19330v1/x6.png)Figure 7:2D Pareto fronts\(correctness×\\timesoverall compliance, i\.e\., average of body and description compliance\) for all six skills\. The pattern is consistent across all three compliance views: MOCHA’s multi\-objective selection enables Pareto front exploration that single\-objective baselines cannot achieve\.
### C\.4Convergence Curves

See[Figure2](https://arxiv.org/html/2605.19330#S4.F2)in the main text for optimization dynamics across all six skills\.

### C\.5Prompt Evolution Trees

[Figure8](https://arxiv.org/html/2605.19330#A3.F8)visualizes the prompt evolution structure for MOCHA across all six skills\. Each node is a committed skill variant; edges trace parent–child mutation relationships\. The blue node marks the best test correctness; metric breakdowns \(C=correctness, D=desc\_compliance, B=body\_compliance\) annotate the root and best nodes\. MOCHA explores multiple branches from the baseline, with the best\-performing variants often emerging from non\-obvious lineages rather than greedy refinement\.

![Refer to caption](https://arxiv.org/html/2605.19330v1/figures/evolution_trees_combined.png)Figure 8:Prompt evolution treesfor MOCHA across all six skills \( shown for one seed\)\. Each node is a committed skill variant; node labels show candidate ID and mean test score \(%\)\.Bluenode = best test correctness; blue edges = path from root\. Metric annotations \(C/D/B\) at root and best node reveal how MOCHA trades compliance for correctness gains\. Grey nodes = other committed candidates\.
### C\.6Per\-Task Qualitative Analysis

We provide qualitative comparisons of the MOCHA\-optimized skill versus the GEPA baseline for each task\. FEVER is covered in the main text \([Figure4](https://arxiv.org/html/2605.19330#S4.F4)\)\. Below we summarize the remaining five tasks\.

GPQA \(Graduate STEM QA\)\.GEPA returns the seed skill unchanged: a single\-line template \(“Given fieldsquestion, produce fieldsanswer”, 57 body tokens\)\. MOCHA discovers a 3,418\-token skill with a 6\-step expert verification protocol: \(1\) parse the question and identify the scientific domain, \(2\) establish foundational principles \(laws, equations, variables\), \(3\) solve methodically with unit tracking, \(4\) evaluate*every*answer choice independently, \(5\) challenge the answer using a “devil’s advocate” technique, and \(6\) output in the required format\. The skill includes domain\-specific error avoidance \(stereochemistry traps, redshift calculations, reduction reaction selectivity\)\. Test correctness: MOCHA\.636\.636vs\. GEPA\.592\.592\(\+4\.4\+4\.4pp\)\.

TheoremQA \(Mathematical Reasoning\)\.GEPA partially optimizes \(3,002 tokens\) but produces a verbose, loosely structured skill\. MOCHA produces a leaner \(2,517 tokens\) but more targeted skill emphasizing*explicit theorem naming*\(e\.g\., “Midsegment Theorem”, “Shannon Channel Capacity”, “Kraft inequality”\) and mandating that every matrix operation term be written out individually\. The key innovation: MOCHA requires verification by an alternative method \(e\.g\., cofactor expansion*and*row reduction for determinants\)\. Test correctness: MOCHA\.762\.762vs\. GEPA\.656\.656\(\+10\.6\+10\.6pp\)\.

HoVer \(Multi\-hop Claim Verification\)\.GEPA returns the seed skill \(67 body tokens\)\. MOCHA discovers a 2,141\-token skill with a rigorous 5\-step verification procedure: decompose the claim into atomic sub\-claims, check each against evidence with explicit quoting, apply a strict “all\-or\-nothing” rule \(SUPPORTED only if*all*sub\-claims are confirmed\), and watch for subtle entity swaps \(wrong names, incorrect dates, swapped roles\)\. The all\-or\-nothing rule prevents lenient verdicts on partially supported claims\. Test correctness: MOCHA\.660\.660vs\. GEPA\.618\.618\(\+4\.2\+4\.2pp\)\.

HotpotQA \(Multi\-hop QA\)\.Both methods partially optimize\. GEPA produces a 1,970\-token skill with answer format rules and 5\-step reasoning\. MOCHA produces a 2,593\-token skill with stricter output minimalism: yes/no questions require*only*lowercase “yes” or “no” \(no elaboration\), and entity answers prohibit parenthetical clarifications\. This task shows the smallest MOCHA advantage \(\.600\.600vs\.\.602\.602\), within standard deviation, consistent with the mild objective conflict observed in[Table1](https://arxiv.org/html/2605.19330#S4.T1)\.

DebugBench \(Code Debugging\)\.GEPA returns the seed skill \(75 body tokens\)\. MOCHA discovers a 2,315\-token skill enumerating common bug patterns: operator confusion \(=vs\.==,\+=vs\.\-=\), reference errors \(undefined functions, wrong variable names\), logic errors \(off\-by\-one, wrong loop bounds\), and syntactic traps \(semicolons afterif/forconditions that cause unconditional execution\)\. The skill mandates “surgical” minimal fixes, changing only the buggy line\(s\)\. Test correctness: MOCHA\.666\.666vs\. GEPA\.615\.615\(\+5\.1\+5\.1pp\)\.

Summary\.Across all six tasks, MOCHA’s multi\-objective selection machinery enables the optimizer to*commit and refine*increasingly structured skill variants, while baselines either return the seed unchanged \(4/6 tasks\) or produce less targeted instructions\. The skill quality differences are qualitative, not just quantitative: MOCHA skills contain domain\-specific reasoning protocols, explicit error avoidance, and structured output formatting absent from baseline skills\.

[Figures9](https://arxiv.org/html/2605.19330#A3.F9),[10](https://arxiv.org/html/2605.19330#A3.F10),[11](https://arxiv.org/html/2605.19330#A3.F11),[12](https://arxiv.org/html/2605.19330#A3.F12)and[13](https://arxiv.org/html/2605.19330#A3.F13)compare the seed skill template against the MOCHA\-optimized SKILL\.md for each task \(oracle candidate, i\.e\., best test correctness across all seeds\)\. FEVER is shown in the main text \([Figure4](https://arxiv.org/html/2605.19330#S4.F4)\)\.

Seed Skill — GPQA \(Graduate STEM QA\)Correctness: 0\.59\-\-\- name:gpqa\_solvercompatibility:dspy\>=2\.5, python\>=3\.10 metadata:task: gpqa, modules: 1allowed\-tools:\[\] description:Graduate\-level STEM multiple\-choice question answering✓52/1,024 chars \-\-\- \# execute\.predict Given the fields ‘question’, produce the fields ‘answer’\. \(no additional instructions\) ✓57/5,000 chars ▶\\blacktrianglerightReturned unchanged by TextGrad, ProTeGi, and GEPA after 1000 rollouts

MOCHA\-Optimized — GPQA \(Graduate STEM QA\)Correctness: 0\.71\-\-\- description:Graduate\-level STEM multiple\-choice question answering requiring rigorous step\-by\-step scientific reasoning, deep domain expertise in physics, chemistry, biology, and mathematics, careful elimination of wrong answers through explicit testing of every option, adversarial self\-checking for common pitfalls, and systematic verification before committing to the final answer letter\.✓∼\\sim360/1,024 chars \-\-\- \# execute\.predict You are an expert\-level STEM scientist\. Step 1: Deep Problem Parsing\-\-\- Read question THREE times\. 1st: identify domain\. 2nd: extract values, conditions, reagents\. 3rd: identify exactly what is asked\. Note how answer choices differ from each other\. Step 2: Solve from First Principles\-\-\- Work through fundamental laws/mechanisms\. Write all equations explicitly\. ∙\\bulletChemistry \(organic\): Draw full structures incl\. stereochemistry\. Identify reactive site considering steric & electronic effects at EVERY position\. For named reactions \(Wittig, Tebbe, Petasis, McMurry\), recall exact mechanisms\. ∙\\bulletPhysics: Set up equations, verify units at every step\. ∙\\bulletBiology: Map all crosses explicitly\. Consider linkage, epistasis, penetrance\. Step 3: Test EVERY Answer Choice\-\-\- For EACH option A\-\-D, write verdict: ‘‘A: fails because\.\.\.’’, ‘‘B: matches because\.\.\.’’ Step 4: Adversarial Self\-Check \(MANDATORY\)\-\-\- Assume first answer is WRONG\. Actively seek errors: ∙\\bulletSteric hindrance: correctly identify LEAST hindered site? ∙\\bulletReaction selectivity: correct reacting position? ∙\\bulletSign/direction errors, symmetry, meso compounds, approximation validity ∙\\bulletNamed reaction specifics: Tebbe converts esters to enol ethers, Simmons\-Smith gives cyclopropanation Step 5: Compare Top Two Candidates\-\-\- Head\-to\-head on single distinguishing factor\. Step 6: Commit\-\-\- Single letter: A, B, C, or D\. Favor standard mechanism over exotic\. Key Error Patterns:\(1\) Steric effects in enolization: t\-BuOK favors less substituted kinetic enolate\. \(2\) Pathway sequences: trace causally\. \(3\) Organometallic specificity: Cp2TiMeCl vs Cp2TiMe2\. \(4\) Don’t conflate similar reactions\. ✓∼\\sim3,418/5,000 chars

Figure 9:GPQA: Seed skill vs\. MOCHA\-optimized\.The seed skill \(top, red\) is a single\-line template returned unchanged by all three baselines\. MOCHA \(bottom, green\) discovers a 6\-step expert verification protocol with adversarial self\-checking and domain\-specific error patterns for organic chemistry, physics, and genetics\. Correctness improves from\.59\.59to\.71\.71\.Seed Skill — TheoremQA \(Mathematical Reasoning\)Correctness: 0\.53\-\-\- name:theoremqa\_solvercompatibility:dspy\>=2\.5, python\>=3\.10 metadata:task: theoremqa, modules: 1allowed\-tools:\[\] description:Theorem\-based QA across math, physics, and CS domains✓49/1,024 chars \-\-\- \# execute\.predict Given the fields ‘problem’, produce the fields ‘answer’\. \(no additional instructions\) ✓57/5,000 chars ▶\\blacktrianglerightPartially optimized by baselines but with verbose, loosely structured output

MOCHA\-Optimized — TheoremQA \(Mathematical Reasoning\)Correctness: 0\.82\-\-\- description:Solves theorem\-based questions across math, physics, finance, EE, and CS by carefully applying relevant theorems, tracking signs/units, and returning precisely formatted answers as decimal numbers, booleans, or lists\.✓∼\\sim215/1,024 chars \-\-\- \# execute\.predict You are an expert problem solver across mathematics, physics, EE, CS, and finance\. Reasoning Protocol: 1\. Identify the domain and relevant theorem\(s\): State which theorem, formula, or principle applies\. 2\. Define all variables and given quantities explicitly: Write out every given value with correct signs and units\. 3\. Apply the theorem step by step: Show each algebraic/logical step\. Double\-check: ∙\\bulletSigns: Pay extreme attention to negative signs\. Never drop them\. ∙\\bulletPowers of 10: Verify exponent arithmetic carefully\. ∙\\bulletUnits: Track throughout\. Convert as needed but CHECK expected units\. 4\. Domain\-specific rules: ∙\\bulletResistance with geometry: Axial:R=ρ​L/\(π​\(Ro2−Ri2\)\)R=\\rho L/\(\\pi\(R\_\{o\}^\{2\}\-R\_\{i\}^\{2\}\)\)\. Radial:R=\(ρ/2​π​L\)​ln⁡\(Ro/Ri\)R=\(\\rho/2\\pi L\)\\ln\(R\_\{o\}/R\_\{i\}\)\. ∙\\bulletStopping times:TTis stopping time iff\{T≤t\}∈ℱt\\\{T\\leq t\\\}\\in\\mathcal\{F\}\_\{t\}\. Sum of non\-negative stopping times IS a stopping time\. ∙\\bulletIteration methods: For Aitken’sΔ2\\Delta^\{2\}, count iterations of the ACCELERATED method only\. CRITICAL Formatting Rules: ∙\\bulletIf multiple sub\-parts, return ONLY the answer to the LAST part asked\. ∙\\bulletReturn ONLY the final answer\-\-\-no explanations, no units, no labels\. ∙\\bulletConvert ALL symbolic/fractional answers to decimals: 4/9→\\to0\.4444, sqrt\(2\)→\\to1\.4142\. ∙\\bulletFloats ALWAYS include decimal point: 3\.0 not 3\. ∙\\bulletRound to 4 decimal places unless exact\. ✓∼\\sim2,517/5,000 chars

Figure 10:TheoremQA: Seed skill vs\. MOCHA\-optimized\.Baselines partially optimize but produce verbose, loosely structured output\. MOCHA discovers a lean skill with theorem identification, sign/unit tracking, domain\-specific templates, and strict formatting rules\. Correctness improves from\.53\.53to\.82\.82\.Seed Skill — HoVer \(Multi\-hop Claim Verification\)Correctness: 0\.62\-\-\- name:hover\_verificationcompatibility:dspy\>=2\.5, python\>=3\.10 metadata:task: hover\_singlecall, modules: 1allowed\-tools:\[retriever\] description:Claim verification with retrieval tool✓38/1,024 chars \-\-\- \# execute\.predict Given the fields ‘claim’, ‘evidence’, produce the fields ‘verdict’\. \(no additional instructions\) ✓67/5,000 chars ▶\\blacktrianglerightReturned unchanged by TextGrad, ProTeGi, and GEPA after 1000 rollouts

MOCHA\-Optimized — HoVer \(Multi\-hop Claim Verification\)Correctness: 0\.67\-\-\- description:Multi\-hop claim verification against retrieved evidence passages\. Decomposes complex claims into sub\-claims, retrieves and chains evidence, and produces a binary SUPPORTED or NOT\_SUPPORTED verdict\.✓∼\\sim198/1,024 chars \-\-\- \# execute\.predict You are an expert fact\-checker performing multi\-hop claim verification\. Critical Rules for Verdict: 1\.Binary output only: exactly SUPPORTED or NOT\_SUPPORTED\. Never ‘‘PARTIALLY SUPPORTED’’ or any other value\. 2\.Default toward SUPPORTED when evidence is consistent\. Do NOT require every detail to be explicitly stated\-\-\-implicit support and reasonable inference count\. 3\.Only NOT\_SUPPORTED when evidenceactively contradictsthe claim\. Reasoning Strategy: Step 1: Decompose claim into atomic sub\-claims\. Step 2: Map evidence to sub\-claims\. Note direct vs\. inferential support\. Step 3: Use retriever tool to fill gaps with targeted queries\. Step 4: Chain reasoning across passages\. Follow entity links completely\. Step 5: Check for explicit contradictions\. Step 6: For comparative/temporal claims, extract specific quantities and compare directly\. Step 7: Produce verdict\. Common Mistakes to Avoid: ∙\\bulletDo NOT default to NOT\_SUPPORTED because evidence is incomplete\-\-\-retrieve more\. ∙\\bulletDo NOT require verbatim confirmation\. Logical inference counts\. ∙\\bulletDo NOT conflate ‘‘not mentioned’’ with ‘‘contradicted\.’’ ∙\\bulletAbsence of evidence is not evidence of absence\-\-\-retrieve more evidence\. ✓∼\\sim2,141/5,000 chars

Figure 11:HoVer: Seed skill vs\. MOCHA\-optimized\.The seed skill \(top, red\) is returned unchanged by all baselines\. MOCHA \(bottom, green\) discovers a 7\-step verification procedure with “default toward SUPPORTED” bias and retriever\-augmented gap filling\. Correctness improves from\.62\.62to\.67\.67\.Seed Skill — HotpotQA \(Multi\-hop QA\)Correctness: 0\.34\-\-\- name:hotpotqa\_qacompatibility:dspy\>=2\.5, python\>=3\.10 metadata:task: hotpotqa\_singlecall, modules: 1allowed\-tools:\[retriever\] description:Question answering with retrieval tool✓38/1,024 chars \-\-\- \# execute\.predict Given the fields ‘question’, ‘evidence’, produce the fields ‘answer’\. \(no additional instructions\) ✓67/5,000 chars ▶\\blacktrianglerightPartially optimized by all baselines

MOCHA\-Optimized — HotpotQA \(Multi\-hop QA\)Correctness: 0\.66\-\-\- description:Multi\-hop question answering over retrieved evidence passages, producing short exact answers extracted verbatim from evidence text✓∼\\sim131/1,024 chars \-\-\- \# execute\.predict Given the fields ‘question’, ‘evidence’, produce the fields ‘answer’\. Instructions for Reasoning: 1\. Decompose: Break question into sub\-questions\. Identify what entity/fact each hop requires\. 2\. Extract: Read every evidence piece\. Extract all names \(full formal names\), dates, nicknames, roles, locations\-\-\-even from parenthetical remarks\. 3\. Retrieve: If evidence is insufficient, call retriever with targeted queries\. Do NOT give up\. 4\. Chain: Connect facts across passages\. Entity A in passage 1→\\toEntity B in passage 2\. 5\. Synthesize: Determine final answer\. Critical Rules for Answer Field: ∙\\bulletShort exact phrase\-\-\-name, date, number, place, or brief noun phrase\. ∙\\bulletEXACT form from evidence: ‘‘Jerral Wayne Jones Sr\.’’ NOT ‘‘Jerry Jones’’\. ‘‘Dayton, Ohio’’ NOT ‘‘Dayton’’\. ∙\\bulletCopy verbatim whenever possible\. Preserve location qualifiers\. ∙\\bulletUse the most complete, formal name version found in evidence\. ∙\\bulletDo NOT add explanations, parenthetical remarks, or full sentences\. ∙\\bulletStrip ‘‘the answer is’’ but keep geographic qualifiers, suffixes \(Sr\., Jr\., III\)\. ∙\\bulletNever say ‘‘cannot be determined’’\-\-\-use the retriever tool first\. Correct formatting examples: Good: Dayton, Ohio\|\|Bad: Dayton Good: Jerral Wayne Jones Sr\.\|\|Bad: Jerry Jones Good: Tony Yoka\|\|Bad: Tony Yoka, a French boxer Good: 1953\|\|Bad: The answer is 1953 ✓∼\\sim2,593/5,000 chars

Figure 12:HotpotQA: Seed skill vs\. MOCHA\-optimized\.Both baselines and MOCHA partially optimize this task\. MOCHA discovers a skill emphasizing verbatim extraction \(exact name forms, location qualifiers\) with explicit good/bad formatting examples\. Correctness improves from\.34\.34to\.66\.66\.Seed Skill — DebugBench \(Code Debugging\)Correctness: 0\.62\-\-\- name:debugbench\_fixercompatibility:dspy\>=2\.5, python\>=3\.10 metadata:task: debugbench\_singlecall, modules: 1allowed\-tools:\[\] description:Single\-call code debugging \(buggy\_code \-\> fixed\_code\)✓48/1,024 chars \-\-\- \# execute\.predict Given the fields ‘buggy\_code’, ‘bug\_type’, produce the fields ‘fixed\_code’\. \(no additional instructions\) ✓75/5,000 chars ▶\\blacktrianglerightReturned unchanged by TextGrad, ProTeGi, and GEPA after 1000 rollouts

MOCHA\-Optimized — DebugBench \(Code Debugging\)Correctness: 0\.73\-\-\- description:Fix bugs in code based on the specified bug\_type classification\. For single\-category bugs, make the minimal change needed\. For multiple errors, fix all bugs found\. Do not refactor or improve beyond what the bug\_type requires\.✓∼\\sim225/1,024 chars \-\-\- \# execute\.predict You are a precise code debugger\. You receive buggy\_code and a bug\_type classification\. Critical Rules: 1\. Understand bug\_type first\-\-\-it determines fixing strategy: ∙\\bulletreference error: Wrong variable/function/method name\. Fix ONLY the incorrect reference\(s\)\. ∙\\bulletsyntax error: Missing colon, semicolon, bracket, wrong operator syntax\. Fix ONLY syntax\. ∙\\bulletlogic error: Off\-by\-one, wrong comparison, wrong return, wrong condition\. Fix ONLY logic\. ∙\\bullettype error: Wrong type usage or conversion\. ∙\\bulletmultiple error: 2\-\-4 bugs across categories\. Find and fix ALL\. 2\. Conservative fixing principle: When uncertain, do NOT change\. A wrong fix is worse than a missing fix\. 3\. Reproduce the rest EXACTLY\-\-\-preserve all indentation, spacing, comments, structure\. Reasoning Process: Step 1: Read bug\_type\. Single\-category or multiple? Step 2: Understand algorithm PURPOSE before making changes\. Step 3: For each bug, state: \(a\) exact line, \(b\) what is wrong, \(c\) fix, \(d\) why it is definitely a bug\. Step 4: For multiple error\-\-\-count bugs\. Typical: 2\-\-4\. If only 1, look harder\. If\>\>5, reconsider\. Step 5: For reference errors\-\-\-replace with correct name that EXISTS in scope\. Step 6: Apply fixes\. Double\-check each is minimal and correct\. Step 7: Re\-read output\. Confirm only bug\_type\-matching changes were made\. Common Pitfalls: ∙\\bulletDo NOT change loop bounds unless certain they cause error\. ∙\\bulletDo NOT treat multiple error as license to rewrite\-\-\-each fix is a small, targeted change\. ∙\\bulletWhen variable exists elsewhere in code, prefer that exact spelling as the fix\. ∙\\bulletFor C\+\+/Java: common bugs include missing semicolons, = vs ==, wrong variable names\. ✓∼\\sim2,315/5,000 chars

Figure 13:DebugBench: Seed skill vs\. MOCHA\-optimized\.The seed template \(top, red\) provides no debugging strategy\. MOCHA \(bottom, green\) develops a category\-aware protocol: classify by bug type, apply type\-specific heuristics \(reference→\\toscope check, logic→\\toboundary check, multiple→\\tocount 2–4\), and follow a “conservative fixing principle” that prevents over\-correction on multi\-bug inputs\.
### C\.7Per\-Task Ablation Detail

[Table5](https://arxiv.org/html/2605.19330#A3.T5)provides per\-task correctness for each MOCHA variant and all baselines, complementing the aggregate view in[Table3](https://arxiv.org/html/2605.19330#S4.T3)\.

Table 5:Per\-task correctnessfor all methods \(mean±\\pmstd, 5 seeds\)\. Bold = best per task\.Observations\.\(1\) The w/o HVC variant \(pure exploitation\) achieves the highest correctness on 4/6 tasks, consistent with its position at the exploitation end of the spectrum \([Table3](https://arxiv.org/html/2605.19330#S4.T3)\)\. \(2\) Full MOCHA achieves the best or near\-best result on HoVer, where the exploration→\\toexploitation transition prevents the premature convergence observed in w/o Annealing \(−4\.6\-4\.6pp\)\. \(3\) HotpotQA remains challenging for all MOCHA variants: ProTeGi’s UCB beam search \(\.622\.622\) outperforms all MOCHA variants, suggesting that this task’s flat objective landscape favors single\-objective exploitation\. \(4\) All MOCHA variants substantially outperform all baselines on GPQA, FEVER, and DebugBench—tasks where baselines return the seed unchanged\.

### C\.8Ablation: Hypervolume Heatmap

![Refer to caption](https://arxiv.org/html/2605.19330v1/x7.png)Figure 14:Ablation heatmap: CorrectnessΔ\\Deltaover GEPA for each MOCHA variant across six skills\. All MOCHA variants achieve substantial gains on TheoremQA and FEVER\. Removing HVC gating shifts toward exploitation \(highest per\-task correctness\); removing annealing shifts toward exploration \(highest Pareto diversity\)\. See[Table3](https://arxiv.org/html/2605.19330#S4.T3)for the aggregate exploration–exploitation spectrum\.