GRAFT: Execution-Guided Genetic Recombination for Agent Design

arXiv cs.AI Papers

Summary

The paper introduces GRAFT, an execution-guided genetic recombination method for automated LLM agent design that uses parent-specific execution evidence to guide semantic crossover, achieving 82.0% mean test accuracy across six reasoning and programming settings—9.8 points above the strongest baseline.

arXiv:2611.23509v1 Announce Type: new Abstract: Automated agent design searches for executable architectures that improve LLM-based agent performance, with evolutionary methods using language-model crossover to recombine promising mechanisms for task decomposition, model coordination, and verification. This search, however, is typically guided by aggregate fitness alone, which hides the instance-level successes and failures behind each score. We introduce Execution Guided Genetic Recombination for Agent Design (GRAFT), which uses aggregate fitness for selection and survival and parent-specific execution evidence to guide semantic crossover with integrated mutation. Its feedback harness reuses bounded samples of successful and failed runs from fitness evaluation, withholding reference answers and requiring no additional task executions. We evaluate GRAFT across six reasoning and programming settings using open-weight optimizer and executor models. It achieves a mean test accuracy of 82.0%, exceeding the strongest baseline by 9.8% points. Controlled ablations show that adding execution evidence improves test accuracy with initialization, search rules, repair policies, and evaluation budgets held fixed. Our results support the value of parent-specific execution evidence for evolutionary agent design.
Original Article
View Cached Full Text

Cached at: 11/20/26, 09:05 AM

# GRAFT: Execution-Guided Genetic Recombination for Agent Design
Source: [https://arxiv.org/html/2611.23509](https://arxiv.org/html/2611.23509)
Mohammed Saidul IslamAffiliation:Vector InstituteXiaodan ZhuAffiliation:York UniversityAffiliation:Queen’s UniversityCorrespondence:[sevdaebr@yorku\.ca](mailto:email@domain)Elham DolatabadiAffiliation:York UniversityAffiliation:Vector Institute

###### Abstract

Automated agent design searches for executable architectures that improve LLM\-based agent performance, with evolutionary methods using language\-model crossover to recombine promising mechanisms for task decomposition, model coordination, and verification\. This search, however, is typically guided by aggregate fitness alone, which hides the instance\-level successes and failures behind each score\. We introduce Execution\-Guided Genetic Recombination for Agent Design \(GRAFT\), which uses aggregate fitness for selection and survival and parent\-specific execution evidence to guide semantic crossover with integrated mutation\. Its feedback harness reuses bounded samples of successful and failed runs from fitness evaluation, withholding reference answers and requiring no additional task executions\. We evaluateGRAFTacross six reasoning and programming settings using open\-weight optimizer and executor models\. It achieves a mean test accuracy of 82\.0%, exceeding the strongest baseline by 9\.8% points\. Controlled ablations show that adding execution evidence improves test accuracy with initialization, search rules, repair policies, and evaluation budgets held fixed\. Our results support the value of parent\-specific execution evidence for evolutionary agent design\. All code is open\-sourced at[https://anonymous\.4open\.science/r/GRAFT\-B1EB/](https://anonymous.4open.science/r/GRAFT-B1EB/)\.

Figure 1:\(a\) The GA loop shared by GA\-only search and GRAFT\. GRAFT also passes execution samples from the evaluation step to the crossover step\. \(b\) An illustrative crossover on Game24\. Both parents score 50%, but their execution samples show different failures\. Both children inherit Parent B’s check that the value equals 24, and each adds one step that neither parent has \(*new*; integrated mutation\)\. Without samples, the new step and combination are untargeted, so the answer6×46\\times 4for \[1, 3, 4, 6\] still passes\. With samples, the child adds a check that each number is used once, which rejects it\.## 1Introduction

Architectural decisions, including task decomposition, the orchestration of model calls, and the validation of intermediate outputs, shape the performance of LLM\-based agents\([Bajoria et al\., 2026](https://arxiv.org/html/2611.23509#bib.bib43)\)\. Automated agent design therefore searches over executable programs to find architectures that, with a fixed execution model, perform well on unseen instances of a target task\([Gullí, 2025](https://arxiv.org/html/2611.23509#bib.bib44)\)\.

Automated search methods, including ADAS\([Hu et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib40)\), AFlow\([Zhang et al\., 2025b](https://arxiv.org/html/2611.23509#bib.bib42)\), and SwarmAgentic\([Zhang et al\., 2025c](https://arxiv.org/html/2611.23509#bib.bib18)\), discover agent architectures through iterative generation and evaluation\. Evolutionary approaches can reuse promising mechanisms through language\-model crossover, which generates offspring conditioned on multiple parent programs, allowing an LLM to combine and adapt mechanisms expressed in their code\([Meyerson et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib41)\)\. AgentBreeder applies evolutionary variation to agent scaffolds, providing a direct precedent for evolutionary search over agent architectures\([Rosser and Foerster, 2026](https://arxiv.org/html/2611.23509#bib.bib22)\)\.

Evolutionary search is typically guided by aggregate fitness scores, which summarize overall performance without revealing the instance\-level successes, failures, and behavioral patterns underlying those scores\([Guo et al\., 2023](https://arxiv.org/html/2611.23509#bib.bib20);[Ye et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib21);[Meyerson et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib41)\)\. Prior work has shown that richer evaluation feedback can guide prompt optimization, workflow refinement, and evolutionary search by providing qualitative, gradient\-like signals for improvement\([Agrawal et al\., 2026](https://arxiv.org/html/2611.23509#bib.bib1);[Lee et al\., 2026](https://arxiv.org/html/2611.23509#bib.bib6);[Lee et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib5);[Cheng et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib13);[Yuksekgonul et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib17);[Pryzant et al\., 2023](https://arxiv.org/html/2611.23509#bib.bib14);[Yang et al\., 2021](https://arxiv.org/html/2611.23509#bib.bib15);[Liu et al\., 2026](https://arxiv.org/html/2611.23509#bib.bib50)\)\. Yet, its role in guiding semantic crossover and mutation in agent architecture search remains underexplored\.

We investigate this gap by varying the information supplied to LLM\-based semantic crossover while holding initialization, search rules, repair policies, and the evaluation budget fixed\. We introduceExecution\-Guided Genetic Recombination for Agent Design \(GRAFT\), a population\-based framework for automated agentic system design\. InGRAFT, aggregate fitness governs parent selection and survival, while an LLM\-based variation operator generates offspring through semantic crossover with integrated mutation\. Building on Language Model Crossover\([Meyerson et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib41)\), the crossover operator receives parent\-specific execution evidence supplied by a feedback harness, alongside parent code and fitness \(Figure[1](https://arxiv.org/html/2611.23509#S0.F1)\)\. We evaluateGRAFTon six benchmarks spanning abstract reasoning, multilingual mathematics, program synthesis, and arithmetic puzzles, using open\-weight optimizer and executor models\. WithQwen3\.5\-35B\-A3Bas both optimizer and executor,GRAFTachieves the highest average test accuracy \(82\.0%\), exceeding the strongest baseline by 9\.8 percentage points\. On ARC, with a fixed optimizer,GRAFTachieves the highest mean accuracy across three executors \(38\.1%\), exceeding the strongest baseline by 4\.9 percentage points\.

Our contributions are as follows:

1. 1\.Execution\-Guided Genetic Recombination for Agent Design\.We introduceGRAFT, a framework that augments LLM\-based semantic crossover with parent\-specific execution evidence to guide the combination and revision of executable agent programs\.
2. 2\.A controlled evaluation of architecture search and execution feedback\.We isolate the role of execution evidence by varying the information supplied to crossover while holding initialization, search rules, repair policies, and the evaluation budget fixed\. We assess the performance ofGRAFTacross four tasks and six benchmarks and analyze its behavior under different executor models, enabling a systematic comparison with existing automated agent\-design methodologies\.

## 2Related Work

#### Automated agent design\.

Automated designers optimize different aspects of agentic systems\. DSPy optimizes prompts and demonstrations\([Khattab et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib3)\), while GPTSwarm optimizes node prompts and graph connectivity\([Zhuge et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib19)\)\. Workflow and architecture search methods include AFlow’s Monte Carlo tree search\([Zhang et al\., 2025b](https://arxiv.org/html/2611.23509#bib.bib42)\), MaAS’s learned agentic supernet\([Zhang et al\., 2025a](https://arxiv.org/html/2611.23509#bib.bib45)\), and W4S’s reinforcement\-trained meta\-agent\([Nie et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib46)\)\. ADAS searches over executable agent programs using an archive of previous designs\([Hu et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib40)\), while Gödel Agent and DGM support self\-improvement through agent\-code modification\([Yin et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib47);[Zhang et al\., 2026](https://arxiv.org/html/2611.23509#bib.bib49)\)\. SwarmAgentic maintains a population of systems and applies PSO\-inspired, language\-driven updates\([Zhang et al\., 2025c](https://arxiv.org/html/2611.23509#bib.bib18)\)\.

#### LLM\-driven evolutionary search\.

LLMs can serve as variation operators within evolutionary search\. ELM uses LLM\-generated mutations\([Lehman et al\., 2023](https://arxiv.org/html/2611.23509#bib.bib7)\), while LMX generates offspring conditioned on multiple parents\([Meyerson et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib41)\)\. FunSearch, AlphaEvolve, and ShinkaEvolve apply LLM\-driven evolution to program discovery\([Romera\-Paredes et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib12);[Novikov et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib11);[Lange et al\., 2026](https://arxiv.org/html/2611.23509#bib.bib4)\)\. Closer to our setting, AgentBreeder evolves executable agent scaffolds through LLM\-based crossover and mutation\([Rosser and Foerster, 2026](https://arxiv.org/html/2611.23509#bib.bib22)\)\. These approaches motivate combining population\-based search with semantic variation over agent programs\.

#### Feedback\-guided optimization\.

Behavioral evidence can complement aggregate fitness in guiding search\. Genetic programming provides precedents through case\-wise performance in lexicase selection\([Helmuth et al\., 2014](https://arxiv.org/html/2611.23509#bib.bib2)\)and program\-output semantics in geometric semantic variation\([Moraglio et al\., 2012](https://arxiv.org/html/2611.23509#bib.bib10)\)\. ProTeGi and TextGrad use natural\-language feedback as textual gradients\([Pryzant et al\., 2023](https://arxiv.org/html/2611.23509#bib.bib14);[Yuksekgonul et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib17)\), while Trace combines execution traces with output feedback\([Cheng et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib13)\)\. ReEvo supplies reflections on parental performance to heuristic crossover\([Ye et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib21)\)\. GEPA combines trajectory\-based reflection with instance\-wise Pareto selection\([Agrawal et al\., 2026](https://arxiv.org/html/2611.23509#bib.bib1)\); Mind Evolution uses evaluator feedback to refine candidate solutions\([Lee et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib5)\); and Meta\-Harness provides access to prior programs, scores, and execution traces\([Lee et al\., 2026](https://arxiv.org/html/2611.23509#bib.bib6)\)\. Our focus is on using parent\-specific execution evidence to guide LMX crossover within a genetic search method for high\-performing agent architectures\.

## 3Harness\-Guided Genetic Agent Design

### 3\.1Problem Formulation

Each candidate agentic system,SS, is an executable Python program that takes a task instance as input and returns an answer\. The search space comprises programs specifying the agent’s architecture, including the organization of model calls, intermediate processing, and output generation\. Each candidate is evaluated on a fixed validation set and assigned a fitness scoreJ⁡\(S\)J\(S\)reflecting its task performance\.

LetPt=\{Si\(t\)\}i=1NP\_\{t\}=\\\{S\_\{i\}^\{\(t\)\}\\\}\_\{i=1\}^\{N\}denote a population ofNNcandidate agents at iterationtt\. Each candidate is evaluated on a validation set and assigned a fitness scoreJ⁡\(Si\(t\)\)J\(S\_\{i\}^\{\(t\)\}\)reflecting its performance on the target task\. The population evolves through iterative selection and variation to discover higher\-performing architectures\. Search proceeds for at mostnniterations, subject to the available design budget and predefined stopping conditions\. If search terminates at iterationT≤nT\\leq n, the best\-performing agent is selected as:

Sbest∈arg⁡maxS∈PT⁡J⁡\(S\)\.S\_\{\\mathrm\{best\}\}\\in\\arg\\max\_\{S\\in P\_\{T\}\}J\(S\)\.\(1\)

### 3\.2Execution\-Guided Genetic Recombination for Agent Design

Our approach,GRAFT, combines fitness\-based evolutionary selection with execution\-guided semantic crossover within a genetic algorithm\. Aggregate fitness determines which architectures are selected and retained, while parent\-specific execution evidence informs how the LLM constructs offspring\. The guidance is qualitative: the operator uses observed behavior to propose architectural changes, and fitness is assessed through execution\.

#### Initialization and parent selection\.

The initial populationP0P\_\{0\}containsNNhand\-designed agent programs\. Evaluating a programSSon the validation set yields its fitnessJ⁡\(S\)J\(S\)and per\-example execution recordsR⁡\(S\)R\(S\)\. These values are cached together under the evaluated program’s identity\. At each iterationtt, the search attemptsλ\\lambdaoffspring proposals\. Fitness\-based tournament selection chooses two distinct parent programs,SaS\_\{a\}andSbS\_\{b\}, fromPtP\_\{t\}; each tournament selects a highest\-fitness member amongkksampled candidates\. Distinct parents may have identical aggregate fitness\.

#### Execution\-Feedback Harness\.

For each selected parent, the harness retrieves its cached fitness and execution records to construct a compact representation of its observed performance:

H⁡\(S\)=\(J⁡\(S\),E\+​\(S\),E−​\(S\)\),H\(S\)=\\bigl\(J\(S\),E^\{\+\}\(S\),E^\{\-\}\(S\)\\bigr\),\(2\)whereE\+​\(S\)E^\{\+\}\(S\)andE−​\(S\)E^\{\-\}\(S\)denote bounded samples of successful and failed executions, respectively, drawn fromR⁡\(S\)R\(S\)\. Each record contains the agent’s response, correctness outcome, and available evaluator feedback\. Successful and failed examples are sampled separately in validation order, subject to the limits specified in the experimental setup\. The evidence remains associated with its source parent, allowing the variation operator to distinguish their observed behaviors\. Reference answers are withheld from the proposal LLM\. Because the harness reuses records collected during fitness evaluation, constructing this feedback requires no additional task executions\.

#### Harness\-Guided Semantic Crossover\.

We build on Language Model crossover parent\-conditioned generation, in which an LLM produces offspring by recombining information from multiple parent programs\. InGRAFT, the variation modelMvarM\_\{\\mathrm\{var\}\}receives both parent programs and their execution\-feedback harnesses to generate a complete offspring program\.

Letℒt\\mathcal\{L\}\_\{t\}denote the elite\-population context andπ\\pithe fixed variation instructions which is specific to each task\. Offspring generation is expressed as

S~c←GG​\-​LMXMvar​\(CLOSEOPENSa,Sb,H⁡\(Sa\),H⁡\(Sb\),ℒt;π\)\.\\begin\{array\}\[\]\{@\{\}l@\{\}\}\\widetilde\{S\}\_\{c\}\\leftarrow\\mathrm\{GG\\mbox\{\-\}LMX\}\_\{M\_\{\\mathrm\{var\}\}\}\\bigl\(\\\\ \\qquad S\_\{a\},S\_\{b\},H\(S\_\{a\}\),H\(S\_\{b\}\),\\mathcal\{L\}\_\{t\};\\pi\\bigr\)\.\\end\{array\}\(3\)The prompt directs the LLM to preserve or combine useful parental mechanisms, address observed failure patterns, and introduce architectural novelty using the elite population as context\. Crossover and mutation are integrated within the same proposal step\. Bounded revision passes check the proposed program’s recombination and implementation\. Algorithm[3\.2](https://arxiv.org/html/2611.23509#S3.SS2.SSS0.Px4)summarizes the GG\-LMX operator\. The completeGRAFTsearch procedure is presented in Algorithm[D](https://arxiv.org/html/2611.23509#A4)in Appendix D\.

#### Evaluation and Elitist Replacement\.

Each proposal is evaluated using the common validation protocol and the fixed revision and retry policy\. The evaluation routine returns the final program versionScS\_\{c\}together with its fitness and execution records, or reports an unresolved failure\. Fitness and evidence are always associated with the program version that produced them\.

LetQtQ\_\{t\}denote the valid, evaluated offspring admitted to replacement\. Because proposal attempts may fail,\|Qt\|≤λ\|Q\_\{t\}\|\\leq\\lambda\. The next population is

Pt\+1=TopN​\(Pt∪Qt,J\),P\_\{t\+1\}=\\mathrm\{Top\}\_\{N\}\\bigl\(P\_\{t\}\\cup Q\_\{t\};J\\bigr\),\(4\)whereTopN\\mathrm\{Top\}\_\{N\}retains theNNhighest\-fitness candidates, with ties favoring incumbents\. Retained programs keep their cached fitness and execution evidence\. Search ends after at mostnniterations, when the design budget is exhausted, or when the specified stopping condition is met\. It returns a highest\-fitness systemSbestS\_\{\\mathrm\{best\}\}from the final population\.

Algorithm 1GG\-LMXOffspring Generation

Require:Elite contextℒt\\mathcal\{L\}\_\{t\}, proposal LLMMvarM\_\{\\mathrm\{var\}\}, and variation instructionsπ\\pi\.

## 4Experiments

### 4\.1Experimental Setup

#### Tasks\.

We evaluate agentic systems across six benchmarks spanning two broad task categories: reasoning and coding\. We use ARC\([Chollet, 2019](https://arxiv.org/html/2611.23509#bib.bib23)\)to assess abstract reasoning, MGSM\([Shi et al\., 2022](https://arxiv.org/html/2611.23509#bib.bib24)\)and Game of 24\([Yao et al\., 2023](https://arxiv.org/html/2611.23509#bib.bib48)\)test multilingual mathematical reasoning and arithmetic puzzle solving, and SuperGPQA\([Du et al\., 2026](https://arxiv.org/html/2611.23509#bib.bib27)\)for domain\-specific knowledge and reasoning\. We consider two disjoint subsets of SuperGPQA: SuperGPQA\-STEM, covering science, engineering, and medicine, and SuperGPQA\-Hard\-Calc, focusing on challenging calculation\-based questions across disciplines\. We use MBPP\+\([Austin et al\., 2021](https://arxiv.org/html/2611.23509#bib.bib25);[Liu et al\., 2023](https://arxiv.org/html/2611.23509#bib.bib26)\)to evaluate code generation with expanded test suites\. In addition in the optimizer\-variation study, we include CRUXEval\-I\([Gu et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib28)\)to assess code execution reasoning through input prediction\. Further benchmark details are provided in Appendix[A](https://arxiv.org/html/2611.23509#A1)\.

Table 1:Held\-out test scores \(%\) across six evaluation settings spanning five benchmarks, using Qwen3\.5\-35B as both optimizer and executor\. Average denotes the unweighted mean of all six scores\. Automated methods receive the same candidate\-evaluation budget \(Appendix[C](https://arxiv.org/html/2611.23509#A3)\)\. Reference architectures are evaluated without search\. Within each section, thebestscore per column is bolded and thesecond\-bestis underlined\.Methodgemma4\-12Bgemma\-3\-27bMeta\-Llama\-3\.18B\-InstructAverageCoT\([Wei et al\., 2022](https://arxiv.org/html/2611.23509#bib.bib33)\)35\.015\.83\.818\.2Quality\-Diversity\([Lu et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib38)\)33\.919\.22\.118\.4LLM Debate\([Du et al\., 2023](https://arxiv.org/html/2611.23509#bib.bib36)\)43\.920\.04\.222\.7Reflexion\([Shinn et al\., 2023](https://arxiv.org/html/2611.23509#bib.bib16)\)43\.920\.82\.922\.5CoT\-SC\([Wang et al\., 2022](https://arxiv.org/html/2611.23509#bib.bib34)\)41\.118\.32\.520\.6Role Assignment\([Xu et al\., 2023](https://arxiv.org/html/2611.23509#bib.bib39)\)27\.816\.74\.216\.2ADAS\([Hu et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib40)\)51\.134\.214\.233\.2AFlow\([Zhang et al\., 2025b](https://arxiv.org/html/2611.23509#bib.bib42)\)42\.519\.23\.321\.7SwarmAgentic\([Zhang et al\., 2025c](https://arxiv.org/html/2611.23509#bib.bib18)\)49\.220\.02\.523\.9GRAFT66\.140\.08\.338\.1

Table 2:ARC test scores \(%\) with three executors and the fixed Qwen3\.5\-35B optimizer\. Average denotes the unweighted mean across executors\. Evaluation budgets matched\. Reference architectures are evaluated directly with each executor\.
#### Baselines\.

We compare our framework against three automated architecture\-search methods: ADAS\([Hu et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib40)\), which generates programs using an archive of prior designs; AFlow\([Zhang et al\., 2025b](https://arxiv.org/html/2611.23509#bib.bib42)\), which performs search with Monte Carlo tree search; and SwarmAgentic\([Zhang et al\., 2025c](https://arxiv.org/html/2611.23509#bib.bib18)\), which uses language\-guided swarm optimization\. We also report the hand\-designed agents that initialize ADAS andGRAFT\(e\.g\., CoT, CoT\-SC, Self\-Refine, and LLM Debate, Step\-back Abstraction, Role Assignment, Quality\-Diversity \), reported and cited in the corresponding results tables, without architecture search explained in Appendix[B](https://arxiv.org/html/2611.23509#A2)\.

#### Models\.

We use separate LLMs for architecture optimization and task execution: the optimizer proposes candidate architectures, while the executor performs the model calls specified by those architectures\. All models are open\-weight and hosted locally, keeping task data, agent outputs, and execution feedback within the evaluation environment throughout the search, an important consideration in privacy\-sensitive domains\. In the main experiments, we useQwen3\.5\-35B\-A3B\-FP8\([Team, 2026](https://arxiv.org/html/2611.23509#bib.bib29)\)as both the optimizer and executor\. For the ARC executor study, we fixQwen3\.5\-35B\-A3B\-FP8as the optimizer and compareQwen3\.5\-9B\([Team, 2026](https://arxiv.org/html/2611.23509#bib.bib29)\),Gemma\-4\-12B\-it\([Team et al\., 2026](https://arxiv.org/html/2611.23509#bib.bib30)\), andGemma\-3\-27B\-it\([Team et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib31)\)as executors\. On CRUXEval\-I, we compareQwen3\.5\-35B\-A3B\-FP8andgpt\-oss\-120b\([Agarwal et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib32)\)as optimizers with a fixedQwen3\.5\-9Bexecutor\. Additional executors includeQwen3\-8B\([Yang et al\., 2025](https://arxiv.org/html/2611.23509#bib.bib8)\)for the case study andLlama\-3\.1 8B\([Grattafiori et al\., 2024](https://arxiv.org/html/2611.23509#bib.bib9)\)on ARC\.

#### Execution settings\.

Our framework maintains five candidate architectures and generates two offspring per generation\. Number of generation is specific to each task and is defined as variable\. Automated methods are evaluated under matched candidate\-evaluation budgets, except SwarmAgentic, which receives a larger budget, using the same task splits, executor, and scoring protocol within each configuration\. Full model, decoding, execution, and budget\-accounting settings are provided in Appendix[C](https://arxiv.org/html/2611.23509#A3)\.

### 4\.2Results

#### Overall Performance

Table[1](https://arxiv.org/html/2611.23509#S4.T1)reports held\-out test accuracy withQwen3\.5\-35B\-A3Bas both optimizer and executor\.GRAFTachieves the highest average accuracy of 82\.0% across the six benchmark settings, outperforming all automated and hand\-designed baselines\. The largest improvement occurs on SuperGPQA\-Hard\-Calc, where GRAFT reaches 96\.5% compared with 67\.2% for the strongest baseline on this task\. We also observe thatGRAFTremains competitive where hand\-designed approaches are strongest\. On Game24, LLM Debate outperforms all other automated design baselines, whileGRAFTmatches its score of 99\.2%\.

#### Executor choice

Table[2](https://arxiv.org/html/2611.23509#S4.T2)reports ARC test performance with three executor models while keeping theQwen3\.5\-35Boptimizer fixed\. Hand\-designed reference architectures are evaluated directly with each executor\.GRAFTachieves the highest scores withgemma4\-12Bandgemma\-3\-27b, reaching 66\.1% and 40\.0%, respectively\. These results exceed ADAS, the strongest baseline in both configurations, by 15\.0 and 5\.8 percentage points\. Across the three executors,GRAFTobtains the highest average score of 38\.1%\.

#### Optimizer choice\.

Table[3](https://arxiv.org/html/2611.23509#S4.T3)compares two architecture optimizers on CRUXEval\-I while fixing the executor toQwen3\.5\-9B\.GRAFTranks first withgpt\-oss\-120b\(60\.2%\) and second withQwen3\.5\-35B\(53\.9%\)\. It is the only automated method to exceed 50% accuracy with both optimizers, although the relative ranking of search methods changes across configurations\.

Table 3:CRUXEval\-I test scores \(%\) using two architecture optimizers and the fixed Qwen3\.5\-9B executor\. Evaluation budgets are matched\.Figure 2:Search on ARC with optimizerQwen3\.5\-35Band executorQwen3\.5\-9B\. Left: best validation accuracy found so far versus the number of candidate systems evaluated\. Right: test accuracy of each method’s best\-validation system, with 95% bootstrap confidence intervals\. Dashed lines: hand\-designed agents\.

## 5Analysis and Ablation

Figure 3:Effect of the behavioral\-feedback harness\. Test accuracy \(%\) of the best\-by\-validation agent found by genetic search alone \(GA\-only\) and byGRAFT\(genetic search with the harness\) on MGSM and ARC,Qwen3\.5\-35B\-A3Bas both optimizer and executor\.#### Contribution of Execution Evidence\.

Figure[3](https://arxiv.org/html/2611.23509#S5.F3)comparesGRAFTwith GA\-only on MGSM and ARC task\. GA\-only retains parent code, aggregate fitness, elite context, and the same crossover and integrated mutation instructions, but omits per\-example execution evidence\. Initialization, selection, replacement, and design budgets are identical across conditions\. We report the test accuracy of the agent selected by validation performance in each condition\. Adding the harness increases test accuracy from 56\.7% to 91\.6% on MGSM, a gain of 34\.9 percentage points, and from 56\.7% to 58\.0% on ARC, a gain of 1\.3 percentage points\.

Our diagnostics suggest that the larger observed improvement on MGSM reflects how directly execution evidence identifies actionable program changes\. For example repeated error messages may expose an answer\-type mismatch, which high\-performing offspring address by converting answers to formats that are expected by the evaluator\. A single correction can therefore prevent the same interface failure, whereas aggregate fitness alone does not identify its shared cause\. This improvement illustrates task\-specific interface adaptation, an important aspect of automated agent design\. Even when the executor does not consistently produce the expected representation, the surrounding scaffold can enforce it through explicit normalization\.

#### Search Dynamics and Test Performance\.

Figure[2](https://arxiv.org/html/2611.23509#S4.F2)compares ARC search trajectories and the test performance of the agents selected by validation accuracy, usingQwen3\.5\-35Bas the optimizer andQwen3\.5\-9Bas the executor\. The best\-so\-far validation accuracy ofGRAFTrises from approximately 43% to 63% during the first half of the plotted search and then plateaus\. On the test set\.GRAFTachieves 49\.2%, compared with 45\.0% for both ADAS and AFlow and 41\.7% for SwarmAgentic\. The experiment shows that its validation–test gap is also the smallest among the compared methods \(13\.3 percentage points\)\.

## 6Case Study: Execution Evidence on CRUXEval\-I

Figure[4](https://arxiv.org/html/2611.23509#S6.F4)follows one GRAFT run on CRUXEval\-I withgpt\-oss\-120bas the optimizer andQwen3\-8Bas the executor\. In this task, an agent receives a Python functionffand a target output and must predict an input for whichffreturns that output\. For CRUXEval\-I, each failed execution record also contains the evaluator’s message, for example the exception raised whenffis called on the predicted input\.

#### Observed failure\.

The hand\-designed agents often predicted the right values in the wrong form: they returned all arguments offfas a single tuple, so callingffraised aTypeErrorfor a missing positional argument \(Figure[4](https://arxiv.org/html/2611.23509#S6.F4)a\)\. In every seed that produced execution records, this error accounted for three to five of the six failed samples shown to the proposal LLM\. Aggregate fitness alone does not reveal this cause: it only reports that, for example, the best seed solves 43\.8% of the validation tasks\.

Figure 4:Execution evidence in a GRAFT run on CRUXEval\-I \(optimizer gpt\-oss\-120b, executor Qwen3\-8B\)\. \(a\) A failure sample from the harness evidence of a hand\-designed seed: the agent returns both arguments offfas one tuple, so callingffraises aTypeError\. \(b\) Design notes of offspring from generations 5, 6 and 10, each with a short summary, a verbatim excerpt and its validation accuracy\. \(c\) Best validation accuracy so far in each generation\. The dashed lines mark the test accuracy of best seed and best agents from automated methods\.
#### Changes in offspring\.

The error remained visible in the harness evidence of the parents selected in later generations, and the design rationales of their offspring address it directly \(Figure[4](https://arxiv.org/html/2611.23509#S6.F4)b\)\. In generation 5, the LLM added an agent that reports the number and order offf’s positional arguments \(validation accuracy 53\.9%\); in generation 6, it required the final answer to contain a single expression and never a tuple \(59\.4%\); and in generation 10, it unpacked the predicted arguments before callingff\(68\.8%\)\. In the execution records of the best offspring, the error appears in only one or two of the six failed samples\.

#### Outcome and limitations\.

The best system, found in generation 12, reaches 72\.7% validation and 68\.0% test accuracy, compared with 41\.5% for the best hand\-designed agent \(Figure[4](https://arxiv.org/html/2611.23509#S6.F4)c\)\. This example shows how the proposal LLM uses execution evidence when it is available\. In this case study we did not run a matched GA\-only search in this configuration, however we got the test accuracy for SwarmAgent, Aflow and ADAS equal to 52\.3, 64\.1, and 47\.3 respectively with the same candidate\-evaluation budget, it is worth mentioning that each generation with GRAFT equals 2 candidate evaluation since two offspring is generated in each generation\.

## 7Conclusion

#### Summary\.

We introducedGRAFT, which uses aggregate fitness to select parents and survivors, while parent\-specific execution feedback guides semantic crossover with integrated mutation\. The harness provides bounded samples of successful and failed validation runs, reusing records from fitness evaluation without exposing reference answers or requiring additional task executions\. Experiments across reasoning and programming tasks show gains over hand\-designed agents and competitive performance against automated architecture\-search methods, with rankings varying across optimizer and executor configurations\. These findings support combining evolutionary search with execution feedback for automated agent design\.

#### Limitations\.

Our evaluation covers selected reasoning and programming benchmarks under specific model configurations and search budgets\. The results do not establish performance on open\-ended, interactive, or long\-horizon agent tasks\. Moreover, evaluating the search framework across different tasks and models does not demonstrate that a discovered architecture transfers to new tasks or executors without further optimization\. Held\-out test evaluation addresses performance on unseen instances within the evaluated setting, but does not establish robustness to distribution shift\.

#### Broader Impact and Societal Considerations\.

Automated agent design may reduce the engineering effort required to build task\-specific systems and make architecture exploration more accessible to researchers and developers\. Our use of open\-weight models supports reproducibility and local deployment, including settings where task inputs and execution records need to remain within controlled infrastructure\. Automating architecture discovery could also facilitate the development of agents for harmful or unauthorized activities\. Deployment decisions should therefore consider misuse risks and performance under realistic conditions, with human oversight appropriate to the application\.

## References

- Agarwalet al\.\(2025\)S\. Agarwal, L\. Ahmad, J\. Ai, S\. Altman, A\. Applebaum, E\. Arbus, R\. K\. Arora, Y\. Bai, B\. Baker, H\. Bao,et al\.Gpt\-oss\-120b & gpt\-oss\-20b model card\.arXiv preprint arXiv:2508\.10925\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px3.p1.1)\.
- Agrawalet al\.\(2026\)L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang,et al\.Gepa: reflective prompt evolution can outperform reinforcement learning\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 8479–8565\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p3.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px3.p1.1)\.
- Austinet al\.\(2021\)J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le,et al\.Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px1.p1.1)\.
- Bajoriaet al\.\(2026\)S\. Bajoria, S\. Ranjan, A\. M, V\. Suri, V\. Hassija, V\. Chamola, and A\. HussainFrom language models to agentic ai: a survey of autonomous, action\-enabled, and collaborative llm agents\.Cognitive Computation18\(1\),pp\. 103\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p1.1)\.
- Chenget al\.\(2024\)C\. Cheng, A\. Nie, and A\. SwaminathanTrace is the next autodiff: generative optimization with rich feedback, execution traces, and llms\.Advances in Neural Information Processing Systems37,pp\. 71596–71642\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p3.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px3.p1.1)\.
- Chollet \(2019\)F\. CholletOn the measure of intelligence\.arXiv preprint arXiv:1911\.01547\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px1.p1.1)\.
- Duet al\.\(2026\)X\. Du, Y\. Yao, K\. Ma, B\. Wang, T\. Zheng, M\. Liu, Y\. Liang, X\. Jin, Z\. Wei, C\. Zheng,et al\.Supergpqa: scaling llm evaluation across 285 graduate disciplines\.Advances in Neural Information Processing Systems38\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px1.p1.1)\.
- Duet al\.\(2023\)Y\. Du, S\. Li, A\. Torralba, J\. B\. Tenenbaum, and I\. MordatchImproving factuality and reasoning in language models through multiagent debate\.arXiv preprint arXiv:2305\.14325\.Cited by:[Table 1](https://arxiv.org/html/2611.23509#S4.T1.2.1.7.1),[Table 2](https://arxiv.org/html/2611.23509#S4.T2.2.1.4.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px3.p1.1)\.
- Guet al\.\(2024\)A\. Gu, B\. Rozière, H\. Leather, A\. Solar\-Lezama, G\. Synnaeve, and S\. I\. WangCruxeval: a benchmark for code reasoning, understanding and execution\.arXiv preprint arXiv:2401\.03065\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px1.p1.1)\.
- Gullí \(2025\)A\. GullíAgentic design patterns: a hands\-on guide to building intelligent systems\.Springer Nature\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p1.1)\.
- Guoet al\.\(2023\)Q\. Guo, R\. Wang, J\. Guo, B\. Li, K\. Song, X\. Tan, G\. Liu, J\. Bian, and Y\. YangEvoprompt: connecting llms with evolutionary algorithms yields powerful prompt optimizers\.arXiv preprint arXiv:2309\.08532\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p3.1)\.
- Helmuthet al\.\(2014\)T\. Helmuth, L\. Spector, and J\. MathesonSolving uncompromising problems with lexicase selection\.IEEE Transactions on Evolutionary Computation19\(5\),pp\. 630–643\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px3.p1.1)\.
- Huet al\.\(2025\)S\. Hu, C\. Lu, and J\. CluneAutomated design of agentic systems\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 21344–21377\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p2.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2611.23509#S4.T1.2.1.12.1),[Table 2](https://arxiv.org/html/2611.23509#S4.T2.2.1.8.1)\.
- Khattabet al\.\(2024\)O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. Haq, A\. Sharma, T\. Joshi, H\. Moazam, H\. Miller,et al\.Dspy: compiling declarative language model calls into state\-of\-the\-art pipelines\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 54928–54958\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px1.p1.1)\.
- Langeet al\.\(2026\)R\. Lange, Y\. Imajuku, and E\. CetinShinkaevolve: towards open\-ended and sample\-efficient program evolution\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 74026–74078\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px2.p1.1)\.
- Leeet al\.\(2025\)K\. Lee, I\. Fischer, Y\. Wu, D\. Marwood, S\. Baluja, D\. Schuurmans, and X\. ChenEvolving deeper llm thinking\.arXiv preprint arXiv:2501\.09891\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p3.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px3.p1.1)\.
- Leeet al\.\(2026\)Y\. Lee, R\. Nair, Q\. Zhang, K\. Lee, O\. Khattab, and C\. FinnMeta\-harness: end\-to\-end optimization of model harnesses\.arXiv preprint arXiv:2603\.28052\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p3.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px3.p1.1)\.
- Lehmanet al\.\(2023\)J\. Lehman, J\. Gordon, S\. Jain, K\. Ndousse, C\. Yeh, and K\. O\. StanleyEvolution through large models\.InHandbook of evolutionary machine learning,pp\. 331–366\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2023\)J\. Liu, C\. S\. Xia, Y\. Wang, and L\. ZhangIs your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation\.Advances in neural information processing systems36,pp\. 21558–21572\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px1.p1.1)\.
- Liuet al\.\(2026\)S\. Liu, S\. Agarwal, M\. Maheswaran, M\. Cemri, Z\. Li, Q\. Mang, A\. Naren, E\. Boneh, A\. Cheng, M\. Z\. Pan,et al\.Evox: meta\-evolution for automated discovery\.arXiv preprint arXiv:2602\.23413\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p3.1)\.
- Luet al\.\(2025\)C\. Lu, S\. Hu, and J\. CluneIntelligent go\-explore: standing on the shoulders of giant foundation models\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 21276–21301\.Cited by:[Table 1](https://arxiv.org/html/2611.23509#S4.T1.2.1.9.1),[Table 2](https://arxiv.org/html/2611.23509#S4.T2.2.1.3.1)\.
- Madaanet al\.\(2023\)A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang,et al\.Self\-refine: iterative refinement with self\-feedback\.Advances in neural information processing systems36,pp\. 46534–46594\.Cited by:[Table 1](https://arxiv.org/html/2611.23509#S4.T1.2.1.6.1)\.
- Meyersonet al\.\(2024\)E\. Meyerson, M\. J\. Nelson, H\. Bradley, A\. Gaier, A\. Moradi, A\. K\. Hoover, and J\. LehmanLanguage model crossover: variation through few\-shot prompting\.ACM Transactions on Evolutionary Learning4\(4\),pp\. 1–40\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p2.1),[§1](https://arxiv.org/html/2611.23509#S1.p3.1),[§1](https://arxiv.org/html/2611.23509#S1.p4.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px2.p1.1)\.
- Moraglioet al\.\(2012\)A\. Moraglio, K\. Krawiec, and C\. G\. JohnsonGeometric semantic genetic programming\.InInternational Conference on Parallel Problem Solving from Nature,pp\. 21–31\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px3.p1.1)\.
- Nieet al\.\(2025\)F\. Nie, L\. Feng, H\. Ye, W\. Liang, P\. Lu, H\. Yao, A\. Alahi, and J\. ZouWeak\-for\-strong: training weak meta\-agent to harness strong executors\.arXiv preprint arXiv:2504\.04785\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px1.p1.1)\.
- Novikovet al\.\(2025\)A\. Novikov, N\. Vũ, M\. Eisenberger, E\. Dupont, P\. Huang, A\. Z\. Wagner, S\. Shirobokov, B\. Kozlovskii, F\. J\. Ruiz, A\. Mehrabian,et al\.Alphaevolve: a coding agent for scientific and algorithmic discovery\.arXiv preprint arXiv:2506\.13131\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px2.p1.1)\.
- Pryzantet al\.\(2023\)R\. Pryzant, D\. Iter, J\. Li, Y\. Lee, C\. Zhu, and M\. ZengAutomatic prompt optimization with “gradient descent” and beam search\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 7957–7968\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p3.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px3.p1.1)\.
- Romera\-Paredeset al\.\(2024\)B\. Romera\-Paredes, M\. Barekatain, A\. Novikov, M\. Balog, M\. P\. Kumar, E\. Dupont, F\. J\. Ruiz, J\. S\. Ellenberg, P\. Wang, O\. Fawzi,et al\.Mathematical discoveries from program search with large language models\.Nature625\(7995\),pp\. 468–475\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px2.p1.1)\.
- Rosser and Foerster \(2026\)J\. Rosser and J\. FoersterAgentbreeder: mitigating the ai safety risks of multi\-agent scaffolds via self\-improvement\.Advances in Neural Information Processing Systems38,pp\. 149564–149594\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p2.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px2.p1.1)\.
- Shiet al\.\(2022\)F\. Shi, M\. Suzgun, M\. Freitag, X\. Wang, S\. Srivats, S\. Vosoughi, H\. W\. Chung, Y\. Tay, S\. Ruder, D\. Zhou,et al\.Language models are multilingual chain\-of\-thought reasoners\.arXiv preprint arXiv:2210\.03057\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px1.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: Language Agents with Verbal Reinforcement Learning\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html)Cited by:[Table 2](https://arxiv.org/html/2611.23509#S4.T2.2.1.5.1)\.
- Teamet al\.\(2026\)G\. Team, S\. E\. Abd, V\. Aggarwal, R\. Algayres, A\. Andreev, O\. Bachem, I\. Ballantyne, C\. Brick, V\. Cărbune, M\. Casbon,et al\.Gemma 4 technical report\.arXiv preprint arXiv:2607\.02770\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px3.p1.1)\.
- Teamet al\.\(2025\)G\. Team, A\. Kamath, J\. Ferret, S\. Pathak, N\. Vieillard, R\. Merhej, S\. Perrin, T\. Matejovicova, A\. Ramé, M\. Rivière,et al\.Gemma 3 technical report\.arXiv preprint arXiv:2503\.19786\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px3.p1.1)\.
- Team \(2026\)Q\. TeamQwen3\. 5\-omni technical report\.arXiv preprint arXiv:2604\.15804\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px3.p1.1)\.
- Wanget al\.\(2022\)X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. ZhouSelf\-consistency improves chain of thought reasoning in language models\.arXiv preprint arXiv:2203\.11171\.Cited by:[Table 1](https://arxiv.org/html/2611.23509#S4.T1.2.1.5.1),[Table 2](https://arxiv.org/html/2611.23509#S4.T2.2.1.6.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[Table 1](https://arxiv.org/html/2611.23509#S4.T1.2.1.4.1),[Table 2](https://arxiv.org/html/2611.23509#S4.T2.2.1.2.1)\.
- Xuet al\.\(2023\)B\. Xu, A\. Yang, J\. Lin, Q\. Wang, C\. Zhou, Y\. Zhang, and Z\. MaoExpertprompting: instructing large language models to be distinguished experts\.arXiv preprint arXiv:2305\.14688\.Cited by:[Table 1](https://arxiv.org/html/2611.23509#S4.T1.2.1.10.1),[Table 2](https://arxiv.org/html/2611.23509#S4.T2.2.1.7.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px3.p1.1)\.
- Yanget al\.\(2021\)S\. Yang, Y\. Tian, C\. He, X\. Zhang, K\. C\. Tan, and Y\. JinA gradient\-guided evolutionary approach to training deep neural networks\.IEEE Transactions on Neural Networks and Learning Systems33\(9\),pp\. 4861–4875\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p3.1)\.
- Yaoet al\.\(2023\)S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. Griffiths, Y\. Cao, and K\. NarasimhanTree of thoughts: deliberate problem solving with large language models\.Advances in neural information processing systems36,pp\. 11809–11822\.Cited by:[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px1.p1.1)\.
- Yeet al\.\(2024\)H\. Ye, J\. Wang, Z\. Cao, F\. Berto, C\. Hua, H\. Kim, J\. Park, and G\. SongReevo: large language models as hyper\-heuristics with reflective evolution\.Advances in neural information processing systems37,pp\. 43571–43608\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p3.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px3.p1.1)\.
- Yinet al\.\(2025\)X\. Yin, X\. Wang, L\. Pan, L\. Lin, X\. Wan, and W\. Y\. WangGödel agent: a self\-referential agent framework for recursively self\-improvement\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 27890–27913\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px1.p1.1)\.
- Yuksekgonulet al\.\(2025\)M\. Yuksekgonul, F\. Bianchi, J\. Boen, S\. Liu, P\. Lu, Z\. Huang, C\. Guestrin, and J\. ZouOptimizing generative ai by backpropagating language model feedback\.Nature639\(8055\),pp\. 609–616\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p3.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px3.p1.1)\.
- Zhanget al\.\(2025a\)G\. Zhang, L\. Niu, J\. Fang, K\. Wang, L\. Bai, and X\. WangMulti\-agent architecture search via agentic supernet\.arXiv preprint arXiv:2502\.04180\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2026\)J\. Zhang, S\. Hu, C\. Lu, R\. Lange, and J\. CluneDarwin gödel machine: open\-ended evolution of self\-improving agents\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 104223–104294\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2025b\)J\. Zhang, J\. Xiang, Z\. Yu, F\. Teng, X\. Chen, J\. Chen, M\. Zhuge, X\. Cheng, S\. Hong, J\. Wang,et al\.Aflow: automating agentic workflow generation\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 34040–34077\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p2.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2611.23509#S4.T1.2.1.13.1),[Table 2](https://arxiv.org/html/2611.23509#S4.T2.2.1.9.1)\.
- Zhanget al\.\(2025c\)Y\. Zhang, C\. Lin, S\. Tang, H\. Chen, S\. Zhou, Y\. Ma, and V\. TrespSwarmagentic: towards fully automated agentic system generation via swarm intelligence\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 1778–1818\.Cited by:[§1](https://arxiv.org/html/2611.23509#S1.p2.1),[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2611.23509#S4.SS1.SSS0.Px2.p1.1),[Table 1](https://arxiv.org/html/2611.23509#S4.T1.2.1.14.1),[Table 2](https://arxiv.org/html/2611.23509#S4.T2.2.1.10.1)\.
- Zhenget al\.\(2024\)H\. S\. Zheng, S\. Mishra, X\. Chen, H\. Cheng, E\. H\. Chi, Q\. V\. Le, and D\. ZhouTake a step back: evoking reasoning via abstraction in large language models\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 20279–20316\.Cited by:[Table 1](https://arxiv.org/html/2611.23509#S4.T1.2.1.8.1)\.
- Zhugeet al\.\(2024\)M\. Zhuge, W\. Wang, L\. Kirsch, F\. Faccio, D\. Khizbullin, and J\. SchmidhuberLanguage agents as optimizable graphs\.arXiv preprint arXiv:2402\.16823\.Cited by:[§2](https://arxiv.org/html/2611.23509#S2.SS0.SSS0.Px1.p1.1)\.

## Appendix ABenchmark Details

For each benchmark we fix a validation split, used as the fitness signal during search, and a disjoint test split, used only to evaluate the final selected system\. All methods use the same splits within a configuration\. TableGRAFT: Execution\-Guided Genetic Recombination for Agent Designlists the splits and scoring rules\.

Table 4:Benchmark splits and scoring\.
## Appendix BHand\-Designed Reference Agents

The reference agents are the hand\-designed programs released with ADAS\. They form the initial population of ADAS andGRAFT, and are evaluated without search on the test split with the same executor as the searched systems\. Table[B](https://arxiv.org/html/2611.23509#A2)summarizes them\.

Table 5:Hand\-designed reference agents\. Roles are task specific \(for MGSM, e\.g\., Math Professor, Grade School Teacher\)\.

## Appendix CImplementation Details

#### Common protocol\.

All models are open\-weight and served with vLLM\. Within each configuration, all methods use the same validation and test splits, executor, and scoring rule\. The candidate\-evaluation budget is 30 for most of the experiments, however it could be variable for optimizing and agent for a specific task\. Also, for SwarmAgentic’s larger budget described below, because of keeping the method’s default optimization process\. Each method returns the system with the highest validation fitness, which is evaluated once optimization finished on the test split\.

#### GRAFT\.

The population holdsN=5N=5programs, initialized with the hand\-designed agents\. Each iteration selects parents by tournaments of sizek=3k=3and proposesλ=2\\lambda=2offspring, for 15 iterations mostly to use 30 candidate evaluation budget\. The harness keeps up to three successful and six failed executions per program, taken in validation order, with inputs and outputs truncated to a task\-specific length\.

#### Baselines\.

ADASuses the released implementation with the same initial agents, running 30 iterations of archive\-conditioned proposal\.AFlowstarts from its blank workflow template and runs 30 optimization rounds with task\-appropriate operators \(e\.g\., Custom, ScEnsemble, and Programmer for mathematics\)\. The best\-validation round is tested\.SwarmAgenticuses its released default schedule of five particles and ten iterations \(50 evaluated systems\) and returns its global best system\. This exceeds the 30\-proposal budget of the other methods\.

## Appendix DAlgorithm

Algorithm[D](https://arxiv.org/html/2611.23509#A4)summarizes the complete GRAFT search procedure\.

Algorithm 2AI Agentic System Search withGradient guided\-LMX, GRAFT

Require:Population sizeN≥2N\\geq 2, iteration limitnn, proposal slotsλ\\lambda, tournament sizekk, GG\-LMX, and fixed revision, retry, budget, and stopping policies\.

Similar Articles