HeurEvo: Agentic Evolution of Hybrid Solver-Augmented Heuristics for Time-Critical Mathematical Optimization

# Agentic Evolution of HybridSolver-Augmented Heuristics forTime-Critical Mathematical Optimization
Source: [https://arxiv.org/html/2609.36303](https://arxiv.org/html/2609.36303)
## HeurEvo: Agentic Evolution of Hybrid Solver\-Augmented Heuristics for Time\-Critical Mathematical Optimization

Hugo BarbalhoAffiliation:Purdue University Microsoft ResearchKonstantina MellouAffiliation:Purdue University Microsoft ResearchMarco MolinaroAffiliation:Purdue University Microsoft ResearchJing GaoIshai MenacheAffiliation:Purdue University Microsoft ResearchXinzhi ZhangAffiliation:Purdue University Microsoft ResearchSirui LiAffiliation:Purdue University Microsoft Research

###### Abstract

Recent advances in agentic heuristic design use AI agents and execution feedback to automate algorithm discovery for challenging optimization problems\. In many practical settings, high\-quality solutions must be obtained under strict runtime constraints, motivating hybrid approaches that combine problem\-specific heuristics with powerful mathematical programming solvers\. However, existing approaches typically improve heuristic components within predefined procedures or tune solver configurations in isolation\. This limits holistic adaptation of where to allocate computation, how to leverage solvers, and how to refine the overall algorithmic structure\. To address these limitations, we propose HeurEvo, anautomated plan–code–component co\-evolution frameworkthat jointly evolves the high\-level algorithmic structures, their implementations, and a shared pool of reusable components\. A planner determines which algorithmic components to use, how to combine them, and how to allocate runtime across stages, a coder realizes the resulting plan as executable code, while a component evolver updates the shared component pool\. Within an island\-based evolutionary framework, plans and implementations co\-evolve with feedback from an interpreter agent that analyzes execution results and identifies opportunities for improvement\. Across diverse combinatorial optimization benchmarks and challenging MIPLIB instances, HeurEvo finds high\-quality solutions within tight runtime budgets, often matching or surpassing state\-of\-the\-art optimization solvers given hours or days of computation\. On several nonlinear geometry problems such as hexagon packing, it also improves upon the best previously reported results\. These results highlight the value of jointly searching over algorithmic structure and implementation for agentic heuristic design\.

## 1Introduction

Mathematical optimization supports decisions in production scheduling, delivery routing, and resource allocation, where a solution is useful only if it can be obtained in time\. Under a short runtime budget, a practical approach is to combine problem\-specific heuristics and mathematical programming solvers in a multi\-step algorithm[Talbi \(2002\)](https://arxiv.org/html/2609.36303#bib.bib52);[Fischetti & Lodi \(2003\)](https://arxiv.org/html/2609.36303#bib.bib15)\. Heuristics can quickly construct or repair solutions, while a solver can optimize selected subproblems or polish a promising incumbent within the broader search[Danna et al\. \(2005\)](https://arxiv.org/html/2609.36303#bib.bib8)\. Such hybrid algorithms have proved useful in practical applications, such as industrial production planning[Lee et al\. \(2023\)](https://arxiv.org/html/2609.36303#bib.bib30)\. Their success, however, depends on designing a complete solver\-augmented algorithm suited to both the problem and its available solving time\.

Recent methods use AI agents to automate heuristic and algorithm discovery, as well as solver configuration tuning\. However, prior automatic heuristic\-design methods typically restrict their search to individual components, such as scoring or update rules, within a fixed algorithmic framework such as ant colony optimization \(ACO\) or guided local search \(GLS\)[Liu et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib34);[Ye et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib59);[Wu et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib55)\. Solver\-oriented work, in turn, searches over solver configurations while retaining the solver’s overall procedure[Luo et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib36)\. General\-purpose code\-evolution methods[Cemri et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib7);[Liu et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib35);[Sharma \(2025\)](https://arxiv.org/html/2609.36303#bib.bib49)broaden the search space by modifying entire programs, but they do not explicitly separate high\-level algorithm structure from low\-level implementation\. This leaves the design of complete hybrid optimization algorithms underexplored, especially how to combine heuristic and solver methods and allocate a shared runtime budget among them, which motivates our research question:

*How can agents jointly evolve the composition and implementation of hybrid optimization algorithms to obtain better solutions within a fixed execution\-time budget?*

We answer this question withHeurEvo, an automatedplan–code–component co\-evolution frameworkthat jointly evolves high\-level algorithmic plans, their executable implementations, and a shared pool of reusable components\. A planner constructs and revises plans by selecting components, ordering their subgoals, and allocating runtime across stages\. A coder implements each plan and refines its code and runtime allocation, while the planner revisits the algorithmic composition when code\-level improvements stall\. A component evolver uses experience accumulated during search to refine the shared component pool, allowing subsequent plans to build on improved components\. An interpreter analyzes execution results and step traces to guide this evolution\. Within an island\-based evolutionary framework, different plans and their implementations evolve in separate populations while sharing knowledge through the component pool\.

Figure[1](https://arxiv.org/html/2609.36303#S1.F1)illustrates the resulting behavior\. The evolved hybrid algorithm first combines greedy construction and large neighborhood search \(LNS\) to obtain a promising solution, then uses Gurobi to optimize selected subproblems, followed by solver\-based polishing and local cleanup\. By jointly choosing these components, their ordering, and their runtime allocation,HeurEvomakes effective use of the available execution budget and achieves a lower objective gap than the illustrated programs produced by whole\-program evolution and ReEvo’s fixed ACO framework with a single LLM\-evolved subcomponent, which is the heuristic function guiding node selection[Ye et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib59)\.

Our goal is to discover fast hybrid heuristics that generalize across related problem instances, producing high\-quality solutions within a tight execution budget \(two minutes in our experiments\) without rerunning the discovery process for each new instance\. We evaluateHeurEvoon 11 synthetic tasks, six MIPLIB\-NL\-derived problems[Li et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib32), and four nonlinear geometry families\. On the synthetic tasks,HeurEvoachieves mean gaps of−2\.22%\-2\.22\\%on training instances and−1\.45%\-1\.45\\%on held\-out test instances relative to Gurobi’s 48\-hour incumbents[Gurobi Optimization, LLC \(2026\)](https://arxiv.org/html/2609.36303#bib.bib21), outperforming these incumbents on eight training tasks and seven test tasks\. On the MIPLIB\-NL problems,HeurEvoimproves upon AdaEvolve on four of six training problems\. For nonlinear geometry, combining aHeurEvophase with subsequent AdaEvolve refinement produces several objectives numerically better than the reported reference values\. Ablation studies further support the benefit of combining adaptive plan revision with component evolution\.

##### Our contributions\.

- •Structured search over complete hybrid algorithms\.We formulate automated heuristic design as an explicit search over complete solver\-augmented algorithms\. An evolvable plan selects heuristic and solver components, specifies their execution order and subgoals, and allocates a shared runtime budget across them\. This makes algorithm composition a first\-class search variable\.
- •Plan–code–component co\-evolution\.We introduceHeurEvo, a plan–code–component co\-evolution framework that jointly evolves algorithmic plans, their executable implementations, and a shared pool of reusable components\. Code evolution improves implementations under the current plan; adaptive plan review revises the algorithmic structure when code\-level progress stalls; and component evolution refines reusable components using execution experience accumulated across islands\. Together, these mechanisms allow experience from previous executions to improve both current algorithms and future planning\.
- •Strong empirical performance under tight runtime budgets\.Across synthetic combinatorial optimization, MIPLIB\-NL\-derived, and nonlinear geometry benchmarks,HeurEvoachieves strong performance with a two\-minute execution budget, often matching or improving upon far longer\-running solver baselines and prior code\-evolution methods\. It also contributes to new best reported results on several nonlinear geometry problems, with ablations supporting the value of adaptive plan review and component evolution\.

Figure 1:Comparison of heuristics generated from HeurEvo, AdaEvolve, and ReEvo \(ACO\) for an orienteering problem\.Left:Stages \(or steps\) of the selected heuristic programs; bold values at the bottom report their measured objective gaps \(lower is better\)\.Right:Objective\-gap trajectories on the same orienteering instance\. AdaEvolve’s program remains inactive after 52\.2 seconds\.

## 2Related Work

##### Island\-based evolutionary program search\.

LLM\-guided evolutionary program search has emerged as a general paradigm for automated algorithm discovery, from seminal systems such as FunSearch[Romera\-Paredes et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib48)and AlphaEvolve[Novikov et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib44)to a growing set of follow\-up frameworks[Lehman et al\. \(2023\)](https://arxiv.org/html/2609.36303#bib.bib31);[Lange et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib29);[Jiang et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib22);[Wang et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib54);[Assumpção et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib2);[Khrulkov et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib23);[Wan et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib53)\. At their core, these frameworks use LLMs to generate candidate programs and execution feedback to iteratively improve them through population\-based search\. Island\-based variants extend this process across multiple semi\-independent populations, preserving diverse search trajectories and reducing premature convergence\. Notably, OpenEvolve[Sharma \(2025\)](https://arxiv.org/html/2609.36303#bib.bib49)provides a competitive open\-source implementation that combines MAP\-Elites[Mouret & Clune \(2015\)](https://arxiv.org/html/2609.36303#bib.bib41), exploratory and elite parent sampling, and migration across islands\. AdaEvolve[Cemri et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib7)and EvoX[Liu et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib35)further extend this paradigm: AdaEvolve adaptively controls exploration and evaluation allocation while introducing high\-level tactics to redirect stalled search, whereas EvoX co\-evolves candidate solutions with search policies that determine which candidates to build on and how new variants are generated\.

Our work differs primarily in the object being evolved\. Rather than treating the candidate primarily as a program,HeurEvoexplicitly represents a hybrid optimization algorithm as a composition of heuristic and solver components together with an executable implementation\. It then co\-evolves the components, their composition into a plan, and the corresponding code, allowing execution feedback to revise both algorithmic structure and implementation\.

##### Automatic Heuristic Design for Optimization Problems\.

A growing body of work applies LLM\-based automatic heuristic design to optimization problems[Liu et al\. \(2023\)](https://arxiv.org/html/2609.36303#bib.bib33);[Stein & Bäck \(2025\)](https://arxiv.org/html/2609.36303#bib.bib50);[Dat et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib10);[Yao et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib57);[Yang et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib56);[Kuang et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib27)\. Early approaches such as FunSearch[Romera\-Paredes et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib48)and EoH[Liu et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib34)demonstrated that LLMs can iteratively discover effective optimization heuristics through program generation and execution feedback\. ReEvo[Ye et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib59)augments this process with short\- and long\-term reflection, MCTS\-AHD[Zheng et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib60)uses Monte Carlo tree search to explore promising heuristic lineages, and RefineEvo[Wu et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib55)adaptively selects and refines evolutionary operators based on search progress\. More recent work broadens the unit of evolution beyond a single heuristic component\. MILP\-Evo[Nie et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib43)jointly evolves cut\-selection and branching rules within a branch\-and\-cut solver\. MOTIF[Kiet et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib24)jointly optimizes multiple interacting algorithmic components, while MuEvo[Lv et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib37)co\-evolves heuristic ensembles and adaptively manages their constituent components\. These methods substantially expand the search space, but operate within predefined algorithmic roles or solver structures\. At the other extreme, concurrent work such as ATLAS[Yazdani et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib58)evolves entire executable programs, allowing broad structural changes but leaving algorithm composition implicit in the code\.

HeurEvoinstead makes the structure of a complete hybrid optimization algorithm an explicit object of evolution\. Its plan can select heterogeneous heuristic and solver components, organize them into ordered steps with explicit subgoals, and allocate a shared runtime budget across them, while code evolution separately refines how each step is implemented\. Component evolution further updates the reusable building blocks from which future plans are constructed\.

## 3Method

### 3\.1Problem Formulation and Hybrid Algorithm Composition

##### Problem formulation\.

Let𝒫\\mathcal\{P\}denote a distribution over instances of a target optimization problem family\. We construct a training set𝒟=\{dj\}j=1N\\mathcal\{D\}=\\\{d\_\{j\}\\\}\_\{j=1\}^\{N\}ofNNinstances, wheredj∼𝒫d\_\{j\}\\sim\\mathcal\{P\}\.

Given𝒟\\mathcal\{D\}and a per\-instance runtime budgetBB, our objective is to discover an executable program that achieves high average solution quality\. We represent a hybrid algorithmGGas an ordered composition ofmmheuristic or solver operations, i\.e\.G=g1⊙g2⊙⋯⊙gmG=g\_\{1\}\\odot g\_\{2\}\\odot\\cdots\\odot g\_\{m\}, where⊙\\odotdenotes sequential composition and intermediate results are passed between steps\. LetTi​\(G,d\)T\_\{i\}\(G;d\)denote the runtime of each operationgig\_\{i\}when executed as part ofGGon instancedd, including its associated overhead, and letBBdenote the total runtime budget\. Furthermore, letf⁡\(G,d\)f\(G;d\)denote the fitness score of the returned solution, with larger values indicating better solution quality\. Under a fixed execution environment, we formulate the program search problem as

maxGF⁡\(G\)=1\|𝒟\|​∑d∈𝒟f⁡\(G,d\)s\.t\.T⁡\(G,d\)=∑i=1mTi​\(G,d\)≤B,∀d∈𝒟\.\\displaystyle\\max\_\{G\}\\quad F\(G\)=\\frac\{1\}\{\|\\mathcal\{D\}\|\}\\sum\_\{d\\in\\mathcal\{D\}\}f\(G;d\)\\qquad\\text\{s\.t\.\}\\qquad T\(G;d\)=\\sum\_\{i=1\}^\{m\}T\_\{i\}\(G;d\)\\leq B,\\;\\;\\forall d\\in\\mathcal\{D\}\.\(1\)

##### Component, plan, and code\.

A*component*is a reusable high\-level optimization method that admits multiple concrete implementations, such as greedy construction, local search, or calls to a mathematical programming solver\. The*component library*contains a set of such reusable components\. A*plan*specifies how these components are composed into an algorithm\. It selects and orders a sequence of components and, for each component, specifies \(1\) a*subgoal*, i\.e\., the intermediate outcome the component should achieve; \(2\) implementation guidance, such as the neighborhood used for local search; and \(3\) a time budget\. We refer to a component together with these specifications as a*step*\. Since steps are executed sequentially, we index them from 1 to the total number of steps\.

Executable*code*instantiates the plan as an algorithmGG, passing intermediate results between steps while respecting the total per\-instance runtime budgetBB\. Code evolution may modify implementation guidance and per\-step time allocations while preserving the*plan structure*, defined as the ordered sequence of components and their associated subgoals\. Changes to this structure require plan evolution\. Table[1](https://arxiv.org/html/2609.36303#S3.T1)shows an example plan produced during an actual traveling salesperson problem \(TSP\) run\.

Table 1:An example plan for the traveling salesperson problem\.StepComponentSubgoalImplementation detailsTime budget1Euclidean tour constructionProduce feasible and diverse incumbentsBuild a small portfolio of nearest\-neighbor and insertion tours; retain a diverse elite set99s2Local tour improvementRapidly reduce tour length with cheap movesApply candidate\-restricted 2\-opt and Or\-opt descent; perturb and restart only after stagnation2020s3Candidate\-edge restricted MIPEscape local minima through nonlocal edge changesWarm\-start Gurobi on a sparse graph containing proximity, incumbent, elite, and repair arcs3232s4Large\-neighborhood searchRepair costly regions missed by the sparse global searchFix most of the incumbent, destroy a long\-edge or geographic region, and use Gurobi to reconnect that region exactly4242s5Residual anytime primal portfolioUse the residual budget on the most productive search modeAdaptively select among Gurobi LNS micro\-repair, Gurobi sparse\-MIP retry, and heuristic local\-search restart using same\-run progressRemaining time

### 3\.2Our Plan\-Code\-Component Co\-Evolution Framework

The representation above makes explicit three coupled design decisions: which components to use for a given problem, how to compose them, and how to implement each step under a shared runtime budget\. To address these coupled decisions, we introduceHeurEvo, a*plan–code–component co\-evolution framework*that jointly evolves the components themselves, the plans that compose them, and the code that implements those plans\.

To structure this search, we initialize a shared*component library*and generate alternative plans from it\. Each plan defines a plan\-conditioned*island*whose programs share the same ordered subgoals and selected components\. Islands maintain separate active code populations and plan\-specific feedback while sharing the component library and an archive of evaluated programs\. Figure[2](https://arxiv.org/html/2609.36303#S3.F2)\(left\) summarizes the resulting evolution workflow\.

Figure 2:HeurEvoworkflow and examples of its three levels of evolution\.Left:Simplified workflow\. UCB selects an island for code evaluation; Interpreter feedback guides code refinement, plan revision when progress stalls, and updates to the shared component library\.Right:Example edits\.*Top:*Plan evolution replaces a stalled mixed\-integer programming \(MIP\) step with large\-neighborhood search \(LNS\)\.*Middle:*Code evolution changes Gurobi’sMIPFocussetting while preserving the plan structure\.*Bottom:*Component evolution refines reusable LNS guidance to encourage balanced use of removal rules\.The examples in Figure[2](https://arxiv.org/html/2609.36303#S3.F2)\(right\) illustrate three complementary levels of evolution:

- •Component\-level evolutionaims to refine the high\-level search space\. Based on accumulated feedback and evaluation evidence, the*component evolver*revises existing component descriptions, introduces previously absent components, and removes components that remain ineffective across evaluated plans and implementations\.
- •Plan\-level evolutionaims to discover more effective compositions of heuristics and solver operations when code\-level improvements plateau\. It may add, remove, replace, or reorder steps, or change their subgoals or selected components\. The*planner*considers programs under the current plan, the performance of plans on other islands, and feedback accumulated from code\-level evolution on the current island\. It proposes a revised plan with preliminary implementation procedures, which the Plan\-based coding agent \(coder\) translates into a new executable code\.
- •Code\-level evolutionaims to improve solution quality through better step implementations under a fixed plan\. Given a parent code, inspiration programs, and feedback from previous evaluations, the*coder*generates a child while preserving the plan’s ordered steps\. It may revise concrete algorithms, data structures, parameters, solver settings, stopping rules, time allocations, and interactions between adjacent steps\.

Detailed agent specifications are provided in Appendix[C\.2](https://arxiv.org/html/2609.36303#A3.SS2); the prompt templates for initial code generation and plan\-aware code evolution are given in Appendices[D\.3](https://arxiv.org/html/2609.36303#A4.SS3)and[D\.4](https://arxiv.org/html/2609.36303#A4.SS4), respectively\.

##### Evaluation and interpreter feedback\.

Each child is evaluated on𝒟\\mathcal\{D\}under budgetBBand checked for feasibility and execution failures\. Based on these evaluation results, the Interpreter compares each validated child with its sampled parent in terms of objective quality, feasibility, runtime, material code changes, and step\-level execution traces\. Code\-level feedback guides subsequent code updates on the selected island; plan\-level feedback accumulates for the island’s next plan review; and component\-level evidence can be aggregated across islands\. Separately, scheduling progress is measured against the island’s best current\-plan code rather than the sampled parent\. Before the next update, the selected island’s population, feedback, and scheduling statistics are updated\. Retained programs and their reports enter the shared archive, whereas the active plans and populations of other islands remain unchanged\. Appendix[C\.2](https://arxiv.org/html/2609.36303#A3.SS2.SSS0.Px8)provides further details about the Interpreter, while Section[3\.3](https://arxiv.org/html/2609.36303#S3.SS3)describes how these updates are scheduled\.

### 3\.3Island\-Based Evolution Procedure

HeurEvomaintainsKKislands, each associated with a current plan and an active population of validated programs\. Following AdaEvolve\([Cemri et al\., 2026](https://arxiv.org/html/2609.36303#bib.bib7)\), we use an upper confidence bound \(UCB\) rule to allocate search across islands\. Our controller additionally determines whether the selected island should continue improving code under its current plan or revise the plan itself\. At initialization, each island is assigned a compositional plan together with a valid seed code \(see Appendix[C\.3](https://arxiv.org/html/2609.36303#A3.SS3)\)\.

##### Island selection\.

Before each round, letr¯k\\bar\{r\}\_\{k\}denote the discounted average reward for islandkkandnkn\_\{k\}its cumulative number of completed code updates\. Letnsum=∑k=1Knkn\_\{\\mathrm\{sum\}\}=\\sum\_\{k=1\}^\{K\}n\_\{k\}be the total count across islands\. We select the next island ask⋆=arg⁡maxk∈\{1,…,K\}⁡\[r¯k\+C​log⁡nsumnk\],k^\{\\star\}=\\arg\\max\_\{k\\in\\\{1,\\ldots,K\\\}\}\\left\[\\bar\{r\}\_\{k\}\+C\\sqrt\{\\frac\{\\log n\_\{\\mathrm\{sum\}\}\}\{n\_\{k\}\}\}\\right\],where the first term favors exploitation of islands that have recently earned higher rewards, while the second promotes exploration of islands that have been visited less often;CCcontrols the balance between the two\. Initialization ensuresnk\>0n\_\{k\}\>0\. Our reward measures how much each code update helps an island close the gap to the current global best\. Any update that improves the island’s best solution receives a positive reward, with larger rewards for improvements that close a larger fraction of its gap to the current global best\. Non\-improving updates receive zero reward\. Exact reward and discounting definitions are provided in Appendix[C\.4](https://arxiv.org/html/2609.36303#A3.SS4)\.

##### Updating an island\.

We next describe the three mechanisms that govern evolution within and across islands: code refinement, plan review, and component evolution\. The first two determine how the selected island is updated, while component evolution uses accumulated experience to refine the shared component pool\.

After selecting an island, the controller decides whether to continue code\-level improvement under the current plan or to review the plan itself\. We measure recent progress by tracking whether successive code updates improve the island’s current best code\. If progress exceeds a plateau thresholdτ\\tau, the controller continuescode refinement; otherwise, it triggers aplan review\. The precise progress metric, smoothing rule, and reset behavior are described in Appendix[C\.4](https://arxiv.org/html/2609.36303#A3.SS4)\.

Code refinement\.When the plan composition remains unchanged, evolution proceeds similarly to AdaEvolve\([Cemri et al\., 2026](https://arxiv.org/html/2609.36303#bib.bib7)\)\. Parents are drawn from the selected island’s current population, while programs from other islands or earlier generations may serve as inspiration\. The coder refines the implementation while preserving the plan’s selected components and ordered subgoals\.

Plan review\.When progress stalls,HeurEvorevisits the algorithmic composition\. The planner uses accumulated feedback from the selected island, together with evidence from other islands, to propose a revised plan\. Once validated, the revised plan initializes a new population for that island, followed byW≥1W\\geq 1local code updates before global island selection resumes\. Programs from the previous plan remain available as inspiration but are no longer eligible as parents\.

Component evolution\.Experience accumulated within the islands is also used to refine the shared component pool, allowing improved component descriptions and execution feedback to transfer across islands\. These updates do not directly modify an island’s current plan, which changes only through plan review\. Algorithm[C2](https://arxiv.org/html/2609.36303#alg2)in Appendix[C\.4](https://arxiv.org/html/2609.36303#A3.SS4)presents the full evolution procedure\.

## 4Experiments

### 4\.1Experimental Setup

##### Dataset\.

We consider three families of data:1\) Standard synthetic problems\.The 11 tasks are traveling salesman \(TSP\), clustered and uniform capacitated vehicle routing \(C\-CVRP and U\-CVRP\), orienteering \(OP\), job\-shop scheduling \(JSS\), permutation flow\-shop scheduling \(PFSS\), resource\-constrained project scheduling \(RCPS\), offline bin packing \(OBP\), Max\-Cut, maximum independent set \(MIS\), and decoupling\-capacitor placement \(Decap\)\. Instances use literature\-based generation settings with task\-specific adaptations; details are in Appendix[B\.2](https://arxiv.org/html/2609.36303#A2.SS2)\.2\) MIPLIB\-derived problems\.Six complex real\-world optimization problems are selected from MIPLIB\-NL[Li et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib32): course scheduling \(comp12\-2idx,comp21\-2idx\), graph drawing \(graphdraw\-grafo2,graphdraw\-mainerd\), mining\-project selection \(opm2\-z12\-s8\), and PCB assembly\-line configuration \(sct32\)\. All six were selected because Gurobi did not certify optimality within two hours in the initial screening\. Appendix[B\.3](https://arxiv.org/html/2609.36303#A2.SS3)provides details\.3\) Nonlinear geometry problems\.We further evolve heuristics for four problems studied in AlphaEvolve[Novikov et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib44): circle packing in a square, circle packing in a rectangle, min\-max distance ratio, and hexagon packing \(Appendix[B\.4](https://arxiv.org/html/2609.36303#A2.SS4)\)\.

##### Baselines\.

We compare against six LLM\-based methods from two groups\. The first comprises general\-purpose program\-evolution frameworks, including OpenEvolve[Sharma \(2025\)](https://arxiv.org/html/2609.36303#bib.bib49), AdaEvolve[Cemri et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib7), and EvoX[Liu et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib35)\. The second comprises automatic heuristic design \(AHD\) methods, including EoH[Liu et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib34), ReEvo[Ye et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib59), and RefineEvo[Wu et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib55)\. We instantiate the AHD baselines within problem\-specific heuristic frameworks or solvers, including ant colony optimization \(ACO\)[Dorigo et al\. \(2006\)](https://arxiv.org/html/2609.36303#bib.bib11), guided local search \(GLS\)[Arnold & Sörensen \(2019\)](https://arxiv.org/html/2609.36303#bib.bib1), constructive heuristics, and Gurobi[Gurobi Optimization, LLC \(2026\)](https://arxiv.org/html/2609.36303#bib.bib21)\.

##### Evaluation Metrics\.

All evolution runs, including baselines, use a budget of 200 generated programs and a two\-minute evaluation limit per instance\. For method objectiveffand Gurobi’s 48\-hour incumbentf48​hf\_\{48\\mathrm\{h\}\}, the signed gap isg=s⋅\(f−f48​h\)/\|f48​h\|g=s\\cdot\(f\-f\_\{48\\mathrm\{h\}\}\)/\|f\_\{48\\mathrm\{h\}\}\|, withs=1s=1for minimization ands=−1s=\-1for maximization\. A positive gap indicates a worse objective than Gurobi’s incumbent, while a negative gap indicates a better objective\. For synthetic problems, we report the mean signed gap across five training instances sampled with different data seeds from the same task distribution; two validation instances select the program and four held\-out test instances evaluate it\. MIPLIB is evaluated on five training instances per task; no validation or test evaluation is reported\. Evolution is guided by the fitness functions defined in Appendix[B](https://arxiv.org/html/2609.36303#A2)\. For nonlinear problems, each heuristic evolves on an individual instance and is compared with the best objective reported in prior literature\. We use GPT\-5\.3\-Codex for standard synthetic problems and GPT\-5\.6\-sol for the more challenging MIPLIB\-derived and nonlinear geometry problems\. Implementation details are in Appendix[A](https://arxiv.org/html/2609.36303#A1.SS0.SSS0.Px1)\.

### 4\.2Main results

##### Performance on Synthetic Dataset\.

Table[2](https://arxiv.org/html/2609.36303#S4.T2)summarizes the mean objective gaps across the 11 synthetic problems, comparingHeurEvowith the code\-evolution baselines OpenEvolve, AdaEvolve, and EvoX\.HeurEvoachieves mean training and test gaps of−2\.22%\-2\.22\\%and−1\.45%\-1\.45\\%, respectively; baseline training means range from−0\.15%\-0\.15\\%to\+1\.22%\+1\.22\\%, and their reported test means are all positive \(OpenEvolve excludes OP from its test mean\)\. It discovers heuristics that produce better solutions than Gurobi does within 48 hours on 8 of the 11 training problem classes\. When transferred to the test set, these heuristics remain effective, outperforming Gurobi’s 48\-hour solutions on 7 of the 11 problem classes\. As shown in Figure[3](https://arxiv.org/html/2609.36303#S4.F3),HeurEvomakes substantial gains early in evolution and maintains the lowest mean training gap throughout the subsequent search, indicating more efficient use of the program\-generation budget\. Table[3](https://arxiv.org/html/2609.36303#S4.T3)further examines five representative problem classes spanning routing, scheduling, and graph optimization\.HeurEvoattains the best or tied\-best training gaps on all five among the full\-coverage program\-evolution baselines, with clear test\-time advantages on OP, C\-CVRP, and RCPS, showing that these gains extend beyond the training instances\. Full results, evaluation coverage, and problem\-selection details are provided in Appendix[A\.1](https://arxiv.org/html/2609.36303#A1.SS1)\.

Mean gap \(%\)Beat GurobiMethodTrainTestTrainTestOpenEvolve\+0\.71\+0\.71\+0\.41‡\+0\.41^\{\\ddagger\}5/115/114/114/11AdaEvolve−0\.15\-0\.15\+0\.91\+0\.917/117/11𝟕/𝟏𝟏\\mathbf\{7/11\}EvoX\+1\.22\+1\.22\+2\.24\+2\.245/115/114/114/11HeurEvo−2\.22\\mathbf\{\-2\.22\}−1\.45\\mathbf\{\-1\.45\}𝟖/𝟏𝟏\\mathbf\{8/11\}𝟕/𝟏𝟏\\mathbf\{7/11\}

Table 2:Average synthetic performance\.‡The OpenEvolve test mean excludes OP, whose reported test value has incomplete evaluation coverage\.

Figure 3:Training gap versus program budget\.
Table 3:Representative synthetic objective gaps \(%\)\. Since AHD methods do not introduce a host function or an initial heuristic solution for the selected problems, their results are left blank\.‡Incomplete evaluation coverage; excluded from the OpenEvolve test mean\.MethodOPC\-CVRPRCPSPFSSMISTrainValTestTrainValTestTrainValTestTrainValTestTrainValTestEoH \(GLS\)–––––––––\-2\.65\-2\.61\-2\.20–––ReEvo \(ACO\)\+4\.16\+1\.32\+7\.40\+6\.58\+9\.77\+8\.28–––––––––RefineEvo \(ACO\)–––\+5\.06\+9\.59\+8\.18–––––––––OpenEvolve\+16\.65\+16\.65\+16\.10\+16\.10\+22\.61‡\+22\.61^\{\\ddagger\}\+1\.93\+1\.93\+4\.57\+4\.57\+2\.96\+2\.96\+0\.64\+0\.64−0\.57\\mathbf\{\-0\.57\}\+1\.74\+1\.74−0\.48\-0\.48\+0\.41\+0\.41−0\.27\-0\.27−0\.04\\mathbf\{\-0\.04\}\+0\.39\+0\.39\+0\.14\\mathbf\{\+0\.14\}AdaEvolve\+16\.92\+16\.92\+15\.73\+15\.73\+21\.75\+21\.75−1\.77\-1\.77−0\.84\-0\.84−1\.62\-1\.62\+1\.49\+1\.49\+1\.21\+1\.21\+0\.70\+0\.70−2\.19\-2\.19−1\.84\-1\.84−2\.21\\mathbf\{\-2\.21\}\+0\.15\+0\.15\+0\.29\+0\.29\+0\.45\+0\.45EvoX\+17\.96\+17\.96\+16\.94\+16\.94\+21\.57\+21\.57\+4\.50\+4\.50\+2\.14\+2\.14\+2\.17\+2\.17\+2\.70\+2\.70\+3\.05\+3\.05\+0\.91\+0\.91−0\.30\-0\.30−0\.15\-0\.15−0\.71\-0\.71\+0\.47\+0\.47\+1\.17\+1\.17\+1\.88\+1\.88HeurEvo−3\.80\\mathbf\{\-3\.80\}−3\.80\\mathbf\{\-3\.80\}−1\.94\\mathbf\{\-1\.94\}−3\.36\\mathbf\{\-3\.36\}−3\.36\\mathbf\{\-3\.36\}−4\.38\\mathbf\{\-4\.38\}\+0\.12\\mathbf\{\+0\.12\}\+0\.12\+0\.120\.00\\mathbf\{0\.00\}−2\.25\-2\.25−2\.25\-2\.25−1\.85\-1\.85−0\.04\\mathbf\{\-0\.04\}−0\.04\\mathbf\{\-0\.04\}\+0\.14\\mathbf\{\+0\.14\}

##### Performance on MIPLIB Problems\.

Table[4](https://arxiv.org/html/2609.36303#S4.T4)compares mean training objectives on the six MIPLIB\-derived problems using GPT\-5\.6\-sol\.HeurEvoachieves the better objective on four of the six problems\. Relative to AdaEvolve, it reduces the objective by10\.70%10\.70\\%oncomp12\-2idx,3\.60%3\.60\\%ongraphdraw\-grafo2, and4\.71%4\.71\\%ongraphdraw\-mainerd, and increases it by0\.80%0\.80\\%on the maximization problemopm2\-z12\-s8\. AdaEvolve achieves the better objective oncomp21\-2idx\(100\.40100\.40versus103\.20103\.20\) andsct32\(−3\.01\-3\.01versus−2\.77\-2\.77\)\. Appendix[A\.2](https://arxiv.org/html/2609.36303#A1.SS2)reports the corresponding training trajectories\.

Table 4:Training objectives on MIPLIB\.Methodcomp12\-2idx↓\\downarrowcomp21\-2idx↓\\downarrowgraphdraw\-grafo2↓\\downarrowgraphdraw\-mainerd↓\\downarrowopm2\-z12\-s8↑\\uparrowsct32↓\\downarrowAdaEvolve366\.20100\.4085061\.3047940\.4058057\.20\-3\.01HeurEvo327\.00103\.2082003\.1045681\.0058519\.40\-2\.77

##### Performance on Nonlinear Problems\.

Table[5](https://arxiv.org/html/2609.36303#S4.T5)reports eight selected nonlinear task\-size configurations\. The reported results use aHeurEvophase followed by AdaEvolve refinement\. They improve on the standalone AdaEvolve baseline in all eight configurations and are numerically better than the listed reference values in five: rectangle packing withn=26n=26, the three distance\-ratio configurations, and hexagon packing withn=12n=12\. Rectangle packing withn=21n=21matches the reference at the displayed precision, while the hexagon\-packing objectives forn=11n=11andn=14n=14remain slightly worse than their references\. Table[A11](https://arxiv.org/html/2609.36303#A1.T11)reports all 14 configurations and the reference sources\. Comparisons against rounded references are limited to the reported precision and do not by themselves establish an improvement over an unrounded solution\.

Table 5:Nonlinear objectives versus reported best values\.Underlineindicates the part better than AdaEvolve;Boldindicates the part better than the reported best\-known value\.MethodCircle packing\-rectangle↑\\uparrowMin\-max distance ratio↓\\downarrowHexagon packing↓\\downarrown=21n=21n=26n=26n=16,d=2n=16,d=2n=21,d=2n=21,d=2n=22,d=2n=22,d=2n=11n=11n=12n=12n=14n=14SOTA2\.36583237592\.639320559012\.889229917\.7749919\.053983\.92450089733\.94164206754\.2689949573AdaEvolve2\.36424812672\.635531315112\.889229907717\.774980554219\.05398450983\.93267585593\.94164230044\.2723920772HeurEvo2\.36583237592\.639320564312\.889229107217\.774980341719\.05397691363\.92450113093\.94164206254\.2689950806

### 4\.3Ablation Studies

##### Effectiveness of Each Module\.

To assess the contribution of each evolutionary component in HeurEvo, we compare the following five ablation variants: 1\)*Fixed plan:*no plan or component evolution; 2\)*Component only:*evolve components while keeping plans fixed; 3\)*Always\-plan:*review plans every round with fixed components; 4\)*Always\-plan \+ component:*combine per\-round plan reviews with component evolution; 5\)*Adaptive plan only:*review plans when progress stalls, with fixed components\. Always\-plan variants review the selected island’s plan every round; adaptive review occurs only when its smoothed relative improvement falls below the plateau threshold, otherwise continuing code refinement\. The full pipeline,HeurEvo\(Ours\), combines adaptive plan review with component evolution\. As shown in Table[6](https://arxiv.org/html/2609.36303#S4.T6), the full pipeline achieves the best mean training, validation, and test gaps of−2\.22%\-2\.22\\%,−2\.22%\-2\.22\\%, and−1\.45%\-1\.45\\%, respectively\. Compared with the Adaptive plan only, it lowers the training and test gaps by0\.870\.87and1\.241\.24percentage points, respectively; its test gap is also0\.750\.75percentage points lower than the Always\-plan \+ component\. These results support combining adaptive plan review with component evolution for both training performance and generalization\.

Table 6:Component ablation ofHeurEvo\.VariantAveraged performanceOPC\-CVRPRCPSPFSSTrainValTestTrainValTestTrainValTestTrainValTestTrainValTestFixed plan−1\.19\-1\.19\+0\.06\+0\.06\+0\.25\+0\.25\+1\.94\+1\.94\+8\.1\+8\.1\+7\.97\+7\.97−3\.15\-3\.15−2\.3\-2\.3−3\.26\-3\.26\+1\.39\+1\.39\+1\.8\+1\.8\+0\.39\+0\.39−0\.83\-0\.83−1\.0\-1\.0−0\.50\-0\.50Component only−1\.45\-1\.45−0\.38\-0\.38\+0\.08\+0\.08−1\.67\-1\.67\+0\.9\+0\.9\+3\.86\+3\.86−3\.09\-3\.09−2\.4\-2\.4−3\.47\-3\.47\+1\.37\+1\.37\+1\.2\+1\.2\+1\.61\+1\.61−0\.79\-0\.79−0\.3\-0\.3−0\.69\-0\.69Always\-plan−1\.67\-1\.67−0\.36\-0\.36−0\.27\-0\.27\+0\.45\+0\.45\+8\.0\+8\.0\+8\.12\+8\.12−2\.84\-2\.84−2\.0\-2\.0−2\.95\-2\.95\+0\.50\+0\.500\.0\\mathbf\{0\.0\}\+0\.43\+0\.43−1\.52\-1\.52−1\.6\-1\.6−1\.67\-1\.67Always\-plan \+ component−1\.89\-1\.89−1\.38\-1\.38−0\.70\-0\.70−2\.67\-2\.67−8\.9\\mathbf\{\-8\.9\}−2\.39\\mathbf\{\-2\.39\}−3\.20\-3\.20−2\.6\-2\.6−3\.62\-3\.62−0\.35\\mathbf\{\-0\.35\}0\.0\\mathbf\{0\.0\}\+0\.43\+0\.43−1\.16\-1\.16−0\.9\-0\.9−0\.93\-0\.93Adaptive plan only−1\.35\-1\.35−1\.01\-1\.01−0\.21\-0\.21−0\.87\-0\.87−3\.8\-3\.8\+3\.76\+3\.76−2\.48\-2\.48−2\.2\-2\.2−3\.00\-3\.00\+0\.87\+0\.87\+0\.1\+0\.1\+0\.65\+0\.65−1\.48\-1\.48−1\.0\-1\.0−0\.64\-0\.64HeurEvo\(Ours\)−2\.22\\mathbf\{\-2\.22\}−2\.22\\mathbf\{\-2\.22\}−1\.45\\mathbf\{\-1\.45\}−3\.80\\mathbf\{\-3\.80\}−3\.80\-3\.80−1\.94\-1\.94−3\.36\\mathbf\{\-3\.36\}−3\.36\\mathbf\{\-3\.36\}−4\.38\\mathbf\{\-4\.38\}\+0\.12\+0\.12\+0\.12\+0\.120\.00\\mathbf\{0\.00\}−2\.25\\mathbf\{\-2\.25\}−2\.25\\mathbf\{\-2\.25\}−1\.85\\mathbf\{\-1\.85\}

Table 7:Island initialization ablation\.VariantAverage gap \(%\)TrainValTestSingle island−1\.31\-1\.31−0\.98\-0\.98−0\.75\-0\.75Shared plan−1\.76\-1\.76−1\.24\-1\.24−1\.23\-1\.23Partitioned library−1\.41\-1\.41−1\.69\-1\.69−1\.57\\mathbf\{\-1\.57\}HeurEvo\(Ours\)−2\.22\\mathbf\{\-2\.22\}−2\.22\\mathbf\{\-2\.22\}−1\.45\-1\.45

##### Effect of Island Count and Plan Initialization\.

Table[7](https://arxiv.org/html/2609.36303#S4.T7)compares three alternatives: 1\)*Single island:*one plan with the full component library; 2\)*Shared plan:*five islands initialized with the same plan and full library; 3\)*Partitioned library:*five distinct plans generated from separate component\-library partitions\. Our default uses five distinct initial plans generated from the full library\. It achieves the lowest training gap \(−2\.22%\-2\.22\\%\), improving on these alternatives by0\.910\.91,0\.460\.46, and0\.810\.81percentage points, respectively, supporting both initial\-plan diversity and access to the full component library\.

We include the full table of ablation experiments in Appendix[A\.4](https://arxiv.org/html/2609.36303#A1.SS4)\.

## 5Conclusion

We presentedHeurEvo, a plan–code–component co\-evolution framework for designing hybrid, solver\-augmented heuristics for time\-critical optimization\.HeurEvojointly evolves algorithmic structure, implementation, and reusable components through execution feedback, using plan\-conditioned islands, adaptive plan review, and a shared component library\. With a two\-minute execution budget, the evolved heuristics outperform Gurobi’s 48\-hour solutions on seven of eleven synthetic tasks and several competitive LLM\-based evolution baselines, with additional gains on MIPLIB and nonlinear problems\. These results demonstrate the value of jointly evolving algorithmic structure and implementation rather than optimizing heuristic components in isolation\.

## AI Use Statement

We used large language models \(LLMs\) in the proposed framework and experiments, to polish the authors’ initial manuscript draft, and to generate code for constructing optimization instances\. The authors reviewed the revised text, and the generated instances were checked against the experimental assumptions and requirements using human\-written evaluators\. Coding agents, including Codex and Claude Code, assisted with implementation, which human experts reviewed for consistency with the described framework\. The authors take full responsibility for the manuscript and research artifacts, including the accuracy of claims and results, the validity of data and code, and the correctness and attribution of references\.

## Reproducibility Statement

To support reproducibility, we document the experimental setup, models, and implementation details in Section[4\.1](https://arxiv.org/html/2609.36303#S4.SS1)and Appendix[A](https://arxiv.org/html/2609.36303#A1), with experimental hyperparameters summarized in Table[A8](https://arxiv.org/html/2609.36303#A1.T8)\. Appendix[B](https://arxiv.org/html/2609.36303#A2)describes the data sources, generation and synthesis procedures, and dataset splits, while Appendix[D](https://arxiv.org/html/2609.36303#A4)provides the prompt templates\. We will publicly release the code and datasets, including the data\-generation scripts\.

## References

- Arnold & Sörensen \(2019\)Florian Arnold and Kenneth Sörensen\.Knowledge\-guided local search for the vehicle routing problem\.*Computers & Operations Research*, 105:32–46, 2019\.
- Assumpção et al\. \(2025\)Henrique Assumpção, Diego Ferreira, Leandro Campos, and Fabricio Murai\.Codeevolve: an open source evolutionary coding agent for algorithmic discovery and optimization\.*arXiv preprint arXiv:2510\.14150*, 2025\.
- Barabási & Albert \(1999\)Albert\-László Barabási and Réka Albert\.Emergence of scaling in random networks\.*Science*, 286\(5439\):509–512, 1999\.doi:10\.1126/science\.286\.5439\.509\.URL[https://doi\.org/10\.1126/science\.286\.5439\.509](https://doi.org/10.1126/science.286.5439.509)\.
- Berthold et al\. \(2026a\)Timo Berthold, Dominik Kamp, Gioni Mexi, Sebastian Pokutta, and Imre Pólik\.Global optimization for combinatorial geometry problems revisited in the era of LLMs, 2026a\.URL[https://arxiv\.org/abs/2601\.05943v2](https://arxiv.org/abs/2601.05943v2)\.Version 2\. Table 1 gives squared distance ratios rounded upward to five decimal places\.
- Berthold et al\. \(2026b\)Timo Berthold, Dominik Kamp, Gioni Mexi, Sebastian Pokutta, and Imre Pólik\.Global optimization for combinatorial geometry problems revisited in the era of llms\.*arXiv preprint arXiv:2601\.05943*, 2026b\.
- Berthold et al\. \(2026c\)Timo Berthold, Dominik Kamp, Gioni Mexi, Sebastian Pokutta, and Imre Pólik\.Out\-of\-the\-box global optimization for packing problems: New models and improved solutions, 2026c\.URL[https://arxiv\.org/abs/2605\.04850](https://arxiv.org/abs/2605.04850)\.
- Cemri et al\. \(2026\)Mert Cemri, Shubham Agrawal, Akshat Gupta, Shu Liu, Audrey Cheng, Qiuyang Mang, Ashwin Naren, Lutfi Eren Erdogan, Koushik Sen, Matei Zaharia, et al\.Adaevolve: Adaptive llm driven zeroth\-order optimization\.*arXiv preprint arXiv:2602\.20133*, 2026\.
- Danna et al\. \(2005\)Emilie Danna, Edward Rothberg, and Claude Le Pape\.Exploring relaxation induced neighborhoods to improve MIP solutions\.*Mathematical Programming*, 102\(1\):71–90, 2005\.doi:10\.1007/s10107\-004\-0518\-7\.
- Dantzig & Ramser \(1959\)G\. B\. Dantzig and J\. H\. Ramser\.The truck dispatching problem\.*Management Science*, 6\(1\):80–91, 1959\.doi:10\.1287/mnsc\.6\.1\.80\.
- Dat et al\. \(2025\)Pham Vu Tuan Dat, Long Doan, and Huynh Thi Thanh Binh\.Hsevo: Elevating automatic heuristic design with diversity\-driven harmony search and genetic algorithm using llms\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, volume 39, pp\. 26931–26938, 2025\.
- Dorigo et al\. \(2006\)Marco Dorigo, Mauro Birattari, and Thomas Stutzle\.Ant colony optimization\.*IEEE computational intelligence magazine*, 1\(4\):28–39, 2006\.
- e Silva & Santos \(2017\)Cézar Augusto N e Silva and Haroldo Gambini Santos\.Drawing graphs with mathematical programming and variable neighborhood search\.*Electronic Notes in Discrete Mathematics*, 58:207–214, 2017\.
- Erdős & Rényi \(1959\)Paul Erdős and Alfréd Rényi\.On random graphs\. I\.*Publicationes Mathematicae Debrecen*, 6\(3\-4\):290–297, 1959\.doi:10\.5486/PMD\.1959\.6\.3\-4\.12\.URL[https://doi\.org/10\.5486/PMD\.1959\.6\.3\-4\.12](https://doi.org/10.5486/PMD.1959.6.3-4.12)\.
- Falkenauer \(1996\)Emanuel Falkenauer\.A hybrid grouping genetic algorithm for bin packing\.*Journal of Heuristics*, 2\(1\):5–30, 1996\.doi:10\.1007/BF00226291\.URL[https://doi\.org/10\.1007/BF00226291](https://doi.org/10.1007/BF00226291)\.
- Fischetti & Lodi \(2003\)Matteo Fischetti and Andrea Lodi\.Local branching\.*Mathematical programming*, 98\(1\):23–47, 2003\.
- Georgiev et al\. \(2025\)Bogdan Georgiev, Javier Gómez\-Serrano, Terence Tao, and Adam Zsolt Wagner\.Mathematical exploration and discovery at scale\.*arXiv preprint arXiv:2511\.02864*, 2025\.doi:10\.48550/arXiv\.2511\.02864\.URL[https://arxiv\.org/abs/2511\.02864](https://arxiv.org/abs/2511.02864)\.
- Gilbert \(1959\)E\. N\. Gilbert\.Random graphs\.*The Annals of Mathematical Statistics*, 30\(4\):1141–1144, 1959\.doi:10\.1214/aoms/1177706098\.URL[https://doi\.org/10\.1214/aoms/1177706098](https://doi.org/10.1214/aoms/1177706098)\.
- Gleixner et al\. \(2021\)Ambros Gleixner, Gregor Hendel, Gerald Gamrath, Tobias Achterberg, Michael Bastubbe, Timo Berthold, Philipp M\. Christophel, Kati Jarck, Thorsten Koch, Jeff Linderoth, Marco Lübbecke, Hans D\. Mittelmann, Derya Ozyurt, Ted K\. Ralphs, Domenico Salvagnin, and Yuji Shinano\.MIPLIB 2017: Data\-driven compilation of the 6th mixed\-integer programming library\.*Mathematical Programming Computation*, 13\(3\):443–490, 2021\.doi:10\.1007/s12532\-020\-00194\-3\.
- Goemans & Williamson \(1995\)Michel X\. Goemans and David P\. Williamson\.Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming\.*Journal of the ACM*, 42\(6\):1115–1145, 1995\.doi:10\.1145/227683\.227684\.
- Golden et al\. \(1987\)Bruce L\. Golden, Larry Levy, and Rakesh Vohra\.The orienteering problem\.*Naval Research Logistics*, 34\(3\):307–318, 1987\.doi:10\.1002/1520\-6750\(198706\)34:3<307::AID\-NAV3220340302\>3\.0\.CO;2\-D\.
- Gurobi Optimization, LLC \(2026\)Gurobi Optimization, LLC\.Gurobi Optimizer Reference Manual, 2026\.URL[https://www\.gurobi\.com](https://www.gurobi.com/)\.
- Jiang et al\. \(2026\)Jiachen Jiang, Tianyu Ding, and Zhihui Zhu\.Deltaevolve: Accelerating scientific discovery through momentum\-driven evolution\.*arXiv preprint arXiv:2602\.02919*, 2026\.
- Khrulkov et al\. \(2025\)Valentin Khrulkov, Andrey Galichin, Denis Bashkirov, Dmitry Vinichenko, Oleg Travkin, Roman Alferov, Andrey Kuznetsov, and Ivan Oseledets\.Gigaevo: An open source optimization framework powered by llms and evolution algorithms\.*arXiv preprint arXiv:2511\.17592*, 2025\.
- Kiet et al\. \(2026\)Nguyen Viet Tuan Kiet, Tung Dao, Cong Dao Tran, and Huynh Thi Thanh Binh\.Motif: Multi\-strategy optimization via turn\-based interactive framework\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, volume 40, pp\. 37000–37008, 2026\.
- Koch et al\. \(2011\)Thorsten Koch, Tobias Achterberg, Erling Andersen, Oliver Bastert, Timo Berthold, Robert E Bixby, Emilie Danna, Gerald Gamrath, Ambros M Gleixner, Stefan Heinz, et al\.Miplib 2010: mixed integer programming library version 5\.*Mathematical Programming Computation*, 3\(2\):103–163, 2011\.
- Kravatskiy et al\. \(2026\)Alexey Kravatskiy, Valentin Khrulkov, and Ivan Oseledets\.ImprovEvolve: Basin\-hopping meets LLM\-guided evolutionary search, 2026\.URL[https://arxiv\.org/abs/2602\.10233v2](https://arxiv.org/abs/2602.10233v2)\.Version 2, 26 June 2026\. Version 1 was titled ImprovEvolve: Ask AlphaEvolve to Improve the Input Solution and Then Improvise\.
- Kuang et al\. \(2026\)Mingen Kuang, Xudong Deng, Xi Lin, Ye Fan, Jianyong Sun, and Jialong Shi\.Llm\-driven co\-evolutionary automated heuristic design for bi\-component coupled combinatorial optimization\.*arXiv preprint arXiv:2606\.00718*, 2026\.
- Lach & Lübbecke \(2012\)Gerald Lach and Marco E Lübbecke\.Curriculum based course timetabling: new solutions to udine benchmark instances\.*Annals of Operations Research*, 194\(1\):255–272, 2012\.
- Lange et al\. \(2026\)Robert Lange, Yuki Imajuku, and Edoardo Cetin\.Shinkaevolve: Towards open\-ended and sample\-efficient program evolution\.In*International Conference on Learning Representations*, volume 2026, pp\. 74026–74078, 2026\.
- Lee et al\. \(2023\)Jonghwa Lee, Byung\-In Kim, and Sang Hun Kim\.A matheuristic algorithm for block assignment problem in long\-term production planning in the shipbuilding industry\.*Computers & Industrial Engineering*, 184:109603, 2023\.doi:10\.1016/j\.cie\.2023\.109603\.
- Lehman et al\. \(2023\)Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, and Kenneth O Stanley\.Evolution through large models\.In*Handbook of evolutionary machine learning*, pp\. 331–366\. Springer, 2023\.
- Li et al\. \(2026\)Zhong Li, Hongliang Lu, Tao Wei, Yuxuan Chen, Wenyu Liu, Yuan Lan, Fan Zhang, and Zaiwen Wen\.Constructing industrial\-scale optimization modeling benchmark\.*arXiv preprint arXiv:2602\.10450*, 2026\.
- Liu et al\. \(2023\)Fei Liu, Xialiang Tong, Mingxuan Yuan, and Qingfu Zhang\.Algorithm evolution using large language model\.*arXiv preprint arXiv:2311\.15249*, 2023\.
- Liu et al\. \(2024\)Fei Liu, Xialiang Tong, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang\.Evolution of heuristics: Towards efficient automatic algorithm design using large language model\.*arXiv preprint arXiv:2401\.02051*, 2024\.
- Liu et al\. \(2026\)Shu Liu, Shubham Agarwal, Monishwaran Maheswaran, Mert Cemri, Zhifei Li, Qiuyang Mang, Ashwin Naren, Ethan Boneh, Audrey Cheng, Melissa Z Pan, et al\.Evox: Meta\-evolution for automated discovery\.*arXiv preprint arXiv:2602\.23413*, 2026\.
- Luo et al\. \(2026\)Yidong Luo, Xuemin Chen, Chenguang Wang, Fangzhou Zhu, Tao Zhong, and Tianshu Yu\.Grimip: A general framework for instance\-specific configuration of mip solvers using llms\.*arXiv preprint arXiv:2606\.23299*, 2026\.
- Lv et al\. \(2026\)Haoze Lv, Ning Lu, Shengcai Liu, Shaofeng Zhang, and Ke Tang\.Muevo: Llm\-driven evolution of multi\-heuristic ensemble\.*arXiv preprint arXiv:2608\.03636*, 2026\.
- Manne \(1960\)Alan S\. Manne\.On the job\-shop scheduling problem\.*Operations Research*, 8\(2\):219–223, 1960\.
- Martello & Toth \(1990\)Silvano Martello and Paolo Toth\.Lower bounds and reduction procedures for the bin packing problem\.*Discrete Applied Mathematics*, 28\(1\):59–70, 1990\.
- Miller et al\. \(1960\)C\. E\. Miller, A\. W\. Tucker, and R\. A\. Zemlin\.Integer programming formulation of traveling salesman problems\.*Journal of the ACM*, 7\(4\):326–329, 1960\.doi:10\.1145/321043\.321046\.
- Mouret & Clune \(2015\)Jean\-Baptiste Mouret and Jeff Clune\.Illuminating search spaces by mapping elites\.*arXiv preprint arXiv:1504\.04909*, 2015\.
- Nemhauser & Trotter \(1975\)G\. L\. Nemhauser and L\. E\. Trotter, Jr\.Vertex packings: Structural properties and algorithms\.*Mathematical Programming*, 8:232–248, 1975\.doi:10\.1007/BF01580444\.
- Nie et al\. \(2026\)Jinbiao Nie, Kewei Feng, Xiaoyuan Zhang, Shan Yin, Zizhuo Wang, and Bin Dong\.Milp\-evo: Closed\-loop fully automatic design of milp solvers\.*arXiv preprint arXiv:2607\.18252*, 2026\.
- Novikov et al\. \(2025\)Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po\-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al\.Alphaevolve: A coding agent for scientific and algorithmic discovery\.*arXiv preprint arXiv:2506\.13131*, 2025\.
- OpenAI \(2026a\)OpenAI\.Introducing gpt\-5\.3\-codex, 2026a\.URL[https://developers\.openai\.com/api/docs/models/gpt\-5\.3\-codex](https://developers.openai.com/api/docs/models/gpt-5.3-codex)\.
- OpenAI \(2026b\)OpenAI\.GPT\-5\.6 Sol API\.[https://chatgpt\.com](https://chatgpt.com/), 2026b\.
- Pritsker et al\. \(1969\)A\. A\. B\. Pritsker, L\. J\. Watters, and P\. M\. Wolfe\.Multiproject scheduling with limited resources: A zero\-one programming approach\.*Management Science*, 16\(1\):93–108, 1969\.doi:10\.1287/mnsc\.16\.1\.93\.
- Romera\-Paredes et al\. \(2024\)Bernardino Romera\-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al\.Mathematical discoveries from program search with large language models\.*Nature*, 625\(7995\):468–475, 2024\.
- Sharma \(2025\)Asankhaya Sharma\.Openevolve: an open\-source evolutionary coding agent, 2025\.URL[https://github\.com/algorithmicsuperintelligence/openevolve](https://github.com/algorithmicsuperintelligence/openevolve)\.
- Stein & Bäck \(2025\)Niki van Stein and Thomas Bäck\.Llamea: A large language model evolutionary algorithm for automatically generating metaheuristics\.*Trans\. Evol\. Comp*, 29\(2\):331–345, April 2025\.ISSN 1089\-778X\.
- Taillard \(1993\)E\. Taillard\.Benchmarks for basic scheduling problems\.*European Journal of Operational Research*, 64\(2\):278–285, 1993\.
- Talbi \(2002\)E\-G Talbi\.A taxonomy of hybrid metaheuristics\.*Journal of heuristics*, 8\(5\):541–564, 2002\.
- Wan et al\. \(2025\)Chunhui Wan, Xunan Dai, Zhuo Wang, Minglei Li, Yanpeng Wang, Yinan Mao, Yu Lan, and Zhiwen Xiao\.Loongflow: Directed evolutionary search via a cognitive plan\-execute\-summarize paradigm\.*arXiv preprint arXiv:2512\.24077*, 2025\.
- Wang et al\. \(2025\)Yiping Wang, Shao\-Rong Su, Zhiyuan Zeng, Eva Xu, Liliang Ren, Xinyu Yang, Zeyi Huang, Xuehai He, Luyao Ma, Baolin Peng, et al\.Thetaevolve: Test\-time learning on open problems\.*arXiv preprint arXiv:2511\.23473*, 2025\.
- Wu et al\. \(2026\)Yang Wu, Junran Pan, Yifan Zhang, Ning Xu, Fanshuo Zeng, and Jian Cheng\.Refineevo: Planning\-guided heuristic evolution with bidirectional experience\.*arXiv preprint arXiv:2607\.11358*, 2026\.
- Yang et al\. \(2025\)Xianliang Yang, Ling Zhang, Haolong Qian, Lei Song, and Jiang Bian\.Heuragenix: Leveraging llms for solving complex combinatorial optimization challenges\.*arXiv preprint arXiv:2506\.15196*, 2025\.
- Yao et al\. \(2025\)Shunyu Yao, Fei Liu, Xi Lin, Zhichao Lu, Zhenkun Wang, and Qingfu Zhang\.Multi\-objective evolution of heuristic using large language model\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, volume 39, pp\. 27144–27152, 2025\.
- Yazdani et al\. \(2026\)Danial Yazdani, Mohammad Nabi Omidvar, Yuan Sun, Maksud Ibrahimov, and Xiaodong Li\.Atlas: Scaffold\-free algorithm synthesis by llms via embedding\-guided quality\-diversity search\.*arXiv preprint arXiv:2608\.15546*, 2026\.
- Ye et al\. \(2024\)Haoran Ye, Jiarui Wang, Zhiguang Cao, Federico Berto, Chuanbo Hua, Haeyeon Kim, Jinkyoo Park, and Guojie Song\.Reevo: Large language models as hyper\-heuristics with reflective evolution\.*Advances in neural information processing systems*, 37:43571–43608, 2024\.
- Zheng et al\. \(2025\)Zhi Zheng, Zhuoliang Xie, Zhenkun Wang, and Bryan Hooi\.Monte carlo tree search for comprehensive exploration in llm\-based automatic heuristic design\.*arXiv preprint arXiv:2501\.08603*, 2025\.

## Appendix AAdditional Experiments

##### Implementation Details\.

We use GPT\-5\.3\-Codex[OpenAI \(2026a\)](https://arxiv.org/html/2609.36303#bib.bib45)for the synthetic experiments and GPT\-5\.6\-sol[OpenAI \(2026b\)](https://arxiv.org/html/2609.36303#bib.bib46)for both the MIPLIB\-NL and nonlinear experiments\. All experiments are conducted on a multi\-node computing cluster, with each node equipped with 128 CPU cores and 1000 GB of memory\. During evolution, each generated program is executed single\-threaded on a single CPU core and is subject to a 120\-second runtime limit per problem instance\. Gurobi 13\.0\.2 is the only optimization solver permitted in the generated programs\. All evolution experiments, including the baselines, are limited to generating 200 programs\. For the synthetic experiments, generated programs are evaluated on the validation set during model selection\. After evolution, the program with the best validation performance is selected and evaluated once on the held\-out test set\. Table[A8](https://arxiv.org/html/2609.36303#A1.T8)summarizes the complete hyperparameter configuration used forHeurEvo\.

Table A8:Hyperparameters of HeurEvo\. Symbols follow Appendix[C](https://arxiv.org/html/2609.36303#A3); a dash marks a quantity the analysis does not name\. Values describe the default configuration; the ablation studies vary the specified components\.SymbolDescriptionValueSearch budget and initializationKKNumber of islands \(one initial plan per island\)5–Validated seed programs generated per plan1MdbgM\_\{\\mathrm\{dbg\}\}Debugging attempts per generation call3BBPer\-instance execution budget \(seconds\)120Island scheduling \(Eqs\. C1, C3\)ρ\\rhoDiscount factor ofRR,VV, andHH0\.9CCUCB exploration coefficient1\.41WWLocal code visits per island at initialization \(Alg\. C1\)3Code evolution \(OpenEvolve backend\)\|ℐj,h\(k\)\|\|\\mathcal\{I\}^\{\(k\)\}\_\{j,h\}\|Inspiration programs sampled per visit2–Top\-ranked programs shown in the prompt2–Ancestor programs shown in the prompt \(metrics only\)3–Elite / exploration / exploitation sampling ratio0\.1 / 0\.2 / 0\.7–MAP\-Elites feature dimensionscomplexity, diversity–MAP\-Elites bins per dimension10–Population size / archive size1000 / 100Plan evolution \(Alg\. C2\)τ\\tauPlateau threshold;Hj−1\(k\)<τH^\{\(k\)\}\_\{j\-1\}<\\tautriggers a plan review0\.01WWLocal code visits after an accepted plan proposal3–Evidence: top programs under the current plan2–Evidence: inspiration programs from the same island3–Evidence: top programs from each other island0–Evidence: inspiration programs across islands5Component evolution \(Eq\. C8\)–Rounds between Component Improver calls1CIC\_\{I\}Sensitivity to the contribution scoreQj​\(s\)Q\_\{j\}\(s\)2\.0IminI\_\{\\min\}Minimum review probability0\.1ImaxI\_\{\\max\}Maximum review probability0\.7
### A\.1Standard Synthetic problems

Tables[A9](https://arxiv.org/html/2609.36303#A1.T9)and[A10](https://arxiv.org/html/2609.36303#A1.T10)report all 11 problems on the training and test sets, respectively\. EoH[Liu et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib34), ReEvo[Ye et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib59), and RefineEvo[Wu et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib55)retain separate constructive, ACO, and GLS variants wherever supported\. In other words, their sparse task coverage prevents them from running all 11 tasks and excludes them from Table[2](https://arxiv.org/html/2609.36303#S4.T2), which reports the mean objective gap across the problem classes \(excluding OP from the OpenEvolve test mean because of incomplete evaluation coverage\)\. Among the full\-coverage methods,HeurEvoachieves the lowest mean gap on both splits, with−2\.22%\-2\.22\\%on training and−1\.45%\-1\.45\\%on test, compared with−0\.15%\-0\.15\\%and\+0\.91%\+0\.91\\%for AdaEvolve, respectively\. This corresponds to reductions of2\.072\.07and2\.362\.36percentage points over AdaEvolve\. At the individual\-problem level,HeurEvoobtains the best or tied\-best gap on six of the 11 problems on both training and test: C\-CVRP, U\-CVRP, OP, Decap, RCPS, and MIS\. It also achieves negative gaps on eight training problems and seven test problems, showing that the gains are not confined to the training instances\. The largest improvement over the full\-coverage baselines occurs on OP, whereHeurEvochanges the gap from\+16\.92%\+16\.92\\%for AdaEvolve to−3\.80%\-3\.80\\%on training and from\+21\.75%\+21\.75\\%to−1\.94%\-1\.94\\%on test\.

Table A9:Full synthetic training gaps \(%\)\.MethodC\-CVRPU\-CVRPTSPBPOPDecapJSSPFSSRCPSMax\-CutMISEoH \(Constr\.\)–––\+3\.53–––––––EoH \(GLS\)––\+0\.68––––\-2\.65–––ReEvo \(ACO\)\+6\.58\+9\.66\+8\.60\+1\.14\+4\.16––––––ReEvo \(Constr\.\)––\+16\.02––––––––ReEvo \(GLS\)––\-0\.83––––––––RefineEvo \(ACO\)\+5\.06\+9\.10\+9\.60––––––––RefineEvo \(Constr\.\)\+12\.13\+16\.10\+20\.07\+2\.29–––––––RefineEvo \(GLS\)––\-0\.83––––––––OpenEvolve\+1\.93\-1\.16\+1\.53\+0\.45\+16\.65\+1\.63\-12\.54\-0\.48\+0\.64\-0\.80\-0\.04AdaEvolve\-1\.77\-2\.06\-0\.80\-0\.45\+16\.92\+1\.63\-12\.46\-2\.19\+1\.49\-2\.08\+0\.15EvoX\+4\.50\-1\.63\+1\.00\-0\.20\+17\.96\+1\.50\-12\.40\-0\.30\+2\.70\-0\.17\+0\.47HeurEvo\-3\.36\-3\.19\-0\.82\+0\.45\-3\.80\+1\.40\-11\.60\-2\.25\+0\.12\-1\.32\-0\.04

Table A10:Full synthetic test gaps \(%\)\.MethodC\-CVRPU\-CVRPTSPBPOPDecapJSSPFSSRCPSMax\-CutMISEoH \(Constr\.\)–––\+3\.30–––––––EoH \(GLS\)––\+0\.60––––\-2\.20–––ReEvo \(ACO\)\+8\.28\+13\.30\+9\.96\+1\.56\+7\.40––––––ReEvo \(Constr\.\)––\+15\.37––––––––ReEvo \(GLS\)––\-0\.79––––––––RefineEvo \(ACO\)\+8\.18\+12\.26\+12\.72––––––––RefineEvo \(Constr\.\)\+9\.91\+19\.12\+23\.22\+2\.37–––––––RefineEvo \(GLS\)––\-0\.79––––––––OpenEvolve\+2\.96\-1\.30\+4\.67\+0\.69\+22\.61\+2\.04\-6\.30\-0\.27\+1\.74\-0\.31\+0\.14AdaEvolve\-1\.62\-2\.18\-0\.38\-0\.32\+21\.75\+2\.02\-6\.42\-2\.21\+0\.70\-1\.78\+0\.45EvoX\+2\.17\-0\.20\+3\.14\-0\.06\+21\.57\+1\.95\-6\.16\-0\.71\+0\.91\+0\.11\+1\.88HeurEvo\-4\.38\-3\.48\-0\.70\+0\.69\-1\.94\+1\.90\-5\.52\-1\.850\.00\-0\.85\+0\.14

### A\.2MIPLIB\-derived problems

Earlier in Table[4](https://arxiv.org/html/2609.36303#S4.T4), we report the averaged training objectives on MIPLIB problems\. Figure[4](https://arxiv.org/html/2609.36303#A1.F4)demonstrates the training performance of each method, i\.e\., mean objectives vs\. the number of generated programs\. The trajectories reveal different improvement patterns across problems\. Oncomp12\-2idx,graphdraw\-mainerd, andopm2\-z12\-s8,HeurEvoestablishes an advantage within the first few dozen generated programs and maintains it through the end of the run\. Ongraphdraw\-grafo2, AdaEvolve leads during the middle of training, but late\-stage improvements allowHeurEvoto finish with a lower objective of82003\.1082003\.10, compared with85061\.3085061\.30for AdaEvolve\. However,HeurEvodoes not dominate on every problem\. AdaEvolve achieves a lower final objective oncomp21\-2idx, reaching100\.40100\.40compared with103\.20103\.20forHeurEvo\. Onsct32,HeurEvoreaches an objective near−2\.77\-2\.77within roughly 30 generated programs but makes little subsequent progress, whereas AdaEvolve overtakes it after approximately 140 programs and finishes at−3\.01\-3\.01\. These contrasting trajectories show that early advantages do not always determine the final ranking and that improvements after extended plateaus can remain important\.

\(a\)comp12\-2idx\(b\)comp21\-2idx\(c\)graphdraw\-grafo2\(d\)graphdraw\-mainerd\(e\)opm2\-z12\-s8\(f\)sct32
Figure 4:Best objective versus number of generated programs for HeurEvo and AdaEvolve \(solver model GPT\-5\.6\) on the six MIPLIB instances\. Objectives are averaged over the five training seeds; the value at the end of each curve is the final best objective\.
### A\.3Non\-linear Geometry Problems

The reported nonlinear results combine aHeurEvophase with subsequent AdaEvolve refinement\. Table[A11](https://arxiv.org/html/2609.36303#A1.T11)contains all 14 nonlinear task\-size configurations under GPT\-5\.6\-sol, which is extended from the main Table[5](https://arxiv.org/html/2609.36303#S4.T5)\. In this table, we offer rich information; for example, we show the output of each best\-known solution\. Moreover, we demonstrate the performance where the proposedHeurEvoachieves poorer performance\. Across all 14 configurations,HeurEvoachieves better objectives than AdaEvolve in 11, matches it at the reported precision in one, and performs worse in two\. Its improvements cover all three rectangle\-packing configurations, all four distance\-ratio configurations, and four of the five hexagon\-packing configurations\. However, these improvements vary in magnitude\. For example, on hexagon packing withn=11n=11,HeurEvoreduces the AdaEvolve objective to3\.92450113093\.9245011309at the second decimal place, approaching the best\-known solution\. In contrast, its improvement over AdaEvolve forn=16n=16is only1\.812×10−71\.812\\times 10^\{\-7\}, and the resulting objective of4\.53249193584\.5324919358remains slightly above the reported reference\.

Table A11:Complete nonlinear training objectives\. ForHeurEvo,Underlineindicates the part better than AdaEvolve, andBoldindicates the part better than the reported best\-known value\.ProblemnnddBest result sourceReported referenceAdaEvolveHeurEvoCircle packing \(rectangle\)↑\\uparrow21–EinsteinArenaa2\.36583237591851562\.36424812672\.3658323759Circle packing \(rectangle\)↑\\uparrow26–Laib2\.6393205589877593362\.63553131512\.6393205643Circle packing \(rectangle\)↑\\uparrow27–Dutton / Laib,c2\.6915233606710186522\.68344102772\.6891174195Circle packing \(square\)↑\\uparrow26–Packomaniad2\.6359830849192\.63598304842\.6359775049Circle packing \(square\)↑\\uparrow32–[Berthold et al\. \(2026a\)](https://arxiv.org/html/2609.36303#bib.bib4)d2\.9395727712052\.93957277062\.9395727706Distance ratio↓\\downarrow143Sun–Samantae4\.1654\.16578347464\.1657820926Distance ratio↓\\downarrow162Together\-AIf12\.889229912\.889229907712\.8892291072Distance ratio↓\\downarrow212[Berthold et al\. \(2026a\)](https://arxiv.org/html/2609.36303#bib.bib4)17\.7749917\.774980554217\.7749803417Distance ratio↓\\downarrow222[Berthold et al\. \(2026a\)](https://arxiv.org/html/2609.36303#bib.bib4)19\.0539819\.053984509819\.0539769136Hexagon packing↓\\downarrow11–Schellhorng3\.92450089729875253\.93267585593\.9245011309Hexagon packing↓\\downarrow12–[Berthold et al\. \(2026c\)](https://arxiv.org/html/2609.36303#bib.bib6)h3\.9416420675369493\.94164230043\.9416420625Hexagon packing↓\\downarrow14–[Berthold et al\. \(2026c\)](https://arxiv.org/html/2609.36303#bib.bib6)h4\.2689949572520014\.27239207724\.2689950806Hexagon packing↓\\downarrow15–[Kravatskiy et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib26)g,h4\.4472711489871054\.44727187654\.4494139689Hexagon packing↓\\downarrow16–[Kravatskiy et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib26)g4\.527464\.53249211704\.5324919358

### A\.4Full ablation results on synthetic problems

Earlier in Table[6](https://arxiv.org/html/2609.36303#S4.T6), we show the effectiveness of various components by showing the mean objective gap across all 11 synthetic tasks and choose the four most representative problems to analyze\. Tables[A12](https://arxiv.org/html/2609.36303#A1.T12)and[A13](https://arxiv.org/html/2609.36303#A1.T13)extend from Table[6](https://arxiv.org/html/2609.36303#S4.T6)with all 11 synthetic tasks performance\. Across all 11 tasks,HeurEvoachieves the lowest gap on five tasks in each split, leading on C\-CVRP, U\-CVRP, TSP, and PFSS in both training and testing\. It outperforms*Adaptive plan only*on nine tasks and*Component only*on eight tasks in each split\. Compared with*Always\-plan \+ component*, it achieves lower gaps on seven training tasks and six test tasks\. These results support combining adaptive plan review with component evolution for both training performance and generalization\.

Table A12:Synthetic ablation training gaps \(%\)\.MethodC\-CVRPU\-CVRPTSPBPOPDecapJSSPFSSRCPSMax\-CutMISFixed plan\-3\.15\-2\.25\+1\.23\+0\.45\+1\.94\+1\.47\-12\.29\-0\.83\+1\.39\-1\.19\+0\.19Adaptive plan only\-2\.48\-0\.68\+1\.11\+0\.45\-0\.87\+1\.43\-12\.32\-1\.48\+0\.87\-1\.18\+0\.27Component only\-3\.09\-2\.47\+1\.98\+0\.40\-1\.67\+1\.30\-12\.28\-0\.79\+1\.37\-1\.08\+0\.35Earlier full configuration\-3\.04\-1\.82\+1\.45\+0\.45\-0\.75\+1\.45\-12\.32\-1\.15\+0\.82\-1\.83\+0\.23Always\-plan \+ component\-3\.20\-1\.78\+0\.40\+0\.45\-2\.67\+1\.47\-12\.32\-1\.16\-0\.35\-1\.69\+0\.11Always\-plan\-2\.84\-2\.04\-0\.23\+0\.10\+0\.45\+1\.44\-12\.23\-1\.52\+0\.50\-1\.89\-0\.12HeurEvo\(Ours\)\-3\.36\-3\.19\-0\.82\+0\.45\-3\.80\+1\.40\-11\.60\-2\.25\+0\.12\-1\.32\-0\.04

Table A13:Synthetic ablation test gaps \(%\)\.MethodC\-CVRPU\-CVRPTSPBPOPDecapJSSPFSSRCPSMax\-CutMISFixed plan\-3\.26\-1\.81\+2\.53\+0\.69\+7\.97\+2\.66\-6\.30\-0\.50\+0\.39\-0\.86\+1\.20Adaptive plan only\-3\.00\-1\.81\+2\.44\+0\.69\+3\.76\+2\.01\-6\.57\-0\.64\+0\.65\-0\.79\+0\.90Component only\-3\.47\-1\.59\+4\.17\+0\.69\+3\.86\+1\.62\-6\.41\-0\.69\+1\.61\-0\.70\+1\.79Earlier full configuration\-3\.45\-1\.06\+1\.78\+0\.69\+7\.04\+1\.98\-6\.42\-0\.89\+0\.57\-1\.54\+0\.57Always\-plan \+ component\-3\.62\+0\.09\+1\.97\+0\.69\-2\.39\+1\.82\-6\.19\-0\.93\+0\.43\-1\.08\+1\.52Always\-plan\-2\.95\-1\.96\+0\.18\+0\.38\+8\.12\+2\.03\-6\.17\-1\.67\+0\.43\-1\.44\+0\.11HeurEvo\(Ours\)\-4\.38\-3\.48\-0\.70\+0\.69\-1\.94\+1\.90\-5\.52\-1\.850\.00\-0\.85\+0\.14

### A\.5Island initialization ablations

Tables[A14](https://arxiv.org/html/2609.36303#A1.T14)and[A15](https://arxiv.org/html/2609.36303#A1.T15)report objective gaps of every problem for the*Single island*,*Shared plan*, and*Partitioned library*variants in Table[7](https://arxiv.org/html/2609.36303#S4.T7)\. Compared with*Single island*,*Shared plan*achieves lower test gaps on six tasks, including a reduction from\+2\.09%\+2\.09\\%to−2\.96%\-2\.96\\%on U\-CVRP, showing that multiple islands can help even when initialized with the same plan\. With the island count fixed at five,HeurEvouses distinct initial plans and improves upon*Shared plan*on seven validation tasks\. It also lowers the test gaps on U\-CVRP, OP, and RCPS, with the OP gap decreasing from\+3\.85%\+3\.85\\%to−1\.94%\-1\.94\\%\. Although individual tasks favor different configurations, these results support the importance of both the number of islands and the diversity of their initial plans\.

Table A14:Island ablation validation gaps \(%\)\. Bold indicates the lowest gap in each column, including ties\.VariantC\-CVRPU\-CVRPTSPBPOPDecapJSSPFSSRCPSMax\-CutMISSingle island\-2\.88\+3\.10\-0\.92\+0\.49\-0\.06\+1\.05\-7\.07\-2\.65\-0\.57\-1\.32\+0\.10Shared plan\-2\.69\-1\.93\-0\.98\+0\.62\-0\.97\+1\.55\-6\.61\-2\.35\+1\.21\-1\.41\-0\.10Partitioned library\-1\.88\-1\.94\-0\.93\+0\.49\-3\.98\+1\.17\-6\.74\-1\.86\-1\.78\-1\.23\+0\.10HeurEvo\(Ours\)\-3\.36\-3\.19\-0\.82\+0\.45\-3\.80\+1\.40\-11\.60\-2\.25\+0\.12\-1\.32\-0\.04

Table A15:Island ablation test gaps \(%\)\. Bold indicates the lowest gap in each column, including ties\.VariantC\-CVRPU\-CVRPTSPBPOPDecapJSSPFSSRCPSMax\-CutMISSingle island\-4\.87\+2\.09\-0\.79\+0\.50\+4\.18\+1\.20\-6\.90\-2\.31\+0\.29\-1\.600\.00Shared plan\-4\.55\-2\.96\-0\.84\+0\.50\+3\.85\+1\.06\-7\.46\-2\.05\+0\.58\-1\.42\-0\.26Partitioned library\-3\.58\-3\.74\-0\.78\+0\.50\-1\.46\+1\.45\-6\.90\-1\.52\+0\.29\-1\.51\-0\.07HeurEvo\(Ours\)\-4\.38\-3\.48\-0\.70\+0\.69\-1\.94\+1\.90\-5\.52\-1\.850\.00\-0\.85\+0\.14

## Appendix BDataset Details

### B\.1Dataset Composition

We consider three data types: Synthetic Problems, MIPLIB\-Derived Problems, and Nonlinear Geometry Benchmarks\. The first two form the instance collections summarized below; the nonlinear configurations are described in Section[B\.4](https://arxiv.org/html/2609.36303#A2.SS4)\. In these two collections, a task denotes one configured synthetic problem or one MIPLIB\-NL base problem together with its perturbations\. The 11 synthetic tasks represent 10 mathematical problem families because capacitated vehicle routing has two configurations\. The six MIPLIB\-derived tasks cover four application domains\.

Table[B16](https://arxiv.org/html/2609.36303#A2.T16)counts the instances used in the reported experiments\. Each synthetic task uses 11 instances:seed0–seed4for training,seed5–seed6for validation, andseed7–seed10for testing\. The MIPLIB experiments reported here use five training instances per task,seed0–seed4\. These are instance seeds, rather than independent seeds of heuristic\-evolution runs\. Section[B\.2](https://arxiv.org/html/2609.36303#A2.SS2)qualifies the effective diversity of the decap\-placement task\.

##### Instance Format\.

All our data follow the MIPLIB\-NL format[Li et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib32)\. Each instance includes a natural\-language problem description, large input data supplied as CSV tables, and an initial Gurobi program encoding the mathematical formulation\. A standardized solution\-output format and a task\-specific validator allow generated programs to be evaluated consistently: the validator checks the output solution’s feasibility and the consistency of its reported objective\.

Table B16:Instances used in the reported experiments\. Dashes indicate splits outside the reported evaluation\.CollectionTasksTrainVal\.TestTotalSynthetic Problems11552244121MIPLIB\-Derived Problems630––30Total17852244151
##### Data Availability\.

We will soon open\-source both our synthetic dataset and our MIPLIB\-NL\-derived dataset, together with the generation code, instance specifications, reference models, and split definitions\.

##### Fitness Score Function Definition\.

We use task\-specific fitness functions to evaluate candidate programs\. As defined earlier in Section[3\.1](https://arxiv.org/html/2609.36303#S3.SS1),𝒟\\mathcal\{D\}denotes the training set for a target task, andd∈𝒟d\\in\\mathcal\{D\}represents an individual instance\. Letobj⁡\(G,d\)\\operatorname\{obj\}\(G;d\)denote the verified objective value of the feasible solution returned by programGGon instanceddwithin the runtime budgetBB\. For the synthetic and MIPLIB\-derived problems \(Tables[B17](https://arxiv.org/html/2609.36303#A2.T17)and[B18](https://arxiv.org/html/2609.36303#A2.T18)\), we define

f⁡\(G,d\)=−\|obj⁡\(G;d\)−b48​h​\(d\)\|\|obj⁡\(G,d\)\|\+ϵ,f\(G;d\)=\-\\frac\{\\left\|\\operatorname\{obj\}\(G;d\)\-b\_\{48\\mathrm\{h\}\}\(d\)\\right\|\}\{\\left\|\\operatorname\{obj\}\(G;d\)\\right\|\+\\epsilon\},\(2\)whereb48​h​\(d\)b\_\{48\\mathrm\{h\}\}\(d\)is the best objective bound obtained from a Gurobi reference run with a 48\-hour time limit\. This is a lower bound for minimization problems and an upper bound for maximization problems, rather than the objective value of Gurobi’s best feasible solution\. The bound is fixed for each instance across program evaluations, andϵ\>0\\epsilon\>0prevents division by zero\.

For the nonlinear geometry problems \(Appendix[B\.4](https://arxiv.org/html/2609.36303#A2.SS4)\), we directly transform the objective according to its optimization direction,

f⁡\(G,d\)=−s⁡\(d\)​obj⁡\(G;d\),s⁡\(d\)=\{\+1,for minimization,−1,for maximization\.f\(G;d\)=\-s\(d\)\\operatorname\{obj\}\(G;d\),\\qquad s\(d\)=\\begin\{cases\}\+1,&\\text\{for minimization\},\\\\ \-1,&\\text\{for maximization\}\.\\end\{cases\}\(3\)Evolution maximizes the mean training fitness,F⁡\(G\)=\|𝒟\|−1​∑d∈𝒟f⁡\(G,d\)F\(G\)=\|\\mathcal\{D\}\|^\{\-1\}\\sum\_\{d\\in\\mathcal\{D\}\}f\(G;d\)\. Each nonlinear configuration is optimized separately, so its training set contains a single instance\. These fitness scores guide program evolution and are distinct from the signed gaps relative to Gurobi’s 48\-hour incumbent reported in the experimental results\.

### B\.2Synthetic Problems

Table[B17](https://arxiv.org/html/2609.36303#A2.T17)describes the 11 synthetic tasks and how we generate their instances\. We follow published sampling recipes where noted and specify our changes in the corresponding entries\. Problem sizes are fixed within a task; realized edge counts, precedence counts, and demand\-derived fleet sizes can vary\. Numerical ranges below are measured across all 11 stored instances per task\. Integer\-uniform ranges include both endpoints\. For TSP, both CVRP configurations, and orienteering, coordinates are stored to six decimal places, and distances between stored pointspip\_\{i\}andpjp\_\{j\}are

di​j=round⁡\(1000​∥pi−pj∥2\)\.d\_\{ij\}=\\operatorname\{round\}\\\!\\left\(1000\\lVert p\_\{i\}\-p\_\{j\}\\rVert\_\{2\}\\right\)\.\(4\)
Decap placement requires a further qualification\. Its randomized target weights cancel against inversely scaled base requirements, leaving essentially the same optimization model across seeds, apart from small rounding differences\. Its nominal split therefore does not measure generalization to substantively different instances\.

Table B17:Synthetic problem definitions and generation settings\.TaskDefinition, generation details, and referencesClustered CVRPMinimize total distance while serving each customer exactly once using exactly 22 depot\-return routes, each carrying at most 55 demand units\.There are 120 customers in four Gaussian clusters with coordinate standard deviation 0\.08, clipped to the unit square\. The depot is at\(0\.1,0\.1\)\(0\.1,0\.1\); customer demands are uniform integers from 1 to 13\. Each customer independently chooses a cluster center uniformly from\(0\.25,0\.25\)\(0\.25,0\.25\),\(0\.75,0\.30\)\(0\.75,0\.30\),\(0\.35,0\.78\)\(0\.35,0\.78\), and\(0\.78,0\.75\)\(0\.78,0\.75\)before its coordinates are sampled\.This is our custom clustered generator for the CVRP of[Dantzig & Ramser \(1959\)](https://arxiv.org/html/2609.36303#bib.bib9), not a reproduction of a published instance set\.Uniform CVRPMinimize total distance with complete single\-visit service, depot\-return routes, and vehicle capacity 50\.The 200 customers are sampled uniformly from the unit square, with a depot at\(0\.5,0\.5\)\(0\.5,0\.5\)and demands uniform on the integers 1–9\. The fleet size is fixed for each instance and ranges from 20 to 22 vehicles:K=max⁡\(⌈∑iqi50⌉\+1,KFFD\),K=\\max\\\!\\left\(\\left\\lceil\\frac\{\\sum\_\{i\}q\_\{i\}\}\{50\}\\right\\rceil\+1,K\_\{\\mathrm\{FFD\}\}\\right\),whereKFFDK\_\{\\mathrm\{FFD\}\}is the number of capacity\-50 bins obtained by first\-fit decreasing on the demands\.Coordinates, demands, depot, and capacity follow the ACO benchmark sampling of ReEvo[Ye et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib59); the exact fleet\-size constraint and integer distances are our adaptations\.Decap\-Placement ProxyMinimize capacitor cost subject to placing at most one capacitor type at each site and meeting every target’s required cumulative synthetic effect\.A30×3030\\times 30grid supplies 900 sites and 900 targets\. Four types give 3,600 site–type choices; effects decay with distance inside a type\-specific radius\. Typekkcontributesek/\(1\+ds​t\)e\_\{k\}/\(1\+d\_\{st\}\)when the Euclidean grid distanceds​t≤ρkd\_\{st\}\\leq\\rho\_\{k\}, and zero otherwise\. The four\(cost,ek,ρk\)\(\\mathrm\{cost\},e\_\{k\},\\rho\_\{k\}\)tuples are\(1\.0,1\.4,1\.25\)\(1\.0,1\.4,1\.25\),\(2\.1,2\.6,2\.25\)\(2\.1,2\.6,2\.25\),\(3\.8,4\.3,3\.5\)\(3\.8,4\.3,3\.5\), and\(6\.4,7\.0,5\.25\)\(6\.4,7\.0,5\.25\)\.Before rounding, targetttrequires0\.22​mt​Ft0\.22m\_\{t\}F\_\{t\}, whereFtF\_\{t\}is the strongest\-type all\-site coverage andmt=0\.90\+0\.20​∥pt−\(14\.5,14\.5\)∥2/\(14\.5​2\)m\_\{t\}=0\.90\+0\.20\\lVert p\_\{t\}\-\(14\.5,14\.5\)\\rVert\_\{2\}/\(14\.5\\sqrt\{2\}\)\. Target\-weight jitter is uniform on\[−0\.06,0\.06\]\[\-0\.06,0\.06\]but cancels against inversely scaled base requirements, up to rounding\. This covering proxy is inspired by ReEvo’s application[Ye et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib59), not its electrical simulator or data distribution; see Section[B\.2](https://arxiv.org/html/2609.36303#A2.SS2)for the diversity qualification\.Job\-Shop Schedulingjob\_shop\_scheduling\. Minimize makespan while preserving each job’s operation order and preventing machine conflicts\. Operations cannot be interrupted\.There are 100 jobs and 20 machines \(2,000 operations\)\. Every job visits every machine once in a random order, with processing times uniform on the integers 1–99\. This follows Taillard’s random\-permutation and processing\-time recipe[Taillard \(1993\)](https://arxiv.org/html/2609.36303#bib.bib51), with fresh seeds; the scheduling formulation follows[Manne \(1960\)](https://arxiv.org/html/2609.36303#bib.bib38)\.Max\-Cutmax\_cut\. Partition the vertices into two sets to maximize the number of crossing edges[Goemans & Williamson \(1995\)](https://arxiv.org/html/2609.36303#bib.bib19)\.Each graph has 2,000 vertices and 10,882–11,072 edges\. Our hybrid generator combines Barabási–Albert\-style preferential attachment[Barabási & Albert \(1999\)](https://arxiv.org/html/2609.36303#bib.bib3), initialized with a five\-vertex clique and attachment parameter 4, with an independent Erdős–RényiG⁡\(n,p\)G\(n,p\)edge overlay atp=0\.0015p=0\.0015[Erdős & Rényi \(1959\)](https://arxiv.org/html/2609.36303#bib.bib13);[Gilbert \(1959\)](https://arxiv.org/html/2609.36303#bib.bib17)\. Duplicate edges are merged; all stored edge weights equal one\. This is our custom combination of the two graph models\.Maximum Independent Setmaximum\_independent\_set\. Maximize the cardinality of a vertex subset containing no adjacent pair\.Each preferential\-attachment graph has 1,500 vertices and 11,964 edges, uses attachment parameter 8, and starts from a nine\-vertex clique\. We follow the preferential\-attachment principle of[Barabási & Albert \(1999\)](https://arxiv.org/html/2609.36303#bib.bib3)with this explicit initialization and use the unweighted independent\-set objective[Nemhauser & Trotter \(1975\)](https://arxiv.org/html/2609.36303#bib.bib42)\.Offline Bin Packing \(BP\)offline\_bin\_packing\. Minimize the number of bins while assigning each item to exactly one bin without exceeding capacity\. All items are available before packing[Martello & Toth \(1990\)](https://arxiv.org/html/2609.36303#bib.bib39)\.Each instance has 1,000 items with independently sampled uniform integer sizes from 20 to 100 and bin capacity 150, following Falkenauer’s uniform benchmark recipe[Falkenauer \(1996\)](https://arxiv.org/html/2609.36303#bib.bib14)\. These are newly generated instances, not the original benchmark files\.Orienteeringorienteering\. Maximize the prize collected on one depot\-return tour visiting a subset of customers within a travel\-distance budget\.The 500 nodes comprise a centered depot and 499 uniformly distributed customers\. The distance budget isround⁡\(1000⋅0\.4​500\)=8944\\operatorname\{round\}\(1000\\cdot 0\.4\\sqrt\{500\}\)=8944\. Customer prizes areπi=clip\[1,100\]⁡\(round⁡\(15\+75​ri\+ϵi\)\),\\pi\_\{i\}=\\operatorname\{clip\}\_\{\[1,100\]\}\\\!\\left\(\\operatorname\{round\}\(15\+75r\_\{i\}\+\\epsilon\_\{i\}\)\\right\),whereϵi∼𝒩⁡\(0,82\)\\epsilon\_\{i\}\\sim\\mathcal\{N\}\(0,8^\{2\}\)andrir\_\{i\}is the distance from the centered depot divided by0\.5\\sqrt\{0\.5\}, computed before coordinate rounding\.The distance\-correlated prize principle follows ReEvo[Ye et al\. \(2024\)](https://arxiv.org/html/2609.36303#bib.bib59); noise, normalization, centered depot, and budget are our adaptations to the orienteering problem[Golden et al\. \(1987\)](https://arxiv.org/html/2609.36303#bib.bib20)\.Permutation Flow\-Shop Schedulingpermutation\_flow\_shop\_scheduling\. Minimize makespan\. All jobs follow the same machine order, and all machines process jobs in the same chosen permutation\. Operations of a job cannot overlap, and each machine processes at most one job at a time\.The 100 jobs and 20 machines have 2,000 independently sampled uniform integer processing times from 1 to 100\. This is a Taillard\-style synthetic recipe[Taillard \(1993\)](https://arxiv.org/html/2609.36303#bib.bib51), using an upper endpoint of 100 rather than Taillard’s 99; we do not reuse the published benchmark instances\.Resource\-Constrained Project Schedulingresource\_constrained\_project\_scheduling\. Minimize makespan subject to finish–start precedence and renewable\-resource capacities during each activity’s uninterrupted execution\.Instances have 85 activities, five resources, and 212–265 precedence arcs, with no dummy activities\. Durations are uniform integers from 1 to 8 and capacities from 5 to 12\. The first resource has comparatively larger capacity\-relative demands: activity demands are uniform integers from zero to⌊C0/2⌋\\lfloor C\_\{0\}/2\\rfloorfor resource zero and to⌊Cr/3⌋\\lfloor C\_\{r\}/3\\rfloorotherwise\.Each pairi<ji<jreceives an arc with probability 0\.06; consecutive pairs receive an independent additional opportunity with probability 0\.35, with duplicates merged\. This is our custom acyclic\-network recipe for the classical formulation[Pritsker et al\. \(1969\)](https://arxiv.org/html/2609.36303#bib.bib47), not a replication of PSPLIB generation\.Traveling Salesmantraveling\_salesman\. Minimize the length of a Hamiltonian cycle visiting every node exactly once and returning to its start\.Each instance contains 350 nodes drawn from three Gaussian clusters with coordinate standard deviation 0\.08 and clipped to the unit square\. Each node independently chooses one of\(0\.2,0\.25\)\(0\.2,0\.25\),\(0\.78,0\.24\)\(0\.78,0\.24\), and\(0\.5,0\.75\)\(0\.5,0\.75\)with equal probability\. This is our custom clustered Euclidean generator; the reference model uses the subtour formulation of[Miller et al\. \(1960\)](https://arxiv.org/html/2609.36303#bib.bib40)\.
### B\.3MIPLIB\-Derived Problems

Table[B18](https://arxiv.org/html/2609.36303#A2.T18)describes the six MIPLIB\-derived tasks\. MIPLIB\-NL[Li et al\. \(2026\)](https://arxiv.org/html/2609.36303#bib.bib32)reconstructs natural\-language optimization problems from MIPLIB 2017[Gleixner et al\. \(2021\)](https://arxiv.org/html/2609.36303#bib.bib18)\. We selected these six base problems because Gurobi[Gurobi Optimization, LLC \(2026\)](https://arxiv.org/html/2609.36303#bib.bib21)did not certify optimality within two hours \(7,200 seconds\) per instance\. This selection criterion refers to the initial screening, not subsequent reference re\-solves or a claim about all instances in MIPLIB\-NL\. The tasks comprise two curriculum\-based course\-scheduling problems \(comp12\-2idx,comp21\-2idx\)[Lach & Lübbecke \(2012\)](https://arxiv.org/html/2609.36303#bib.bib28), two graph\-drawing problems \(graphdraw\-grafo2,graphdraw\-mainerd\)[e Silva & Santos \(2017\)](https://arxiv.org/html/2609.36303#bib.bib12), mining\-project selection \(opm2\-z12\-s8\), and PCB assembly\-line configuration \(sct32\) from MIPLIB[Koch et al\. \(2011\)](https://arxiv.org/html/2609.36303#bib.bib25)\. We use these original IDs consistently in the dataset and result tables\. For each task, the five training instances comprise the unperturbed reconstructed base instance \(seed0\) and four task\-specific perturbations \(seed1–seed4\)\. The dimensions below are fixed across these training instances\.

Table B18:MIPLIB\-derived problem definitions\.Problem IDDefinition and instance configurationcomp12\-2idxCoursescheduling\.Schedule all required lectures into available periods while avoiding course conflicts and respecting the number of rooms\. Minimize penalties for room\-capacity shortfalls, insufficient teaching\-day spread, and isolated curriculum lectures\.There are 88 courses, 218 lectures, six days with six periods each, 11 rooms, 146 curricula, and 519 course\-conflict pairs\.comp21\-2idxCoursescheduling\.Solve the same time\-assignment problem and three\-part penalty objective, with a different timetable, room inventory, and curriculum structure\. Room capacities are represented by capacity thresholds and penalized shortfalls\.There are 94 courses, 327 lectures, five days with five periods each, 18 rooms, 78 curricula, and 302 conflict pairs\.graphdraw\-grafo2Graphdrawing\.Place entities at integer coordinates inside their permitted windows, enforcing size\-dependent non\-overlap and consistent directional relations\. Minimize weighted connection\-distance terms plus deviation from preferred anchors\.This is the entity\-placement phase of graph drawing, with 67 entities, 73 connections, and 2,211 entity pairs\. All edge weights equal 122\.graphdraw\-mainerdGraphdrawing\.Solve the same entity\-placement problem on a smaller graph, with placement windows, pairwise separations, directional consistency, and centering terms\.There are 31 entities, 33 connections, and 465 entity pairs\. All edge weights equal 126\.opm2\-z12\-s8Mining\-projectselection\.Select mining blocks/projects to maximize total net present value\. Selecting a project requires selecting its predecessors, and the total use of each resource must respect its capacity\.There are 10,800 binary project decisions, eight resources, and 319,500 dependency arcs\. Every resource has capacity 3,704,832\.sct32PCBassembly\-lineconfiguration\.Assign each operation’s full workload across eligible workstation options, using binary decisions on discrete lanes and fractional assignments where permitted\. Minimize staffing cost minus weighted checkpoint performance and discrete\-routing rewards, subject to capacities, checkpoint constraints, priority\-token eligibility and budget, and policy checks\.There are 552 operations, ten workstation options, seven board variants, 27 checkpoints, 67 policy checks, and a priority\-token budget of six\.#### B\.3\.1Generating MIPLIB Training Instances

We construct the five training instances for each of the six reconstructed base problems as follows\. For each problem,seed0retains the base data, andseed1–seed4vary selected relationships and coefficients while preserving table dimensions, course–capacity keys, or graph degree sequences, as appropriate\. Forsct32, we use the upstream generator’s numerical\-jitter mode\. Across all six tasks, the variable count remains unchanged; linear constraint counts may vary for timetabling \(Table\)\.

##### Course Timetabling\.

For both tasks, we relocate unavailable periods while preserving their total count for each course\. We first sample full\-day unavailability and then remaining slots without replacement, using the base instance’s corresponding day and day–period frequencies plus one as sampling weights\. We retain the existing curriculum\-induced conflict pairs and apply degree\-preserving double\-edge swaps to the remaining conflicts: 83 pairs forcomp12\-2idxand 32 forcomp21\-2idx\. We also permute lecture\-count/minimum\-day pairs across courses, preserving their multiset and total lecture count\. A permutation is accepted only if each course’s lecture count fits its available periods and each curriculum’s lecture load does not exceed the base instance’s maximum curriculum load\. Finally, we permute capacity\-penalty weights within each capacity class\. Curricula, room inventories, course identifiers, and course–capacity penalty keys are retained\.

##### Graph Drawing\.

Degree\-preserving edge swaps modify the connection graph while retaining entity data, placement windows, sizes, edge count, degree sequence, and the constant edge weight\. Edge\-distance coefficients are then recomputed from the unchanged geometry\.

##### Mining\-Project Selection\.

Dependency arcs and resource capacities are retained\. Project values and resource costs are independently multiplied by factors fromU⁡\(0\.9,1\.1\)U\(0\.9,1\.1\)and rounded to integers; stored resource\-cost entries are bounded below by one, while omitted entries remain zero\. Thus, dependency structure is preserved, while changes to resource coefficients can change the feasible set\.

##### PCB Assembly\-Line Configuration\.

The upstream generator’s jitter mode independently multiplies entries in designated numeric data\-table columns by factors fromU⁡\(0\.9,1\.1\)U\(0\.9,1\.1\), using floating\-point conversion and nonnegative clipping without integer rounding\. These columns contain capacities, workloads, performance coefficients, weights, and constraint right\-hand sides\. Identifiers, categorical flags, table structure, and scalar model parameters are retained\.

##### Feasibility Checks\.

We check the timetable variable counts against an explicit counting formula and verify that the graph\-drawing coefficient formulas reproduce the base data\. We then construct each instance’s Gurobi model to record its size before optimization, obtain a reference solution, and pass the reported solution to the task\-specific verifier to check feasibility and objective consistency\. Generation\-time reference runs used a 120\-second budget with two threads, with additional 600\-second, four\-thread runs forgraphdraw\-grafo2\. The generation\-time verification records contain aVALIDverdict for each of the 30 training instances; these checks are separate from both the initial hardness screening and subsequent longer reference re\-solves\. The validated reference objectives are feasible\-solution values; a successful verification does not certify optimality\. The timetabling guards are screening conditions, not a guarantee of feasibility\. If a draw is proven infeasible, we resample it using a new random\-stream attempt index and repeat validation\.

Seeds, resampling attempts, and perturbation details are recorded with each instance, together with its feasibility\-check results\.

#### B\.3\.2MIPLIB Model Sizes

Table[B19](https://arxiv.org/html/2609.36303#A2.T19)reports the numbers of decision variables and constraints in the Gurobi models used for the six MIPLIB\-derived problems\. Each problem has the same number of variables across its 11 instances\. General constraints are reported separately from linear constraints\.

Table B19:Formulation sizes before presolve for 11 synthetic and six MIPLIB\-derived tasks\. Ranges cover synthetic instances; “varies” marks MIPLIB constraint counts affected by perturbation\.ProblemVariablesLinearconstraintsGeneralconstraintsKey instance parametersC\-CVRP14,64014,5220120 customers; 22 vehicles; capacity 55U\-CVRP40,40040,2020200 customers; 20–22 vehicles; capacity 50OP250,498250,5010500 nodes; travel budget 8,944TSP122,499122,1520350 nodesJSSP101,001201,9000100 jobs; 20 machines; 2,000 operationsPFSSP12,0004,0810100 jobs; 20 machinesRCPSP28,646–34,3582,140–2,449085 activities; 5 resources; 212–265 precedencesBP1,001,0002,00001,000 items; bin capacity 150comp12\-2idx11,726varies088 courses; 36 time slots; 11 rooms; 519 conflictscomp21\-2idx10,911varies094 courses; 25 time slots; 18 rooms; 302 conflictsMax\-Cut12,882–13,07243,528–44,28802,000 vertices; 10,882–11,072 edgesMIS1,50011,96401,500 vertices; 11,964 edgesgraphdraw\-grafo29,258194,6119,13667 entities; 73 relationships; 2,211 entity pairsgraphdraw\-mainerd2,05018,8011,99231 entities; 33 relationships; 465 entity pairsDCP3,6001,8000900 sites; 900 targets; 4 capacitor typesopm2\-z12\-s810,800319,508010,800 projects; 8 resources; 319,500 dependenciessct326,6073,8850552 operations; 10 stations; 7 variants; 27 checkpoints

### B\.4Nonlinear Geometry Benchmarks

We take our nonlinear geometry benchmarks from[Berthold et al\. \(2026b\)](https://arxiv.org/html/2609.36303#bib.bib5), who revisit problems studied in AlphaEvolve[Georgiev et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib16);[Novikov et al\. \(2025\)](https://arxiv.org/html/2609.36303#bib.bib44)by solving those with nonlinear programming solvers\. We retain these AlphaEvolve problem definitions and use the nonlinear formulations presented by[Berthold et al\. \(2026b\)](https://arxiv.org/html/2609.36303#bib.bib5)\. We manually wrote our own natural\-language problem descriptions and mathematical formulation files for these settings\. We evolve heuristics separately for each problem–size configuration\.

Circle packing maximizes the sum of variable radii in either a unit square or a rectangle of perimeter four with an optimized aspect ratio\. The distance\-ratio problem minimizes the ratio of maximum to minimum pairwise distances between points; our formulation uses squared distances and fixes the minimum squared distance to one\. Hexagon packing minimizes the side length of a regular hexagonal container holding non\-overlapping unit\-side regular hexagons with free translations and rotations\.

Table[B20](https://arxiv.org/html/2609.36303#A2.T20)lists all 14 configurations in the nonlinear collection\. Here,nndenotes the number of circles, points, or hexagons, anddddenotes the dimension of the point configuration\.

Table B20:Nonlinear problem names and size configurations\.Problem NameSize Configurationscircle\_packing\_squaren∈\{26,32\}n\\in\\\{26,32\\\}circle\_packing\_rectanglen∈\{21,26,27\}n\\in\\\{21,26,27\\\}distance\_ratiod=2,n∈\{16,21,22\}d=2,\\ n\\in\\\{16,21,22\\\};
d=3,n=14d=3,\\ n=14hexagon\_packingn∈\{11,12,14,15,16\}n\\in\\\{11,12,14,15,16\\\}These fixed geometry configurations do not use separate instances for training, validation, and testing\. Seed directories identify problem sizes rather than random samples\. These benchmarks are additional to the 151 instances used in the reported experiments in Table[B16](https://arxiv.org/html/2609.36303#A2.T16)\.

## Appendix CAdditional Method Details

### C\.1Notations

The indicesk,j,h,ik,j,h,iidentify the island, global round, local update, and program step, respectively\. The tagrole\\mathrm\{role\}isseed\\mathrm\{seed\},par\\mathrm\{par\},child\\mathrm\{child\}, or⋆\\star\. Omithhwhen no local update is involved\. For shared best programs,scope\\mathrm\{scope\}isglobal\\mathrm\{global\}orarchive\\mathrm\{archive\}\.

Table C1:AnnotationsAnnotationDescription𝒜,𝒜j\\mathcal\{A\},\\mathcal\{A\}\_\{j\}Shared Program Database of retained, validated programs\.𝒜j\(k\)\\mathcal\{A\}\_\{j\}^\{\(k\)\}All retained programs from islandkk, including earlier plans\.𝒜j\(k,par\),𝒞j\(k\)\\mathcal\{A\}\_\{j\}^\{\(k,\\mathrm\{par\}\)\},\\mathcal\{C\}\_\{j\}^\{\(k\)\}Parent\-eligible programs under the current plan generation\.𝒜j\(k,ins\)\\mathcal\{A\}\_\{j\}^\{\(k,\\mathrm\{ins\}\)\}Available inspiration programs for islandkk\.𝒟,B\\mathcal\{D\},BTraining set and per\-instance execution budget\.f⁡\(G,d\),F⁡\(G\)f\(G;d\),F\(G\)Instance fitness and average training fitness; larger is better\.Fj,h\(k,role\)F\_\{j,h\}^\{\(k,\\mathrm\{role\}\)\}Fitness of the correspondingly indexed programGG; shared scopes use the same convention\.gi,Gg\_\{i\},GStep implementation and complete programG=g1⊙⋯⊙gmG=g\_\{1\}\\odot\\cdots\\odot g\_\{m\}\.𝒢⁡\(P\)\\mathcal\{G\}\(P\)Programs preserving the ordered subgoals and components ofPP\.Gj,h\(k,role\)G\_\{j,h\}^\{\(k,\\mathrm\{role\}\)\}Seed, parent, child, or current\-plan incumbent, as indicated by the role tag\.Gj\(scope,⋆\)G\_\{j\}^\{\(\\mathrm\{scope\},\\star\)\}Best current\-island incumbent \(global\) or best retained program \(archive\); addhhfor local updates\.Hj,h\(k\)H\_\{j,h\}^\{\(k\)\}Recent\-progress estimate controlling plan review\.ℐj,h\(k\)\\mathcal\{I\}\_\{j,h\}^\{\(k\)\}Inspiration programs actually sampled for an update\.K,J,WK,J,WIsland count, global\-round count, and local code visits after an accepted plan proposal\.MdbgM\_\{\\mathrm\{dbg\}\}Debugging\-attempt limit for bounded generation calls\.nj\(k\),Njn\_\{j\}^\{\(k\)\},N\_\{j\}Island and total code\-improvement visit counts; retries do not create extra visits\.Pj\(k\)P\_\{j\}^\{\(k\)\}Plan associated with islandkk\.Qj​\(s\)Q\_\{j\}\(s\)Mean component contribution since its last description change\.Rj,h\(k\),rj,h\(k\)R\_\{j,h\}^\{\(k\)\},r\_\{j,h\}^\{\(k\)\}Discounted accumulated reward and current\-visit reward\.ℛj\\mathcal\{R\}\_\{j\}Existing components selected for review\.s,𝒮j,sj,i\(k\)s,\\mathcal\{S\}\_\{j\},s\_\{j,i\}^\{\(k\)\}Component, shared library, and component selected at stepii\.uj,i\(k\),aj,i\(k\)u\_\{j,i\}^\{\(k\)\},a\_\{j,i\}^\{\(k\)\}Step subgoal and preliminary implementation annotation\.Vj,h\(k\)V\_\{j,h\}^\{\(k\)\}Discounted exposure statistic\.βj,i\(k\),ζj,i\(k\)\\beta\_\{j,i\}^\{\(k\)\},\\zeta\_\{j,i\}^\{\(k\)\}Step time allocation and step type\.δj,h\(k\)\\delta\_\{j,h\}^\{\(k\)\}Nonnegative relative improvement in a code visit\.Δj,h,i\(k\),𝚫j​\(s\)\\Delta\_\{j,h,i\}^\{\(k\)\},\\bm\{\\Delta\}\_\{j\}\(s\)Step contribution and component history since its last description change\.Φj,h\(k\)\\Phi\_\{j,h\}^\{\(k\)\}Interpreter report for a parent–child comparison\.⊥\\botNo valid program obtained; not a numerical fitness\.
### C\.2Agent Description

##### Component Proposer\.

The Component Proposer receives the target problem description and proposes 10–20 relevant high\-level components\. The descriptions should cover distinct approaches without duplicating components that differ only in implementation details\. Each component should be specific enough to guide planning while supporting multiple concrete algorithms\.

##### Planner\.

The Planner selects components from the library and organizes them intoKKdifferent plans, with one plan assigned to each island\. Each plan contains an ordered sequence of steps\. The*subgoal and selected component*are fixed during ordinary code\-level evolution\. The*implementation annotation, step type, and time allocation*are revisable implementation fields\. The annotation describes a preliminary procedure that the Coder can follow directly\. This preserves the distinction between the plan’s structural specification and its implementation guidance\. The Planner distinguishes three step types, listed below\. It assigns time allocations according to these execution requirements and the shared per\-instance budget\.

- •Complete\-algorithm\.Runs a prescribed finite procedure to completion according to its algorithmic stopping rule\. Its runtime is estimated or measured based on the procedure and problem instance\.
- •Budget\-controlled\.Performs search within an allocated time limit\. Its execution is controlled by a local runtime budget\.
- •Hybrid\.Combines a required complete\-algorithm component with budget\-controlled search\. It requires a minimum time allocation for the mandatory component, with additional time available for search\.

##### Coder\.

The Coder translates a supplied plan into an initial executable program\. The initial\-generation prompt requires all plan steps to be implemented in order, following their annotations, step types, and runtime fields\. Intermediate outputs are passed between steps\. After each step, the program reports a solution and cumulative execution time, along with the step’s objective value or gap and execution time\. The Code Improver handles subsequent implementation changes, runtime reallocation, and permitted step deactivation\.

##### Debugger\.

The Debugger handles programs rejected because of compilation errors, invalid solutions, time\-limit exceedances, or other execution failures\. It supports two modes\. In*direct repair*, the agent receives the rejected source code and its error message\. It revises that implementation to address the reported failure while preserving the applicable plan requirements\. In*error\-aware regeneration*, the agent generates replacement code from the original generation context, augmented with the error message\. Rather than requiring a modification of the rejected implementation, this mode warns the agent against repeating the same failure during a new generation attempt\. Every repaired or regenerated candidate is evaluated again\. During evolution, debugging is bounded byMdbgM\_\{\\mathrm\{dbg\}\}\. During initialization, seed generation continues until validation succeeds, and the main evolution procedure begins only after every island has a valid seed\.

##### Code Improver\.

The Code Improver receives a parent program, inspiration programs, the current plan, the component library, and any available code\-level revision suggestions\. It generates a child while preserving the ordered subgoals and selected components\. Its revisions may change concrete algorithms, data structures, parameters, implementation annotations, step types, stopping rules, time allocations, and interactions between adjacent steps\. The parent and inspirations are selected by the system’s*SelectCode*function\. The Code Improver may also deactivate a step judged unnecessary, harmful, or too expensive by assigning it zero runtime while retaining its step identifier, component assignment, subgoal, and step type\. The reason for deactivation is recorded in the step metadata\. When a component has been removed from the active library, the Code Improver also receives guidance to bypass the corresponding step\. The Plan Improver remains responsible for deleting the recorded plan entry or replacing its assigned component\.

##### Plan Improver\.

The Plan Improver receives the selected island’s current plan and programs, details of the best\-performing plan, and sampled plans from other islands\. These records include components, subgoals, implementation annotations, time allocations, and measured performance\. The agent also receives the complete active component library and accumulated plan\-level feedback\. The proposed plan may add, remove, replace, or reorder steps and change their subgoals or selected components\. It includes preliminary implementation procedures and time allocations for the Coder\. If the current plan references a removed component, the proposal must delete the affected step or replace its component with an available alternative\. Every proposal that produces a valid seed starts a new plan generation, even when its content matches the preceding plan\.

##### Component Improver\.

The Component Improver reviews selected components using their contribution histories and aggregated Interpreter feedback\. It may refine descriptions, remove ineffective components, or consider missing high\-level components suggested by the Interpreter\. Only existing components used by the selected island’s current plan are eligible for revision or removal during that round\. Existing components absent from the plan remain unchanged\. Suggestions for genuinely new components are considered separately\. A description revision clears that component’s numerical contribution history\. A removal deletes its description from the active library but does not immediately rewrite existing programs or plans\. Subsequent code improvement receives a suggestion to bypass the affected step, while subsequent plan improvement must delete or replace it\.

##### Interpreter\.

The Interpreter compares the parent and child using their source code and implementation feedback\. This feedback includes each step’s objective value or gap, execution time, and relevant execution diagnostics\. The actual optimization solutions, such as complete schedules or decision\-variable assignments, are not included in the comparison context\. The Interpreter explains measured outcomes rather than replacing the evaluator’s assessment\. Its report contains four components, detailed below\. Code\-level insights are associated with the parent and child\. Plan\-level reports accumulate for the current plan generation; reports from retired generations remain historical records rather than entering the new generation’s active feedback pool\. Component\-level reports accumulate across relevant evaluations and islands\. A proposed addition must describe a missing high\-level search family rather than only an algorithm, neighborhood, parameter, or solver setting\.

- •Change Summary:Records the measured outcome as*better*,*worse*, or*equivalent*\. Each material change identifies the affected step, change type, parent and child behaviors, observed effect, and attribution confidence\. The summary becomes part of the child’s evolution history\.
- •Code\-level Insight:Assesses implementation changes under the unchanged subgoals and components\. The newly generated code is identified as*effective*,*regression*,*neutral*, or*not established*\. To support the argument, the interpreter includes supporting evidence and a concrete implementation direction\. These insights are available to later Code Improver calls\.
- •Plan\-level Insight:Assesses step usefulness, component suitability, organization, and runtime allocation\. Step assessments use*helpful*,*supporting*,*implementation limited*,*component mismatch*,*subgoal unhelpful*, or*not established*\. Structural recommendations use*revise*,*add*, or*remove*\.
- •Component\-level Insight:Produces one record per unique component used by the plan, consolidating evidence when several steps share that component\. It records observed implementations, transferable lessons, and proposed operations such as*keep*,*propose revise*, or*propose add*\.

### C\.3Initialization

Initialization constructs the component library, generatesKKplans, and obtains one validated seed for every island\. It then performsWWlocal code\-improvement visits per island before global scheduling begins\. Component descriptions remain unchanged during these initial local visits\. Algorithm[C1](https://arxiv.org/html/2609.36303#alg1)gives the initialization procedure\.

Algorithm C1The Initialization Procedure of HeurEvo\.1:Problem description,

KKislands, training set

𝒟\\mathcal\{D\}, per\-instance budget

BB, local steps

W≥1W\\geq 1, and debugging limit

MdbgM\_\{\\mathrm\{dbg\}\}\.

2:

𝒮0←𝖯𝗋𝗈𝗉𝗈𝗌𝖾𝖢𝗈𝗆𝗉𝗈𝗇𝖾𝗇𝗍𝗌⁡\(problem\),𝒜←∅\\mathcal\{S\}\_\{0\}\\leftarrow\\mathsf\{ProposeComponents\}\(\\text\{problem\}\),\\quad\\mathcal\{A\}\\leftarrow\\varnothing
3:

\{P0\(k\)\}k=1K←𝖢𝗋𝖾𝖺𝗍𝖾𝖯𝗅𝖺𝗇𝗌⁡\(𝒮0,K,B\)\\\{P\_\{0\}^\{\(k\)\}\\\}\_\{k=1\}^\{K\}\\leftarrow\\mathsf\{CreatePlans\}\(\\mathcal\{S\}\_\{0\},K,B\)
4:Initialize an empty contribution array for each

s∈𝒮0s\\in\\mathcal\{S\}\_\{0\}
5:// Seed generation

6:for

k=1,…,Kk=1,\\ldots,Kdo

7:

\(G0\(k,seed\),F0\(k,seed\)\)←𝖢𝗈𝖽𝖾𝖯𝗅𝖺𝗇⁡\(P0\(k\),𝒮0,B\)\(G\_\{0\}^\{\(k,\\mathrm\{seed\}\)\},F\_\{0\}^\{\(k,\\mathrm\{seed\}\)\}\)\\leftarrow\\mathsf\{CodePlan\}\(P\_\{0\}^\{\(k\)\},\\mathcal\{S\}\_\{0\},B\)
8:Insert the validated seed into

𝒜\\mathcal\{A\}and associate it with island

kk
9:

𝒜0,0\(k,par\)←\{G0\(k,seed\)\},F0,0\(k,⋆\)←F0\(k,seed\)\\mathcal\{A\}\_\{0,0\}^\{\(k,\\mathrm\{par\}\)\}\\leftarrow\\\{G\_\{0\}^\{\(k,\\mathrm\{seed\}\)\}\\\},\\quad F\_\{0,0\}^\{\(k,\\star\)\}\\leftarrow F\_\{0\}^\{\(k,\\mathrm\{seed\}\)\}
10:

\(R0,0\(k\),V0,0\(k\),H0,0\(k\),n0,0\(k\)\)←\(0,1,0,0\)\(R\_\{0,0\}^\{\(k\)\},V\_\{0,0\}^\{\(k\)\},H\_\{0,0\}^\{\(k\)\},n\_\{0,0\}^\{\(k\)\}\)\\leftarrow\(0,1,0,0\)
11:endfor

12:// Local code evolution

13:for

k=1,…,Kk=1,\\ldots,Kdo

14:for

h=1,…,Wh=1,\\ldots,Wdo

15:

\(G0,h\(k,par\),ℐ0,h\(k\)\)←𝖲𝖾𝗅𝖾𝖼𝗍𝖢𝗈𝖽𝖾⁡\(𝒜0,h−1\(k,par\),𝒜\)\(G\_\{0,h\}^\{\(k,\\mathrm\{par\}\)\},\\mathcal\{I\}\_\{0,h\}^\{\(k\)\}\)\\leftarrow\\mathsf\{SelectCode\}\(\\mathcal\{A\}\_\{0,h\-1\}^\{\(k,\\mathrm\{par\}\)\},\\mathcal\{A\}\)
16:

\(G0,h\(k,child\),F0,h\(k,child\)\)←𝖨𝗆𝗉𝗋𝗈𝗏𝖾𝖢𝗈𝖽𝖾⁡\(k,G0,h\(k,par\),ℐ0,h\(k\)\)\(G\_\{0,h\}^\{\(k,\\mathrm\{child\}\)\},F\_\{0,h\}^\{\(k,\\mathrm\{child\}\)\}\)\\leftarrow\\mathsf\{ImproveCode\}\(k,G\_\{0,h\}^\{\(k,\\mathrm\{par\}\)\},\\mathcal\{I\}\_\{0,h\}^\{\(k\)\}\)
17:if

G0,h\(k,child\)≠⊥G\_\{0,h\}^\{\(k,\\mathrm\{child\}\)\}\\neq\\botthen

18:

Φ0,h\(k\)←𝖨𝗇𝗍𝖾𝗋𝗉𝗋𝖾𝗍⁡\(G0,h\(k,par\),G0,h\(k,child\)\)\\Phi\_\{0,h\}^\{\(k\)\}\\leftarrow\\mathsf\{Interpret\}\(G\_\{0,h\}^\{\(k,\\mathrm\{par\}\)\},G\_\{0,h\}^\{\(k,\\mathrm\{child\}\)\}\)
19:

𝖴𝗉𝖽𝖺𝗍𝖾𝖲𝗍𝖺𝗍𝖾⁡\(k,G0,h\(k,child\),F0,h\(k,child\),Φ0,h\(k\)\)\\mathsf\{UpdateState\}\(k,G\_\{0,h\}^\{\(k,\\mathrm\{child\}\)\},F\_\{0,h\}^\{\(k,\\mathrm\{child\}\)\},\\Phi\_\{0,h\}^\{\(k\)\}\)⊳\\trianglerightEqs\.[C2](https://arxiv.org/html/2609.36303#A3.E2)–[C3](https://arxiv.org/html/2609.36303#A3.E3)

20:else

21:Apply Eq\.[C3](https://arxiv.org/html/2609.36303#A3.E3)with

r0,h\(k\)=δ0,h\(k\)=0r\_\{0,h\}^\{\(k\)\}=\\delta\_\{0,h\}^\{\(k\)\}=0; add no program

22:Retain the parent\-eligible collection and incumbent for the next local update

23:endif

24:endfor

25:Record the final plan, programs, feedback, and statistics of island

kkat index

00
26:endfor

27:

𝒜0←𝒜,N0←∑k=1Kn0\(k\)=K​W\\mathcal\{A\}\_\{0\}\\leftarrow\\mathcal\{A\},\\quad N\_\{0\}\\leftarrow\\sum\_\{k=1\}^\{K\}n\_\{0\}^\{\(k\)\}=KW
28:return

KKinitialized islands,

𝒮0\\mathcal\{S\}\_\{0\}, and

𝒜0\\mathcal\{A\}\_\{0\}

In Algorithm[C1](https://arxiv.org/html/2609.36303#alg1),*CodePlan*and*ImproveCode*include generation, evaluation, and debugging internally\. Each successful call returns the validated program together with its average training fitness\. Evaluation records remain available for subsequent interpretation\.

During initialization,*CodePlan*continues repairing or regenerating a seed until validation succeeds\.*ImproveCode*instead uses bounded debugging\. If no valid child is obtained within that limit, it returns\(⊥,⊥\)\(\\bot,\\bot\)\. The rejected candidate is not inserted into the Program Database, and the local visit receives zero reward and zero relative improvement\.

Each valid seed initializes its island’s parent\-eligible collection and incumbent\. Before local code improvement,R,V,H,nR,V,H,nare initialized to0,1,0,00,1,0,0, respectively\. Seed generation does not count as code improvement\.

Every scheduled*ImproveCode*call counts as one visit, whether it succeeds immediately, requires debugging, or remains invalid\. The successful update or the failure update incrementsnnexactly once\. Consequently,

n0\(k\)=W,N0=K​W\.\\displaystyle n\_\{0\}^\{\(k\)\}=W,\\qquad N\_\{0\}=KW\.The requirementW≥1W\\geq 1ensures that every island has a positive visit count before the first UCB selection\. Initialization is separate from theJJglobal scheduling rounds\.

### C\.4Evolution

##### Location of the deferred details\.

The exact scheduling rules and complete evolution algorithm deferred from Section[3\.3](https://arxiv.org/html/2609.36303#S3.SS3)are collected here\. References below to the “main\-text” selection or update rules refer to Eqs\.[C1](https://arxiv.org/html/2609.36303#A3.E1)–[C3](https://arxiv.org/html/2609.36303#A3.E3)in this subsection\. Initialization and its starting values remain in Appendix[C\.3](https://arxiv.org/html/2609.36303#A3.SS3); restart, timeout, and invalid\-child handling are specified in the operator descriptions below\.

##### UCB island\-selection rule\.

At the beginning of global roundjj, the controller selects an island using its pre\-round statistics:

kj\\displaystyle k\_\{j\}=arg⁡maxk⁡\[Rj−1\(k\)Vj−1\(k\)\+C​log⁡Nj−1nj−1\(k\)\],\\displaystyle=\\arg\\max\_\{k\}\\left\[\\frac\{R\_\{j\-1\}^\{\(k\)\}\}\{V\_\{j\-1\}^\{\(k\)\}\}\+C\\sqrt\{\\frac\{\\log N\_\{j\-1\}\}\{n\_\{j\-1\}^\{\(k\)\}\}\}\\right\],\(C1\)Nj−1\\displaystyle N\_\{j\-1\}=∑ℓ=1Knj−1\(ℓ\)\.\\displaystyle=\\sum\_\{\\ell=1\}^\{K\}n\_\{j\-1\}^\{\(\\ell\)\}\.Here,RRandVVsummarize discounted rewards and exposure\. The countnnrecords scheduled code\-improvement visits to one island, andNNtotals these visits across islands\. The coefficientCCcontrols exploration\. The first term favors recent normalized progress; the second encourages exploration of under\-visited islands\. Counts include unsuccessful visits, not individual debugging attempts\. Initialization withW≥1W\\geq 1ensures positive counts before the first selection\.

##### Normalized reward and relative improvement\.

Lethhindex local code\-improvement visits in roundjj, withk=kjk=k\_\{j\}\. For a validated child, letFj,h\(k,child\)F\_\{j,h\}^\{\(k,\\mathrm\{child\}\)\}denote its average training fitness under budgetBB\. Using the selected island’s incumbentFj,h−1\(k,⋆\)F\_\{j,h\-1\}^\{\(k,\\star\)\}and the current global incumbentFj,h−1\(global,⋆\)F\_\{j,h\-1\}^\{\(\\mathrm\{global\},\\star\)\}immediately before evaluation, define

rj,h\(k\)\\displaystyle r\_\{j,h\}^\{\(k\)\}=clip⁡\(Fj,h\(k,child\)−Fj,h−1\(k,⋆\)max⁡\(Fj,h−1\(global,⋆\)−Fj,h−1\(k,⋆\),0\)\+ϵ,0,1\),\\displaystyle=\\operatorname\{clip\}\\\!\\left\(\\frac\{F\_\{j,h\}^\{\(k,\\mathrm\{child\}\)\}\-F\_\{j,h\-1\}^\{\(k,\\star\)\}\}\{\\max\\\!\\left\(F\_\{j,h\-1\}^\{\(\\mathrm\{global\},\\star\)\}\-F\_\{j,h\-1\}^\{\(k,\\star\)\},0\\right\)\+\\epsilon\},0,1\\right\),\(C2\)δj,h\(k\)\\displaystyle\\delta\_\{j,h\}^\{\(k\)\}=max⁡\(Fj,h\(k,child\)−Fj,h−1\(k,⋆\)\|Fj,h−1\(k,⋆\)\|\+ϵ,0\)\.\\displaystyle=\\max\\\!\\left\(\\frac\{F\_\{j,h\}^\{\(k,\\mathrm\{child\}\)\}\-F\_\{j,h\-1\}^\{\(k,\\star\)\}\}\{\\left\|F\_\{j,h\-1\}^\{\(k,\\star\)\}\\right\|\+\\epsilon\},0\\right\)\.Here,clip⁡\(z,0,1\)\\operatorname\{clip\}\(z,0,1\)truncateszzto\[0,1\]\[0,1\], andϵ\>0\\epsilon\>0prevents division by zero\. The reward compares the child’s gain over the island’s incumbent with that island’s gap to the current global incumbent, both measured before the update\. For example, if these incumbents have fitness8080and100100, respectively, a child with fitness9090receives reward10/\(20\+ϵ\)≈0\.510/\(20\+\\epsilon\)\\approx 0\.5\. A child that does not improve on the island’s incumbent receives zero reward\. If the island already matches the global incumbent, the denominator isϵ\\epsilon, so any gain of at leastϵ\\epsilonreceives reward11\. In contrast,δ\\deltameasures nonnegative relative improvement within the island\. These comparisons precede incumbent replacement\. The Interpreter instead compares the child with its sampled parent\.

##### Discounted scheduling updates\.

Each scheduled code\-improvement visit updates the selected island’s statistics:

Rj,h\(k\)\\displaystyle R\_\{j,h\}^\{\(k\)\}=ρ​Rj,h−1\(k\)\+rj,h\(k\),\\displaystyle=\\rho R\_\{j,h\-1\}^\{\(k\)\}\+r\_\{j,h\}^\{\(k\)\},Vj,h\(k\)\\displaystyle\\qquad V\_\{j,h\}^\{\(k\)\}=ρ​Vj,h−1\(k\)\+1,\\displaystyle=\\rho V\_\{j,h\-1\}^\{\(k\)\}\+1,\(C3\)Hj,h\(k\)\\displaystyle H\_\{j,h\}^\{\(k\)\}=ρ​Hj,h−1\(k\)\+\(1−ρ\)​δj,h\(k\),\\displaystyle=\\rho H\_\{j,h\-1\}^\{\(k\)\}\+\(1\-\\rho\)\\delta\_\{j,h\}^\{\(k\)\},nj,h\(k\)\\displaystyle n\_\{j,h\}^\{\(k\)\}=nj,h−1\(k\)\+1\.\\displaystyle=n\_\{j,h\-1\}^\{\(k\)\}\+1\.The factorρ∈\[0,1\)\\rho\\in\[0,1\)controls discounting, andHHis the recent\-progress signal used for plan review\. If no valid child is obtained, the same update is applied once withr=δ=0r=\\delta=0, without fabricating a fitness or replacing the incumbent\. Non\-selected islands retain their statistics\.

##### Plateau criterion and complete procedure\.

For the selected island,Hj−1\(k\)≥τH\_\{j\-1\}^\{\(k\)\}\\geq\\tauresults in one ordinary code\-improvement visit, whereasHj−1\(k\)<τH\_\{j\-1\}^\{\(k\)\}<\\tautriggers a plan review\. A proposal starts a new plan generation only after its seed is validated, followed byWWconsecutive local visits without another island selection or threshold test\. If seed generation fails, the old plan and programs remain active and the round proceeds with one ordinary code\-improvement visit\.

Algorithm[C2](https://arxiv.org/html/2609.36303#alg2)makes this update order explicit\. As in Algorithm[C1](https://arxiv.org/html/2609.36303#alg1),*CodePlan*and*ImproveCode*include evaluation and debugging internally and return a validated program and its measured fitness, or\(⊥,⊥\)\(\\bot,\\bot\)after bounded failure during evolution\. The temporary proposalP~\\widetilde\{P\}does not replace the current plan until*RestartIsland*succeeds\.

Algorithm C2The Evolution Procedure of HeurEvo\.1:

KKinitialized islands, training set

𝒟\\mathcal\{D\}, per\-instance budget

BB, rounds

JJ, local steps

W≥1W\\geq 1, threshold

τ\\tau, library

𝒮0\\mathcal\{S\}\_\{0\}, codebase archive

𝒜0\\mathcal\{A\}\_\{0\}, and debugging limit

MdbgM\_\{\\mathrm\{dbg\}\}\.

2:for

j=1,…,Jj=1,\\ldots,Jdo

3:Carry forward all islands’ plans, populations, feedback, and statistics

4:

k←𝖲𝖾𝗅𝖾𝖼𝗍𝖨𝗌𝗅𝖺𝗇𝖽⁡\(j\),L←1k\\leftarrow\\mathsf\{SelectIsland\}\(j\),\\quad L\\leftarrow 1⊳\\trianglerightEq\.[C1](https://arxiv.org/html/2609.36303#A3.E1)

5:Initialize local state

h=0h=0from the preceding round

6:if

Hj−1\(k\)<τH\_\{j\-1\}^\{\(k\)\}<\\tauthen

7:

P~←𝖨𝗆𝗉𝗋𝗈𝗏𝖾𝖯𝗅𝖺𝗇⁡\(k\)\\widetilde\{P\}\\leftarrow\\mathsf\{ImprovePlan\}\(k\)
8:

\(Gj\(k,seed\),Fj\(k,seed\)\)←𝖢𝗈𝖽𝖾𝖯𝗅𝖺𝗇⁡\(P~,𝒮j−1,B\)\(G\_\{j\}^\{\(k,\\mathrm\{seed\}\)\},F\_\{j\}^\{\(k,\\mathrm\{seed\}\)\}\)\\leftarrow\\mathsf\{CodePlan\}\(\\widetilde\{P\},\\mathcal\{S\}\_\{j\-1\},B\)
9:if

Gj\(k,seed\)≠⊥G\_\{j\}^\{\(k,\\mathrm\{seed\}\)\}\\neq\\botthen

10:

𝖱𝖾𝗌𝗍𝖺𝗋𝗍𝖨𝗌𝗅𝖺𝗇𝖽⁡\(k,P~,Gj\(k,seed\),Fj\(k,seed\)\)\\mathsf\{RestartIsland\}\(k,\\widetilde\{P\},G\_\{j\}^\{\(k,\\mathrm\{seed\}\)\},F\_\{j\}^\{\(k,\\mathrm\{seed\}\)\}\)
11:

L←WL\\leftarrow W
12:endif

13:endif

14:for

h=1,…,Lh=1,\\ldots,Ldo

15:Refresh the global incumbent from the current island incumbents

16:

\(Gj,h\(k,par\),ℐj,h\(k\)\)←𝖲𝖾𝗅𝖾𝖼𝗍𝖢𝗈𝖽𝖾⁡\(𝒞j,h−1\(k\),𝒜j,h−1\)\(G\_\{j,h\}^\{\(k,\\mathrm\{par\}\)\},\\mathcal\{I\}\_\{j,h\}^\{\(k\)\}\)\\leftarrow\\mathsf\{SelectCode\}\(\\mathcal\{C\}\_\{j,h\-1\}^\{\(k\)\},\\mathcal\{A\}\_\{j,h\-1\}\)
17:

\(Gj,h\(k,child\),Fj,h\(k,child\)\)←𝖨𝗆𝗉𝗋𝗈𝗏𝖾𝖢𝗈𝖽𝖾⁡\(k,Gj,h\(k,par\),ℐj,h\(k\)\)\(G\_\{j,h\}^\{\(k,\\mathrm\{child\}\)\},F\_\{j,h\}^\{\(k,\\mathrm\{child\}\)\}\)\\leftarrow\\mathsf\{ImproveCode\}\(k,G\_\{j,h\}^\{\(k,\\mathrm\{par\}\)\},\\mathcal\{I\}\_\{j,h\}^\{\(k\)\}\)
18:if

Gj,h\(k,child\)≠⊥G\_\{j,h\}^\{\(k,\\mathrm\{child\}\)\}\\neq\\botthen

19:

Φj,h\(k\)←𝖨𝗇𝗍𝖾𝗋𝗉𝗋𝖾𝗍⁡\(Gj,h\(k,par\),Gj,h\(k,child\)\)\\Phi\_\{j,h\}^\{\(k\)\}\\leftarrow\\mathsf\{Interpret\}\(G\_\{j,h\}^\{\(k,\\mathrm\{par\}\)\},G\_\{j,h\}^\{\(k,\\mathrm\{child\}\)\}\)
20:

𝖴𝗉𝖽𝖺𝗍𝖾𝖲𝗍𝖺𝗍𝖾⁡\(k,Gj,h\(k,child\),Fj,h\(k,child\),Φj,h\(k\)\)\\mathsf\{UpdateState\}\(k,G\_\{j,h\}^\{\(k,\\mathrm\{child\}\)\},F\_\{j,h\}^\{\(k,\\mathrm\{child\}\)\},\\Phi\_\{j,h\}^\{\(k\)\}\)⊳\\trianglerightEqs\.[C2](https://arxiv.org/html/2609.36303#A3.E2)–[C3](https://arxiv.org/html/2609.36303#A3.E3)

21:else

22:Apply Eq\.[C3](https://arxiv.org/html/2609.36303#A3.E3)with

rj,h\(k\)=δj,h\(k\)=0r\_\{j,h\}^\{\(k\)\}=\\delta\_\{j,h\}^\{\(k\)\}=0
23:Retain programs, incumbent, and feedback; add no program

24:endif

25:endfor

26:Record the selected island’s final local state and archive at round

jj
27:

Nj←∑ℓ=1Knj\(ℓ\)N\_\{j\}\\leftarrow\\sum\_\{\\ell=1\}^\{K\}n\_\{j\}^\{\(\\ell\)\}
28:

ℛj←𝖲𝖾𝗅𝖾𝖼𝗍𝖢𝗈𝗆𝗉𝗈𝗇𝖾𝗇𝗍𝗌⁡\(𝒮j−1\)\\mathcal\{R\}\_\{j\}\\leftarrow\\mathsf\{SelectComponents\}\(\\mathcal\{S\}\_\{j\-1\}\)⊳\\trianglerightEq\.[C8](https://arxiv.org/html/2609.36303#A3.E8)

29:

𝒮j←𝖨𝗆𝗉𝗋𝗈𝗏𝖾𝖢𝗈𝗆𝗉𝗈𝗇𝖾𝗇𝗍𝗌⁡\(𝒮j−1,ℛj\)\\mathcal\{S\}\_\{j\}\\leftarrow\\mathsf\{ImproveComponents\}\(\\mathcal\{S\}\_\{j\-1\},\\mathcal\{R\}\_\{j\}\)
30:endfor

31:return

arg​maxG∈𝒜J⁡F​\(G\)\\displaystyle\\operatorname\*\{arg\\,max\}\_\{G\\in\\mathcal\{A\}\_\{J\}\}F\(G\)

##### Compact controller notation in the main text\.

The pseudocode in Section[C\.4](https://arxiv.org/html/2609.36303#A3.SS4)suppresses round indices\. Immediately before global roundjj, the compact variables correspond to

k⋆=kj,r¯k=Rj−1\(k\)Vj−1\(k\),nk=nj−1\(k\),ntot=Nj−1,Hk=Hj−1\(k\)\.k^\{\\star\}=k\_\{j\},\\quad\\bar\{r\}\_\{k\}=\\frac\{R\_\{j\-1\}^\{\(k\)\}\}\{V\_\{j\-1\}^\{\(k\)\}\},\\quad n\_\{k\}=n\_\{j\-1\}^\{\(k\)\},\\quad n\_\{\\mathrm\{tot\}\}=N\_\{j\-1\},\\quad H\_\{k\}=H\_\{j\-1\}^\{\(k\)\}\.They express the same UCB selection and plateau criterion specified above, without changing the reward or update rules\.

##### System functions and agent\-response calls\.

System functions execute prescribed sampling, database, evaluation, and numerical\-update rules\. Agent\-response calls construct prompts, obtain structured responses, and parse the requested outputs\.

Table C2:System functions and agent\-response calls\.Call categoryOperatorsSystem functions*SelectIsland*,*SelectCode*,*RestartIsland*,*UpdateState*,*SelectComponents*Agent\-response calls*ImprovePlan*,*CodePlan*,*ImproveCode*,*Interpret*,*ImproveComponents**CodePlan*and*ImproveCode*additionally include system\-controlled evaluation and debugging\. Their agents propose source code, but the returned fitness is measured by execution rather than supplied by an agent\. The internal evaluator also determines whether a candidate must be sent to the Debugger\. No separate*EvaluateAndDebug*call is required in the pseudocode\.

##### Island selection \(*SelectIsland*\)\.

*SelectIsland*computes the UCB scores using Eq\.[C1](https://arxiv.org/html/2609.36303#A3.E1)in the main text and returns the selected island index\. The decision uses the statistics available before the current round\.

For an ordinary code\-improvement visit, local indexh=0h=0refers to the programs, incumbents, and statistics carried from the preceding global round\. After an accepted plan proposal,h=0h=0instead refers to the new seed and the quantities initialized by*RestartIsland*\.

The cumulative countsnnandNNretain their history across plan generations\. They are distinct fromVV, the discounted exposure statistic that is reset when a new generation starts\.

##### Plan proposal \(*ImprovePlan*\)\.

*ImprovePlan*gathers the selected island’s plan and programs, cross\-island plan details, active component descriptions, and accumulated plan\-level feedback\. It passes them to the Plan Improver and parses the returned proposal\.

The existing plan remains available while the proposal is being implemented and validated\. A proposal does not replace the current plan before a valid seed is obtained\. Plan equality is not an acceptance criterion\.

##### Seed generation and validation \(*CodePlan*\)\.

*CodePlan*passes the proposed plan, component descriptions, and execution budget to the Coder\. It evaluates the generated program and invokes the Debugger when compilation, validity, or runtime checks fail\.

A successful call returns

\(G\(seed\),F\(seed\)\),\\left\(G^\{\(\\mathrm\{seed\}\)\},F^\{\(\\mathrm\{seed\}\)\}\\right\),where the program is the final validated implementation and the fitness comes from its evaluation\.

Initialization continues this process until a valid seed is obtained\. During plan evolution, the configured debugging limit applies\. If debugging is exhausted,*CodePlan*returns\(⊥,⊥\)\(\\bot,\\bot\)\. The proposal is then discarded, the island retains its previous plan and programs, and the round resumes with one ordinary code\-improvement visit\.

A failed seed\-generation call does not increment the code\-improvement visit count and does not itself trigger a scheduling\-reward update\.

##### Island restart \(*RestartIsland*\)\.

*RestartIsland*is invoked only after a proposed plan produces a valid seed\. It installs the proposed plan and initializes its parent\-eligible collection and incumbent with that seed\. This handling applies even when the proposal has identical content to the previous plan or its seed has lower fitness than the previous incumbent\.

Programs from the previous generation remain in the codebase according to the database’s retention policy, but they are no longer eligible parents for the new generation\. Previous plan\-level feedback remains archived, and a new current\-generation feedback pool is started\.

The function resetsR,V,HR,V,Hto0,1,00,1,0, respectively, without resettingnnorNN\. The seed establishes a new within\-generation baseline and does not receive an improvement reward against the retired incumbent\.

The island then receivesWWconsecutive local code\-improvement visits\. Neither island selection nor the plan\-evolution threshold is reconsidered between these visits\.

##### Parent and inspiration selection \(*SelectCode*\)\.

*SelectCode*samples one parent from the selected island’s current parent\-eligible collection and inspiration programs from the codebase\. The parent must belong to the current plan generation, while inspirations may provide examples from other plans\.

The input𝒜\\mathcal\{A\}identifies the complete codebase\. The collection𝒜\(k,ins\)\\mathcal\{A\}^\{\(k,\\mathrm\{ins\}\)\}identifies the programs available as inspirations for the selected island under the configured database policy\. The returnedℐj,h\(k\)\\mathcal\{I\}\_\{j,h\}^\{\(k\)\}contains the programs actually sampled for that visit\.

The reference backends use related but distinct mechanisms\. OpenEvolve’s documented parent\-selection mechanisms combine random island sampling, elite selection, and fitness\-weighted sampling\. Its configuration supports both high\-performing and diverse programs as prompt examples\([Sharma, 2025](https://arxiv.org/html/2609.36303#bib.bib49)\)\. AdaEvolve’s published sampling subroutine switches between uniform parent selection with diverse inspirations and parent selection from the highest\-fitness quartile with high\-fitness inspirations\([Cemri et al\., 2026](https://arxiv.org/html/2609.36303#bib.bib7)\)\.

HeurEvo applies its current\-plan eligibility restriction through this database interface\. Parent and inspiration selection are repeated before every local update, so later visits can use validated programs produced earlier in the same round\.

##### Code generation and validation \(*ImproveCode*\)\.

*ImproveCode*constructs the Code Improver’s prompt from the sampled programs, current plan, component library, and available code\-level feedback\. It then evaluates the generated child and performs bounded debugging when necessary\.

A successful call returns the final validated child and its measured average training fitness\. If debugging produces a valid repaired or regenerated program, that final program—not an earlier rejected version—is returned\.

When no valid child is obtained, the call returns\(⊥,⊥\)\(\\bot,\\bot\)\. Intermediate rejected versions are not inserted into the Program Database\. All internal generation and debugging attempts remain part of the same code\-improvement visit\.

A valid child with degraded fitness is different from a rejected child\. It can still be interpreted and submitted to the database’s ordinary admission and retention procedure\.

##### Program comparison \(*Interpret*\)\.

*Interpret*retrieves the selected parent’s evaluation record and the final validated child’s record\. It supplies the source code and per\-step implementation feedback to the Interpreter and parses the four\-part report\.

When debugging has modified the child, the comparison uses the validated result and the originally selected parent\. No additional program evaluation is required merely to construct this comparison\.

An unrepaired child does not receive a normal parent–child performance interpretation because no valid child fitness is available\. Its diagnostics have already been used by the Debugger\.

##### Database insertion and updates \(*UpdateState*\)\.

*UpdateState*computes the normalized reward and relative improvement for a valid child using the incumbent fitnesses immediately before the child’s evaluation\. It then applies Eqs\.[C2](https://arxiv.org/html/2609.36303#A3.E2)–[C3](https://arxiv.org/html/2609.36303#A3.E3)in the main text\. Incumbent replacement occurs after these comparisons\.

The Interpreter and scheduler use different references\. The Interpreter compares the child with its sampled parent\. Scheduling measures progress against the selected island’s incumbent, which need not be the sampled parent\.

The validated child is submitted to the Program Database with its measured fitness, island and plan\-generation association, implementation measurements, and Interpreter report\. The database manages retention and parent eligibility\. Feedback is made available to subsequent calls, and measured component contributions are appended to the appropriate arrays\.

If*ImproveCode*returns failure, the visit receives

rj,h\(k\)=0,δj,h\(k\)=0\.r\_\{j,h\}^\{\(k\)\}=0,\\qquad\\delta\_\{j,h\}^\{\(k\)\}=0\.The main\-text update rule is applied with these values, including one increment ofnn\. No numerical child fitness is fabricated, no program is inserted, and the incumbent remains unchanged\. The discounting rule reduces previously accumulated positive reward when no new reward is obtained\.

A failed visit inside aWW\-visit block does not terminate the block\. The procedure continues with the next local visit using the available parent\-eligible programs\. Rejected attempts do not automatically contribute zero\-valued component observations, because scheduling rewards and measured step contributions serve different purposes\.

##### Component selection \(*SelectComponents*\)\.

*SelectComponents*considers existing components that are both present in the active library and used by the selected island’s current plan\. A component also needs at least one contribution observation collected under its current description\.

The function computes the review probabilities defined in Appendix[C\.5](https://arxiv.org/html/2609.36303#A3.SS5)and returnsℛj\\mathcal\{R\}\_\{j\}\. Existing components absent from the selected plan remain unchanged during that round\. New\-component suggestions are considered separately from the contribution\-based selection of existing components\.

##### Component revision and deferred removal \(*ImproveComponents*\)\.

*ImproveComponents*passes the selected components, contribution histories, aggregated feedback, and addition suggestions to the Component Improver\. Selection requests a review rather than requiring an edit\.

When a description is revised, only that component’s contribution array is cleared\. Arrays for unchanged components are retained\. New components begin with empty arrays\.

When a component is removed, its description is deleted from the active library\. Existing plans and stored programs are not immediately rewritten, and their measured fitness values remain associated with the original programs\. Subsequent code\-improvement prompts suggest bypassing the affected step with zero allocated time\. Any child implementing that suggestion must pass ordinary validation\. Deletion or replacement of the recorded plan entry occurs when the Plan Improver next revises the affected plan\. It must delete that entry or replace its component with an available alternative\.

##### Completion of a global round\.

Local visits are processed sequentially\. Each visit uses the latest parent\-eligible collection, codebase, incumbent, and feedback\. The global reference is refreshed from the current island incumbents before each child evaluation\.

After the final local visit, the selected island’s resulting programs, feedback, and statistics are recorded for roundjj\. Non\-selected islands retain their corresponding quantities\. Component review then produces the library used by subsequent rounds\.

The current global incumbent is the best program among the current island plans\. The archive\-best program is the best retained program in the codebase, which may belong to a retired generation\. Algorithm[C2](https://arxiv.org/html/2609.36303#alg2)returns the latter\.

### C\.5Component Contribution and Review

##### Intermediate fitness\.

For a program withmmsteps, letyiy\_\{i\}denote its average best\-so\-far training fitness after stepii, with final fitnessy=ymy=y\_\{m\}\. The recorded sequence satisfies

y1≤y2≤⋯≤ym=y\.y\_\{1\}\\leq y\_\{2\}\\leq\\cdots\\leq y\_\{m\}=y\.These are higher\-is\-better fitness values, rather than raw minimization objectives or gaps\. Equality is allowed when a step does not improve the incumbent\.

Letygy\_\{g\}denote the current global incumbent immediately before evaluation\. The first\-step reference is the mean of the recorded first\-step observations available to the contribution calculation,

y¯1=1Mfirst​∑q=1Mfirstyq,1\.\\bar\{y\}\_\{1\}=\\frac\{1\}\{M\_\{\\mathrm\{first\}\}\}\\sum\_\{q=1\}^\{M\_\{\\mathrm\{first\}\}\}y\_\{q,1\}\.\(C4\)Here,MfirstM\_\{\\mathrm\{first\}\}counts first\-step observations and is distinct from the code\-improvement visit countNN\.

##### Contributions in multi\-step programs\.

Form≥2m\\geq 2, the observed contribution of stepiiis

Δi=\{0,i=1,y−y¯1≤0,clip⁡\(y1−y¯1y−y¯1\+ϵ,0,1\),i=1,y−y¯1\>0,clip⁡\(yi−yi−1y−yi−1\+ϵ,0,1\),2≤i<m,clip⁡\(yi−yi−1max⁡\(yg,y\)−yi−1\+ϵ,0,1\),i=m\.\\Delta\_\{i\}=\\begin\{cases\}0,&i=1,\\;y\-\\bar\{y\}\_\{1\}\\leq 0,\\\\\[5\.0pt\] \\operatorname\{clip\}\\\!\\left\(\\dfrac\{y\_\{1\}\-\\bar\{y\}\_\{1\}\}\{y\-\\bar\{y\}\_\{1\}\+\\epsilon\},0,1\\right\),&i=1,\\;y\-\\bar\{y\}\_\{1\}\>0,\\\\\[10\.0pt\] \\operatorname\{clip\}\\\!\\left\(\\dfrac\{y\_\{i\}\-y\_\{i\-1\}\}\{y\-y\_\{i\-1\}\+\\epsilon\},0,1\\right\),&2\\leq i<m,\\\\\[10\.0pt\] \\operatorname\{clip\}\\\!\\left\(\\dfrac\{y\_\{i\}\-y\_\{i\-1\}\}\{\\max\(y\_\{g\},y\)\-y\_\{i\-1\}\+\\epsilon\},0,1\\right\),&i=m\.\\end\{cases\}\(C5\)The first step is compared with the historical first\-step reference\. Intermediate steps are normalized by the remaining improvement obtained by the complete program\. The final step is normalized by the remaining distance to the better of the program’s final fitness and the pre\-evaluation global incumbent\. The score prioritizes component review rather than estimating a causal effect\.

With full indices restored, the contribution of stepiiin local updatehhis denoted byΔj,h,i\(k\)\\Delta\_\{j,h,i\}^\{\(k\)\}\.

##### Contributions in single\-step programs\.

Form=1m=1, the only step is also the final step\. Settingy0=y¯1y\_\{0\}=\\bar\{y\}\_\{1\}, its contribution is

Δ1=\{0,max⁡\(yg,y1\)≤y¯1,clip⁡\(y1−y¯1max⁡\(yg,y1\)−y¯1\+ϵ,0,1\),max⁡\(yg,y1\)\>y¯1\.\\Delta\_\{1\}=\\begin\{cases\}0,&\\max\(y\_\{g\},y\_\{1\}\)\\leq\\bar\{y\}\_\{1\},\\\\\[7\.0pt\] \\operatorname\{clip\}\\\!\\left\(\\dfrac\{y\_\{1\}\-\\bar\{y\}\_\{1\}\}\{\\max\(y\_\{g\},y\_\{1\}\)\-\\bar\{y\}\_\{1\}\+\\epsilon\},0,1\\right\),&\\max\(y\_\{g\},y\_\{1\}\)\>\\bar\{y\}\_\{1\}\.\\end\{cases\}\(C6\)The special case assigns zero contribution when the historical reference is at least as large as both the program’s outcome and the current global incumbent\. Otherwise, the denominator is strictly positive\.

##### Per\-component contribution histories\.

Each component maintains an independent contribution array,

𝚫j​\(s\)=\[Δs\[1\],…,Δs\[Mj​\(s\)\]\]\.\\bm\{\\Delta\}\_\{j\}\(s\)=\\left\[\\Delta\_\{s\}^\{\[1\]\},\\ldots,\\Delta\_\{s\}^\{\[M\_\{j\}\(s\)\]\}\\right\]\.The array contains observations collected since that component’s most recent description change\. Each measured occurrence contributes one observation\. A component used in several steps can therefore receive several numerical observations from one program, while its qualitative Interpreter feedback is consolidated into one component\-level record\.

For a nonempty array,

Qj​\(s\)=1Mj​\(s\)​∑q=1Mj​\(s\)Δs\[q\]\.Q\_\{j\}\(s\)=\\frac\{1\}\{M\_\{j\}\(s\)\}\\sum\_\{q=1\}^\{M\_\{j\}\(s\)\}\\Delta\_\{s\}^\{\[q\]\}\.\(C7\)A description revision clears the corresponding array\. Observations associated with the previous description do not enter the revised component’s numerical average\. Historical Interpreter reports can remain available as contextual evidence\. New components begin with empty arrays, and unchanged components retain their existing observations\.

##### Review eligibility and probability\.

An existing component is eligible for review when it remains in the active library, appears in the selected island’s current plan, and has at least one contribution observation under its current description\. Thus,

ℛj⊆\{s∈𝒮j−1:sis used byPj\(kj\),Mj\(s\)≥1\}\.\\mathcal\{R\}\_\{j\}\\subseteq\\left\\\{s\\in\\mathcal\{S\}\_\{j\-1\}:\\;s\\text\{ is used by \}P\_\{j\}^\{\(k\_\{j\}\)\},\\,M\_\{j\}\(s\)\\geq 1\\right\\\}\.For an eligible component, the review probability is

pj​\(s\)=Imin\+Imax−Imin1\+CI​Qj​\(s\),p\_\{j\}\(s\)=I\_\{\\min\}\+\\frac\{I\_\{\\max\}\-I\_\{\\min\}\}\{1\+C\_\{I\}\\sqrt\{Q\_\{j\}\(s\)\}\},\(C8\)where0≤Imin≤Imax≤10\\leq I\_\{\\min\}\\leq I\_\{\\max\}\\leq 1andCI\>0C\_\{I\}\>0\. A lower contribution score increases the probability of review without making revision mandatory\.

There is no requirement to accumulateWWobservations before reviewing a component\. TheWWconsecutive code\-improvement visits after an accepted plan proposal provide opportunities to gather multiple observations under unchanged descriptions, but they do not impose a minimum history length\. After a description revision, one new measured contribution is sufficient for eligibility\.

Unrepaired visits do not produce successful step measurements and therefore do not guarantee additional contribution observations\. Suggestions for adding new components are assessed separately from the numerical review of existing components\.

## Appendix DPrompt Templates

This appendix presents the prompt templates for initial solver generation and plan\-aware code evolution, together with their shared task context and solution\-reporting requirements\. Fields in braces are filled with task\-specific information, optimization plans, and evaluation feedback\.

### D\.1Task context and substitution fields

The sharedproblem\_contextcontains the natural\-language problem description, a runtime command and parameter values, CSV table names and locations, column descriptions, the mathematical model, and the reference Gurobi solver code\. The reference implementation is authoritative when it disagrees with a textual description\. The first selected training instance supplies the representative prompt context\. The resulting program is evaluated on the selected training instances\. The table descriptions identify the data that the program must read at execution time; CSV rows are not embedded in the prompt\. Runtime parameter values must be parsed by the generated program and can vary across executions\.

Fields in braces are replaced when a message is constructed\. Their meanings are:

- •problem\_context: the shared task context described above\.
- •time\_limit: the configured per\-instance runtime allowance in seconds\.
- •formatted\_planandstrategy\_context: the validated runtime\-aware plan and the descriptions of the strategy families selected by that plan\. In the evolution prompt,planprovides the island’s fixed strategy and subgoal for each step, andselected\_strategiessupplies the corresponding strategy descriptions\.
- •solution\_output\_requirements: the task\-specific contract for a complete solution report that can be checked by the verifier\.intermediate\_output\_requirementsinserts the shared reporting contract reproduced in Section[D\.2](https://arxiv.org/html/2609.36303#A4.SS2)\.
- •metrics,fitness\_description,improvement\_areas, andartifacts: the parent program’s recorded performance, the scoring rule, comparisons with previous attempts, and execution feedback\.
- •previous\_attempts,other\_context\_programs, andsearch\_guidance: the selected ancestor history, top\-performing and inspiration programs, and available guidance from prior evaluations\.
- •current\_programandlanguage: the parent source code and its language\.timeout\_warningreiterates the configured runtime allowance and the requirement to preserve the program’s output format\.

Instance\-specific models, program histories, plans, and output requirements are represented by placeholders so that the templates remain reusable across tasks\.

### D\.2Shared intermediate\-report contract

Both user prompts require a complete solution report after each plan step and at termination\. The following instructions specify the shared reporting format and cumulative elapsed\-time measurements\.

Shared intermediate\-report contractThe program must print one complete solution report after every plan step, not only at the end\. The body of every \`<stepK\>\` block and the \`<final\>\` block must independently follow the complete solution report contract above\.After a step that does not improve the incumbent, repeat the complete report for the current best incumbent\. If no incumbent exists, print the complete no\-feasible\-solution report permitted by the contract\.Use this stdout structure:<step1\>\# One complete solution report after the first plan step\.</step1\>Cumulative output time: <elapsed seconds\> seconds<step2\>\# One complete solution report after the second plan step\.</step2\>Cumulative output time: <elapsed seconds\> seconds\.\.\.<stepN\>\# One complete solution report after the N\-th plan step\.</stepN\>Cumulative output time: <elapsed seconds\> seconds<final\>\# One complete final solution report\.</final\>Rules:1\. Print one \`<stepK\>\.\.\.</stepK\>\` block for each plan step, in plan order\.2\. Number step tags consecutively by plan step output: \`<step1\>\`, \`<step2\>\`, \.\.\., \`<stepN\>\`\.3\. Print each step report only after that step’s required operation has run\. Never print a partial solution, debug text, traceback, summary, or "same as above" inside a report block\.4\. Print \`Cumulative output time: <elapsed seconds\> seconds\` immediately after each \`</stepK\>\` tag, using wall\-clock time since program start\. This timer is for reporting only and must not affect control flow\.5\. The \`<final\>\` block is mandatory\. Even when it duplicates \`<stepN\>\`, repeat the complete solution report\.

### D\.3Initial program generation

Unlike subsequent code evolution, initial program generation follows the supplied annotations and runtime fields and is instructed to implement every plan step\. The system message defines the agent’s role, and the user message supplies the task, plan, and implementation requirements\.

Initial\-generation system messageYou are an expert optimization coding agent\. Your task is to translate a provided optimization plan into a complete, executable Python program\.

Initial\-generation user\-message template\{problem\_context\}\# TaskSolve the optimization problem above by generating one complete, standalone Python program that faithfully implements every step of the supplied optimization plan and returns the best feasible solution found\.The supplied plan is the algorithmic specification for solving this problem\. Every plan step is mandatory, and each \`Annotation\` describes an operation that must appear in the executable implementation\. Do not replace a required strategy or operation merely because another method is easier to implement or faster to run\.\# Solver PolicyGurobi \(\`gurobipy\`\) is the only permitted mathematical optimization backend\. If a plan step uses a solver, optimizer, mathematical\-programming backend, or optimization subproblem, implement it with \`gurobipy\`\. Do not import or call another optimizer\.Standard Python and non\-optimizer libraries may be used for deterministic computation, data handling, numerical operations, and validation\. Using Gurobi as the backend does not permit replacing the algorithm, formulation, transformation, neighborhood, repair mechanism, or other operation required by the plan\.\{strategy\_context\}\# Step\-by\-Step Plan to Implement\{formatted\_plan\}\# Runtime\-Aware Plan\-Following ContractThe Planner has already classified and scheduled every step for a per\-instance runtime budget of \{time\_limit\} seconds, and that schedule has been validated\. It is not yours to revisit: do not classify a step again, choose or revise an estimate, reallocate runtime between steps, or write a new Reason\. Your task is to implement what each \`Annotation\` describes, under the runtime semantics the plan already declares\.The step\-by\-step plan above is the implementation\-facing rendering of a validated strict JSON plan\. Every field it shows \-\- \`Strategy\`, \`Subgoal\`, \`Annotation\`, \`Step type\`, \`Estimated running time\`, and \`Reason\` \-\- is authoritative as written\.Implement the plan so that its declared runtime semantics hold:1\. For a Complete\-algorithm step, execute the finite operation completely to its scale\-derived stopping rule, without an elapsed\-time cutoff\.2\. For a Budget\-controlled step, run one shared local budget \`B\` for the whole step, equal to that step’s \`Estimated running time\`\. \`B\` is a floor as well as a ceiling: the step must not overrun it and must not finish before it has spent \`B\` on useful work of its own Strategy\. Repeated bounded calls and nested loops within the step share that same \`B\`; they do not receive separate budgets\. When one pass converges before \`B\` is spent, use the remaining budget for further work of the same Strategy \-\- for example, another restart from a different starting point, a perturbation of the incumbent, a wider neighborhood, or a finer parameter \-\- and keep the best feasible result found\. A step whose \`Estimated running time\` is \`remaining time\` takes its \`B\` from the remaining\-allowance rule below instead\.3\. For a Hybrid step, execute the complete \`C\` portion unconditionally and then run one shared local budget \`B\` for the bounded portion only\. The bounded portion follows the same floor\-and\-ceiling and useful\-work requirements as a Budget\-controlled step: it must spend all of \`B\` on work of its own Strategy without overrunning it\. \`Reason\` states how the step’s \`Estimated running time\` splits into \`C \+ B\`\.Time counts toward \`B\` only when the step is performing useful work required by its own \`Annotation\` and Strategy\. Sleeping, busy\-waiting, repeatedly printing, repeatedly validating an unchanged incumbent solely to consume time, executing an empty or unconditional\-break loop, or repeating work that cannot produce a new candidate, decision, or required artifact does not count as spending \`B\`\.The plan’s last step may state \`remaining time\` in place of a number\. It is the one step whose budget the plan does not fix: record one shared program start with a monotonic clock at the beginning of the timed execution, and set that step’s \`B\` to \{time\_limit\} seconds minus the elapsed time since that start, clamped at zero\. Every part of the step, including its own setup, model construction, and result extraction, is charged to that same \`B\`, and enough of \`B\` is reserved for the step’s required report\. A solver time limit set inside this step bounds the solve alone, so derive it from the budget still remaining at the moment of the call \-\- after model construction \-\- minus the reserve that extraction and the report will need\. Its bounded improvement work is permitted bounded work rather than an unconditional core: when no usable time remains, skip the improvement and still produce the step’s required report\. This shared start time measures that step’s budget only; it is not a deadline over any other step\.A local time check may control only budget\-controlled work\. It must never skip a complete portion, an entire mandatory step, or any later plan step\. The generated source must not contain or enforce the total per\-instance budget or any program\-wide deadline other than the remaining\-time step’s own \`B\` described above, and must not sum, rescale, or otherwise recompute the step estimates\.\# Annotation Fidelity RequirementsBefore coding, privately split every \`Annotation\` into its concrete required claims and map each claim to executable statements\. Only clauses explicitly marked optional may be omitted\.\- Every required artifact must materially affect a later model, candidate choice, solver setting, acceptance decision, repair, validation, or required operation\. Computing it and then ignoring it is not implementation\.\- Implement the defining mechanics of every named algorithm, formulation, transformation, derivative rule, stopping rule, and conditional rule\. Do not substitute a proxy or generic alternative unless the \`Annotation\` permits it\.\- Prepare valid fallback input before each mandatory operation so that a missing incumbent, empty preferred input, or failed bounded call cannot turn the step into a no\-op\.\- Handle expected failures explicitly\. Do not hide mandatory work behind broad exception handling\.\- Each step’s required operation runs after the preceding step’s report and before its own \`<stepK\>\` report\. Two \`<stepK\>\` reports with no computation between them mean the later step was never implemented\.\- When one step produces guidance for another operation, pass the exact artifact forward and make it affect the consumer’s behavior\.\- Use only solver\-variable and constraint keys that were actually created, with an explicit missing\-value policy\.\# Program Implementation RequirementsThe code must:1\. Read all required CSV files\.2\. Implement every plan step in order\. A step is implemented only when every non\-optional claim in its \`Annotation\` is realized by executable operations during that step’s own execution interval\. The operations must run even if they find no improvement\. Failure to improve the incumbent is permitted; failure to perform the required search, construction, transformation, repair, validation, or solver operation is not\.3\. Maintain the best feasible incumbent found so far and pass it to later steps when applicable\.4\. Execute every step unconditionally\. Every Budget\-controlled step must spend its full \`B\`, and every Hybrid step must execute its complete \`C\` portion and then spend its full bounded \`B\`\. A step\-local bound governs how long that step’s work runs; it does not license the step to finish early\. When a pass converges with budget left, spend the remainder on further work of the same Strategy \-\- another restart, a perturbation of the incumbent, a wider neighborhood \-\- and keep the best result\. A step\-local bound may limit improvement intensity but may not skip the step’s mandatory core or a later step\.5\. Use strategy\-faithful fallback construction, repair, and validation when preferred input or a solver result is unavailable\.6\. Express every mathematical optimization relationship and subproblem through \`gurobipy\`\.7\. Make every preprocessing or guidance artifact affect the operation specified by its \`Annotation\`\.8\. Follow the complete\-solution\-report contract exactly after every step and for the final report\.\#\# Code comment requirementsIn the generated Python code, clearly mark the plan structure using Python comments only\. These comments are metadata for later agents; they must not be printed to stdout\.For every plan step, include exactly one plan comment block, in plan order\. Every field in the block is copied from the plan: the step number, \`Strategy\`, \`Subgoal\`, \`Annotation\`, \`Step type\`, \`Estimated running time\`, and \`Reason\`\. Reproduce each value exactly as the plan states it\. Do not rename, summarize, rephrase, reorder, or omit a field; do not add punctuation that the plan does not already have; and do not write a value of your own for any of them\. The executable implementation itself must faithfully realize the copied \`Annotation\`; do not add a separate \`Approach\` field\.Every plan step is mandatory\. Implement every step with substantive executable logic\. Execute every step unconditionally and in plan order\. Never skip, omit, disable, defer, or replace a step, including when preferred input is empty, a solver call fails, or a step’s budget is small\. A step\-local bound may control only that step’s permitted bounded work; it must not prevent the step’s mandatory core or any later step from executing and producing its report\.For every implemented step, include comments in this form before the relevant code, copying every value from the plan:\# \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\# Step <number\>:\# Strategy: <Strategy\>\# Subgoal: <Subgoal\>\# Annotation: <Annotation\>\# Step type: <Step type\>\# Estimated running time: <Estimated running time\>\# Reason: <Reason\>\# \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-Never combine, merge, or jointly execute two plan steps\. Each step must have its own execution interval, beginning only after the preceding step’s report and ending before its own report\. Work performed before that interval cannot be credited to the step\.Immediately before returning, privately audit every copied \`Annotation\` against the executable code and correct any clause represented only by a comment, unused computation, proxy strategy, conditional no\-op path, or optional call\.\# Code Quality RequirementsThe script must run without manual edits and must not hard\-code the final answer\. Use reasonable numerical tolerances for feasibility checks and for extracting integer or binary values\.Use only standard Python and commonly available non\-optimizer libraries\. \`gurobipy\` must be the sole mathematical optimization backend\.Before returning, privately verify that every plan step has exactly one comment block whose seven fields appear in order and match the plan exactly, every helper’s return shape matches all call sites, and no mandatory operation can be lost through a missing key, empty input, failed solver call, or broad exception handler\. Do not add runtime code that reads its own source or validates its own comment metadata\.\# Program Output RequirementsThe following contract defines one complete solution report:<solution\_output\_requirements\>\{solution\_output\_requirements\}</solution\_output\_requirements\>\{intermediate\_output\_requirements\}\# Response FormatReturn exactly one \`<code\>\.\.\.</code\>\` block\. Do not include Markdown fences or any text outside this block\.<code\>\# Complete executable Python code only\. Do not use Markdown fences here\.</code\>

### D\.4Plan\-aware code evolution

The code\-evolution prompt requests targeted SEARCH/REPLACE edits under a fixed recorded plan\. It preserves step identifiers, component assignments, and subgoals while allowing implementation changes, runtime reallocation, and step deactivation through zero\-time allocation\. The experimental template is reproduced below with its original field names\. The edits must preserve the plan’s strategies and subgoals, respect the runtime allowance, and retain the required solution\-reporting format\.

Code\-evolution system messageYou are an expert optimization engineer tasked with iteratively improving a codebase\. Your job is to analyze the current program and suggest improvements based on feedback from previous attempts\. Focus on making targeted changes that will increase the program’s performance metrics\.

Plan\-aware code\-evolution user\-message template\{problem\_context\}\# Program Output RequirementsThe following contract defines one complete solution report:<solution\_output\_requirements\>\{solution\_output\_requirements\}</solution\_output\_requirements\>\{intermediate\_output\_requirements\}\# Current Program Information\- Current performance metrics: \{metrics\}\- Areas identified for improvement:\- \{improvement\_areas\}\- \{fitness\_description\}\- Plan implemented by the current program:\{plan\}\#\# Last Execution Output\{artifacts\}\# Program Evolution History\#\# Previous Attempts\{previous\_attempts\}\{other\_context\_programs\}\# Current Program\`\`\`\{language\}\{current\_program\}\`\`\`\{search\_guidance\}\#\# Available Problem\-specific Strategies\{selected\_strategies\}\# TaskImprove the Current Program so that it scores better on the metric named under "\# Current Program Information", by finding a better implementation of the plan it already carries\. Every constraint below is binding\.Answer with SEARCH/REPLACE diffs and nothing else:<<<<<<< SEARCH\# Original code to find and replace \(must match exactly\)=======\# New replacement code\>\>\>\>\>\>\> REPLACEExample of valid diff format:<<<<<<< SEARCHfor i in range\(m\):for j in range\(p\):for k in range\(n\):C\[i, j\] \+= A\[i, k\] \* B\[k, j\]=======\# Reorder loops for better memory access patternfor i in range\(m\):for k in range\(n\):for j in range\(p\):C\[i, j\] \+= A\[i, k\] \* B\[k, j\]\>\>\>\>\>\>\> REPLACERules for the answer itself:\- Send as many blocks as you need\. Every SEARCH section must match code in "\# Current Program" character\-for\-character, including whitespace and indentation\. Do not paraphrase or reformat it\.\- Explain your reasoning for each change\.\- Give every function a concise docstring naming the approach it implements\.\- Do not write a Changes Description\. This workflow derives one by comparing the step comments before and after\.\# Constraints\#\# The plan is fixed; the implementation is notEach plan step appears in the code as one comment block, with exactly these fields in this order:\# \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\# Step <number\>:\# Strategy: <Strategy\>\# Subgoal: <Subgoal\>\# Annotation: <Annotation\>\# Step type: <Complete\-algorithm step \| Budget\-controlled step \| Hybrid step\>\# Estimated running time: <number seconds \| remaining time\>\# Reason: <basis for the step type and the number\>\# \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\| Field \| May you change it? \|\| \-\-\- \| \-\-\- \|\| Step, Strategy, Subgoal \| \*\*No\.\*\* Reproduce them byte\-for\-byte\. Do not rename, rephrase, summarize, shorten or expand them\. \|\| Annotation \| \*\*Yes, and you must\*\* whenever the implementation changes\. \|\| Step type, Estimated running time, Reason \| \*\*Yes\*\*, following "Runtime budget" below\. \|You CANNOT add, delete, renumber, reorder or merge steps\.The Annotation states the main operation, the variables, constraints or structures it touches, and why that operation serves the fixed Strategy and Subgoal\. It must describe the code you are returning, never the code you replaced\.Before returning, confirm that every step of the Current Program is still present in the same order, that Step, Strategy and Subgoal are untouched, and that every Annotation matches the code you actually wrote\.\#\# What you are free to changeAnything inside a step, as long as it stays inside the same strategy family: a materially different mechanism from that family, compatible tactics combined, better data structures or solver formulations, and tuning such as neighborhood size, candidate ordering, iteration counts, restart intensity, stopping rules and solver settings\.The strategy descriptions above are a design space, not text to copy\. Use them to find another way to implement a fixed Strategy for its fixed Subgoal\. Never relabel a step’s Strategy or change its Subgoal because some other family appears elsewhere in this prompt, and never copy a description verbatim into plan metadata\.To drop a step you judge unnecessary, harmful or too expensive, keep its Step, Strategy, Subgoal and Step type unchanged, and set \`Estimated running time: 0 seconds\`\. This is the one case where the estimate is not the \`C\`, \`B\` or \`C \+ B\` that its step type would otherwise imply\. Its \`Reason\` still opens with the exact step type, and then says the step was dropped and why\.\#\# Runtime budget\`Estimated running time\` is the time a step is expected to occupy in the next run\. What that number is depends on the step type, and only one of the three is yours to choose:1\. \*\*Complete\-algorithm step\.\*\* The work has a finite, scale\-derived stopping rule and runs to completion without an elapsed\-time cutoff\. The number is its natural runtime \`C\`, which you observe or predict but do not pick\.2\. \*\*Budget\-controlled step\.\*\* The work is bounded by a solver limit, loop deadline, iteration budget, or equivalent local bound\. The number is the one shared local budget \`B\` you enforce in the code for the whole step, including its repeated calls and nested loops\. This is the number you allocate\.3\. \*\*Hybrid step\.\*\* An unconditional complete portion plus bounded work\. The number is \`C \+ B\`, with one shared \`B\` covering the bounded portion only\.A last step should be a \`Budget\-controlled step\` and should read \`remaining time\` in place of a number for "Estimated running time"\. That is not an estimate you derive: it means the step takes the remaining runtime allowance, the total minus actual elapsed time since a shared program start, clamped at zero\. Keep the value, and keep the code that computes it, while that step still ends the program in useful bounded improvement\. Replace it with a number only by also replacing that computation with one shared local budget \`B\` you enforce yourself, and say so in its \`Reason\`\.\#\#\# First, measure"\#\# Last Execution Output" reports each step’s \`Cumulative Time\`: wall\-clock seconds since program start, averaged over the evaluation instances\. It is cumulative, so it is not the cost of that one step \-\- a step’s own measured runtime is its cumulative time minus the previous step’s, taking zero before the first step\. Derive every step’s measured runtime that way before changing any number\. Where a run was cut short and a step never reported a cumulative time, it has no measurement: say so in its \`Reason\` and estimate it the way a reimplemented step is estimated below\.The numbers currently in the code are predictions made before anything ran, and are often far from what the program actually did\. Prefer a measurement to an inherited number wherever a measurement exists\.\#\#\# Then decide each step\- \*\*A step you leave as it is\.\*\* Its measured runtime is its cost, so use that number\.\- \*\*A step whose measurement contradicts its type\.\*\* A budget\-controlled or hybrid step that finished well inside its budget was stopped by its own rule rather than by the bound, which is what a complete portion does\. Correct the \`Step type\` \-\- to a complete\-algorithm step, or to a hybrid one with a smaller bounded portion \-\- and let the number follow\. Do not keep a type whose \`B\` the code never actually reached\. A step that consumed essentially all of its budget was genuinely bounded by it, and that \`B\` is yours to keep or raise\.\- \*\*A step you reimplement\*\* \(its Annotation changes\)\. The measurement describes code you are replacing, so derive the number afresh\. Name the dominant work first \-\- how many moves, passes, solver calls or candidate evaluations the new code performs, and how the cost of one of them scales \-\- then turn that into seconds by calibrating against a step of this same program whose measured runtime you already know\. Same machine, same instances, so a measured anchor is far more reliable than an assumed operations\-per\-second rate\.Every step’s \`Reason\` opens with its exact step type followed by a period, and then says where its number came from: that it is measured, which step it was calibrated against, why the step was dropped, or that the step takes the remaining allowance instead of a number\.\#\#\# Then allocateOnly \`B\` is yours to allocate\. \`C\` is whatever the complete work costs, so a plan of complete\-algorithm steps has nothing to reallocate\.\- \`sum\(Estimated running time\) <= \{time\_limit\} seconds\`, counting only the steps that state a number\.\- Spend the allowance rather than dividing it evenly\. Give more to a budget\-controlled step that is still improving the incumbent, less to one that is stagnant or disproportionately expensive, and zero to one you are dropping\. Reassign what a shrunk estimate frees to a step that can use it; when no step can, leave the remainder unspent rather than padding a step with useless work merely to consume time\.\- Never enforce the total in code\. The returned source must contain no global time\-limit constant, program\-wide deadline, or remaining\-time logic, except the computation that gives a \`remaining time\` step its own budget\. A step may bound its own work with a local iteration count, solver limit or timer; that bound may limit only that step’s search intensity, and must never skip the step’s mandatory core or any later step\.\- \`gurobipy\` is the only permitted optimization backend\.\{timeout\_warning\}