Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks

# Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks
Source: [https://arxiv.org/html/2606.29082](https://arxiv.org/html/2606.29082)
11affiliationtext:University of Minnesota22affiliationtext:Carnegie Mellon University33affiliationtext:KAIST44affiliationtext:University of Cambridge55affiliationtext:Hanyang University66affiliationtext:AmazonSeungone KimMinki KangAlistair CheongZerui ChenSeungho HanTaehee JungThis work is independent of the author’s position at Amazon and does not relate to any work conducted at Amazon\.Dongyeop Kang

###### Abstract

Would experience designing faster GPU kernels also help close in on a long\-standing open mathematical conjecture? Large Language Models \(LLMs\) integrated into evolutionary search have recently produced state\-of\-the\-art solutions on optimization tasks, including open mathematical conjectures, GPU kernel design, scientific law discovery, and combinatorial puzzles\. To achieve this, prior work applied search scaffolds to one target task at a time, so every new problem is approached from scratch and the experience accumulated during search is discarded once the model finishes its attempt\. This leaves the capability of iteratively evolving a solution \(e\.g\., knowing which part to mutate and how, deciding when to backtrack\) entirely in the scaffold rather than in the model itself\. Whether the model itself could acquire this capability and reuse it across different tasks has been largely unexamined\. To address this, we introduceEvolution Fine\-Tuning\(EFT\), a mid\-training paradigm that teaches LLMs to evolve solutions across tasks by converting evolutionary search trajectories into supervision\. We constructℱ\\mathcal\{F\}inch Collection, a 156K\-trajectory dataset spanning 10 domains and 371 optimization tasks, and fine\-tune open\-source LLMs from 2B to 9B parameters\. Empirically, EFT confers cross\-task generalization: across 22 held\-out tasks, our models surpass their base counterparts by 10\.22% on average\. Furthermore, when paired with test\-time RL, our model matches state\-of\-the\-art performance on two circle\-packing tasks and outperforms its base\-model counterpart on the Erdős minimum\-overlap problem\. EFT thus serves as a “practice phase” for general\-purpose discovery agents that do not solve new problems from scratch\.

![Refer to caption](https://arxiv.org/html/2606.29082v1/x1.png)Figure 1:\(Left\)Evolution Fine\-Tuning\(EFT\)serves as “mid\-training”, boostingℱ\\mathcal\{F\}inch’s discovery capability on the Erdős minimum overlap problem under both test\-time search and learning\.\(Right\)On NP\-hard competitive programming \(CALICO, UC Berkeley contest\), EFT enables cross\-discovery transfer:ℱ\\mathcal\{F\}inch solves problems by combining strategies acquired from diverse domains, such as combinatorial optimization, recommender systems, robust statistics/computer vision, and numerical optimization\. In contrast, the base model without EFT relies on a single, repetitive strategy\.## 1Introduction

Some of the most consequential problems in mathematics, algorithm engineering, and the natural sciences are optimization tasks: problems for which a candidate solution can be scored against an objective, but for which the optimal solution is not directly computable\[kirkpatrick1983optimization\]\. Concrete examples include open mathematical conjectures such as the Erdős minimum\-overlap problem\[erdHos1955some,white2023new\], the design of high\-performance GPU kernels\[ouyang2025kernelbench\], and the discovery of new scientific laws from data\[shojaee2025llm\]\.

Recently, large language models \(LLMs\) combined with evolutionary search methods have begun to produce state\-of\-the\-art solutions across such tasks: at each iteration, the LLM proposes new candidate solutions, a scaffold scores them and updates a population of high\-scoring candidates, and the loop continues until a strong solution emerges\[romera2024mathematical,novikov2025alphaevolve,lange2025shinkaevolve\]\. Two methodological branches dominate this line of work: \(1\)*Test\-time search*methods use a fixed, typically proprietary LLM as the mutation operator and rely on the scaffold’s parent selection and prompting logic to drive improvement\[assumpccao2025codeevolve,yan2026pacevolve,cemri2026adaevolve,liu2026evox\]\. \(2\)*Test\-time learning*methods additionally update the LLM’s weights during the search process, allowing the model to specialize to the target task as it explores\[wang2025thetaevolve,yuksekgonul2026learning\]\.

Despite empirical successes, both branches share a common limitation: the discovery capability \(i\.e\., the skill of iteratively improving a solution, knowing what to mutate, what to keep, and when to backtrack\) is constructed during each search rather than internalized into the model itself\. Specifically,

1. 1\.Test\-time search methods often rely on proprietary, frontier\-scale LLMs as their mutation operator, because the scaffold demands consistent, high\-quality proposals at every iteration\. In our experiments, we observe that open\-source models smaller than 9B parameters fail to follow evolutionary trajectories within such scaffolds and yield substantially weaker performance \(Figure[1](https://arxiv.org/html/2606.29082#S0.F1), left\)\.
2. 2\.Test\-time learning methods alleviate this by allowing smaller LLMs to adapt their weights based on their own search experience, and have produced new best\-known solutions on several mathematical problems\[wang2025thetaevolve,yuksekgonul2026learning\]\. However, these updates are tailored to a single search loop and a single task; the strategies the model discovers are not consolidated into reusable capability, so the model cannot compose strategies from prior tasks when tackling a new one \(Figure[1](https://arxiv.org/html/2606.29082#S0.F1), right\)\.
3. 3\.More fundamentally, in neither branch does the model itself acquire the evolving capability\. Test\-time search\-based methods do not update the model at all, leaving the capability in the search procedure by design\. Test\-time learning\-based methods update the model through test\-time RL, but the updates serve to find a solution within a single search loop rather than to internalize the discovery capability itself, and they are discarded once the task is solved\.

One promising direction to address these limitations is to have the LLM itselfmeta\-learnthe discovery capability \(i\.e\., learning how to evolve solutions across optimization tasks\)\. The core challenge in doing so is that optimization tasks are NP\-hard and lack ground truth optimal solutions, making the standard supervised learning recipe of collecting \(problem, answer\) pairs unavailable\. To circumvent this, we proposeEvolution Fine\-Tuning \(EFT\), a mid\-training paradigm that treats thetrajectoriesof search runs as the supervision signal, thereby internalizing the discovery capability into the model itself\. We constructℱ\\mathcal\{F\}inch Collection, a large\-scale dataset of 156K such trajectories collected using a widely used search scaffold, i\.e\., OpenEvolve\[openevolve\], and a recent large model, Qwen3\.5\-397B\-A17B\[qwen35blog\], across 10 domains and 371 tasks\. Usingℱ\\mathcal\{F\}inch Collection, we fine\-tune open\-source models from 2B to 9B, and obtain a new model family,![[Uncaptioned image]](https://arxiv.org/html/2606.29082v1/logos/finch_icon.png)ℱ\\mathcal\{F\}inch\-\{2, 4, 8, 9\}B\.

![Refer to caption](https://arxiv.org/html/2606.29082v1/x2.png)Figure 2:A Concept of Evolution Fine\-Tuning \(EFT\)To demonstrate the cross\-task generalization conferred byℱ\\mathcal\{F\}inch Collection, we first employℱ\\mathcal\{F\}inch as mutation operators in test\-time search scaffolds\.ℱ\\mathcal\{F\}inch outperforms its base counterparts on 22 held\-out tasks and achieves performance comparable to best\-known solutions previously obtained by a proprietary model, despite using a much smaller open\-source backbone\. Notably, as shown in Figure[1](https://arxiv.org/html/2606.29082#S0.F1)\(right\), when solving a competitive programming task, we observe that while base LLM tends to apply only in\-domain strategy \(i\.e\., Gauss\-Seidel uniform weight\),ℱ\\mathcal\{F\}inch transfers strategies across domains \(e\.g\., applying log\-domain alternating least squares from recommender system, Levenberg\-Marquardt from numerical optimization to solve a competitive programming problem\), suggesting thatEFTgives rise to emergent behaviors in discovery tasks\. Furthermore, scaling the number of training tasks inℱ\\mathcal\{F\}inch Collection from 15 to 355 improvesℱ\\mathcal\{F\}inch’s held\-out performance by14\.1%on average across held\-out tasks\. Finally, to assess whetherℱ\\mathcal\{F\}inch can also learn from its own search experience, we apply test\-time learning to bothℱ\\mathcal\{F\}inch \(i\.e\., w/EFT\) and base LLMs \(i\.e\., w/oEFT\) on three tasks\. We find thatℱ\\mathcal\{F\}inch achieves state\-of\-the\-art performance on two circle packing tasks and outperforms its base\-model counterpart on the Erdős Minimum Overlap Problem\.

## 2Preliminaries

#### Optimization setup\.

We consider an optimization taskτ∈𝒯\\tau\\in\\mathcal\{T\}with an initial candidate solutionx0x\_\{0\}and an iteration budgetTT\. The candidate set generated during search is denoted by𝒳=\{x0,…,xT\}\\mathcal\{X\}=\\\{x\_\{0\},\\ldots,x\_\{T\}\\\}, wherex0x\_\{0\}may be a program\[novikov2025alphaevolve\], math construction\[imajuku2025ale\], or prompt\[openevolve\], depending on the task\. At iterationtt, an evolutionary scaffold𝒮\\mathcal\{S\}uses a mutation operatorℳθ\\mathcal\{M\}\_\{\\theta\}, typically an LLM, to produce a new candidate from the parent solution and search history:xt=𝒮​\(xt−1,I,ℋt−1;ℳθ\)x\_\{t\}=\\mathcal\{S\}\(x\_\{t\-1\},I,\\mathcal\{H\}\_\{t\-1\};\\mathcal\{M\}\_\{\\theta\}\), whereℋt−1\\mathcal\{H\}\_\{t\-1\}contains selected prior candidates and feedback\. An evaluatorℰ\\mathcal\{E\}assigns a score and auxiliary artifacts such as logs or natural\-language feedback\. The goal is

x⋆=arg​optx∈𝒳⁡ℰ​\(x\),opt∈\{max,min\},x^\{\\star\}=\\operatorname\*\{arg\\,opt\}\_\{x\\in\\mathcal\{X\}\}\\mathcal\{E\}\(x\),\\qquad\\mathrm\{opt\}\\in\\\{\\max,\\min\\\},\(1\)where the optimization direction is determined by task:max\\maxfor accuracy andmin\\minfor c5 bound in Erdos problem\.

Table 1:Comparison of evolutionary methods\. Scaffold indicates whether the method provides search or learning support for evolution\. Train and Test denote scaffolding at training or test time, respectively\. OS denotes open sourcing\.MethodMainContributionScaffoldTrainingOSSearchLearnTrainTestParadigm\#TasksAlphaEvolve\[novikov2025alphaevolve\]Search✓✗✗✗––✗OpenEvolve\[openevolve\]Search✓✗✗✗––✓ShinkaEvolve\[lange2025shinkaevolve\]Search✓✗✗✗––✓GEPA\[agrawal2025gepa\]Search✓✗✗✗––✓PAC\-Evolve\[yan2026pacevolve\]Search✓✗✗✗––✗AdaEvolve\[cemri2026adaevolve\]Search✓✗✗✗––✓EvoX\[liu2026evox\]Search✓✗✗✗––✓DGM\[zhang2025darwin\]Search✓✗✗✗––✓HyperAgent\[zhang2026hyperagents\]Search✓✗✗✗––✓CORAL\[qu2026coral\]Search✓✗✗✗––✓ThetaEvolve\[wang2025thetaevolve\]Learning✗✓✗✓RL1✓TTT\-Discover\[yuksekgonul2026learning\]Learning✗✓✗✓RL1✓EFT \(Ours\)Model, data✓✓✓✓SFT, RL371✓

![Refer to caption](https://arxiv.org/html/2606.29082v1/x3.png)Figure 3:An overview of the optimization task groups \(a total of 371 tasks\) inℱ\\mathcal\{F\}inch Collection, where bubble size indicates the number of tasks in each group\.
#### Discovery\.

Following prior work\[yuksekgonul2026learning\], we callx⋆x^\{\\star\}adiscoveryif it improves upon the previous best\-known solutionxsotax\_\{\\mathrm\{sota\}\}within budgetTT, i\.e\.,ℰ​\(x⋆\)\>ℰ​\(xsota\)\\mathcal\{E\}\(x^\{\\star\}\)\>\\mathcal\{E\}\(x\_\{\\mathrm\{sota\}\}\)for maximization tasks, with the inequality reversed for minimization tasks\.

#### Evolutionary Search Scaffold\.

To discover novel solutionsτ\\tau, existing LLM\-based evolutionary methods use either search\-based or learning\-based scaffolds\. Search\-based scaffolds keepθ\\thetafixed and rely on external search mechanisms, while learning\-based scaffolds updateθ\\thetaat test time\. Both approaches are fundamentally built upon four core modular components: \(i\) construct a prompt from the parent solution, task instruction, history, and feedback; \(ii\) generate a candidate usingℳθ\\mathcal\{M\}\_\{\\theta\}through eitherdiff\-based editorfull rewrite; \(iii\) evaluate the candidate withℰ\\mathcal\{E\}; and \(iv\) store eligible candidates in a population database𝒟\\mathcal\{D\}\. The updated database𝒟t\\mathcal\{D\}\_\{t\}then informs the next iteration until the compute budget or target score is reached\. We summarize existing evolutionary methods in Tab[3](https://arxiv.org/html/2606.29082#S2.F3.4)\.

## 3Evolution Fine\-Tuning

Evolutionary scaffolds can discover strong solutions at test time, but the discovery procedure is typically external to the model\. We introduceEvolution Fine\-Tuning \(EFT\), a mid\-training procedure that transfers this test\-time discovery behavior into smaller open\-source LLMs\. EFT converts evolutionary search trajectories into supervised training examples, so that the model learns to act as a stronger mutation operator before deployment\. As summarized in Table[3](https://arxiv.org/html/2606.29082#S2.F3.4), EFT is orthogonal to the choice of test\-time scaffold: the resulting model can be used inside search\-based scaffolds with frozen weights, or further adapted by learning\-based scaffolds such as test\-time RL\.

![Refer to caption](https://arxiv.org/html/2606.29082v1/x4.png)Figure 4:Overview of theℱ\\mathcal\{F\}inch Collection construction pipeline, consisting of \(1\) seed optimization task collection, \(2\) trajectory collection via an evolutionary search scaffold \(i\.e\., OpenEvolve\[openevolve\]\), and \(3\) trajectory filtering for unrecoverable cases, breakage cases, and candidate solutions that incur systematic errors \(e\.g\., timeout errors\)\.### 3\.1ℱ\\mathcal\{F\}inch Collection Construction

#### Overview\.

Figure[4](https://arxiv.org/html/2606.29082#S3.F4)illustrates the construction ofℱ\\mathcal\{F\}inch Collection\. Each task consists of a task specification, an initial candidate solution, an evaluator, and configuration files\. We run an evolutionary scaffold over these task files to produce a sequence of parent\-to\-child solution transitions\. Each transition records the prompt, parent solution, generated candidate, evaluator output, score change, and execution artifacts\. We then remove trajectories whose feedback is unreliable or whose transitions would provide misleading supervision\. The final collection contains approximately 156K evolutionary trajectories across 371 optimization tasks\.

#### Step 1: Seed Optimization Task Collection\.

Training data for optimization is difficult to synthesize because many target problems are NL\-hard, lack known global optima, and require expert\-designed evaluators \(see Table[8](https://arxiv.org/html/2606.29082#A2.T8)in Appendix[B](https://arxiv.org/html/2606.29082#A2)\)\. Instead of generating artificial tasks, we source seed tasks from existing optimization benchmarks whose objectives are executable and externally validated\. We select tasks according to three criteria: \(i\) the task should require nontrivial search, \(ii\) should not reduce to matching a known ground\-truth answer, and \(iii\) should provide a deterministic evaluator that assigns a continuous or comparable score to candidate solutions\.

In total, as shown in Figure[3](https://arxiv.org/html/2606.29082#S2.F3.4), we collect 371 seed tasks from 10 benchmarks, spanning from mathematical discovery, competitive programming, heuristic optimization, numerical algorithm optimization, symbolic regression, GPU kernel optimization, constructive search, and biological denoising benchmarks\. These include AlphaEvolve’s mathematical discovery problems\[novikov2025alphaevolve\], FrontierCS\[mang2025frontiercs\], ALE\-Bench\[imajuku2025ale\], AlgoTune\[press2025algotune\], GPU Mode\[gpumode\], LLM\-SRBench\[shojaee2025llm\], Function Minimization and K\-Module tasks from OpenEvolve\[openevolve\], scRNA\-seq denoising\[luecken2025defining,yuksekgonul2026learning\], and variants of Erdős problems\[feng2026semi,erdosproblems\]\. The full task list is provided in Appendix[E](https://arxiv.org/html/2606.29082#A5)\.

#### Step 2: Evolutionary Trajectory Collection\.

For each seed taskτ\\tau, we run an evolutionary scaffold𝒮\\mathcal\{S\}for a fixed budget of iterations\. At iterationtt, the scaffold selects a parent solutionxt−1x\_\{t\-1\}and constructs a prompt from the task instructionII, the parent, the search historyℋt−1\\mathcal\{H\}\_\{t\-1\}, and evaluator artifacts\. For each task,NτN\_\{\\tau\}trajectories are generated whereNτ<TN\_\{\\tau\}<T, where trajectories flagged as errors are discarded\. A teacher mutation operatorℳθ\\mathcal\{M\}\_\{\\theta\}then generates a candidate solutionxtx\_\{t\}, which is executed by the task evaluatorℰ\\mathcal\{E\}\. The resulting trajectory stores\(I,xt−1,ℋt−1,xt,ℰ​\(xt\),ℱt\)\(I,x\_\{t\-1\},\\mathcal\{H\}\_\{t\-1\},x\_\{t\},\\mathcal\{E\}\(x\_\{t\}\),\\mathcal\{F\}\_\{t\}\), whereℱt\\mathcal\{F\}\_\{t\}includes execution logs, error traces, and evaluator feedback\.

We instantiateℳ\\mathcal\{M\}with OpenEvolve\[openevolve\]and use Qwen3\.5\-397B\-A17B\[qwen35blog\]as the teacher mutation operator\. To expose the student model to both local refinement and global exploration, we collect trajectories under two mutation strategies:diff\-based editandfull rewrite\. Thediff\-based editstrategy asks the model to edit an existing solution and therefore emphasizes exploitation, whilefull rewriteasks the model to rewrite the solution and therefore encourages broader exploration\. For diversity, we run the scaffold multiple times per task with stochastic decoding\. Unless otherwise specified, we use temperature0\.70\.7, top\-p=0\.95p=0\.95, and a maximum generation length of 30K tokens\. In total, we obtain 172,997 trajectories\.

#### Step 3: Trajectory Filtering\.

Raw evolutionary traces contain failures that should not be imitated\. From 172,997 raw trajectories, we retain 156,731 \(90\.6%\) by applying the following criteria:

- •First, we removesystematic errors, where the evaluator output is unreliable or the score delta cannot be computed, removing 6,321 trajectories \(3\.7%\) in total\. Examples include missing parent scores \(57\.5%\), timeout errors \(14\.2%\), syntax\-check failures, import failures, and evaluator crashes\. The complete list of systematic errors is provided in Table[9](https://arxiv.org/html/2606.29082#A3.T9)\.
- •Second, we removeunrecoverable & breakage cases: Unrecoverable cases are both parent and child solutions are erroneous \(294; 0\.2%\), and breakage cases are an error\-free parent yields an erroneous child \(1,281; 0\.8%\)\. Both constitute hard negatives, destabilizing the training signal\.
- •Lastly, we removeexcessively long inputs, to keep training stable and tractable\. We dsicard responses longer than 16,384 tokens and examples whose total serialized input\-output length exceeds 32,768 tokens, removing 8,370 \(5\.0%\)\.

After filtering,ℱ\\mathcal\{F\}inch Collection contains approximately 156K trajectories across 371 tasks\.

![Refer to caption](https://arxiv.org/html/2606.29082v1/x5.png)

Figure 5:Distribution of trajectory improvement inℱ\\mathcal\{F\}inch Collection\.Table 2:Effect of improvement type on EFT\.Each retained trajectory is converted into a supervised fine\-tuning instance\. The input consists of the same information available to the mutation operator inside the scaffold: the task instruction, parent solution, selected history, previous scores, and evaluator artifacts\. The target output is the teacher\-generated candidate solution\. This format directly trains the model to map an evolutionary state to a plausible next mutation:\(I,xt−1,ℋt−1,ℱt−1\)↦xt\(I,x\_\{t\-1\},\\mathcal\{H\}\_\{t\-1\},\\mathcal\{F\}\_\{t\-1\}\)\\mapsto x\_\{t\}\.

When task scores are available, we classify each trajectory based on its improvement outcome \(Δ\\Delta\), i\.e\.,Δ=ℰ​\(xt\)−ℰ​\(xt−1\)\\Delta=\\mathcal\{E\}\(x\_\{t\}\)\-\\mathcal\{E\}\(x\_\{t\-1\}\), into three categories:Δ\>0\\Delta\>0\(Imp\),Δ=0\\Delta=0\(NC\), andΔ<0\\Delta<0\(Reg\)\. As shown in Figure[2](https://arxiv.org/html/2606.29082#S3.T2), this yields 61,802 \(39\.4%\)Imp, 30,130 \(19\.2%\)NC, and 64,799 \(41\.3%\)Regtrajectories\. In this work, we primarily useImptrajectories for evolution fine\-tuning to preventℱ\\mathcal\{F\}inch from imitating non\-improving behaviors due toReg, as shown in Table[2](https://arxiv.org/html/2606.29082#S3.T2)\. Furthermore, to enableℱ\\mathcal\{F\}inch to learn how to evolve parent programs effectively within the solution space, we leverage bothImpandRegtrajectories through a preference learning algorithm \(i\.e\., KTO\[ethayarajh2024kto\]\)111To maximize the contrastive signal that guidesℱ\\mathcal\{F\}inch toward self\-judging which solutions are promising and which fall short, we restrict KTO training toImpandRegin this work\. Nevertheless, we believeNCis also a valuable resource for internalizing discovery capability, as suggested in Table[8](https://arxiv.org/html/2606.29082#S4.F8)\.\. Consequently, ourℱ\\mathcal\{F\}inch Collection provides a comprehensive set of improvement trajectories, serving as a valuable resource for internalizing discovery capabilities into LLMs\.

\(a\) Tasks and trajectories by task group

\(b\) Languages and mutation strategies

![Refer to caption](https://arxiv.org/html/2606.29082v1/x6.png)

\(c\) Trajectory length distribution by improvement type

![Refer to caption](https://arxiv.org/html/2606.29082v1/x7.png)

\(d\) Top 20% packages used in initial program

Figure 6:Dataset composition and trajectory characteristics\. \(a\) Number of tasks and trajectories for each task group\. \(b\) Programming language and mutation strategy proportions across trajectories\. \(c\) Distribution of trajectory lengths by improvement type \(Imp,NC, andReg\)\. \(d\) Top\-20% packages used in initial programs\.

### 3\.2Analysis ofℱ\\mathcal\{F\}inch Collection

Overall,ℱ\\mathcal\{F\}inch Collection contains 156,731 \(~156K\) evolutionary search trajectories, covering 371 tasks across 10 task groups\. As shown in Figure[6](https://arxiv.org/html/2606.29082#S3.F6)\(a\), Competitive Programming occupies the largest share with 172 individual tasks and is derived from FrontierCS\[mang2025frontiercs\], followed by Numerical Algorithm Optimization, which builds upon ALE\-Bench\[imajuku2025ale\]\. In Figure[6](https://arxiv.org/html/2606.29082#S3.F6)\(b\), 68\.5% of trajectories inℱ\\mathcal\{F\}inch Collection are based on Python, with a considerable portion in C\+\+ \(31\.5%\)\. Regarding mutation strategy,ℱ\\mathcal\{F\}inch Collection exhibits a well\-balanced distribution betweendiff\-based edit\(50\.3%\) andfull rewrite\(49\.7%\)\. Together, these statistics suggest that models trained onℱ\\mathcal\{F\}inch Collection acquire optimization capabilities across both Python and C\+\+, as well as proficiency in bothdiff\-based editandfull rewritestrategies\. Moreover, Figure[6](https://arxiv.org/html/2606.29082#S3.F6)\(c\) shows that the input length \(avg\. 8,902 tokens\) is substantially longer than the output length \(avg\. 6,865 tokens, 1\.3×\\times\), indicating thatℱ\\mathcal\{F\}inch learns to optimize parent programs by selectively leveraging useful feedback signals from a large context of prior programs\. In addition, withinNC, the median output length is 2\.9×\\timeslonger, suggesting that Qwen3\.5\-397B\-A17B tends to engage in more extensive reasoning even when no improvement is achieved\. Meanwhile, the output length inRegtrajectories is comparable to that inImpones, implying that regressions arise not from insufficient reasoning, but from misguided exploration within the solution space\. Moreover, in Figure[6](https://arxiv.org/html/2606.29082#S3.F6)\(d\),ℱ\\mathcal\{F\}inch Collection most frequently utilizesnumpy, followed bybits/stdc\+\+, suggesting that the dataset spans both numerical computing and competitive programming workloads, thereby exposing models to diverse optimization patterns across multiple domains\.

### 3\.3![[Uncaptioned image]](https://arxiv.org/html/2606.29082v1/logos/finch_icon.png)ℱ\\mathcal\{F\}inch: Evolution Fine\-Tuned Language Model

We introduce a new family of evolution fine\-tuned LLMs,![[Uncaptioned image]](https://arxiv.org/html/2606.29082v1/logos/finch_icon.png)ℱ\\mathcal\{F\}inch, trained onℱ\\mathcal\{F\}inch Collection across 355 tasks, excluding 16 held\-out tasks\. As base models, we use the Qwen3\.5\[qwen35blog\]series at three model sizes—2B, 4B, and 9B—as well as the Qwen3\-8B\[yang2025qwen3\]forℱ\\mathcal\{F\}inch\-8B\. As shown in Figure[3](https://arxiv.org/html/2606.29082#S2.F3.4), the trajectory distribution is highly imbalanced, with Symbolic Regression \(SR\) trajectories constituting the majority, which could adversely affect training\. To mitigate this, we use only one trajectory per task during training222Recall from Step 2 of dataset construction that three trajectories are collected per task\.\. We fine\-tune the base models using onlyImptrajectories fromℱ\\mathcal\{F\}inch Collection via full SFT, resulting in a total of 30,445 training trajectories\. For validation, we use 900 trajectories uniformly sampled across tasks\. For training, we employ the LLaMA\-Factory\[zheng2024llamafactory\]framework\. All models are trained for one epoch with a global batch size of 128 and a learning rate of1​e\-​51\\text\{e\-\}5\. All experiments are conducted on eight NVIDIA H200 140GB GPUs\.

## 4Experiments

### 4\.1Experimental Setup

Table 3:Performance comparison of![[Uncaptioned image]](https://arxiv.org/html/2606.29082v1/logos/finch_icon.png)ℱ\\mathcal\{F\}inch to base models across Mathematical Discovery, Algorithm Engineering, and System Performance benchmarks\.†Scores reported in\[qu2026coral\]\.‡Scores reported in\[liu2026evox\]\.Δ\\Deltadenotes the relative improvement \(%\) ofℱ\\mathcal\{F\}inch over the corresponding same\-size base model, sign\-adjusted so that positive values always indicate improvement regardless of metric direction\.Avg\. Gainis the arithmetic mean of availableΔ\\Deltavalues, excludingahc058where the base score is near zero and yields a disproportionately large ratio\.#### Evaluation Tasks\.

To evaluate thecross\-task discovery generalizationofℱ\\mathcal\{F\}inch, we use a diverse set of optimization tasks that are, to the extent possible, disjoint from those used to trainℱ\\mathcal\{F\}inch\. These tasks are drawn from benchmarks widely adopted in prior works\[cemri2026adaevolve,liu2026evox,skydiscover2026,yuksekgonul2026learning,ye2026evaluation\]\. In total, we evaluateℱ\\mathcal\{F\}inch across five domains and 22 tasks:\(1\) Mathematical Discovery, including the Erdős Minimum Overlap Problem \(Erdos\), First Autocorrelation Inequality \(AC1\), Second Autocorrelation Inequality \(AC2\), Circle Packing in a Unit Square withn=26n=26\(CP\), and Hadamard Maximum Determinant \(Hadamard\);\(2\) Algorithm Engineering, including two tasks, ahc039 and ahc048;\(3\) System Performance, including four tasks, EPLB, PRISM, LLM\-SQL, and Transaction;\(4\) Competitive Programming, comprising six tasks, each scored on a scale from 0 to 100\. These include Problem 263 \(P263\) from CALICO333[https://frontier\-cs\.org/blog/calico/](https://frontier-cs.org/blog/calico/), UC Berkeley’s official programming contest featuring open\-ended optimization problems, and five newly added FrontierCS v1\.1 problems, Problem 301–305 \(P301–305\); and\(5\) Algorithmic Heuristics, including five tasks, Convolve2D Full Fill \(Convolve2D\), PolynomialReal \(Polynomial\), Positive Semidefinite Cone Projection \(PSD\), 2D Affine Transform \(Affine Transform\), and FFT Convolution \(FFT Conv\.\)\. Although some tasks are drawn from the same benchmark suite, we treat them as independent evaluation tasks because discovery tasks often differ substantially in their objectives, search spaces, and solution forms\. For evaluation, following the definition of discovery, we report the maximum task\-specific score achieved withinTTiterations\. Each optimization task uses its own task\-specific metric, such as thec5\_boundfor the Erdős overlap problem\. Detailed descriptions of these task\-specific metrics are provided in Appendix[E](https://arxiv.org/html/2606.29082#A5)\.

#### Baselines\.

We evaluateℱ\\mathcal\{F\}inch’s capability as a mutation operator when combined with a test\-time search scaffold; throughout this work, we use OpenEvolve\[openevolve\]as the default scaffold unless otherwise noted\. Specifically, we measure the relative improvement ofℱ\\mathcal\{F\}inch over the initial program score and compare it against two baselines: \(1\) the base model with a search scaffold; and \(2\) the base model with a learning scaffold \(e\.g\., TTT\-Discover\[yuksekgonul2026learning\]\)\. However, since running the original TTT\-Discover is prohibitively expensive—each task requires up to 50 epochs to reproduce, costing approximately 500 USD on average—we instead adopt nanodiscover444[https://github\.com/cheongalc/nanodiscover/](https://github.com/cheongalc/nanodiscover/), an open\-source reproduction of TTT\-Discover that does not depend on the Tinker API\. A detailed description of nanodiscover is provided in Appendix[D](https://arxiv.org/html/2606.29082#A4)\.

#### Inference Details\.

For the search\-based scaffold, following prior work\[liu2026evox\], we setT=100T=100with a parallel evaluation size of 1, temperature 0\.7, top\-pp0\.95, and a maximum of 30K tokens\. For the remaining scaffold\-specific hyperparameters, such as the island size, we adopt the default settings specified for each task\. For the learning\-based scaffold, we follow the same configurations as the original TTT\-Discover, which is presented in Appendix[D](https://arxiv.org/html/2606.29082#A4)\.

### 4\.2Impact of Evolution Fine\-Tuning

#### EFT improves cross\-task discovery generalization across all model scales\.

Table[3](https://arxiv.org/html/2606.29082#S4.T3)shows thatℱ\\mathcal\{F\}inch, evolution fine\-tuned onℱ\\mathcal\{F\}inch Collection, achieves substantial relative performance gains across 22 tasks, with the largest improvements observed on ahc058 \(\+290\.59%\) and Transaction \(\+74\.30%\)\. In Table[7](https://arxiv.org/html/2606.29082#S4.F7.11)and Figure[7](https://arxiv.org/html/2606.29082#S4.F7.11), we observe that EFT is also effective on complex optimization\-intensive tasks, including NP\-hard competitive programming and algorithmic heuristics\. Moreover, larger models benefit more fromℱ\\mathcal\{F\}inch Collection:ℱ\\mathcal\{F\}inch\-9B obtains a larger relative gain \(\+10\.31%\) thanℱ\\mathcal\{F\}inch\-4B \(\+3\.40%\)\. Our findings further demonstrate that EFT enables smaller models to match or even exceed the performance of non\-EFT models twice their size; for instance,ℱ\\mathcal\{F\}inch\-4B achieves 0\.386460 on Erdos, comparable to Qwen3\-8B’s 0\.403585\. Taken together, these results suggest thatℱ\\mathcal\{F\}inch Collection enables LLMs to internalize discovery capabilities, thereby exhibiting cross\-task discovery generalization\.

Table 4:Competitive Programming performance of![Refer to caption](https://arxiv.org/html/2606.29082v1/logos/finch_icon.png)ℱ\\mathcal\{F\}inch compared to base models across six optimization tasks\.
![Refer to caption](https://arxiv.org/html/2606.29082v1/x8.png)Figure 7:Algorithmic Heuristics performance of![Refer to caption](https://arxiv.org/html/2606.29082v1/logos/finch_icon.png)ℱ\\mathcal\{F\}inch compared to Qwen3\.5\-9B\.Table 5:Effect of Offline RL \(CP: avg\. competitive programming score\)\.
Table 6:Performance of![[Uncaptioned image]](https://arxiv.org/html/2606.29082v1/logos/finch_icon.png)ℱ\\mathcal\{F\}inch combined with online \(test\-time\) RL scaffolds across math optimization tasks\.

#### Teachingℱ\\mathcal\{F\}inch to distinguish good from bad solutions further enhances cross\-task discovery generalization\.

After EFT, we further trainℱ\\mathcal\{F\}inch using preference learning \(KTO\[ethayarajh2024kto\]\) onImpandRegjointly, in order to examine whether discovery capability can be further internalized by enablingℱ\\mathcal\{F\}inch to self\-judge which solutions are good and which are not\. As shown in Table[4\.2](https://arxiv.org/html/2606.29082#S4.SS2.SSS0.Px1), offline RL consistently improvesℱ\\mathcal\{F\}inch’s discovery capability; notably,ℱ\\mathcal\{F\}inch\-8B with KTO surpasses the best human score on both the AC1 and AC2 tasks\. These results suggest that \(1\)ℱ\\mathcal\{F\}inch Collection provides a useful and complementary training signal beyond supervised fine\-tuning, and \(2\) internalizing the ability to discriminate good from bad solutions is a viable path toward instilling discovery skills directly into the model’s parameters, rather than relying solely on test\-time search\.

![Refer to caption](https://arxiv.org/html/2606.29082v1/x9.png)Figure 8:Scaling trends with increasing numbers of training tasks inℱ\\mathcal\{F\}inch Collection, evaluated on AC2, CP \(n=26n=26\), and PRISM\.
Table 7:Effect of improvement type on EFT \(CP: avg\. competitive programming score\)

#### EFT serves as mid\-training for test\-time RL\.

We apply test\-time RL toℱ\\mathcal\{F\}inch using nanodiscover\. Table[4\.2](https://arxiv.org/html/2606.29082#S4.SS2.SSS0.Px1)shows thatℱ\\mathcal\{F\}inch achieves the best performance on two circle\-packing tests and improves performance on Erdős \(\+3\.2%\)\. These results suggest that EFT can serve as a form of mid\-training that strengthens test\-time RL\. However, compared with the original TTT\-Discover using GPT\-OSS\-120B,ℱ\\mathcal\{F\}inch still achieves lower performance, indicating that it remains difficult for smaller models \(e\.g\., 8B\-scale\) to discover push\-frontier solutions\.

## 5Analysis

![Refer to caption](https://arxiv.org/html/2606.29082v1/x10.png)Figure 9:Case Study on Convolve2D\.#### Scaling the number of tasks improves cross\-task discovery transfer\.

As shown in Figure[8](https://arxiv.org/html/2606.29082#S4.F8), performance on AC2, CP \(n=26n=26\), and PRISM exhibits a clear positive scaling trend as the number of training tasks inℱ\\mathcal\{F\}inch Collection increases\. These results indicate thatℱ\\mathcal\{F\}inch Collection provides a scalable training signal: increasing the amount ofℱ\\mathcal\{F\}inch Collection consistently improves discovery performance\. This suggests that the gains from EFT are not merely task\-specific, but can further grow with larger task collections, more trajectories, and increased model capacity\.

#### ℱ\\mathcal\{F\}inch can transfer discovery pattern by adopting different domain knowledge\.

Figure[9](https://arxiv.org/html/2606.29082#S5.F9)presents a case study on the Convolve2D task\. In this example,ℱ\\mathcal\{F\}inch\-9B modifies the implementation fromscipytojaxto improve computational efficiency\. We hypothesize that this behavior emerges becauseℱ\\mathcal\{F\}inch Collection contains a substantial number of trajectories involving thejaxlibrary, particularly from tasks such as uncertainty inequalities and matrix multiplication, as shown in Figure[6](https://arxiv.org/html/2606.29082#S3.F6)\(d\)\. These results suggest thatℱ\\mathcal\{F\}inch Collection enables LLMs to internalize transferable discovery patterns across domains\.

## 6Related Work

#### LLM\-driven Evolutionary Search Scaffolds\.

Recent work shows that LLM\-driven evolutionary search scaffolds can produce novel solutions across diverse optimization tasks\. These methods can be broadly categorized into*search\-based*and*learning\-based*approaches\. Search\-based scaffolds primarily differ in how they archive candidate solutions and select parents for mutation\. For instance, AlphaEvolve\[novikov2025alphaevolve\]employs MAP\-Elites with island\-based populations to balance exploration and exploitation, while OpenEvolve adopts a similar MAP\-Elites framework with periodic migration\. CodeEvolve\[assumpccao2025codeevolve\]integrates island\-based genetic algorithms with inspiration\-based crossover and meta\-prompting, and ShinkaEvolve\[lange2025shinkaevolve\]replaces fixed quality–diversity grids with a sample\-efficient regime combining weighted sampling, novelty rejection, and bandit\-based LLM selection\. Other methods explore alternative selection and adaptation strategies: GEPA operates along Pareto frontiers; PACEvolve\[yan2026pacevolve\]and AdaEvolve\[cemri2026adaevolve\]emphasize long\-horizon, progress\-aware, and adaptive proposal mechanisms; and EvoX\[liu2026evox\]and SkyDiscover\[skydiscover2026\]study meta\-evolution and unified discovery frameworks\. Application\-specific extensions include KernelEvolve\[liao2025kernelevolve\]and Kernel\-Smith\[du2026kernel\]for GPU kernel optimization, while the Darwin Gödel Machine\[zhang2025darwin\], HyperAgents\[zhang2026hyperagents\], and CORAL\[qu2026coral\]investigate open\-ended self\-improvement and multi\-agent evolution\. Meta\-Harness\[lee2026meta\]further shifts the optimization target from candidate solutions to the*harness*itself\. In contrast, learning\-based approaches such as ThetaEvolve\[wang2025thetaevolve\]and TTT\-Discover\[yuksekgonul2026learning\]demonstrate that combining evolutionary search with test\-time learning enables LLMs to internalize discovery capabilities\.

#### Benchmarks for Optimization and Scientific Discovery\.

Progress in LLM\-driven discovery has been driven by benchmarks that emphasize long\-horizon, open\-ended problem solving under strong baselines\. KernelBench\[ouyang2025kernelbench\]and GSO\[shetty2025gso\]evaluate iterative optimization of GPU kernels and programs, while AlgoTune\[press2025algotune\]measures performance gains in numerical routines\. ALE\-Bench\[imajuku2025ale\]focuses on long\-horizon algorithm engineering tasks derived from programming contests\. FrontierCS\[mang2025frontiercs\]and AutoLab\[autolab\-2026\]introduce*living*benchmarks that evolve alongside frontier models and support end\-to\-end scientific–engineering loops\. For scientific\-law discovery, LLM\-SRBench\[shojaee2025llm\]evaluates whether models can recover symbolic equations from data, while\[lin2025can\]examine the rediscovery of empirical scaling laws\.

## 7Conclusion

In this work, we show that evolution fine\-tuning \(EFT\) — fine\-tuning LLMs on a large collection of evolutionary trajectories spanning 371 tasks — improves the model’s discovery capability on 22 held\-out tasks, demonstrating cross\-task discovery generalization\. Furthermore, we show thatℱ\\mathcal\{F\}inch exhibits a synergistic effect when combined with test\-time RL\.

## Limitations

#### Mixed Test\-Time Search Scaffolds\.

In this work, we use only the OpenEvolve scaffold for both collecting trajectories and evaluatingℱ\\mathcal\{F\}inch\. A concern is that a model trained on OpenEvolve\-style trajectories may not generalize well when combined with stronger scaffolds, such as EvoX\[liu2026evox\]\. To mitigate this issue in future work, collecting trajectories from mixed scaffolds will be necessary as a form of template input variation\[longpre2023flan\]to improve cross\-scaffold generalization\.

#### Extending Test\-Time RL Experiments to Diverse and Realistic Tasks\.

In this work, we demonstrate thatℱ\\mathcal\{F\}inch exhibits a positive synergistic effect with test\-time RL only on mathematical tasks\. However, many practical tasks exist in the real world, such as kernel engineering\. It is therefore important to verify whether our model also exhibits a positive synergistic effect with test\-time RL on these tasks in order to establish the broader effectiveness of our approach\.

#### Beyond Language Modality and Single\-turn\.

Scientific discovery is not limited to the language modality alone\. In many scientific domains, researchers must also interpret visual observations from experiments\[sun2025scienceboard,ma2026orion\], making multi\-modal reasoning an essential component of the discovery process\. Similar to how we internalize discovery capabilities into LLMs in this work, our training recipe can be naturally extended to the multi\-modal setting\. In particular, recent advances in distillation techniques\[lee2024collavo,lee2024moai,lee2024meteor,lee2024phantom,lee2024trol,lee2025genrecal,lee2025vlsi,lee2026unified,kang2026agent,lee2026masking,lee2026recursive,cho2026spatialclaw,yu2026hide,kim2026and\]for Vision–Language Models are highly complementary to our approach and could enableEvolution Fine\-Tuningfor multi\-modal scientific discovery\. Furthermore, our current framework trains the model to generate a child solution conditioned on the evolutionary history and parent program in a single turn\. An important future direction is to extend this paradigm to multi\-turn interactions\[lee2024dialogcc,lee2024stark,lee2024thanos,lee2024large,lee2025multiverse,lee2025refinebench\], allowing the model to continually reason over previously explored lineages and iteratively learn to evolve solutions in more promising directions\.

## Acknowledgement

This research was supported by the “Advanced GPU Utilization Support Program” funded by the Government of the Republic of Korea \(Ministry of Science and ICT\)\. We thank the SkyDiscover team for their valuable feedback on the dataset construction process, the use of the SkyDiscover framework, and the overall direction of this research\. In particular, we would like to thank Shu Liu, Shubham Agarwal, and Mert Cemri for their insightful comments and discussions\. We also thank the OpenEvolve team, especially Ritik Vijayvergiya and Asankhaya Sharma, for their helpful feedback on the use of the OpenEvolve framework and for their thoughtful comments on this work\. We are also grateful to the authors of ALE\-Bench, especially Yuki Imajuku, and the AtCoder team for authorizing the public release of the evolutionary search trajectories derived from their CC BY\-ND 4\.0 licensed dataset\. We thank Minnesota NLP group members for valuable feedback\. In addition, we thank Byung\-Kwan Lee, a Research Scientist at NVIDIA, for providing valuable feedback during the early stages of this project\. Finally, we sincerely thank Tahee Jung from Amazon for providing extensive and valuable feedback throughout this project\.

## References

## Appendix ABroader Impacts

EFT democratizes LLM\-driven discovery by transferring optimization capabilities from expensive proprietary models to small open\-weight models, reducing search costs, enabling fully local discovery pipelines, and providingℱ\\mathcal\{F\}inch Collection as a reusable resource for future research\. At the same time, these capabilities may be misused for harmful optimization objectives, reward hacking, or over\-reliance on automatically generated discoveries without sufficient verification\. To mitigate these risks, we train only on public scientific and engineering benchmarks, preserve the upstream safety properties of base models, and recommend human oversight, scoring\-function auditing, and trajectory logging for downstream deployment\.

## Appendix BFurther Details and Discussion

### B\.1Why is the model namedℱ\\mathcal\{F\}inch?

Darwin’s finches, despite belonging to the same species, have evolved differently in response to the diverse ecological environments of the Galápagos Islands\[darwin2025origin,grant2002unpredictable\]\. This observation illustrates their remarkable ability to adapt to a wide range of environmental conditions and to thrive within them\.

Inspired by this phenomenon, we name our modelℱ\\mathcal\{F\}inch\. Analogous to Darwin’s finches, our model is designed to adapt across diverse environments—here corresponding to different optimization tasks—and to effectively operate within them\. In particular, it reflects the model’s ability to flexibly adapt to various tasks \(e\.g\., mathematics, kernel engineering, biology denoising\) and to discover better solutions within each task setting\.

### B\.2Definition of Scaffolds

The termscaffoldis borrowed from developmental psychology\. Vygotsky\[vygotsky2011interaction\]characterizes scaffolding as external support that elevates a learner’sassisted performancebeyond their unaided reach, and further distinguishes between progress driven byexternally suppliedstructure and progress driven by the learner’sinternalizationof past experience\. Following this distinction, we taxonomize evolutionary search scaffolds intosupervisedandunsupervisedscaffolds\.

#### Definition: Supervised Scaffold\.

A supervised scaffold \(i\.e\.,, search\-based\)\[openevolve,lange2025shinkaevolve,cemri2026adaevolve,liu2026evox,skydiscover2026\]keeps the mutation operator’s parametersθ\\thetafixed and drives discovery through a hand\-engineered modular framework that compensates for the LLM’s limited intrinsic capacity for discovery\. It plays the role of an external tutor, as the policies governing how candidate solutions are selected and stored are typically designed by human experts\.

#### Definition: Unsupervised Scaffold\.

An unsupervised scaffold \(i\.e\.,, learning\-based\)\[wang2025thetaevolve,yuksekgonul2026learning\]updatesθ\\thetaat test time via reinforcement learning \(RL\) over the LLM’s self\-generated experiences\. Prior works still retain a database and a selection strategy \(e\.g\.,, PUCT\), but the surrounding framework is comparatively lightweight, corresponding to Vygotsky’sinternalizationphase\.

Strong performance under a supervised scaffold is not evidence thatℳθ\\mathcal\{M\}\_\{\\theta\}hasinternalizeddiscovery: sinceθ\\thetais frozen, any gain is jointly attributable to the LLM and its external framework, and corresponds to Vygotsky’s assisted performance\. Genuine internalization requires improvements to accrue toℳθ\\mathcal\{M\}\_\{\\theta\}itself—persisting even after the scaffolding is stripped away—which motivates the unsupervised regime\. In this work, our goal is to equip the LLM with internalized discovery capability, enabling it to operate effectively under both supervised and unsupervised scaffolds\.

Table 8:Key differences across three standard tasks: Reasoning vs\. Agentic vs\. Optimization\.

### B\.3Distinguishing Optimization from Reasoning and Agentic Tasks

We position optimization \(or discovery\) tasks as fundamentally distinct from both reasoning and agentic tasks, as summarized in Table[8](https://arxiv.org/html/2606.29082#A2.T8)\.

Unlike reasoning tasks, which assume well\-defined problems with known ground\-truth solutions, optimization tasks are inherently open\-ended: the objective is not to recover a known answer, but to discover improved or entirely novel solutions\. This lack of ground truth shifts the success criterion from correctness to relative improvement over prior best\-known results\.

Compared to agentic tasks, which emphasize executing sequences of actions to satisfy predefined acceptance criteria in real\-world environments, optimization tasks require searching over a substantially larger and more abstract solution space \(e\.g\., programs, algorithms, or mathematical constructions\)\. As a result, they exhibit longer time horizons and demand iterative refinement rather than single\-pass execution\.

These differences lead to a distinct evaluation paradigm\. While reasoning and agentic tasks are typically evaluated using binary success signals, optimization tasks rely on deterministic but continuous metrics that enable partial progress to be measured and accumulated over time\.

Taken together, optimization tasks necessitate a different role for language models: instead of acting purely as a reasoning engine or a planner, the model serves as a mutation operator within an evolutionary search process, iteratively proposing candidate solutions that can be evaluated and improved\. This distinction motivates the design ofℱ\\mathcal\{F\}inch, which is specifically tailored to support discovery\-driven optimization\.

## Appendix CAdditional Details of Dataset Construction Method

### C\.1Systematic Error Breakdown Analysis

Table 9:Breakdown of system\-level errors filtered out \(6,321 trajectories\)\. Counts and percentages are calculated over the total of 6,321 trajectories\.Error categoryCount%Descriptionparent\_missing\_combined\_score3,64157\.60Parent program itself never produced acombined\_score, so the parent→\\tochild improvement delta is undefined regardless of what the child did\.artifact\_error\_type:timeout90114\.25Evaluator wall\-clock timeout: child program ran past its time budget \(artifacts\.timeout = True,error\_type = "timeout"\)\.failure\_stage:correctness79412\.56Child program ran to completion but produced numerically incorrect outputs \(the evaluator’s correctness check rejected the result\)\.child\_metrics\_error\_string6059\.57child\_metrics\.errorholds a human\-readable failure message \(e\.g\."C1 mismatch: reported X, computed Y"\) rather than a clean numeric metric\.artifact\_error\_type:TimeoutError1642\.59AsyncTimeoutErrorraised from the evaluator’s task wrapper \(typically duringcascade\_setup,stage1, orstage2\)\.failure\_stage:benchmark1472\.33Failure inside the benchmark/timing harness*after*the correctness check passed \(e\.g\. a crash during the performance\-measurement loop\)\.artifact\_error\_string610\.97artifacts\.errorcontains a runtime exception string \+ traceback \(e\.g\.NameError, syntax error, JAX trace error\) with no structurederror\_typeorfailure\_stage\.failure\_stage:syntax\_check50\.08Generated code failed the pre\-execution syntax check \(could not be parsed or compiled\)\.failure\_stage:import20\.03Generated module failed to import \(missing symbol,ImportError, or top\-level exception during import\)\.failure\_stage:validation10\.02Generated solution failed schema/shape validation before execution\.Total6,321100\.00

### C\.2Benchmark Licenses

We construct our seed tasks from publicly available benchmarks, all of which are released under permissive open\-source licenses \(e\.g\., MIT, Apache 2\.0, or CC\-BY 4\.0\)\. Table[10](https://arxiv.org/html/2606.29082#A3.T10)summarizes the license terms for the corresponding GitHub repositories and associated datasets for each benchmark\. Our use of these resources is limited to non\-commercial academic research and complies with the terms of their respective licenses, including all attribution requirements\. Accordingly,ℱ\\mathcal\{F\}inch Collection is released under the CC\-BY 4\.0 license, while our code andℱ\\mathcal\{F\}inch weights are released under the Apache 2\.0 license\. Both are compatible with the licenses of all upstream benchmarks listed above\. In particular, the tasks sourced from ALE\-Bench\[imajuku2025ale\]are derived from a dataset released under the CC BY\-ND 4\.0 license\. We sincerely thank the AtCoder team for granting permission to publicly release the evolutionary search trajectories derived from their CC BY\-ND 4\.0 licensed dataset\.

Table 10:Benchmark Licenses

## Appendix DAdditional Implementation Details

#### Implementation details forℱ\\mathcal\{F\}inch\-8B nanodiscover runs\.

nanodiscover is an open\-source reproduction of TTT\-Discover that does not depend on the Tinker API\. It is publicly available at[https://github\.com/cheongalc/nanodiscover](https://github.com/cheongalc/nanodiscover)\. Each search epoch in nanodiscover consists of five stages mirroring the TTT\-Discover pipeline: \(1\) sampling parent solutions from the archive, \(2\) generating child solutions from parent solutions, \(3\) evaluating child solutions, \(4\) updating the archive, and \(5\) test\-time training\. Ray Data LLM \(which orchestrates vLLM under the hood\) is used for step \(2\), while DeepSpeed is used for step \(5\)\. Unless otherwise noted, all hyperparameters were matched to TTT\-Discover as closely as possible\.

All runs were conducted for 50 epochs on a single node with 4 GPUs and 96 logical CPU cores\. For the Erdős task, the prompt informed the model that the evaluation budget was 1000 seconds, while the actual timeout was set to 1100 seconds\. For both circle\-packing tasks, the model was not informed of the evaluation budget, and the actual timeout was 530 seconds\. These timing configurations follow the TTT\-Discover setup\.

## Appendix EFull List of Optimization Tasks inℱ\\mathcal\{F\}inch Collection

Table 19:Per\-task descriptions for theCompetitive Programming \(Frontier\-CS\)task group \(172 tasks\) evaluated underℱ\\mathcal\{F\}inch Collection\.circle\_packing\_rectmax​∑ri\\max\\sum r\_\{i\}Pack equal\-radius circles inside an axis\-aligned rectangle without overlap; maximize the common radius under boundary and non\-overlap constraints\.erdos\_min\_overlapmin⁡M​\(n\)\\min M\(n\)Construct a witness function \(step function on\[−1,1\]\[\-1,1\]\) that gives a constructive upper bound on Erdős’s minimum\-overlap constantM​\(n\)M\(n\)\.first\_autocorr\_ineqmin⁡C1\\min C\_\{1\}First autocorrelation inequality: minimize∥f∗f∥∞\\lVert f\*f\\rVert\_\{\\infty\}over functionsf≥0f\\geq 0supported on\[−1/4,1/4\]\[\-1/4,1/4\]with∫f=1\\int f=1\.second\_autocorr\_ineqmax⁡C2\\max C\_\{2\}Second autocorrelation inequality: extremal∥f∗f∥\\lVert f\*f\\rVertunder prescribed support and mass constraints\.third\_autocorr\_ineqmin⁡C3\\min C\_\{3\}Third autocorrelation inequality: tighten the upper bound on the third autocorrelation constant arising in additive combinatorics\.heilbronn\_trianglemax⁡min\\max\\minareaPlacennpoints in the unit square\[0,1\]2\[0,1\]^\{2\}; maximize the area of the smallest triangle formed by any three points\.heilbronn\_convex\_13max⁡min\\max\\minareaHeilbronn\-on\-a\-convex\-region variant withn=13n\{=\}13points: maximize the minimum convex\-hull area over all subsets ofk\>3k\>3points\.heilbronn\_convex\_14max⁡min\\max\\minareaHeilbronn\-on\-a\-convex\-region variant withn=14n\{=\}14points \(otherwise as above\)\.hexagon\_packing\_11min\\minenclosingPackn=11n\{=\}11unit regular hexagons inside the smallest enclosing regular hexagon\.hexagon\_packing\_12min\\minenclosingPackn=12n\{=\}12unit regular hexagons inside the smallest enclosing regular hexagon\.minimizing\_max\_min\_dist\_2max⁡dmin/dmax\\max\\,d\_\{\\min\}/d\_\{\\max\}Place points in\[0,1\]2\[0,1\]^\{2\}so that the ratio of minimum to maximum pairwise distance is as close to 1 as possible \(uniform 2D point distribution\)\.minimizing\_max\_min\_dist\_3max⁡dmin/dmax\\max\\,d\_\{\\min\}/d\_\{\\max\}Same as above but for points in\[0,1\]3\[0,1\]^\{3\}\(uniform 3D point distribution\)\.signal\_processingmulti\-objectiveDesign a causal real\-time filter for a noisy non\-stationary time series, balancing fidelity, smoothness, lag, and false\-trend detection\.sums\_diffs\_finite\_setsmin⁡\|A\+A\|/\|A−A\|\\min\|A\{\+\}A\|/\|A\{\-\}A\|Construct a finite setA⊂ℤA\\subset\\mathbb\{Z\}minimizing the ratio between sumset and difference\-set cardinalities \(additive combinatorics\)\.