Agentic Neural Architecture Search

# Agentic Neural Architecture Search
Source: [https://arxiv.org/html/2607.07984](https://arxiv.org/html/2607.07984)
###### Abstract

Neural architecture search \(NAS\) methods have grown increasingly efficient, yet they remain bounded by manually engineered search spaces that require substantial domain expertise and must be rebuilt for every new task\. Large language models \(LLMs\) can generate architectures in an open\-ended space, but how to optimally divide the labor between LLM\-driven design and NAS\-driven search remains unexplored\. We propose a mechanism that bridges these two paradigms: an LLM produces a high\-quality seed architecture, then decomposes it into a*slotted architecture*—a scaffold with named, interchangeable module slots that automatically defines a bounded, task\-specific search space for conventional NAS to explore, without manual engineering\. We instantiate this mechanism inAgentNAS, a modular three\-phase pipeline in which each component’s contribution can be measured independently\. On 17 tasks spanning classification, dense regression, segmentation, and multi\-label tagging across diverse modalities \(NAS\-Bench\-360 and Unseen NAS\), AgentNAS establishes a new state of the art on 11 tasks, outperforming published baselines including task\-specific expert designs\. Ablation studies show that the two search mechanisms are broadly complementary: the LLM\-generated seed already surpasses published baselines on the majority of tasks, and NAS delivers additional gains in most cases through combinatorial recombination across slots—a mode of search that independent LLM samples cannot replicate\. These patterns hold across three LLMs of different capability levels, confirming that the division of labor is robust\. Our code is available at[https://github\.com/alroimfebruary/AgentNAS](https://github.com/alroimfebruary/AgentNAS)\.

Neural architecture search, Code generation, LLM agents, Algorithmic discovery, Automated machine learning

## 1Introduction

Neural architecture search \(NAS\) aims to automatically find a neural architecture, replacing manual design of neural networks\. Studies on weight\-sharing methods\(Liuet al\.,[2019b](https://arxiv.org/html/2607.07984#bib.bib5); Phamet al\.,[2018](https://arxiv.org/html/2607.07984#bib.bib6)\)or evolutionary algorithms\(Realet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib12)\)have made the search algorithm increasingly efficient\. Nevertheless, every method still presupposes a hand\-engineered search space that demands domain expertise and must be rebuilt for each new task\(Elskenet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib46); Whiteet al\.,[2023](https://arxiv.org/html/2607.07984#bib.bib47)\)\. Large language models \(LLMs\) have begun to loosen this constraint: by encoding broad architectural priors from the research literature, they can generate or mutate network code in an open\-ended space, serving as mutation operators\(Chenet al\.,[2023](https://arxiv.org/html/2607.07984#bib.bib34); Morriset al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib49); Nasiret al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib36); Zhuet al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib50)\), optimizers\(Zhenget al\.,[2023](https://arxiv.org/html/2607.07984#bib.bib35); Yuet al\.,[2023](https://arxiv.org/html/2607.07984#bib.bib48)\), and hyperparameter tuners\(Zhanget al\.,[2023](https://arxiv.org/html/2607.07984#bib.bib38); Liuet al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib39)\)\. However, all of these approaches conflate two distinct roles—*expanding*the space of possible architectures and*exploring*it—making it difficult to study when each capability adds value\. This conflation leaves two fundamental questions unanswered:*can an LLM replace NAS?*and*if not, what is the optimal division of labor between LLMs and NAS?*An LLM can propose diverse architectures from its learned prior, but each proposal is largely an independent sample—it does not systematically explore how multiple design choices interact\. NAS excels at precisely this kind of combinatorial search, yet it requires a bounded space to search within\. The two capabilities are complementary in principle, but studying their interaction requires a framework that \(i\) lets an LLM construct the search space itself, \(ii\) lets conventional NAS explore it, and \(iii\) cleanly separates the contributions of each so they can be measured independently\. We propose a mechanism that provides exactly this separation: an LLM first produces a high\-quality seed architecture, then decomposes it into a*slotted architecture*—a scaffold with named, interchangeable module slots that automatically defines a bounded, task\-specific search space for conventional NAS to explore, without manual engineering\. The key insight is that this decomposition preserves the LLM’s macro\-structural decisions \(depth, width progression, backbone type\) while exposing combinatorial degrees of freedom at the module level, precisely where NAS is most effective\. We instantiate this mechanism inAgentNAS, a modular three\-phase pipeline whose separation of concerns allows each component’s contribution to be measured independently\.

On 17 tasks from NAS\-Bench\-360\(Tuet al\.,[2022](https://arxiv.org/html/2607.07984#bib.bib43)\)and Unseen NAS\(Geadaet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib44)\)—spanning classification, dense regression, segmentation, and multi\-label tagging across diverse modalities—AgentNAS establishes a new state of the art on 11 tasks, outperforming published baselines including task\-specific expert designs\. Ablation studies show that the two search mechanisms are broadly complementary: even when the LLM fails to improve further, NAS delivers additional gains on top of the seed in most cases\.

1. 1\.We presentAgentNAS, a mechanism by which an LLM decomposes a seed architecture into a*slotted architecture*, automatically constructing a task\-specific search space amenable to conventional NAS without manual engineering\.
2. 2\.We conduct an empirical study on the LLM–NAS division of labor, showing that the two search mechanisms are broadly complementary and that NAS outperforms matched\-budget LLM sampling\.
3. 3\.AgentNAS achieves state\-of\-the\-art results on 11 of 17 diverse tasks, including a blind benchmark where the LLM receives no domain metadata and must discover effective architectures from data alone\.

## 2Related Work

### 2\.1Neural Architecture Search

Neural architecture search \(NAS\) aims to automate the design of neural network architectures, replacing the trial\-and\-error process that has traditionally relied on human expertise\. The pioneering work ofZoph and Le \([2017](https://arxiv.org/html/2607.07984#bib.bib1)\)first demonstrated that a recurrent controller trained with reinforcement learning could discover architectures competitive with hand\-designed models, albeit at enormous computational cost\. Following this work, two large axes of research emerged\. The first is weight\-sharing approaches\(Benderet al\.,[2018](https://arxiv.org/html/2607.07984#bib.bib2),[2020](https://arxiv.org/html/2607.07984#bib.bib3); Caiet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib4); Liuet al\.,[2019b](https://arxiv.org/html/2607.07984#bib.bib5); Phamet al\.,[2018](https://arxiv.org/html/2607.07984#bib.bib6); Xuet al\.,[2020](https://arxiv.org/html/2607.07984#bib.bib7)\), where a single over\-parameterized supernet is trained once and individual architectures are evaluated as its sub\-networks\. The second is evolutionary methods\(Caiet al\.,[2020](https://arxiv.org/html/2607.07984#bib.bib8); Luet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib10); Realet al\.,[2017](https://arxiv.org/html/2607.07984#bib.bib11),[2019](https://arxiv.org/html/2607.07984#bib.bib12)\), which explore sparse search spaces through population\-based optimization\. Although these two paradigms differ in mechanism, both operate within a fixed, human\-designed search space—a constraint that has shaped the trajectory of the field\.

The introduction of tabular benchmarks providing pre\-computed architecture metadata\(Yinget al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib31); Dong and Yang,[2020](https://arxiv.org/html/2607.07984#bib.bib32); Donget al\.,[2022](https://arxiv.org/html/2607.07984#bib.bib33)\)further reinforced this tendency: because these benchmarks are built on convolution\-based ResNet\(Heet al\.,[2016](https://arxiv.org/html/2607.07984#bib.bib15)\)\-style search spaces for image classification tasks\(Krizhevskyet al\.,[2009](https://arxiv.org/html/2607.07984#bib.bib51); Denget al\.,[2009](https://arxiv.org/html/2607.07984#bib.bib14)\), the majority of subsequent NAS research has been evaluated within the same narrow setting\. Nonetheless, several works have demonstrated the applicability of NAS beyond this setting, targeting transformers\(Soet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib18); Yanget al\.,[2023](https://arxiv.org/html/2607.07984#bib.bib13); Liet al\.,[2021a](https://arxiv.org/html/2607.07984#bib.bib16)\), graph neural networks\(Gaoet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib19); Li and King,[2020](https://arxiv.org/html/2607.07984#bib.bib20); Zhouet al\.,[2022](https://arxiv.org/html/2607.07984#bib.bib21)\), and generative adversarial models\(Gonget al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib22); Gaoet al\.,[2020](https://arxiv.org/html/2607.07984#bib.bib23)\)\.

Across all of these efforts—whether they differ in search strategy, efficiency technique, or target domain—a common limitation persists: the search space itself must be manually engineered\. Several studies have shown that the expressiveness of the search space is at least as important as the choice of search algorithm\(Geadaet al\.,[2020](https://arxiv.org/html/2607.07984#bib.bib24); Liuet al\.,[2019a](https://arxiv.org/html/2607.07984#bib.bib25),[2018](https://arxiv.org/html/2607.07984#bib.bib26); Robertset al\.,[2021](https://arxiv.org/html/2607.07984#bib.bib27); Ruet al\.,[2020](https://arxiv.org/html/2607.07984#bib.bib28)\), yet designing such spaces remains a labor\-intensive process that demands substantial domain expertise\.Realet al\.\([2020](https://arxiv.org/html/2607.07984#bib.bib29)\)attempted to sidestep this problem by composing architectures from the most primitive operations, but the approach required a prohibitive amount of computation\. More recent works based on context\-free grammars \(CFGs\)\(Schrodiet al\.,[2023](https://arxiv.org/html/2607.07984#bib.bib17); Ericssonet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib30)\)offer a more principled way to extend the search space, but they are not compatible with well\-studied supernet\-based search strategies and ultimately shift the manual engineering burden from architecture design to grammar design\. Despite these advances, the search space remains a manual design artifact, constraining the diversity of discoverable architectures even as search algorithms continue to improve\.

### 2\.2Large Language Models for Neural Architecture Search

The limitation identified above—that the search space itself is the bottleneck—has motivated a fundamentally different line of work that leverages large language models \(LLMs\) for architecture design\. Rather than searching within a predefined space, LLMs can directly generate or manipulate architecture code, opening a near\-infinite, code\-level search space\. The most prevalent application of LLMs in this context is as code\-level mutation and crossover operators within evolutionary frameworks\(Chenet al\.,[2023](https://arxiv.org/html/2607.07984#bib.bib34); Morriset al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib49); Nasiret al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib36); Zhuet al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib50)\), or as direct architecture generators that produce candidate networks from natural\-language or code\-level specifications\(Yanget al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib41); Rahman and Chakraborty,[2024](https://arxiv.org/html/2607.07984#bib.bib42)\)\. Beyond generation, LLMs have also been employed as optimizers\(Zhenget al\.,[2023](https://arxiv.org/html/2607.07984#bib.bib35); Yuet al\.,[2023](https://arxiv.org/html/2607.07984#bib.bib48)\), rerankers\(Huet al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib37)\), principle extractors\(Zhouet al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib40)\), and hyperparameter tuners\(Zhanget al\.,[2023](https://arxiv.org/html/2607.07984#bib.bib38); Liuet al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib39)\)\. However, a critical gap remains\. While these works demonstrate that LLMs can operate in an open\-ended search space, they primarily use the LLM as a search operator or optimizer within the search loop\. To our knowledge, no prior work uses the LLM to construct a bounded, task\-specific search space that is then handed off to a conventional NAS algorithm\. As a consequence, the question of how to divide labor between LLM\-driven design and NAS\-driven search—and under what conditions each component adds value—has not been empirically studied\.

### 2\.3Automated Scientific Discovery

More broadly, recent work has demonstrated that combining LLMs, multi\-agent systems, programmatic evaluators, and evolutionary search can discover novel algorithms\(Romera\-Paredeset al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib52); Novikovet al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib54); Presset al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib87)\)or conduct scientific research autonomously\(Luet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib55); Yamadaet al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib56); Schmidgall and Moor,[2025](https://arxiv.org/html/2607.07984#bib.bib57); Schmidgallet al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib58)\)\. Notably, this paradigm has been extended to scientific machine learning problems such as discovering PINNs configurations and PDE surrogates\(Wuwuet al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib89); Songet al\.,[2026](https://arxiv.org/html/2607.07984#bib.bib90); Jiang and Karniadakis,[2026](https://arxiv.org/html/2607.07984#bib.bib88)\), while concurrent work expands the neural operator design space itself through architectures grounded in state\-space models\(Tiwariet al\.,[2025](https://arxiv.org/html/2607.07984#bib.bib91); Song and Jiang,[2026](https://arxiv.org/html/2607.07984#bib.bib92),[2025](https://arxiv.org/html/2607.07984#bib.bib93)\)\. These efforts share the broader objective of automating architecture discovery for mathematical and scientific computation, while AgentNAS targets general\-purpose NAS that can naturally extend to such domains, as illustrated by our Darcy Flow experiments\.

## 3Method

![Refer to caption](https://arxiv.org/html/2607.07984v1/figures/pipeline-overview.png)Figure 1:The AgentNAS pipeline\. Phase 1 iteratively prompts an LLM to propose, implement, and train candidate architectures, returning a seed architecture and its training recipe\. Given the updated leaderboard and self\-generated summary with the proxy scores of proposed architectures, an LLM iteratively improves architectures until it saturates\. Phase 2 decomposes the seed into a slotted architecture with per\-slot alternatives, defining a task\-specific search space\. Phase 3 explores this space with conventional NAS\. Each square represents a slot; a complete sequence indicates a candidate architecture\. We show examples of the three most standard NAS algorithms on the slotted architectures search space\.AgentNAS is a three\-phase pipeline that leverages the architectural priors of an LLM to construct a task\-specific search space, which is then explored by conventional NAS\. Motivated by the observation ofEricssonet al\.\([2024](https://arxiv.org/html/2607.07984#bib.bib30)\)that bootstrapping NAS with human\-designed architectures is beneficial, our pipeline uses the LLM to produce both a strong seed architecture and the search space around it\. An overview of the system is shown in Figure[1](https://arxiv.org/html/2607.07984#S3.F1), while Figure[2](https://arxiv.org/html/2607.07984#S4.F2)shows an example of the slotted architecture on the*Spherical*task\. The details of the three phases are described below and further detailed descriptions of pipeline components can be found in Appendix[A\.7](https://arxiv.org/html/2607.07984#A1.SS7)\.

### 3\.1Phase 1: LLM\-Driven Seed Search

Phase 1 iteratively refines architectures via a*Planner–Code Generator–Explorer*loop, returning a seed architecture and its training recipe \(optimizer, learning rate, schedule, augmentation, batch size, epochs\)\. The Planner selects among three actions at each step:<EXPLORE\>to launch a separate agent to inspect data characteristics via a sandboxed code environment,<EXPERIMENT\>to propose and evaluate candidate architectures, and<STOP\>to conclude the search\. The search budget is 160 evaluations; each candidate is implemented by the code generator agent and trained under a30%30\\%\-epoch proxy with GPU time and VRAM constraints, and the top 8 architectures are fully trained after termination\. Only validation metrics are used for selection; test metrics are never exposed to the pipeline\. Full details can be found at Appendix[A\.7](https://arxiv.org/html/2607.07984#A1.SS7)and Appendix[A\.8](https://arxiv.org/html/2607.07984#A1.SS8)\.

### 3\.2Phase 2: Slotted Architecture Synthesis

Phase 2 constructs a*slotted architecture*: the seed equipped with named, interchangeable module slots that define the Phase 3 search space\. The slotted architecture mostly preserves the seed’s macro\-structural decisions—depth, width progression, stage structure, and backbone type—while exposing combinatorial degrees of freedom at the module level, where NAS is most effective\. A*Slot Planner*receives the seed’s source code, its training recipe, and the Data Explorer’s report, then decomposes the seed into a scaffold with two structurally distinct slot types\.

The most straightforward approach would be to create slots for every operation, but this enlarges the search space excessively \(up to∼1090\{\\sim\}10^\{90\}configurations in our preliminary experiment\)\. Instead, motivated by the observation in hierarchical NAS\(Tanet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib59); Wuet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib60)\)and cell\-based NAS\(Liuet al\.,[2019b](https://arxiv.org/html/2607.07984#bib.bib5)\)that stacking a well\-designed block leads to a good architecture, we design slot types to leverage the LLM’s slot\-design capability and NAS’ combinatorial search capability\.

##### Modularization\.

When the seed uses a block\-stack pattern where operations such as convolution, normalization, and activation are composed as separate sequential primitives, a separate*Modularizer Agent*first groups them into coherent module\-level units before decomposition\. We observe that without this process, sometimes Slot Planner tends to decompose the architecture into much smaller granularities, enabling topological mutations but making the search very unstable\. This observation is empirically documented in Appendix[A\.7\.3](https://arxiv.org/html/2607.07984#A1.SS7.SSS3)\.

##### Module\-level slots\.

Slot Planner establishes a detailed plan for translating a seed architecture’s components into module\-level slots, whose alternatives are implemented by a code generator agent as a completenn\.Modulethat performs a coarse, internally coherent transformation at one stage of the network\. This slot type mutates the seed by*replacing*components while preserving the internal coherence of each block\.

##### Additive glue slots\.

Based on the idea that NAS often tends to result in unconventional structures, we let Slot Planner make*additive*slots, which are interleaved between module\-level slots and applied to a module’s output in the forward pass\. Each glue slot defaults to identity, so the seed’s behavior is preserved unless NAS actively selects an alternative\. This allows NAS to introduce complementary components—such as additional operations, activation functions, or skip connections—without disturbing the internal design of adjacent modules\.

##### Learning rate slot\.

Since the LLM tunes the training recipe along with the architecture, the learning rate is optimized to the seed architecture\. However, since module\-level replacement often alters the parameter size, the optimal learning rate often differs\. We add a virtual slot for learning rate, which includes five multipliers of the original training recipe:μ∈\{0\.25,0\.5,1\.0,2\.0,4\.0\}\\mu\\\!\\in\\\!\\\{0\.25,0\.5,1\.0,2\.0,4\.0\\\}\. We find that in several scenarios, learning rate is actively altered, as described in Appendix[A\.2](https://arxiv.org/html/2607.07984#A1.SS2)\.

### 3\.3Phase 3: Slotted Neural Architecture Search

The slotted architecture is compatible with most NAS methods that operate over a discrete, categorical search space\. We default to regularized evolution\(Realet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib12)\), and additionally evaluate random search\(Li and Talwalkar,[2019](https://arxiv.org/html/2607.07984#bib.bib86)\)and GDAS\(Dong and Yang,[2019](https://arxiv.org/html/2607.07984#bib.bib61)\), a Gumbel\-Softmax variant of DARTS\(Liuet al\.,[2019b](https://arxiv.org/html/2607.07984#bib.bib5)\), in Appendix[A\.3](https://arxiv.org/html/2607.07984#A1.SS3)\. Candidates are trained under the same proxy protocol as Phase 1\. After the search, the top 8 candidates are fully trained and evaluated following the same protocol as Phase 1\. Implementation details are in Appendix[A\.7\.2](https://arxiv.org/html/2607.07984#A1.SS7.SSS2)\.

## 4Experiments

### 4\.1Benchmarks

We evaluate on 17 tasks drawn from two diverse\-task benchmark suites\.

##### NAS\-Bench\-360

\(Tuet al\.,[2022](https://arxiv.org/html/2607.07984#bib.bib43)\)contributes 10 tasks spanning 1D and 2D classification, multi\-label audio tagging \(FSD50K\), dense regression \(Darcy Flow, PSICOV\), and segmentation \(Cosmic\)\.

##### Unseen NAS

\(Geadaet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib44)\)contributes 7 classification tasks deliberately constructed to stress domain\-agnostic architecture search, including chess\-board outcome prediction \(Chesseract\) and authorship attribution from character\-level features \(Gutenberg\)\. Unseen NAS was originally administered as a competition in which participants had no access to the evaluation datasets and could not identify their domains\. The organizers deliberately obfuscated data shapes and split sizes to resemble well\-known benchmarks, preventing participants from inferring the true task domain\. We replicate this blind setting: the LLM receives no domain metadata, and data shapes alone do not reveal the underlying task\. The LLM may inspect the data programmatically through the Data Explorer, but cannot leverage domain\-specific prior knowledge\. We find that the LLM fully discovers only Chesseract, as shown in Table[12](https://arxiv.org/html/2607.07984#A1.T12)in Appendix\.

In particular, Unseen NAS classifies its tasks into Type\-1 and Type\-2: Type\-1 tasks are problems for which an expert could reasonably design a strong architecture using prior domain knowledge, while Type\-2 tasks are problems where effective architecture design without task\-specific tools would be nearly impossible for a human—and indeed, no competition participant correctly identified the domain of any Type\-2 dataset\. Language and Gutenberg are Type\-2\. The Isabella task, another Type\-2 task, is excluded because the data rights holder does not permit redistribution, as confirmed through personal communication with the dataset authors\. Full details on the benchmark subtasks are in Appendix[A\.6](https://arxiv.org/html/2607.07984#A1.SS6)\.

### 4\.2Baselines

![Refer to caption](https://arxiv.org/html/2607.07984v1/figures/spherical.png)Figure 2:Slotted architecture for theSphericaltask\. Module\-level slots such as*stacked DWP\-SE\-GELU*or*stacked MBConv\-SE\-SiLU*are replaceable, while additive glue slots provide optional normalizations or activation functions between modules\. The architecture discovered by NAS is unconventional yet substantially stronger\.We compare against baselines reported by the official benchmark papers and einspace\(Ericssonet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib30)\), grouping them by category\.

##### NAS\-Bench\-360\.

We compare with four baselines: \(i\) WideResNet\-16\-4\(Zagoruyko and Komodakis,[2016](https://arxiv.org/html/2607.07984#bib.bib62)\)as a fixed architecture baseline, \(ii\) GAEA DARTS, a differentiable NAS method, \(iii\) a task\-specific expert\-designed architecture provided by the benchmark, and \(iv\) einspace\(Ericssonet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib30)\)with regularized evolution starting from WideResNet\-16\-4, which represents the current state of the art on this suite\. Results for baselines \(i\)–\(iii\) are taken from the official NAS\-Bench\-360 evaluation\(Tuet al\.,[2022](https://arxiv.org/html/2607.07984#bib.bib43)\); einspace results are taken from its paper\(Ericssonet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib30)\)\.

##### Unseen NAS\.

We compare with three groups: \(i\) ResNet\-18\(Heet al\.,[2016](https://arxiv.org/html/2607.07984#bib.bib15)\)as a fixed architecture baseline, \(ii\) the per\-task best among six conventional NAS methods \(PC\-DARTS\(Xuet al\.,[2020](https://arxiv.org/html/2607.07984#bib.bib7)\), DrNAS\(Chenet al\.,[2021](https://arxiv.org/html/2607.07984#bib.bib63)\), BonsaiNet\(Geadaet al\.,[2020](https://arxiv.org/html/2607.07984#bib.bib24)\), random search on the DARTS and BonsaiNet search spaces, and regularized evolution on the hNB201 space\), and \(iii\) the per\-task best among all einspace variants \(random search, random sampling, and regularized evolution with various seed sets\)\. Full per\-method results are in Appendix[A\.5](https://arxiv.org/html/2607.07984#A1.SS5)\. Results for baselines \(i\)–\(ii\) are taken from the official Unseen NAS evaluation\(Geadaet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib44)\); einspace results are taken from its paper\(Ericssonet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib30)\)\.

### 4\.3Implementation Details

##### Choice of LLMs\.

All LLM roles \(Planner, Code Generator, Data Explorer, Slot Planner\) use Claude Sonnet 4\.6\(Anthropic,[2026](https://arxiv.org/html/2607.07984#bib.bib45)\)at temperature0\.70\.7, each with a separate context window\. The same model and prompt templates are used across all1717tasks without per\-task tuning\. Ablation studies in Section[5](https://arxiv.org/html/2607.07984#S5)additionally employ two models of varying capability: Claude Haiku 4\.5 and Claude Opus 4\.6\.

##### Compute\.

Each architecture is trained on a single RTX 2080 Ti GPU \(1111GB VRAM\)\. Pipeline budget details and runtime are documented in Appendix[A\.1](https://arxiv.org/html/2607.07984#A1.SS1)\.

##### Reporting\.

We use a*non\-seed*best selection rule: across Phase 1 and Phase 3, we select the architecture with the best validation metric and report its test score\. Since we initialize the population with the seed architecture, if NAS never improves the validation metric, in principle the seed becomes the output\. To unmask such a null\-NAS scenario, we report the second\-best architecture’s score in this case\.

### 4\.4Results

Table 1:NAS\-Bench\-360 results \(lower is better\)\. AgentNAS\-LLM is Phase 1 only; AgentNAS\-Full is the complete pipeline\.Bestandsecond bestper row\.TaskMetricWRN16\-4GAEADARTSExperteinspace\(WRN\)AgentNAS\-LLMAgentNAS\-FullCIFAR\-1001−acc1\-\\text\{acc\}\. \(%\)23\.3524\.0219\.3921\.4716\.7415\.50Spherical1−acc1\-\\text\{acc\}\. \(%\)85\.7748\.2367\.4166\.3745\.0838\.71NinaPro1−acc1\-\\text\{acc\}\. \(%\)6\.7817\.678\.736\.378\.358\.04FSD50K1−1\-mAP0\.920\.940\.620\.650\.560\.53Darcy FlowRel\. L20\.0730\.0260\.0080\.0140\.0050\.006PSICOVMAE83\.842\.943\.354\.383\.373\.27Cosmic1−1\-AUROC0\.2450\.2290\.1270\.7300\.0650\.051ECG1−1\-F10\.430\.340\.280\.460\.310\.27Satellite1−acc1\-\\text\{acc\}\. \(%\)15\.4912\.5119\.8012\.5511\.1110\.85DeepSea1−1\-AUROC0\.400\.360\.300\.360\.260\.28Average rank↓\\downarrow5\.004\.253\.604\.352\.301\.50

Table 2:Unseen NAS test errors \(%, lower is better\)\. NAS Best†: per\-task best among six conventional NAS methods; einspace Best‡: per\-task best among all einspace variants\. Full per\-method breakdown in Appendix[A\.5](https://arxiv.org/html/2607.07984#A1.SS5)\.Bestandsecond bestper row\.TaskRN18NASBest†einspaceBest‡AgentNAS\-LLMAgentNAS\-FullAddNIST6\.642\.092\.282\.822\.42Language7\.847\.572\.080\.850\.49MultNIST8\.641\.903\.633\.543\.53CIFARTile52\.877\.7237\.2411\.208\.48Gutenberg56\.6850\.8845\.9844\.6344\.17GeoClassing9\.923\.974\.691\.600\.97Chesseract40\.6531\.1738\.1433\.8431\.39Average error↓\\downarrow26\.1815\.0419\.1514\.0713\.06Average rank↓\\downarrow5\.002\.143\.432\.711\.71Tables[1](https://arxiv.org/html/2607.07984#S4.T1)and[2](https://arxiv.org/html/2607.07984#S4.T2)summarize our main results on NAS\-Bench\-360 and Unseen NAS, respectively\.

##### Overall performance\.

AgentNAS establishes a new state of the art on 11 out of 17 tasks across both benchmarks\. On NAS\-Bench\-360, it achieves the best result on 8 of 10 tasks, with an average rank of 1\.5 compared to3\.603\.60for the next\-best baseline \(Expert\)\. On Unseen NAS—where the pipeline operates under the same blind setting as the original competition participants—AgentNAS achieves the lowest average error13\.06%13\.06\\%vs\.15\.04%15\.04\\%for the per\-task\-best conventional NAS method\.

##### LLM seed strength\.

The LLM\-only Phase 1 seed, before any NAS, already matches or exceeds all published baselines on the majority of tasks\. This demonstrates that an LLM with access to a sandboxed training environment can serve as a strong architecture designer, producing competitive networks without manual intervention or predefined search spaces\.

##### Additive value of NAS\.

Phase 3 NAS delivers additional gains on top of the seed on the majority of tasks—most notably Spherical \(45\.08→38\.7145\.08\\\!\\to\\\!38\.71\), CIFARTile \(11\.20→8\.4811\.20\\\!\\to\\\!8\.48\), and Chesseract \(33\.84→31\.3933\.84\\\!\\to\\\!31\.39\)\. On a smaller number of tasks, the seed is already near the performance ceiling and NAS matches but does not improve upon it \(e\.g\., Darcy Flow, GeoClassing\)\. Section[5](https://arxiv.org/html/2607.07984#S5)examines the division of labor between these two phases in detail\.

##### Limitations\.

AgentNAS underperforms the per\-task\-best baseline on 6 tasks: NinaPro \(einspace\), PSICOV \(GAEA DARTS\), AddNIST \(Bonsai\-Net\), MultNIST \(DrNAS\), CIFARTile \(PC\-DARTS\), and Chesseract \(random search on Bonsai\-Net space\)\. These gaps indicate that LLM\-generated search spaces do not universally dominate hand\-designed ones, and that certain task–search\-space pairings in conventional NAS remain effective\.

## 5Analysis

To understand the contribution of each component, we conduct ablation studies on six representative tasks—Spherical, Chesseract, Language, NinaPro, CIFARTile, and Darcy Flow—selected to cover both benchmark suites, multiple modalities, and a range of NAS improvement magnitudes\.

### 5\.1LLM Seed Search vs\. Fixed\-Seed NAS

Table 3:Fixed\-seed ablation \(lower is better\)\. The LLM\-driven Phase 1 is replaced with a fixed baseline \(WRN16\-4 or ResNet\-18\)\.Bestandsecond bestper row\. Darcy Flow is OOM for the fixed seed on our hardware constraint of 11GB VRAM; its value is from Table[1](https://arxiv.org/html/2607.07984#S4.T1)\. We could not reproduce Fixed Seed experimental results, so we report both\. This discrepancy is also reported in einspace\(Ericssonet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib30)\), particularly regarding WRN16\-4 for NAS\-Bench\-360\. We note that einspace searched 500 architectures, while both our Fixed Seed \+ NAS and AgentNAS\-LLM only searches for 160 architectures\.TaskFixed Seed\(einspace\)Fixed SeedeinspaceFixed Seed\+ NASAgentNAS\-LLMSpherical76\.3271\.6566\.3753\.5145\.08Ninapro10\.327\.136\.376\.688\.35Darcy Flow0\.0320\.0730\.014OOM0\.005Language7\.847\.403\.163\.350\.85CIFARTile55\.8852\.8739\.3521\.9911\.20Chesseract44\.4640\.6539\.6940\.1933\.84

AgentNAS combines two sources of strength: the LLM produces a high\-quality seed \(Phase 1\), and NAS refines it within an automatically constructed search space \(Phases 2–3\)\. To disentangle these contributions, we compare three configurations in Table[3](https://arxiv.org/html/2607.07984#S5.T3): \(i\) a fixed published architecture \(WideResNet\-16\-4 or ResNet\-18\) refined by Phase 2–3, \(ii\) the same architecture refined by einspace\(Ericssonet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib30)\)with regularized evolution over 500 evaluations, and \(iii\) the full AgentNAS pipeline\.

##### LLM seed search is more budget\-efficient than fixed\-seed NAS\.

The full pipeline outperforms the fixed\-seed variant on every completed task, often by a wide margin\. This confirms that the LLM’s ability to design a task\-adapted seed provides a stronger starting point than beginning NAS from a generic architecture\.

##### Phase 2 still constructs a strong search space\.

Even with a fixed seed, Phase 2–3 with 160 evaluations outperforms einspace with 500 evaluations on Spherical and CIFARTile\. This suggests that the LLM\-constructed slotted space is itself more effective than the grammar\-based space of einspace, independent of seed quality\. Combined, these two results justify the pipeline’s design: the LLM contributes a strong seed*and*a strong search space, and NAS exploits the latter to improve the former\.

### 5\.2NAS Complements LLM Sampling

![Refer to caption](https://arxiv.org/html/2607.07984v1/figures/ablation.png)Figure 3:Budget–performance trajectories \(lower is better\)\.Solid: LLM\-only search;dashed: LLM \+ NAS\. Each point reports the best non\-seed architecture\. Colors: Sonnet 4\.6 \(blue\), Opus 4\.6 \(pink\), Haiku 4\.5 \(green\)\. On Darcyflow, Haiku, starting from a 320\-budget architecture, created degenerate search spaces that could not pass our validation gate even with multiple trials, so we report the NAS result from a 160\-budget architecture instead, while Haiku keeps improving until 480 budget solely with LLM sampling\.LLM sampling produces independent proposals and saturates quickly, whereas NAS combinatorially recombines modules across slots, exploring interactions that independent samples cannot\. To quantify this complementarity, Figure[3](https://arxiv.org/html/2607.07984#S5.F3)compares two budget\-allocation strategies\. Starting from a 160\-evaluation LLM search, we iteratively invest 160 additional evaluations in further LLM sampling as long as it keeps improving, up to a total budget of 480, and run NAS at the point where the LLM can no longer improve\.

##### LLM sampling saturates; NAS continues to improve\.

On the majority of tasks, LLM sampling saturates within 480 evaluations across all three LLMs, while NAS delivers additional gains beyond the saturation point\. On*Spherical*, all models saturate at 160 while NAS continues to improve for Sonnet\. On*Chesseract*, saturation occurs at 160–480 depending on the model, but NAS consistently yields further improvement\. On*Language*, Sonnet benefits from both mechanisms simultaneously, with LLM\-320\+NAS slightly outperforming LLM\-480\.

##### Edge cases\.

On*CIFARTile*, Opus already reaches a strong level at 160 evaluations and neither further LLM sampling nor NAS improves upon it, while Sonnet and Haiku still benefit from NAS\. On*Darcy Flow*, Sonnet saturates early and NAS fails to improve, whereas Opus benefits from both continued LLM sampling and NAS—a difference we trace to the quality of the search space in Section[5\.3](https://arxiv.org/html/2607.07984#S5.SS3)\.

##### Consistency across LLMs\.

The complementarity of NAS is robust across model families and capability levels\. While seed quality scales with model capability—Haiku consistently starts at a worse level—the pattern that NAS adds value after LLM saturation holds in most configurations\. The per\-task regimes \(NAS improves, both improve, neither improves\) are largely consistent across LLMs, with Darcy Flow being a notable exception where search space quality, rather than seed quality alone, determines NAS success\.

##### Effect of the search algorithm\.

We additionally compare regularized evolution, random search, and GDAS within the same slotted space\. The differences between search algorithms are modest \(full results in Appendix[A\.3](https://arxiv.org/html/2607.07984#A1.SS3)\)\. We note, however, that we have not optimized the hyperparameters of each search algorithm, so future work may reveal larger differences through tuning\.

##### Effect of random seeds\.

Across three random seeds \(\{42,43,44\}\\\{42,43,44\\\}\), AgentNAS\-Full exhibits moderate variance \(standard deviations from0\.380\.38to2\.102\.10percentage points; full results in Appendix[A\.4](https://arxiv.org/html/2607.07984#A1.SS4)\), with all seeds remaining competitive against published baselines\.

### 5\.3What Does NAS Change?

The preceding sections establish*that*NAS improves upon the LLM seed on the majority of tasks\. Here we examine*how*, focusing on three tasks that illustrate distinct regimes\.

##### Chesseract: functional compensation\.

Chesseract is a three\-class prediction task \(win/draw/loss\) over 8×\\times8 chess board states encoded as 12\-channel inputs \(2 players×\\times6 piece types\)\. The three LLMs produce structurally diverse seeds—a chess\-specific architecture with piece\-type branches \(Sonnet\), a generic Transformer \(Opus\), and a CNN backbone \(Haiku\)\. Across all three, NAS exhibits a consistent*compensatory*pattern: it adds capabilities the seed lacks while leaving already\-effective components intact\. This is clearest in*gating*\. The Transformer and CNN seeds lack dynamic feature control; NAS introduces it via ReZero scaling, sigmoid gates, and gated FFN variants \(GLU/SwiGLU\)\. The Sonnet seed already contains piece\-type\-specific sigmoid gates; NAS reorganizes them, removing per\-branch gates and placing a consolidated gated composition at the merge point\. In*spatial mixing*, the global\-only Transformer seed receives depthwise 3×\\times3 convolutions for explicit local mixing over the 8×\\times8 board, while the local\-only CNN seed is refined but not augmented with global attention\. NAS also selectively prunes redundant components—removing skip connections or attention layers in stages where they contribute little\.

##### CIFARTile: consistent channel attention\.

CIFARTile presents 2×\\times2 tiled CIFAR\-100 images as a 4\-class classification task\. Across three seeds \(RegNet, ResNet, Bottleneck variants\), NAS consistently strengthens channel attention: adding SE blocks where none existed, or layering complementary mechanisms \(Coordinate Attention, ECA\) on top of existing SE modules\. This pattern—*if absent, add it; if present, diversify it*—suggests that channel recalibration is uniformly beneficial for this task, where discriminative signals may be concentrated in specific spatial tiles\.

##### Darcy Flow: search space quality determines NAS success\.

Darcy Flow is dense regression over a PDE: given a permeability field, predict the pressure field governed by an elliptic equation\. All three LLMs produce Fourier Neural Operator \(FNO\) seeds, but NAS outcomes diverge sharply: Sonnet’s NAS fails to improve, while Opus’s succeeds\.

The difference traces to how each LLM constructs the search space\. Sonnet’s search space retains the seed’s einsum\-based channel\-mixing spectral convolution \(bimw,iom\-\>bomw\) across all slot alternatives, leaving NAS with no meaningful variation in the spectral layers\. Opus, by contrast, includes an alternative that replaces channel\-mixing spectral convolution with element\-wise spectral filtering, where each channel’s frequency components are independently scaled by a lightweight\(channel, modes\)\-shaped weight\. This factorization separates spectral processing from channel mixing—the latter is handled by dedicated1×11\\\!\\times\\\!1convolutions—yielding a simpler, more parameter\-efficient design that NAS selects\.

This result highlights that the LLM’s contribution extends beyond seed quality to search space design: a more capable LLM can populate slots with structurally diverse, physically meaningful alternatives that give NAS room to discover effective recombinations\.

## 6Conclusion

We proposed a mechanism by which an LLM decomposes a seed architecture into a*slotted architecture*, automatically constructing a task\-specific search space that conventional NAS can explore without manual engineering\. Instantiated in AgentNAS, this approach establishes a new state of the art on 11 of 17 diverse tasks, demonstrating that LLM\-driven design and NAS\-driven search are broadly complementary: the LLM provides a strong seed and search space, while NAS discovers additional gains through combinatorial recombination that independent LLM samples cannot replicate\.

##### Limitations\.

Our evaluation is subject to several constraints\. First, hardware limitations \(8×8\\timesRTX 2080 Ti, 11 GB VRAM\) and per\-candidate time caps impose an implicit bias toward smaller architectures\. Second, evaluation budgets are not perfectly controlled: both LLM and NAS searches terminate either when a patience criterion is met or at a 160\-evaluation cap, and the LLM may stop early, so actual budgets vary across tasks and models\. Third, validation\-based selection can misalign with test performance, as observed on DeepSea where NAS improves validation metrics but regresses on test\. Fourth, weaker LLMs \(e\.g\., Haiku\) occasionally fail to construct viable slotted architectures, indicating a minimum capability threshold\.

##### Future work\.

Three directions are most promising\. The question of what makes a good LLM\-constructed search space—and how to systematically improve it—remains largely unexplored\. Additionally, our search algorithms \(regularized evolution, GDAS\) and their hyperparameters are not optimized; exploring alternatives, especially efficient weight\-sharing methods, within the slotted framework is a natural next step\. Finally, extending the pipeline to generative tasks would test its further generality, though this likely requires relaxing the current hardware and time constraints\.

## Acknowledgements

This work was supported by Institute of Information & Communications Technology Planning & Evaluation \(IITP\) grant funded by the Korea government \(MSIT\) \(No\. IITP\-2026\-RS\-2024\-00360227, Leading Generative AI Human Resources Development, No\. RS\-2025\-25442824, AI Star Fellowship Program \(Ulsan National Institute of Science and Technology\), & No\. RS\-2020\-II201336, Artificial Intelligence graduate school support \(UNIST\)\)\.

## Impact Statement

This paper presents work whose goal is to advance the field of Machine Learning\. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here\.

## References

- B\. Adhikari \(2020\)A fully open\-source framework for deep learning protein real\-valued distances\.Scientific reports10\(1\),pp\. 13374\.Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px6.p1.1)\.
- Anthropic \(2026\)Introducing Claude Sonnet 4\.6\.Note:Accessed: 2026\-04\-28External Links:[Link](https://www.anthropic.com/news/claude-sonnet-4-6)Cited by:[§4\.3](https://arxiv.org/html/2607.07984#S4.SS3.SSS0.Px1.p1.2)\.
- M\. Atzori, A\. Gijsberts, S\. Heynen, A\. M\. Hager, O\. Deriaz, P\. Van Der Smagt, C\. Castellini, B\. Caputo, and H\. Müller \(2012\)Building the ninapro database: a resource for the biorobotics community\.In2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics \(BioRob\),pp\. 1258–1265\.Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px3.p1.1)\.
- G\. Bender, P\. Kindermans, B\. Zoph, V\. Vasudevan, and Q\. V\. Le \(2018\)Understanding and simplifying one\-shot architecture search\.InProceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10\-15, 2018,J\. G\. Dy and A\. Krause \(Eds\.\),Proceedings of Machine Learning Research, Vol\.80,pp\. 549–558\.External Links:[Link](http://proceedings.mlr.press/v80/bender18a.html)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p1.1)\.
- G\. Bender, H\. Liu, B\. Chen, G\. Chu, S\. Cheng, P\. Kindermans, and Q\. V\. Le \(2020\)Can weight sharing outperform random architecture search? an investigation with tunas\.In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13\-19, 2020,pp\. 14311–14320\.External Links:[Link](https://openaccess.thecvf.com/content%5C_CVPR%5C_2020/papers/Bender%5C_Can%5C_Weight%5C_Sharing%5C_Outperform%5C_Random%5C_Architecture%5C_Search%5C_An%5C_Investigation%5C_With%5C_CVPR%5C_2020%5C_paper.pdf),[Document](https://dx.doi.org/10.1109/CVPR42600.2020.01433)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p1.1)\.
- H\. Cai, C\. Gan, T\. Wang, Z\. Zhang, and S\. Han \(2020\)Once\-for\-all: train one network and specialize it for efficient deployment\.In8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\-30, 2020,External Links:[Link](https://openreview.net/forum?id=HylxE1HKwS)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p1.1)\.
- H\. Cai, L\. Zhu, and S\. Han \(2019\)ProxylessNAS: direct neural architecture search on target task and hardware\.In7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6\-9, 2019,External Links:[Link](https://openreview.net/forum?id=HylVB3AqYm)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p1.1)\.
- A\. Chen, D\. Dohan, and D\. R\. So \(2023\)EvoPrompting: language models for code\-level neural architecture search\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper%5C_files/paper/2023/hash/184c1e18d00d7752805324da48ad25be-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- X\. Chen, R\. Wang, M\. Cheng, X\. Tang, and C\. Hsieh \(2021\)DrNAS: dirichlet neural architecture search\.In9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3\-7, 2021,External Links:[Link](https://openreview.net/forum?id=9FWas6YbmB3)Cited by:[§4\.2](https://arxiv.org/html/2607.07984#S4.SS2.SSS0.Px2.p1.1)\.
- G\. D\. Clifford, C\. Liu, B\. Moody, L\. H\. Lehman, I\. Silva, Q\. Li, A\. E\. W\. Johnson, and R\. G\. Mark \(2017\)AF classification from a short single lead ECG recording: the physionet computing in cardiology challenge 2017\.InComputing in Cardiology, CinC 2007, Rennes, France, September 24\-27, 2017,External Links:[Link](https://doi.org/10.22489/CinC.2017.065-469),[Document](https://dx.doi.org/10.22489/CINC.2017.065-469)Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px8.p1.1)\.
- T\. S\. Cohen, M\. Geiger, J\. Köhler, and M\. Welling \(2018\)Spherical cnns\.In6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 \- May 3, 2018, Conference Track Proceedings,External Links:[Link](https://openreview.net/forum?id=Hkbd5xZRb)Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px2.p1.1)\.
- U\. Côté\-Allard, C\. L\. Fall, A\. Drouin, A\. Campeau\-Lecours, C\. Gosselin, K\. Glette, F\. Laviolette, and B\. Gosselin \(2019\)Deep learning for electromyographic hand gesture signal classification using transfer learning\.IEEE transactions on neural systems and rehabilitation engineering27\(4\),pp\. 760–771\.Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px3.p1.1)\.
- J\. Deng, W\. Dong, R\. Socher, L\. Li, K\. Li, and L\. Fei\-Fei \(2009\)ImageNet: A large\-scale hierarchical image database\.In2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition \(CVPR 2009\), 20\-25 June 2009, Miami, Florida, USA,pp\. 248–255\.External Links:[Link](https://doi.org/10.1109/CVPR.2009.5206848),[Document](https://dx.doi.org/10.1109/CVPR.2009.5206848)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- X\. Dong, L\. Liu, K\. Musial, and B\. Gabrys \(2022\)NATS\-bench: benchmarking NAS algorithms for architecture topology and size\.IEEE Trans\. Pattern Anal\. Mach\. Intell\.44\(7\),pp\. 3634–3646\.External Links:[Link](https://doi.org/10.1109/TPAMI.2021.3054824),[Document](https://dx.doi.org/10.1109/TPAMI.2021.3054824)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- X\. Dong and Y\. Yang \(2019\)Searching for a robust neural architecture in four GPU hours\.InIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16\-20, 2019,pp\. 1761–1770\.External Links:[Link](http://openaccess.thecvf.com/content%5C_CVPR%5C_2019/html/Dong%5C_Searching%5C_for%5C_a%5C_Robust%5C_Neural%5C_Architecture%5C_in%5C_Four%5C_GPU%5C_Hours%5C_CVPR%5C_2019%5C_paper.html),[Document](https://dx.doi.org/10.1109/CVPR.2019.00186)Cited by:[§A\.3](https://arxiv.org/html/2607.07984#A1.SS3.p1.1),[Table 9](https://arxiv.org/html/2607.07984#A1.T9),[Table 9](https://arxiv.org/html/2607.07984#A1.T9.5.2),[§3\.3](https://arxiv.org/html/2607.07984#S3.SS3.p1.1)\.
- X\. Dong and Y\. Yang \(2020\)NAS\-bench\-201: extending the scope of reproducible neural architecture search\.In8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\-30, 2020,External Links:[Link](https://openreview.net/forum?id=HJxyZkBKDr)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- T\. Elsken, J\. H\. Metzen, and F\. Hutter \(2019\)Neural architecture search: A survey\.J\. Mach\. Learn\. Res\.20,pp\. 55:1–55:21\.External Links:[Link](https://jmlr.org/papers/v20/18-598.html)Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1)\.
- L\. Ericsson, M\. Espinosa, C\. Yang, A\. Antoniou, A\. Storkey, S\. B\. Cohen, S\. McDonagh, and E\. J\. Crowley \(2024\)Einspace: searching for neural architectures from fundamental operations\.InAdvances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 \- 15, 2024,A\. Globersons, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. M\. Tomczak, and C\. Zhang \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper%5C_files/paper/2024/hash/0382cb76309820f71c6eacd47b36ce71-Abstract-Conference.html)Cited by:[§A\.2](https://arxiv.org/html/2607.07984#A1.SS2.p1.1),[Table 11](https://arxiv.org/html/2607.07984#A1.T11),[Table 11](https://arxiv.org/html/2607.07984#A1.T11.10.2),[Table 4](https://arxiv.org/html/2607.07984#A1.T4),[Table 4](https://arxiv.org/html/2607.07984#A1.T4.2.1),[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p3.1),[§3](https://arxiv.org/html/2607.07984#S3.p1.1),[§4\.2](https://arxiv.org/html/2607.07984#S4.SS2.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2607.07984#S4.SS2.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2607.07984#S4.SS2.p1.1),[§5\.1](https://arxiv.org/html/2607.07984#S5.SS1.p1.1),[Table 3](https://arxiv.org/html/2607.07984#S5.T3),[Table 3](https://arxiv.org/html/2607.07984#S5.T3.5.2)\.
- E\. Feingold, P\. Good, M\. Guyer, S\. Kamholz, L\. Liefer, K\. Wetterstrand, F\. Collins, T\. Gingeras, D\. Kampa, E\. Sekinger,et al\.\(2004\)The encode \(encyclopedia of dna elements\) project\.Science306\(5696\),pp\. 636–640\.Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px10.p1.1)\.
- E\. Fonseca, X\. Favory, J\. Pons, F\. Font, and X\. Serra \(2022\)FSD50K: an open dataset of human\-labeled sound events\.IEEE ACM Trans\. Audio Speech Lang\. Process\.30,pp\. 829–852\.External Links:[Link](https://doi.org/10.1109/TASLP.2021.3133208),[Document](https://dx.doi.org/10.1109/TASLP.2021.3133208)Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px4.p1.1)\.
- C\. Gao, Y\. Chen, S\. Liu, Z\. Tan, and S\. Yan \(2020\)AdversarialNAS: adversarial neural architecture search for gans\.In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13\-19, 2020,pp\. 5679–5688\.External Links:[Link](https://openaccess.thecvf.com/content%5C_CVPR%5C_2020/html/Gao%5C_AdversarialNAS%5C_Adversarial%5C_Neural%5C_Architecture%5C_Search%5C_for%5C_GANs%5C_CVPR%5C_2020%5C_paper.html),[Document](https://dx.doi.org/10.1109/CVPR42600.2020.00572)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- Y\. Gao, H\. Yang, P\. Zhang, C\. Zhou, and Y\. Hu \(2019\)GraphNAS: graph neural architecture search with reinforcement learning\.CoRRabs/1904\.09981\.External Links:[Link](http://arxiv.org/abs/1904.09981),1904\.09981Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- R\. Geada, D\. Prangle, and A\. S\. McGough \(2020\)Bonsai\-net: one\-shot neural architecture search via differentiable pruners\.CoRRabs/2006\.09264\.External Links:[Link](https://arxiv.org/abs/2006.09264),2006\.09264Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p3.1),[§4\.2](https://arxiv.org/html/2607.07984#S4.SS2.SSS0.Px2.p1.1)\.
- R\. Geada, D\. Towers, M\. Forshaw, A\. Atapour\-Abarghouei, and A\. S\. McGough \(2024\)Insights from the use of previously unseen neural architecture search datasets\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16\-22, 2024,pp\. 22541–22550\.External Links:[Link](https://doi.org/10.1109/CVPR52733.2024.02127),[Document](https://dx.doi.org/10.1109/CVPR52733.2024.02127)Cited by:[§A\.6\.2](https://arxiv.org/html/2607.07984#A1.SS6.SSS2.p1.1),[Table 11](https://arxiv.org/html/2607.07984#A1.T11),[Table 11](https://arxiv.org/html/2607.07984#A1.T11.10.2),[Table 4](https://arxiv.org/html/2607.07984#A1.T4),[Table 4](https://arxiv.org/html/2607.07984#A1.T4.2.1),[§1](https://arxiv.org/html/2607.07984#S1.p2.1),[§4\.1](https://arxiv.org/html/2607.07984#S4.SS1.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2607.07984#S4.SS2.SSS0.Px2.p1.1)\.
- J\. F\. Gemmeke, D\. P\. W\. Ellis, D\. Freedman, A\. Jansen, W\. Lawrence, R\. C\. Moore, M\. Plakal, and M\. Ritter \(2017\)Audio set: an ontology and human\-labeled dataset for audio events\.In2017 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2017, New Orleans, LA, USA, March 5\-9, 2017,pp\. 776–780\.External Links:[Link](https://doi.org/10.1109/ICASSP.2017.7952261),[Document](https://dx.doi.org/10.1109/ICASSP.2017.7952261)Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px4.p1.1)\.
- X\. Gong, S\. Chang, Y\. Jiang, and Z\. Wang \(2019\)AutoGAN: neural architecture search for generative adversarial networks\.In2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea \(South\), October 27 \- November 2, 2019,pp\. 3223–3233\.External Links:[Link](https://doi.org/10.1109/ICCV.2019.00332),[Document](https://dx.doi.org/10.1109/ICCV.2019.00332)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- K\. He, X\. Zhang, S\. Ren, and J\. Sun \(2016\)Deep residual learning for image recognition\.In2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27\-30, 2016,pp\. 770–778\.External Links:[Link](https://doi.org/10.1109/CVPR.2016.90),[Document](https://dx.doi.org/10.1109/CVPR.2016.90)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1),[§4\.2](https://arxiv.org/html/2607.07984#S4.SS2.SSS0.Px2.p1.1)\.
- Y\. Hu, J\. Liu, K\. Wang, J\. Zheng, W\. Shi, M\. Zhang, Q\. Dou, R\. Liu, A\. Zhou, and H\. Li \(2025\)LM\-searcher: cross\-domain neural architecture search with llms via unified numerical encoding\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, EMNLP 2025, Suzhou, China, November 4\-9, 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),pp\. 9408–9421\.External Links:[Link](https://doi.org/10.18653/v1/2025.emnlp-main.478),[Document](https://dx.doi.org/10.18653/V1/2025.EMNLP-MAIN.478)Cited by:[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- Q\. Jiang and G\. Karniadakis \(2026\)Agenticsciml: collaborative multi\-agent systems for emergent discovery in scientific machine learning\.npj Artificial Intelligence\.Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- A\. Krizhevsky, G\. Hinton,et al\.\(2009\)Learning multiple layers of features from tiny images\.Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px1.p1.1),[§A\.6\.2](https://arxiv.org/html/2607.07984#A1.SS6.SSS2.Px4.p1.1),[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- Y\. LeCun, C\. Cortes, and C\. Burges \(2010\)MNIST handwritten digit database\.ATT Labs \[Online\]\. Available: http://yann\.lecun\.com/exdb/mnist2\.Cited by:[§A\.6\.2](https://arxiv.org/html/2607.07984#A1.SS6.SSS2.Px1.p1.2)\.
- C\. Li, T\. Tang, G\. Wang, J\. Peng, B\. Wang, X\. Liang, and X\. Chang \(2021a\)BossNAS: exploring hybrid cnn\-transformers with block\-wisely self\-supervised neural architecture search\.In2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10\-17, 2021,pp\. 12261–12271\.External Links:[Link](https://doi.org/10.1109/ICCV48922.2021.01206),[Document](https://dx.doi.org/10.1109/ICCV48922.2021.01206)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- L\. Li and A\. Talwalkar \(2019\)Random search and reproducibility for neural architecture search\.InProceedings of the Thirty\-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, Tel Aviv, Israel, July 22\-25, 2019,A\. Globerson and R\. Silva \(Eds\.\),Proceedings of Machine Learning Research, Vol\.115,pp\. 367–377\.External Links:[Link](http://proceedings.mlr.press/v115/li20c.html)Cited by:[§A\.3](https://arxiv.org/html/2607.07984#A1.SS3.p1.1),[Table 9](https://arxiv.org/html/2607.07984#A1.T9),[Table 9](https://arxiv.org/html/2607.07984#A1.T9.5.2),[§3\.3](https://arxiv.org/html/2607.07984#S3.SS3.p1.1)\.
- Y\. Li and I\. King \(2020\)AutoGraph: automated graph neural network\.InNeural Information Processing \- 27th International Conference, ICONIP 2020, Bangkok, Thailand, November 23\-27, 2020, Proceedings, Part II,H\. Yang, K\. Pasupa, A\. C\. Leung, J\. T\. Kwok, J\. H\. Chan, and I\. King \(Eds\.\),Lecture Notes in Computer Science, Vol\.12533,pp\. 189–201\.External Links:[Link](https://doi.org/10.1007/978-3-030-63833-7%5C_16),[Document](https://dx.doi.org/10.1007/978-3-030-63833-7%5F16)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- Z\. Li, N\. B\. Kovachki, K\. Azizzadenesheli, B\. Liu, K\. Bhattacharya, A\. M\. Stuart, and A\. Anandkumar \(2021b\)Fourier neural operator for parametric partial differential equations\.In9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3\-7, 2021,External Links:[Link](https://openreview.net/forum?id=c8P9NQVtmnO)Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px5.p1.1)\.
- C\. Liu, L\. Chen, F\. Schroff, H\. Adam, W\. Hua, A\. L\. Yuille, and L\. Fei\-Fei \(2019a\)Auto\-deeplab: hierarchical neural architecture search for semantic image segmentation\.InIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16\-20, 2019,pp\. 82–92\.External Links:[Link](http://openaccess.thecvf.com/content%5C_CVPR%5C_2019/html/Liu%5C_Auto-DeepLab%5C_Hierarchical%5C_Neural%5C_Architecture%5C_Search%5C_for%5C_Semantic%5C_Image%5C_Segmentation%5C_CVPR%5C_2019%5C_paper.html),[Document](https://dx.doi.org/10.1109/CVPR.2019.00017)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p3.1)\.
- H\. Liu, K\. Simonyan, O\. Vinyals, C\. Fernando, and K\. Kavukcuoglu \(2018\)Hierarchical representations for efficient architecture search\.In6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 \- May 3, 2018, Conference Track Proceedings,External Links:[Link](https://openreview.net/forum?id=BJQRKzbA-)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p3.1)\.
- H\. Liu, K\. Simonyan, and Y\. Yang \(2019b\)DARTS: differentiable architecture search\.In7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6\-9, 2019,External Links:[Link](https://openreview.net/forum?id=S1eYHoC5FX)Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p1.1),[§3\.2](https://arxiv.org/html/2607.07984#S3.SS2.p2.1),[§3\.3](https://arxiv.org/html/2607.07984#S3.SS3.p1.1)\.
- S\. Liu, C\. Gao, and Y\. Li \(2025\)AgentHPO: large language model agent for hyper\-parameter optimization\.InConference on Parsimony and Learning, Stanford University, USA, 24\-27 March 2025,B\. Chen, S\. Liu, M\. Pilanci, W\. Su, J\. Sulam, Y\. Wang, and Z\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.280,pp\. 1146–1169\.External Links:[Link](https://proceedings.mlr.press/v280/liu25c.html)Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- C\. Lu, C\. Lu, R\. T\. Lange, J\. N\. Foerster, J\. Clune, and D\. Ha \(2024\)The AI scientist: towards fully automated open\-ended scientific discovery\.CoRRabs/2408\.06292\.External Links:[Link](https://doi.org/10.48550/arXiv.2408.06292),[Document](https://dx.doi.org/10.48550/ARXIV.2408.06292),2408\.06292Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- Z\. Lu, I\. Whalen, V\. Boddeti, Y\. D\. Dhebar, K\. Deb, E\. D\. Goodman, and W\. Banzhaf \(2019\)NSGA\-net: neural architecture search using multi\-objective genetic algorithm\.InProceedings of the Genetic and Evolutionary Computation Conference, GECCO 2019, Prague, Czech Republic, July 13\-17, 2019,A\. Auger and T\. Stützle \(Eds\.\),pp\. 419–427\.External Links:[Link](https://doi.org/10.1145/3321707.3321729),[Document](https://dx.doi.org/10.1145/3321707.3321729)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p1.1)\.
- C\. Morris, M\. Jurado, and J\. Zutty \(2024\)LLM guided evolution \- the automation of models advancing models\.InProceedings of the Genetic and Evolutionary Computation Conference, GECCO 2024, Melbourne, VIC, Australia, July 14\-18, 2024,X\. Li and J\. Handl \(Eds\.\),External Links:[Link](https://doi.org/10.1145/3638529.3654178),[Document](https://dx.doi.org/10.1145/3638529.3654178)Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- M\. U\. Nasir, S\. Earle, J\. Togelius, S\. James, and C\. W\. Cleghorn \(2024\)LLMatic: neural architecture search via large language models and quality diversity optimization\.InProceedings of the Genetic and Evolutionary Computation Conference, GECCO 2024, Melbourne, VIC, Australia, July 14\-18, 2024,X\. Li and J\. Handl \(Eds\.\),External Links:[Link](https://doi.org/10.1145/3638529.3654017),[Document](https://dx.doi.org/10.1145/3638529.3654017)Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- A\. Novikov, N\. Vu, M\. Eisenberger, E\. Dupont, P\. Huang, A\. Z\. Wagner, S\. Shirobokov, B\. Kozlovskii, F\. J\. R\. Ruiz, A\. Mehrabian, M\. P\. Kumar, A\. See, S\. Chaudhuri, G\. Holland, A\. Davies, S\. Nowozin, P\. Kohli, and M\. Balog \(2025\)AlphaEvolve: A coding agent for scientific and algorithmic discovery\.CoRRabs/2506\.13131\.External Links:[Link](https://doi.org/10.48550/arXiv.2506.13131),[Document](https://dx.doi.org/10.48550/ARXIV.2506.13131),2506\.13131Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- F\. Petitjean, J\. Inglada, and P\. Gançarski \(2012\)Satellite image time series analysis under time warping\.IEEE Trans\. Geosci\. Remote\. Sens\.50\(8\),pp\. 3081–3095\.External Links:[Link](https://doi.org/10.1109/TGRS.2011.2179050),[Document](https://dx.doi.org/10.1109/TGRS.2011.2179050)Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px9.p1.1)\.
- H\. Pham, M\. Y\. Guan, B\. Zoph, Q\. V\. Le, and J\. Dean \(2018\)Efficient neural architecture search via parameter sharing\.InProceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10\-15, 2018,J\. G\. Dy and A\. Krause \(Eds\.\),Proceedings of Machine Learning Research, Vol\.80,pp\. 4092–4101\.External Links:[Link](http://proceedings.mlr.press/v80/pham18a.html)Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p1.1)\.
- O\. Press, B\. Amos, H\. Zhao, Y\. Wu, S\. K\. Ainsworth, D\. Krupke, P\. Kidger, T\. Sajed, B\. Stellato, J\. Park, N\. Bosch, E\. Meril, A\. Steppi, A\. Zharmagambetov, F\. Zhang, D\. Pérez\-Piñeiro, A\. Mercurio, N\. Zhan, T\. Abramovich, K\. Lieret, H\. Zhang, S\. Huang, M\. Bethge, and O\. Press \(2025\)AlgoTune: can language models speed up general\-purpose numerical programs?\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diago, CA, USA, December 2\-7, 2025 / Mexico City, Mexico, November 30 \- December 5, 2025,D\. Belgrave, C\. Zhang, L\. N\. Montoya, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, N\. Chen, I\. V\. M\. Ruíz, and A\. Loaiza\-Bonilla \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper%5C_files/paper/2025/hash/9a648e8e1014c2427156dcb5465cd488-Abstract-Datasets%5C_and%5C_Benchmarks%5C_Track.html)Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- M\. H\. Rahman and P\. Chakraborty \(2024\)LeMo\-nade: multi\-parameter neural architecture discovery with llms\.CoRRabs/2402\.18443\.External Links:[Link](https://doi.org/10.48550/arXiv.2402.18443),[Document](https://dx.doi.org/10.48550/ARXIV.2402.18443),2402\.18443Cited by:[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- E\. Real, A\. Aggarwal, Y\. Huang, and Q\. V\. Le \(2019\)Regularized evolution for image classifier architecture search\.InThe Thirty\-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty\-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 \- February 1, 2019,pp\. 4780–4789\.External Links:[Link](https://doi.org/10.1609/aaai.v33i01.33014780),[Document](https://dx.doi.org/10.1609/AAAI.V33I01.33014780)Cited by:[§A\.3](https://arxiv.org/html/2607.07984#A1.SS3.p1.1),[Table 9](https://arxiv.org/html/2607.07984#A1.T9),[Table 9](https://arxiv.org/html/2607.07984#A1.T9.5.2),[§1](https://arxiv.org/html/2607.07984#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p1.1),[§3\.3](https://arxiv.org/html/2607.07984#S3.SS3.p1.1)\.
- E\. Real, C\. Liang, D\. R\. So, and Q\. V\. Le \(2020\)AutoML\-zero: evolving machine learning algorithms from scratch\.InProceedings of the 37th International Conference on Machine Learning, ICML 2020, 13\-18 July 2020, Virtual Event,Proceedings of Machine Learning Research, Vol\.119,pp\. 8007–8019\.External Links:[Link](http://proceedings.mlr.press/v119/real20a.html)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p3.1)\.
- E\. Real, S\. Moore, A\. Selle, S\. Saxena, Y\. I\. Leon\-Suematsu, J\. Tan, Q\. V\. Le, and A\. Kurakin \(2017\)Large\-scale evolution of image classifiers\.InProceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6\-11 August 2017,D\. Precup and Y\. W\. Teh \(Eds\.\),Proceedings of Machine Learning Research, Vol\.70,pp\. 2902–2911\.External Links:[Link](http://proceedings.mlr.press/v70/real17a.html)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p1.1)\.
- N\. Roberts, M\. Khodak, T\. Dao, L\. Li, C\. Ré, and A\. Talwalkar \(2021\)Rethinking neural operations for diverse tasks\.InAdvances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6\-14, 2021, virtual,M\. Ranzato, A\. Beygelzimer, Y\. N\. Dauphin, P\. Liang, and J\. W\. Vaughan \(Eds\.\),pp\. 15855–15869\.External Links:[Link](https://proceedings.neurips.cc/paper/2021/hash/84fdbc3ac902561c00871c9b0c226756-Abstract.html)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p3.1)\.
- B\. Romera\-Paredes, M\. Barekatain, A\. Novikov, M\. Balog, M\. P\. Kumar, E\. Dupont, F\. J\. R\. Ruiz, J\. S\. Ellenberg, P\. Wang, O\. Fawzi, P\. Kohli, and A\. Fawzi \(2024\)Mathematical discoveries from program search with large language models\.Nat\.625\(7995\),pp\. 468–475\.External Links:[Link](https://doi.org/10.1038/s41586-023-06924-6),[Document](https://dx.doi.org/10.1038/S41586-023-06924-6)Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- R\. Ru, P\. M\. Esperança, and F\. M\. Carlucci \(2020\)Neural architecture generator optimization\.InAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\-12, 2020, virtual,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. Balcan, and H\. Lin \(Eds\.\),External Links:[Link](https://proceedings.neurips.cc/paper/2020/hash/8c53d30ad023ce50140181f713059ddf-Abstract.html)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p3.1)\.
- S\. Schmidgall and M\. Moor \(2025\)AgentRxiv: towards collaborative autonomous research\.CoRRabs/2503\.18102\.External Links:[Link](https://doi.org/10.48550/arXiv.2503.18102),[Document](https://dx.doi.org/10.48550/ARXIV.2503.18102),2503\.18102Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- S\. Schmidgall, Y\. Su, Z\. Wang, X\. Sun, J\. Wu, X\. Yu, J\. Liu, M\. Moor, Z\. Liu, and E\. Barsoum \(2025\)Agent laboratory: using LLM agents as research assistants\.InFindings of the Association for Computational Linguistics: EMNLP 2025, Suzhou, China, November 4\-9, 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),pp\. 5977–6043\.External Links:[Link](https://doi.org/10.18653/v1/2025.findings-emnlp.320),[Document](https://dx.doi.org/10.18653/V1/2025.FINDINGS-EMNLP.320)Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- S\. Schrodi, D\. Stoll, B\. Ru, R\. S\. Sukthanker, T\. Brox, and F\. Hutter \(2023\)Construction of hierarchical neural architecture search spaces based on context\-free grammars\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper%5C_files/paper/2023/hash/4869f3f967dfe954439408dd92c50ee1-Abstract-Conference.html)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p3.1)\.
- D\. R\. So, Q\. V\. Le, and C\. Liang \(2019\)The evolved transformer\.InProceedings of the 36th International Conference on Machine Learning, ICML 2019, 9\-15 June 2019, Long Beach, California, USA,K\. Chaudhuri and R\. Salakhutdinov \(Eds\.\),Proceedings of Machine Learning Research, Vol\.97,pp\. 5877–5886\.External Links:[Link](http://proceedings.mlr.press/v97/so19a.html)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- Z\. Song and Z\. Jiang \(2025\)Adaptive fourier decomposition\-guided neural operator design for inverse pde problems\.External Links:[Link](https://openreview.net/forum?id=k5Z1tSidYY)Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- Z\. Song and Z\. Jiang \(2026\)Adaptive mamba neural operators\.External Links:[Link](https://openreview.net/forum?id=OenyzvFZPs)Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- Z\. Song, X\. Zhen, and Z\. Jiang \(2026\)Can large language models design effective neural operators for solving partial differential equations?\.In3rd AI for Math Workshop: Toward Self\-Evolving Scientific Agents,External Links:[Link](https://openreview.net/forum?id=6GCIuS0oO0&)Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- G\. Sumbul, M\. Charfuelan, B\. Demir, and V\. Markl \(2019\)Bigearthnet: A large\-scale benchmark archive for remote sensing image understanding\.In2019 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2019, Yokohama, Japan, July 28 \- August 2, 2019,pp\. 5901–5904\.External Links:[Link](https://doi.org/10.1109/IGARSS.2019.8900532),[Document](https://dx.doi.org/10.1109/IGARSS.2019.8900532)Cited by:[§A\.6\.2](https://arxiv.org/html/2607.07984#A1.SS6.SSS2.Px7.p1.1)\.
- M\. Tan, B\. Chen, R\. Pang, V\. Vasudevan, M\. Sandler, A\. Howard, and Q\. V\. Le \(2019\)MnasNet: platform\-aware neural architecture search for mobile\.InIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16\-20, 2019,pp\. 2820–2828\.External Links:[Link](http://openaccess.thecvf.com/content%5C_CVPR%5C_2019/html/Tan%5C_MnasNet%5C_Platform-Aware%5C_Neural%5C_Architecture%5C_Search%5C_for%5C_Mobile%5C_CVPR%5C_2019%5C_paper.html),[Document](https://dx.doi.org/10.1109/CVPR.2019.00293)Cited by:[§3\.2](https://arxiv.org/html/2607.07984#S3.SS2.p2.1)\.
- K\. Tiwari, N\. Dutta, N\. M\. A\. Krishnan, and P\. A\. P\. \(2025\)Latent mamba operator for partial differential equations\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267\.External Links:[Link](https://proceedings.mlr.press/v267/tiwari25a.html)Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- D\. Towers, R\. Geada, A\. Atapour\-Abarghouei, and A\. S\. McGough \(2023a\)AddNIST Dataset\.Newcastle University\(en\)\.Note:https://data\.ncl\.ac\.uk/articles/dataset/AddNIST\_Dataset/245 74354/1External Links:[Link](https://data.ncl.ac.uk/articles/dataset/AddNIST%5C_Dataset/24574354/1),[Document](https://dx.doi.org/10.25405/data.ncl.24574354.v1)Cited by:[§A\.6\.2](https://arxiv.org/html/2607.07984#A1.SS6.SSS2.Px1.p1.2)\.
- D\. Towers, R\. Geada, A\. Atapour\-Abarghouei, and A\. S\. McGough \(2023b\)Chesseract Dataset\.Newcastle University\(en\)\.Note:https://data\.ncl\.ac\.uk/articles/dataset/Chesseract\_Dataset/241 18743/2External Links:[Link](https://data.ncl.ac.uk/articles/dataset/Chesseract%5C_Dataset/24118743/2),[Document](https://dx.doi.org/10.25405/data.ncl.24118743.v2)Cited by:[§A\.6\.2](https://arxiv.org/html/2607.07984#A1.SS6.SSS2.Px8.p1.1)\.
- D\. Towers, R\. Geada, A\. Atapour\-Abarghouei, and A\. S\. McGough \(2023c\)CIFARTile Dataset\.Newcastle University\(en\)\.Note:https://data\.ncl\.ac\.uk/articles/dataset/CIFARTile\_Dataset/24 551539/1External Links:[Link](https://data.ncl.ac.uk/articles/dataset/CIFARTile%5C_Dataset/24551539/1),[Document](https://dx.doi.org/10.25405/data.ncl.24551539.v1)Cited by:[§A\.6\.2](https://arxiv.org/html/2607.07984#A1.SS6.SSS2.Px4.p1.1)\.
- D\. Towers, R\. Geada, A\. Atapour\-Abarghouei, and A\. S\. McGough \(2023d\)GeoClassing Dataset\.Newcastle University\(en\)\.Note:https://data\.ncl\.ac\.uk/articles/dataset/GeoClassing\_Dataset/2 4050256/3External Links:[Link](https://data.ncl.ac.uk/articles/dataset/GeoClassing%5C_Dataset/24050256/3),[Document](https://dx.doi.org/10.25405/data.ncl.24050256.v3)Cited by:[§A\.6\.2](https://arxiv.org/html/2607.07984#A1.SS6.SSS2.Px7.p1.1)\.
- D\. Towers, R\. Geada, A\. Atapour\-Abarghouei, and A\. S\. McGough \(2023e\)Gutenberg Dataset\.Newcastle University\(en\)\.Note:https://data\.ncl\.ac\.uk/articles/dataset/Gutenberg\_Dataset/245 74753/1External Links:[Link](https://data.ncl.ac.uk/articles/dataset/Gutenberg%5C_Dataset/24574753/1),[Document](https://dx.doi.org/10.25405/data.ncl.24574753.v1)Cited by:[§A\.6\.2](https://arxiv.org/html/2607.07984#A1.SS6.SSS2.Px5.p1.1)\.
- D\. Towers, R\. Geada, A\. Atapour\-Abarghouei, and A\. S\. McGough \(2023f\)Language Dataset\.Newcastle University\(en\)\.Note:https://data\.ncl\.ac\.uk/articles/dataset/Language\_Dataset/245 74729/1External Links:[Link](https://data.ncl.ac.uk/articles/dataset/Language%5C_Dataset/24574729/1),[Document](https://dx.doi.org/10.25405/data.ncl.24574729.v1)Cited by:[§A\.6\.2](https://arxiv.org/html/2607.07984#A1.SS6.SSS2.Px2.p1.1)\.
- D\. Towers, R\. Geada, A\. Atapour\-Abarghouei, and A\. S\. McGough \(2023g\)MultNIST Dataset\.Newcastle University\(en\)\.Note:https://data\.ncl\.ac\.uk/articles/dataset/MultNIST\_Dataset/245 74678/1External Links:[Link](https://data.ncl.ac.uk/articles/dataset/MultNIST%5C_Dataset/24574678/1),[Document](https://dx.doi.org/10.25405/data.ncl.24574678.v1)Cited by:[§A\.6\.2](https://arxiv.org/html/2607.07984#A1.SS6.SSS2.Px3.p1.2)\.
- R\. Tu, N\. Roberts, M\. Khodak, J\. Shen, F\. Sala, and A\. Talwalkar \(2022\)NAS\-bench\-360: benchmarking neural architecture search on diverse tasks\.InAdvances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 \- December 9, 2022,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper%5C_files/paper/2022/hash/506630e4a43bb9d64a49f98b9ba934e9-Abstract-Datasets%5C_and%5C_Benchmarks.html)Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.p1.1),[Table 6](https://arxiv.org/html/2607.07984#A1.T6),[Table 6](https://arxiv.org/html/2607.07984#A1.T6.2.1),[§1](https://arxiv.org/html/2607.07984#S1.p2.1),[§4\.1](https://arxiv.org/html/2607.07984#S4.SS1.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2607.07984#S4.SS2.SSS0.Px1.p1.1)\.
- C\. White, M\. Safari, R\. S\. Sukthanker, B\. Ru, T\. Elsken, A\. Zela, D\. Dey, and F\. Hutter \(2023\)Neural architecture search: insights from 1000 papers\.CoRRabs/2301\.08727\.External Links:[Link](https://doi.org/10.48550/arXiv.2301.08727),[Document](https://dx.doi.org/10.48550/ARXIV.2301.08727),2301\.08727Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1)\.
- B\. Wu, X\. Dai, P\. Zhang, Y\. Wang, F\. Sun, Y\. Wu, Y\. Tian, P\. Vajda, Y\. Jia, and K\. Keutzer \(2019\)FBNet: hardware\-aware efficient convnet design via differentiable neural architecture search\.InIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16\-20, 2019,pp\. 10734–10742\.External Links:[Link](http://openaccess.thecvf.com/content%5C_CVPR%5C_2019/html/Wu%5C_FBNet%5C_Hardware-Aware%5C_Efficient%5C_ConvNet%5C_Design%5C_via%5C_Differentiable%5C_Neural%5C_Architecture%5C_Search%5C_CVPR%5C_2019%5C_paper.html),[Document](https://dx.doi.org/10.1109/CVPR.2019.01099)Cited by:[§3\.2](https://arxiv.org/html/2607.07984#S3.SS2.p2.1)\.
- Q\. Wuwu, C\. Gao, T\. Chen, Y\. Huang, Y\. Zhang, J\. Wang, J\. Li, H\. Zhou, and S\. Zhang \(2025\)PINNsAgent: automated PDE surrogation with large language models\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267\.External Links:[Link](https://proceedings.mlr.press/v267/wuwu25a.html)Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- Y\. Xu, L\. Xie, X\. Zhang, X\. Chen, G\. Qi, Q\. Tian, and H\. Xiong \(2020\)PC\-DARTS: partial channel connections for memory\-efficient architecture search\.In8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\-30, 2020,External Links:[Link](https://openreview.net/forum?id=BJlS634tPr)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p1.1),[§4\.2](https://arxiv.org/html/2607.07984#S4.SS2.SSS0.Px2.p1.1)\.
- Y\. Yamada, R\. T\. Lange, C\. Lu, S\. Hu, C\. Lu, J\. N\. Foerster, J\. Clune, and D\. Ha \(2025\)The AI scientist\-v2: workshop\-level automated scientific discovery via agentic tree search\.CoRRabs/2504\.08066\.External Links:[Link](https://doi.org/10.48550/arXiv.2504.08066),[Document](https://dx.doi.org/10.48550/ARXIV.2504.08066),2504\.08066Cited by:[§2\.3](https://arxiv.org/html/2607.07984#S2.SS3.p1.1)\.
- S\. Yang, X\. Yu, Y\. Tian, X\. Yan, H\. Ma, and X\. Zhang \(2023\)Evolutionary neural architecture search for transformer in knowledge tracing\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper%5C_files/paper/2023/hash/3e53d82a1113e3d240059a9195668edc-Abstract-Conference.html)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- Z\. Yang, W\. Zeng, S\. Jin, C\. Qian, P\. Luo, and W\. Liu \(2025\)NADER: neural architecture design via multi\-agent collaboration\.InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11\-15, 2025,pp\. 4452–4461\.External Links:[Link](https://openaccess.thecvf.com/content/CVPR2025/html/Yang%5C_NADER%5C_Neural%5C_Architecture%5C_Design%5C_via%5C_Multi-Agent%5C_Collaboration%5C_CVPR%5C_2025%5C_paper.html),[Document](https://dx.doi.org/10.1109/CVPR52734.2025.00420)Cited by:[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- C\. Ying, A\. Klein, E\. Christiansen, E\. Real, K\. Murphy, and F\. Hutter \(2019\)NAS\-bench\-101: towards reproducible neural architecture search\.InProceedings of the 36th International Conference on Machine Learning, ICML 2019, 9\-15 June 2019, Long Beach, California, USA,K\. Chaudhuri and R\. Salakhutdinov \(Eds\.\),Proceedings of Machine Learning Research, Vol\.97,pp\. 7105–7114\.External Links:[Link](http://proceedings.mlr.press/v97/ying19a.html)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- C\. Yu, X\. Liu, W\. Feng, C\. Tang, and J\. Lv \(2023\)GPT\-NAS: evolutionary neural architecture search with the generative pre\-trained model\.CoRRabs/2305\.05351\.External Links:[Link](https://doi.org/10.48550/arXiv.2305.05351),[Document](https://dx.doi.org/10.48550/ARXIV.2305.05351),2305\.05351Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- S\. Zagoruyko and N\. Komodakis \(2016\)Wide residual networks\.InProceedings of the British Machine Vision Conference 2016, BMVC 2016, York, UK, September 19\-22, 2016,R\. C\. Wilson, E\. R\. Hancock, and W\. A\. P\. Smith \(Eds\.\),External Links:[Link](https://bmva-archive.org.uk/bmvc/2016/papers/paper087/index.html)Cited by:[§4\.2](https://arxiv.org/html/2607.07984#S4.SS2.SSS0.Px1.p1.1)\.
- K\. Zhang and J\. S\. Bloom \(2020\)Deepcr: cosmic ray rejection with deep learning\.The Astrophysical Journal889\(1\),pp\. 24\.Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px7.p1.1)\.
- M\. R\. Zhang, N\. Desai, J\. Bae, J\. Lorraine, and J\. Ba \(2023\)Using large language models for hyperparameter optimization\.CoRRabs/2312\.04528\.External Links:[Link](https://doi.org/10.48550/arXiv.2312.04528),[Document](https://dx.doi.org/10.48550/ARXIV.2312.04528),2312\.04528Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- Z\. Zhang, E\. M\. Cofer, and O\. G\. Troyanskaya \(2021\)AMBIENT: accelerated convolutional neural network architecture search for regulatory genomics\.bioRxiv,pp\. 2021–02\.Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px10.p1.1)\.
- M\. Zheng, X\. Su, S\. You, F\. Wang, C\. Qian, C\. Xu, and S\. Albanie \(2023\)Can GPT\-4 perform neural architecture search?\.CoRRabs/2304\.10970\.External Links:[Link](https://doi.org/10.48550/arXiv.2304.10970),[Document](https://dx.doi.org/10.48550/ARXIV.2304.10970),2304\.10970Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- J\. Zhou and O\. G\. Troyanskaya \(2015\)Predicting effects of noncoding variants with deep learning–based sequence model\.Nature methods12\(10\),pp\. 931–934\.Cited by:[§A\.6\.1](https://arxiv.org/html/2607.07984#A1.SS6.SSS1.Px10.p1.1)\.
- K\. Zhou, X\. Huang, Q\. Song, R\. Chen, and X\. Hu \(2022\)Auto\-gnn: neural architecture search of graph neural networks\.Frontiers Big Data5\.External Links:[Link](https://doi.org/10.3389/fdata.2022.1029307),[Document](https://dx.doi.org/10.3389/FDATA.2022.1029307)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p2.1)\.
- X\. Zhou, X\. Wu, L\. Feng, Z\. Lu, and K\. C\. Tan \(2025\)Design principle transfer in neural architecture search via large language models\.InThirty\-Ninth AAAI Conference on Artificial Intelligence, Thirty\-Seventh Conference on Innovative Applications of Artificial Intelligence, Fifteenth Symposium on Educational Advances in Artificial Intelligence, AAAI 2025, Philadelphia, PA, USA, February 25 \- March 4, 2025,T\. Walsh, J\. Shah, and Z\. Kolter \(Eds\.\),pp\. 23000–23008\.External Links:[Link](https://doi.org/10.1609/aaai.v39i21.34463),[Document](https://dx.doi.org/10.1609/AAAI.V39I21.34463)Cited by:[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- H\. Zhu, G\. L\. Zhang, and S\. Huang \(2025\)LLM\-nas: llm\-driven hardware\-aware neural architecture search\.arXiv preprint arXiv:2510\.01472\.Cited by:[§1](https://arxiv.org/html/2607.07984#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.07984#S2.SS2.p1.1)\.
- B\. Zoph and Q\. V\. Le \(2017\)Neural architecture search with reinforcement learning\.In5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24\-26, 2017, Conference Track Proceedings,External Links:[Link](https://openreview.net/forum?id=r1Ue8Hcxg)Cited by:[§2\.1](https://arxiv.org/html/2607.07984#S2.SS1.p1.1)\.

## Appendix AAppendix

### A\.1Efficiency Analysis

We report search cost, parameter counts, and FLOPs for all tasks across both benchmarks\. Tables[6](https://arxiv.org/html/2607.07984#A1.T6)and[7](https://arxiv.org/html/2607.07984#A1.T7)cover NAS\-Bench\-360; Tables[4](https://arxiv.org/html/2607.07984#A1.T4)and[5](https://arxiv.org/html/2607.07984#A1.T5)cover Unseen NAS\. We separate*comparison*tables \(including baselines where available\) from*transparency*tables \(our method only, for metrics where no baseline data exists\)\.

##### Search cost\.

AgentNAS is more expensive than supernet\-based methods such as GAEA, which amortize search cost across a shared network\. This is expected: our pipeline trains each candidate individually through regularized evolution, as einspace\. Compared to einspace, which is the most directly comparable baseline, AgentNAS is in a similar range on Unseen NAS tasks, with the balance depending on task\-specific training cost per candidate\. On NAS\-Bench\-360, GPU\-hour totals confirm that the pipeline’s cost scales primarily with per\-candidate training time\.

##### Parameter counts\.

In Unseen NAS, AgentNAS results in parameter counts much smaller than einspace in general, and even comparable to those of conventional NAS methods\. In NAS\-Bench\-360, AgentNAS architectures tend to be larger than GAEA and expert\-designed baselines\. This is most pronounced on tasks where the LLM designs deep multi\-stage networks \(e\.g\., CIFAR\-100: 23\.4M vs\. 4\.9M for GAEA; Satellite: 67\.3M vs\. 3\.4M\)\. Expert architectures are typically the most parameter\-efficient, reflecting the benefit of domain\-specific engineering\. Notably, Phase 3 tends to reduce parameter count relative to the Phase 1 seed \(e\.g\., Spherical: 15\.4M vs\. 19\.7M; Satellite: 54\.7M vs\. 67\.3M\), indicating that NAS can tighten the architecture as well as expand it\.

##### FLOPs\.

Despite larger parameter counts, AgentNAS architectures are frequently more compute\-efficient than baselines in terms of FLOPs\. On NAS\-Bench\-360, our architectures use fewer FLOPs than GAEA on 6 of 10 tasks\. This pattern implies that the AgentNAS search space may generally favor parameter\-heavy but compute\-efficient operations, such as depthwise separable convolutions or mobile inverted bottleneck blocks\.

Table 4:Wall\-clock runtime and parameter counts on Unseen NAS tasks\. Reference methods use internal GPU clusters\(Geadaet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib44)\); our experiments use8×8\\timesRTX 2080 Ti\. RE\(einspace\) is the RE\(Mix\) variant; RE\(hNB201\) searches from scratch\. PC\-DARTS entries are missing where logs were unavailable, which is denoted byEricssonet al\.\([2024](https://arxiv.org/html/2607.07984#bib.bib30)\)\.AddNISTLanguageMultNISTCIFARTileGutenbergGeoClassingChesseractWall\-clock \(hours\)DrNAS1091125132310PC\-DARTS4–5129–2RE\(hNB201\)157221599379RE\(einspace\)55713262426542Ours \(P1\)675698646759Ours \(P3\)112193021911Ours \(P1\+P3\)7878711678669\#Params \(×106\\times 10^\{6\}\)DrNAS4453344PC\-DARTS3–332–2RE\(hNB201\)1113111RE\(einspace\)2012551411Ours \(P1\)611131112Ours \(P3\)11113172

Table 5:GPU time and FLOPs of AgentNAS architectures on Unseen NAS tasks\. No reference data is available for these metrics\. Experiments use8×8\\timesRTX 2080 Ti\.AddNISTLanguageMultNISTCIFARTileGutenbergGeoClassingChesseractGPU time \(hours\)Ours \(P1\)103141441651412540Ours \(P3\)67141352021211465Ours \(P1\+P3\)1702827836726239105FLOPs \(×106\\times 10^\{6\}\)Ours \(P1\)223171400204122020592Ours \(P3\)151181400280411701130

Table 6:GPU time, parameter counts, and FLOPs on NAS\-Bench\-360 tasks\. GAEA values are from\(Tuet al\.,[2022](https://arxiv.org/html/2607.07984#bib.bib43)\)using a single V100 16 GB; params and FLOPs are means over 3 seeds\. Our experiments use8×8\\timesRTX 2080 Ti\.\*Expert models contain non\-standard modules without FLOPs count\(Tuet al\.,[2022](https://arxiv.org/html/2607.07984#bib.bib43)\)\.CIFAR\-100SphericalDarcy FlowPSICOVCosmicNinaProFSD50KECGSatelliteDeepSEAGPU time \(hours\)GAEA9\.516\.56\.51821\.50\.5371402839\.5Ours \(P1\)9912511119943425831642294Ours \(P3\)186228121191515240383106399Ours \(P1\+P3\)285353233390949498698148693\#Params \(×106\\times 10^\{6\}\)GAEA4\.91\.70\.60\.50\.43\.40\.83\.33\.42\.9Expert3\.10\.161\.20\.60\.101\.40\.3516\.50\.4860\.9Ours \(P1\)23\.419\.71\.81\.012\.91\.210\.30\.6467\.335\.8Ours \(P3\)26\.015\.41\.20\.5911\.21\.210\.30\.2454\.725\.0FLOPs \(×109\\times 10^\{9\}\)GAEA1\.421\.919\.3317\.7414\.270\.892\.572\.280\.112\.01Expert1\.18n/a\*n/a\*0\.011\.960\.020\.660\.700\.010\.12Ours \(P1\)3\.360\.883\.6516\.648\.210\.230\.570\.630\.075\.71Ours \(P3\)3\.510\.831\.8615\.68\.430\.190\.590\.490\.055\.04\*Expert models contain non\-standard modules without FLOPs count\.

Table 7:Wall\-clock runtime of AgentNAS on NAS\-Bench\-360 tasks\. No wall\-clock reference is available for GAEA\. Experiments use8×8\\timesRTX 2080 Ti\.CIFAR\-100SphericalDarcy FlowPSICOVCosmicNinaProFSD50KECGSatelliteDeepSEAWall\-clock \(hours\)Ours \(P1\)152419603926680989Ours \(P3\)2730185633135631553Ours \(P1\+P3\)43543611573310114323142

### A\.2Training Recipes Generated by LLMs

In this section, we describe the training recipes defined by each run\. NAS uses the same recipes, except for the learning rate\. We compare the training recipes with those of einspace\(Ericssonet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib30)\)\.

Table 8:Comparison of hyperparameters: einspace’s fixed recipe vs\. our dynamically searched configurations\. einspace uses a single fixed recipe per benchmark suite, whereas ours tailors each hyperparameter per dataset\. When NAS chose different learning rates, we denote the change with→\\rightarrow\.einspace \(fixed\)Ours \(LLM\-searched\)DatasetEpochsBatchLearningWeightEpochsBatchLearningWeightsizeratedecaysizeratedecayAddNIST642560\.043×10−43\\times 10^\{\-4\}2001280\.003→0\.0120\.003\\rightarrow 0\.0120\.02Language642560\.043×10−43\\times 10^\{\-4\}2002563×10−4→6×10−43\\times 10^\{\-4\}\\rightarrow 6\\times 10^\{\-4\}0\.01MultNIST642560\.043×10−43\\times 10^\{\-4\}3001280\.25×10−45\\times 10^\{\-4\}CIFARTile642560\.043×10−43\\times 10^\{\-4\}3001280\.0010\.05Gutenberg642560\.043×10−43\\times 10^\{\-4\}1502563×10−4→7\.5×10−53\\times 10^\{\-4\}\\rightarrow 7\.5\\times 10^\{\-5\}1×10−41\\times 10^\{\-4\}GeoClassing642560\.043×10−43\\times 10^\{\-4\}2001280\.15×10−45\\times 10^\{\-4\}Chesseract642560\.043×10−43\\times 10^\{\-4\}2002563×10−43\\times 10^\{\-4\}5×10−45\\times 10^\{\-4\}CIFAR1002001280\.15×10−45\\times 10^\{\-4\}2001280\.15×10−45\\times 10^\{\-4\}Spherical2001280\.15×10−45\\times 10^\{\-4\}2001280\.001→0\.0040\.001\\rightarrow 0\.0040\.05NinaPro2001280\.15×10−45\\times 10^\{\-4\}300320\.0010\.01FSD50K2002560\.15×10−45\\times 10^\{\-4\}1001923×10−43\\times 10^\{\-4\}0\.1Darcy Flow20040\.0015×10−45\\times 10^\{\-4\}100052×10−4→4×10−42\\times 10^\{\-4\}\\rightarrow 4\\times 10^\{\-4\}1×10−41\\times 10^\{\-4\}PSICOV20080\.0015×10−45\\times 10^\{\-4\}20043×10−43\\times 10^\{\-4\}5×10−55\\times 10^\{\-5\}Cosmic20080\.0015×10−45\\times 10^\{\-4\}150160\.002→0\.0040\.002\\rightarrow 0\.0041×10−41\\times 10^\{\-4\}ECG2002560\.15×10−45\\times 10^\{\-4\}100640\.001→2\.5×10−40\.001\\rightarrow 2\.5\\times 10^\{\-4\}1×10−41\\times 10^\{\-4\}Satellite20040960\.15×10−45\\times 10^\{\-4\}20010240\.001→0\.0020\.001\\rightarrow 0\.0021×10−41\\times 10^\{\-4\}Deepsea2002560\.15×10−45\\times 10^\{\-4\}1201283×10−4→1\.2×10−33\\times 10^\{\-4\}\\rightarrow 1\.2\\times 10^\{\-3\}0\.01

### A\.3Search Algorithm Ablation

We compare three search algorithms within the same slotted search space: random search \(RS\) following the protocol ofLi and Talwalkar \([2019](https://arxiv.org/html/2607.07984#bib.bib86)\), GDAS\(Dong and Yang,[2019](https://arxiv.org/html/2607.07984#bib.bib61)\), and regularized evolution \(RE\)\(Realet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib12)\)as used by AgentNAS Phase 3\. All methods use the same seed architecture, training recipe, and evaluation protocol; the only difference is the search strategy\. For GDAS, the virtual learning rate slot is excluded because GDAS operates over a continuous relaxation that is incompatible with this slot type\. Table[9](https://arxiv.org/html/2607.07984#A1.T9)reports the best non\-seed architecture found by each method\.

Table 9:Search\-method comparison within the AgentNAS slotted search space \(lower is better\)\. AgentNAS\-RS: random search\(Li and Talwalkar,[2019](https://arxiv.org/html/2607.07984#bib.bib86)\)\. AgentNAS\-GDAS: GDAS\(Dong and Yang,[2019](https://arxiv.org/html/2607.07984#bib.bib61)\), excluding the virtual learning rate slot\. AgentNAS\-RE: regularized evolution\(Realet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib12)\)\(AgentNAS Phase 3 default\)\. All results report the best non\-seed architecture\.Bestandsecond bestper row\.TaskMetricAgentNAS\-RSAgentNAS\-GDASAgentNAS\-RESpherical0\-1 err\. \(%\)43\.9644\.9638\.71NinaPro0\-1 err\. \(%\)7\.287\.598\.04Darcy FlowRel\. L20\.00560\.03410\.0061Language0\-1 err\. \(%\)0\.460\.460\.49CIFARTile0\-1 err\. \(%\)11\.2875\.578\.48Chesseract0\-1 err\. \(%\)33\.5134\.2231\.39RE and RS perform comparably overall, with neither method consistently dominating the other\. RE achieves the best result on Spherical, CIFARTile, and Chesseract—tasks where the search space is large enough for evolutionary recombination to discover beneficial slot combinations\. RS wins on NinaPro, Darcy Flow, and Language, suggesting that on tasks where the performance landscape is relatively smooth or the effective search space is small, uniform sampling is sufficient\. GDAS performs substantially worse on two tasks: Darcy Flow \(0\.03410\.0341vs\.0\.00560\.0056for RS\) and CIFARTile \(75\.5775\.57vs\.8\.488\.48for RE\)\. We have not optimized GDAS hyperparameters for the slotted setting; with tuning, the gap may narrow\.

These results confirm that the slotted search space is compatible with diverse NAS methods, and that the gains reported in the main text are not specific to the choice of regularized evolution\. The modest differences between RE and RS also suggest that the search space constructed by Phase 2 is well\-structured: even uniform sampling finds competitive architectures, indicating that the LLM\-defined space contains a high density of strong candidates\.

### A\.4Multiple Random Seed Ablation

Table 10:Multi\-seed \(\{42, 43, 44\}\) variability ofAgentNAS\-Fullper task\. Each cell reports the ever\-best val\-final architecture’s test metric, parameter count, and FLOPs\. Mean±\\pmstandard deviation is computed over three random seeds\.test metric per seed \(val\-selected arch\)TaskPhaseseed 42seed 43seed 44test metric \(mean±\\pmstd\)params \[M\] \(mean±\\pmstd\)FLOPs \(mean±\\pmstd\)ChesseractP10\.33840\.37380\.39990\.3707±\\pm0\.03092\.01±\\pm1\.21403±\\pm290 MP30\.31390\.34710\.31350\.3249±\\pm0\.01931\.21±\\pm0\.95491±\\pm555 MCIFARTileP10\.11200\.10010\.13680\.1163±\\pm0\.018713\.92±\\pm19\.411\.21±\\pm1\.36 GP30\.08480\.09040\.11740\.0975±\\pm0\.01748\.23±\\pm9\.09765±\\pm508 MDarcyFlowP10\.00520\.00470\.00810\.0060±\\pm0\.001821\.54±\\pm34\.904\.98±\\pm2\.33 GP30\.00610\.01160\.00640\.0080±\\pm0\.003114\.10±\\pm12\.832\.71±\\pm1\.04 GLanguageP10\.00850\.00220\.03390\.0149±\\pm0\.01683\.83±\\pm4\.8013±\\pm10 MP30\.00490\.02500\.01860\.0162±\\pm0\.01030\.86±\\pm0\.057±\\pm10 MNinaproP10\.08350\.08500\.07740\.0819±\\pm0\.00401\.79±\\pm2\.06124±\\pm114 MP30\.08040\.09100\.09260\.0880±\\pm0\.00661\.63±\\pm1\.8199±\\pm133 MSphericalP10\.45080\.44270\.42380\.4391±\\pm0\.013917\.45±\\pm7\.291\.48±\\pm1\.41 GP30\.38710\.42330\.39860\.4030±\\pm0\.018512\.90±\\pm3\.231\.26±\\pm1\.10 G

Ninaproseed\-43 phase\-3 had a FLOPs recording bug; that seed is excluded from the phase\-3 FLOPs aggregate\.

### A\.5Full Baseline Results

Table 11:Full per\-method test errors \(%, lower is better\) on Unseen NAS benchmarks\(Geadaet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib44)\)\. Results are transcribed from the original dataset\(Geadaet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib44)\)and einspace\(Ericssonet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib30)\)\.Bestandsecondbest performance per dataset\.BaselinesRegularised evolution \(RE\)hNB201einspaceRand\. SearchRand\.Sampl\.AgentNASDatasetRN18PC\-DARTSDrNASBonsai\-NetRE\(Scratch\)RE\(RN18\)RE\(Mix\)RE\(Scratch\)DARTSBonsaieinspaceeinspaceLLMFullAddNIST6\.643\.402\.942\.096\.182\.462\.2816\.132\.9365\.8333\.0089\.872\.822\.42Language7\.849\.8811\.4512\.357\.573\.162\.0811\.889\.8823\.1712\.9964\.740\.850\.49MultNIST8\.643\.321\.902\.836\.563\.637\.756\.283\.4560\.2433\.9181\.133\.543\.53CIFARTile52\.877\.7218\.928\.5341\.6939\.3537\.2469\.119\.2675\.2469\.1074\.7511\.208\.48Gutenberg56\.6850\.8853\.3851\.4356\.3045\.9849\.8463\.3052\.2871\.0060\.4280\.3144\.6344\.17GeoClassing9\.925\.393\.974\.347\.674\.694\.8739\.574\.4636\.4430\.8775\.651\.600\.97Chesseract40\.6542\.8041\.7639\.2436\.0839\.6938\.1440\.5040\.8431\.1738\.5455\.1733\.8431\.39

### A\.6Benchmark Details

#### A\.6\.1NAS\-Bench\-360 Dataset Descriptions

NAS\-Bench\-360\(Tuet al\.,[2022](https://arxiv.org/html/2607.07984#bib.bib43)\)is a benchmark suite designed to evaluate neural architecture search methods across diverse domains\. It comprises ten tasks spanning image classification, signal processing, scientific computing, and sequence analysis\. Below we briefly describe each task\.

##### CIFAR\-100\.

A standard image classification task consisting of natural RGB images categorized into 100 fine\-grained classes\(Krizhevskyet al\.,[2009](https://arxiv.org/html/2607.07984#bib.bib51)\)\.

##### Spherical\.

CIFAR\-100 images are projected onto the northern hemisphere of a sphere with random rotations applied, followingCohenet al\.\([2018](https://arxiv.org/html/2607.07984#bib.bib64)\)\. The resulting60×6060\\times 60RGB images simulate distorted signals common in omnidirectional vision and meteorological sensing\.

##### NinaPro\.

A hand gesture classification task using electromyography \(EMG\) signals from the NinaPro DB5 dataset\(Atzoriet al\.,[2012](https://arxiv.org/html/2607.07984#bib.bib65)\)\. EMG recordings from two Myo armbands worn by 10 subjects are sampled into 2D representations followingCôté\-Allardet al\.\([2019](https://arxiv.org/html/2607.07984#bib.bib66)\), with 18 gesture classes\.

##### FSD50K\.

A multi\-label sound event classification task derived fromFonsecaet al\.\([2022](https://arxiv.org/html/2607.07984#bib.bib67)\), containing approximately 51,000 audio clips across 200 classes from the AudioSet ontology\(Gemmekeet al\.,[2017](https://arxiv.org/html/2607.07984#bib.bib68)\)\. Performance is measured by mean average precision \(mAP\)\.

##### Darcy Flow\.

A regression task that learns the mapping from initial conditions of a partial differential equation to its solution, using the Darcy Flow dataset fromLiet al\.\([2021b](https://arxiv.org/html/2607.07984#bib.bib69)\)\. Inputs and outputs are 2D grids representing fluid states at different timesteps\. The evaluation metric is mean squared error \(ℓ2\\ell\_\{2\}\)\.

##### PSICOV\.

An inter\-residual protein distance prediction task based onAdhikari \([2020](https://arxiv.org/html/2607.07984#bib.bib71)\)\. Large\-scale 2D features extracted from protein sequences serve as inputs, and the targets are pairwise distance matrices\. Performance is evaluated using mean absolute error on distances below 8Å\(MAE8\)\.

##### Cosmic\.

A pixel\-level binary segmentation task for identifying cosmic ray artifacts in Hubble Space Telescope images of resolved galaxies\(Zhang and Bloom,[2020](https://arxiv.org/html/2607.07984#bib.bib70)\)\. The evaluation metric is the false\-negative rate \(FNR\)\.

##### ECG\.

A cardiac rhythm classification task based on the 2017 PhysioNet Challenge\(Cliffordet al\.,[2017](https://arxiv.org/html/2607.07984#bib.bib72)\)\. ECG recordings sampled at 300 Hz are segmented using a 1,000 ms sliding window with 500 ms stride, and classified into four categories \(normal, disease, other, noisy\)\. Performance is reported as F1\-score\.

##### Satellite\.

A satellite image time series classification task using Formosat\-2 imagery over Toulouse, France\(Petitjeanet al\.,[2012](https://arxiv.org/html/2607.07984#bib.bib73)\)\. Each pixel’s temporal profile across multiple spectral channels is classified into 24 land cover types\.

##### DeepSEA\.

A genomic sequence classification task for predicting chromatin feature activity from DNA sequences\(Zhou and Troyanskaya,[2015](https://arxiv.org/html/2607.07984#bib.bib74)\), based on ENCODE profiles\(Feingoldet al\.,[2004](https://arxiv.org/html/2607.07984#bib.bib75)\)\. FollowingZhanget al\.\([2021](https://arxiv.org/html/2607.07984#bib.bib76)\), 36 out of 919 categories are used with 5% of the training data\. Performance is measured by the area under the ROC curve \(AUROC\)\.

#### A\.6\.2Unseen NAS Dataset Descriptions

Unseen NAS\(Geadaet al\.,[2024](https://arxiv.org/html/2607.07984#bib.bib44)\)is a benchmark suite comprising eight tasks that test neural architecture search methods on diverse and unconventional data modalities\. Tasks are categorized as Type\-1 \(solvable by humans\) or Type\-2 \(requiring non\-trivial pattern recognition beyond human intuition\)\. Below we briefly describe each task\.

##### AddNIST\.

A Type\-1 task built from MNIST\(LeCunet al\.,[2010](https://arxiv.org/html/2607.07984#bib.bib77)\)\. Each3×28×283\\times 28\\times 28image encodes one MNIST digit per color channel, and the label is computed asl=\(r\+g\+b\)−1l=\(r\+g\+b\)\-1, yielding 20 classes\. The task tests whether an architecture can learn to identify digits and perform arithmetic jointly\(Towerset al\.,[2023a](https://arxiv.org/html/2607.07984#bib.bib81)\)\.

##### Language\.

A Type\-2 task where six\-letter words from 10 Latin\-alphabet languages are encoded as binary images via character\-position mappings\(Towerset al\.,[2023f](https://arxiv.org/html/2607.07984#bib.bib79)\)\. The goal is to classify each image into its source language, testing whether encoded character patterns retain sufficient linguistic information for classification\.

##### MultNIST\.

A Type\-1 task similar to AddNIST, with3×28×283\\times 28\\times 28images whose label isl=\(r×g×b\)mod10l=\(r\\times g\\times b\)\\bmod 10, producing 10 classes\(Towerset al\.,[2023g](https://arxiv.org/html/2607.07984#bib.bib78)\)\. The modular arithmetic removes the bias toward larger digit values present in AddNIST\.

##### CIFARTile\.

A Type\-1 task composing four CIFAR\-10\(Krizhevskyet al\.,[2009](https://arxiv.org/html/2607.07984#bib.bib51)\)images into a3×64×643\\times 64\\times 64grid\(Towerset al\.,[2023c](https://arxiv.org/html/2607.07984#bib.bib82)\)\. The label indicates the number of distinct CIFAR\-10 classes in the grid minus one, resulting in 4 classes\. The task requires simultaneous recognition and comparison of multiple objects\.

##### Gutenberg\.

A Type\-2 task derived from Project Gutenberg literary works by six authors \(Thomas Aquinas, Confucius, Hawthorne, Plato, Shakespeare, and Tolstoy\)\(Towerset al\.,[2023e](https://arxiv.org/html/2607.07984#bib.bib80)\)\. Consecutive three\-word sequences are encoded into1×27×181\\times 27\\times 18binary images via character\-position mappings\. The task is to predict the authorship from these spatial letter patterns\.

##### Isabella\.

A Type\-2 task using musical recordings from the Isabella Stewart Gardner Museum\. Five\-second audio snippets are converted into1×64×1281\\times 64\\times 128spectrograms, and the goal is to classify each into one of four composition eras \(Baroque, Classical, Romantic, and 20th Century\)\.

##### GeoClassing\.

A Type\-2 task that repurposes satellite images from BigEarthNet\(Sumbulet al\.,[2019](https://arxiv.org/html/2607.07984#bib.bib85)\)by replacing ground\-cover labels with the country of origin, determined by cross\-referencing ESA Sentinel patch coordinates\(Towerset al\.,[2023d](https://arxiv.org/html/2607.07984#bib.bib84)\)\. Each3×60×603\\times 60\\times 60image is classified into one of 10 European countries based on topological and vegetation cues\.

##### Chesseract\.

A Type\-1 task encoding chess endgame positions from eight grandmasters as12×8×812\\times 8\\times 8images via one\-hot piece\-type encoding\(Towerset al\.,[2023b](https://arxiv.org/html/2607.07984#bib.bib83)\)\. The three classes correspond to game outcomes \(White wins, Draw, Black wins\)\. The 12\-channel representation tests whether architectures can handle non\-standard input dimensions\.

##### Blindness implementation\.

The blind setting, which follows the official protocol of the benchmark, is implemented by extending the explorer\-side filter and by replacing the task identity in every prompt\. Specifically, the Phase 1 task README is replaced with a synthetic minimal spec \(input shape, number of classes / output shape, loss name, metric direction\) under the placeholder nameTASK\_X, and the sameTASK\_Xlabel is propagated to the Code Executor and Slotted Architecture Planner system prompts\. The benchmark loader and the labels themselves are unchanged—blind mode only affects what the LLM agents can read, not what the harness trains on\.

##### What does the LLM infer?

Despite the blinding, the Data Explorer often forms hypotheses about the task domain from data statistics alone\. Table[12](https://arxiv.org/html/2607.07984#A1.T12)summarizes these inferences\. The LLM correctly identifies the domain in two cases:Chesseract, where it maps the 12\-channel input to 2 players×\\times6 piece types and recognizes chess outcome prediction; andLanguage, where it recovers the sequence\-of\-tokens abstraction \(each row encodes one character index in a 24\-symbol alphabet\), prescribing 1\-D sequence models despite not naming the task as language identification\. On the remaining five tasks, the LLM defaults to generic image classification hypotheses—treating AddNIST and MultNIST as MNIST\-family variants without recognizing the multi\-digit arithmetic structure, CIFARTile as standard 4\-class natural\-image classification, GeoClassing as a CIFAR\-10/STL\-10 variant, and Gutenberg as sensor or game\-state data rather than character\-level text\. Notably, these misidentifications do not prevent the pipeline from achieving strong performance: AgentNAS reaches state\-of\-the\-art results on Language and GeoClassing despite the LLM’s incorrect or incomplete domain recognition, suggesting that the pipeline’s effectiveness relies more on data\-driven architecture refinement than on correct domain identification\.

Table 12:LLM domain hypotheses under the blind setting\. The Data Explorer infers a task description from data statistics alone; the verdict indicates whether the true domain was identified\.TaskTrue DomainLLM HypothesisVerdictAddNIST3\-digit sum predictionMNIST\-family RGB image classificationPartial \(MNIST family, missed arithmetic\)ChesseractChess outcome \(W/D/L\)Chess outcome; 12ch = 2 players×\\times6 piecesCorrectCIFARTile2×\\times2 tiled CIFAR\-1004\-class natural\-image classificationIncorrect \(missed tile structure\)GeoClassingSatellite geo\-classificationCIFAR\-10/STL\-10 variantIncorrect \(missed satellite imagery\)GutenbergText genre classificationPose/sensor/game\-state dataIncorrect \(missed text domain\)LanguageWritten language IDLength\-24 token sequences, 1\-D modelsAlmost correct\(abstraction recovered\)MultNIST3\-digit number classificationRGB MNIST variantPartial \(MNIST family, missed multi\-digit\)

##### Does domain knowledge help?

To test whether the blind setting harms performance, we run the full pipeline without blinding on four tasks and compare against the blind results \(Table[13](https://arxiv.org/html/2607.07984#A1.T13)\)\.

Table 13:Blind vs\. non\-blind test error \(%, lower is better\)\. NegativeΔ\\Deltaindicates that removing blindness improves performance\.TaskBlindNon\-blindΔ\\DeltaAddNIST2\.420\.88−1\.54\-1\.54MultNIST3\.531\.03−2\.50\-2\.50Gutenberg44\.1743\.93−0\.24\-0\.24Chesseract31\.3935\.34\+3\.95\+3\.95The results are mixed\. On AddNIST and MultNIST, removing blindness yields substantial gains, indicating that the LLM can leverage domain knowledge \(multi\-digit arithmetic\) to design more effective architectures\. On Gutenberg, the difference is negligible—knowing that the task is authorship classification does not translate into actionable architectural insights\. Most surprisingly, on Chesseract, the blind setting*outperforms*the non\-blind setting by nearly 4 percentage points\. We inspected the LLM logs and found out that under blindness, the Data Explorer systematically benchmarks diverse proxy models \(MLP, CNN, Transformer, Random Forest\) before recommending an architecture, producing a richer exploration report\. When the domain is known, the LLM skips this exploration and directly prescribes architectures based on domain priors, resulting in less thorough data\-driven adaptation\. A similar tendency appears on the other tasks where domain knowledge helps—the LLM explores less when it believes it already knows the answer\. A deeper investigation of this exploration bias is interesting but is beyond the scope of this work\.

### A\.7Implementation Details

This appendix documents the concrete hyperparameters and engineering choices behind the AgentNAS pipeline\. Where Appendix[A\.8](https://arxiv.org/html/2607.07984#A1.SS8)describes*what*each LLM agent sees in its prompt, this appendix describes*how*the surrounding pipeline is configured: the slot generation pipeline \(§[A\.7\.1](https://arxiv.org/html/2607.07984#A1.SS7.SSS1)\) and the search\-process protocol \(§[A\.7\.2](https://arxiv.org/html/2607.07984#A1.SS7.SSS2)\)\.

#### A\.7\.1Slot Generation

##### Slot caps\.

Given the seed architecture, the Slot Planner is instructed to produce exactlymax\_slots=20\\texttt\{max\\\_slots\}=20slots, each with up tomax\_alternatives\_per\_slot=8\\texttt\{max\\\_alternatives\\\_per\\\_slot\}=8alternatives\. If the Planner produces more slots, the post\-parse step caps at2020in declaration order; over\-long alternative lists are truncated to88\. Both caps are enforced after parsing rather than re\-prompting because the Slotted Architecture Planner is a single\-call agent \(§[A\.8](https://arxiv.org/html/2607.07984#A1.SS8)\) and a re\-prompt would amount to a full Phase 2 retry\.

#### A\.7\.2Search Process

##### Phase 1 budget and patience\.

Phase 1 has a hard cap ofmax\_evaluations=160\\texttt\{max\\\_evaluations\}=160architecture evaluations and a patience counter ofpatience=64\\texttt\{patience\}=64*successful*consecutive no\-improvement evaluations\. A failed evaluation \(rejected/oom/error/timeout\) consumes one unit of the160160\-evaluation budget but does*not*increment patience\. The motivation is asymmetric: the budget protects against runaway compute, while patience protects against runaway wall\-clock with no metric movement; counting failures toward patience would let a streak of OOM proposals trigger early termination on a task where the LLM simply needed a few VRAM\-aware retries\. After every batch the planner state and per\-evaluation log are checkpointed tostate\.jsonso that a pod restart resumes exactly where the previous process left off\.

##### Phase 3 budget and patience\.

Phase 3 reuses the samemax\_evaluations=160\\texttt\{max\\\_evaluations\}=160pool and has its own independent patience counternas\_patience=64\\texttt\{nas\\\_patience\}=64, also counted only over successful evaluations\. Evolution additionally caps generations atmax\_generations=10\\texttt\{max\\\_generations\}=10\(initial population\+\+99offspring waves\)\. Withpopulation\_size=16\\texttt\{population\\\_size\}=16this gives a soft upper bound of160160total evaluations, which lines up with the budget ceiling; in practice the patience trigger fires first on tasks where evolution converges within a few generations\.

##### Proxy training scaling\.

The Planner is told to design its training recipe for the*full*schedule\. The pipeline then runs the proxy atproxy\_epoch\_fraction=0\.3\\texttt\{proxy\\\_epoch\\\_fraction\}=0\.3, i\.e\. proxyepochs=max⁡\(1,round​\(0\.3⋅Efull\)\)=\\max\(1,\\mathrm\{round\}\(0\.3\\cdot E\_\{\\mathrm\{full\}\}\)\)\. Thewarmup\_epochsand any time\-based scheduler parameters \(T\_maxfor cosine,step\_sizefor step LR,patiencefor plateau\) are scaled by the same factor and floored at11, so the LR schedule still completes within the shorter run\. On top of the epoch cap, every proxy run is subject to a6060\-minute GPU time hard cap \(proxy\_time\_limit\_s=3600=3600s\), measured via paired CUDA events around each forward/backward step rather than wall clock; data\-loading time and evaluation time are excluded so that a slow disk does not penalize an architecture’s measured fitness\. The same VRAM cap \(max\_vram\_mb=11,264=11\{,\}264, the RTX 2080 Ti’s1111GB\) is enforced viatorch\.cuda\.max\_memory\_allocatedsampled per epoch; an OOM exception during training is caught and reported asstatus="oom"\.

##### Final training: two waves\.

After Phase 1 \(and again after Phase 3\), the top\-88architectures by proxy validation metric are passed through a two\-wave full\-training protocol\. We usetop\_k\_final=8\\texttt\{top\\\_k\\\_final\}=8to fully utilize the88GPUs of one node in parallel\.

- •Wave 1: validation metric\.Each architecture is retrained with the*full*epoch schedule \(proxy\_epoch\_fraction=1\.0=1\.0\) on the train split, using the validation split for best\-checkpoint selection\. The harness records, for every epoch, the validation metric and the model’s state dict; only the state dict at the best validation epoch is kept\. The Wave 1 metric is what every downstream pipeline decision uses\.
- •Wave 2: test metric\.The same architectures are then retrained from scratch on the train\+\+validation*combined*set \(no held\-out split for checkpoint selection\), using the*last*checkpoint, and evaluated once on the held\-out test set\. This wave is run only for analysis and is the test column in the main paper\.

Both waves use the samefinal\_time\_limit\_s=86,400\\texttt\{final\\\_time\\\_limit\\\_s\}=86\{,\}400s \(2424h\) GPU\-time hard cap, again measured via CUDA events\. Failed Wave 1 architectures are reported asstatus="error"or"oom"/"timeout"; the Wave 1 winner is chosen as the lowestval\_final\_metricamong successful entries\.

##### Population initialization for evolution\.

The Phase 3 evolutionary search initializes its population of1616as: the seed \(11individual, choices identical toseed\_choices\), then⌊\(16−1\)/3⌋=5\\lfloor\(16\-1\)/3\\rfloor=5single\-slot mutations of the seed,55double\-slot mutations of the seed, and the remaining55uniformly\-random configurations\. For the*random\-search*ablation, the population is instead drawn as1616i\.i\.d\. uniform samples from the search space\.

##### Regularized Evolution\.

We perform regularized evolution through tournament\-of\-tournament\_size=5\\texttt\{tournament\\\_size\}=5followed by50/5050/50single/double\-slot mutation\. After evaluation, the population is updated by appending offspring and removing the oldest individuals until size1616is restored; ages are incremented at end\-of\-generation\.

#### A\.7\.3Modularization Ablation

Table 14:Effect of the modularization step on tasks with block\-stack seeds \(lower is better\)\. Without modularization, NAS terminates early due to patience and finds weaker architectures\.TaskWithoutmod\.Withmod\.Evals\(without\)Evals\(with\)CIFAR\-10016\.7415\.5080 \(patience hit\)160CIFARTile11\.208\.4880 \(patience hit\)160As described in Section[3\.2](https://arxiv.org/html/2607.07984#S3.SS2), the Slot Planner applies a modularization step when the seed uses a block\-stack pattern, grouping tightly coupled primitives into coherent modules before decomposition\. This step is applied selectively and automatically; it does not alter the seed’s architecture but prevents Phase 2 from fragmenting coupled operation sequences, which can degrade the quality of the resulting search space\. Table[14](https://arxiv.org/html/2607.07984#A1.T14)describes two cases where modularization is activated whose seeds exhibit block\-stack structure\. Without modularization, the search space degenerates: NAS exhausts its patience criterion early \(80 evaluations vs\. the 160\-evaluation cap\) and produces weaker architectures\. With modularization, NAS utilizes the full budget and delivers substantial improvements on both tasks\.

### A\.8Context Management and Prompts for Agents

This appendix details, for each LLM agent in the AgentNAS pipeline, what enters its context window at every call\. Five agents are involved across the three phases:

- •Planner\(§[A\.8\.1](https://arxiv.org/html/2607.07984#A1.SS8.SSS1)\) — Phase 1 controller; stateful, multi\-turn\.
- •Data Explorer\(§[A\.8\.2](https://arxiv.org/html/2607.07984#A1.SS8.SSS2)\) — Phase 1 sandbox; autoregressive code\-execution loop\.
- •Code Executor\(§[A\.8\.3](https://arxiv.org/html/2607.07984#A1.SS8.SSS3)\) — Phase 1 and Phase 2; translates descriptions into PyTorch\.
- •Modularizer\(§[A\.8\.4](https://arxiv.org/html/2607.07984#A1.SS8.SSS4)\) — Phase 1\.5; refactors the seed into per\-stagenn\.Modules\.
- •Slotted Architecture Planner\(§[A\.8\.5](https://arxiv.org/html/2607.07984#A1.SS8.SSS5)\) — Phase 2; single\-call slotted\-architecture designer\.

#### A\.8\.1Planner

Planner system promptYou are the Planner agent for a neural architecture search pipeline\. Your goalis to design the best possible neural network architecture for the given task\.\#\# Available ActionsAt each turn, choose ONE action:1\. <EXPLORE\> \{purpose\} </EXPLORE\>Launch the Data Explorer to analyze the dataset\.2\. <EXPERIMENT\>Propose up to \{n\_gpu\} architectures for parallel evaluation\. For each:<ARCH id="arch\_NNN"\><DESCRIPTION\> \{high\-level architecture description\} </DESCRIPTION\><REASONING\> \{why this design, what you expect\} </REASONING\><TRAINING\_RECIPE\> \{optimizer, lr, scheduler, epochs, \.\.\.\} </TRAINING\_RECIPE\></ARCH\></EXPERIMENT\>To modify a prior architecture, name it by literal arch\_NNN id inDESCRIPTION; the pipeline regex\-injects its full code into the Executor\.3\. <STOP\><FINAL\_SELECTION\> \{arch\_ids for full training\} </FINAL\_SELECTION\></STOP\>After receiving experiment results:<SUMMARY\> \{what worked, what failed, surprises, next direction\} </SUMMARY\>\#\# Constraints\- \{vram\_mb\} MB VRAM per GPU\- Proxy training: \{proxy\_epoch\_pct\}% of proposed epochs, \{time\_limit\_min\}\-min GPU cap\- Final training: full proposed epochs, 24\-hour GPU cap\- \{budget\_remaining\} of \{budget\_total\} architecture evaluations remaining\- Patience: \{patience\_counter\}/\{patience\} consecutive no\-improvement\- NO pre\-trained weights\. All models trained from scratch\.\#\# Performance Baseline\{sota\_info\} Lower metric is better\.\#\# Training\-recipe whitelist \(anything else is silently dropped\)\- optimizer: sgd \| adam \| adamw \(others \-\> AdamW\)\- scheduler: cosine \| step \| warmup\_cosine \| plateau \| none\- augmentations \(harness\-level\): subset of \{mixup, cutmix, random\_erasing\}\- loss: TASK\-FIXED\. Only label\_smoothing and mixup/cutmix are loss\-side knobs\.\- NOT supported: AMP / fp16 / bf16, torch\.compile, grad accum/checkpointing\.\- For task\-specific augs \(e\.g\. time\-shift\), implement INLINE inArchitecture\.forward\(\) gated on self\.training\.\#\# Task\{task\_readme\}\#\# Data Explorer Report\{explorer\_report\}

#### A\.8\.2Data Explorer

Data Explorer system promptYou are a Data Explorer agent for a neural architecture search pipeline\. Yourjob is to analyze a dataset by writing and executing Python code\.\#\# Environment\- Available: numpy, pandas, scipy, matplotlib, seaborn, librosa, h5py,sklearn, torch, and the standard library\.\- Data is at: \{data\_root\}/\{data\_subdir\}/\- You are working in a temporary directory\. Save plots to \./plot\.png\.\#\# WorkflowAt each step, write Python code in one or more \`\`\`python\`\`\` blocks\. ALL blocksin a single turn are concatenated and executed together in one freshsubprocess \(notebook\-style WITHIN a turn; variables/imports do NOT carryacross turns \-\- each turn starts a new process\)\. stdout\+stderr \(truncated to5000 chars\) returns as the next user message\.CRITICAL rules:\- NEVER fabricate "\*\*Execution output:\*\*" or any output block in your ownmessage \-\- outputs come back to you in the NEXT user turn\.\- Do NOT produce <REPORT\> until you have actually executed at least one\`\`\`python\`\`\` block and observed real output\.When done, write:<REPORT\> \{dataset stats; key characteristics; domain observations;recommendations for the Planner\} </REPORT\>\#\# Constraints\- Maximum \{max\_steps\} steps\. On step \{max\_steps\}, <REPORT\> is mandatory\.\- Keep code concise and focused on the investigation purpose\.\#\# Purpose\{purpose\} \# provided by the Planner via <EXPLORE\>

#### A\.8\.3Code Executor

Code Executor system promptYou are a Code Executor for a neural architecture search pipeline\. Your jobis to translate architecture descriptions into working PyTorch code\.\#\# What you produce1\. Model code: a class \`Architecture\` extending \`nn\.Module\`, with\_\_init\_\_\(self\) and forward\(self, x\)\.\- x shape: \(batch, \{input\_shape\}\)\- output shape: \(batch, \{output\_spec\}\)\- Imports: torch, torch\.nn, torch\.nn\.functional, stdlib only\.\- NO pre\-trained weights\. No torchvision\.models, no timm, no checkpoints\.2\. Training config: JSON dict \(schema below\)\. Use ONLY the fields listed\.Wrap model in \`\`\`python\`\`\`, config in \`\`\`json\`\`\`\.\#\# Constraints\- VRAM cap: \{vram\_mb\} MB\.\- Proxy: \{proxy\_epoch\_pct\}% of proposed epochs, \{time\_limit\}\-min GPU cap\.Final: full proposed epochs, 24\-hour GPU cap\.\- Loss: \{loss\_info\} \# CrossEntropyLoss / BCEWithLogitsLoss / regressionOutput raw logits/predictions; do NOT apply softmax/sigmoid\.\- On retry: only fix bugs \(shape mismatches, runtime errors\)\. Do NOT changethe architecture’s functional design\.\#\# Training\-recipe whitelist \(anything else is silently dropped\)\- optimizer: sgd \| adam \| adamw \(others \-\> AdamW\)\- scheduler: cosine \| step \| warmup\_cosine \| plateau \| none\- augmentations: subset of \{mixup, cutmix, random\_erasing\}\- NOT supported: amp/autocast, torch\.compile, grad accumulation\- For unsupported augs \(time\_shift, gaussian\_noise, \.\.\.\), implement INLINEin forward\(\) gated on self\.training:if self\.training:\# e\.g\. shift = int\(torch\.randint\(\-4, 5, \(1,\)\)\.item\(\)\)\# x = torch\.roll\(x, shifts=shift, dims=\-1\) \+ torch\.randn\_like\(x\)\*0\.01\.\.\.\#\# Training config fields honored by the harness\{"optimizer": "adamw","lr": \.\.\., "weight\_decay": \.\.\., "momentum": \.\.\., "betas": \.\.\.,"scheduler": "cosine", "scheduler\_params": \{\.\.\.\}, "warmup\_epochs": \.\.\.,"batch\_size": \.\.\., "epochs": \.\.\.,"augmentations": \[\.\.\.\], "augmentation\_params": \{\.\.\.\},"grad\_clip": \.\.\., "label\_smoothing": \.\.\.\}\#\# Task\{task\_description\}

Code Executor input prompt\#\# Recent Architectures \(code for reference\)\# only included if the Planner’s description mentions arch\_NNN tokens; the\# pipeline regex\-matches them and injects the matching code blocks here\.The Planner’s description may reference these by \`arch\_NNN\` \-\- use them asthe baseline to modify\.\#\#\# \{arch\_id\} \(status=\{\.\.\.\}, metric=\{\.\.\.\}\)\`\`\`python\{full\_prior\_architecture\_code\}\`\`\`\.\.\. \(one block per referenced architecture\)\#\# Architecture to implement\{architecture\_description\} \# from <DESCRIPTION\> in the Planner proposal\#\# Training recipe\{training\_recipe\} \# from <TRAINING\_RECIPE\> in the Planner proposalProduce the \`Architecture\` class and training config JSON\.\# On validation failure, the truncated traceback is fed back as the next user\# turn \(up to 10 retries\):The model failed validation with this error:\`\`\`\{truncated\_traceback\}\`\`\`Please fix the code\. Only fix bugs \-\- do NOT change the architecture design\.

Code Executor system promptYou are implementing a PyTorch nn\.Module for a neural architecture search slot\.\#\# What to produceA complete Python class named \`\{class\_name\}\` that extends \`nn\.Module\`\.\`\`\`pythonclass \{class\_name\}\(nn\.Module\):def \_\_init\_\_\(self\):super\(\)\.\_\_init\_\_\(\)\# \.\.\. your layers here \(NO constructor arguments; dimensions hardcoded\# from this slot’s input/output shape\) \.\.\.\{forward\_signature\}: \# forward\(self, x\) for "single"\# forward\(self, x, residual\) for "dual"\# x shape: \(batch, \{input\_shape\_at\_slot\}\)\# residual shape: \(batch, \{residual\_shape\}\) \# "dual" only\# return shape: \(batch, \{output\_shape\_at\_slot\}\)return x\`\`\`\#\# Interface\{interface\_note\} \# explanation of single vs\. dual semantics for this slot\#\# Slot: \{slot\_description\}\#\# Alternative: \{alternative\_description\}\#\# Constraints\- Imports: torch, torch\.nn, torch\.nn\.functional only\.\- NO pre\-trained weights\. Random initialization only\.\- Class must be fully self\-contained\.\- Wrap code in a \`\`\`python\`\`\` block\.\# User\-side message \(one line\):Implement \`\{class\_name\}\` for: \{alternative\_description\}\# Retry feedback on validation failure \(up to 10 retries\):The module failed validation:\`\`\`\{truncated\_traceback\}\`\`\`Fix it\. Class: \`\{class\_name\}\`, forward signature: \`\{forward\_signature\}\`\.

#### A\.8\.4Modularizer

Modularizer system promptYou are a Modularizer agent\. You refactor a PyTorch Architecture’s sourcecode so that each repeated stage is wrapped in its own self\-containednn\.Module subclass\. Forward\-pass behavior must be IDENTICAL to the original\(bit\-equivalent under torch\.allclose\(rtol=1e\-5, atol=1e\-6\) in eval mode afterthe original’s state\_dict is transplanted into the refactored module\)\.\#\# What you must produce1\. All helper classes the original used \(RegNetBlock, FNOBlock, \.\.\.\)\.Copy them VERBATIM \-\- no renames, no reordering, no layer\-constructionchanges\.2\. NEW per\-stage Stage classes \(Stage1, Stage2, Stage3, \.\.\.\)\. Each must:\- Have a docstring stating the contract, e\.g\.:’’’INTERNAL: BatchNorm \+ ReLU \+ SE \+ residual baked in\. Outernorm/act/skip slots SHOULD be Identity\.’’’\- In \_\_init\_\_, instantiate blocks/parameters in the SAME order with theSAME hyperparameters as the original stage construction\.\- In forward\(self, x\), perform exactly the same operations the originaldid for this stage \-\- INCLUDING any per\-stage outer norm/activation/residual previously written in the top\-level Architecture\.forward\.\- Single tensor in / single tensor out\.3\. Rewritten Architecture class:\- Same helper attributes \(stem, pool, head, \.\.\.\) constructed identically\.\- Each repeated stage replaced by \`self\.stageN = StageN\(\)\` \(instead of\`nn\.ModuleList\(\[\.\.\.\]\)\`\)\.\- Architecture\.forward simplified to:\# data\-augmentation block \(UNCHANGED, copy verbatim\)x = self\.stem\(x\)x = self\.stage1\(x\); x = self\.stage2\(x\); \.\.\.x = self\.pool\(x\)\.flatten\(1\)return self\.head\(x\)\#\# Hard constraints \(any violation rejects the output\)\- No parameter add/remove\. sum\(p\.numel\(\)\) must match exactly\.\- No op\-type changes \(no ReLU<\-\>GELU, BN<\-\>LN, Conv<\-\>DWConv, etc\.\)\.\- No op\-order changes within a stage\.\- Data\-augmentation block: copy verbatim\.\- \_init\_weights logic: copy verbatim\.\- Submodule construction order in \_\_init\_\_: must match original \(RNG align\)\.\- No new imports beyond what original used\.\#\# Output formatReturn ONLY a single \`\`\`python\`\`\` block containing the complete refactoredsource\. No prose outside the code block\.\#\# Original source\`\`\`python\{original\_code\}\`\`\`\#\# Task description\{task\_description\} \# "Task: \{task\_name\}, Input shape: \(batch, \.\.\.\)"\# Initial user\-side message:Refactor the Architecture above per the contract\. Wrap each repeated stage inits own Stage class with internal norm/act/skip baked in, simplify thetop\-level forward, and ensure forward\-pass equivalence\. Return ONE\`\`\`python\`\`\` block\.\# Retry feedback on validation failure \(up to 5 retries\):Validation FAILED:\{failure\_summary\}\# one of, e\.g\.:\# \- "Parameter count mismatch: original=\{\.\.\.\} vs modularized=\{\.\.\.\}\. \.\.\."\# \- "Parameter shapes differ as a sorted multiset\. \.\.\."\# \- "Forward output mismatch: max\_abs\_diff=\{diff\}\. After transplanting the\# original weights into your refactored module, the forward pass produced\# different outputs in eval mode\. Check op order, per\-stage outer\# norm/act/skip lifted into the stage’s forward, and missing operations\."Re\-emit the refactored Architecture, fixing the issue\. The contract \(no paramadds/removes, no op\-type or op\-order changes, identical construction order,augmentation block untouched\) must be respected\.

#### A\.8\.5Slotted Architecture Planner

Grammar/Slot Planner system promptYou are a Grammar Planner for neural architecture search\. Your job is todesign a rich, domain\-aware search space around a seed architecture\.\#\# What you produceA JSON object with \`scaffold\_code\` and \`slots\`\.\#\#\# scaffold\_codeThe main Architecture class with searchable components replaced by slot calls:\- self\.xxx = SLOT\_xxx\(\) in \_\_init\_\_ \-\- NO constructor arguments\. AlwaysSLOT\_xxx\(\) with no args\. Dimensions are hardcoded inside each module fromthe slot’s input/output shape\.\- forward calls:"single" slot: x = self\.xxx\(x\)"dual" slot: x = self\.xxx\(x, residual\)\- Do NOT include \`class SLOT\_xxx\` definitions in scaffold\_code\. The actualclass bodies are inserted automatically before the scaffold duringassembly\. Stub forwards = silent identity collapse\.\- Every slot you instantiate in \_\_init\_\_ must be called from forward\(or from a helper method that forward invokes\)\.\#\#\# slotsDefine 10\-20 fine\-grained slots:\- Per\-stage operation slots: stageN\_op, stageN\_norm, stageN\_act,stageN\_attn, stageN\_pool\- Cross\-cutting parameter slots: width\_stageN, kernel\_stageNSkip/composition can live inside op modules OR as separate "dual" slots\.\#\# Designing good alternativesEach slot: 4\-8 alternatives, including1\. Seed choice \(always first\)\.2\. Domain\-specific novel ops \-\- e\.g\.:\- audio: frequency\-band processing, multi\-scale temporal conv\- images: multi\-scale feature extraction, deformable conv\- 1D sigs: dilated causal conv, wavelet\-inspired blocks3\. Standard alternatives \(conv3x3, conv1x1, depthwise\-separable,squeeze\-excite, self\-attention\)\.4\. Identity \(pass\-through\) and Zero \(outputs zeros\) \-\- let NAS discoverthat a component is unnecessary\.\#\# Slot JSON formatsingle\-input:\{"name": "stageN\_op", "class\_name": "SLOT\_stageN\_op","description": \.\.\., "input\_shape": \[\.\.\.\], "output\_shape": \[\.\.\.\],"interface": "single", "seed\_description": \.\.\.,"alternatives": \[\.\.\.\]\}dual\-input \(skip/composition\):\{ \.\.\., "interface": "dual", "residual\_shape": \[\.\.\.\] \}\#\# Key principles\- More slots = richer search\. 10\-20 is the target\.\- Domain creativity matters; do not just list standard ConvNet blocks\.\- Include identity AND zero for EVERY slot\.\- Input/output shapes must be EXACT\.\- NO pre\-trained weights\.\#\# Seed Architecture\`\`\`python\{seed\_code\}\`\`\`\#\# Seed PerformanceMetric: \{val\_final\_metric\}, Params: \{param\_count\}\#\# Data Explorer Report\{explorer\_report\} \# truncated to 3000 chars\#\# Planner’s Accumulated Insights\{planner\_summary\} \# last 3000 chars of the summaries \(recency\-preserving\)\#\# Task\{task\_description\} \# blind\-ablation runs replace task name with TASK\_X\#\# Recognizing self\-contained stage modulesIf the seed defines a stage as a self\-contained nn\.Module \(with internalnorm/act/residual\), then:\- Create ONE op slot per stage \(the stage as a whole\)\.\- Create OUTER norm/act/skip slots BUT mark their alt0 explicitly as Identitywith the description "Identity \(already inside the stage block\)"\.\- Provide non\-Identity alternatives for the outer slots so NAS can ADD anouter norm/act if useful, but the seed pick is Identity\.DO NOT unwrap the stage into multiple sub\-slots that duplicate functionalityalready inside the block\.