Improving Auto-Design of Neural PDE Solvers with a Domain-Specific Language

# Improving Auto-Design of Neural PDE Solvers with a Domain-Specific Language
Source: [https://arxiv.org/html/2608.04384](https://arxiv.org/html/2608.04384)
###### Abstract

Neural PDE solver auto\-design is fundamentally a search\-space representation problem\. In the space of unrestricted Python programs, valid solvers form an extremely sparse subset: most candidate programs are syntactically incorrect, semantically incompatible, or numerically unstable\. Direct code generation therefore forces an LLM to spend most of its search capacity navigating implementation failures rather than reasoning about solver quality\. ADSL\-PDE addresses this challenge by introducing a structured search state between solver concepts and executable code\. It represents the functional decisions that determine a neural PDE solver—architecture, physical constraints, objectives, sampling, and optimization—while abstracting away low\-level implementation details\. A deterministic compiler maps each valid search state to an executable solver\. In effect, ADSL\-PDE reshapes the search space: it removes large regions of invalid programs, increases the density of meaningful candidates, and preserves the compositional freedom needed to discover previously unseen designs\. Solver evolution can thus operate over design decisions rather than code artifacts\. Built on this representation, our evolutionary agent iteratively proposes, evaluates, and refines solver search states using empirical feedback\. Across multiple PDE benchmarks, ADSL\-PDE improves both search efficiency and optimization stability, achieving more than 52% performance improvement within the first ten evolution iterations\. These results suggest a broader principle for LLM\-driven auto\-design: effective agents do not merely require stronger reasoning, but a search representation that concentrates exploration on valid and consequential decisions\.

## Introduction

Neural partial differential equation \(PDE\) solver auto\-design has emerged as an important research direction in AI for Science, aiming to automatically discover effective solver designs while reducing reliance on human expertise\(Raissiet al\.[2019](https://arxiv.org/html/2608.04384#bib.bib2); Luet al\.[2021](https://arxiv.org/html/2608.04384#bib.bib3); Liet al\.[2021](https://arxiv.org/html/2608.04384#bib.bib4)\)\. This task involves searching over a structured design space that includes network architectures, loss functions, sampling strategies, and training hyperparameters\. Existing approaches can be broadly divided into search\-based and agent\-based methods\. Search\-based methods use neural architecture search, evolutionary algorithms, or Bayesian optimization to identify solver configurations within predefined search spaces\(Elskenet al\.[2019](https://arxiv.org/html/2608.04384#bib.bib5); Zoph and Le[2017](https://arxiv.org/html/2608.04384#bib.bib6); Realet al\.[2019](https://arxiv.org/html/2608.04384#bib.bib7)\)\. Although effective in some cases, these methods are limited by manually designed search spaces and may not capture the full range of neural PDE solver designs\. Agent\-based methods instead use the reasoning and code\-generation abilities of large language models to iteratively generate and improve solvers\(Yaoet al\.[2023](https://arxiv.org/html/2608.04384#bib.bib9); Schicket al\.[2023](https://arxiv.org/html/2608.04384#bib.bib10); Wuwuet al\.[2025](https://arxiv.org/html/2608.04384#bib.bib1)\)\. However, most of them directly generate Python code, requiring agents to handle both solver design and implementation details, such as framework syntax, function calls, and dependency management\. This greatly increases the search complexity and can lead to invalid programs and unstable training\. Therefore, search\-based methods are limited by overly restricted spaces, while agent\-based methods face overly large and implementation\-heavy spaces\. Neither provides an effective representation for reliable and flexible neural PDE solver auto\-design\.

![Refer to caption](https://arxiv.org/html/2608.04384v1/Figures/main.png)Figure 1:Reshaping the neural PDE solver search space with ADSL\-PDE\. \(a\) ADSL\-PDE represents solver designs with a structured schema, maps them to a modular backend, and restricts the agent to editing ADSL programs\. \(b\) This language design removes unproductive regions of the unrestricted Python space, yielding a smaller and denser search space of valid solver candidates\. \(c\) The resulting structured search space enables more effective optimization; quantitative results are reported in Table[3](https://arxiv.org/html/2608.04384#Sx4.T3)\.Domain\-specific languages \(DSLs\) have long been used in software engineering to provide compact and structured representations for specialized tasks\(Fowler[2010](https://arxiv.org/html/2608.04384#bib.bib11); Parr[1998](https://arxiv.org/html/2608.04384#bib.bib12)\)\. Inspired by this idea, we propose ADSL\-PDE \(Agent\-oriented Domain\-Specific Language for PDE Solver Evolution\), a DSL tailored for neural PDE solver design, to address the representation limitations of existing agent\-based auto\-design approaches\. As illustrated in Figure[1](https://arxiv.org/html/2608.04384#Sx1.F1), the core idea of ADSL\-PDE is to reshape the search space for neural PDE solver auto\-design\. Unlike approaches that directly generate Python code, ADSL\-PDE decouples solver design semantics from implementation logic\. The language frontend explicitly represents high\-level design components, including network architectures, physical constraints, loss formulations, and training strategies, while a deterministic backend translates these specifications into executable solvers\. This design allows agents to evolve neural PDE solvers by modifying structured design scripts rather than repeatedly manipulating complex and error\-prone implementation code\.

Our contributions are summarized as follows:

\(1\) We identify representation as a key bottleneck in neural PDE solver auto\-design and highlight the critical role of language design in enabling efficient agent\-based solver evolution\.

\(2\) We propose ADSL\-PDE, an agent\-oriented domain\-specific language and evolutionary framework that transforms neural PDE solver design from unrestricted code generation into structured design\-space exploration\.

\(3\) We validate ADSL\-PDE on multiple PDE benchmarks, where it consistently improves search efficiency and achieves over 52% performance gains within the first ten evolution iterations\.

![Refer to caption](https://arxiv.org/html/2608.04384v1/Figures/combined_score.png)Figure 2:Combined scores on Darcy\-Flow and Burgers\. Solid curves with shaded bands denote DSL\-based results, while lighter dashed curves denote results without DSL\.
## Related Work

### Neural PDE Solvers

Neural PDE solvers are widely used in scientific machine learning\. Physics\-informed neural networks incorporate governing equations into training objectives\(Raissiet al\.[2019](https://arxiv.org/html/2608.04384#bib.bib2)\), while neural operators such as DeepONet, FNO, and U\-NO learn mappings between PDE inputs and solutions\(Luet al\.[2021](https://arxiv.org/html/2608.04384#bib.bib3); Liet al\.[2021](https://arxiv.org/html/2608.04384#bib.bib4); Rahmanet al\.[2022](https://arxiv.org/html/2608.04384#bib.bib16)\)\. Graph\-based simulators support irregular meshes and complex geometries\(Pfaffet al\.[2021](https://arxiv.org/html/2608.04384#bib.bib17)\), and PDEBench and PDENNEval provide systematic evaluations of these solver families\(Takamotoet al\.[2022](https://arxiv.org/html/2608.04384#bib.bib18); Weiet al\.[2024](https://arxiv.org/html/2608.04384#bib.bib19)\)\. However, their performance is strongly task\-dependent: PINNs are sensitive to gradient imbalance, loss weighting, sampling, and optimization\(Wanget al\.[2021](https://arxiv.org/html/2608.04384#bib.bib20),[2023a](https://arxiv.org/html/2608.04384#bib.bib21)\), whereas neural operators and graph\-based methods rely on different data representations and inductive biases\. We therefore focus on representing and exploring solver designs across multiple solver families rather than proposing another fixed architecture\.

### Automated Design of Neural PDE Solvers

Neural solver design has been automated through architecture search, evolutionary algorithms, and hyperparameter optimization\(Elskenet al\.[2019](https://arxiv.org/html/2608.04384#bib.bib5); Zoph and Le[2017](https://arxiv.org/html/2608.04384#bib.bib6); Realet al\.[2019](https://arxiv.org/html/2608.04384#bib.bib7)\)\. PDE\-specific methods such as Auto\-PINN, NAS\-PINN, and PINN\-DARTS search predefined PINN architecture or training spaces\(Wanget al\.[2023b](https://arxiv.org/html/2608.04384#bib.bib22); Wang and Zhong[2024](https://arxiv.org/html/2608.04384#bib.bib23); Li and Chen[2025](https://arxiv.org/html/2608.04384#bib.bib24)\)\. More recent methods, including PINNsAgent, Lang\-PINN, and CodePDE, use LLMs to generate or refine PDE solver implementations\(Wuwuet al\.[2025](https://arxiv.org/html/2608.04384#bib.bib1); Heet al\.[2025](https://arxiv.org/html/2608.04384#bib.bib25); Liet al\.[2025](https://arxiv.org/html/2608.04384#bib.bib26)\)\. AlphaEvolve and OpenEvolve further combine LLM\-based mutation with automated evaluation and population selection\(Novikovet al\.[2025](https://arxiv.org/html/2608.04384#bib.bib27); Sharma[2025b](https://arxiv.org/html/2608.04384#bib.bib28)\)\. Traditional search methods are constrained by predefined spaces, whereas code\-generating agents must handle both solver design and low\-level implementation\. ADSL\-PDE instead evolves structured solver specifications that can be validated and compiled deterministically\.

### Agent\-Oriented Languages and Structured Representations

Domain\-specific languages provide compact abstractions for specialized tasks\(Fowler[2010](https://arxiv.org/html/2608.04384#bib.bib11)\), and recent studies suggest that representation can influence LLM reasoning and internal feature activation\(Yanget al\.[2026](https://arxiv.org/html/2608.04384#bib.bib29); Zhanget al\.[2025](https://arxiv.org/html/2608.04384#bib.bib30)\)\. LMQL introduces constraints and control flow for language\-model generation\(Beurer\-Kellneret al\.[2023](https://arxiv.org/html/2608.04384#bib.bib31)\), while PAL and Logic\-LM translate natural\-language problems into executable programmatic or symbolic forms\(Gaoet al\.[2023](https://arxiv.org/html/2608.04384#bib.bib32); Panet al\.[2023](https://arxiv.org/html/2608.04384#bib.bib33)\)\. Structured agent interfaces include CodeStruct for syntax\-aware code manipulation\(Kimet al\.[2026](https://arxiv.org/html/2608.04384#bib.bib34)\), ACDL for agent\-context construction\(Peleg\-Pelcet al\.[2026](https://arxiv.org/html/2608.04384#bib.bib35)\), and Extended Scenic for executable driving scenarios\(Safaet al\.[2026](https://arxiv.org/html/2608.04384#bib.bib15)\)\. Unlike these systems, ADSL\-PDE integrates domain semantics, static validation, backend compilation, evaluator feedback, and iterative optimization, using a PDE\-specific language as both the representation and action space for neural solver evolution\.

## Method

### PDE Problem Formulation

We consider a general PDE task defined on a spatial\-temporal domainΩ×\[0,T\]\\Omega\\times\[0,T\], with target solutionu:Ω×\[0,T\]→ℝduu:\\Omega\\times\[0,T\]\\rightarrow\\mathbb\{R\}^\{d\_\{u\}\}\. The problem is written as

𝒩​\[u\]​\(x,t\)\\displaystyle\\mathcal\{N\}\[u\]\(x,t\)=f​\(x,t\),\\displaystyle=f\(x,t\),\(x,t\)∈Ω×\[0,T\],\\displaystyle\(x,t\)\\in\\Omega\\times\[0,T\],\(1\)ℬ​\[u\]​\(x,t\)\\displaystyle\\mathcal\{B\}\[u\]\(x,t\)=g​\(x,t\),\\displaystyle=g\(x,t\),\(x,t\)∈∂Ω×\[0,T\],\\displaystyle\(x,t\)\\in\\partial\\Omega\\times\[0,T\],u​\(x,0\)\\displaystyle u\(x,0\)=u0​\(x\),\\displaystyle=u\_\{0\}\(x\),x∈Ω,\\displaystyle x\\in\\Omega,where𝒩\\mathcal\{N\}andℬ\\mathcal\{B\}denote the differential and boundary operators, respectively\.

A neural PDE solver is determined by a coupled set of design choices rather than by its network architecture alone\. We represent a solver design as the structured schema

z=\(𝒯,ℱ,𝒜,𝒞,𝒮,ℛ,ℰ\),z=\(\\mathcal\{T\},\\mathcal\{F\},\\mathcal\{A\},\\mathcal\{C\},\\mathcal\{S\},\\mathcal\{R\},\\mathcal\{E\}\),\(2\)where𝒯\\mathcal\{T\}denotes the PDE task,ℱ\\mathcal\{F\}the solver family,𝒜\\mathcal\{A\}the neural architecture,𝒞\\mathcal\{C\}the physical constraints,𝒮\\mathcal\{S\}the sampling strategy,ℛ\\mathcal\{R\}the training recipe, andℰ\\mathcal\{E\}the evaluation protocol\. Given a task𝒯\\mathcal\{T\}, solver auto\-design searches for a valid schemaz∈𝒵𝒯validz\\in\\mathcal\{Z\}^\{\\mathrm\{valid\}\}\_\{\\mathcal\{T\}\}with strong validation performance\. ADSL\-PDE represents and evolves solver designs at this schema level rather than directly searching over implementation code\.

### Overview of ADSL\-PDE

The framework of ADSL\-PDE consists of two coupled components: a language system and an evolution system\. The language system defines how solver designs are expressed, checked, and executed, while the evolution system defines how an LLM agent modifies solver designs under feedback\.

Given an initial DSL program, the LLM agent proposes a modified program by editing solver\-level fields rather than low\-level implementation code\. The parser converts the program into a typed intermediate representation \(IR\), where solver components and their dependencies are explicitly represented\. The verifier then checks whether the IR satisfies task\-level, solver\-level, and backend\-level constraints\. Only verified candidates are passed to the compiler, which instantiates an executable neural PDE solver under a deterministic backend\.

The compiled solver is first evaluated through a quick training stage\. Promising candidates are selected for full training, while invalid or poorly performing candidates are converted into structured feedback\. This feedback is aligned with the DSL fields edited by the agent, allowing subsequent modifications to target meaningful solver\-design decisions such as loss weighting, sampling density, architecture depth, activation functions, or optimization schedules\. In this way, ADSL\-PDE transforms agent\-based solver evolution from unrestricted code generation into verifiable solver\-schema search\.

Table 1:Comparison with manually designed function\-learning and operator\-learning baselines\. The reported metric is MSE or relativeL2L\_\{2\}error, depending on the benchmark, and lower values indicate better performance\. The geometric mean is computed over available entries, excluding missing results\. The best result for each benchmark is shown in bold\.
### ADSL Language Design

ADSL\-PDE is designed as an agent\-facing language for neural PDE solver design\. Its purpose is not merely to provide a configuration format, but to reshape the action space of the LLM agent\. Instead of allowing the agent to generate arbitrary Python code, ADSL exposes a compact set of solver\-level primitives that correspond to the components of the solver schema defined above\. A complete ADSL program contains typed blocks for the PDE task, solver family, network architecture, physical constraints, sampler, trainer, evaluator, and evolution policy\.

The front\-end language follows a principle of controlled expressiveness\. It is more restrictive than Python because the agent cannot define arbitrary functions, import external libraries, modify backend training loops, or bypass the compiler\. At the same time, it is more expressive than a fixed hyperparameter table because it allows the agent to modify meaningful PDE\-solver decisions, including architecture depth and width, activation functions, residual and boundary loss weights, adaptive sampling strategies, optimizer choices, learning\-rate schedules, and training stages\. Moreover, at a frequency controlled by a dedicated hyperparameter, the agent can synthesize new solver implementations by combining design elements from high\-performing solvers and, after validation, register them in the backend code library\. This design removes implementation\-level degrees of freedom while preserving solver\-level flexibility\.

The language primitives are aligned with PDE solver semantics\. Thetaskblock specifies the equation, domain, variables, boundary and initial conditions, and evaluation metric\. Thesolverblock determines the solver family and automatic differentiation mode\. Thenetworkblock defines the neural parameterization\. Thephysicsblock specifies residual, boundary, data, and constraint losses\. Thesamplerblock controls how interior, boundary, and data points are selected\. Thetrainerblock describes the optimization recipe, while theevaluatorandevolutionblocks define model selection and editable mutation scopes\.

After parsing, an ADSL program is converted into an IR process\. The IR serves as the canonical representation of the solver schema\. It removes superficial syntactic variations in the front\-end program and represents the solver as a dependency graph of typed components\. This representation makes the dependencies among solver components explicit and enables static checking before execution\.

The verifier enforces the validity of the solver schema before compilation\. It checks whether the PDE task is complete, whether component input\-output types are consistent, whether tensor shapes and variable references are compatible, whether the selected solver family can support the requested losses and derivative orders, and whether every IR primitive has a corresponding backend implementation\. For instance, if a candidate refers to an undefined physical variable, uses an unsupported sampler for a given domain, or selects a solver configuration that cannot compute the required derivatives, the verifier rejects the candidate before training\. The rejection is not returned as a generic runtime failure, but as diagnostic feedback tied to the corresponding DSL field\.

The backend compiler grounds a verified IR into an executable solver\. The compiler instantiates neural modules, automatic differentiation residuals, loss functions, samplers, optimizers, schedulers, training loops, checkpointing, and evaluation routines according to the verified IR\. It is deterministic: the same IR always produces the same solver implementation\. Importantly, the compiler does not perform hidden search, tune hyperparameters, or introduce task\-specific heuristics beyond the DSL specification\. This separation is essential for attributing the improvement to language\-mediated solver search rather than uncontrolled backend optimization\.

Through this language design, ADSL\-PDE provides a middle ground between fixed search spaces and unrestricted code generation\. It gives the agent access to expressive PDE solver\-design primitives while enforcing type, feasibility, and executability constraints before training\. This language system eliminates unproductive regions of the agent’s search space, thereby improving the efficiency of agent\-driven code evolution\.

### Language\-Guided Evolution Framework

We build a language\-guided evolutionary framework on top of ADSL\-PDE\. The population is divided into method islands, where each island corresponds to a neural solver family, such as PINN, FNO, U\-NO, and so on\. Each solver family instantiates the unified solver schema with its own architecture, loss construction, sampling mechanism, data interface, and training procedure\. The island therefore defines a family\-specific search space while maintaining a common representation across heterogeneous solvers\.

In each evolution round, the framework selects an island and a parent ADSL program\. Conditioned on the parent, the island constraints, and historical feedback, the LLM agent modifies valid solver\-level fields within that family\. The resulting program is parsed into a typed IR, verified against both the unified schema and the selected solver family, and then compiled into an executable solver\. Invalid candidates are rejected before training and returned as structured diagnostic feedback\.

Valid candidates are first evaluated with a limited quick\-training budget\. Promising candidates are then selected for full training, and the island populations are updated according to validation performance\. The framework returns numerical feedback, including validation error and training stability, together with structural and diagnostic feedback linked to the modified ADSL fields\. This allows the agent to revise solver designs rather than repair low\-level code\.

The framework also supports controlled cross\-island innovation\. At a frequency specified by an evolution hyperparameter, the agent can combine compatible components from high\-performing solvers to create a new solver implementation\. After validation, the implementation is registered in the backend library and can participate in subsequent evolution as a new method island\. The code selection and evolution logic can be swapped with different evolution frameworks, such as OpenEvolve\(Sharma[2025a](https://arxiv.org/html/2608.04384#bib.bib43)\)\. Figure[3](https://arxiv.org/html/2608.04384#Sx3.F3)illustrates the evolutionary genealogy of candidate neural solvers for the Burgers equation across the PINN, MPNN, and UNO method islands\. The framework explores diverse solver designs through within\-island evolution and cross\-island migration, while filtering invalid candidates and retaining the globally best program\.

![Refer to caption](https://arxiv.org/html/2608.04384v1/Figures/evolution_tree.png)Figure 3:The complete evolution path of the Burgers equation under ADSL\-PDE\.

## Experiments

We evaluate ADSL\-PDE from three perspectives: final solver accuracy, comparison with search\-based and LLM\-agent baselines, and representation\-level reliability\. The main experiments are designed to test whether a structured agent\-facing language can improve neural PDE solver auto\-design beyond manually designed solvers, direct search baselines, and prompt\-driven agent methods\.

### Experimental Setup

#### Benchmarks\.

We evaluate ADSL\-PDE on a diverse set of PDE benchmarks covering one\-dimensional, two\-dimensional, three\-dimensional, and high\-dimensional tasks\. The benchmark set includes advection, diffusion\-reaction, Burgers, diffusion\-sorption, Allen–Cahn, Cahn–Hilliard, Darcy flow, shallow\-water, Black\-Scholes\-Barenblatt, heat, Navier–Stokes, Poisson, and Kuramoto–Sivashinsky equations\. These tasks cover convection\-dominated dynamics, nonlinear diffusion, multi\-field coupled systems, elliptic equations, and high\-dimensional PDEs\.

#### Baselines\.

We compare ADSL\-PDE with two groups of baselines\. The first group consists of manually designed neural PDE solvers, including function\-learning methods and operator\-learning methods\. Specifically, we compare with PINN\(Raissiet al\.[2019](https://arxiv.org/html/2608.04384#bib.bib2)\), DFVM\(Cen and Zou[2024](https://arxiv.org/html/2608.04384#bib.bib36)\), RFM\(Chenet al\.[2022](https://arxiv.org/html/2608.04384#bib.bib37)\), PINO\(Liet al\.[2024](https://arxiv.org/html/2608.04384#bib.bib38)\), U\-NO\(Rahmanet al\.[2022](https://arxiv.org/html/2608.04384#bib.bib16)\), FNO\(Liet al\.[2021](https://arxiv.org/html/2608.04384#bib.bib4)\)and MPNN\(Pfaffet al\.[2021](https://arxiv.org/html/2608.04384#bib.bib17)\)\. This comparison evaluates whether ADSL\-PDE can discover solver designs that are competitive with human\-designed neural PDE solvers\. The second group consists of search\-based and LLM\-agent\-based auto\-design baselines, including RandomAgent, BayesianAgent, PINNsAgent\(Wuwuet al\.[2025](https://arxiv.org/html/2608.04384#bib.bib1)\), Lang\-PINN\(Heet al\.[2025](https://arxiv.org/html/2608.04384#bib.bib25)\), and PINNacle\(Haoet al\.[2024](https://arxiv.org/html/2608.04384#bib.bib42)\)\. This comparison evaluates whether the proposed language representation improves agent\-based solver search\.

#### Metrics and implementation\.

We report mean squared error \(MSE\) or relativeL2L\_\{2\}error according to the metric used by each benchmark\. Lower values indicate better predictive accuracy\. To summarize performance across heterogeneous tasks with different error scales, we additionally report the geometric mean over available entries\. For representation\-level reliability, we report valid candidate rate \(VCR\), search efficiency \(SE\), and token\-normalized search efficiency \(Token\-SE\)\. Unless otherwise specified, the evolutionary search uses DeepSeek\-V4\-Pro as the LLM backbone\. All representation ablations use the same PDE tasks, search budget, and backend training protocol\. For ADSL\-PDE, the LLM agent only modifies the front\-end DSL program; each candidate is parsed into IR, statically verified, compiled by the deterministic backend, and evaluated by the same training and validation pipeline\.

### Main Results against Manually Designed Solvers

Table[1](https://arxiv.org/html/2608.04384#Sx3.T1)compares ADSL\-PDE with manually designed neural PDE solvers\. ADSL\-PDE achieves the best result on all listed benchmark entries and obtains the lowest geometric mean error\. The result indicates that the proposed language representation does not sacrifice solver expressiveness: although the agent is restricted to editing structured DSL programs rather than arbitrary Python code, it can still discover high\-quality solver designs across different PDE families\.

The improvement is especially clear on nonlinear and coupled PDEs\. For example, on 1D Burgers, ADSL\-PDE reduces the error from6\.01×10−26\.01\\times 10^\{\-2\}for U\-NO and6\.03×10−26\.03\\times 10^\{\-2\}for FNO to2\.20×10−32\.20\\times 10^\{\-3\}\. On 2D Burgers, ADSL\-PDE achieves1\.60×10−31\.60\\times 10^\{\-3\}for theuucomponent and9\.00×10−49\.00\\times 10^\{\-4\}for thevvcomponent\. On 2D Darcy Flow, where operator\-learning methods are usually strong baselines, ADSL\-PDE obtains4\.20×10−34\.20\\times 10^\{\-3\}, substantially lower than FNO, U\-NO, and U\-Net\. These results suggest that ADSL\-PDE provides an effective design interface for solver evolution\.

![Refer to caption](https://arxiv.org/html/2608.04384v1/Figures/llm_effciency.png)Figure 4:Best\-so\-far improvement rates over evolution rounds using different LLM backbones\.
### Comparison with Search and LLM\-Agent Baselines

Table 2:Comparison with search\-based and prompt\-driven LLM\-agent baselines\. The reported metric is MSE or relativeL2L\_\{2\}error; lower is better\. Geo\. Mean summarizes performance across all evaluated tasks\.Table[2](https://arxiv.org/html/2608.04384#Sx4.T2)compares ADSL\-PDE with search\-based and prompt\-driven LLM\-agent baselines\. ADSL\-PDE achieves the best result on 8 out of 11 tasks and obtains the lowest geometric mean error across all methods\. Compared with RandomAgent and BayesianAgent, ADSL\-PDE consistently achieves much lower errors, showing that the improvement is not merely due to exploring more candidates\. Instead, the structured DSL provides a more informative and better constrained search space\.

Compared with prompt\-driven LLM\-agent baselines, ADSL\-PDE also achieves stronger overall performance\. For instance, on Poisson\-MA, ADSL\-PDE reduces the error to1\.24×10−41\.24\\times 10^\{\-4\}, while PINNsAgent and PINNacle obtain3\.163\.16and1\.831\.83, respectively\. ADSL\-PDE does not dominate every individual task: Lang\-PINN performs best on KS and Burgers\-C, while PINNacle performs best on Poisson\-ND\. Nevertheless, the overall geometric mean shows that ADSL\-PDE provides the strongest average performance across heterogeneous PDE settings\. This supports the central claim that a structured agent\-facing language improves the reliability and effectiveness of solver auto\-design\.

### Sensitivity to LLM Backbone

We evaluate ADSL\-PDE with Qwen\-3\.5\-Plus, GLM\-5\.1, and DeepSeek\-V4\-Pro under the same search budget and backend protocol\. As shown in Figure[4](https://arxiv.org/html/2608.04384#Sx4.F4), all three models produce consistent improvements as evolution proceeds\. DeepSeek\-V4\-Pro achieves the highest final improvement rate of approximately 52%, while Qwen\-3\.5\-Plus and GLM\-5\.1 reach approximately 42% and 41%, respectively\. Although the final values differ, all backbones discover substantially improved solvers and exhibit similar overall convergence patterns\.

These results suggest that the LLM backbone has a moderate, rather than dominant, influence on ADSL\-PDE\. By restricting generation to structured and valid solver\-level decisions, the language system reduces the dependence of the evolutionary process on a particular LLM\. Therefore, when a modest difference in final performance is acceptable, the LLM backbone can be selected primarily according to inference cost and availability\. A stronger model may still be preferred when maximizing the final solver performance is the primary objective\.

### Ablation of Key Designs

We evaluate the key components of ADSL\-PDE by comparing direct Python generation, structured specification, validated IR, backend support, and the complete system\. Table[3](https://arxiv.org/html/2608.04384#Sx4.T3)reports the average performance improvement per 100K tokens and the candidate pass rate under the same search budget\.

Table 3:Ablation study of the key designs in ADSL\-PDE\. Improvement per 100K tokens measures the average performance gain achieved for every 100K generated tokens\. Pass rate denotes the number of valid candidates among 30 generated candidates\. Higher values are better, and the best results are shown in bold\.Table 4:Search\-space ablation evaluating the semantic compression effect of ADSL\-PDE\. Task performance is measured by MSE, for which lower values are better\. VCR, SE, and Token\-SE measure search reliability and efficiency, for which higher values are better\.Table 5:Performance of different evolution backends with and without ADSL\-PDE\. Results are reported in MSE, and lower values indicate better performance\. Base denotes the original evolution backend\. The best result on each task is shown in bold, and↑\\uparrowindicates that ADSL\-PDE improves upon the corresponding base backend\. “–” denotes unavailable results\.Direct Python generation achieves an average improvement of 12\.10% per 100K tokens, with only 14 of 30 candidates passing the validity checks\. Structured specification slightly improves token efficiency but does not increase the pass rate, indicating that a structured format alone cannot ensure executable solver designs\. Introducing validated IR raises the pass rate to 23/30 by detecting invalid candidates before execution, although validation alone does not directly improve solver quality\. Backend support increases the average improvement to 21\.62%, showing the value of mapping solver specifications to reliable and reusable implementations\.

The complete ADSL\-PDE achieves the highest token\-normalized improvement of 30\.18% while retaining the joint\-best pass rate of 23/30\. These results indicate that structured representation, IR validation, and backend compilation provide complementary benefits\. Their integration improves both the reliability of generated candidates and the efficiency of neural PDE solver evolution\.

### Representation Ablation

We evaluate whether the language representation improves solver evolution by removing unproductive regions from the agent’s search space\. Table[4](https://arxiv.org/html/2608.04384#Sx4.T4)compares direct Python generation, plain configuration search, low\-level DSL search, and full ADSL\-PDE under the same PDE tasks, search budget, and backend protocol\. These representations progressively reduce implementation\-level freedom while preserving solver\-level design choices\.

Full ADSL\-PDE achieves the lowest geometric mean error and the highest VCR, SE, and Token\-SE\. Compared with direct Python generation, it reduces the geometric mean error from4\.27×10−34\.27\\times 10^\{\-3\}to1\.07×10−41\.07\\times 10^\{\-4\}, increases VCR from0\.340\.34to0\.910\.91, and improves SE and Token\-SE from0\.180\.18and0\.060\.06to0\.830\.83and0\.720\.72, respectively\. These results show that ADSL\-PDE concentrates exploration on valid and performance\-relevant solver decisions, thereby improving search efficiency\.

Figure[2](https://arxiv.org/html/2608.04384#Sx1.F2)further shows that DSL\-based evolution improves faster and fluctuates less than direct code generation\. Overall, the representation ablation confirms that reducing irrelevant implementation choices improves both the efficiency and reliability of neural PDE solver evolution\.

### Generality Across Evolution Backends

ADSL\-PDE is designed as a representation layer that can be combined with different evolutionary search methods\. We therefore compare OpenEvolve, PINNsAgent, and OpenAlpha\-Evolve with and without ADSL\-PDE on the same PDE tasks\. The results are reported in Table[5](https://arxiv.org/html/2608.04384#Sx4.T5)\.

ADSL\-PDE improves 10 of the 12 available task–backend comparisons\. The improvement is most consistent with OpenEvolve, for which ADSL\-PDE reduces the error on all five tasks\. When combined with PINNsAgent, ADSL\-PDE improves four of the five tasks, with only a slight degradation on Burgers\-C\. For OpenAlpha\-Evolve, it improves Burgers but does not improve Burgers\-C\.

These results indicate that the effectiveness of ADSL\-PDE is not tied to a specific evolution algorithm\. By providing a structured and constrained solver representation, ADSL\-PDE can generally improve the search performance of different evolutionary backends\. Nevertheless, the gains are task\-dependent and are not guaranteed for every backend–task combination\.

## Discussion

Our results show that ADSL\-PDE is an effective representation layer for neural PDE solver auto\-design\. Its main benefit come from reshaping the search space\. By replacing unrestricted Python generation with structured solver\-level decisions, ADSL\-PDE removes many invalid or implementation\-irrelevant candidates and concentrates agent exploration on choices that directly affect solver performance\. This leads to higher candidate validity, better token efficiency, and more stable evolution across different PDE tasks, LLM backbones, and evolutionary frameworks\.

The study also suggests several principles for designing effective agent\-oriented languages\. First, the language should expose domain\-relevant decisions, such as solver family, architecture, physical constraints, sampling, and training, while hiding low\-level implementation details\. Second, it should provide controlled expressiveness: overly rigid representations restrict innovation, whereas overly general code spaces introduce unnecessary search dimensions\. Third, static validation and deterministic compilation are essential for rejecting invalid candidates before expensive training and for ensuring reproducibility\.

These findings support the central claim of this work: improving neural PDE solver auto\-design requires not only stronger agents, but also a language system that defines a compact, valid, and semantically effective search space\. ADSL\-PDE provides one realization of this principle and offers a general direction for language\-guided scientific solver evolution\. Future work will systematically evaluate how language systems affect LLM\-agent performance across diverse tasks and theoretically derive bounds on their improvements in task completion\.

## References

- L\. Beurer\-Kellner, M\. Fischer, and M\. Vechev \(2023\)Prompting is programming: a query language for large language models\.InProceedings of the 44th ACM SIGPLAN Conference on Programming Language Design and Implementation,External Links:[Document](https://dx.doi.org/10.1145/3591300)Cited by:[Agent\-Oriented Languages and Structured Representations](https://arxiv.org/html/2608.04384#Sx2.SSx3.p1.1)\.
- J\. Cen and Q\. Zou \(2024\)Deep finite volume method for partial differential equations\.Journal of Computational Physics517,pp\. 113307\.External Links:[Document](https://dx.doi.org/10.1016/j.jcp.2024.113307)Cited by:[Baselines\.](https://arxiv.org/html/2608.04384#Sx4.SSx1.SSS0.Px2.p1.1)\.
- J\. Chen, X\. Chi, W\. E, and Z\. Yang \(2022\)Bridging traditional and machine learning\-based algorithms for solving PDEs: the random feature method\.Journal of Machine Learning1,pp\. 268–298\.External Links:[Document](https://dx.doi.org/10.4208/jml.220726)Cited by:[Baselines\.](https://arxiv.org/html/2608.04384#Sx4.SSx1.SSS0.Px2.p1.1)\.
- T\. Elsken, J\. H\. Metzen, and F\. Hutter \(2019\)Neural architecture search: a survey\.Journal of Machine Learning Research20,pp\. 1–21\.Cited by:[Introduction](https://arxiv.org/html/2608.04384#Sx1.p1.1),[Automated Design of Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx2.p1.1)\.
- M\. Fowler \(2010\)Domain\-specific languages\.Addison\-Wesley Professional\.Cited by:[Introduction](https://arxiv.org/html/2608.04384#Sx1.p2.1),[Agent\-Oriented Languages and Structured Representations](https://arxiv.org/html/2608.04384#Sx2.SSx3.p1.1)\.
- L\. Gao, A\. Madaan, S\. Zhou, U\. Alon, P\. Liu, Y\. Yang, J\. Callan, and G\. Neubig \(2023\)PAL: program\-aided language models\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 10764–10799\.Cited by:[Agent\-Oriented Languages and Structured Representations](https://arxiv.org/html/2608.04384#Sx2.SSx3.p1.1)\.
- Z\. Hao, J\. Yao, C\. Su, H\. Su, Z\. Wang, F\. Lu, Z\. Xia, Y\. Zhang, S\. Liu, L\. Lu, and J\. Zhu \(2024\)PINNacle: a comprehensive benchmark of physics\-informed neural networks for solving PDEs\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 76721–76774\.External Links:[Document](https://dx.doi.org/10.52202/079017-2442)Cited by:[Baselines\.](https://arxiv.org/html/2608.04384#Sx4.SSx1.SSS0.Px2.p1.1)\.
- X\. He, L\. You, H\. Tian, B\. Han, I\. Tsang, and Y\. Ong \(2025\)Lang\-PINN: from language to physics\-informed neural networks via a multi\-agent framework\.External Links:2510\.05158,[Document](https://dx.doi.org/10.48550/arXiv.2510.05158)Cited by:[Automated Design of Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx2.p1.1),[Baselines\.](https://arxiv.org/html/2608.04384#Sx4.SSx1.SSS0.Px2.p1.1)\.
- M\. Kim, C\. Hsu, D\. Wang, S\. Garg, V\. Kumar, and M\. K\. Ramanathan \(2026\)CODESTRUCT: code agents over structured action spaces\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics,pp\. 13290–13306\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.607)Cited by:[Agent\-Oriented Languages and Structured Representations](https://arxiv.org/html/2608.04384#Sx2.SSx3.p1.1)\.
- Q\. Li and J\. Chen \(2025\)An unsupervised network architecture search method for solving partial differential equations\.Note:Introduces PINN\-DARTSExternal Links:2504\.11140,[Document](https://dx.doi.org/10.48550/arXiv.2504.11140)Cited by:[Automated Design of Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx2.p1.1)\.
- S\. Li, T\. Marwah, J\. Shen, W\. Sun, A\. Risteski, Y\. Yang, and A\. Talwalkar \(2025\)CodePDE: an inference framework for LLM\-driven PDE solver generation\.External Links:2505\.08783,[Document](https://dx.doi.org/10.48550/arXiv.2505.08783)Cited by:[Automated Design of Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx2.p1.1)\.
- Z\. Li, N\. Kovachki, K\. Azizzadenesheli, B\. Liu, K\. Bhattacharya, A\. Stuart, and A\. Anandkumar \(2021\)Fourier neural operator for parametric partial differential equations\.InInternational Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2608.04384#Sx1.p1.1),[Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx1.p1.1),[Baselines\.](https://arxiv.org/html/2608.04384#Sx4.SSx1.SSS0.Px2.p1.1)\.
- Z\. Li, H\. Zheng, N\. Kovachki, D\. Jin, H\. Chen, B\. Liu, K\. Azizzadenesheli, and A\. Anandkumar \(2024\)Physics\-informed neural operator for learning partial differential equations\.ACM/IMS Journal of Data Science1\(3\),pp\. 1–27\.External Links:[Document](https://dx.doi.org/10.1145/3648506)Cited by:[Baselines\.](https://arxiv.org/html/2608.04384#Sx4.SSx1.SSS0.Px2.p1.1)\.
- L\. Lu, P\. Jin, G\. Pang, Z\. Zhang, and G\. E\. Karniadakis \(2021\)Learning nonlinear operators via deeponet based on the universal approximation theorem of operators\.Nature Machine Intelligence3,pp\. 218–229\.Cited by:[Introduction](https://arxiv.org/html/2608.04384#Sx1.p1.1),[Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx1.p1.1)\.
- A\. Novikov, N\. V u, M\. Eisenberger, E\. Dupont, P\. Huang, A\. Z\. Wagner, S\. Shirobokov, B\. Kozlovskii, F\. J\. R\. Ruiz, A\. Mehrabian, M\. P\. Kumar, A\. See, S\. Chaudhuri, G\. Holland, A\. Davies, S\. Nowozin, P\. Kohli, and M\. Balog \(2025\)AlphaEvolve: a coding agent for scientific and algorithmic discovery\.External Links:2506\.13131,[Document](https://dx.doi.org/10.48550/arXiv.2506.13131)Cited by:[Automated Design of Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx2.p1.1)\.
- L\. Pan, A\. Albalak, X\. Wang, and W\. Wang \(2023\)Logic\-LM: empowering large language models with symbolic solvers for faithful logical reasoning\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 3806–3824\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.248)Cited by:[Agent\-Oriented Languages and Structured Representations](https://arxiv.org/html/2608.04384#Sx2.SSx3.p1.1)\.
- T\. Parr \(1998\)A handbook for language designers\.Technical Report\.Cited by:[Introduction](https://arxiv.org/html/2608.04384#Sx1.p2.1)\.
- N\. Peleg\-Pelc, G\. A\. Kaminka, and Y\. Goldberg \(2026\)A language for describing agentic LLM contexts\.InProceedings of the ACM Conference on AI and Agentic Systems,External Links:[Document](https://dx.doi.org/10.1145/3786335.3813126)Cited by:[Agent\-Oriented Languages and Structured Representations](https://arxiv.org/html/2608.04384#Sx2.SSx3.p1.1)\.
- T\. Pfaff, M\. Fortunato, A\. Sanchez\-Gonzalez, and P\. W\. Battaglia \(2021\)Learning mesh\-based simulation with graph networks\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=roNqYL0_XP)Cited by:[Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx1.p1.1),[Baselines\.](https://arxiv.org/html/2608.04384#Sx4.SSx1.SSS0.Px2.p1.1)\.
- M\. A\. Rahman, Z\. E\. Ross, and K\. Azizzadenesheli \(2022\)U\-NO: U\-shaped neural operators\.External Links:2204\.11127,[Document](https://dx.doi.org/10.48550/arXiv.2204.11127)Cited by:[Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx1.p1.1),[Baselines\.](https://arxiv.org/html/2608.04384#Sx4.SSx1.SSS0.Px2.p1.1)\.
- M\. Raissi, P\. Perdikaris, and G\. E\. Karniadakis \(2019\)Physics\-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations\.Journal of Computational Physics378,pp\. 686–707\.Cited by:[Introduction](https://arxiv.org/html/2608.04384#Sx1.p1.1),[Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx1.p1.1),[Baselines\.](https://arxiv.org/html/2608.04384#Sx4.SSx1.SSS0.Px2.p1.1)\.
- E\. Real, A\. Aggarwal, Y\. Huang, and Q\. V\. Le \(2019\)Regularized evolution for image classifier architecture search\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.33,pp\. 4780–4789\.Cited by:[Introduction](https://arxiv.org/html/2608.04384#Sx1.p1.1),[Automated Design of Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx2.p1.1)\.
- F\. K\. Safa, Y\. Jiang, and X\. Zheng \(2026\)An llm\-driven scenario generation pipeline using an extended scenic dsl for autonomous driving safety validation\.arXiv preprint arXiv:2602\.20644\.Cited by:[Agent\-Oriented Languages and Structured Representations](https://arxiv.org/html/2608.04384#Sx2.SSx3.p1.1)\.
- T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. Scialom \(2023\)Toolformer: language models can teach themselves to use tools\.Advances in neural information processing systems36,pp\. 68539–68551\.Cited by:[Introduction](https://arxiv.org/html/2608.04384#Sx1.p1.1)\.
- A\. Sharma \(2025a\)OpenEvolve: an open\-source evolutionary coding agentExternal Links:[Link](https://github.com/algorithmicsuperintelligence/openevolve)Cited by:[Language\-Guided Evolution Framework](https://arxiv.org/html/2608.04384#Sx3.SSx4.p4.1)\.
- A\. Sharma \(2025b\)OpenEvolve: an open\-source evolutionary coding agentExternal Links:[Link](https://github.com/algorithmicsuperintelligence/openevolve)Cited by:[Automated Design of Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx2.p1.1)\.
- M\. Takamoto, T\. Praditia, R\. Leiteritz, D\. MacKinlay, F\. Alesiani, D\. Pfl"uger, and M\. Niepert \(2022\)PDEBench: an extensive benchmark for scientific machine learning\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 1596–1611\.Cited by:[Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx1.p1.1)\.
- S\. Wang, S\. Sankaran, H\. Wang, and P\. Perdikaris \(2023a\)An expert’s guide to training physics\-informed neural networks\.External Links:2308\.08468,[Document](https://dx.doi.org/10.48550/arXiv.2308.08468)Cited by:[Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx1.p1.1)\.
- S\. Wang, Y\. Teng, and P\. Perdikaris \(2021\)Understanding and mitigating gradient flow pathologies in physics\-informed neural networks\.SIAM Journal on Scientific Computing43\(5\),pp\. A3055–A3081\.External Links:[Document](https://dx.doi.org/10.1137/20M1318043)Cited by:[Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx1.p1.1)\.
- Y\. Wang, X\. Han, C\. Chang, D\. Zha, U\. Braga\-Neto, and X\. Hu \(2023b\)Auto\-PINN: understanding and optimizing physics\-informed neural architecture\.InNeurIPS 2023 Workshop on AI for Science,Note:arXiv:2205\.13748External Links:[Link](https://openreview.net/forum?id=xJFITn0hRx)Cited by:[Automated Design of Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx2.p1.1)\.
- Y\. Wang and L\. Zhong \(2024\)NAS\-PINN: neural architecture search\-guided physics\-informed neural network for solving PDEs\.Journal of Computational Physics496,pp\. 112603\.External Links:[Document](https://dx.doi.org/10.1016/j.jcp.2023.112603)Cited by:[Automated Design of Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx2.p1.1)\.
- P\. Wei, M\. Liu, J\. Cen, Z\. Zhou, L\. Chen, and Q\. Zou \(2024\)PDENNEval: a comprehensive evaluation of neural network methods for solving PDEs\.InProceedings of the Thirty\-Third International Joint Conference on Artificial Intelligence,pp\. 5181–5189\.External Links:[Document](https://dx.doi.org/10.24963/ijcai.2024/573)Cited by:[Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx1.p1.1)\.
- Q\. Wuwu, C\. Gao, T\. Chen, Y\. Huang, Y\. Zhang, J\. Wang, J\. Li, H\. Zhou, and S\. Zhang \(2025\)PINNsAgent: automated pde surrogation with large language models\.arXiv preprint arXiv:2501\.12053\.Cited by:[Introduction](https://arxiv.org/html/2608.04384#Sx1.p1.1),[Automated Design of Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx2.p1.1),[Baselines\.](https://arxiv.org/html/2608.04384#Sx4.SSx1.SSS0.Px2.p1.1)\.
- Z\. Yang, Y\. Liu, J\. Fu, P\. Fu, B\. Han, M\. Sugiyama, and N\. Zheng \(2026\)Shaping schema via language representation as the next frontier for LLM intelligence expanding\.External Links:2605\.09271,[Document](https://dx.doi.org/10.48550/arXiv.2605.09271)Cited by:[Agent\-Oriented Languages and Structured Representations](https://arxiv.org/html/2608.04384#Sx2.SSx3.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.International Conference on Learning Representations\.Cited by:[Introduction](https://arxiv.org/html/2608.04384#Sx1.p1.1)\.
- J\. Zhang, T\. Wang, H\. Wu, Z\. Huang, Y\. Wu, D\. Chen, L\. Song, Y\. Zhang, G\. Rao, and K\. Yu \(2025\)SR\-LLM: rethinking the structured representation in large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics,pp\. 3443–3462\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.172)Cited by:[Agent\-Oriented Languages and Structured Representations](https://arxiv.org/html/2608.04384#Sx2.SSx3.p1.1)\.
- B\. Zoph and Q\. V\. Le \(2017\)Neural architecture search with reinforcement learning\.InInternational Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2608.04384#Sx1.p1.1),[Automated Design of Neural PDE Solvers](https://arxiv.org/html/2608.04384#Sx2.SSx2.p1.1)\.