EvoPINN: Agentic Discovery of Executable Algorithms for Physics-Informed Neural Networks

# EvoPINN: Agentic Discovery of Executable Algorithms for Physics-Informed Neural Networks
Source: [https://arxiv.org/html/2607.26490](https://arxiv.org/html/2607.26490)
Peng YinSchool of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, ChinaInstitute of Automation, Chinese Academy of Sciences, Beijing, ChinaYifan ZhangInstitute of Automation, Chinese Academy of Sciences, Beijing, ChinaJian ChengInstitute of Automation, Chinese Academy of Sciences, Beijing, China

###### Abstract

Physics\-informed neural networks \(PINNs\) have emerged as a powerful paradigm for solving partial differential equations \(PDEs\), yet their performance heavily relies on the manual, trial\-and\-error engineering of neural representations, loss formulations, and optimization dynamics\. While Large Language Models \(LLMs\) offer a promising avenue for automated design, unconstrained code generation often yields mathematically invalid or numerically unstable solutions under strict scientific computing constraints\. To bridge this gap, we proposeEvoPINN, an agentic framework that reformulates PINN development from labor\-intensive manual design into a rigorous, execution\-grounded algorithm discovery problem\. EvoPINN navigates a modular search space by decoupling neural representations from training programs, utilizing an LLM agent to iteratively propose memory\-conditioned programmatic modifications\. To ensure scientific validity, all candidates undergo strict structural verification and budget\-matched PDE evaluation\. Extensive experiments across diverse PDE regimes \(oscillatory, elliptic, dissipative, and nonlinear transport\) demonstrate that EvoPINN discovers PDE\-specialized learning algorithms that significantly reduce relativeL2L\_\{2\}error compared to baselines\. Crucially, EvoPINN autonomously invented SLRC\-PINN, a novel architecture whose performance gains persist under rigorous parameter\-matched comparisons, establishing the viability of execution\-grounded agents for discovering genuinely new scientific computing mechanisms\.

## 1Introduction

Physics\-informed neural networks \(PINNs\) provide a flexible, mesh\-free framework for solving partial differential equations \(PDEs\) by incorporating physical laws into neural\-network trainingRaissiet al\.\([2019](https://arxiv.org/html/2607.26490#bib.bib1)\); Luet al\.\([2021](https://arxiv.org/html/2607.26490#bib.bib2)\)\. Despite their conceptual elegance, their performance depends strongly on PDE\-specific choices regarding coordinate representations, network architectures, loss formulations, collocation sampling, and optimization algorithms\. Different PDE regimes often favor different mechanisms: oscillatory solutions benefit from frequency\-aware representations, whereas nonlinear transport problems require stable optimization and enhanced resolution near steep gradients\. Consequently, adapting PINNs to these distinct physical behaviors requires structural and programmatic interventions, making it a labor\-intensive algorithm design problem with no single design dominating across diverse PDEsWanget al\.\([2021a](https://arxiv.org/html/2607.26490#bib.bib4),[2022](https://arxiv.org/html/2607.26490#bib.bib5)\); Haoet al\.\([2024](https://arxiv.org/html/2607.26490#bib.bib3)\)\.

Existing automated PINN methods primarily focus on tuning hyperparameters or assembling predefined components within human\-specified search spacesWanget al\.\([2023](https://arxiv.org/html/2607.26490#bib.bib37)\); Wang and Zhong \([2024](https://arxiv.org/html/2607.26490#bib.bib38)\); Zhang and Yang \([2024](https://arxiv.org/html/2607.26490#bib.bib39)\)\. More recently, Large Language Model \(LLM\)\-based systems \(e\.g\., PINNsAgentWuwuet al\.\([2025](https://arxiv.org/html/2607.26490#bib.bib30)\), Lang\-PINNHeet al\.\([2025](https://arxiv.org/html/2607.26490#bib.bib31)\)\) have advanced this frontier by automating PINN pipeline construction and configuration selection via natural\-language reasoning and experimental feedback\. While these methods substantially reduce human effort, their exploration remains fundamentally constrained by human\-specified templates and bounded design spaces\. They excel at identifying optimal configurations among known alternatives, but do not readily modify the neural representation and training logic that jointly define the PINN learning algorithm\. Consequently, the broader ambition—discovering genuinely new, executable learning mechanisms—remains largely unexplored for PINNs\. Furthermore, directly leveraging LLMs for open\-ended algorithmic generation in scientific computing presents a critical challenge: unconstrained LLMs frequently produce mathematically invalid, structurally redundant, or numerically unstable code when subjected to the strict physical and computational constraints of PDEs\.

To bridge this gap, we introduceEvoPINN, an execution\-grounded agentic framework for automatically discovering complete PINN learning algorithms\. To tame the instability of open\-ended LLM generation and solve the credit assignment problem during evolution, EvoPINN employs a modular design that decouples the neural representation module from the physics\-informed training program\. An LLM agent iteratively proposes memory\-guided programmatic modifications to one module at a time\. Crucially, EvoPINN ensures scientific validity by subjecting all proposals to a strict verification pipeline—including structural verification, executable testing, and budget\-matched PDE evaluation—updating the algorithm only when a candidate exhibits structural novelty and reduces solution error\. This controlled search process enables EvoPINN to explore representation and training mechanisms beyond a predefined library of PINN configurations\.

We evaluate EvoPINN across oscillatory, elliptic, dissipative, and nonlinear\-transport PDE regimes\. The discovered algorithms consistently improve upon the baseline PINN and achieve performance comparable to or better than strong expert\-designed methods across diverse problems\. Furthermore, our evaluations confirm the robustness of the discovered mechanisms, demonstrating applicability to unseen equation parameters\. Most notably, EvoPINN discovers SLRC\-PINN for Burgers1D—a global–local architecture in which an adaptive bank of localized bases gates a zero\-initialized corrective branch\. To our knowledge, this specific topology has not been reported, and its advantage persists under parameter\-matched comparisons with related PINN architectures\. Together, these results demonstrate that execution\-grounded agents can move automated PINN design beyond hyperparameter tuning toward discovering novel, effective, and reusable learning mechanisms\.

![Refer to caption](https://arxiv.org/html/2607.26490v1/x1.png)Figure 1:Overview of EvoPINN\. A PINN algorithm is represented asA=\(M,T\)A=\(M,T\), whereMMdefines the neural representation andTTdefines the training program\. At each generation, EvoPINN selects one module for modification, uses the current training behavior and earlier attempts to guide the LLM, and updates the current algorithm only when a generated program runs successfully under the shared training budget and reduces the solution error\.
## 2Related Work

### 2\.1Manual Mechanism Design in PINNs

A large body of work improves Physics\-Informed Neural Networks \(PINNs\) through manually designed representation topologies and training algorithms\. Representation\-oriented methods introduce adaptive activation functionsJagtapet al\.\([2020](https://arxiv.org/html/2607.26490#bib.bib9)\), frequency\-aware feature encodings for high\-frequency or multiscale dynamicsSitzmannet al\.\([2020](https://arxiv.org/html/2607.26490#bib.bib12)\); Wanget al\.\([2021b](https://arxiv.org/html/2607.26490#bib.bib11)\), spatial feature embeddingsTanciket al\.\([2020](https://arxiv.org/html/2607.26490#bib.bib40)\), and domain\-decomposition architecturesJagtap and Karniadakis \([2020](https://arxiv.org/html/2607.26490#bib.bib14)\); Moseleyet al\.\([2023](https://arxiv.org/html/2607.26490#bib.bib15)\)\. Training\-oriented methods focus on adaptive collocation sampling strategiesDawet al\.\([2022](https://arxiv.org/html/2607.26490#bib.bib41)\); Wuet al\.\([2023](https://arxiv.org/html/2607.26490#bib.bib6)\), residual objective modificationsYuet al\.\([2022](https://arxiv.org/html/2607.26490#bib.bib8)\), dynamic loss balancingWanget al\.\([2021a](https://arxiv.org/html/2607.26490#bib.bib4),[2022](https://arxiv.org/html/2607.26490#bib.bib5)\), and multi\-stage or curriculum optimization schemesKrishnapriyanet al\.\([2021](https://arxiv.org/html/2607.26490#bib.bib42)\); Yaoet al\.\([2023](https://arxiv.org/html/2607.26490#bib.bib7)\)\. Comprehensive benchmark evaluations demonstrate that no single manually designed PINN variant uniformly dominates across diverse physical regimesHaoet al\.\([2024](https://arxiv.org/html/2607.26490#bib.bib3)\)\. Consequently, adapting PINNs to specific physical behaviors remains an arduous, trial\-and\-error engineering process, underscoring the critical imperative to transition from hand\-crafted mechanisms to automated algorithmic discovery\.

### 2\.2Automated and Agentic PINN Design

Earlier automated PINN design methods focus on optimizing discrete structural or numerical choices within predefined, highly restricted search spaces\. Auto\-PINN performs staged hyperparameter search over network depth, width, and optimizer configurationsWanget al\.\([2023](https://arxiv.org/html/2607.26490#bib.bib37)\), whereas NAS\-PINN utilizes differentiable architecture search to select optimal connectivity patternsWang and Zhong \([2024](https://arxiv.org/html/2607.26490#bib.bib38)\)\. Beyond standard NAS, classical evolutionary computation approaches, such as DPSTEZhang and Yang \([2024](https://arxiv.org/html/2607.26490#bib.bib39)\), explore network structures alongside tree\-based parametric activation functions using dynamic population scheduling\. More recently, LLM\-driven agentic frameworks extend automation to workflow and pipeline construction: PINNsAgent leverages empirical execution feedback to navigate architectures and hyperparameter configurationsWuwuet al\.\([2025](https://arxiv.org/html/2607.26490#bib.bib30)\), while Lang\-PINN generates executable code pipelines directly from natural\-language PDE specificationsHeet al\.\([2025](https://arxiv.org/html/2607.26490#bib.bib31)\)\. However, these approaches remain constrained to selecting configurations, tuning hyperparameters, or composing workflows within pre\-engineered templates\. EvoPINN instead enables the open\-ended discovery of novel PINN learning mechanisms\.

### 2\.3LLM\-Guided Solver and Algorithm Discovery

LLMs have increasingly been used to generate and refine scientific programs\. In the PDE domain, CodePDE and PDE\-SHARP synthesize solver implementations through reasoning or execution feedback, but focus on general PDE solvers rather than PINN\-specific learning algorithmsLiet al\.\([2025](https://arxiv.org/html/2607.26490#bib.bib32)\); Fazliani and Udell \([2025](https://arxiv.org/html/2607.26490#bib.bib33)\)\. Beyond PDE solving, LLM\-guided executable search has been applied to mathematical programs, combinatorial heuristics, reward functions, and optimization algorithmsRomera\-Paredeset al\.\([2024](https://arxiv.org/html/2607.26490#bib.bib26)\); Liuet al\.\([2024](https://arxiv.org/html/2607.26490#bib.bib27)\); Maet al\.\([2024](https://arxiv.org/html/2607.26490#bib.bib24)\); Yeet al\.\([2024](https://arxiv.org/html/2607.26490#bib.bib28)\)\. These methods typically evolve a bounded component, such as a heuristic, reward, or update function, within a fixed algorithmic scaffold\. EvoPINN breaks this limitation by elevating the search object to the complete PINN learning algorithm, which is significantly more challenging\.

## 3Preliminaries of PINNs

Consider a PDE instance𝒫\\mathcal\{P\}defined on a spatio\-temporal domainΩ\\Omega, governed by a differential operatorℱ​\[u\]=0\\mathcal\{F\}\[u\]=0alongside boundary and initial constraintsℬ​\[u\]=0\\mathcal\{B\}\[u\]=0andℐ​\[u\]=0\\mathcal\{I\}\[u\]=0\. A PINN approximates the solution using a differentiable neural networkuθu\_\{\\theta\}parameterized byθ\\theta, and is trained by minimizing a composite physics\-informed loss:

ℒPINN​\(θ\)=λr​ℒr​\(θ\)\+λb​ℒb​\(θ\)\+λi​ℒi​\(θ\),\\mathcal\{L\}\_\{\\mathrm\{PINN\}\}\(\\theta\)=\\lambda\_\{r\}\\mathcal\{L\}\_\{r\}\(\\theta\)\+\\lambda\_\{b\}\\mathcal\{L\}\_\{b\}\(\\theta\)\+\\lambda\_\{i\}\\mathcal\{L\}\_\{i\}\(\\theta\),\(1\)whereℒr\\mathcal\{L\}\_\{r\},ℒb\\mathcal\{L\}\_\{b\}andℒi\\mathcal\{L\}\_\{i\}denote the PDE\-residual, boundary\-condition, and initial\-condition losses, respectively, balanced by their corresponding weightsλr\\lambda\_\{r\},λb\\lambda\_\{b\}, andλi\\lambda\_\{i\}\. In standard practice, both the architectural form ofuθu\_\{\\theta\}and the procedure used to optimize this objective are rigidly specified*a priori*\. It is precisely these two static components—the neural representation and the optimization dynamics—that our framework seeks to autonomously discover\. More mathematical details about PINNs are provided in Appendix A\.

## 4Method

### 4\.1Overview

EvoPINN shifts the paradigm of PINN development from manual design to executable algorithm discovery\. We formally define a PINN algorithm as a tuple:

A=\(M,T\)∈𝒜=ℳ×𝒯,A=\(M,T\)\\in\\mathcal\{A\}=\\mathcal\{M\}\\times\\mathcal\{T\},\(2\)whereMMencompasses the neural representation of the solution, andTTgoverns the program used to train it under the physical constraints\. Together,ℳ\\mathcal\{M\}and𝒯\\mathcal\{T\}constitute an open\-ended algorithmic search space𝒜\\mathcal\{A\}\.

For a target PDE𝒫\\mathcal\{P\}, a training budgetBB, and a random seedξ\\xi, executing an algorithmAAyields the optimized parameters:

θA​\(𝒫,B,ξ\)=Train⁡\(A;𝒫,B,ξ\)\.\\theta\_\{A\}\(\\mathcal\{P\},B,\\xi\)=\\operatorname\{Train\}\(A;\\mathcal\{P\},B,\\xi\)\.\(3\)The efficacy of candidate algorithms is then quantified by their relativeL2L\_\{2\}error against a reference solutionu∗u^\{\*\}on a search reference set𝒳s​e​a​r​c​h\\mathcal\{X\}\_\{search\}:

Ssearch​\(A;𝒫,B,ξ\)=‖uθA​\(𝒫,B,ξ\)​\(𝒳search\)−u∗​\(𝒳search\)‖2‖u∗​\(𝒳search\)‖2\.\\small S\_\{\\mathrm\{search\}\}\(A;\\mathcal\{P\},B,\\xi\)=\\frac\{\\left\\\|u\_\{\\theta\_\{A\}\(\\mathcal\{P\},B,\\xi\)\}\(\\mathcal\{X\}\_\{\\mathrm\{search\}\}\)\-u^\{\*\}\(\\mathcal\{X\}\_\{\\mathrm\{search\}\}\)\\right\\\|\_\{2\}\}\{\\left\\\|u^\{\*\}\(\\mathcal\{X\}\_\{\\mathrm\{search\}\}\)\\right\\\|\_\{2\}\}\.\(4\)For brevity, we write this score asSsearch​\(A\)S\_\{\\mathrm\{search\}\}\(A\)when the PDE, budget, and random seed are clear\.

As illustrated in Figure[1](https://arxiv.org/html/2607.26490#S1.F1), EvoPINN operates through an LLM\-guided evolutionary loop\. The framework initializes with a seed PINN and iteratively refines it overGGgenerations\. In each generationgg, an LLM agent generatesKKprogrammatic modifications, evaluates these candidates under a shared budget, utilizing execution feedback to drive a continuous cycle of algorithmic generation and refinement\.

Utilizing LLMs for evolutionary search has recently shown promise in many problemsLiuet al\.\([2024](https://arxiv.org/html/2607.26490#bib.bib27)\); Romera\-Paredeset al\.\([2024](https://arxiv.org/html/2607.26490#bib.bib26)\)\. To establish the viability of LLM\-driven PINN algorithm discovery without confounding the performance gains with complex meta\-heuristics, we deliberately adopt the straightforward evolutionary loop described above\. However, navigating an open\-ended space that encompasses both neural representations and optimization dynamics introduces unprecedented complexity\. The innovation of EvoPINN lies in the specialized mechanisms designed to make searching this vast, physics\-constrained algorithmic space tractable and scientifically valid\. Specifically, realizing this open\-ended search presents three fundamental challenges:

- •Credit Assignment in Joint Spaces:Modifying both the neural representation and the training program simultaneously obfuscates the attribution of performance gains\. To isolate the impact of each programmatic change, Section[4\.2](https://arxiv.org/html/2607.26490#S4.SS2)introduces an adaptively scheduled, one\-module\-at\-a\-time evolution strategy\.
- •Sparse Reward Guidance for Program Generation:A scalarL2L\_\{2\}provides little actionable feedback to guide the LLM’s proposals\. Section[4\.3](https://arxiv.org/html/2607.26490#S4.SS3)bridges this gap by using training diagnostics and evolutionary memory to inform targeted programmatic mutations\.
- •Algorithmic Validity and Evaluation Fairness:Unconstrained LLMs frequently generate code that is structurally redundant, mathematically invalid, or computationally disproportionate\. Section[4\.4](https://arxiv.org/html/2607.26490#S4.SS4)establishes a rigorous structural verification and budget\-matched evaluation protocol to ensure that all discoveries are both genuinely novel and rigorously validated\.

### 4\.2Module\-Wise Evolution and Adaptive Scheduling

Changing the representation and training program simultaneously makes improvements difficult to attribute\. EvoPINN therefore adopts module\-wise evolution: at each generation, it modifies either the representation moduleMMor the training\-program moduleTTwhile keeping the other fixed\.

At generationgg, let the current algorithm beAg=\(Mg,Tg\)A\_\{g\}=\(M\_\{g\},T\_\{g\}\), and letcg∈\{M,T\}c\_\{g\}\\in\\\{M,T\\\}denote the module selected for modification\. Fori=1,…,Ki=1,\\ldots,K, theiith candidate is

Ag\(i\)=\{\(M~g\(i\),Tg\),cg=M,\(Mg,T~g\(i\)\),cg=T,A\_\{g\}^\{\(i\)\}=\\begin\{cases\}\(\\widetilde\{M\}\_\{g\}^\{\(i\)\},T\_\{g\}\),&c\_\{g\}=M,\\\\ \(M\_\{g\},\\widetilde\{T\}\_\{g\}^\{\(i\)\}\),&c\_\{g\}=T,\\end\{cases\}\(5\)where the tilde denotes a newly generated replacement for the selected module\. This decoupling makes performance changes easier to associate with the modified module\. Over successive generations, accepted changes in one module naturally alter the optimization landscape for the other, allowing complex, synergistic PINN algorithms to emerge iteratively\.

Because the relative benefit of modifying each module can change during search, fixed alternation may allocate evaluations inefficiently\. EvoPINN instead uses a UCB\-style scheduler to dynamically determine which module to evolve next\. After each completed generation, the selected module receives a reward that reflects the outcome and quality of its candidate batch\. This reward incorporates key evaluation metrics, such as the magnitude of the improvement and the diversity or novelty of the generated mechanisms \(details in Appendix B\)\.

Letr¯g​\(c\)\\bar\{r\}\_\{g\}\(c\)denote the mean recent reward of modulecc, and letNg,cN\_\{g,c\}denote its number of recent observations\. EvoPINN dynamically selects the next module to evolve according to:

cg=arg⁡maxc∈\{M,T\}⁡\[r¯g​\(c\)\+β​log⁡\(Ng\+1\)Ng,c\],c\_\{g\}=\\arg\\max\_\{c\\in\\\{M,T\\\}\}\\left\[\\bar\{r\}\_\{g\}\(c\)\+\\beta\\sqrt\{\\frac\{\\log\(N\_\{g\}\+1\)\}\{N\_\{g,c\}\}\}\\right\],\(6\)whereNg=Ng,M\+Ng,TN\_\{g\}=N\_\{g,M\}\+N\_\{g,T\}\. The first term favors modules with stronger recent rewards, while the second favors less explored modules, withβ\>0\\beta\>0balancing exploitation and exploration\. To ensure proper initialization, any untried module is deterministically selected before this rule is applied\.

### 4\.3Diagnosis\-Guided Program Generation

After EvoPINN selects a module, it must determine how that module should be changed\. A scalar relativeL2L\_\{2\}error provides sparse, non\-actionable guidance for generating proposals\. To bridge this semantic gap, EvoPINN conditions the LLM’s generation process on a rich, structured context comprising three key components: training diagnostics, evolutionary memory, and diversified search focuses\.

Training Diagnostics\.Rather than relying solely on the final solution error, EvoPINN condenses the training trace into a compact diagnostic summaryDgD\_\{g\}\. This diagnosis captures convergence progress, late\-stage stability, and the relation between the physics losses and solution error\. After module selection,DgD\_\{g\}is supplied to the LLM to guide targeted modifications within the selected module\. For example, it may motivate richer basis structures during representation search or revised optimizer staging and collocation policies during training\-program search\. Detailed methods for constructingDgD\_\{g\}are provided in Appendix B\.

Evolutionary Memory\.To prevent redundant exploration, the LLM is provided with an evolutionary memory\. This memory bank archives previously successful architectural motifs, identifies persistently failing proposal families, and logs recent modifications applied to the selected module\. By maintaining this historical context, the LLM is constrained to build upon promising trajectories rather than repeatedly proposing known suboptimal configurations\.

Diversified Search Focuses\.Naive parallel LLM generation frequently suffers from mode collapse, yielding similar programs\. To enforce batch diversity, EvoPINN injects a distinct search focus into each of theKKLLM calls\. For the representation module, these focuses might direct the LLM toward feature construction, architectural organization, or coordinate transformation\. For the training program, focuses may emphasize optimization dynamics, loss coordination, or collocation sampling strategies\. These focuses operate as high\-level exploratory priors, encouraging structural diversity without rigidly prescribing specific implementations\.

### 4\.4Program Verification and Algorithm Update

A critical challenge in open\-ended algorithmic search is that unconstrained LLM generation frequently yields code that is structurally redundant, mathematically invalid, or computationally unfair\. To ensure that only genuinely novel and strictly beneficial mechanisms are retained, EvoPINN subjects all candidate programs to a rigorous, multi\-stage verification and evaluation pipeline\.

Structural Verification\.EvoPINN compares the generated module with its parent using both standard and normalized abstract syntax trees \(ASTs\)\. Parsing the programs into standard ASTs excludes non\-computational differences such as comments and formatting, while the normalized AST additionally canonicalizes non\-interface identifiers and literal constants\. A candidate must differ from its parent under both representations, filtering out cosmetic edits, identifier renaming, and constant\-only changes\. This deliberately excludes pure hyperparameter mutations because EvoPINN targets mechanism discovery rather than parameter tuning\. The complete normalization and comparison procedure is provided in Appendix B\.

Executable Testing and Repair\.Candidates that pass structural verification are assembled with their counterpart module to form a complete algorithm\. Before full\-scale training, they are executed on a small PINN instance to rigorously test for interface compatibility, numerical stability, and automatic differentiability\. Failed candidates are iteratively repaired using execution feedback, up to a predefined limit\. More details are provided in Appendix B\.

Budget\-Matched Evaluation\.To guarantee fair performance attribution, all candidates are evaluated under a shared training budgetBgB\_\{g\}and random seedξg\\xi\_\{g\}\. This budget constrains the total number of optimization steps, the initial collocation counts, and the configured adaptive\-sampling controls\. While candidate algorithms are free to creatively reallocate these resources \(e\.g\., via adaptive sampling strategies or novel loss\-balancing dynamics\), they cannot arbitrarily increase the computational footprint, thereby preventing naive resource\-scaling advantages\. The complete agentic search pipeline across generations is formalized as Algorithm 1 in Appendix B\.

## 5Experiments

Our experiments are designed to verify that the algorithmic mechanisms discovered by EvoPINN are both effective and robustly generalizable\. We first evaluate the frozen discovered algorithms against seed and expert\-designed PINNs across four PDE benchmarks\. Next, we compare EvoPINN across distinct automated search paradigms under matched evaluation budgets and a unified evaluation pipeline\. Beyond aggregate accuracy, we provide a mechanistic analysis of the discovered SLRC\-PINN architecture, perform targeted framework ablations, and demonstrate zero\-shot transferability to neighboring PDE parameters\.

### 5\.1Experimental Setup

##### Benchmarks\.

We evaluate EvoPINN on four diverse PDE regimes: elliptic Poisson2D with COMSOL reference data, nonlinear\-transport Burgers1D with numerical reference data, oscillatory Wave1D with an analytic reference solution, and dissipative Heat2D with an analytic reference solution\. These benchmarks respectively challenge the models with complex\-domain approximation, steep\-gradient resolution, high\-frequency structures, and anisotropic diffusion\.

##### Search and retraining protocol\.

The seed algorithmA0A\_\{0\}is a five\-hidden\-layer tanh feed\-forward network of width 100, optimized via Adam followed by L\-BFGS\. Each search spans seven generations with eight proposals per generation, yielding a maximum of 56 full candidate evaluations\. To rigorously decouple algorithm discovery from final assessment, candidate selection during evolution uses only the search\-time reference set\(𝒳search\)\(\\mathcal\{X\}\_\{\\mathrm\{search\}\}\)\. The resulting algorithm is then frozen and retrained from scratch across five independent evaluation seeds, and its final relativeL2L\_\{2\}error is computed on a disjoint reporting set\.

##### Baselines\.

We benchmark EvoPINN against the seed algorithm and an extensive pool of 16 expert\-designed PINN variants, covering advanced techniques such as Fourier and SIREN representations, residual\-adaptive sampling, curriculum learning, weighted objectives, geometry\-aware models, and Neural Tangent Kernel \(NTK\) balancing\. To ensure a fair comparison, the expert variants were first screened under a unified evaluation pipeline\. The best\-performing expert for each PDE was then fixed and retrained from scratch using the same five evaluation seeds as EvoPINN\. The expert mechanisms identified were: boundary\-aware NTK with an L\-BFGS prior for Poisson2D, multiscale Fourier features for Burgers1D, Fourier features with capped NTK for Wave1D, and anisotropic multiscale Fourier features for Heat2D\.

##### Implementation details\.

Candidate programs are generated using the gpt\-5\.4\-mini model at a temperature of 0\.7\. To guarantee fair attribution of performance, all candidate algorithms operate under a shared, generation\-specific nominal budget \(with a full\-scale limit of 20,000 optimization steps and common initial collocation counts sampled via a Hammersley sequence\)\. Candidate evaluations are executed in isolated subprocesses with a 7,200\-second timeout; runs producing errors or non\-finite outputs are automatically rejected\. Our framework is built upon DeepXDE 1\.15\.0 and PyTorch 2\.6\.0\. Executed on NVIDIA RTX 3090 24GB GPUs, the autonomous algorithm discovery process proved computationally tractable, accumulating between 14\.70 and 51\.78 GPU\-hours depending on the complexity of the target PDE\. Comprehensive details and hyperparameters are provided in Appendix D\.

### 5\.2Performance across PDE Benchmarks

Table 1:Relative L2 error evaluated across five independent runs \(mean±\\pmsample std\.; lower is better\)\. All methods use the shared training\-resource budget\.We first investigate whether the performance gains from evolution generalize when the discovered algorithms are frozen and trained from scratch\. As reported in Table[1](https://arxiv.org/html/2607.26490#S5.T1), frozen EvoPINN algorithms consistently outperform the seed PINN across all four benchmarks\. Against highly specialized expert designs, EvoPINN achieves the lowest mean error on Poisson2D, Burgers1D, and Wave1D\. While the expert\-designed anisotropic Fourier baseline retains an edge on Heat2D, EvoPINN remains competitive\. These results confirm that the discovered algorithmic improvements are robust and effective across diverse physical regimes\.

### 5\.3Comparison with Automated Search Frameworks

To evaluate execution\-grounded evolution, we benchmark EvoPINN against three distinct search paradigms under matched computational budgets:\(i\) LLM Best\-of\-56:samples 56 independent algorithms without execution feedback or search memory, selecting the top search\-set candidate;\(ii\) DPSTE\(Zhang and Yang,[2024](https://arxiv.org/html/2607.26490#bib.bib39)\): performs evolutionary architecture search over fixed hyperparameter spaces \(e\.g\., depth, width, shortcuts, and activations\); and\(iii\) PINNsAgent\(Wuwuet al\.,[2025](https://arxiv.org/html/2607.26490#bib.bib30)\): uses a multi\-agent workflow and memory\-based reasoning to iteratively explore and optimize PINN architectures\. Each strategy is allocated a maximum of 56 candidate evaluations\. The best algorithm identified by each strategy is selected using only the search\-time reference set, frozen, and retrained from scratch using the same five independent evaluation seeds\. Final errors are computed on the reporting set\.

Table 2:Search strategy comparison on Burgers1D and Wave1D \(mean±\\pmstd\)\. Best results are in bold\.As presented in Table[2](https://arxiv.org/html/2607.26490#S5.T2), EvoPINN achieves the lowest error across all tasks\. While PINNsAgent substantially outperforms traditional search \(DPSTE\) via agentic reasoning, its exploration remains inherently bounded by predefined configuration templates\. In contrast, EvoPINN leverages execution\-grounded diagnostic feedback and decoupled module\-wise mutation to move beyond configuration selection, synthesizing novel, executable algorithmic mechanisms that handle complex PDE optimization dynamics\.

While these aggregate performance metrics confirm that EvoPINN repeatedly identifies superior learning algorithms, they do not fully reveal the mathematical mechanisms driving these gains\. To examine whether our agentic evolution synthesizes genuinely new computational topologies rather than merely tuning configurations, we next present a mechanistic interpretation of the discovered SLRC\-PINN architecture\.

### 5\.4Discovery of SLRC\-PINN

Across independent search runs on Burgers1D, EvoPINN repeatedly evolved architectures incorporating localized solution\-enrichment mechanisms\. A representative candidate pairs a continuous global base predictor with a zero\-initialized, selectively activated local corrective pathway\. We term this architecture theSelf\-Localizing Residual\-Correction PINN \(SLRC\-PINN\)\. Rather than partitioning the spatial domain or forcing competing subnetworks to fit local regions, SLRC\-PINN introduces a dynamic execution topology that enables continuous global prediction with localized, self\-organizing solution enrichment\.

##### Mathematical Formulation\.

Letq=\(x,t\)q=\(x,t\)denote a spatiotemporal coordinate\. SLRC\-PINN constructsK=6K=6localized basis functions with learned centers and input\-dependent widths:

dk2​\(q\)=∥q−ck∥22,σk​\(q\)=exp⁡\(ℓk\)​\(1\+softplus⁡\(sk​\(q\)\)\+0\.2\),bk​\(q\)=exp⁡\[−dk2​\(q\)2​σk​\(q\)2\]​sigmoid⁡\(ak​\(q\)\)​sigmoid⁡\(mk​\(q\)\),\\begin\{split\}d\_\{k\}^\{2\}\(q\)&=\\lVert q\-c\_\{k\}\\rVert\_\{2\}^\{2\},\\\\ \\sigma\_\{k\}\(q\)&=\\exp\(\\ell\_\{k\}\)\\bigl\(1\+\\operatorname\{softplus\}\(s\_\{k\}\(q\)\)\+0\.2\\bigr\),\\\\ b\_\{k\}\(q\)&=\\exp\\\!\\left\[\-\\frac\{d\_\{k\}^\{2\}\(q\)\}\{2\\sigma\_\{k\}\(q\)^\{2\}\}\\right\]\\operatorname\{sigmoid\}\(a\_\{k\}\(q\)\)\\operatorname\{sigmoid\}\(m\_\{k\}\(q\)\),\\end\{split\}\(7\)whereaka\_\{k\},mkm\_\{k\}, andsks\_\{k\}are learned coordinate projections\. The basis responses and coordinates are processed by a gated residual backbone, feeding into two distinct decoders: a global background decoderuG​\(q\)u\_\{G\}\(q\)and a local corrective decoderuL​\(q\)u\_\{L\}\(q\)\. The combined prediction is formulated as:

u​\(q\)=uG​\(q\)\+sigmoid⁡\(w𝖳​b​\(q\)\+b0\)​\(1K​∑k=1Kbk​\(q\)\)⏟α​\(q\)​uL​\(q\)\.\\small u\(q\)=u\_\{G\}\(q\)\+\\underbrace\{\\operatorname\{sigmoid\}\(w^\{\\mathsf\{T\}\}b\(q\)\+b\_\{0\}\)\\left\(\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}b\_\{k\}\(q\)\\right\)\}\_\{\\alpha\(q\)\}u\_\{L\}\(q\)\.\(8\)

##### Core Topological Principles\.

The principal innovation of SLRC\-PINN lies not in simply adding another neural branch, but in allowing the model to autonomously govern where, when, and how localized corrections emerge\. This design is characterized by three tightly coupled principles:

1. 1\.Persistent Global Anchoring:The prediction topologyu​\(q\)=uG​\(q\)\+α​\(q\)​uL​\(q\)u\(q\)=u\_\{G\}\(q\)\+\\alpha\(q\)u\_\{L\}\(q\)is inherently asymmetric\. Unlike traditional domain decomposition \(e\.g\., FBPINNsMoseleyet al\.\([2023](https://arxiv.org/html/2607.26490#bib.bib15)\)\) or ensemble\-gating approaches \(e\.g\., APINNsHuet al\.\([2023](https://arxiv.org/html/2607.26490#bib.bib34)\)\) that route predictions across competing subnetworks, SLRC\-PINN retains a persistent, unsuppressed global representationuG​\(q\)u\_\{G\}\(q\)across the entire domain\. The local branchuL​\(q\)u\_\{L\}\(q\)serves purely as an additive enrichment, removing the need for artificial interface matching or inter\-branch competition\.
2. 2\.Self\-Organized Corrective Support:Rather than relying on fixed subdomains or hand\-crafted spatial masks, the gating fieldα​\(q\)∈\(0,1\)\\alpha\(q\)\\in\(0,1\)dynamically converts the learned geometry of the basis bank into aC∞C^\{\\infty\}\-differentiable routing field\. The network automatically identifies unresolved spatial structures \(e\.g\., steep viscous shock waves\) and allocates corrective capacity end\-to\-end directly from the PDE loss, avoiding the parasitic gradient discontinuities typical of hard\-partitioning methods\.
3. 3\.Function\-Preserving Progressive Expansion:By zero\-initializing the final linear layer of the corrective decoder,uL​\(q\)≡0u\_\{L\}\(q\)\\equiv 0andu​\(q\)≡uG​\(q\)u\(q\)\\equiv u\_\{G\}\(q\)at step0\. This guarantees that the model initiates optimization on the smooth global solution manifold without cold\-start turbulence\. Localized expressive capacity is subsequently introduced as an emergent, smooth continuation as demanded by PDE residual minimization\.

##### Controlled Empirical Validation\.

To verify that SLRC\-PINN’s superior accuracy stems from its discovered global\-local topology rather than raw model capacity, we compare the frozen SLRC\-PINN against a parameter\-matched global MLP \(width 230\) and architecture\-faithful reimplementations of representative baselines \(APINN, FBPINN, and HyResPINNs\) under identical evaluation protocols\.

Table 3:Controlled Burgers1D architecture comparison across five identical evaluation seeds\. All models operate under matched capacity and identical training budgets\.As shown in Table[3](https://arxiv.org/html/2607.26490#S5.T3), SLRC\-PINN achieves the lowest mean \(2\.04×10−42\.04\\times 10^\{\-4\}\) and median relativeL2L\_\{2\}error\. Crucially, it reduces the mean error by55\.1%55\.1\\%compared to the parameter\-matched global MLP and consistently outperforms domain\-decomposition variants\. This confirms that the performance gains are directly driven by its self\-organizing, function\-preserving execution topology\.

Furthermore, EvoPINN’s programmatic discoveries extend beyond neural representations\. On Poisson2D, the agent autonomously invented theTelemetry\-Conditioned Resampling and Optimizer Policy\. Due to space limitations, Appendix C provides a detailed mechanism audit and evaluation of this discovered training policy\.

### 5\.5Ablation Study

While the preceding experiments demonstrate EvoPINN’s ability to discover effective learning mechanisms, they do not isolate the contributions of specific framework components\. To address this, we conduct targeted ablations on Burgers1D and Wave1D, focusing on two core design choices: conditioning LLM proposals on dynamic training diagnostics, and the decoupled module\-wise evolution strategy\. All variants are evaluated under a strictly shared configuration \(identical seed, proposal budget, and evaluation protocol\) to ensure a fair comparison\. We evaluate four ablated configurations: removing training diagnostic summary \(w/o diagnosis\), mutating both modules simultaneously \(w/o modular\), and restricting the search exclusively to either the representation \(repr\-only\) or the training program \(train\-only\)\.

Table 4:Ablation study of core EvoPINN components\. ‘Evals’ indicates the number of successfully executed candidate programs out of 56 total proposals\.As shown in Table[4](https://arxiv.org/html/2607.26490#S5.T4), the full EvoPINN framework achieves superior accuracy across both benchmarks\. These ablations reveal three critical insights: First, omitting diagnostics degrades accuracy—particularly on Wave1D—proving that diagnostic feedback on loss dynamics is essential for targeted LLM mutations\. Second, joint multi\-module mutations cause execution instability, whereas adaptively scheduled, one\-module\-at\-a\-time evolution provides a useful scaffold for candidate validity\. Finally, the inferior performance of single\-module searches confirms that representation topology and optimization dynamics are intrinsically coupled, necessitating joint algorithmic co\-design\.

### 5\.6Transfer to Neighboring PDE Parameters

The preceding experiments evaluate the discovered algorithms on the PDE instances used during search\. We further investigate their zero\-shot transferability to neighboring equation parameters\. Specifically, we freeze the algorithms discovered on Wave1D ata=4\.0a=4\.0and Heat2D atkx=20​πk\_\{x\}=20\\pi, and directly retrain them at shifted parameters, i\.e\.,a∈\{3\.5,4\.5\}a\\in\\\{3\.5,4\.5\\\}andkx∈\{15​π,25​π\}k\_\{x\}\\in\\\{15\\pi,25\\pi\\\}\.

Table 5:Transfer to neighboring PDE parameters with EvoPINN modules and expert identities fixed on the source instance\. The source parameters area=4\.0a=4\.0andkx=20​πk\_\{x\}=20\\pi\.Table[5](https://arxiv.org/html/2607.26490#S5.T5)demonstrates that the algorithms discovered by EvoPINN exhibit strong parameter adaptability across both physical regimes\. On Wave1D, the parameteraagoverns the spatial\-temporal oscillation frequencies and boundary condition profiles of the high\-frequency wave mode\. Notably, shiftingaafrom4\.04\.0to half\-integer values \(3\.53\.5and4\.54\.5\) introduces frequency scale shifts and transitions the right boundary from a static homogeneous condition \(u​\(1,t\)=0u\(1,t\)=0\) to a time\-varying non\-homogeneous driver\. Even under these combined frequency and boundary profile shifts, frozen EvoPINN consistently reduces the seed\-PINN error by over an order of magnitude across all settings, maintaining superior or competitive accuracy compared to the expert baseline\.

This generalization advantage becomes particularly evident on Heat2D, where frozen EvoPINN consistently yields the highest solution precision across both original and shifted frequencies\. Notably, while the expert baseline suffers marked performance degradation under spatial frequency shifts due to rigid basis tuning, frozen EvoPINN retains a steady order\-of\-magnitude accuracy\. This contrast indicates that EvoPINN synthesizes inherently frequency\-resilient learning mechanisms rather than overfitting to specific physical parameters, enabling direct zero\-shot deployment to neighboring equations without additional search overhead\.

## 6Conclusion and Future Work

In this work, we transition automated PINN design from passive configuration tuning to open\-ended, execution\-grounded algorithm discovery\. By decoupling neural representations from training dynamics and enforcing strict structural and computational contracts, EvoPINN enables LLM agents to systematically navigate physics\-constrained design spaces without numerical instability\. Across diverse PDE regimes, the discovered algorithms achieve consistent performance gains and demonstrate strong zero\-shot transferability to neighboring equation parameters\. Crucially, the autonomous discovery and sustained advantage of SLRC\-PINN confirm that execution\-grounded agents can invent non\-trivial, scientifically valid computational topologies\. Broadly, EvoPINN establishes a practical blueprint for integrating generative models with numerical computing, paving the way for autonomous algorithmic discovery across AI for Science\. Despite these promising results, our approach has certain limitations\. First, the creativity of the discovered mechanisms depends in part on the underlying LLM’s code synthesis proficiency and scientific reasoning capacity\. Second, evaluating each candidate via full\-budget PINN training incurs non\-trivial cost during search\. Future work will explore proxy objectives or surrogate evaluations to significantly accelerate candidate screening, and investigate the integration of scientifically fine\-tuned language models to further expand the boundaries of automated algorithm discovery\.

## References

- A\. Daw, J\. Bu, S\. Wang, P\. Perdikaris, and A\. Karpatne \(2022\)Rethinking the importance of sampling in physics\-informed neural networks\.arXiv preprint arXiv:2207\.02338\.Cited by:[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- S\. Fazliani and M\. Udell \(2025\)PDE\-SHARP: PDE solver hybrids through analysis and refinement passes\.arXiv preprint arXiv:2511\.00183\.External Links:2511\.00183Cited by:[§2\.3](https://arxiv.org/html/2607.26490#S2.SS3.p1.1)\.
- Z\. Hao, J\. Yao, C\. Su, H\. Su, Z\. Wang, F\. Lu, Z\. Xia, Y\. Zhang, S\. Liu, L\. Lu, and J\. Zhu \(2024\)PINNacle: a comprehensive benchmark of physics\-informed neural networks for solving PDEs\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2607.26490#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- X\. He, L\. You, H\. Tian, B\. Han, I\. Tsang, and Y\. Ong \(2025\)Lang\-PINN: from language to physics\-informed neural networks via a multi\-agent framework\.arXiv preprint arXiv:2510\.05158\.External Links:2510\.05158Cited by:[§1](https://arxiv.org/html/2607.26490#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.26490#S2.SS2.p1.1)\.
- Z\. Hu, A\. D\. Jagtap, G\. E\. Karniadakis, and K\. Kawaguchi \(2023\)Augmented physics\-informed neural networks \(APINNs\): a gating network\-based soft domain decomposition methodology\.Engineering Applications of Artificial Intelligence126,pp\. 107183\.External Links:[Document](https://dx.doi.org/10.1016/j.engappai.2023.107183)Cited by:[item 1](https://arxiv.org/html/2607.26490#S5.I1.i1.p1.3)\.
- A\. D\. Jagtap and G\. E\. Karniadakis \(2020\)Extended physics\-informed neural networks \(XPINNs\): a generalized space\-time domain decomposition based deep learning framework for nonlinear partial differential equations\.Communications in Computational Physics28\(5\),pp\. 2002–2041\.External Links:[Document](https://dx.doi.org/10.4208/cicp.OA-2020-0164)Cited by:[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- A\. D\. Jagtap, K\. Kawaguchi, and G\. E\. Karniadakis \(2020\)Adaptive activation functions accelerate convergence in deep and physics\-informed neural networks\.Journal of Computational Physics404,pp\. 109136\.External Links:[Document](https://dx.doi.org/10.1016/j.jcp.2019.109136)Cited by:[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- A\. Krishnapriyan, A\. Gholami, S\. Zhe, R\. Kirby, and M\. W\. Mahoney \(2021\)Characterizing possible failure modes in physics\-informed neural networks\.Advances in neural information processing systems34,pp\. 26548–26560\.Cited by:[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- S\. Li, T\. Marwah, J\. Shen, W\. Sun, A\. Risteski, Y\. Yang, and A\. Talwalkar \(2025\)CodePDE: an inference framework for LLM\-driven PDE solver generation\.arXiv preprint arXiv:2505\.08783\.External Links:2505\.08783Cited by:[§2\.3](https://arxiv.org/html/2607.26490#S2.SS3.p1.1)\.
- F\. Liu, X\. Tong, M\. Yuan, X\. Lin, F\. Luo, Z\. Wang, Z\. Lu, and Q\. Zhang \(2024\)Evolution of heuristics: towards efficient automatic algorithm design using large language model\.arXiv preprint arXiv:2401\.02051\.Cited by:[§2\.3](https://arxiv.org/html/2607.26490#S2.SS3.p1.1),[§4\.1](https://arxiv.org/html/2607.26490#S4.SS1.p4.1)\.
- L\. Lu, X\. Meng, Z\. Mao, and G\. E\. Karniadakis \(2021\)DeepXDE: a deep learning library for solving differential equations\.SIAM Review63\(1\),pp\. 208–228\.External Links:[Document](https://dx.doi.org/10.1137/19M1274067)Cited by:[§1](https://arxiv.org/html/2607.26490#S1.p1.1)\.
- Y\. J\. Ma, W\. Liang, G\. Wang, D\. Huang, O\. Bastani, D\. Jayaraman, Y\. Zhu, L\. Fan, and A\. Anandkumar \(2024\)Eureka: human\-level reward design via coding large language models\.InInternational Conference on Learning Representations,Cited by:[§2\.3](https://arxiv.org/html/2607.26490#S2.SS3.p1.1)\.
- B\. Moseley, A\. Markham, and T\. Nissen\-Meyer \(2023\)Finite basis physics\-informed neural networks \(FBPINNs\): a scalable domain decomposition approach for solving differential equations\.Advances in Computational Mathematics49\(4\),pp\. 62\.External Links:[Document](https://dx.doi.org/10.1007/s10444-023-10065-9)Cited by:[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1),[item 1](https://arxiv.org/html/2607.26490#S5.I1.i1.p1.3)\.
- M\. Raissi, P\. Perdikaris, and G\. E\. Karniadakis \(2019\)Physics\-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations\.Journal of Computational Physics378,pp\. 686–707\.External Links:[Document](https://dx.doi.org/10.1016/j.jcp.2018.10.045)Cited by:[§1](https://arxiv.org/html/2607.26490#S1.p1.1)\.
- B\. Romera\-Paredes, M\. Barekatain, A\. Novikov, M\. Balog, M\. P\. Kumar, E\. Dupont, F\. J\. R\. Ruiz, J\. S\. Ellenberg, P\. Wang, O\. Fawzi, P\. Kohli, and A\. Fawzi \(2024\)Mathematical discoveries from program search with large language models\.Nature625\(7995\),pp\. 468–475\.External Links:ISSN 1476\-4687,[Link](https://doi.org/10.1038/s41586-023-06924-6),[Document](https://dx.doi.org/10.1038/s41586-023-06924-6)Cited by:[§2\.3](https://arxiv.org/html/2607.26490#S2.SS3.p1.1),[§4\.1](https://arxiv.org/html/2607.26490#S4.SS1.p4.1)\.
- V\. Sitzmann, J\. N\. P\. Martel, A\. W\. Bergman, D\. B\. Lindell, and G\. Wetzstein \(2020\)Implicit neural representations with periodic activation functions\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 7462–7473\.Cited by:[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- M\. Tancik, P\. Srinivasan, B\. Mildenhall, S\. Fridovich\-Keil, N\. Raghavan, U\. Singhal, R\. Ramamoorthi, J\. Barron, and R\. Ng \(2020\)Fourier features let networks learn high frequency functions in low dimensional domains\.Advances in neural information processing systems33,pp\. 7537–7547\.Cited by:[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- S\. Wang, Y\. Teng, and P\. Perdikaris \(2021a\)Understanding and mitigating gradient flow pathologies in physics\-informed neural networks\.SIAM Journal on Scientific Computing43\(5\),pp\. A3055–A3081\.External Links:[Document](https://dx.doi.org/10.1137/20M1318043)Cited by:[§1](https://arxiv.org/html/2607.26490#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- S\. Wang, H\. Wang, and P\. Perdikaris \(2021b\)On the eigenvector bias of Fourier feature networks: from regression to solving multi\-scale PDEs with physics\-informed neural networks\.Computer Methods in Applied Mechanics and Engineering384,pp\. 113938\.External Links:[Document](https://dx.doi.org/10.1016/j.cma.2021.113938)Cited by:[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- S\. Wang, X\. Yu, and P\. Perdikaris \(2022\)When and why PINNs fail to train: a neural tangent kernel perspective\.Journal of Computational Physics449,pp\. 110768\.External Links:[Document](https://dx.doi.org/10.1016/j.jcp.2021.110768)Cited by:[§1](https://arxiv.org/html/2607.26490#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- Y\. Wang, X\. Han, C\. Chang, D\. Zha, U\. Braga\-Neto, and X\. Hu \(2023\)Auto\-pinn: understanding and optimizing physics\-informed neural architecture\.External Links:2205\.13748,[Link](https://arxiv.org/abs/2205.13748)Cited by:[§1](https://arxiv.org/html/2607.26490#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.26490#S2.SS2.p1.1)\.
- Y\. Wang and L\. Zhong \(2024\)NAS\-PINN: neural architecture search\-guided physics\-informed neural network for solving PDEs\.Journal of Computational Physics496,pp\. 112603\.External Links:[Document](https://dx.doi.org/10.1016/j.jcp.2023.112603)Cited by:[§1](https://arxiv.org/html/2607.26490#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.26490#S2.SS2.p1.1)\.
- C\. Wu, M\. Zhu, Q\. Tan, Y\. Kartha, and L\. Lu \(2023\)A comprehensive study of non\-adaptive and residual\-based adaptive sampling for physics\-informed neural networks\.Computer Methods in Applied Mechanics and Engineering403,pp\. 115671\.External Links:[Document](https://dx.doi.org/10.1016/j.cma.2022.115671)Cited by:[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- Q\. Wuwu, C\. Gao, T\. Chen, Y\. Huang, Y\. Zhang, J\. Wang, J\. Li, H\. Zhou, and S\. Zhang \(2025\)PINNsAgent: automated PDE surrogation with large language models\.arXiv preprint arXiv:2501\.12053\.External Links:2501\.12053Cited by:[§1](https://arxiv.org/html/2607.26490#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.26490#S2.SS2.p1.1),[§5\.3](https://arxiv.org/html/2607.26490#S5.SS3.p1.1)\.
- J\. Yao, C\. Su, Z\. Hao, S\. Liu, H\. Su, and J\. Zhu \(2023\)MultiAdam: parameter\-wise scale\-invariant optimizer for multiscale training of physics\-informed neural networks\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 39702–39721\.Cited by:[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- H\. Ye, J\. Wang, Z\. Cao, F\. Berto, C\. Hua, H\. Kim, J\. Park, and G\. Song \(2024\)ReEvo: large language models as hyper\-heuristics with reflective evolution\.Advances in Neural Information Processing Systems\.Cited by:[§2\.3](https://arxiv.org/html/2607.26490#S2.SS3.p1.1)\.
- J\. Yu, L\. Lu, X\. Meng, and G\. E\. Karniadakis \(2022\)Gradient\-enhanced physics\-informed neural networks for forward and inverse PDE problems\.Computer Methods in Applied Mechanics and Engineering393,pp\. 114823\.External Links:[Document](https://dx.doi.org/10.1016/j.cma.2022.114823)Cited by:[§2\.1](https://arxiv.org/html/2607.26490#S2.SS1.p1.1)\.
- B\. Zhang and C\. Yang \(2024\)Discovering physics\-informed neural networks model for solving partial differential equations through evolutionary computation\.Swarm and Evolutionary Computation88,pp\. 101589\.Cited by:[§1](https://arxiv.org/html/2607.26490#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.26490#S2.SS2.p1.1),[§5\.3](https://arxiv.org/html/2607.26490#S5.SS3.p1.1)\.

## Appendix AMathematical Background of Physics\-Informed Neural Networks

### A\.1Governing Equations and Neural Approximation

LetΩ⊆ℝd\\Omega\\subseteq\\mathbb\{R\}^\{d\}be add\-dimensional spatial domain,𝐱∈Ω\\mathbf\{x\}\\in\\Omegaa spatial coordinate, and𝐳=\(𝐱,t\)∈Ω×\[0,Tmax\]\\mathbf\{z\}=\(\\mathbf\{x\},t\)\\in\\Omega\\times\[0,T\_\{\\max\}\]a space–time coordinate for a time\-dependent problem, wherettis time andTmaxT\_\{\\max\}is the final time\. The target fieldu:Ω×\[0,Tmax\]→ℝmu:\\Omega\\times\[0,T\_\{\\max\}\]\\to\\mathbb\{R\}^\{m\}, withmmoutput components, satisfies

ℱ​\[u\]​\(𝐳\)=0,𝐳∈Ω×\(0,Tmax\],\\mathcal\{F\}\[u\]\(\\mathbf\{z\}\)=0,\\qquad\\mathbf\{z\}\\in\\Omega\\times\(0,T\_\{\\max\}\],\(9\)together with boundary and, when applicable, initial conditions

ℬ​\[u\]​\(𝐳\)\\displaystyle\\mathcal\{B\}\[u\]\(\\mathbf\{z\}\)=0,\\displaystyle=0,𝐳∈∂Ω×\[0,Tmax\],\\displaystyle\\mathbf\{z\}\\in\\partial\\Omega\\times\[0,T\_\{\\max\}\],\(10\)ℐ​\[u\]​\(𝐱\)\\displaystyle\\mathcal\{I\}\[u\]\(\\mathbf\{x\}\)=0,\\displaystyle=0,𝐱∈Ω,t=0\.\\displaystyle\\mathbf\{x\}\\in\\Omega,\\;t=0\.\(11\)Hereℱ\\mathcal\{F\},ℬ\\mathcal\{B\}, andℐ\\mathcal\{I\}are the differential, boundary\-residual, and initial\-residual operators, respectively;∂Ω\\partial\\Omegadenotes the spatial boundary\. Prescribed boundary and initial data are included inℬ\\mathcal\{B\}andℐ\\mathcal\{I\}\. For a steady problem, the temporal coordinate and initial condition are omitted\.

A PINN replaces the unknown field by a differentiable neural approximationuθ:ℝdz→ℝmu\_\{\\theta\}:\\mathbb\{R\}^\{d\_\{z\}\}\\to\\mathbb\{R\}^\{m\}, wheredz=d\+1d\_\{z\}=d\+1for a time\-dependent problem anddz=dd\_\{z\}=dfor a steady problem\. The parameter vectorθ\\thetacontains all trainable weights\. Unlike a supervised regressor, a PINN is trained primarily from violations of the governing equation and its physical constraints\. Automatic differentiation applies the chain rule throughuθu\_\{\\theta\}to evaluate the derivatives required byℱ\\mathcal\{F\}; for example,utu\_\{t\},∇u\\nabla u, and∇2u\\nabla^\{2\}uare computed at arbitrary coordinates without constructing a mesh\-based differentiation stencil\.

The pointwise residual fields are

rθ​\(𝐳\)\\displaystyle r\_\{\\theta\}\(\\mathbf\{z\}\)=ℱ​\[uθ\]​\(𝐳\),\\displaystyle=\\mathcal\{F\}\[u\_\{\\theta\}\]\(\\mathbf\{z\}\),\(12\)bθ​\(𝐳\)\\displaystyle b\_\{\\theta\}\(\\mathbf\{z\}\)=ℬ​\[uθ\]​\(𝐳\),\\displaystyle=\\mathcal\{B\}\[u\_\{\\theta\}\]\(\\mathbf\{z\}\),\(13\)iθ​\(𝐱\)\\displaystyle i\_\{\\theta\}\(\\mathbf\{x\}\)=ℐ​\[uθ\]​\(𝐱\)\.\\displaystyle=\\mathcal\{I\}\[u\_\{\\theta\}\]\(\\mathbf\{x\}\)\.\(14\)For systems of PDEs these quantities may be vector valued\. Their derivatives remain coupled through the shared network parameters, so a single forward representation must satisfy the equation and all conditions simultaneously\.

### A\.2Collocation Loss and Constraint Enforcement

Let𝒳r\\mathcal\{X\}\_\{r\},𝒳b\\mathcal\{X\}\_\{b\}, and𝒳i\\mathcal\{X\}\_\{i\}be interior, boundary, and initial collocation sets with cardinalitiesNrN\_\{r\},NbN\_\{b\}, andNiN\_\{i\}\. The standard empirical residual losses are

ℒr​\(θ\)\\displaystyle\\mathcal\{L\}\_\{r\}\(\\theta\)=1Nr​∑𝐳∈𝒳r‖rθ​\(𝐳\)‖22,\\displaystyle=\\frac\{1\}\{N\_\{r\}\}\\sum\_\{\\mathbf\{z\}\\in\\mathcal\{X\}\_\{r\}\}\\left\\\|r\_\{\\theta\}\(\\mathbf\{z\}\)\\right\\\|\_\{2\}^\{2\},\(15\)ℒb​\(θ\)\\displaystyle\\mathcal\{L\}\_\{b\}\(\\theta\)=1Nb​∑𝐳∈𝒳b‖bθ​\(𝐳\)‖22,\\displaystyle=\\frac\{1\}\{N\_\{b\}\}\\sum\_\{\\mathbf\{z\}\\in\\mathcal\{X\}\_\{b\}\}\\left\\\|b\_\{\\theta\}\(\\mathbf\{z\}\)\\right\\\|\_\{2\}^\{2\},\(16\)ℒi​\(θ\)\\displaystyle\\mathcal\{L\}\_\{i\}\(\\theta\)=1Ni​∑𝐱∈𝒳i‖iθ​\(𝐱\)‖22\.\\displaystyle=\\frac\{1\}\{N\_\{i\}\}\\sum\_\{\\mathbf\{x\}\\in\\mathcal\{X\}\_\{i\}\}\\left\\\|i\_\{\\theta\}\(\\mathbf\{x\}\)\\right\\\|\_\{2\}^\{2\}\.\(17\)The initial loss is defined only for time\-dependent problems withNi\>0N\_\{i\}\>0\. The composite soft\-constraint objective is

ℒPINN​\(θ\)=λr​ℒr\+λb​ℒb\+λi​ℒi,\\mathcal\{L\}\_\{\\mathrm\{PINN\}\}\(\\theta\)=\\lambda\_\{r\}\\mathcal\{L\}\_\{r\}\+\\lambda\_\{b\}\\mathcal\{L\}\_\{b\}\+\\lambda\_\{i\}\\mathcal\{L\}\_\{i\},\(18\)whereλr\\lambda\_\{r\},λb\\lambda\_\{b\}, andλi\\lambda\_\{i\}are nonnegative weights for the interior, boundary, and initial terms\. The initial term is omitted for steady problems\. The collocation coordinates encode physical constraints; they are not labeled samples of the unknown solution\.

Equation \([18](https://arxiv.org/html/2607.26490#A1.E18)\) imposes boundary and initial conditions*softly*\. When an analytic constraint\-preserving transformation is available, they can instead be imposed*hard*\. For example, a scalar Dirichlet condition can be represented as

uθ​\(𝐳\)=ubc​\(𝐳\)\+ϕ​\(𝐳\)​u^θ​\(𝐳\),ϕ\|∂Ω=0,u\_\{\\theta\}\(\\mathbf\{z\}\)=u\_\{\\mathrm\{bc\}\}\(\\mathbf\{z\}\)\+\\phi\(\\mathbf\{z\}\)\\widehat\{u\}\_\{\\theta\}\(\\mathbf\{z\}\),\\qquad\\phi\|\_\{\\partial\\Omega\}=0,\(19\)whereubcu\_\{\\mathrm\{bc\}\}satisfies the prescribed boundary values\. The boundary is then satisfied for everyθ\\theta, and the corresponding penalty may be removed\. Soft enforcement is more general, whereas hard enforcement reduces competition among loss terms when a suitable transformation exists\.

Collocation sets may be fixed, resampled, or adapted during training\. Their finite\-sample objective approximates a continuous residual norm; for example,

ℒr​\(θ\)≈1\|ΩT\|​∫ΩT‖ℱ​\[uθ\]​\(𝐳\)‖22​𝑑𝐳,\\mathcal\{L\}\_\{r\}\(\\theta\)\\approx\\frac\{1\}\{\|\\Omega\_\{T\}\|\}\\int\_\{\\Omega\_\{T\}\}\\left\\\|\\mathcal\{F\}\[u\_\{\\theta\}\]\(\\mathbf\{z\}\)\\right\\\|\_\{2\}^\{2\}\\,d\\mathbf\{z\},\(20\)whereΩT=Ω×\(0,Tmax\]\\Omega\_\{T\}=\\Omega\\times\(0,T\_\{\\max\}\]for a time\-dependent problem\. Consequently, point placement determines which regions of the physical domain provide gradient information to the optimizer\.

### A\.3Training and Solution Error

For a fixed representation and training procedure, PINN training seeks

θ⋆∈arg⁡minθ⁡ℒPINN​\(θ\)\.\\theta^\{\\star\}\\in\\arg\\min\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{PINN\}\}\(\\theta\)\.\(21\)Gradient\-based updates use

θk\+1=θk−ηk​Pk​∇θℒPINN​\(θk\),\\theta\_\{k\+1\}=\\theta\_\{k\}\-\\eta\_\{k\}P\_\{k\}\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{PINN\}\}\(\\theta\_\{k\}\),\(22\)whereηk\\eta\_\{k\}is a step size andPkP\_\{k\}is the identity for a basic first\-order method or an optimizer\-dependent preconditioner\. In practice, Adam is often used for robust stochastic progress and L\-BFGS for later quasi\-Newton refinement\. These phases are not interchangeable: their effectiveness depends on loss scaling, collocation dynamics, and the state at which an optimizer transition occurs\.

A small empirical physics loss does not by itself guarantee a small solution error\. The learned error reflects several interacting sources: the approximation capacity and inductive bias ofuθu\_\{\\theta\}, finite collocation coverage, imbalance among residual terms, and incomplete optimization of a nonconvex objective\. For a reference solutionu∗u^\{\*\}on an evaluation set𝒳\\mathcal\{X\}, this distinction is measured by

Rel⁡\-​L2​\(uθ,u∗;𝒳\)=‖uθ​\(𝒳\)−u∗​\(𝒳\)‖2‖u∗​\(𝒳\)‖2\.\\operatorname\{Rel\}\\text\{\-\}L\_\{2\}\(u\_\{\\theta\},u^\{\*\};\\mathcal\{X\}\)=\\frac\{\\left\\\|u\_\{\\theta\}\(\\mathcal\{X\}\)\-u^\{\*\}\(\\mathcal\{X\}\)\\right\\\|\_\{2\}\}\{\\left\\\|u^\{\*\}\(\\mathcal\{X\}\)\\right\\\|\_\{2\}\}\.\(23\)Reference values are used for evaluation and outer\-loop candidate selection in this work, never as labels in the inner PINN objective\.

## Appendix BAgentic Search Pipeline and Verification Protocols

To complement the component\-level descriptions in the main text, this appendix specifies the outer evolutionary control loop of EvoPINN\. The pipeline integrates telemetry diagnosis, UCB module scheduling, diversified generation, structural verification, and lineage updates acrossGGgenerations \(Algorithm[1](https://arxiv.org/html/2607.26490#alg1)\)\.

Algorithm 1EvoPINN Agentic Search Framework0:Target PDE

𝒫\\mathcal\{P\}; seed algorithm

A0=\(M0,T0\)A\_\{0\}=\(M\_\{0\},T\_\{0\}\); budgets

\{Bg\}g=0G−1\\\{B\_\{g\}\\\}\_\{g=0\}^\{G\-1\}; seeds

\{ξg\}g=0G−1\\\{\\xi\_\{g\}\\\}\_\{g=0\}^\{G\-1\}; total generations

GG; batch size

KpropK\_\{\\mathrm\{prop\}\}\.

0:Globally evolved algorithm

AGA\_\{G\}\.

1:Initialize evolutionary memory

R0←∅R\_\{0\}\\leftarrow\\varnothingand UCB scheduler statistics

2:for

g=0,…,G−1g=0,\\ldots,G\-1do

3:Evaluate active algorithm

AgA\_\{g\}under

\(𝒫,Bg,ξg\)\(\\mathcal\{P\},B\_\{g\},\\xi\_\{g\}\)and parse telemetry

EgE\_\{g\}
4:Compute deterministic diagnostic state

Dg←Diagnose⁡\(Eg\)D\_\{g\}\\leftarrow\\operatorname\{Diagnose\}\(E\_\{g\}\)
5:Select search module

cg∈\{M,T\}c\_\{g\}\\in\\\{M,T\\\}via recent\-window UCB scheduler

6:Assign

KpropK\_\{\\mathrm\{prop\}\}mechanism focuses

\{zg\(i\)\}i=1Kprop\\\{z\_\{g\}^\{\(i\)\}\\\}\_\{i=1\}^\{K\_\{\\mathrm\{prop\}\}\}to diversify candidate context

7:Generate

KpropK\_\{\\mathrm\{prop\}\}module replacements for

cgc\_\{g\}via LLM conditioned on context

𝒞g\(i\)\\mathcal\{C\}\_\{g\}^\{\(i\)\}
8:Form full candidate algorithms

\{Ag\(i\)\}i=1Kprop\\\{A\_\{g\}^\{\(i\)\}\\\}\_\{i=1\}^\{K\_\{\\mathrm\{prop\}\}\}by combining replacements with unchanged module

9:Apply structural AST verification \(

GstructG\_\{\\mathrm\{struct\}\}\) and pre\-training smoke tests \(

GpreG\_\{\\mathrm\{pre\}\}\)

10:Train all verified candidates under

\(𝒫,Bg,ξg\)\(\\mathcal\{P\},B\_\{g\},\\xi\_\{g\}\)in process\-isolated environments

11:Let

𝒱g\\mathcal\{V\}\_\{g\}be the set of candidates completing full evaluation with finite scores

12:if

𝒱g≠∅\\mathcal\{V\}\_\{g\}\\neq\\varnothingthen

13:

A^g←arg⁡minA∈𝒱g⁡Ssearch,g​\(A\)\\widehat\{A\}\_\{g\}\\leftarrow\\arg\\min\_\{A\\in\\mathcal\{V\}\_\{g\}\}S\_\{\\mathrm\{search\},g\}\(A\)
14:if

Ssearch,g​\(A^g\)<Ssearch,g​\(Ag\)S\_\{\\mathrm\{search\},g\}\(\\widehat\{A\}\_\{g\}\)<S\_\{\\mathrm\{search\},g\}\(A\_\{g\}\)then

15:

Ag\+1←A^gA\_\{g\+1\}\\leftarrow\\widehat\{A\}\_\{g\}\{Accept strict improvement into active lineage\}

16:else

17:

Ag\+1←AgA\_\{g\+1\}\\leftarrow A\_\{g\}\{Retain parent algorithm\}

18:endif

19:else

20:

Ag\+1←AgA\_\{g\+1\}\\leftarrow A\_\{g\}
21:endif

22:Update scheduler rewards

rc,gr\_\{c,g\}and record outcomes into evolutionary memory

Rg\+1R\_\{g\+1\}
23:endfor

24:return

AGA\_\{G\}

### B\.1Diagnostic State and UCB Module Scheduling

During each parent run, EvoPINN records the total and component\-wise physics\-informed losses at fixed checkpoints\. From this trace it reports five diagnostic cues to the search agent: overall loss contraction, the least\-squares trend over the late checkpoints, how strongly the late\-stage loss oscillates relative to the early stage, the final contribution of each physical constraint, and the log\-scale gap between the final training loss and the search\-time solution error\. These cues distinguish, for example, a run that is still improving from one that has plateaued, and a genuinely accurate solution from one that achieves a small physics loss without a correspondingly small solution error\.

For the two normalized quantities used by TCROP below, letLkL\_\{k\}denote the recorded total physics\-informed loss at checkpointkk, and letKckptK\_\{\\mathrm\{ckpt\}\}be the final recorded checkpoint\. We compute the loss contraction asρconv=LKckpt/\(L1\+ϵ\)\\rho\_\{\\mathrm\{conv\}\}=L\_\{K\_\{\\mathrm\{ckpt\}\}\}/\(L\_\{1\}\+\\epsilon\)and the relative late\-stage oscillation asρosc=Std⁡\{Lk:k∈𝒲late\}/\(Std⁡\{Lk:k∈𝒲early\}\+ϵ\)\\rho\_\{\\mathrm\{osc\}\}=\\operatorname\{Std\}\\\{L\_\{k\}:k\\in\\mathcal\{W\}\_\{\\mathrm\{late\}\}\\\}/\(\\operatorname\{Std\}\\\{L\_\{k\}:k\\in\\mathcal\{W\}\_\{\\mathrm\{early\}\}\\\}\+\\epsilon\)\. Here𝒲early\\mathcal\{W\}\_\{\\mathrm\{early\}\}and𝒲late\\mathcal\{W\}\_\{\\mathrm\{late\}\}are the fixed early and late checkpoint windows, respectively, andϵ\>0\\epsilon\>0is a numerical safeguard\.

The diagnostic summary guides proposal generation after module selection; it does not enter the UCB scheduler or replace the relative\-L2L\_\{2\}selection objective\. Loss components also depend on their weighting scheme, so the summary is used to interpret trends within a run rather than to compare raw loss values across different objectives or PDEs\.

The scheduler treats representation search \(MM\) and training\-program search \(TT\) as two arms\. After each generation, the selected arm receives a reward in\[0,1\]\[0,1\]based on four observable outcomes: whether the generation produced an accepted child, the normalized performance improvement, the fraction of executable candidates, and the batch’s structural novelty\. Table[9](https://arxiv.org/html/2607.26490#A4.T9)reports the reward weights and history length\.

Letr¯g​\(c\)\\bar\{r\}\_\{g\}\(c\)be the mean reward of moduleccover the recent history,Ng,cN\_\{g,c\}its number of observations in that history, andNg=Ng,M\+Ng,TN\_\{g\}=N\_\{g,M\}\+N\_\{g,T\}\. For an observed module, the next module is chosen by

cg=arg⁡maxc∈\{M,T\}⁡\[r¯g​\(c\)\+β​log⁡\(Ng\+1\)Ng,c\]\.c\_\{g\}=\\arg\\max\_\{c\\in\\\{M,T\\\}\}\\left\[\\bar\{r\}\_\{g\}\(c\)\+\\beta\\sqrt\{\\frac\{\\log\(N\_\{g\}\+1\)\}\{N\_\{g,c\}\}\}\\right\]\.\(24\)whereβ\\betacontrols exploration\. An unobserved module receives initialization priority rather than being inserted into the fraction with a zero denominator\. The concrete coefficients are given in Table[9](https://arxiv.org/html/2607.26490#A4.T9)\.

### B\.2Context Construction, Memory, and Proposal Diversity

After module selection, each proposal receives the target PDE, parent algorithm and score, diagnostic summary, selected module, a structured account of the parent mechanism, and the run\-scoped memory of earlier attempts\. The PDE specification contains the equation, domain, input\-output dimensions, derivative orders, physical conditions, and relevant structure such as transport, diffusion, wave propagation, anisotropy, periodicity, or multiple fields\. It never contains reference\-solution values\.

Representation focuses include coordinate and frequency bases, spatial and temporal organization, field coupling, localized structure, stable backbones, transport interactions, and global–local scale separation\. Training focuses include optimizer staging, collocation, objective shaping, residual\-driven refinement, diagnostic feedback, curricula, and time\-causal optimization\. Each proposal is assigned one focus from the relevant family, with coverage balanced across the batch\. This unified rule replaces PDE\-specific proposal quotas while preserving mechanism diversity\.

After evaluation, the memory records the selected module, assigned focus, mechanism family, score change, and execution outcome of each attempt\. Before the next generation, these records are summarized into promising mechanisms, regressions, neutral changes, repeated ideas, and failure modes\. The summary encourages refinement of successful ideas and discourages repetition without prescribing the next implementation\. Memory is reset for each independent search and is not transferred across PDEs\.

### B\.3Executable Generation and Verification

The generator returns one replacement module together with a structured mechanism description\. The two modules obey fixed interface contracts, denoted abstractly byMMandTT, so the selected replacement can be composed with the unchanged parent module\.

Verification begins with a structural comparison\. LetHrawH\_\{\\mathrm\{raw\}\}remove comments, documentation strings, and source locations from the parsed program, and letHnormH\_\{\\mathrm\{norm\}\}additionally normalize private identifiers and literal values while preserving public interfaces and framework operators\. The proposal is rejected if it matches the parent at any of the three levels: the original source, the comment\-free AST, or the normalized AST\. This removes exact copies, cosmetic rewrites, and changes confined to private names or literal constants\. It remains a practical test of structural novelty, not a proof of semantic novelty\.

Every structurally distinct proposal must then pass four pre\-training checks: it must parse, load as a module, satisfy the relevant interface contract, and complete a smoke test\. The smoke test composes the proposal with the unchanged parent module and exercises the target PDE’s forward computation, required derivatives, backward propagation, optimizer construction, sampling interfaces, device placement, finite\-value checks, resource bounds, and a short training run\. Failure at any stage prevents full\-budget evaluation\.

When a repairable failure occurs, the validation evidence is returned to the generator for a bounded number of self\-repair attempts\. Failures associated with an invalid mechanism or resource violation trigger regeneration from the parent rather than local patching\. Every repaired or regenerated proposal repeats the complete verification sequence\. After a full\-budget run, a final consistency gate checks agreement between the assigned focus and the reported mechanism metadata\. Only candidates that pass all gates and produce a finite score enter selection\.

### B\.4Evaluation and Reporting Separation

Every verified candidate is trained under the generation’s shared envelope and seed, then scored by the relative\-L2L\_\{2\}objective defined in the main paper\. Algorithm[1](https://arxiv.org/html/2607.26490#alg1)gives the complete selection and lineage update, so it is not repeated here\. Search\-time acceptance uses one full\-budget evaluation per candidate\. The frozen winner is subsequently retrained from scratch on independent seeds and reporting points; the single\-run search score is never presented as the final result\.

## Appendix CBenchmark Mathematics, Reference Topologies, and SLRC\-PINN

### C\.1Benchmark Equations and Reference Solutions

The four benchmarks span steady elliptic, nonlinear transport\-diffusion, hyperbolic, and anisotropic parabolic dynamics\.

##### Poisson2D\.

On

Ω=\\displaystyle\\Omega=\[−0\.5,0\.5\]2∖\\displaystyle\[\-5,5\]^\{2\}\\setminus\(25\)⋃\(a,b\)∈\{±0\.3\}2\{\(x,y\):\(x−a\)2\+\(y−b\)2<0\.12\},\\displaystyle\\bigcup\_\{\(a,b\)\\in\\\{\\pm 0\.3\\\}^\{2\}\}\\left\\\{\(x,y\):\(x\-a\)^\{2\}\+\(y\-b\)^\{2\}<0\.1^\{2\}\\right\\\},where\(x,y\)\(x,y\)are Cartesian coordinates and\(a,b\)\(a,b\)ranges over the four hole centers, the solution satisfies

ux​x\+uy​y=0,u\|∂\[−0\.5,0\.5\]2=1,u\|∂Ωhole=0\.u\_\{xx\}\+u\_\{yy\}=0,\\quad u\|\_\{\\partial\[\-0\.5,0\.5\]^\{2\}\}=1,\\quad u\|\_\{\\partial\\Omega\_\{\\mathrm\{hole\}\}\}=0\.\(26\)Hereux​x=∂2u/∂x2u\_\{xx\}=\\partial^\{2\}u/\\partial x^\{2\}anduy​y=∂2u/∂y2u\_\{yy\}=\\partial^\{2\}u/\\partial y^\{2\}, while∂Ωhole\\partial\\Omega\_\{\\mathrm\{hole\}\}denotes the union of the four circular hole boundaries\. Final reporting uses the finite\-element reference field used in our experiments\.

##### Burgers1D\.

For\(x,t\)∈\[−1,1\]×\[0,1\]\(x,t\)\\in\[\-1,1\]\\times\[0,1\],

ut\+u​ux−ν​ux​x=0,ν=0\.01π,u\_\{t\}\+uu\_\{x\}\-\\nu u\_\{xx\}=0,\\qquad\\nu=\\frac\{0\.01\}\{\\pi\},\(27\)where subscripts denote partial derivatives andν\\nuis the viscosity, withu​\(x,0\)=−sin⁡\(π​x\)u\(x,0\)=\-\\sin\(\\pi x\)andu​\(−1,t\)=u​\(1,t\)=0u\(\-1,t\)=u\(1,t\)=0\. Final reporting uses the complete numerical reference grid used in our experiments\.

##### Wave1D\.

For\(x,t\)∈\[0,1\]2\(x,t\)\\in\[0,1\]^\{2\},

ut​t−4​ux​x=0,u\_\{tt\}\-4u\_\{xx\}=0,\(28\)whereut​tu\_\{tt\}andux​xu\_\{xx\}denote the second derivatives with respect to time and space, respectively\. The boundary values and initial displacement are induced by the exact solution

u∗​\(x,t\)=\\displaystyle u^\{\*\}\(x,t\)=\{\}sin⁡\(π​x\)​cos⁡\(2​π​t\)\\displaystyle\\sin\(\\pi x\)\\cos\(2\\pi t\)\(29\)\+12​sin⁡\(a​π​x\)​cos⁡\(2​a​π​t\),a=4,\\displaystyle\+\\tfrac\{1\}\{2\}\\sin\(a\\pi x\)\\cos\(2a\\pi t\),\\qquad a=4,together with the initial velocity conditionut​\(x,0\)=0u\_\{t\}\(x,0\)=0\.

##### Heat2D\.

For\(x,y,t\)∈\[0,1\]2×\[0,5\]\(x,y,t\)\\in\[0,1\]^\{2\}\\times\[0,5\],

ut−κx​ux​x−κy​uy​y=0,u\_\{t\}\-\\kappa\_\{x\}u\_\{xx\}\-\\kappa\_\{y\}u\_\{yy\}=0,\(30\)whereκx\\kappa\_\{x\}andκy\\kappa\_\{y\}are the diffusivities along thexxandyydirections, with zero spatial boundary values and the exact initial condition induced by

u∗​\(x,y,t\)=sin⁡\(kx​x\)​sin⁡\(ky​y\)​exp⁡\[−\(κx​kx2\+κy​ky2\)​t\],u^\{\*\}\(x,y,t\)=\\sin\(k\_\{x\}x\)\\sin\(k\_\{y\}y\)\\exp\[\-\(\\kappa\_\{x\}k\_\{x\}^\{2\}\+\\kappa\_\{y\}k\_\{y\}^\{2\}\)t\],\(31\)where

κx=\(500​π\)−2,κy=π−2,\(kx,ky\)=\(20​π,π\)\.\\kappa\_\{x\}=\(500\\pi\)^\{\-2\},\\quad\\kappa\_\{y\}=\\pi^\{\-2\},\\quad\(k\_\{x\},k\_\{y\}\)=\(20\\pi,\\pi\)\.\(32\)Herekxk\_\{x\}andkyk\_\{y\}are the corresponding spatial wavenumbers\. For Wave1D and Heat2D, final errors are evaluated on independently sampled interior coordinates from the analytic solution\.

### C\.2Reference Topologies of the Frozen Algorithms

The reporting experiments retrain fixed algorithms from scratch; they do not invoke the generator\. Table[6](https://arxiv.org/html/2607.26490#A3.T6)summarizes the principal representation topology, training mechanism, and fixed structural settings of each frozen algorithm\.

Table 6:Frozen EvoPINN algorithms used for independent reporting retraining\.
### C\.3SLRC\-PINN Implementation Notes

EvoPINN discovered SLRC\-PINN during the Burgers1D search as an accepted representation change rather than through post\-search manual design\. The main paper gives the complete mathematical topology; here we record the implementation details needed to interpret that formulation\. The input isq=\(x,t\)∈ℝ2q=\(x,t\)\\in\\mathbb\{R\}^\{2\}, and the basis bank contains six learned centers, log\-scales, and coordinate\-dependent amplitude, modulation, and scale projections\. The raw coordinate and all six basis responses are concatenated before the gated residual backbone\. Separate linear decoders produce the global prediction and local correction, while a learned scalar gate combines the basis responses to modulate only the corrective branch\.

The final linear layer of the local decoder is initialized with zero weights and zero bias\. ThereforeuL​\(q\)≡0u\_\{L\}\(q\)\\equiv 0and the initial network function is exactly the global branch, independent of the initial gate values\. All basis centers, widths, projections, backbone parameters, decoders, and gate parameters are subsequently optimized jointly by the same physics\-informed objective\. No subdomain labels, interface losses, or manually specified localization masks are used\.

### C\.4Controlled SLRC\-PINN Comparison

The controlled study compares SLRC\-PINN with the parameter\-matched global MLP, APINN, FBPINN, and HyResPINNs listed in the main paper\. All methods use the same Burgers equation, Hammersley collocation, physics\-informed mean\-squared objective, 20,000 Adam–L\-BFGS optimization steps, relative\-L2L\_\{2\}scorer, and evaluation seeds\{0,1,2,3,4\}\\\{0,1,2,3,4\\\}\. Parameter counts are computed after construction and document the parameter\-matched comparison; they do not alter the training budget\. APINN, FBPINN, and HyResPINNs preserve their defining architectural structures under the shared PyTorch/DeepXDE runner rather than claiming exact reproduction of the numerical protocols in their original studies\.

SLRC\-PINN is a mechanism\-discovery case study and is distinct from the frozen Burgers1D algorithm used in the main benchmark table\. The former isolates and tests the discovered global–local mechanism; the latter is selected strictly by the search objective for the principal comparison\.

### C\.5TCROP: Telemetry\-Conditioned Program Discovery

Beyond neural representation search, EvoPINN successfully synthesized an executable training policy for Poisson2D, termed theTelemetry\-Conditioned Resampling and Optimizer Policy \(TCROP\)\. This case study examines its formulation, structural mechanisms, and optimization dynamics\. TCROP is a particularly revealing discovery because it shows that agentic evolution can produce a coordinated training program, rather than merely select an optimizer or tune an isolated scalar hyperparameter\.

#### Algorithmic Formulation and Policy Mapping

TCROP maps the compressed diagnostic statesgs\_\{g\}of a parent run to a training scheduleπ​\(sg\)=\(BAdam,BLBFGS,rrefresh\)\\pi\(s\_\{g\}\)=\(B\_\{\\mathrm\{Adam\}\},B\_\{\\mathrm\{LBFGS\}\},r\_\{\\mathrm\{refresh\}\}\), whereBAdamB\_\{\\mathrm\{Adam\}\}andBLBFGSB\_\{\\mathrm\{LBFGS\}\}are the step allocations for the two optimizers andrrefreshr\_\{\\mathrm\{refresh\}\}is the collocation\-refresh period\. The state is

sg=\(\\displaystyle s\_\{g\}=\\bigl\(zconv,ztrend,zosc,ρconv,ρosc,\\displaystyle z\_\{\\mathrm\{conv\}\},z\_\{\\mathrm\{trend\}\},z\_\{\\mathrm\{osc\}\},\\rho\_\{\\mathrm\{conv\}\},\\rho\_\{\\mathrm\{osc\}\},\(33\)slate,srecent,nlog,d\)\.\\displaystyle s\_\{\\mathrm\{late\}\},s\_\{\\mathrm\{recent\}\},n\_\{\\mathrm\{log\}\},d\\bigr\)\.wherezconvz\_\{\\mathrm\{conv\}\},ztrendz\_\{\\mathrm\{trend\}\}, andzoscz\_\{\\mathrm\{osc\}\}are the categorical convergence, trend, and oscillation labels produced by the deterministic diagnostic map in Appendix[B\.1](https://arxiv.org/html/2607.26490#A2.SS1);ρconv\\rho\_\{\\mathrm\{conv\}\}andρosc\\rho\_\{\\mathrm\{osc\}\}are the continuous contraction and relative\-oscillation statistics defined there;nlogn\_\{\\mathrm\{log\}\}is the number of valid logged checkpoints; andddis the dominant\-constraint label\. The synthesized second\-order readiness condition is

δsmooth=\\displaystyle\\delta\_\{\\mathrm\{smooth\}\}=\{\}𝕀​\[zosc=low\]\\displaystyle\\mathbb\{I\}\[z\_\{\\mathrm\{osc\}\}=\\text\{low\}\]∨𝕀​\[ρosc<0\.03\],\\displaystyle\\lor\\mathbb\{I\}\[\\rho\_\{\\mathrm\{osc\}\}<0\.03\],\(34\)δplateau=\\displaystyle\\delta\_\{\\mathrm\{plateau\}\}=\{\}𝕀​\[\|slate\|<2\.5×10−4\]\\displaystyle\\mathbb\{I\}\[\|s\_\{\\mathrm\{late\}\}\|<2\.5\\times 10^\{\-4\}\]∧𝕀​\[\|srecent\|<2\.5×10−4\],\\displaystyle\\land\\mathbb\{I\}\[\|s\_\{\\mathrm\{recent\}\}\|<2\.5\\times 10^\{\-4\}\],\(35\)δready=\\displaystyle\\delta\_\{\\mathrm\{ready\}\}=\{\}δsmooth∧\(δconv∨δplateau\)\\displaystyle\\delta\_\{\\mathrm\{smooth\}\}\\land\\bigl\(\\delta\_\{\\mathrm\{conv\}\}\\lor\\delta\_\{\\mathrm\{plateau\}\}\\bigr\)∧𝕀​\[nlog≥10\]\.\\displaystyle\\land\\mathbb\{I\}\[n\_\{\\mathrm\{log\}\}\\geq 10\]\.\(36\)where∨\\lorand∧\\landdenote logical OR and AND;slates\_\{\\mathrm\{late\}\}andsrecents\_\{\\mathrm\{recent\}\}are least\-squares loss slopes over the fixed late and most\-recent checkpoint windows, respectively; andδsmooth\\delta\_\{\\mathrm\{smooth\}\},δplateau\\delta\_\{\\mathrm\{plateau\}\}, andδready\\delta\_\{\\mathrm\{ready\}\}are binary indicators of smoothness, plateauing, and second\-order readiness\. Finally,δconv=𝕀​\[zconv=high\]\\delta\_\{\\mathrm\{conv\}\}=\\mathbb\{I\}\[z\_\{\\mathrm\{conv\}\}=\\text\{high\}\]indicates a well\-converged residual trajectory\.

For the reported Poisson2D trajectory, the logged diagnostic state satisfiesδready=1\\delta\_\{\\mathrm\{ready\}\}=1\. TCROP therefore adjusts the standard training allocation\. Under a total nominal resource envelope ofB=20,000B=20\{,\}000steps, it allocates:

BAdam=17,200,BLBFGS=2,800,B\_\{\\mathrm\{Adam\}\}=17\{,\}200,\\qquad B\_\{\\mathrm\{LBFGS\}\}=2\{,\}800,\(37\)paired with an Adam learning rate of5×10−45\\times 10^\{\-4\}and an active global collocation refresh gate enabled at periodrrefresh=1,000r\_\{\\mathrm\{refresh\}\}=1\{,\}000\.

#### Mechanistic Advantage: Programmatic Synergy of Policy and Resampling

To interpret TCROP’s superior accuracy, we examine the two mechanisms that the generated program coordinates\. Rather than relying on a rigid, static optimizer handoff, TCROP combines:

1. 1\.Telemetry\-Driven Budget Allocation:Under the observed readiness state, TCROP extends the initial Adam phase toBAdam=17,200B\_\{\\mathrm\{Adam\}\}=17\{,\}200steps before assigningBLBFGS=2,800B\_\{\\mathrm\{LBFGS\}\}=2\{,\}800steps to quasi\-Newton refinement\. The resulting combined policy substantially outperforms the conventional equal\-split baseline\.
2. 2\.Synergistic Global Collocation Renewal:During Adam, TCROP activates global resampling every1,0001\{,\}000steps, producing 17 point renewals across the interior and boundary domains\. These renewals repeatedly expose the optimizer to fresh physical constraints and are designed to reduce early over\-specialization to a fixed collocation set before L\-BFGS\.

This programmatic synergy highlights a key strength of EvoPINN: the framework does not merely tune scalar hyperparameters, but autonomously synthesizes holistic, execution\-grounded training policies from observed convergence behavior\.

#### Two\-Phase Optimization Dynamics: Stochastic Preconditioning and Second\-Order Refinement

We empirically validate the discovered TCROP policy against standard optimization baselines across five paired random seeds under identical initializations and resource envelopes \(Table[7](https://arxiv.org/html/2607.26490#A3.T7)\)\. The discovered TCROP policy achieves a27\.5% reduction in geometric\-mean relativeL2L\_\{2\}errorcompared to the conventional10​k/10​k10\\text\{k\}/10\\text\{k\}handoff, while simultaneously lowering seed variability \(CV=19\.0%\\mathrm\{CV\}=19\.0\\%\)\.

Table 7:Controlled Poisson2D comparison over five paired seeds\. GM denotes geometric mean, and time is measured in minutes\. TCROP dynamically discovers and executes the17\.2​k/2\.8​k17\.2\\text\{k\}/2\.8\\text\{k\}optimizer allocation paired with gated global collocation renewals\.##### Optimization\-Trajectory Analysis\.

The optimizer\-transition statistics provide a direct view of how the two protocols enter L\-BFGS:

- •Substantially Lower Entry Loss:At step17,20017\{,\}200, the geometric\-mean training loss under TCROP reaches6\.90×10−26\.90\\times 10^\{\-2\}\. In contrast, the conventional handoff at step10,00010\{,\}000leaves an entry loss of2\.81×10−12\.81\\times 10^\{\-1\}—which is4\.1×4\.1\\timeshigher\.
- •Precondition\-and\-Polish Effect:The extended Adam phase and periodic renewals deliver a markedly lower\-loss state to L\-BFGS\. Together with the lower final error, this supports the interpretation that TCROP creates a more favorable entry point for second\-order refinement\.

##### Execution\-Fidelity Control\.

A stateless control executing the exact discovered17\.2​k/2\.8​k17\.2\\text\{k\}/2\.8\\text\{k\}schedule and refresh actions reproduces TCROP’s accuracy \(5\.06×10−35\.06\\times 10^\{\-3\}\)\. This control verifies that the gain is carried by the executable program emitted by the search rather than by hidden outer\-loop state at reporting time\. It is not intended to isolate the individual causal contribution of the budget split and resampling; the scientific claim here concerns the effectiveness and reproducibility of the discovered joint policy\.

## Appendix DImplementation and Evaluation Protocols

### D\.1Unified Budget and Evaluation Protocol

The generation\-specific resource envelope is

Bg=\(Nr,g\(0\),Nb,g\(0\),Ni,g\(0\),Ktrain,g,𝒪g,𝒬g\),B\_\{g\}=\\left\(N\_\{r,g\}^\{\(0\)\},N\_\{b,g\}^\{\(0\)\},N\_\{i,g\}^\{\(0\)\},K\_\{\\mathrm\{train\},g\},\\mathcal\{O\}\_\{g\},\\mathcal\{Q\}\_\{g\}\\right\),\(38\)where the sample counts specify the initial counts of interior, boundary, and initial constraints,Ktrain,gK\_\{\\mathrm\{train\},g\}is the total optimization\-step limit,𝒪g\\mathcal\{O\}\_\{g\}is the permitted optimizer family, and𝒬g\\mathcal\{Q\}\_\{g\}contains the configured adaptive\-sampling controls\. All candidates within a generation share the sameBgB\_\{g\}, which may be adjusted between generations\. A candidate may reorganize optimization phases, losses, and sampling using these controls, but it may not alter the PDE, physical constraints, reference points, score definition, or total step budget\. Matching the envelope aligns the principal training resources; it does not imply identical wall\-clock time or floating\-point operation count\.

Search\-time selection and final reporting use distinct reference point sets\. The outer loop can access only the search set used to score candidates\. After selection, the algorithm is frozen, retrained from scratch with previously unused reporting seeds, and evaluated on independently prepared reporting points\. No reporting coordinate is used for candidate selection\. This separation prevents the main reported results from reusing the point set that drives outer\-loop evolution\.

Table 8:Approximate cost recorded for the designated source search of each benchmark\.Table 9:Implementation hyperparameters collected in one place\. These values specify the reported protocol but are not presented as universal properties of EvoPINN\.
### D\.2Search, Retraining, and Expert Controls

Three independent searches are run for each PDE\. The principal frozen algorithm is the best archived algorithm from the designated source search with seed 1234, chosen by its lowest full\-evaluationSsearchS\_\{\\mathrm\{search\}\}before any reporting retraining\. The remaining searches provide robustness evidence and do not enlarge the selection pool for the main comparison\. The seed PINN, selected expert, and frozen EvoPINN are then retrained from scratch with the same five reporting seeds\. Expert configurations are selected with seed 1234 during a separate screening stage; screening runs are excluded from reported expert statistics\. Ablation and neighboring\-instance experiments follow the budgets stated in the main paper and do not replace the principal reporting protocol\. In the neighboring\-instance experiments, the discovered representation and training\-program modules \(and the selected expert identities\) are kept fixed, while network parameters are reinitialized and trained from scratch at each shifted parameter value; no additional algorithm search is performed\.

Candidate evaluations run in isolated processes and must complete within the configured time limit with a finite score\. Failed or timed\-out candidates cannot enter selection\. Random states for the numerical and learning frameworks are reset before each evaluation\. This controls the intended random initial conditions but does not imply bitwise\-identical accelerator execution\.

### D\.3Prompt and Interaction Protocol

Each proposal prompt supplies the PDE, parent algorithm, selected module, mechanism focus, resource envelope, diagnostic state, and summarized search history\. It requests a complete executable replacement while prohibiting changes to the benchmark, reference data, scoring rule, and unchanged module\. Repair prompts add validation evidence from the failed proposal but never reveal reference\-solution values or reporting results\. Initial and repaired proposals pass the same structural, interface, resource, and execution gates\.

### D\.4Hardware and Computational Cost

The experiments were executed on NVIDIA GeForce RTX 3090 GPUs with 24 GB of memory using DeepXDE 1\.15\.0 and PyTorch 2\.6\.0 under a common Python, NumPy, CUDA, and cuDNN environment\. The approximate execution costs of the four designated source\-search runs are reported in Table[8](https://arxiv.org/html/2607.26490#A4.T8)\. These totals include both failed and successful candidate attempts and therefore represent the observed search cost under our experimental environment rather than a hardware\-independent complexity measure\.