MEMENTO: Memory-Guided Memetic Code-as-Policy Evolution

# MEMENTO: Memory-Guided Memetic Code-as-Policy Evolution
Source: [https://arxiv.org/html/2607.22832](https://arxiv.org/html/2607.22832)
\\nameAlkis Sygkounas\\addralkis\.sygkounas@oru\.se \\addrCenter for Applied Autonomous Sensor Systems \(AASS\), Örebro University, Sweden\\nameVictor Aregbede\\addrvictor\.aregbede@oru\.se \\addrCenter for Applied Autonomous Sensor Systems \(AASS\), Örebro University, Sweden\\nameAmy Loutfi\\addramy\.loutfi@oru\.se \\addrÖrebro University, 70281 Örebro, Sweden and Linköing University, 58183 Linköping, Sweden\\nameAndreas Persson\\addrandreas\.persson@oru\.se \\addrCenter for Applied Autonomous Sensor Systems \(AASS\), Örebro University, Sweden

###### Abstract

Long\-horizon embodied tasks require policies that execute many dependent actions before task success can be observed\. Representing policies as executable control programs \(code\-as\-policy\) enables their decision logic to be inspected and revised after rollout evaluation\. Revised programs can then be executed and compared by rollout performance, framing policy improvement as execution\-guided program search\. Evolutionary methods driven by large language models \(LLMs\) provide a natural mechanism for this search by generating variants and selecting high\-performing candidates\. However, existing approaches primarily select among independently generated variants and lack a sequential local improvement phase\. We introduceMEMENTO, a memory\-guided single\-elite memetic framework for code\-as\-policy evolution\.MEMENTOfirst evolves a rollout evaluator that maps policy rollouts to scalar fitness and structured feedback metrics\. Fitness selects accepted candidates and the next elite, while feedback metrics condition policy proposals generated by memory\-guided hill\-climbing, macro\-mutation, and crossover\. We evaluateMEMENTOon two long\-horizon embodied domains: Robosuite Franka Tower\-of\-Hanoi manipulation and AI2\-THOR household interaction\.MEMENTOoutperforms Eureka and REvolve, adapted as code\-as\-policy evolutionary baselines, in task success and generalization to held\-out Robosuite object configurations and unseen AI2\-THOR scenes\. Ablations show that zero\-shot generation and unevolved evaluators fail to solve either domain, and that removing policy\-search branches reduces performance\. Finally, we deploy the best\-evolved Robosuite policy on a physical Franka robot, demonstrating the feasibility of sim\-to\-real transfer of the evolved code\-as\-policy\. Code, prompts, and videos are available at:https://github\.com/sygkounas/MEMENTO\.

Keywords

> Memetic algorithms, genetic programming, code\-as\-policy, large language models, sim\-to\-real transfer\.

## 1Introduction

Long\-horizon embodied control tasks require policies to complete many dependent actions before task success can be evaluated\(Meeset al\.,[2022](https://arxiv.org/html/2607.22832#bib.bib36); Shridharet al\.,[2020](https://arxiv.org/html/2607.22832#bib.bib7); Srivastavaet al\.,[2022](https://arxiv.org/html/2607.22832#bib.bib32)\)\. Earlier actions determine the task state encountered by later actions\(Shridharet al\.,[2020](https://arxiv.org/html/2607.22832#bib.bib7); Srivastavaet al\.,[2022](https://arxiv.org/html/2607.22832#bib.bib32)\)\. Correct action order is therefore insufficient; each action must also be physically executed from the observed task state\. This creates a long\-horizon credit\-assignment problem because task success is observed only after many dependent actions\(Arjona\-Medinaet al\.,[2019](https://arxiv.org/html/2607.22832#bib.bib44)\)\. Existing approaches reduce the effect of delayed feedback by adding external structure to policy learning or by changing how experience is generated and credited\. Demonstration\-based and hierarchical methods can constrain long\-horizon behavior through trajectories, subgoal orderings, or reusable skills, while reinforcement\-learning methods make delayed task feedback more usable through exploration, relabeling, temporal abstraction, or return redistribution\(Zhaoet al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib1); Wanget al\.,[2023a](https://arxiv.org/html/2607.22832#bib.bib3); Shridharet al\.,[2020](https://arxiv.org/html/2607.22832#bib.bib7); Padmakumaret al\.,[2022](https://arxiv.org/html/2607.22832#bib.bib9); Andreaset al\.,[2017](https://arxiv.org/html/2607.22832#bib.bib10); Andrychowiczet al\.,[2017](https://arxiv.org/html/2607.22832#bib.bib12); Pathaket al\.,[2017](https://arxiv.org/html/2607.22832#bib.bib14); Ecoffetet al\.,[2021](https://arxiv.org/html/2607.22832#bib.bib16); Suttonet al\.,[1999](https://arxiv.org/html/2607.22832#bib.bib17); Arjona\-Medinaet al\.,[2019](https://arxiv.org/html/2607.22832#bib.bib44)\)\. These approaches improve long\-horizon learning, but they do not directly expose the decision logic of a failed policy or identify which part of that logic should be revised\.

When policies are represented as executable programs, decision logic is written directly as program code and can thereby be inspected and revised\. Large language models \(LLMs\) generate robot policy code that processes perception outputs and parameterizes control\-primitive APIs, a formulation termed code\-as\-policy\(Lianget al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib20)\)\. This formulation has been extended to contact\-rich control and video\-conditioned policy generation\(Burnset al\.,[2024](https://arxiv.org/html/2607.22832#bib.bib56); Xieet al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib57)\)\. In long\-horizon tasks, however, a generated program can execute without errors while its control logic fails to achieve the task\(Liuet al\.,[2024](https://arxiv.org/html/2607.22832#bib.bib66)\), and failure feedback from rollouts has, therefore, been used to characterize policy errors and guide refinement\(Yinget al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib58)\)\. Code\-as\-policy improvement can then be formulated as execution\-guided program search, in which candidate policy programs are executed in the task environment, evaluated based on observed behavior, and revised in response to failure feedback\. Binary task success alone, however, is too sparse to guide the revision of a long\-horizon policy program, and structured rollout feedback indicating which policy decisions to revise is difficult to specify by hand\.

In genetic programming, candidate programs are executed and assigned a fitness score based on performance, which is then used to select programs for variation by micro\-mutations that make small local changes, macro\-mutations that modify larger program components, and crossover\(Koza,[1992](https://arxiv.org/html/2607.22832#bib.bib27); Brameier and Banzhaf,[2007](https://arxiv.org/html/2607.22832#bib.bib59); Angeline,[1997](https://arxiv.org/html/2607.22832#bib.bib60)\)\. High\-performing programs often require many candidate evaluations\(Echevarrietaet al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib61)\), making undirected syntactic variation costly in rollout\-evaluated embodied domains\. Language models have since been used as variation operators over executable programs, replacing syntactic program modifications with prompt\-conditioned code proposals\(Lehmanet al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib62); Hemberget al\.,[2024](https://arxiv.org/html/2607.22832#bib.bib63)\)\. LLM\-driven code evolution has also been applied to executable policy programs, where candidate controllers are evaluated in task environments and selected for further revision\(Huet al\.,[2026](https://arxiv.org/html/2607.22832#bib.bib64)\)\. In embodied control, LLM\-guided evolutionary methods have evolved reward code via LLM\-generated variation, with candidate reward functions selected based on policy performance\(Maet al\.,[2024](https://arxiv.org/html/2607.22832#bib.bib8); Hazraet al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib21)\)\. However, existing code\-as\-policy and reward\-code evolution methods primarily select among independently generated variants; they neither refine an accepted candidate through a local\-search chain nor retain a memory of revisions that have already failed\. Memetic algorithms support this form of refinement by combining evolutionary variation with local search on selected candidates\(Moscato,[1989](https://arxiv.org/html/2607.22832#bib.bib65); Neri and Cotta,[2012](https://arxiv.org/html/2607.22832#bib.bib38)\)\. The local\-search component enables exploitation of an accepted candidate via iterative program revision, while mutation and crossover introduce additional variation via larger changes and recombination\.

![Refer to caption](https://arxiv.org/html/2607.22832v1/x1.png)Figure 1:Conceptual overview of MEMENTO\. The top row shows the task environments used in the study, including a Robosuite Franka Tower\-of\-Hanoi task and AI2\-THOR household interaction scenes\. The left panel illustrates evaluator evolution: candidate evaluators are generated, judged by an LLM majority vote, and iteratively refined to obtain the evolved evaluatorE⋆E^\{\\star\}\. The right panel illustrates policy evolution under the fixed evaluatorE⋆E^\{\\star\}\. An initial policyp0p\_\{0\}is evaluated to form the current elite candidate state𝒞g\\mathcal\{C\}\_\{g\}, after which each generation applies three proposal mechanisms: sequential hill\-climbing from the current accepted candidate with rejected\-proposal memoryℋ\\mathcal\{H\}, macro\-mutation from the elite, and crossover between the best hill\-climb and macro\-mutation candidates\. The best resulting candidate is selected as the next elite𝒞g\+1\\mathcal\{C\}\_\{g\+1\}, and the process repeats forGGgenerations\.Local iterative improvement, search memory, and evolutionary variation via micro\-mutation, macro\-mutation, and crossover have not been combined in embodied policy programs that have been evaluated through environment rollouts\. This limitation is addressed byMEMENTO, a memory\-guided single\-elite memetic search method for code\-as\-policy evolution in long\-horizon embodied tasks\. As illustrated in Figure[1](https://arxiv.org/html/2607.22832#S1.F1),MEMENTOfirst evolves an evaluatorE⋆E^\{\\star\}that maps policy rollouts to scalar fitness and structured feedback metrics and then freezes it for code\-as\-policy search\. Starting from an executable policy program, candidates are generated in each generation through three revision branches: sequential hill\-climbing through micro\-mutations from the current accepted candidate with rejected\-proposal memory, macro\-mutation from the elite, and crossover between the best hill\-climb and macro\-mutation candidates\. The best branch output becomes the elite for the next generation\.MEMENTOis evaluated on two long\-horizon embodied domains: fully observed Robosuite Franka Tower\-of\-Hanoi\(Zhuet al\.,[2020](https://arxiv.org/html/2607.22832#bib.bib50); Havuret al\.,[2013](https://arxiv.org/html/2607.22832#bib.bib48); Dugganet al\.,[2026](https://arxiv.org/html/2607.22832#bib.bib35)\), in which object placements define the legal Hanoi state for subsequent pick\-and\-place moves, and partially observed AI2\-THOR household interaction\(Kolveet al\.,[2017](https://arxiv.org/html/2607.22832#bib.bib30)\), in which valid object interactions depend on egocentric visibility and scene\-specific object identifiers\. The final Robosuite Tower\-of\-Hanoi policy is also deployed on a physical Franka robot, showcasing the feasibility of transferring policies evolved through simulation to real\-world settings\. In particular, the contributions are as follows:

- •a single\-elite memetic search framework for executable policy programs that combines sequential local improvement with rejected\-proposal memory, macro\-mutation, and crossover;
- •an evaluator\-evolution stage that produces a fixed evaluator before policy search, providing scalar fitness for selection and structured rollout feedback metrics for conditioning code\-as\-policy revisions;
- •an empirical evaluation on two long\-horizon embodied domains, including zero\-shot, evaluator, and operator ablations, generalization across object\-geometry and scene variations;
- •a physical Franka deployment demonstrating sim\-to\-real transfer of the evolved Robosuite controller\.

## 2Related Work

##### Long\-horizon embodied control\.

Long\-horizon embodied tasks have been approached by constraining policy optimization with prior information\. Extended manipulation behavior has been learned from teleoperated or language\-annotated demonstrations in ACT\(Zhaoet al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib1)\), Diffusion Policy\(Chiet al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib2)\), MimicPlay\(Wanget al\.,[2023a](https://arxiv.org/html/2607.22832#bib.bib3)\), and BLADE\(Liuet al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib4)\), and demonstration and interaction data for compositional tasks are provided by the ALFRED\(Shridharet al\.,[2020](https://arxiv.org/html/2607.22832#bib.bib7)\), TEACh\(Padmakumaret al\.,[2022](https://arxiv.org/html/2607.22832#bib.bib9)\), CALVIN\(Meeset al\.,[2022](https://arxiv.org/html/2607.22832#bib.bib36)\), and BEHAVIOR\(Srivastavaet al\.,[2022](https://arxiv.org/html/2607.22832#bib.bib32)\)benchmarks\. Task structure has also been specified through subtask sequences or program decompositions, as in Policy Sketches\(Andreaset al\.,[2017](https://arxiv.org/html/2607.22832#bib.bib10)\)and Neural Task Programming\(Xuet al\.,[2018](https://arxiv.org/html/2607.22832#bib.bib11)\), and ordering\-constrained manipulation has been studied in rearrangement and Tower\-of\-Hanoi settings\(Dugganet al\.,[2026](https://arxiv.org/html/2607.22832#bib.bib35); Kimet al\.,[2026](https://arxiv.org/html/2607.22832#bib.bib34)\)\. Sparse\-reward reinforcement learning instead addresses delayed task feedback through goal relabeling, exploration, temporal abstraction, return decomposition, and model\-based rollouts\(Andrychowiczet al\.,[2017](https://arxiv.org/html/2607.22832#bib.bib12); Pathaket al\.,[2017](https://arxiv.org/html/2607.22832#bib.bib14); Ecoffetet al\.,[2021](https://arxiv.org/html/2607.22832#bib.bib16); Suttonet al\.,[1999](https://arxiv.org/html/2607.22832#bib.bib17); Arjona\-Medinaet al\.,[2019](https://arxiv.org/html/2607.22832#bib.bib44); Hafneret al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib67)\)\. Across these approaches, delayed feedback is addressed by adding supervision, imposing temporal structure, or changing how experience is generated and credited during learning\.

##### Language models for robot programs\.

Language models have been used to map task context to robot behavior, either by selecting among predefined skills or by generating executable programs\. Action selection has been grounded with learned affordance and value estimates in SayCan\(Ichteret al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib19)\), and execution feedback and language\-conditioned spatial reasoning have been incorporated into language\-model planning in Inner Monologue\(Huanget al\.,[2023b](https://arxiv.org/html/2607.22832#bib.bib23)\), VoxPoser\(Huanget al\.,[2023a](https://arxiv.org/html/2607.22832#bib.bib24)\), and LLM\-Planner\(Songet al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib22)\)\. Program\-structured task plans have been produced from program\-like prompts in ProgPrompt\(Singhet al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib68)\)\. Policy programs that process perception outputs and parameterize control\-primitive APIs have been generated in Code as Policies\(Lianget al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib20)\), and in RoboPro\(Xieet al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib57)\)and GenCHiP\(Burnset al\.,[2024](https://arxiv.org/html/2607.22832#bib.bib56)\)for visually grounded and contact\-rich control\. Rollout failures in generated policy programs have been characterized and used for failure\-feedback refinement in RoboInspector\(Yinget al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib58)\), and execution summaries have been used to explain failed robot trials and propose corrective actions in REFLECT\(Liuet al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib69)\)\. Together, these works show how language models can produce, ground, and refine robot programs from task context and execution feedback\.

##### LLM\-driven evolutionary search\.

This line of research follows genetic programming, in which executable programs are varied through mutation and crossover and selected for fitness\(Koza,[1992](https://arxiv.org/html/2607.22832#bib.bib27)\)\. Language models have since been used as the variation operators, with model\-conditioned edits replacing syntactic operators in ELM\(Lehmanet al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib62)\)and LLM\_GP\(Hemberget al\.,[2024](https://arxiv.org/html/2607.22832#bib.bib63)\)\. The methods that follow differ primarily in the object being searched\. Discrete prompts have been evolved through model\-generated mutation and crossover in EvoPrompt\(Guoet al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib28)\)and self\-referentially in Promptbreeder\(Fernandoet al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib29)\)\. Executable programs have been evolved by pairing a frozen model with a fixed evaluator in FunSearch\(Romera\-Paredeset al\.,[2024](https://arxiv.org/html/2607.22832#bib.bib5)\)and, at codebase scale, in AlphaEvolve\(Novikovet al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib70)\), and heuristics have been evolved through accumulated reflective feedback in ReEvo\(Yeet al\.,[2024](https://arxiv.org/html/2607.22832#bib.bib71)\)and through memetic local search in EoH\-S\(Liuet al\.,[2026](https://arxiv.org/html/2607.22832#bib.bib72)\)\. Reward code has been evolved from model\-generated variation selected by policy performance in Eureka\(Maet al\.,[2024](https://arxiv.org/html/2607.22832#bib.bib8)\), and with population\-based crossover and human\-feedback fitness in REvolve\(Hazraet al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib21)\)\. REvolve has also been extended from reward\-function discovery to the discovery of executable reinforcement\-learning algorithms, in which candidate update rules are evaluated through full training runs\(Sygkounaset al\.,[2026b](https://arxiv.org/html/2607.22832#bib.bib51)\), while policy and environment programs are co\-evolved in COvolve\(Sygkounaset al\.,[2026a](https://arxiv.org/html/2607.22832#bib.bib52)\)\. Programmatic control policies have been evolved directly in MLES\(Huet al\.,[2026](https://arxiv.org/html/2607.22832#bib.bib64)\), where multimodal models generate candidate policy programs, selection uses quantitative task metrics, and new proposals are conditioned on behavioral evidence from rollouts\. In these methods, the evaluator is fixed or externally specified, and the search object is a prompt, a heuristic, a reward, a program, or a program pair\.

##### Memetic and memory\-based search\.

Memetic algorithms combine population\-level variation with local improvement of selected candidates\(Moscato,[1989](https://arxiv.org/html/2607.22832#bib.bib65); Neri and Cotta,[2012](https://arxiv.org/html/2607.22832#bib.bib38)\)\. Recombination has been coupled with local refinement in crossover hill\-climbing\(Lozanoet al\.,[2004](https://arxiv.org/html/2607.22832#bib.bib73)\), and local search has been concentrated on promising candidates through local\-search chains\(Molinaet al\.,[2010](https://arxiv.org/html/2607.22832#bib.bib74)\)\. Memory\-based metaheuristics such as tabu search use search history to restrict recently explored moves and bias local improvement toward unexplored neighborhoods\(Glover and Laguna,[1997](https://arxiv.org/html/2607.22832#bib.bib75)\)\.MEMENTOadapts this structure to model\-generated code\-as\-policy: a single accepted candidate is refined by a memory\-guided local\-search branch that records rejected proposals, along with macro\-mutation and crossover, all of which are selected by the evolved rollout evaluator\.

## 3Methodology

Sequential decision problems are modeled as finite\-horizon POMDPsℳPO=\(𝒮,𝒜,𝒪,T,Z,R,ρ0,L\)\\mathcal\{M\}\_\{\\mathrm\{PO\}\}=\(\\mathcal\{S\},\\mathcal\{A\},\\mathcal\{O\},T,Z,R,\\rho\_\{0\},L\), where𝒮\\mathcal\{S\}is the state space,𝒜\\mathcal\{A\}the action space,𝒪\\mathcal\{O\}the observation space,T:𝒮×𝒜→Δ​\(𝒮\)T:\\mathcal\{S\}\\times\\mathcal\{A\}\\to\\Delta\(\\mathcal\{S\}\)the transition function,Z:𝒮×𝒜→Δ​\(𝒪\)Z:\\mathcal\{S\}\\times\\mathcal\{A\}\\to\\Delta\(\\mathcal\{O\}\)the observation function,R:𝒮×𝒜→ℝR:\\mathcal\{S\}\\times\\mathcal\{A\}\\to\\mathbb\{R\}the environment reward,ρ0\\rho\_\{0\}the initial\-state distribution, andLLthe episode horizon\(Kaelblinget al\.,[1998](https://arxiv.org/html/2607.22832#bib.bib42); Puterman,[1994](https://arxiv.org/html/2607.22832#bib.bib43)\)\. At timet=0,…,L−1t=0,\\ldots,L\-1, the environment has statest∈𝒮s\_\{t\}\\in\\mathcal\{S\}, the agent receives observationot∈𝒪o\_\{t\}\\in\\mathcal\{O\}, selects actionat∈𝒜a\_\{t\}\\in\\mathcal\{A\}, and the environment transitions according toTT\. Task completion is represented by an episode\-level success indicatorσ​\(τ\)∈\{0,1\}\\sigma\(\\tau\)\\in\\\{0,1\\\}, defined on an executed trajectoryτ\\tau\.

The method does not directly optimize the environment rewardRR\. Instead, it searches over executable code\-as\-policy programs\. Each candidate programp∈𝒫p\\in\\mathcal\{P\}defines a deterministic stateful policy with internal program\-state space𝒵p\\mathcal\{Z\}\_\{p\}\. At the start of each episode, the program initializes its internal statez0∈𝒵pz\_\{0\}\\in\\mathcal\{Z\}\_\{p\}\. At timesteptt, the executable program maps the current observation and internal program state to an action and an updated program state,

\(at,zt\+1\)=p​\(ot,zt\),ot∈𝒪,at∈𝒜\.\(a\_\{t\},z\_\{t\+1\}\)=p\(o\_\{t\},z\_\{t\}\),\\qquad o\_\{t\}\\in\\mathcal\{O\},\\;a\_\{t\}\\in\\mathcal\{A\}\.The induced policy is denoted byπp\\pi\_\{p\}\. Executingπp\\pi\_\{p\}inℳPO\\mathcal\{M\}\_\{\\mathrm\{PO\}\}for episodeeeinduces a trajectoryτe​\(p\)\\tau\_\{e\}\(p\)\. For a rollout budget ofNrollN\_\{\\mathrm\{roll\}\}episodes, let

𝒯Nroll​\(p\)=\{τ1​\(p\),…,τNroll​\(p\)\}\\mathcal\{T\}\_\{N\_\{\\mathrm\{roll\}\}\}\(p\)=\\\{\\tau\_\{1\}\(p\),\\ldots,\\tau\_\{N\_\{\\mathrm\{roll\}\}\}\(p\)\\\}denote the resulting rollout set\. A fixed evaluatorEEmaps this rollout set to program\-level fitness and feedback metrics,

E​\(𝒯Nroll​\(p\)\)=\(F​\(p\),𝐌​\(p\)\)\.E\\\!\\left\(\\mathcal\{T\}\_\{N\_\{\\mathrm\{roll\}\}\}\(p\)\\right\)=\\bigl\(F\(p\),\\mathbf\{M\}\(p\)\\bigr\)\.HereF​\(p\)∈\[0,1\]F\(p\)\\in\[0,1\]is the aggregate evaluator fitness over theNrollN\_\{\\mathrm\{roll\}\}episodes, and𝐌​\(p\)\\mathbf\{M\}\(p\)is the collection of rollout\-level feedback metrics used to condition later code\-as\-policy revisions\. The empirical success rate is computed from the task success indicator,

S​\(p\)=1Nroll​∑e=1Nrollσ​\(τe​\(p\)\)\.S\(p\)=\\frac\{1\}\{N\_\{\\mathrm\{roll\}\}\}\\sum\_\{e=1\}^\{N\_\{\\mathrm\{roll\}\}\}\\sigma\\\!\\left\(\\tau\_\{e\}\(p\)\\right\)\.Evaluating a program defines the candidate state𝒞​\(p\)=\(p,F​\(p\),𝐌​\(p\),S​\(p\)\)\\mathcal\{C\}\(p\)=\(p,F\(p\),\\mathbf\{M\}\(p\),S\(p\)\), which stores the program, the evaluator outputs, and the task\-derived success rate\. For any candidate state𝒞\\mathcal\{C\},F​\(𝒞\)F\(\\mathcal\{C\}\)denotes the stored fitness value in𝒞\\mathcal\{C\}\.

### 3\.1Evaluator Evolution

Evaluator evolution searches for evaluator programs that map rollout data to scalar fitness and feedback metrics for code\-as\-policy search\. The initial policyp0p\_\{0\}is executed forNrollN\_\{\\mathrm\{roll\}\}episodes, yielding the fixed rollout set𝒯0=𝒯Nroll​\(p0\)\\mathcal\{T\}\_\{0\}=\\mathcal\{T\}\_\{N\_\{\\mathrm\{roll\}\}\}\(p\_\{0\}\)and empirical success rateS0=S​\(p0\)S\_\{0\}=S\(p\_\{0\}\)\. The same rollout set is used for all evaluator candidates during evaluator evolution\. At evaluator generationii, the search maintains a poolℰi=\{Ei\(1\),…,Ei\(n\)\}\\mathcal\{E\}\_\{i\}=\\\{E\_\{i\}^\{\(1\)\},\\ldots,E\_\{i\}^\{\(n\)\}\\\}ofnnevaluator programs\. The initial poolℰ0\\mathcal\{E\}\_\{0\}containsnnLLM\-generated evaluator programs\. Fori\>0i\>0,ℰi\\mathcal\{E\}\_\{i\}is obtained by applying the evaluator macro\-mutation operatorqevalmacroq\_\{\\mathrm\{eval\}\}^\{\\mathrm\{macro\}\}to the selected evaluator from the previous generation,Ei−1⋆E\_\{i\-1\}^\{\\star\}\.

Each evaluator candidate is applied to the shared rollout set and produces candidate\-specific program\-level outputs,

Ei\(j\)​\(𝒯0\)=\(Fi\(j\)​\(p0\),𝐌i\(j\)​\(p0\)\)\.E\_\{i\}^\{\(j\)\}\(\\mathcal\{T\}\_\{0\}\)=\\bigl\(F\_\{i\}^\{\(j\)\}\(p\_\{0\}\),\\mathbf\{M\}\_\{i\}^\{\(j\)\}\(p\_\{0\}\)\\bigr\)\.The empirical success rateS0S\_\{0\}is computed from the task success indicator and is shared across the evaluated pool\. Let

𝒟i=\{\(Ei\(j\),Fi\(j\)​\(p0\),𝐌i\(j\)​\(p0\),S0\)\}j=1n\\mathcal\{D\}\_\{i\}=\\left\\\{\\bigl\(E\_\{i\}^\{\(j\)\},F\_\{i\}^\{\(j\)\}\(p\_\{0\}\),\\mathbf\{M\}\_\{i\}^\{\(j\)\}\(p\_\{0\}\),S\_\{0\}\\bigr\)\\right\\\}\_\{j=1\}^\{n\}denote the scored evaluator pool\. The selected evaluator is chosen by majority voting overkkindependent LLM judgments of𝒟i\\mathcal\{D\}\_\{i\},

Ei⋆=𝒱k​\(𝒟i\)\.E\_\{i\}^\{\\star\}=\\mathcal\{V\}\_\{k\}\(\\mathcal\{D\}\_\{i\}\)\.The voting step uses LLM\-as\-judge evaluation to compare the scored evaluator pool, aggregating by majority overkkindependent judgments\(Zhenget al\.,[2023](https://arxiv.org/html/2607.22832#bib.bib40); Wanget al\.,[2023b](https://arxiv.org/html/2607.22832#bib.bib41)\)\. AfterTevalT\_\{\\mathrm\{eval\}\}evaluator generations,E⋆=ETeval−1⋆E^\{\\star\}=E\_\{T\_\{\\mathrm\{eval\}\}\-1\}^\{\\star\}is fixed and used for code\-as\-policy evolution\. The full evaluator evolution process is outlined in Algorithm[1](https://arxiv.org/html/2607.22832#alg1)\.

Algorithm 1Evaluator Evolution1:Require:initial policy

p0p\_\{0\}, pool size

nn, rollout budget

NrollN\_\{\\mathrm\{roll\}\}, voting budget

kk, evaluator generations

TevalT\_\{\\mathrm\{eval\}\}
2:Execute

p0p\_\{0\}for

NrollN\_\{\\mathrm\{roll\}\}episodes; store

𝒯0=𝒯Nroll​\(p0\)\\mathcal\{T\}\_\{0\}=\\mathcal\{T\}\_\{N\_\{\\mathrm\{roll\}\}\}\(p\_\{0\}\)and

S0=S​\(p0\)S\_\{0\}=S\(p\_\{0\}\)
3:Generate the initial evaluator pool

\{E0\(1\),…,E0\(n\)\}\\\{E\_\{0\}^\{\(1\)\},\\ldots,E\_\{0\}^\{\(n\)\}\\\}
4:for

i=0i=0to

Teval−1T\_\{\\mathrm\{eval\}\}\-1do

5:

𝒟i←∅\\mathcal\{D\}\_\{i\}\\leftarrow\\emptyset
6:for

j=1j=1to

nndo

7:

\(Fi\(j\),𝐌i\(j\)\)←Ei\(j\)​\(𝒯0\)\(F\_\{i\}^\{\(j\)\},\\mathbf\{M\}\_\{i\}^\{\(j\)\}\)\\leftarrow E\_\{i\}^\{\(j\)\}\(\\mathcal\{T\}\_\{0\}\)
8:Add

\(Ei\(j\),Fi\(j\),𝐌i\(j\),S0\)\(E\_\{i\}^\{\(j\)\},F\_\{i\}^\{\(j\)\},\\mathbf\{M\}\_\{i\}^\{\(j\)\},S\_\{0\}\)to

𝒟i\\mathcal\{D\}\_\{i\}
9:endfor

10:

Ei⋆←𝒱k​\(𝒟i\)E\_\{i\}^\{\\star\}\\leftarrow\\mathcal\{V\}\_\{k\}\(\\mathcal\{D\}\_\{i\}\)⊳\\trianglerightmajority vote overkkindependent LLM judgments

11:

\{Ei\+1\(j\)\}j=1n←\{qevalmacro​\(Ei⋆\)\}j=1n\\\{E\_\{i\+1\}^\{\(j\)\}\\\}\_\{j=1\}^\{n\}\\leftarrow\\\{q\_\{\\mathrm\{eval\}\}^\{\\mathrm\{macro\}\}\(E\_\{i\}^\{\\star\}\)\\\}\_\{j=1\}^\{n\}⊳\\trianglerightunused wheni=Teval−1i=T\_\{\\mathrm\{eval\}\}\-1

12:endfor

13:return

E⋆←ETeval−1⋆E^\{\\star\}\\leftarrow E\_\{T\_\{\\mathrm\{eval\}\}\-1\}^\{\\star\}

### 3\.2Policy Evolution Under the Fixed Evaluator

AfterE⋆E^\{\\star\}is fixed, policy search proceeds from a single elite candidate state𝒞g=𝒞​\(pg\)\\mathcal\{C\}\_\{g\}=\\mathcal\{C\}\(p\_\{g\}\)\. Each generation explores three branches: a hill\-climb branch of sequential micro\-mutations, a macro\-mutation branch, and a crossover branch \(cf\. Figure[1](https://arxiv.org/html/2607.22832#S1.F1)\)\. Letqhcq\_\{\\mathrm\{hc\}\},qmacroq\_\{\\mathrm\{macro\}\}, andqcrossq\_\{\\mathrm\{cross\}\}denote LLM\-conditioned proposal distributions that take evaluated candidate states as input and return new policy programs\. Proposals are conditioned on the program text and the evaluator outputsFFand𝐌\\mathbf\{M\}of the input candidate states; the success rateSSis recorded for reporting and is not used during search\. The complete policy evolution procedure is presented in Algorithm[2](https://arxiv.org/html/2607.22832#alg2)\.

Algorithm 2Single\-Elite Memetic Code\-as\-Policy Evolution1:Initial policy

p0p\_\{0\}; evaluator

E⋆E^\{\\star\}; generations

GG; budgets

Nroll,Khc,Kmacro,KcrossN\_\{\\mathrm\{roll\}\},K\_\{\\mathrm\{hc\}\},K\_\{\\mathrm\{macro\}\},K\_\{\\mathrm\{cross\}\}
2:

𝒞0←𝒞​\(p0\)\\mathcal\{C\}\_\{0\}\\leftarrow\\mathcal\{C\}\(p\_\{0\}\)⊳\\trianglerightevaluate initial policy underE⋆E^\{\\star\}

3:for

g=0g=0to

G−1G\-1do

4:

𝒞hc←𝒞g\\mathcal\{C\}^\{\\mathrm\{hc\}\}\\leftarrow\\mathcal\{C\}\_\{g\},

ℋ←∅\\mathcal\{H\}\\leftarrow\\emptyset⊳\\trianglerighthill\-climb branch

5:for

h=1h=1to

KhcK\_\{\\mathrm\{hc\}\}do

6:

p∼qhc​\(𝒞hc,ℋ\)p\\sim q\_\{\\mathrm\{hc\}\}\(\\mathcal\{C\}^\{\\mathrm\{hc\}\},\\mathcal\{H\}\)
7:

𝒞new←𝒞​\(p\)\\mathcal\{C\}^\{\\mathrm\{new\}\}\\leftarrow\\mathcal\{C\}\(p\)⊳\\trianglerightexecuteπp\\pi\_\{p\}forNrollN\_\{\\mathrm\{roll\}\}episodes, evaluate underE⋆E^\{\\star\}

8:if

F​\(𝒞new\)≥F​\(𝒞hc\)F\(\\mathcal\{C\}^\{\\mathrm\{new\}\}\)\\geq F\(\\mathcal\{C\}^\{\\mathrm\{hc\}\}\)then

9:

𝒞hc←𝒞new\\mathcal\{C\}^\{\\mathrm\{hc\}\}\\leftarrow\\mathcal\{C\}^\{\\mathrm\{new\}\}
10:else

11:

ℋ←append⁡\(ℋ,history​\(p\)\)\\mathcal\{H\}\\leftarrow\\operatorname\{append\}\(\\mathcal\{H\},\\mathrm\{history\}\(p\)\)
12:endif

13:endfor

14:for

u=1u=1to

KmacroK\_\{\\mathrm\{macro\}\}do⊳\\trianglerightmacro\-mutation branch

15:

pu∼qmacro​\(𝒞g\)p\_\{u\}\\sim q\_\{\\mathrm\{macro\}\}\(\\mathcal\{C\}\_\{g\}\);

𝒞umacro←𝒞​\(pu\)\\mathcal\{C\}\_\{u\}^\{\\mathrm\{macro\}\}\\leftarrow\\mathcal\{C\}\(p\_\{u\}\)
16:endfor

17:

𝒞macro←arg​max𝒞∈\{𝒞g,𝒞1macro,…,𝒞Kmacromacro\}⁡F​\(𝒞\)\\mathcal\{C\}^\{\\mathrm\{macro\}\}\\leftarrow\\operatorname\*\{arg\\,max\}\_\{\\mathcal\{C\}\\in\\\{\\mathcal\{C\}\_\{g\},\\mathcal\{C\}\_\{1\}^\{\\mathrm\{macro\}\},\\ldots,\\mathcal\{C\}\_\{K\_\{\\mathrm\{macro\}\}\}^\{\\mathrm\{macro\}\}\\\}\}F\(\\mathcal\{C\}\)
18:for

r=1r=1to

KcrossK\_\{\\mathrm\{cross\}\}do⊳\\trianglerightcrossover branch

19:

pr∼qcross​\(𝒞hc,𝒞macro\)p\_\{r\}\\sim q\_\{\\mathrm\{cross\}\}\(\\mathcal\{C\}^\{\\mathrm\{hc\}\},\\mathcal\{C\}^\{\\mathrm\{macro\}\}\);

𝒞rcross←𝒞​\(pr\)\\mathcal\{C\}\_\{r\}^\{\\mathrm\{cross\}\}\\leftarrow\\mathcal\{C\}\(p\_\{r\}\)
20:endfor

21:

𝒞cross←arg​max𝒞∈\{𝒞1cross,…,𝒞Kcrosscross\}⁡F​\(𝒞\)\\mathcal\{C\}^\{\\mathrm\{cross\}\}\\leftarrow\\operatorname\*\{arg\\,max\}\_\{\\mathcal\{C\}\\in\\\{\\mathcal\{C\}\_\{1\}^\{\\mathrm\{cross\}\},\\ldots,\\mathcal\{C\}\_\{K\_\{\\mathrm\{cross\}\}\}^\{\\mathrm\{cross\}\}\\\}\}F\(\\mathcal\{C\}\)
22:

𝒞g\+1←arg​max𝒞∈\{𝒞hc,𝒞macro,𝒞cross\}⁡F​\(𝒞\)\\mathcal\{C\}\_\{g\+1\}\\leftarrow\\operatorname\*\{arg\\,max\}\_\{\\mathcal\{C\}\\in\\\{\\mathcal\{C\}^\{\\mathrm\{hc\}\},\\mathcal\{C\}^\{\\mathrm\{macro\}\},\\mathcal\{C\}^\{\\mathrm\{cross\}\}\\\}\}F\(\\mathcal\{C\}\)⊳\\trianglerightelitist selection

23:endfor

24:return

𝒞G\\mathcal\{C\}\_\{G\}

##### Hill\-climb branch\.

For any proposed policy programpp, lethistory​\(p\)\\mathrm\{history\}\(p\)denote the edit summary embedded in the program text, describing the change made relative to its parent\. The hill\-climb branch maintains a current accepted candidate state𝒞g,hhc=𝒞​\(pg,hhc\)\\mathcal\{C\}\_\{g,h\}^\{\\mathrm\{hc\}\}=\\mathcal\{C\}\(p\_\{g,h\}^\{\\mathrm\{hc\}\}\)and a rejected\-proposal memoryℋg,h\\mathcal\{H\}\_\{g,h\}, whereℋg,h\\mathcal\{H\}\_\{g,h\}is an ordered list of edit summaries from rejected hill\-climb proposals in generationgg\. Edit summaries of accepted proposals remain embedded in the accepted program text and are therefore visible to subsequent proposals through𝒞g,hhc\\mathcal\{C\}\_\{g,h\}^\{\\mathrm\{hc\}\};ℋg,h\\mathcal\{H\}\_\{g,h\}records only rejected edits\. The branch is initialized from the generation elite, with𝒞g,0hc=𝒞g\\mathcal\{C\}\_\{g,0\}^\{\\mathrm\{hc\}\}=\\mathcal\{C\}\_\{g\}, and starts with an empty rejected\-proposal memory,ℋg,0=∅\\mathcal\{H\}\_\{g,0\}=\\emptyset\. At steph=1,…,Khch=1,\\ldots,K\_\{\\mathrm\{hc\}\}, a new policy program is sampled from the current accepted candidate state and the rejected\-proposal memory:

pg,hnew∼qhc​\(𝒞g,h−1hc,ℋg,h−1\)\.p\_\{g,h\}^\{\\mathrm\{new\}\}\\sim q\_\{\\mathrm\{hc\}\}\\\!\\left\(\\mathcal\{C\}\_\{g,h\-1\}^\{\\mathrm\{hc\}\},\\mathcal\{H\}\_\{g,h\-1\}\\right\)\.The proposed program is executed and evaluated underE⋆E^\{\\star\}, producing

𝒞g,hnew=𝒞​\(pg,hnew\)\.\\mathcal\{C\}\_\{g,h\}^\{\\mathrm\{new\}\}=\\mathcal\{C\}\(p\_\{g,h\}^\{\\mathrm\{new\}\}\)\.The proposal is accepted if its evaluated fitness is at least that of the current accepted program:

𝒞g,hhc=\{𝒞g,hnew,if​F​\(𝒞g,hnew\)≥F​\(𝒞g,h−1hc\),𝒞g,h−1hc,otherwise\.\\mathcal\{C\}\_\{g,h\}^\{\\mathrm\{hc\}\}=\\begin\{cases\}\\mathcal\{C\}\_\{g,h\}^\{\\mathrm\{new\}\},&\\text\{if \}F\(\\mathcal\{C\}\_\{g,h\}^\{\\mathrm\{new\}\}\)\\geq F\(\\mathcal\{C\}\_\{g,h\-1\}^\{\\mathrm\{hc\}\}\),\\\\\[2\.0pt\] \\mathcal\{C\}\_\{g,h\-1\}^\{\\mathrm\{hc\}\},&\\text\{otherwise\.\}\\end\{cases\}\(1\)If the proposal is accepted, the rejected\-proposal memory is unchanged:ℋg,h=ℋg,h−1\\mathcal\{H\}\_\{g,h\}=\\mathcal\{H\}\_\{g,h\-1\}\. If the proposal is rejected, its edit summary is appended to the memory:ℋg,h=append⁡\(ℋg,h−1,history​\(pg,hnew\)\)\\mathcal\{H\}\_\{g,h\}=\\operatorname\{append\}\(\\mathcal\{H\}\_\{g,h\-1\},\\mathrm\{history\}\(p\_\{g,h\}^\{\\mathrm\{new\}\}\)\)\. The branch output is the accepted candidate state after the final hill\-climb step,

𝒞ghc=𝒞g,Khchc\.\\mathcal\{C\}\_\{g\}^\{\\mathrm\{hc\}\}=\\mathcal\{C\}\_\{g,K\_\{\\mathrm\{hc\}\}\}^\{\\mathrm\{hc\}\}\.

##### Macro\-mutation branch\.

Foru=1,…,Kmacrou=1,\\ldots,K\_\{\\mathrm\{macro\}\}, an independent mutation of the generation elite is sampled and evaluated,

pg,umacro∼qmacro​\(𝒞g\),𝒞g,umacro=𝒞​\(pg,umacro\)\.p\_\{g,u\}^\{\\mathrm\{macro\}\}\\sim q\_\{\\mathrm\{macro\}\}\(\\mathcal\{C\}\_\{g\}\),\\qquad\\mathcal\{C\}\_\{g,u\}^\{\\mathrm\{macro\}\}=\\mathcal\{C\}\(p\_\{g,u\}^\{\\mathrm\{macro\}\}\)\.The branch output is the candidate state with the highest fitness among the generation elite and the macro\-mutation candidates:

𝒞gmacro=arg​max𝒞∈\{𝒞g,𝒞g,1macro,…,𝒞g,Kmacromacro\}⁡F​\(𝒞\)\.\\mathcal\{C\}\_\{g\}^\{\\mathrm\{macro\}\}=\\operatorname\*\{arg\\,max\}\_\{\\mathcal\{C\}\\in\\\{\\mathcal\{C\}\_\{g\},\\mathcal\{C\}\_\{g,1\}^\{\\mathrm\{macro\}\},\\ldots,\\mathcal\{C\}\_\{g,K\_\{\\mathrm\{macro\}\}\}^\{\\mathrm\{macro\}\}\\\}\}F\(\\mathcal\{C\}\)\.\(2\)

##### Crossover branch\.

Forr=1,…,Kcrossr=1,\\ldots,K\_\{\\mathrm\{cross\}\}, a recombination of the two branch outputs is sampled and evaluated,

pg,rcross∼qcross​\(𝒞ghc,𝒞gmacro\),𝒞g,rcross=𝒞​\(pg,rcross\)\.p\_\{g,r\}^\{\\mathrm\{cross\}\}\\sim q\_\{\\mathrm\{cross\}\}\\\!\\left\(\\mathcal\{C\}\_\{g\}^\{\\mathrm\{hc\}\},\\mathcal\{C\}\_\{g\}^\{\\mathrm\{macro\}\}\\right\),\\qquad\\mathcal\{C\}\_\{g,r\}^\{\\mathrm\{cross\}\}=\\mathcal\{C\}\(p\_\{g,r\}^\{\\mathrm\{cross\}\}\)\.The crossover proposal is conditioned on both selected candidate states, using their policy programs and evaluator outputs\. The branch output is the evaluated crossover candidate with the highest program fitness:

𝒞gcross=arg​max𝒞∈\{𝒞g,1cross,…,𝒞g,Kcrosscross\}⁡F​\(𝒞\)\.\\mathcal\{C\}\_\{g\}^\{\\mathrm\{cross\}\}=\\operatorname\*\{arg\\,max\}\_\{\\mathcal\{C\}\\in\\\{\\mathcal\{C\}\_\{g,1\}^\{\\mathrm\{cross\}\},\\ldots,\\mathcal\{C\}\_\{g,K\_\{\\mathrm\{cross\}\}\}^\{\\mathrm\{cross\}\}\\\}\}F\(\\mathcal\{C\}\)\.\(3\)

##### Generation elite selection\.

The branch output with the highest program fitness is selected as the next\-generation elite and used as the parent for generationg\+1g\+1:

𝒞g\+1=arg​max𝒞∈\{𝒞ghc,𝒞gmacro,𝒞gcross\}⁡F​\(𝒞\)\.\\mathcal\{C\}\_\{g\+1\}=\\operatorname\*\{arg\\,max\}\_\{\\mathcal\{C\}\\in\\\{\\mathcal\{C\}\_\{g\}^\{\\mathrm\{hc\}\},\\mathcal\{C\}\_\{g\}^\{\\mathrm\{macro\}\},\\mathcal\{C\}\_\{g\}^\{\\mathrm\{cross\}\}\\\}\}F\(\\mathcal\{C\}\)\.\(4\)

## 4Experiments

We evaluateMEMENTOon two long\-horizon embodied domains with different sources of execution dependence\. In Robosuite Franka Tower\-of\-Hanoi\(Zhuet al\.,[2020](https://arxiv.org/html/2607.22832#bib.bib50); Havuret al\.,[2013](https://arxiv.org/html/2607.22832#bib.bib48); Dugganet al\.,[2026](https://arxiv.org/html/2607.22832#bib.bib35)\), each pick\-and\-place operation changes the object configuration that determines the remaining legal moves\. In AI2\-THOR household interaction\(Kolveet al\.,[2017](https://arxiv.org/html/2607.22832#bib.bib30)\), valid actions are constrained by egocentric visibility, scene\-specific object identifiers, and object\-interaction preconditions\. The two domains, therefore, test code\-as\-policy search under structured manipulation state and partially observed household interaction\. Figure[2](https://arxiv.org/html/2607.22832#S4.F2)shows the training task settings used in the experiments\.

![Refer to caption](https://arxiv.org/html/2607.22832v1/x2.png)

\(a\) Robosuite Franka Tower\-of\-Hanoi

![Refer to caption](https://arxiv.org/html/2607.22832v1/x3.png)FloorPlan1![Refer to caption](https://arxiv.org/html/2607.22832v1/x4.png)FloorPlan2![Refer to caption](https://arxiv.org/html/2607.22832v1/x5.png)FloorPlan3
\(b\) AI2\-THOR training kitchens

Figure 2:Training task environments\. The Robosuite task is a four\-object Tower\-of\-Hanoi manipulation problem in which the policy must move the full stack from the source container to the target container while preserving legal size order\. The AI2\-THOR task uses three training kitchens,FloorPlan1–FloorPlan3, which differ in layout, object placement, visibility, and receptacle configuration\.##### Robosuite Franka Tower\-of\-Hanoi\.

The Robosuite domain is a four\-cube Tower\-of\-Hanoi task with a Franka Panda arm and three containers\. At reset, all cubes are stacked on container A in legal size order\. Success requires moving the complete tower to container C while leaving containers A and B empty\. An episode terminates on success, timeout, or an invalid Hanoi placement in which a larger cube is placed on a smaller cube\. Fornncubes, the shortest legal solution requires2n−12^\{n\}\-1transfers; forn=4n=4, this gives1515ordered pick\-and\-place operations\.

##### AI2\-THOR household interaction\.

The AI2\-THOR domain uses the taskapple\_in\_microwave\_on\_bread\_in\_fridge\. Success requires placing the Apple in the Microwave, toggling the Microwave on, and placing the Bread in the Fridge\. The agent receives egocentric observations, and object actions are valid only when the corresponding scene\-specific object identifier appears in the current action space\. An episode terminates on success, policy\-issueddone, or the step limit\. Policy evolution usesFloorPlan1–FloorPlan3; cross\-scene transfer is evaluated on held\-out kitchensFloorPlan4–FloorPlan29\.

The full observation and action interfaces are reported in Appendix[A\.2](https://arxiv.org/html/2607.22832#A1.SS2)and Appendix[B](https://arxiv.org/html/2607.22832#A2); Robosuite domain\-randomization details are reported in Appendix[A\.3](https://arxiv.org/html/2607.22832#A1.SS3)\. Held\-out Robosuite object variants are shown in Appendix[A\.5](https://arxiv.org/html/2607.22832#A1.SS5), and additional AI2\-THOR scene examples are shown in Appendix[B](https://arxiv.org/html/2607.22832#A2)\.

##### Training setup\.

MEMENTOis run forG=5G=5generations\. Generation 0 denotes the evaluated initial policyp0p\_\{0\}and involves no policy search; each subsequent generation applies the three proposal branches\. Each generation usesKhc=10K\_\{\\mathrm\{hc\}\}=10hill\-climb proposals,Kmacro=10K\_\{\\mathrm\{macro\}\}=10macro\-mutation proposals, andKcross=4K\_\{\\mathrm\{cross\}\}=4crossover proposals, giving a policy\-candidate budget ofB=Khc\+Kmacro\+Kcross=24B=K\_\{\\mathrm\{hc\}\}\+K\_\{\\mathrm\{macro\}\}\+K\_\{\\mathrm\{cross\}\}=24new candidates per generation\. Each candidate policy is evaluated forNroll=10N\_\{\\mathrm\{roll\}\}=10rollout episodes under the fixed evolved evaluatorE⋆E^\{\\star\}\. Evaluator evolution uses a pool ofn=8n=8evaluator candidates per generation forTeval=5T\_\{\\mathrm\{eval\}\}=5evaluator generations, with selection by majority vote overk=10k=10independent LLM judgments\. Results are reported as mean±\\pmstandard deviation across three random seeds\. Curves report per\-generation best\-so\-far fitness and empirical success rate, and tables report final\-generation performance\. Evaluator and code\-as\-policy proposals are generated with GPT\-5\.2\(OpenAI,[2025](https://arxiv.org/html/2607.22832#bib.bib53)\)\.

##### Baselines\.

Eureka\(Maet al\.,[2024](https://arxiv.org/html/2607.22832#bib.bib8)\)and REvolve\(Hazraet al\.,[2025](https://arxiv.org/html/2607.22832#bib.bib21)\)are adapted from reward\-code evolution to code\-as\-policy evolution: the search object is the executable policy program, and candidate fitness is computed by the same fixed evolved evaluatorE⋆E^\{\\star\}used by MEMENTO\. All methods start from the same seed\-specific initial policy, use the same language model and domain prompt templates, and receive the same budget ofB=24B=24evaluated candidates per generation forG=5G=5generations\. The adapted Eureka samples independent macro\-mutations of the current best program in each generation and selects the highest\-fitness candidate\. The adapted REvolve maintains a population of policy programs with fitness\-based selection and LLM\-generated crossover and macro\-mutation, withE⋆E^\{\\star\}fitness replacing its human\-feedback fitness\.

## 5Results

### 5\.1Main Evolution Results

We first evaluate whether the proposed search structure improves code\-as\-policy programs more effectively than LLM\-guided evolutionary baselines under the same rollout budget\. Figure[3](https://arxiv.org/html/2607.22832#S5.F3)reports best\-so\-far evaluator fitness and empirical success rate across generations forMEMENTO, REvolve, and Eureka\. Results are reported as mean±\\pmstandard deviation across three random seeds, with all methods initialized from the same seed\-specific policy at generation0\.

![Refer to caption](https://arxiv.org/html/2607.22832v1/x6.png)

\(a\) Robosuite Tower\-of\-Hanoi

![Refer to caption](https://arxiv.org/html/2607.22832v1/x7.png)

\(b\) AI2\-THOR household interaction

Figure 3:Main evolution results on Robosuite Tower\-of\-Hanoi and AI2\-THOR household interaction\. Subfigure \(a\) reports Robosuite and Subfigure \(b\) reports AI2\-THOR\. Within each subfigure, the left plot shows the best\-so\-far evaluator fitness, and the right plot shows the empirical success rate over generations\. Lines denote the mean across three random seeds, and shaded regions denote one standard deviation forMEMENTO, REvolve, and Eureka\.By generation 5,MEMENTOachieves the strongest final performance in both domains\. On Robosuite Tower\-of\-Hanoi,MEMENTOreaches0\.99±0\.02\\mathbf\{0\.99\\pm 0\.02\}fitness and0\.97±0\.06\\mathbf\{0\.97\\pm 0\.06\}success, compared with0\.72±0\.080\.72\\pm 0\.08fitness and0\.53±0\.120\.53\\pm 0\.12success for REvolve\. Eureka reaches0\.45±0\.040\.45\\pm 0\.04fitness and produces no successful rollouts\. On AI2\-THOR,MEMENTOreaches0\.98±0\.00\\mathbf\{0\.98\\pm 0\.00\}fitness and1\.00±0\.00\\mathbf\{1\.00\\pm 0\.00\}success, compared with0\.67±0\.050\.67\\pm 0\.05fitness and0\.40±0\.100\.40\\pm 0\.10success for REvolve; Eureka again produces no successful rollouts\. These results show that, under the same rollout evaluator and candidate budget,MEMENTO’s memory\-guided local refinement, macro\-mutation, and crossover produce task\-completing code\-as\-policy programs more reliably than the mutation\-only and population\-style evolutionary baselines\.

Because candidate selection is performed using the fixed evaluator fitnessFFreturned byE⋆E^\{\\star\}, while empirical successSSis computed independently from the task\-success predicate, we examined the correspondence between these two quantities\. For each domain, the per\-generation selected candidates, i\.e\., the elite𝒞g\\mathcal\{C\}\_\{g\}forMEMENTOand the reported best candidate for each baseline, provide5454paired values\(F,S\)\(F,S\), obtained from three methods, three seeds, and six generations\. Fitness and success are positively correlated in both domains: Robosuite Tower\-of\-Hanoi gives Pearsonr=0\.922r=0\.922and Spearmanρ=0\.755\\rho=0\.755, while AI2\-THOR gives Pearsonr=0\.871r=0\.871and Spearmanρ=0\.801\\rho=0\.801\. These correlations indicate that the fixed evaluator provides a selection signal aligned with task success: candidates with higher evaluator fitness generally also achieve higher empirical success\.

### 5\.2Final Performance and Generalization

Table[1](https://arxiv.org/html/2607.22832#S5.T1)reports the success rate of the final selected policy after search on the original evaluation domains and on held\-out generalization domains\. For Robosuite Franka Tower\-of\-Hanoi, Cube denotes the cube\-based environment used during policy evolution, while Asym\. and Cyl\. replace the manipulated cubes with held\-out object geometries\. These variants preserve the Hanoi objective, object identifiers, container layout, and structured observation interface, but change the grasp envelope and contact geometry of the manipulated objects; details and examples are given in Appendix[A\.5](https://arxiv.org/html/2607.22832#A1.SS5)\. For AI2\-THOR, Search scenes denote the kitchens used during policy evolution, while Held\-out scenes denote unseen kitchens with different layouts, object instances, visibility patterns, receptacle placements, and scene\-specific object identifiers; environment details and scene examples are given in Appendix[B](https://arxiv.org/html/2607.22832#A2)\. SAC\+HER is included as a Robosuite\-only interaction\-learning baseline on the original cube task\. Unlike the LLM\-guided policy\-code methods, SAC\+HER is trained with goal\-conditioned HER rewards rather than evaluator\-conditioned code revision; its observation interface, training configuration, and sparse/dense reward definitions are reported in Appendices[A\.6](https://arxiv.org/html/2607.22832#A1.SS6)–[A\.9](https://arxiv.org/html/2607.22832#A1.SS9)\.

Table 1:Final success rate on original and held\-out generalization domains\. Cube is the Robosuite Tower\-of\-Hanoi environment used during policy evolution\. Asym\. and Cyl\. are held\-out Robosuite object\-geometry variants that preserve the Hanoi task while changing the shape of the manipulated object\. Search scenes are the AI2\-THOR kitchens used during policy evolution, and held\-out scenes are unseen kitchensFloorPlan4\-\-29\. SAC\+HER is evaluated only on the original Robosuite cube task\. Entries are mean±\\pmstandard deviation across three seeds when available; “–” denotes not evaluated\.Table[1](https://arxiv.org/html/2607.22832#S5.T1)shows thatMEMENTOachieves the highest final success in both training domains and maintains substantial success under domain\-specific generalization tests\. In Robosuite, the final policy transfers from cubic training objects to asymmetric stackable objects and cylindrical disks, indicating that the evolved program relies on Hanoi state structure, legal stack ordering, and closed\-loop placement geometry rather than memorizing cube\-specific contacts\. In AI2\-THOR, the final policy transfers from the three search kitchens to unseen kitchens with new layouts, object placements, visibility constraints, and object identifiers\. REvolve partially solves the original domains, but loses most performance on the held\-out Robosuite geometries and AI2\-THOR kitchens, while Eureka produces no successful rollouts in either domain\. These results indicate that the evolved programs rely on object\-centric state assignments, legal subgoal ordering, and observation\-conditioned action targets, with limited dependence on cube\-specific contacts or memorized kitchen layouts\.

### 5\.3Ablations

The ablations use the same three\-seed evaluation protocol as the main comparison\. All evolutionary code\-as\-policy search ablations use the same total candidate budget as the main experiments,B=24B=24evaluated policy candidates per generation\. The zero\-shot no\-evolution baseline uses the same one\-generation candidate budget of2424policy candidates, without evolutionary updates\. We isolate four design choices: whether evolutionary code\-as\-policy search is needed at all, whether an evolved evaluator is needed to guide code\-as\-policy search, whether the code\-as\-policy search branches must be used jointly, and whether restricting AI2\-THOR search to a single training scene reduces cross\-scene generalization\.

##### Zero\-shot policy generation\.

We first test whether the LLM can solve the tasks without evolutionary policy search\. In this setting, we sample2424zero\-shot policy candidates and evaluate the best candidate, but without evolutionary generations\. This baseline tests whether the domains can be solved by single\-round code generation alone\.

On Robosuite Tower\-of\-Hanoi, zero\-shot policy generation reaches0\.37±0\.120\.37\\pm 0\.12fitness and0\.00±0\.000\.00\\pm 0\.00success\. On AI2\-THOR, it reaches0\.22±0\.100\.22\\pm 0\.10fitness and0\.00±0\.000\.00\\pm 0\.00success\. Thus, zero\-shot policies achieve partial dense progress but never satisfy the full task\-success predicate in either domain\. By contrast, fullMEMENTOreaches0\.99±0\.02\\mathbf\{0\.99\\pm 0\.02\}fitness and0\.97±0\.06\\mathbf\{0\.97\\pm 0\.06\}success on Robosuite, and0\.98±0\.00\\mathbf\{0\.98\\pm 0\.00\}fitness and1\.00±0\.00\\mathbf\{1\.00\\pm 0\.00\}success on AI2\-THOR\. Figure[4](https://arxiv.org/html/2607.22832#S5.F4)shows this comparison\. The zero\-shot baseline is shown as a horizontal reference because it is not an evolutionary trajectory\.

![Refer to caption](https://arxiv.org/html/2607.22832v1/x8.png)Figure 4:Zero\-shot policy\-generation baseline compared with fullMEMENTO\. Left: Robosuite Tower\-of\-Hanoi\. Right: AI2\-THOR household interaction\. The dashed horizontal curve shows the no\-evolution zero\-shot fitness baseline from2424policy candidates, and diamond markers show its empirical success\. The solid curve showsMEMENTObest\-so\-far fitness, and square markers showMEMENTOempirical success\. Shaded regions and error bars denote one standard deviation across three seeds\.
##### Evaluator evolution\.

We next test whether evaluator evolution is needed once policy search is run\. We replace the evolved evaluator from Section[3\.1](https://arxiv.org/html/2607.22832#S3.SS1)with one zero\-shot LLM\-generated evaluator per seed and run the same policy\-search procedure\. The evaluator still returns scalar fitness and feedback metrics, but it is not mutated or selected through evaluator evolution\. Figure[5](https://arxiv.org/html/2607.22832#S5.F5)compares this no\-evaluator\-evolution variant with fullMEMENTO\.

At generation 5, the no\-evaluator\-evolution variant reaches0\.42±0\.050\.42\\pm 0\.05fitness and0\.00±0\.000\.00\\pm 0\.00success on Robosuite Tower\-of\-Hanoi, and0\.33±0\.050\.33\\pm 0\.05fitness and0\.00±0\.000\.00\\pm 0\.00success on AI2\-THOR\. FullMEMENTOreaches0\.99±0\.02\\mathbf\{0\.99\\pm 0\.02\}fitness and0\.97±0\.06\\mathbf\{0\.97\\pm 0\.06\}success on Robosuite, and0\.98±0\.00\\mathbf\{0\.98\\pm 0\.00\}fitness and1\.00±0\.00\\mathbf\{1\.00\\pm 0\.00\}success on AI2\-THOR\. The zero\-shot evaluators support limited fitness improvement but do not guide policy search to task completion\. This indicates that evaluator evolution is needed to align rollout feedback with the long\-horizon success conditions\.

![Refer to caption](https://arxiv.org/html/2607.22832v1/x9.png)Figure 5:Evaluator\-evolution ablation on Robosuite Tower\-of\-Hanoi and AI2\-THOR household interaction\. Each subfigure compares fullMEMENTOwith a variant that uses one zero\-shot LLM\-generated evaluator per seed and does not perform evaluator evolution\. Solid curves show best\-so\-far evaluator fitness, and square markers show empirical success across generations\. Lines and markers denote means across three seeds, and shaded regions denote one standard deviation\.
##### Operator branches\.

We then evaluate whether the code\-as\-policy search branches must be used jointly\. The evolved evaluator is kept fixed, and fullMEMENTOis compared with two ablated variants: Hill\-only and Without\-Crossover\. Hill\-only reallocates the full candidate budget to sequential hill\-climbing, usingKhc=24K\_\{\\mathrm\{hc\}\}=24,Kmacro=0K\_\{\\mathrm\{macro\}\}=0, andKcross=0K\_\{\\mathrm\{cross\}\}=0\. Without\-Crossover removes recombination and reallocates the crossover budget to macro\-mutation, usingKhc=12K\_\{\\mathrm\{hc\}\}=12,Kmacro=12K\_\{\\mathrm\{macro\}\}=12, andKcross=0K\_\{\\mathrm\{cross\}\}=0\. A macro\-mutation\-only variant is not included because it matches the mutation\-only structure of the adapted Eureka baseline\. Figure[6](https://arxiv.org/html/2607.22832#S5.F6)reports the resulting fitness and success curves\.

At generation 5, Hill\-only reaches0\.50±0\.040\.50\\pm 0\.04fitness on Robosuite Tower\-of\-Hanoi and0\.46±0\.040\.46\\pm 0\.04on AI2\-THOR, while Without\-Crossover reaches0\.55±0\.020\.55\\pm 0\.02and0\.51±0\.040\.51\\pm 0\.04, respectively\. Neither ablated variant produces successful rollouts in either domain\. In contrast, fullMEMENTOreaches0\.99±0\.02\\mathbf\{0\.99\\pm 0\.02\}fitness and0\.97±0\.06\\mathbf\{0\.97\\pm 0\.06\}success on Robosuite, and0\.98±0\.00\\mathbf\{0\.98\\pm 0\.00\}fitness and1\.00±0\.00\\mathbf\{1\.00\\pm 0\.00\}success on AI2\-THOR\. These results indicate that, under the tested candidate budget, local refinement and macro\-mutation alone are insufficient: the crossover branch is needed forMEMENTOto combine partial executable strategies into task\-completing policies\.

![Refer to caption](https://arxiv.org/html/2607.22832v1/x10.png)Figure 6:Operator\-branch ablations on Robosuite Tower\-of\-Hanoi and AI2\-THOR household interaction\. Each subfigure compares Hill\-only, Without\-Crossover, and fullMEMENTO\. Solid curves show best\-so\-far evaluator fitness, and hollow square markers show empirical success across generations\. Hill\-only retains sequential accepted\-candidate refinement with rejected\-proposal memory but removes macro\-mutation and crossover\. Without\-Crossover retains hill\-climbing and macro\-mutation but removes crossover between its best candidates\.
##### Search\-scene diversity\.

Finally, we test whether AI2\-THOR policy evolution requires multiple training kitchens\. In this ablation,MEMENTOis restricted to a single training kitchen during policy evolution and is then evaluated acrossFloorPlan1–FloorPlan29\. The resulting policies solve the search scene but generalize poorly across kitchens, reaching0\.09±0\.020\.09\\pm 0\.02success across seeds\(2/29,3/29,3/29\)\(2/29,3/29,3/29\)\. This suggests that single\-scene policy evolution overfits to scene\-specific object placements, navigation routes, and interaction viewpoints\. In contrast, multi\-scene policy evolution exposes the policy search to scene\-dependent object identifiers, visibility constraints, receptacle locations, and valid\-action preconditions\. Together with the held\-out scene results in Table[1](https://arxiv.org/html/2607.22832#S5.T1), this shows that search\-scene diversity is necessary for AI2\-THOR cross\-scene generalization\.

### 5\.4Sim\-to\-Real Transfer

The physical deployment tests whether the evolved program logic completes the task when its object\-centric inputs are estimated from real RGB\-D perception, and its selected moves are executed through physical contact\. The best\-performing Robosuite Tower\-of\-Hanoi code\-as\-policy across the threeMEMENTOseeds, denotedpbestp\_\{\\mathrm\{best\}\}, was deployed on a physical Franka robot\. The evolved task\-level code is kept fixed\. During the simulation search, the selected policy acquired a closed\-loop execution structure that reduces dependence on exact simulator dynamics: object poses are filtered before target computation, pick\-and\-place targets are re\-derived from the current cube and container estimates, and contact\-sensitive execution is guarded by alignment checks, placement centering, grasp verification, and post\-placement retry\. The symbolic Hanoi ordering remains fixed, while the execution of each scheduled move is adapted to the currently estimated configuration of objects\. Additional details of the evolved Hanoi policy are given in Appendix[C\.2\.1](https://arxiv.org/html/2607.22832#A3.SS2.SSS1)\.

Simulator object states, and end\-effector control increments are replaced by RGB\-D object\-pose estimates and Franka motion commands\. During the simulation search, observation noise and calibration bias were sampled from the randomization model in Appendix[A\.3](https://arxiv.org/html/2607.22832#A1.SS3)\. In the physical trials, analogous errors arise from segmentation, depth measurement, camera calibration, and contact execution\. In simulation, Robosuite provides object poses and robot state, and actions are applied through the Robosuite end\-effector controller\. In the physical setup, object and container poses are estimated from RGB\-D observations using SAM3 segmentation\(Carionet al\.,[2026](https://arxiv.org/html/2607.22832#bib.bib54)\), and policy\-selected pick\-and\-place targets are executed by the robot motion planner and gripper controller\.

Before the final robot evaluation, a small set of execution constants was recalibrated for the physical cell overNcal=3N\_\{\\mathrm\{cal\}\}=3LLM\-assisted calibration steps\. The calibration variables, real\-robot metrics, and calibration procedure are detailed in Appendix[A\.4](https://arxiv.org/html/2607.22832#A1.SS4)\. The real\-robot metrics correspond to a subset of the evaluator’s feedback categories and are computed by a reduced evaluatorERE\_\{R\}\. They guide calibration only; no fitness is computed, and no policy search is performed during deployment\.

The calibrated code\-as\-policy achieved a success rate of0\.900\.90overNreal=10N\_\{\\mathrm\{real\}\}=10physical trials\. In all1010trials, the task\-level code followed the fixed Hanoi ordering and rederived execution targets from the estimated object configuration; the single unsuccessful trial was due to a perception error\.

## 6Limitations and Future Work

The experiments use two task formulations: four\-cube Tower\-of\-Hanoi manipulation in Robosuite and a long\-horizon household\-interaction task in AI2\-THOR\. Since each evolved program is conditioned on a task\-specific prompt, evaluator, and success predicate, broader task coverage would require running the full evaluator\-evolution and code\-as\-policy search procedure on additional task definitions\. Model sensitivity is not isolated, as evaluator and policy proposals are generated using a single language model\. The evaluatorE⋆E^\{\\star\}is evolved from the initial\-policy rollout set𝒯0\\mathcal\{T\}\_\{0\}and then fixed during code\-as\-policy search, so the experiments do not test whether evaluator evolution should be reintroduced when later policy elites expose failures absent in the initial rollouts\. The hill\-climb branch is sequential within each generation because each proposal depends on the currently accepted candidate and the rejection memory, thereby limiting within\-branch parallelism relative to population\-based variation\.

Future work should therefore extend the current single\-elite formulation to a population\-based code\-as\-policy search\. Multiple islands could maintain separate policy elites, run branch\-level search independently, and periodically exchange high\-fitness candidates through migration\(Cantú\-Paz and others,[1998](https://arxiv.org/html/2607.22832#bib.bib33)\)\. This would add population diversity and reduce the sequential bottleneck of a single hill\-climb chain\. In addition, future work should also study adaptive evaluator–policy co\-evolution, in whichE⋆E^\{\\star\}is periodically re\-evolved from rollouts of later elites or from failed and ambiguous candidates collected during the search, rather than only fromp0p\_\{0\}\. Finally, evaluator and policy generation with multiple language models should also be tested to measure model sensitivity\.

## 7Conclusion

This work introducedMEMENTO, a memory\-guided single\-elite memetic framework for evolving executable code\-as\-policy programs in long\-horizon embodied tasks\. The method first evolves a rollout evaluator that maps executions to scalar fitness and task\-specific feedback metrics, then uses the fixed evaluator to guide policy search through sequential hill\-climbing with rejected\-proposal memory, macro\-mutation, and crossover between the best hill\-climb and macro\-mutation candidates\. On the Robosuite Franka Tower\-of\-Hanoi and AI2\-THOR household interaction tasks,MEMENTOachieved higher performance than LLM\-guided evolutionary baselines adapted to code\-as\-policy search, including on held\-out object geometries and unseen household scenes\. Ablations showed that zero\-shot policy generation, policy search with an unevolved zero\-shot evaluator, and removal of search branches each reduced final performance\. Finally, the best evolved Robosuite Tower\-of\-Hanoi code\-as\-policy was deployed on a physical Franka robot\. These results support memory\-guided memetic search as a mechanism to improve executable embodied policies using sparse rollout feedback\.

## References

- Modular multitask reinforcement learning with policy sketches\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 166–175\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- M\. Andrychowicz, F\. Wolski, A\. Ray, J\. Schneider, R\. Fong, P\. Welinder, B\. McGrew, J\. Tobin, P\. Abbeel, and W\. Zaremba \(2017\)Hindsight experience replay\.InAdvances in Neural Information Processing Systems,Vol\.30\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- P\. J\. Angeline \(1997\)Subtree crossover: building block engine or macromutation?\.InGenetic Programming 1997: Proceedings of the Second Annual Conference,pp\. 9–17\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p3.1)\.
- J\. A\. Arjona\-Medina, M\. Gillhofer, M\. Widrich, T\. Unterthiner, J\. Brandstetter, and S\. Hochreiter \(2019\)RUDDER: return decomposition for delayed rewards\.InAdvances in Neural Information Processing Systems,Vol\.32,pp\. 13544–13555\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- M\. F\. Brameier and W\. Banzhaf \(2007\)Linear genetic programming\.Springer\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p3.1)\.
- K\. Burns, A\. Jain, K\. Go, F\. Xia, M\. Stark, S\. Schaal, and K\. Hausman \(2024\)GenCHiP: generating robot policy code for high\-precision and contact\-rich manipulation tasks\.In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems \(IROS\),pp\. 9596–9603\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p2.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px2.p1.1)\.
- E\. Cantú\-Pazet al\.\(1998\)A survey of parallel genetic algorithms\.Calculateurs paralleles, reseaux et systems repartis10\(2\),pp\. 141–171\.Cited by:[§6](https://arxiv.org/html/2607.22832#S6.p2.2)\.
- N\. Carion, L\. Gustafson, Y\. Hu, S\. Debnath, R\. Hu, D\. Suris, C\. Ryali, K\. V\. Alwala, H\. Khedr, A\. Huang, J\. Lei, T\. Ma, B\. Guo, A\. Kalla, M\. Marks, J\. Greer, M\. Wang, P\. Sun, R\. Rädle, T\. Afouras, E\. Mavroudi, K\. Xu, T\. Wu, Y\. Zhou, L\. Momeni, R\. Hazra, S\. Ding, S\. Vaze, F\. Porcher, F\. Li, S\. Li, A\. Kamath, H\. K\. Cheng, P\. Dollár, N\. Ravi, K\. Saenko, P\. Zhang, and C\. Feichtenhofer \(2026\)SAM 3: segment anything with concepts\.InInternational Conference on Learning Representations,Cited by:[§A\.4](https://arxiv.org/html/2607.22832#A1.SS4.SSS0.Px3.p1.1),[§5\.4](https://arxiv.org/html/2607.22832#S5.SS4.p2.1)\.
- C\. Chi, Z\. Xu, S\. Feng, E\. Cousineau, Y\. Du, B\. Burchfiel, R\. Tedrake, and S\. Song \(2025\)Diffusion policy: visuomotor policy learning via action diffusion\.The International Journal of Robotics Research44\(10–11\),pp\. 1684–1704\.External Links:[Document](https://dx.doi.org/10.1177/02783649241273668)Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- T\. Duggan, P\. Lorang, H\. Lu, and M\. Scheutz \(2026\)The price is not right: neuro\-symbolic methods outperform vlas on structured long\-horizon manipulation tasks with significantly lower energy consumption\.arXiv preprint arXiv:2602\.19260\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p4.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2607.22832#S4.p1.1)\.
- J\. Echevarrieta, E\. Arza, and A\. Pérez \(2025\)Speeding\-up evolutionary algorithms to solve black\-box optimization problems\.IEEE Transactions on Evolutionary Computation29\(1\),pp\. 117–131\.External Links:[Document](https://dx.doi.org/10.1109/TEVC.2024.3352450)Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p3.1)\.
- A\. Ecoffet, J\. Huizinga, J\. Lehman, K\. O\. Stanley, and J\. Clune \(2021\)Go\-explore: a new approach for hard\-exploration problems\.Artificial Intelligence297,pp\. 103503\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Fernando, D\. Banarse, H\. Michalewski, S\. Osindero, and T\. Rocktäschel \(2023\)Promptbreeder: self\-referential self\-improvement via prompt evolution\.External Links:2309\.16797Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- F\. Glover and M\. Laguna \(1997\)Tabu search\.Springer,New York\.External Links:[Document](https://dx.doi.org/10.1007/978-1-4615-6089-0)Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px4.p1.1)\.
- Q\. Guo, R\. Wang, J\. Guo, B\. Li, K\. Song, X\. Tan, G\. Liu, J\. Bian, and Y\. Yang \(2023\)EvoPrompt: connecting large language models with evolutionary algorithms yields powerful prompt optimizers\.External Links:2309\.08532Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- D\. Hafner, J\. Pasukonis, J\. Ba, and T\. Lillicrap \(2025\)Mastering diverse control tasks through world models\.Nature640,pp\. 647–653\.External Links:[Document](https://dx.doi.org/10.1038/s41586-025-08744-2)Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- G\. Havur, K\. Haspalamutgil, C\. Palaz, E\. Erdem, and V\. Patoglu \(2013\)A case study on the tower of hanoi challenge: representation, reasoning and execution\.InProceedings of the IEEE International Conference on Robotics and Automation,pp\. 4552–4559\.External Links:[Document](https://dx.doi.org/10.1109/ICRA.2013.6631224)Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p4.1),[§4](https://arxiv.org/html/2607.22832#S4.p1.1)\.
- R\. Hazra, A\. Sygkounas, A\. Persson, A\. Loutfi, and P\. Z\. Dos Martires \(2025\)REvolve: reward evolution with large language models using human feedback\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p3.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2607.22832#S4.SS0.SSS0.Px4.p1.4)\.
- E\. Hemberg, S\. Moskal, and U\. O’Reilly \(2024\)Evolving code with a large language model\.Genetic Programming and Evolvable Machines25\(2\),pp\. 21\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p3.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- Q\. Hu, X\. Tong, M\. Yuan, F\. Liu, Z\. Lu, and Q\. Zhang \(2026\)Multimodal LLM\-assisted evolutionary search for programmatic control policies\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p3.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- W\. Huang, C\. Wang, R\. Zhang, Y\. Li, J\. Wu, and L\. Fei\-Fei \(2023a\)VoxPoser: composable 3d value maps for robotic manipulation with language models\.InProceedings of the 7th Conference on Robot Learning,Proceedings of Machine Learning Research, Vol\.229,pp\. 540–562\.Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px2.p1.1)\.
- W\. Huang, F\. Xia, T\. Xiao, H\. Chan, J\. Liang, P\. Florence, A\. Zeng, J\. Tompson, I\. Mordatch, Y\. Chebotar, P\. Sermanet, N\. Jackson, N\. Brown, L\. Luu, S\. Levine, K\. Hausman, and B\. Ichter \(2023b\)Inner monologue: embodied reasoning through planning with language models\.InProceedings of the 6th Conference on Robot Learning,Proceedings of Machine Learning Research, Vol\.205,pp\. 1769–1782\.Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px2.p1.1)\.
- B\. Ichter, A\. Brohan, Y\. Chebotar, C\. Finn, K\. Hausman, A\. Herzog, D\. Ho, J\. Ibarz, A\. Irpan, E\. Jang, R\. Julian, D\. Kalashnikov, S\. Levine, Y\. Lu, C\. Parada, K\. Rao, P\. Sermanet, A\. T\. Toshev, V\. Vanhoucke, F\. Xia, T\. Xiao, P\. Xu, M\. Yan, N\. Brown, M\. Ahn, O\. Cortes, N\. Sievers, C\. Tan, S\. Xu, D\. Reyes, J\. Rettinghouse, J\. Quiambao, P\. Pastor, L\. Luu, K\. Lee, Y\. Kuang, S\. Jesmonth, N\. J\. Joshi, K\. Jeffrey, R\. Jauregui Ruano, J\. Hsu, K\. Gopalakrishnan, B\. David, A\. Zeng, and C\. Fu \(2023\)Do as i can, not as i say: grounding language in robotic affordances\.InProceedings of the 6th Conference on Robot Learning,Proceedings of Machine Learning Research, Vol\.205,pp\. 287–318\.Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px2.p1.1)\.
- L\. P\. Kaelbling, M\. L\. Littman, and A\. R\. Cassandra \(1998\)Planning and acting in partially observable stochastic domains\.Artificial Intelligence101\(1–2\),pp\. 99–134\.External Links:[Document](https://dx.doi.org/10.1016/S0004-3702%2898%2900023-X)Cited by:[§3](https://arxiv.org/html/2607.22832#S3.p1.16)\.
- M\. Kim, K\. Chen, A\. Shrestha, L\. Fuxin, S\. Lee, and A\. Fern \(2026\)Humanoid hanoi: investigating shared whole\-body control for skill\-based box rearrangement\.External Links:2602\.13850Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- E\. Kolve, R\. Mottaghi, W\. Han, E\. VanderBilt, L\. Weihs, A\. Herrasti, M\. Deitke, K\. Ehsani, D\. Gordon, Y\. Zhu, A\. Kembhavi, A\. Gupta, and A\. Farhadi \(2017\)AI2\-THOR: an interactive 3d environment for visual ai\.External Links:1712\.05474Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p4.1),[§4](https://arxiv.org/html/2607.22832#S4.p1.1)\.
- J\. R\. Koza \(1992\)Genetic programming: on the programming of computers by means of natural selection\.MIT Press,Cambridge, MA, USA\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p3.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- J\. Lehman, J\. Gordon, S\. Jain, K\. Ndousse, C\. Yeh, and K\. O\. Stanley \(2023\)Evolution through large models\.InHandbook of Evolutionary Machine Learning,pp\. 331–366\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p3.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- J\. Liang, W\. Huang, F\. Xia, P\. Xu, K\. Hausman, B\. Ichter, P\. Florence, and A\. Zeng \(2023\)Code as policies: language model programs for embodied control\.In2023 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 9493–9500\.External Links:[Document](https://dx.doi.org/10.1109/ICRA48891.2023.10160591)Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p2.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px2.p1.1)\.
- F\. Liu, Y\. Liu, Q\. Zhang, X\. Tong, and M\. Yuan \(2026\)EoH\-S: evolution of heuristic set using LLMs for automated heuristic design\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 37090–37098\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v40i43.41038)Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- S\. Liu, H\. Yuan, M\. Hu, Y\. Li, Y\. Chen, S\. Liu, Z\. Lu, and J\. Jia \(2024\)RL\-GPT: integrating reinforcement learning and code\-as\-policy\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 28430–28459\.External Links:[Link](https://openreview.net/forum?id=LEzx6QRkRH)Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p2.1)\.
- W\. Liu, N\. Nie, R\. Zhang, J\. Mao, and J\. Wu \(2025\)Learning compositional behaviors from demonstration and language\.InProceedings of the 8th Conference on Robot Learning,Proceedings of Machine Learning Research, Vol\.270,pp\. 1992–2028\.Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Liu, A\. Bahety, and S\. Song \(2023\)REFLECT: summarizing robot experiences for failure explanation and correction\.InProceedings of The 7th Conference on Robot Learning,Proceedings of Machine Learning Research, Vol\.229,pp\. 3468–3484\.Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px2.p1.1)\.
- M\. Lozano, F\. Herrera, N\. Krasnogor, and D\. Molina \(2004\)Real\-coded memetic algorithms with crossover hill\-climbing\.Evolutionary Computation12\(3\),pp\. 273–302\.External Links:[Document](https://dx.doi.org/10.1162/1063656041774983)Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px4.p1.1)\.
- Y\. J\. Ma, W\. Liang, G\. Wang, D\. Huang, O\. Bastani, D\. Jayaraman, Y\. Zhu, L\. Fan, and A\. Anandkumar \(2024\)Eureka: human\-level reward design via coding large language models\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p3.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2607.22832#S4.SS0.SSS0.Px4.p1.4)\.
- O\. Mees, L\. Hermann, E\. Rosete\-Beas, and W\. Burgard \(2022\)CALVIN: a benchmark for language\-conditioned policy learning for long\-horizon robot manipulation tasks\.IEEE Robotics and Automation Letters7\(3\),pp\. 7327–7334\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- D\. Molina, M\. Lozano, C\. García\-Martínez, and F\. Herrera \(2010\)Memetic algorithms for continuous optimisation based on local search chains\.Evolutionary Computation18\(1\),pp\. 27–63\.External Links:[Document](https://dx.doi.org/10.1162/evco.2010.18.1.18102)Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px4.p1.1)\.
- P\. Moscato \(1989\)On evolution, search, optimization, genetic algorithms and martial arts: towards memetic algorithms\.Caltech Concurrent Computation Program, C3P Report826,pp\. 37\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p3.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px4.p1.1)\.
- F\. Neri and C\. Cotta \(2012\)Memetic algorithms and memetic computing optimization: a literature review\.Swarm and Evolutionary Computation2,pp\. 1–14\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p3.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px4.p1.1)\.
- A\. Novikov, N\. Vũ, M\. Eisenberger, E\. Dupont, P\. Huang, A\. Z\. Wagner, S\. Shirobokov, B\. Kozlovskii, F\. J\. R\. Ruiz, A\. Mehrabian, M\. P\. Kumar, A\. See, S\. Chaudhuri, G\. Holland, A\. Davies, S\. Nowozin, P\. Kohli, and M\. Balog \(2025\)AlphaEvolve: a coding agent for scientific and algorithmic discovery\.Technical reportGoogle DeepMind\.Note:Technical report, arXiv:2506\.13131Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- OpenAI \(2025\)Introducing gpt\-5\.2\.Note:https://openai\.com/index/introducing\-gpt\-5\-2/Accessed: 2026\-06\-09Cited by:[§4](https://arxiv.org/html/2607.22832#S4.SS0.SSS0.Px3.p1.12)\.
- A\. Padmakumar, J\. Thomason, A\. Shrivastava, P\. Lange, A\. Narayan\-Chen, S\. Gella, R\. Piramuthu, G\. Tur, and D\. Hakkani\-Tur \(2022\)TEACh: task\-driven embodied agents that chat\.InProceedings of the AAAI Conference on Artificial Intelligence,pp\. 2017–2025\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- D\. Pathak, P\. Agrawal, A\. A\. Efros, and T\. Darrell \(2017\)Curiosity\-driven exploration by self\-supervised prediction\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops,Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- M\. L\. Puterman \(1994\)Markov decision processes: discrete stochastic dynamic programming\.Wiley Series in Probability and Statistics,John Wiley & Sons\.External Links:ISBN 9780471619772,[Document](https://dx.doi.org/10.1002/9780470316887)Cited by:[§3](https://arxiv.org/html/2607.22832#S3.p1.16)\.
- B\. Romera\-Paredes, M\. Barekatain, A\. Novikov, M\. Balog, M\. P\. Kumar, E\. Dupont, F\. J\. R\. Ruiz, J\. S\. Ellenberg, P\. Wang, O\. Fawzi, P\. Kohli, and A\. Fawzi \(2024\)Mathematical discoveries from program search with large language models\.Nature625\(7995\),pp\. 468–475\.External Links:[Document](https://dx.doi.org/10.1038/s41586-023-06924-6)Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- M\. Shridhar, J\. Thomason, D\. Gordon, Y\. Bisk, W\. Han, R\. Mottaghi, L\. Zettlemoyer, and D\. Fox \(2020\)ALFRED: a benchmark for interpreting grounded instructions for everyday tasks\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 10740–10749\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- I\. Singh, V\. Blukis, A\. Mousavian, A\. Goyal, D\. Xu, J\. Tremblay, D\. Fox, J\. Thomason, and A\. Garg \(2023\)ProgPrompt: program generation for situated robot task planning using large language models\.Autonomous Robots47,pp\. 999–1012\.External Links:[Document](https://dx.doi.org/10.1007/s10514-023-10135-3)Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px2.p1.1)\.
- C\. H\. Song, J\. Wu, C\. Washington, B\. M\. Sadler, W\. Chao, and Y\. Su \(2023\)LLM\-Planner: few\-shot grounded planning for embodied agents with large language models\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 2998–3009\.Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Srivastava, C\. Li, M\. Lingelbach, R\. Martín\-Martín, F\. Xia, K\. E\. Vainio, Z\. Lian, C\. Gokmen, S\. Buch, K\. Liu, S\. Savarese, H\. Gweon, J\. Wu, and L\. Fei\-Fei \(2022\)BEHAVIOR: benchmark for everyday household activities in virtual, interactive, and ecological environments\.InProceedings of the 5th Conference on Robot Learning,Proceedings of Machine Learning Research, Vol\.164,pp\. 477–490\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- R\. S\. Sutton, D\. Precup, and S\. Singh \(1999\)Between MDPs and semi\-MDPs: a framework for temporal abstraction in reinforcement learning\.Artificial Intelligence112\(1–2\),pp\. 181–211\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Sygkounas, R\. Hazra, A\. Persson, P\. Z\. D\. Martires, and A\. Loutfi \(2026a\)COvolve: adversarial co\-evolution of large\-language\-model\-generated policies and environments via two\-player zero\-sum game\.InProceedings of the Genetic and Evolutionary Computation Conference,GECCO ’26,New York, NY, USA\.External Links:ISBN 979\-8\-4007\-2487\-9,[Document](https://dx.doi.org/10.1145/3795095.3805144)Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Sygkounas, A\. Loutfi, and A\. Persson \(2026b\)Evolutionary discovery of reinforcement learning algorithms via large language models\.InProceedings of the Genetic and Evolutionary Computation Conference,GECCO ’26,New York, NY, USA\.External Links:ISBN 979\-8\-4007\-2487\-9,[Document](https://dx.doi.org/10.1145/3795095.3805180)Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- C\. Wang, L\. Fan, J\. Sun, R\. Zhang, L\. Fei\-Fei, D\. Xu, Y\. Zhu, and A\. Anandkumar \(2023a\)MimicPlay: long\-horizon imitation learning by watching human play\.InProceedings of the 7th Conference on Robot Learning,Proceedings of Machine Learning Research, Vol\.229,pp\. 201–221\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. V\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2023b\)Self\-consistency improves chain of thought reasoning in language models\.InProc\. Int\. Conf\. Learn\. Represent\. \(ICLR\),Cited by:[§3\.1](https://arxiv.org/html/2607.22832#S3.SS1.p2.6)\.
- S\. Xie, H\. Wang, Z\. Xiao, R\. Wang, and X\. Chen \(2025\)Robotic programmer: video instructed policy code generation for robotic manipulation\.In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems \(IROS\),pp\. 14923–14930\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p2.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px2.p1.1)\.
- D\. Xu, S\. Nair, Y\. Zhu, J\. Gao, A\. Garg, L\. Fei\-Fei, and S\. Savarese \(2018\)Neural task programming: learning to generalize across hierarchical tasks\.In2018 IEEE international conference on robotics and automation \(ICRA\),pp\. 3795–3802\.Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- H\. Ye, J\. Wang, Z\. Cao, F\. Berto, C\. Hua, H\. Kim, J\. Park, and G\. Song \(2024\)ReEvo: large language models as hyper\-heuristics with reflective evolution\.Advances in Neural Information Processing Systems37,pp\. 43571–43608\.Cited by:[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px3.p1.1)\.
- C\. Ying, L\. Du, P\. Cheng, and Y\. Shu \(2025\)RoboInspector: unveiling the unreliability of policy code for LLM\-enabled robotic manipulation\.External Links:2508\.21378Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p2.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px2.p1.1)\.
- T\. Z\. Zhao, V\. Kumar, S\. Levine, and C\. Finn \(2023\)Learning fine\-grained bimanual manipulation with low\-cost hardware\.InProceedings of Robotics: Science and Systems,Daegu, Republic of Korea\.External Links:[Document](https://dx.doi.org/10.15607/RSS.2023.XIX.016)Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p1.1),[§2](https://arxiv.org/html/2607.22832#S2.SS0.SSS0.Px1.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-judge with MT\-Bench and Chatbot Arena\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§3\.1](https://arxiv.org/html/2607.22832#S3.SS1.p2.6)\.
- Y\. Zhu, J\. Wong, A\. Mandlekar, R\. Martín\-Martín, A\. Joshi, K\. Lin, A\. Maddukuri, S\. Nasiriany, and Y\. Zhu \(2020\)Robosuite: a modular simulation framework and benchmark for robot learning\.arXiv preprint arXiv:2009\.12293\.Cited by:[§1](https://arxiv.org/html/2607.22832#S1.p4.1),[§4](https://arxiv.org/html/2607.22832#S4.p1.1)\.

The appendix is organized as follows\. Appendix[A](https://arxiv.org/html/2607.22832#A1)details the Robosuite Franka Tower\-of\-Hanoi environment, domain randomization, the physical Franka deployment, held\-out object variants, and the SAC\+HER baselines\. Appendix[B](https://arxiv.org/html/2607.22832#A2)details the AI2\-THOR environment, the code\-as\-policy interface, and the observation space\. Appendix[C](https://arxiv.org/html/2607.22832#A3)reports the evolved evaluators and representative evolved policy programs\.

## Appendix ARobosuite Franka Tower\-of\-Hanoi Environment

### A\.1Task and Simulator Setup

The Robosuite task uses a Franka Panda arm, four cubes, and three containers\. The cubes are ordered by size: extra\-large, large, medium, and small\. At reset, all four cubes are stacked on container A in legal Tower\-of\-Hanoi order, with the extra\-large cube at the bottom and the small cube at the top\. The goal is to move the complete tower to container C, preserve the same size order, and leave containers A and B empty\. An episode terminates on success, illegal Hanoi placement, or timeout after 12000 environment steps\.

![Refer to caption](https://arxiv.org/html/2607.22832v1/x11.png)Figure 7:Initial reset state of the Robosuite Franka Panda Tower\-of\-Hanoi task\. Four cubes are stacked on container A; the target is to move the full tower to container C while preserving the size order\.The cube half\-edge lengths are fixed by the environment:

ssmall=0\.022,smedium=0\.025,slarge=0\.028,sxlarge=0\.031\.s\_\{\\mathrm\{small\}\}=0\.022,\\quad s\_\{\\mathrm\{medium\}\}=0\.025,\\quad s\_\{\\mathrm\{large\}\}=0\.028,\\quad s\_\{\\mathrm\{xlarge\}\}=0\.031\.The containers are plate\-with\-hole objects\. The hole is shifted from the container origin along the container\-local positivexx\-axis\. The local hole interval is

xlocal∈\[0\.02,0\.09\],\|ylocal\|≤0\.035,x\_\{\\mathrm\{local\}\}\\in\[0\.02,0\.09\],\\qquad\|y\_\{\\mathrm\{local\}\}\|\\leq 0\.035,so the nominal hole center is at local offset

hlocal=\[0\.055,0,0\]⊤\.h\_\{\\mathrm\{local\}\}=\[0\.055,0,0\]^\{\\top\}\.For a container with world positionbband orientationRbR\_\{b\}, the placement center used by policy code is

h​\(b,Rb\)=b\+Rb​hlocal\.h\(b,R\_\{b\}\)=b\+R\_\{b\}h\_\{\\mathrm\{local\}\}\.

### A\.2Observation and Action Space

Policy\-code evolution generates a Python classFrankaPolicywith two required methods\. The methodreset\(\)clears episode\-level policy state before each rollout\. The methodcompute\_action\(obs\)is called at every control step and returns a continuous seven\-dimensional action vector:

at=\[Δ​x,Δ​y,Δ​z,Δ​ϕ,Δ​θ,Δ​ψ,Δ​g\]∈\[−1,1\]7\.a\_\{t\}=\[\\Delta x,\\Delta y,\\Delta z,\\Delta\\phi,\\Delta\\theta,\\Delta\\psi,\\Delta g\]\\in\[\-1,1\]^\{7\}\.The first three entries are end\-effector position increments, the next three entries are axis\-angle orientation increments, and the final entry controls the gripper\. Positive gripper values close the gripper, negative values open it, and zero holds the current command\.

The generated policy receives the structured Robosuite observation dictionary directly\. The task\-relevant observation fields are listed in Table[2](https://arxiv.org/html/2607.22832#A1.T2)\. The policy is instructed to use these structured keys rather than packed aggregate vectors\.

Table 2:Observation fields used by policy\-code evolution in the Robosuite Franka Tower\-of\-Hanoi task\.The evolved policy parses the four cube poses and sizes, computes the hole center of each container fromboxes\_posandboxes\_quat, and infers the current cube\-to\-container assignment\. Its symbolic component generates the recursive Tower\-of\-Hanoi move sequence from containerAAto containerCCthrough containerBB, yielding24−1=152^\{4\}\-1=15ordered moves\. For each scheduled move, the policy selects the top observed cube on the source container, recomputes pick\-and\-place targets from the current object and container poses, and executes the move via closed\-loop pick\-and\-place control\. The symbolic Hanoi ordering remains fixed; closed\-loop adaptation comes from observation\-based cube assignment, grasp/lift checks, and geometric target recomputation during execution\.

### A\.3Domain Randomization for Policy\-Code Evolution

At each reset, the environment randomizes observation noise, calibration bias, contact parameters, cube mass, joint damping, actuator gain, world position offset, and initial cube placement\. These perturbations modify both the object\-state observations received by the policy and the underlying MuJoCo dynamics, while preserving the same task definition and success condition\.

Table 3:Heavy domain randomization used for policy\-code evolution in the Robosuite Franka Tower\-of\-Hanoi environment\.The position\-noise standard deviation is sampled once per episode and then used when cube and container positions are observed\. The resulting sensor value is the true object position plus an episode\-level position bias and per\-observation Gaussian noise\. The world XY offset changes the physical placement of containers A, B, and C at reset\. The initial cube XY perturbation applies an independent physical offset to each cube in the initial tower\. Observation latency is not included in Table[3](https://arxiv.org/html/2607.22832#A1.T3), because it is not applied to the returned observations in the implementation used for the experiments\.

### A\.4Physical Franka Deployment

This appendix details the physical Franka deployment used for the sim\-to\-real result in Section[5\.4](https://arxiv.org/html/2607.22832#S5.SS4)\. The deployed policy is the best\-performing Robosuite Tower\-of\-Hanoi code\-as\-policy selected across the MEMENTO seeds\. The selected policy uses a fixed, recursive Tower\-of\-Hanoi move schedule, together with guarded, closed\-loop execution of each pick\-and\-place move\. The evolved program logic is kept fixed, while the simulator observation and action interfaces are replaced by real\-robot perception and execution\.

##### Deployment interface\.

Letpbestp\_\{\\mathrm\{best\}\}denote the selected Robosuite code\-as\-policy\. In simulation,pbestp\_\{\\mathrm\{best\}\}receives the Robosuite observation dictionaryoto\_\{t\}, including robot state, cube poses, cube sizes, and container poses, and returns a seven\-dimensional end\-effector action\. On the physical robot, a deployment wrapper converts RGB\-D observations and robot\-state measurements into the same object\-centric fields consumed by the policy:

otR=ΨR​\(ytR\),o\_\{t\}^\{R\}=\\Psi\_\{R\}\(y\_\{t\}^\{R\}\),whereytRy\_\{t\}^\{R\}denotes the raw real\-robot observation andΨR\\Psi\_\{R\}estimates the cube poses, cube sizes, container poses, and end\-effector pose in the Franka base frame\.

GivenotRo\_\{t\}^\{R\}, the physical execution version of the policy evaluates the same internal finite\-state controller as in simulation\. At each control step, the policy identifies the active manipulation phase, computes the current Cartesian pick or place target, and issues the corresponding gripper command:

\(αt,xt⋆,gt,zt\+1\)=pbestR​\(otR,zt\)\.\(\\alpha\_\{t\},x\_\{t\}^\{\\star\},g\_\{t\},z\_\{t\+1\}\)=p\_\{\\mathrm\{best\}\}^\{R\}\(o\_\{t\}^\{R\},z\_\{t\}\)\.Hereαt\\alpha\_\{t\}is the active manipulation phase,xt⋆x\_\{t\}^\{\\star\}is the Cartesian target,gtg\_\{t\}is the gripper command, andztz\_\{t\}is the policy state\. The physical execution stack maps these phase\-specific targets to Franka motion and gripper commands,

utR=ΓR​\(αt,xt⋆,gt\)\.u\_\{t\}^\{R\}=\\Gamma\_\{R\}\(\\alpha\_\{t\},x\_\{t\}^\{\\star\},g\_\{t\}\)\.Thus,ΨR\\Psi\_\{R\}replaces the simulator observation source andΓR\\Gamma\_\{R\}replaces the simulator action interface, while the Tower\-of\-Hanoi state machine remains fixed\.

##### Preserved program logic\.

The preserved policy components are the object\-centric parts of the evolved code\-as\-policy\. These include the recursive1515\-move Tower\-of\-Hanoi schedule, the active move index, the manipulation finite\-state machine, filtered pose estimates, retry\-conditioned pick offsets, guarded descent, stack\-height\-based placement\-target computation, grasp verification, placement centering, and post\-placement verification\. The deployed policy does not perform global BFS replanning\. Instead, the symbolic Hanoi ordering remains fixed, and recovery is local to the active move: failed grasps or failed post\-placement checks cause the same scheduled move to be retried from the updated object observation\.

##### RGB\-D object grounding\.

Robosuite object\-state observations are replaced by RGB\-D object grounding\. SAM3\-based language\-guided segmentation is queried with the physical object labels used in the task: the four cube colors and the three container markers\(Carionet al\.,[2026](https://arxiv.org/html/2607.22832#bib.bib54)\)\. The resulting segmentation masks are combined with depth measurements and calibrated camera extrinsics to estimate object poses in the Franka base frame\. The estimated cube poses, cube sizes, container poses, and end\-effector pose are written into the same object\-centric fields consumed bypbestp\_\{\\mathrm\{best\}\}\. Cube poses are estimated online during execution\. Container poses are measured once and reused as fixed container locations during the physical trials\.

##### Physical execution\.

Policy\-selected pick\-and\-place targets are executed by the Franka motion and gripper stack\. Long transfers between task locations are executed through motion planning\. Short vertical motions for grasping, lifting, placing, and retreating are executed as local Cartesian motions\. The gripper command is provided by the policy and executed by the physical wrapper\. After each executed phase, the current object configuration is observed again, and the active move targets are recomputed from the updated object\-centric state\. If the grasp verification or post\-placement checking fails, the policy retries the same scheduled Hanoi move rather than advancing the symbolic move index\.

##### Reduced real\-robot evaluator\.

The simulator evaluatorE⋆E^\{\\star\}returns scalar fitness and feedback metrics from simulator rollouts,

E⋆​\(𝒯Nroll​\(p\)\)=\(F​\(p\),𝐌​\(p\)\)\.E^\{\\star\}\(\\mathcal\{T\}\_\{N\_\{\\mathrm\{roll\}\}\}\(p\)\)=\\bigl\(F\(p\),\\mathbf\{M\}\(p\)\\bigr\)\.For physical deployment, a reduced evaluatorERE\_\{R\}, distinct from the evolvedE⋆E^\{\\star\}, computes feedback metrics only from robot execution and returns no fitness\. LetτiR​\(pbest,κi\)\\tau\_\{i\}^\{R\}\(p\_\{\\mathrm\{best\}\},\\kappa\_\{i\}\)denote the physical execution obtained at calibration stepiiwith execution constantsκi\\kappa\_\{i\}\. The reduced evaluator is

ER​\(τiR​\(pbest,κi\)\)=𝐌iR,𝐌iR=\(mi,1R,…,mi,dRR\)\.E\_\{R\}\\bigl\(\\tau\_\{i\}^\{R\}\(p\_\{\\mathrm\{best\}\},\\kappa\_\{i\}\)\\bigr\)=\\mathbf\{M\}\_\{i\}^\{R\},\\qquad\\mathbf\{M\}\_\{i\}^\{R\}=\\bigl\(m\_\{i,1\}^\{R\},\\ldots,m\_\{i,d\_\{R\}\}^\{R\}\\bigr\)\.The metrics in𝐌iR\\mathbf\{M\}\_\{i\}^\{R\}are measured from robot execution and perceived object poses\. They are used to calibrate the deployed controller\.ERE\_\{R\}does not introduce a new policy\-search objective and is not used for selection during MEMENTO search\.

##### LLM\-assisted calibration\.

Calibration adjusts execution constants before the final physical evaluation\. Letκi\\kappa\_\{i\}denote the execution constants at calibration stepii, and letcRc\_\{R\}denote the real\-robot execution code\. At each step, the current deployment is executed on the robot,𝐌iR\\mathbf\{M\}\_\{i\}^\{R\}is computed byERE\_\{R\}, and the language model receives

\(pbest,cR,κi,𝐌iR\)\.\\left\(p\_\{\\mathrm\{best\}\},\\;c\_\{R\},\\;\\kappa\_\{i\},\\;\\mathbf\{M\}\_\{i\}^\{R\}\\right\)\.The calibration proposal is

κi\+1=ℒcal​\(pbest,cR,κi,𝐌iR\),i=0,…,Ncal−1\.\\kappa\_\{i\+1\}=\\mathcal\{L\}\_\{\\mathrm\{cal\}\}\\left\(p\_\{\\mathrm\{best\}\},c\_\{R\},\\kappa\_\{i\},\\mathbf\{M\}\_\{i\}^\{R\}\\right\),\\qquad i=0,\\ldots,N\_\{\\mathrm\{cal\}\}\-1\.We useNcal=3N\_\{\\mathrm\{cal\}\}=3calibration steps\. The final constantsκ⋆\\kappa^\{\\star\}are selected from the tested calibration sequence and used for the final physical evaluation\.

##### Calibrated constants\.

Table[4](https://arxiv.org/html/2607.22832#A1.T4)reports the execution constants changed for physical deployment\. These constants affect pick depth, approach height, container assignment tolerance, and place\-hover height\.

Table 4:Execution constants for physical Franka deployment\. Values are in meters\.
##### Physical target corrections\.

The physical execution wrapper also applies small Cartesian corrections after the policy selects the current pick\-or\-place target\. Pick targets are shifted by a fixed lateral centering offset\. Stack\-placement targets receive a fixed lateral correction when placed onto an existing stack\. During the place\-descend phase, the target height is lowered to seat the cube on the support surface\. These corrections are part of the real execution interface and do not alter move selection, cube\-to\-container assignment, or move retry\.

### A\.5Held\-Out Tower\-of\-Hanoi Object Variants

To evaluate object\-geometry transfer, we use two held\-out Robosuite Tower\-of\-Hanoi variants\. Both variants preserve the Hanoi objective, object identifiers, target layout, and structured policy observation interface while changing the geometry of the manipulated objects\. This tests whether the policy relies on Hanoi state relations, such as target assignment, stack order, and placement targets, rather than on the cubic object geometry used during policy search\.

![Refer to caption](https://arxiv.org/html/2607.22832v1/x12.png)

\(a\) Asymmetric stackable objects

![Refer to caption](https://arxiv.org/html/2607.22832v1/x13.png)

\(b\) Cylindrical disks

Figure 8:Held\-out Robosuite Tower\-of\-Hanoi object variants\. Both variants preserve the Hanoi task and policy observation interface used for the cubic training environment\. The asymmetric\-object variant changes the grasp envelope while retaining a stackable support core\. The cylindrical\-disk variant changes the object contact geometry by replacing cube faces with curved cylinder sides\.The asymmetric\-object variant replaces each cube with a stackable central core and two protruding arms\. The object remains compatible with the Hanoi stacking task, but its side surfaces and grasp points differ from those of a cube with the same scalar size\. The cylindrical\-disk variant replaces each cube with a cylindrical disk\. This keeps flat top–bottom stacking surfaces but changes the lateral contact geometry during grasping and transport\.

Table 5:Difference between the training Robosuite Tower\-of\-Hanoi environment and the held\-out object variants\.
### A\.6SAC\+HER Observation and Action Space

The SAC\+HER baselines use the same Robosuite task and continuous low\-level action space as the policy\-code experiments, but their observation representation differs\. The HER wrapper returns a Gymnasium dictionary observation:

ot=\{otbase,gtach,g⋆\}\.o\_\{t\}=\\left\\\{o\_\{t\}^\{\\mathrm\{base\}\},g\_\{t\}^\{\\mathrm\{ach\}\},g^\{\\star\}\\right\\\}\.The base observationotbaseo\_\{t\}^\{\\mathrm\{base\}\}is the flattened Robosuite object\-state observation\. The achieved goalgtachg\_\{t\}^\{\\mathrm\{ach\}\}is the current Cartesian position of the four cubes,

gtach=\[xtsmall,xtmedium,xtlarge,xtxlarge\]∈ℝ12\.g\_\{t\}^\{\\mathrm\{ach\}\}=\[x\_\{t\}^\{\\mathrm\{small\}\},x\_\{t\}^\{\\mathrm\{medium\}\},x\_\{t\}^\{\\mathrm\{large\}\},x\_\{t\}^\{\\mathrm\{xlarge\}\}\]\\in\\mathbb\{R\}^\{12\}\.The desired goalg⋆g^\{\\star\}is the target tower on container C, with the extra\-large cube at the bottom, followed by the large, medium, and small cubes\. Quaternion goals are disabled, so both achieved and desired goals contain cube positions only\.

The use ofMultiInputPolicyfollows from this dictionary observation\. SAC receivesobservation,achieved\_goal, anddesired\_goalas separate input fields\. The action space is the Robosuite continuous action box exposed by the Franka environment\.

A goal is successful when all four cube positions are withinϵgoal=0\.025\\epsilon\_\{\\mathrm\{goal\}\}=0\.025of their corresponding desired positions:

succ​\(gtach,g⋆\)=𝕀​\[maxj∈\{1,…,4\}⁡‖gt,jach−gj⋆‖2<ϵgoal\]\.\\mathrm\{succ\}\(g\_\{t\}^\{\\mathrm\{ach\}\},g^\{\\star\}\)=\\mathbb\{I\}\\left\[\\max\_\{j\\in\\\{1,\\ldots,4\\\}\}\\left\\\|g\_\{t,j\}^\{\\mathrm\{ach\}\}\-g\_\{j\}^\{\\star\}\\right\\\|\_\{2\}<\\epsilon\_\{\\mathrm\{goal\}\}\\right\]\.

### A\.7SAC\+HER Training Configuration

Both SAC\+HER baselines use Soft Actor\-Critic with Hindsight Experience Replay\. HER uses future\-goal relabeling: additional goals are sampled from states achieved later in the same trajectory, and the reward is recomputed using the sparse or dense goal\-conditioned reward\.

Table 6:Shared SAC\+HER training configuration for the Robosuite Franka Tower\-of\-Hanoi task\.GroupParameterValueEnvironmentRobotPandaEnvironmentCamera observationsdisabledEnvironmentObject observationsenabledEnvironmentControl frequency2020HzEnvironmentEpisode horizon1200012000stepsEnvironmentGoal tolerance0\.0250\.025SACPolicyMultiInputPolicySACTotal timesteps50,000,00050\{,\}000\{,\}000SACLearning rate10−410^\{\-4\}SACReplay buffer size2,000,0002\{,\}000\{,\}000SACLearning starts25,00025\{,\}000SACBatch size512512SACDiscount factorγ\\gamma0\.9950\.995SACTarget smoothingτ\\tau0\.0050\.005SACTrain frequency11stepSACGradient steps11SACEntropy coefficientautoHERReplay bufferHerReplayBufferHERGoal selection strategyfutureHERSampled goals per transition88EvaluationEvaluation frequency1,000,0001\{,\}000\{,\}000stepsEvaluationEvaluation episodes55Episode success, episode fitness, phase scores, milestones, and interaction\-quality metrics are logged at episode termination\. These metrics are used for analysis and comparison with policy\-code evolution\. The SAC update uses only the HER reward defined in Appendix[A\.8](https://arxiv.org/html/2607.22832#A1.SS8)and Appendix[A\.9](https://arxiv.org/html/2607.22832#A1.SS9)\. The SAC\+HER baselines use the goal\-conditioned wrapper described above; they do not use the heavy domain\-randomized environment from Appendix[A\.3](https://arxiv.org/html/2607.22832#A1.SS3)\.

### A\.8Sparse SAC\+HER

The sparse SAC\+HER baseline uses a success\-only goal\-conditioned reward:

rsparse​\(gtach,g⋆\)=\{0,succ​\(gtach,g⋆\)=1,−1,succ​\(gtach,g⋆\)=0\.r\_\{\\mathrm\{sparse\}\}\(g\_\{t\}^\{\\mathrm\{ach\}\},g^\{\\star\}\)=\\begin\{cases\}0,&\\mathrm\{succ\}\(g\_\{t\}^\{\\mathrm\{ach\}\},g^\{\\star\}\)=1,\\\\ \-1,&\\mathrm\{succ\}\(g\_\{t\}^\{\\mathrm\{ach\}\},g^\{\\star\}\)=0\.\\end\{cases\}All unsuccessful configurations receive the same reward, independent of their distance to the target tower or their intermediate stack structure\. During HER replay, relabeled transitions are scored by the same success\-only criterion with respect to the sampled future goal\.

### A\.9Dense SAC\+HER

The dense SAC\+HER baseline uses the same observation representation, action space, SAC configuration, and HER relabeling method, but replaces the success\-only reward with geometric shaping\. Let

dt,j=‖gt,jach−gj⋆‖2d\_\{t,j\}=\\left\\\|g\_\{t,j\}^\{\\mathrm\{ach\}\}\-g\_\{j\}^\{\\star\}\\right\\\|\_\{2\}be the Cartesian position error for cubejj, and let

d¯t=14​∑j=14dt,j\\bar\{d\}\_\{t\}=\\frac\{1\}\{4\}\\sum\_\{j=1\}^\{4\}d\_\{t,j\}be the mean cube\-position error\. The dense reward is

rdense​\(gtach,g⋆\)=−d¯t\+5​succ​\(gtach,g⋆\)−\(1−legal​\(gtach\)\)\.r\_\{\\mathrm\{dense\}\}\(g\_\{t\}^\{\\mathrm\{ach\}\},g^\{\\star\}\)=\-\\bar\{d\}\_\{t\}\+5\\,\\mathrm\{succ\}\(g\_\{t\}^\{\\mathrm\{ach\}\},g^\{\\star\}\)\-\\left\(1\-\\mathrm\{legal\}\(g\_\{t\}^\{\\mathrm\{ach\}\}\)\\right\)\.The legality term checks the vertical cube ordering:

legal​\(gtach\)=𝕀​\[ztxlarge<ztlarge<ztmedium<ztsmall\]\.\\mathrm\{legal\}\(g\_\{t\}^\{\\mathrm\{ach\}\}\)=\\mathbb\{I\}\\left\[z\_\{t\}^\{\\mathrm\{xlarge\}\}<z\_\{t\}^\{\\mathrm\{large\}\}<z\_\{t\}^\{\\mathrm\{medium\}\}<z\_\{t\}^\{\\mathrm\{small\}\}\\right\]\.The dense reward combines distance\-to\-target shaping, an exact success bonus, and a penalty for violating the Tower\-of\-Hanoi ordering constraint\. During HER replay, the same dense reward is recomputed after the desired goal is replaced by the sampled future achieved goal\.

### A\.10Sparse–Dense Difference

The sparse and dense SAC\+HER baselines differ only in the goal\-conditioned reward returned bycompute\_reward\. The sparse baseline uses only final goal achievement\. The dense baseline adds continuous geometric progress and an ordering penalty\. Both baselines use the same goal\-conditioned observation representation, continuous action space, HER relabeling method, SAC architecture, training schedule, and logging metrics\.

## Appendix BAI2\-THOR Household Environment

### B\.1Task and Simulator Setup

The AI2\-THOR domain is a partially observed household interaction task in kitchen scenes\. The task isapple\_in\_microwave\_on\_bread\_in\_fridge\. The agent must find an Apple, a Bread, a Microwave, and a Fridge; place the Apple in the Microwave, toggle the Microwave on, and place the Bread in the Fridge\. Success is defined by

apple​\_​in​\_​microwave∧microwave​\_​on∧bread​\_​in​\_​fridge\.\\mathrm\{apple\\\_in\\\_microwave\}\\;\\wedge\\;\\mathrm\{microwave\\\_on\}\\;\\wedge\\;\\mathrm\{bread\\\_in\\\_fridge\}\.The task is partially observed: the agent receives egocentric observations, and only currently visible objects are exposed for interaction\. Object identifiers are scene\-specific and must be obtained from the observation before they can be used in object actions\.

![Refer to caption](https://arxiv.org/html/2607.22832v1/x14.png)FloorPlan1![Refer to caption](https://arxiv.org/html/2607.22832v1/x15.png)FloorPlan3⋯\\cdots![Refer to caption](https://arxiv.org/html/2607.22832v1/x16.png)FloorPlan29Figure 9:Examples of AI2\-THOR kitchen scenes used in the household interaction domain\. The scenes differ in layout, object placement, visibility, and receptacle configuration\.Table 7:AI2\-THOR simulator configuration\.
### B\.2AI2\-THOR Code\-as\-Policy Interface

Code as policy evolution generates a Python class namedPolicy\. The class is instantiated asPolicy\(seed=0\)\. Before each rollout,reset\(\)clears episode\-level memory\. At each environment step,compute\_action\(obs\)is called once and returns

\(at,ιt\),\(a\_\{t\},\\iota\_\{t\}\),whereata\_\{t\}is an action name andιt\\iota\_\{t\}is either an AI2\-THOR object identifier orNone\. Navigation and camera actions useιt=None\\iota\_\{t\}=\\texttt\{None\}\. Object\-interaction actions require an object identifier that is valid for the current observation\.

Table 8:AI2\-THOR action space exposed to generated policy code\.For every object\-parameterized action, the returned identifier must belong to the corresponding valid\-action list:

ιt∈obs​\["action\_space"\]​\[at\]\.\\iota\_\{t\}\\in\\mathrm\{obs\}\[\\texttt\{"action\\\_space"\}\]\[a\_\{t\}\]\.The valid object identifiers are recomputed at every step from the simulator state, including visibility, distance, inventory state, object state, and AI2\-THOR action preconditions\.

### B\.3Observation Space

The observation is a Python dictionary containing the current task, scene, egocentric agent state, inventory state, task\-object fields, visible object summaries, task\-progress predicates, valid actions, the previous action result, and step counters\.

Table 9:Top\-level AI2\-THOR observation fields\.Theagentfield contains position, rotation, and camera horizon\. Theinventoryfield contains the held object type and held object identifier\. Thetargetsfield reports object existence, object identifier, and visibility for Apple, Bread, Microwave, and Fridge:

targets=\{\\displaystyle\\texttt\{targets\}=\\\{apple\_exists\_in\_scene,apple\_object\_id,apple\_visible,\\displaystyle\\texttt\{apple\\\_exists\\\_in\\\_scene\},\\texttt\{apple\\\_object\\\_id\},\\texttt\{apple\\\_visible\},bread\_exists\_in\_scene,bread\_object\_id,bread\_visible,\\displaystyle\\texttt\{bread\\\_exists\\\_in\\\_scene\},\\texttt\{bread\\\_object\\\_id\},\\texttt\{bread\\\_visible\},microwave\_exists\_in\_scene,microwave\_object\_id,microwave\_visible,\\displaystyle\\texttt\{microwave\\\_exists\\\_in\\\_scene\},\\texttt\{microwave\\\_object\\\_id\},\\texttt\{microwave\\\_visible\},fridge\_exists\_in\_scene,fridge\_object\_id,fridge\_visible\}\.\\displaystyle\\texttt\{fridge\\\_exists\\\_in\\\_scene\},\\texttt\{fridge\\\_object\\\_id\},\\texttt\{fridge\\\_visible\}\\\}\.
Table 10:Task\-progress predicates for the AI2\-THOR household task\.Each entry invisible\_objectsis a summary of one currently visible object\. These summaries expose the object type, exact object identifier, visibility, distance, interaction affordances, current object state, and containing receptacles\.

Table 11:Fields in each visible object summary\.The previous action result is part of the observation\. It contains the previous action name, object identifier, success flag, normalized error category, and raw error message:

last\_action\_result=\{action,object\_id,success,error\_category,error\_message\}\.\\texttt\{last\\\_action\\\_result\}=\\\{\\texttt\{action\},\\texttt\{object\\\_id\},\\texttt\{success\},\\texttt\{error\\\_category\},\\texttt\{error\\\_message\}\\\}\.The error category can indicate blocked movement, collision, a non\-visible object, a non\-reachable object, a failed precondition, a generic failure, or a simulator exception\.

### B\.4Partial Observability and Object Identifiers

The policy does not receive a complete scene graph at every step\.visible\_objectscontains only objects visible from the current egocentric pose\. Task\-object identifiers may therefore be unavailable until the agent changes viewpoint, moves closer, or opens a receptacle\. The exactobjectIdstring is required for object actions, and object identifiers can differ across floor plans\.

The action space is the source of validity for interactions\. Even when a target object identifier is known from memory, an interaction is valid only if that identifier appears in the corresponding list inobs\["action\_space"\]\. This makes the policy interface state\-dependent: the set of valid object actions changes with visibility, distance, object state, inventory state, and simulator preconditions\.

## Appendix CAdditional Results

### C\.1Evaluator Evolution Results

Section[3\.1](https://arxiv.org/html/2607.22832#S3.SS1)defines evaluator evolution and the fixed evaluatorE⋆E^\{\\star\}used during code\-as\-policy search\. Here, we report the task\-specific form of the selected evaluators\. In both domains,E⋆E^\{\\star\}returns scalar fitness for selection and feedback metrics for subsequent code\-as\-policy proposals:

E⋆​\(𝒯Nroll​\(p\)\)=\(F​\(p\),𝐌​\(p\)\)\.E^\{\\star\}\(\\mathcal\{T\}\_\{N\_\{\\mathrm\{roll\}\}\}\(p\)\)=\(F\(p\),\\mathbf\{M\}\(p\)\)\.The empirical success rateS​\(p\)S\(p\)is computed from the task success indicator and recorded alongside\.

#### C\.1\.1Robosuite Franka Tower\-of\-Hanoi

For Robosuite, the selected evaluator is defined for the four\-cube Tower\-of\-Hanoi task with cubes

𝒦=\{xlarge,large,medium,small\}\\mathcal\{K\}=\\\{\\mathrm\{xlarge\},\\mathrm\{large\},\\mathrm\{medium\},\\mathrm\{small\}\\\}and containersA,B,CA,B,C\. Letinj​\(c\)\\mathrm\{in\}\_\{j\}\(c\)denote that cubecclies inside the hole region of containerjj, and letstack​\(c1,c2\)\\mathrm\{stack\}\(c\_\{1\},c\_\{2\}\)denote that cubec2c\_\{2\}is stacked on cubec1c\_\{1\}with the required horizontal and vertical tolerances\. The binary success predicate requires all cubes to be in the target containerCC, containersAAandBBto be empty, and the stack onCCto satisfy the legal Hanoi order:

successe=\\displaystyle\\mathrm\{success\}\_\{e\}=\{\}\(⋀c∈𝒦inC​\(c\)\)∧\(⋀c∈𝒦¬inA​\(c\)∧¬inB​\(c\)\)\\displaystyle\\left\(\\bigwedge\_\{c\\in\\mathcal\{K\}\}\\mathrm\{in\}\_\{C\}\(c\)\\right\)\\wedge\\left\(\\bigwedge\_\{c\\in\\mathcal\{K\}\}\\neg\\mathrm\{in\}\_\{A\}\(c\)\\wedge\\neg\\mathrm\{in\}\_\{B\}\(c\)\\right\)∧stack​\(xlarge,large\)∧stack​\(large,medium\)∧stack​\(medium,small\)\.\\displaystyle\\wedge\\;\\mathrm\{stack\}\(\\mathrm\{xlarge\},\\mathrm\{large\}\)\\wedge\\mathrm\{stack\}\(\\mathrm\{large\},\\mathrm\{medium\}\)\\wedge\\mathrm\{stack\}\(\\mathrm\{medium\},\\mathrm\{small\}\)\.The task success indicator is

Se=𝕀​\{successe\},S\_\{e\}=\\mathbb\{I\}\\\{\\mathrm\{success\}\_\{e\}\\\},computed from the environment success predicate\. The evaluator also returns a dense episode fitnessfe∈\[0,1\]f\_\{e\}\\in\[0,1\]\. The policy\-level fitness is the mean episode fitness over the rollout budget:

F​\(p\)=1Nroll​∑e=1Nrollfe​\(p\)\.F\(p\)=\\frac\{1\}\{N\_\{\\mathrm\{roll\}\}\}\\sum\_\{e=1\}^\{N\_\{\\mathrm\{roll\}\}\}f\_\{e\}\(p\)\.
At each control steptt, the evaluator computes phase scores for approach, grasp/lift, transport, placement, and final arrangement\. The approach score is the maximum proximity score between the end effector and any cube:

At=maxc∈𝒦⁡exp⁡\(−‖xeef,t−xc,t‖0\.12\)\.A\_\{t\}=\\max\_\{c\\in\\mathcal\{K\}\}\\exp\\\!\\left\(\-\\frac\{\\\|x\_\{\\mathrm\{eef\},t\}\-x\_\{c,t\}\\\|\}\{0\.12\}\\right\)\.The grasp/lift score is the maximum normalized lift height:

Gt=maxc∈𝒦⁡clip\[0,1\]​\(zc,t−\(ztable\+sc\)0\.08\),G\_\{t\}=\\max\_\{c\\in\\mathcal\{K\}\}\\mathrm\{clip\}\_\{\[0,1\]\}\\left\(\\frac\{z\_\{c,t\}\-\(z\_\{\\mathrm\{table\}\}\+s\_\{c\}\)\}\{0\.08\}\\right\),wherescs\_\{c\}is the cube half\-edge length\. The transport score rewards lifting a cube while moving it toward the target container:

Tt=maxc∈𝒦⁡ℓc,t​exp⁡\(−dC​\(c,t\)0\.18\),T\_\{t\}=\\max\_\{c\\in\\mathcal\{K\}\}\\ell\_\{c,t\}\\exp\\\!\\left\(\-\\frac\{d\_\{C\}\(c,t\)\}\{0\.18\}\\right\),whereℓc,t\\ell\_\{c,t\}is the normalized lift score of cubecc, anddC​\(c,t\)d\_\{C\}\(c,t\)is the planar error between cubeccand the target hole center in containerCC\.

Placement is measured using continuous hole scores\. Lethj​\(c,t\)∈\[0,1\]h\_\{j\}\(c,t\)\\in\[0,1\]be the score that cubecclies inside the hole of containerjj\. The target\-fill and source\-emptiness terms are

Cfill,t=14​∑c∈𝒦hC​\(c,t\),C\_\{\\mathrm\{fill\},t\}=\\frac\{1\}\{4\}\\sum\_\{c\\in\\mathcal\{K\}\}h\_\{C\}\(c,t\),EA​B,t=12​\(1−14​∑c∈𝒦hA​\(c,t\)\+1−14​∑c∈𝒦hB​\(c,t\)\)\.E\_\{AB,t\}=\\frac\{1\}\{2\}\\left\(1\-\\frac\{1\}\{4\}\\sum\_\{c\\in\\mathcal\{K\}\}h\_\{A\}\(c,t\)\+1\-\\frac\{1\}\{4\}\\sum\_\{c\\in\\mathcal\{K\}\}h\_\{B\}\(c,t\)\\right\)\.The placement score is

Lt=clip\[0,1\]​\(0\.75​Cfill,t\+0\.25​EA​B,t\)\.L\_\{t\}=\\mathrm\{clip\}\_\{\[0,1\]\}\\left\(0\.75\\,C\_\{\\mathrm\{fill\},t\}\+0\.25\\,E\_\{AB,t\}\\right\)\.
The arrangement score evaluates the target stack on containerCC:

Rt=clip\[0,1\]​\(0\.50​g¯t\+16​σxlarge,large,t\+16​σlarge,medium,t\+16​σmedium,small,t\),R\_\{t\}=\\mathrm\{clip\}\_\{\[0,1\]\}\\left\(0\.50\\,\\bar\{g\}\_\{t\}\+\\frac\{1\}\{6\}\\sigma\_\{\\mathrm\{xlarge\},\\mathrm\{large\},t\}\+\\frac\{1\}\{6\}\\sigma\_\{\\mathrm\{large\},\\mathrm\{medium\},t\}\+\\frac\{1\}\{6\}\\sigma\_\{\\mathrm\{medium\},\\mathrm\{small\},t\}\\right\),whereg¯t\\bar\{g\}\_\{t\}is the mean goal\-pose score over the four cubes andσi,j,t\\sigma\_\{i,j,t\}is the continuous score that cubejjis stacked on cubeii\.

The per\-step composite score is

P~t=0\.14​At\+0\.18​Gt\+0\.20​Tt\+0\.20​Lt\+0\.28​Rt\.\\tilde\{P\}\_\{t\}=0\.14\\,A\_\{t\}\+0\.18\\,G\_\{t\}\+0\.20\\,T\_\{t\}\+0\.20\\,L\_\{t\}\+0\.28\\,R\_\{t\}\.Illegal Hanoi placements are penalized by the maximum continuous score for a larger cube being placed on a smaller cube:

Pt=clip\[0,1\]​\(0\.95​P~t​\(1−0\.15​Ut\)\+0\.05​Qint,t\),P\_\{t\}=\\mathrm\{clip\}\_\{\[0,1\]\}\\left\(0\.95\\,\\tilde\{P\}\_\{t\}\(1\-0\.15\\,U\_\{t\}\)\+0\.05\\,Q\_\{\\mathrm\{int\},t\}\\right\),whereUt∈\[0,1\]U\_\{t\}\\in\[0,1\]is the illegal\-stack score andQint,tQ\_\{\\mathrm\{int\},t\}measures interaction smoothness from end\-effector and cube speeds\.

For an episode ending at stepTeT\_\{e\}, the evaluator records the time\-average composite score, the best composite score, and the final arrangement score:

Pauc=1Te​∑t=1TePt,Pbest=max1≤t≤Te⁡Pt,Rfinal=RTe\.P\_\{\\mathrm\{auc\}\}=\\frac\{1\}\{T\_\{e\}\}\\sum\_\{t=1\}^\{T\_\{e\}\}P\_\{t\},\\qquad P\_\{\\mathrm\{best\}\}=\\max\_\{1\\leq t\\leq T\_\{e\}\}P\_\{t\},\\qquad R\_\{\\mathrm\{final\}\}=R\_\{T\_\{e\}\}\.The episode fitness is

fe=clip\[0,1\]​\(0\.45​Pauc\+0\.45​Pbest\+0\.10​Rfinal\)\.f\_\{e\}=\\mathrm\{clip\}\_\{\[0,1\]\}\\left\(0\.45\\,P\_\{\\mathrm\{auc\}\}\+0\.45\\,P\_\{\\mathrm\{best\}\}\+0\.10\\,R\_\{\\mathrm\{final\}\}\\right\)\.If the Tower\-of\-Hanoi success predicate is satisfied,fef\_\{e\}is set to1\.01\.0\.

The feedback metrics returned by the evaluator include maximum and final phase scores for approach, grasp/lift, transport, placement, and arrangement; milestone times for first reach, first lift, first cube inCC, first two cubes inCC, and all cubes inCC; minimum end\-effector–cube distances; maximum lift heights; cube transport speeds; hole scores for containersAA,BB, andCC; target\-fill and source\-emptiness scores; goal\-pose scores; legal stack\-pair scores; drop count; and maximum illegal\-stack score\. These feedback metrics localize failure to reaching, lifting, transport, placement, final stack arrangement, illegal Hanoi placement, or unstable interaction, and are included with scalar fitness when generating subsequent code\-as\-policy proposals\.

#### C\.1\.2AI2\-THOR Household Interaction

For AI2\-THOR, the selected evaluator is defined for the taskapple\_in\_microwave\_on\_bread\_in\_fridge\. The binary success predicate is the same as in Appendix[B\.1](https://arxiv.org/html/2607.22832#A2.SS1):

successe=apple​\_​in​\_​microwave∧microwave​\_​on∧bread​\_​in​\_​fridge\.\\mathrm\{success\}\_\{e\}=\\mathrm\{apple\\\_in\\\_microwave\}\\;\\wedge\\;\\mathrm\{microwave\\\_on\}\\;\\wedge\\;\\mathrm\{bread\\\_in\\\_fridge\}\.The task success indicator is

Se=𝕀​\{successe\},S\_\{e\}=\\mathbb\{I\}\\\{\\mathrm\{success\}\_\{e\}\\\},computed from the environment success predicate\. The evaluator returns a dense episode fitnessfe∈\[0,1\]f\_\{e\}\\in\[0,1\]\. The policy\-level fitness is the mean episode fitness over the rollout budget:

F​\(p\)=1Nroll​∑e=1Nrollfe​\(p\)\.F\(p\)=\\frac\{1\}\{N\_\{\\mathrm\{roll\}\}\}\\sum\_\{e=1\}^\{N\_\{\\mathrm\{roll\}\}\}f\_\{e\}\(p\)\.
At each environment steptt, the evaluator computes a phase\-progress scorePtP\_\{t\}from the current task predicates and accumulated visibility history:

Pt=\\displaystyle P\_\{t\}=\{\}0\.06​papple​\_​seen,t\+0\.06​pbread​\_​seen,t\+0\.08​papple​\_​picked,t\+0\.08​pbread​\_​picked,t\\displaystyle 06\\,p\_\{\\mathrm\{apple\\\_seen\},t\}\+06\\,p\_\{\\mathrm\{bread\\\_seen\},t\}\+08\\,p\_\{\\mathrm\{apple\\\_picked\},t\}\+08\\,p\_\{\\mathrm\{bread\\\_picked\},t\}\+0\.10​pmicrowave​\_​seen,t\+0\.10​pfridge​\_​seen,t\+0\.08​pmicrowave​\_​open,t\+0\.08​pfridge​\_​open,t\\displaystyle\+10\\,p\_\{\\mathrm\{microwave\\\_seen\},t\}\+10\\,p\_\{\\mathrm\{fridge\\\_seen\},t\}\+08\\,p\_\{\\mathrm\{microwave\\\_open\},t\}\+08\\,p\_\{\\mathrm\{fridge\\\_open\},t\}\+0\.16​papple​\_​in​\_​microwave,t\+0\.16​pbread​\_​in​\_​fridge,t\+0\.04​pmicrowave​\_​on,t\.\\displaystyle\+16\\,p\_\{\\mathrm\{apple\\\_in\\\_microwave\},t\}\+16\\,p\_\{\\mathrm\{bread\\\_in\\\_fridge\},t\}\+04\\,p\_\{\\mathrm\{microwave\\\_on\},t\}\.Visibility terms are fractional and depend on whether the object has been observed and, when visible, its distance to the agent\. State\-change terms such as pickup, receptacle opening, object insertion, and toggling are binary once the corresponding predicate is reached\.

For an episode ending at stepTeT\_\{e\}, the evaluator records

Pbest=max0≤t≤Te⁡Pt,Pfinal=PTe\.P\_\{\\mathrm\{best\}\}=\\max\_\{0\\leq t\\leq T\_\{e\}\}P\_\{t\},\\qquad P\_\{\\mathrm\{final\}\}=P\_\{T\_\{e\}\}\.The progress term used in the episode fitness is

P=0\.35​Pbest\+0\.65​Pfinal\.P=0\.35\\,P\_\{\\mathrm\{best\}\}\+0\.65\\,P\_\{\\mathrm\{final\}\}\.Thus,PbestP\_\{\\mathrm\{best\}\}gives credit for the best intermediate task state reached during the rollout, whilePfinalP\_\{\\mathrm\{final\}\}measures the task progress that remains at termination\.

The final episode fitness is

fe=clip\[0,1\]\(\\displaystyle f\_\{e\}=\\mathrm\{clip\}\_\{\[0,1\]\}\\big\(0\.74​P\+0\.14​Qint\+0\.06​Qvis\+0\.06​Qeff\\displaystyle 74\\,P\+14\\,Q\_\{\\mathrm\{int\}\}\+06\\,Q\_\{\\mathrm\{vis\}\}\+06\\,Q\_\{\\mathrm\{eff\}\}−0\.07Cblocked−0\.03Ccarry\)\.\\displaystyle\-07\\,C\_\{\\mathrm\{blocked\}\}\-03\\,C\_\{\\mathrm\{carry\}\}\\big\)\.HereQintQ\_\{\\mathrm\{int\}\}measures valid and task\-relevant object interactions,QvisQ\_\{\\mathrm\{vis\}\}measures visibility of the Apple, Bread, Microwave, and Fridge, andQeffQ\_\{\\mathrm\{eff\}\}rewards earlier progress within the episode horizon\. The penaltyCblockedC\_\{\\mathrm\{blocked\}\}increases with failed movement and long intervals without progress\. The penaltyCcarryC\_\{\\mathrm\{carry\}\}increases when the policy holds the Apple or Bread for many steps without placing it in the required receptacle\.

The evaluator also enforces lower bounds on the milestones offef\_\{e\}\. Any successful Apple or Bread pickup provides at least0\.060\.06fitness\. Observing the Microwave or Fridge gives at least0\.100\.10\. Opening the Microwave or Fridge gives at least0\.180\.18\. Placing either the Apple in the Microwave or the Bread in the Fridge gives at least0\.420\.42\. Completing both placement predicates gives at least0\.720\.72\. Completing both placement predicates and turning on the Microwave gives at least0\.900\.90\. Full task success gives at least0\.980\.98\.

The feedback metrics returned by the evaluator include terminal task predicates, best\- and final\-phase scores, first\-hit times for task milestones, visibility and proximity statistics for all target objects, action success rates, task\-relevant interaction success rates, failed movement rate, and carrying\-stall measures\. The milestone times record when the policy first picks up the Apple or Bread, first observes the Microwave or Fridge, opens each receptacle, places each object in its target receptacle, and toggles the Microwave on\. These feedback metrics identify the unsatisfied AI2\-THOR precondition at the level used by the code\-as\-policy interface: object discovery, valid pickup, receptacle localization, receptacle opening, object placement, Microwave toggling, or premature termination\.

### C\.2Evolved Code\-as\-Policy Programs

We report one successful evolved code\-as\-policy program from each domain to illustrate the executable strategy produced by policy evolution\. Since multiple final policies solve the task, the discussion focuses on the policy structure: task\-state representation, control\-flow logic, closed\-loop execution, and recovery mechanisms\.

#### C\.2\.1Robosuite Franka Tower\-of\-Hanoi

The Franka code\-as\-policy program combines a fixed symbolic Hanoi schedule with guarded closed\-loop execution\. The symbolic component generates the recursive1515\-move solution for four cubes and three containers\. The policy state is

whereiiis the current Hanoi move index,zzis the manipulation state, andrris the retry index\. The move index is advanced only after the corresponding physical placement passes a post\-placement check\.

At each control step, the policy filters the observed end\-effector pose, cube poses, and container poses using separate exponential moving averages\. It then computes the three container hole centers from the current container poses\. For the active Hanoi move, the pick target is the observed pose of the selected cube plus a retry\-dependent planar offset\. The target place is the observed destination hole center with a height determined from the current top surface of the destination stack, excluding the cube being moved\.

The manipulation controller is implemented as a finite\-state machine with pick, grasp\-verification, placement, centering, release, and retreat states\. In the pick stage, the policy moves to a travel height above the selected cube, descends to a pre\-grasp pose, and closes the gripper\. In the grasp\-verification stage, the policy lifts the cube and checks whether it has moved upward from its grasp\-time pose while remaining close to the end effector\. In the placement stage, the policy moves above the destination container and descends only once it is horizontally aligned\. In the centering and release states, the policy centers on the target hole at contact height, then opens the gripper\. After release, the policy checks whether the cube is near the destination hole before advancing the move index\.

Recovery is local to the active Hanoi move\. If grasp verification fails, the gripper is opened, the retry index is incremented, and the same Hanoi move is attempted again with a different planar pick offset\. If the post\-placement check fails, the move index is not advanced, so the same Hanoi move is retried from the updated observation\. Thus, the symbolic Hanoi ordering remains fixed, but the execution of each move is adapted through retry\-conditioned targets\.

The controller also uses contact\-aware motion constraints\. Downward motion is suppressed until the horizontal error is below the corresponding pick or place gate\. Cartesian commands are slew\-limited, and lateral motion is reduced near the stack to avoid disturbing previously placed cubes\. Before release, a centering state applies small lateral corrections until the residual planar error and commanded lateral motion are both small\. The largest cube uses a tighter placement gate and a longer centering window, since its placement forms the base of the final tower\.

#### C\.2\.2AI2\-THOR Household Interaction

The selected AI2\-THOR code\-as\-policy program solves the task by using observation\-conditioned object interactions rather than a fixed action sequence\. The policy maintains memory of object identifiers and parsed object positions for the Apple, Bread, Microwave, and Fridge\. At each step, it selects object identifiers only from the currentobs\["action\_space"\], so pickup, open, put, and toggle actions are issued only when the simulator exposes them as executable\.

Navigation uses the agent grid pose and remembered object positions\. The policy records failedmove\_aheadattempts as blocked grid edges and avoids them in subsequent navigation\. When direct movement toward a target object is obstructed, it switches to an axis\-aligned greedy choice over neighboring grid cells, using blocked\-edge memory and container\-specific bad\-placement cells to avoid repeating failed poses\.

The main domain\-specific recovery mechanism is for receptacle placement\. Ifput\_objectfails at the Microwave or Fridge, the policy marks the current grid cell as a bad placement pose for that receptacle and executes a short pose\-adjustment sequence before retrying the placement from a different viewpoint\. Separate recovery queues are used for Microwave and Fridge placement failures, reflecting the different geometry and visibility constraints of the two receptacles\.

The policy also handles the hidden\-Apple case\. If the Apple is not found during a normal search, the policy probes the Fridge, opens it when a valid Fridge identifier is available, scans the open receptacle with camera actions, and attempts to pick up the Apple from inside\. If the Apple is picked from an open Fridge, the policy closes the Fridge before navigating to the Microwave, avoiding collisions and failed navigation caused by the open door\.