Co-design Gym：具身-策略协同优化的统一基准

# Co-Design Gym: A Unified Benchmark for Embodiment-Policy Co-Optimization
Source: [https://arxiv.org/html/2610.02366](https://arxiv.org/html/2610.02366)
Yordan TsvetkovAffiliation:University of CambridgeLeonard PleissAndrew SpielbergAffiliation:Carnegie Mellon UniversityAffiliation:MorphoAIRika Antonova\*Equal contributionAffiliation:University of CambridgeTechnical University of Munich

###### Abstract

Finding an optimal behaviour policy within a given environment is a widely studied problem in domains as diverse as games, robotics, energy infrastructure, communication networks, and multi\-agent systems\. Numerous benchmarks have been developed to support such research, but the vast majority assume that the agent’s embodiment \(design\) is fixed, focusing instead on policy learning alone\.Liftingthis assumption gives rise to a broader class of problems in which optimizing embodiment and policy separately is highly suboptimal\. An agent’s embodiment strongly shapes which control policies can be discovered, while the optimal embodiment is in turn defined by the policies it admits\. To help the research community study this class of problems explicitly and systematically, we introduceCo\-Design Gym– a suite of benchmark environments for jointly optimizing embodiment and policy\. Our environments span domains such as robotic manipulation and locomotion, multi\-robot cooperation, deformable and soft dynamics, video games, electricity grids, wireless networks, F1 racing, multi\-agent warehouses, and optimal control, offering 20 environment families \(domains\), with over 85 distinct co\-design presets in total\. We further contribute a systematic evaluation of representative co\-design algorithms, characterizing the current state of the art\. Together, these contributions lay the groundwork for cumulative, comparable progress in co\-design\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/figures/fig1_graphic_v7.png)Figure 1:Co\-Design Gym overview with three environment categories\.Native:new domains we developed to study the dynamic interplay between embodiment and policy, covering manipulation, locomotion and deformable body dynamics\.Extended:classic MuJoCo control tasks fromOpenAIGym extended to support morphology variations\.Reframed:existing problems from games,energy, networking, racing, logistics and control, recast as co\-design tasks\.Co\-Design Gym is available athttps://github\.com/camral/codesign\-gym## 1Introduction

An agent’s ability is a function of both itsembodimentand its behavioralpolicy\. An embodiment determines what an agent isable to do: it is an interface for transforming perception into action upon an external environment\. The policy determines what the agentactually does, using a given physical interface\. The process of jointly improving the embodiment and the policy toward performance goals is referred to asco\-design\.

Embodiment and policy are coupled: the quality of an embodiment is the performance of the best policy it can achieve for downstream tasks\. Hence, fully assessing each candidate embodiment requires finding an optimal policy for it on a variety of tasks\. However, running expensive policy optimization to completion before knowing the potential of an embodiment is wasteful\. Tighter co\-optimization is essential, especially when the embodiment and policy subspace structures yield challenging high\-dimensional optimization landscapes\.

The above calls for extending optimization paradigms such as Reinforcement Learning \(RL\), usually framed as singly non\-stationary optimization\. Recognizing co\-design as adoublynon\-stationaryproblem is more suitable, since both the policy and the embodiment can change rapidly\. Finding global or even high\-quality local optima for this extended problem is expensive, and gauging real progress is nontrivial\. To date, no unified benchmark tracks the performance of co\-design methods across a variety of domains, or disentangles progress in embodiment search and policy optimization from progress specific to co\-design\.

Co\-design today stands in a similar place to where RL stood before the introduction of unified benchmarks, when agents still struggled to solve basic tasks\([Mnih et al\., 2015](https://arxiv.org/html/2610.02366#bib.bib64)\)\. Benchmarks and unifying frameworks,e\.g\.OpenAI Gym and Gymnasium\([Brockman et al\., 2016](https://arxiv.org/html/2610.02366#bib.bib39);[Towers et al\., 2024](https://arxiv.org/html/2610.02366#bib.bib57)\), enabled compound progress that accelerated the field\. RL agents have since reached superhuman performance in a wide variety of areas, such as strategic game playing in Go\([Silver et al\., 2016](https://arxiv.org/html/2610.02366#bib.bib61)\)& Dota 2\([Berner et al\., 2019](https://arxiv.org/html/2610.02366#bib.bib62)\), drone racing\([Geles et al\., 2026](https://arxiv.org/html/2610.02366#bib.bib60)\), control of tokamak plasmas for nuclear fusion\([Degrave et al\., 2022](https://arxiv.org/html/2610.02366#bib.bib59)\), and robot locomotion\([Peng et al\., 2017](https://arxiv.org/html/2610.02366#bib.bib41);[Peng et al\., 2018](https://arxiv.org/html/2610.02366#bib.bib42);[Margolis et al\., 2021](https://arxiv.org/html/2610.02366#bib.bib40)\), to name a few\. A unified benchmark for co\-design should spark comparable progress\.

In pursuit of that, we introduceCo\-Design Gym– a unified benchmark for co\-design algorithms\. Our contribution is threefold\. First, we extend the canonical Gymnasium API with an explicit design space, so that any policy\-learning task can be lifted into a co\-design task\. Second, we leverage this generalized API to convert prominent existing policy optimization environments into co\-design tasks\. Finally, we present novel co\-design environments in which embodiment optimization isdecisivefor what the policy can achieve: from reshaping fingertips for precise manipulation of small objects & reshaping a hand for fast in\-hand reorientation, to dynamic ball catching, multi\-robot truck unloading, and evolving soft robots for locomotion\. Overall, we offer 20 environment families and over 85 presets, with accelerated GPU\-based resets that make it practical to evaluate thousands of embodiments in parallel\. Benchmarking representative evolutionary, reinforcement\-learning, hybrid and transformer\-based co\-design methods under matched compute budgets, we find that no single method dominates, and that many tasks remain far from solved\. This leaves substantial headroom for new methods, and we call upon the community to take advantage of this opportunity\.

## 2Background and Related Work

Co\-design problems appear naturally across disciplines in design tasks that feature both a static embodiment and a time\-varying policy component, and in which choices made in each domain can affect choices in the other\. Over the last several decades, methods have been developed for the co\-design of computer systems hardware and software\([De Michell and Gupta, 1997](https://arxiv.org/html/2610.02366#bib.bib1);[Milder et al\., 2012](https://arxiv.org/html/2610.02366#bib.bib2)\), optics and their processing algorithms\([Dowski Jr and Cathey, 1995](https://arxiv.org/html/2610.02366#bib.bib5)\), aeronautical materials and stabilizers\([Salama et al\., 1989](https://arxiv.org/html/2610.02366#bib.bib6)\), and chemical plants and processes\([Luyben and Floudas, 1994](https://arxiv.org/html/2610.02366#bib.bib7)\), just to name a few\. More recently, co\-design of the embodiments and policies of virtual and physical robots has seen growing interest\. This problem is exceptionally relevant now that investment in robotic hardware is at all\-time high: advanced hardware prototyping is finally affordable, thus co\-design solutions don’t have to be confined to simulation\. Robots typically must solve tasks that involve a complex combination of dynamical moves, medium\-to\-long\-term\-planning, a mix of discrete and continuous optimization in both morphological structure and actions all at once, while still enabling fast \(real\-time\) operation of the whole system\. A further challenge is to transfer the success from simulation to reality\. All these aspects make co\-optimization of robot systems particularly challenging, yielding an ideal ground for testing novel co\-design methods\.

Early research built upon the seminal works of[Ventrella \(1994\)](https://arxiv.org/html/2610.02366#bib.bib3)and[Sims \(1994\)](https://arxiv.org/html/2610.02366#bib.bib4)and employed evolutionary algorithms\([Hiller and Lipson, 2011](https://arxiv.org/html/2610.02366#bib.bib8);[Cheney et al\., 2014b](https://arxiv.org/html/2610.02366#bib.bib9);[Cheney et al\., 2014a](https://arxiv.org/html/2610.02366#bib.bib10);[Corucci et al\., 2016](https://arxiv.org/html/2610.02366#bib.bib11)\), a trend that still sees application in modern learning\-based approaches\([Wang et al\., 2019](https://arxiv.org/html/2610.02366#bib.bib12);[Yuan et al\., 2021](https://arxiv.org/html/2610.02366#bib.bib13)\)and are enjoying a renaissance in combination with off\-the\-shelf large models\([Chen et al\., 2026](https://arxiv.org/html/2610.02366#bib.bib29);[Gao et al\., 2026](https://arxiv.org/html/2610.02366#bib.bib30)\)\. While these approaches produce robust specimens\([Bongard, 2011](https://arxiv.org/html/2610.02366#bib.bib14);[Hejna III et al\., 2021](https://arxiv.org/html/2610.02366#bib.bib15)\)their sampling\-based search can be inefficient when simulation time becomes the computational bottleneck\. In such cases, complementary approaches that use gradient\-based search tend to more efficient\([Matthews et al\., 2023](https://arxiv.org/html/2610.02366#bib.bib16)\)\. Given continuous and smoothly differentiable parameterizations of morphology and controllers, these approaches employ unconstrained and constrained optimizers to search over optimal parameterizations of geometry\([Wampler and Popović, 2009](https://arxiv.org/html/2610.02366#bib.bib17);[Coros et al\., 2013](https://arxiv.org/html/2610.02366#bib.bib22);[Spielberg et al\., 2017](https://arxiv.org/html/2610.02366#bib.bib19);[Ha et al\., 2017](https://arxiv.org/html/2610.02366#bib.bib18);[Ma et al\., 2021](https://arxiv.org/html/2610.02366#bib.bib31)\), materials,\([Spielberg et al\., 2023](https://arxiv.org/html/2610.02366#bib.bib20)\), and controllers and observers\([Spielberg et al\., 2019](https://arxiv.org/html/2610.02366#bib.bib21)\)for solving specified tasks\. As the field has begun to mature in the last decade, holistic frameworks have emerged that have driven toward a unifying theory of co\-design\([Censi, 2015](https://arxiv.org/html/2610.02366#bib.bib24);[Zardini, 2023](https://arxiv.org/html/2610.02366#bib.bib23)\)\. Sampling\- and gradient\-based approaches have also begun to merge under the umbrella of generative systems\([Wang et al\., 2023a](https://arxiv.org/html/2610.02366#bib.bib27);[Zhao et al\., 2020](https://arxiv.org/html/2610.02366#bib.bib25);[Xu et al\., 2021](https://arxiv.org/html/2610.02366#bib.bib26);[Dai et al\., 2026](https://arxiv.org/html/2610.02366#bib.bib56);[Ha et al\., 2026](https://arxiv.org/html/2610.02366#bib.bib28)\)\.

Though much of the above work focuses on the algorithmic nature of co\-design algorithms, co\-designed creatures are not confined to the virtual world\. Successful physical demonstrations of co\-designed robots reach back as far as[Park and Asada \(1994\)](https://arxiv.org/html/2610.02366#bib.bib32)and have been applied to form factors such as quadrupeds\([Wang et al\., 2026a](https://arxiv.org/html/2610.02366#bib.bib34)\), grippers\([Xu et al\., 2024](https://arxiv.org/html/2610.02366#bib.bib38);[Fay et al\., 2026](https://arxiv.org/html/2610.02366#bib.bib55)\), full humanoids\([Wang et al\., 2026b](https://arxiv.org/html/2610.02366#bib.bib33)\), and more at large\([Yu et al\., 2026](https://arxiv.org/html/2610.02366#bib.bib37)\)and small scales\([Kriegman et al\., 2020](https://arxiv.org/html/2610.02366#bib.bib35);[Balciunaite et al\., 2026](https://arxiv.org/html/2610.02366#bib.bib36)\)\.

Despite algorithmic progress, evaluation progress has lagged\. Two precedents have made some progress in this regard\. SoftZoo\([Wang et al\., 2023b](https://arxiv.org/html/2610.02366#bib.bib53)\)provides benchmarking for soft robot co\-design algorithms across terrains, materiality, and morphological representations, but lacks physical realism and is limited to 3D elastic dynamics\. EvoGym\([Bhatia et al\., 2022](https://arxiv.org/html/2610.02366#bib.bib54)\)provides an extensive array of tasks and environments and emphasizes complex neural control tasks, but it is also limited to soft robots, and in this case, 2D, low\-fidelity pixel representations\. Further evaluation setups relevant to co\-design exist, but either focus on a narrow set of tasks, or lack realistic environment modeling\. Our work is most inspired by the OpenAI Gym\([Brockman et al\., 2016](https://arxiv.org/html/2610.02366#bib.bib39)\)and its successor, Gymnasium\([Towers et al\., 2024](https://arxiv.org/html/2610.02366#bib.bib57)\), which have been seminal for RL research under a shared framework, but lack the extensibility needed to accommodate search over morphologies and other embodiment aspects\.

## 3An Optimization Perspective on Co\-Design

Co\-design treats embodiment and policy as coupled choices and a single problem\. An embodiment determines aspects of what an agent can do and how it interacts with the environment; a policy determines how those capabilities are used\. The usefulness of an embodiment therefore depends on the policies available to it, while the usefulness of a policy depends on the embodiment through which it acts\. For example, changing a robot’s leg geometry can change which actuations are effective, so evaluating a new geometry using only a controller developed for the previous one may give a misleading picture of its potential\. Hence, these two problems are not independent\.

The termembodimentis not limited to a physical body\. The capabilities of a virtual character or the interfaces available to a software agent can play a similar role\. In each case, design choices are subject to overarching rules and constraints\. Environment structure, tasks, and objectives can also become design variables, but here we focus on embodiment and policy under a shared objective\. It is often useful to select an embodiment before interaction \(the beginning of the episode\) and hold it fixed during execution when the policy acts\. In such cases, we can say that the problem of choosing an embodiment \(design\) and choosing a policy \(actions\) possess a temporal hierarchy, where an embodiment is chosen once, while actions are chosen multiple times within a single episode\.

In order to describe a co\-design problemℐ\\mathcal\{I\}, let𝐄\\mathbf\{E\}be a nonempty set of admissible embodiments\. Each𝐞∈𝐄\\mathbf\{e\}\\in\\mathbf\{E\}gives rise to a control problemM𝐞M\_\{\\mathbf\{e\}\}and a nonempty setΠ𝐞\\Pi\_\{\\mathbf\{e\}\}of policies available to that embodiment\. This set incorporates both what the embodiment can execute and any restrictions imposed on the controller\. We impose no particular form on the policies inΠ𝐞\\Pi\_\{\\mathbf\{e\}\}, beyond the feasibility and controller restrictions specified by the problem\. Their action choices may be probabilistic, history\-dependent, or time\-dependent\. Different embodiments may induce different dynamics, available actions, initial conditions, or information available to the controller\. These details are specified byM𝐞M\_\{\\mathbf\{e\}\}andΠ𝐞\\Pi\_\{\\mathbf\{e\}\}, allowing different embodiments to use different representations\.

Let us useJ⁡\(𝐞,π\)J\(\\mathbf\{e\},\\pi\)to denote the expected performance of policyπ\\piwith embodiment𝐞\\mathbf\{e\}\. For concreteness, in a discrete\-time discounted setting this is

J⁡\(𝐞,π\)=𝔼M𝐞,π​\[∑t=0∞γt​R𝐞​\(St,At\)\],0≤γ<1,J\(\\mathbf\{e\},\\pi\)=\\mathbb\{E\}\_\{M\_\{\\mathbf\{e\}\},\\pi\}\\left\[\\sum\_\{t=0\}^\{\\infty\}\\gamma^\{t\}R\_\{\\mathbf\{e\}\}\(S\_\{t\},A\_\{t\}\)\\right\],\\qquad 0\\leq\\gamma<1,whereR𝐞R\_\{\\mathbf\{e\}\}is the reward associated with the specified task\. We assume that the underlying models and policies define this expectation and that it is finite for every pair considered\. Embodiment\-dependent rewards can include design costs or other effects as long as their values remain comparable under the same objective\. The argument below also applies to other well\-defined scalar performance criteria\.

The feasible choices are the embodiment\-policy pairs

ℱ=\{\(𝐞,π\):𝐞∈𝐄,π∈Π𝐞\}\.\\mathcal\{F\}=\\\{\(\\mathbf\{e\},\\pi\):\\mathbf\{e\}\\in\\mathbf\{E\},\\ \\pi\\in\\Pi\_\{\\mathbf\{e\}\}\\\}\.This set is not necessarily a Cartesian product: a policy executable with one embodiment may be unavailable with another\. It is nevertheless useful to picture performance over these pairs as a joint optimization landscape\. This picture is schematic; neither the embodiment choices nor the policy classes need to form a continuous or finite\-dimensional space and often they may not\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/figures/problem_formulation_fixed.png)Figure 2:An illustrative embodiment\-policy optimization landscape\. The axes𝚷\\mathbf\{\\Pi\}and𝐄\\mathbf\{E\}represent policy and embodiment spaces; surface height represents expected return\. The panels𝐞α\\mathbf\{e\}\_\{\\alpha\}and𝐞β\\mathbf\{e\}\_\{\\beta\}label two fixed\-embodiment slices\. EachJ⋆​\(𝐞\)J^\{\\star\}\(\\mathbf\{e\}\)refers to optimization over only its panel’s embodiments\. In this example, the green slice contains a joint optimum, givingJ⋆​\(𝐞\)=J⋆⁣⋆​\(ℐ\)J^\{\\star\}\(\\mathbf\{e\}\)=J^\{\\star\\star\}\(\\mathcal\{I\}\)\.Figure[2](https://arxiv.org/html/2610.02366#S3.F2)illustrates the interaction between embodiment and policy\. Each colored slice holds an embodiment fixed while allowing the policy to vary\. Changing the embodiment changes the slice, potentially changing both the best available return and the policy that achieves it\. An embodiment’s potential therefore depends on the policies it enables\.

For a fixed embodiment𝐞¯\\bar\{\\mathbf\{e\}\}, policy optimization searches the corresponding slice\. Its best achievable value, understood as a supremum, is

J⋆​\(𝐞¯\)=supπ∈Π𝐞¯J⁡\(𝐞¯,π\)\.J^\{\\star\}\(\\bar\{\\mathbf\{e\}\}\)=\\sup\_\{\\pi\\in\\Pi\_\{\\bar\{\\mathbf\{e\}\}\}\}J\(\\bar\{\\mathbf\{e\}\},\\pi\)\.Co\-design also searches across embodiments, giving

J⋆⁣⋆​\(ℐ\)=sup\(𝐞,π\)∈ℱJ⁡\(𝐞,π\)=sup𝐞∈𝐄J⋆​\(𝐞\)\.J^\{\\star\\star\}\(\\mathcal\{I\}\)=\\sup\_\{\(\\mathbf\{e\},\\pi\)\\in\\mathcal\{F\}\}J\(\\mathbf\{e\},\\pi\)=\\sup\_\{\\mathbf\{e\}\\in\\mathbf\{E\}\}J^\{\\star\}\(\\mathbf\{e\}\)\.The nested expression assesses each embodiment through the best performance available with its policies\. It motivates approaches that optimize policies within a search over embodiments, although co\-design algorithms are not restricted to this organization\. We use suprema because a best policy or embodiment\-policy pair need not be attained\.

In the illustrated example, the highest point on the red slice lies below the highest point on the green slice, thus only the latter slice contains a joint optimum\. Policy optimization within the red slice cannot close this difference: doing so requires another embodiment together with a suitable policy\. The*design gap*is then the difference between a fixed\-embodiment performance ceiling and a global optimum\. When both supremal values are finite, we have

gap⁡\(𝐞¯,ℐ\)=J⋆⁣⋆​\(ℐ\)−J⋆​\(𝐞¯\)≥0\.\\mathrm\{gap\}\(\\bar\{\\mathbf\{e\}\};\\mathcal\{I\}\)=J^\{\\star\\star\}\(\\mathcal\{I\}\)\-J^\{\\star\}\(\\bar\{\\mathbf\{e\}\}\)\\geq 0\.The gap is nonnegative because every pair available with𝐞¯\\bar\{\\mathbf\{e\}\}remains available in the joint problem\. A positive gap means that fixing this embodiment excludes some pairs \(with alternative embodiments\) whose returns exceed its performance ceiling\. A zero gap means that fixing the given embodiment still allows to attain a global optimum\. Thus, in the figure, the red embodiment has a positive design gap and the green embodiment has zero gap\.

This distinction separates two reasons why a learned policy may fall short of the joint performance ceiling\. It may not yet use its embodiment effectively, leaving room for improvement within its slice, or the slice itself may also have a lower ceiling than the joint problem\. Improved policy optimization can address the first shortfall, but cannot remove the second while the embodiment, available policy class, and objective remain fixed\. The design gap captures this remaining opportunity after accounting for the best policies available to the fixed embodiment\.

The same reasoning applies to other restrictions of the joint problem\. Restricting embodiments, policies, or both gives a subproblem with a smaller feasible set\. For any nonemptyℱ′⊆ℱ\\mathcal\{F\}^\{\\prime\}\\subseteq\\mathcal\{F\},

sup\(𝐞,π\)∈ℱ′J⁡\(𝐞,π\)≤sup\(𝐞,π\)∈ℱJ⁡\(𝐞,π\),\\sup\_\{\(\\mathbf\{e\},\\pi\)\\in\\mathcal\{F\}^\{\\prime\}\}J\(\\mathbf\{e\},\\pi\)\\leq\\sup\_\{\(\\mathbf\{e\},\\pi\)\\in\\mathcal\{F\}\}J\(\\mathbf\{e\},\\pi\),provided each retained pair is evaluated in exactly the same way\. Every choice in the restricted problem remains available in the larger one\. If rewards, initial conditions, or other evaluation conditions change for retained pairs, feasible\-set inclusion alone no longer establishes this comparison\.

These comparisons describe available performance, rather than what a particular algorithm will find\. A positive design gap motivates searching over embodiments, but the additional search may require more data or computation\. Under a limited budget, optimizing a policy for a fixed embodiment can still produce a better result than searching jointly over embodiments and policies\.

Determining whether a design gap exists also requires looking beyond a single learning curve\. A plateau at a fixed embodiment establishes neither that its policy has reached the slice’s ceiling nor that other embodiments offer no improvement\. Assessing the gap requires comparisons across embodiments or additional knowledge connecting their behavior, such as a model of how design affects the dynamics\. Benchmarks with explicit design choices allow these comparisons to be studied\. Their measured gains, however, reflect both the embodiments considered and how effectively their policies were optimized; they need not equal the theoretic design gap, which compares supremal returns\.

There is also an important distinction between this explicit separation of design and control and the generality of MDPs\. An MDP can itself represent embodiment choices through actions and state variables\. Our formulation exposes those choices as a separate optimization variable so that their interaction with policy can be discussed directly\. In this sense,policy optimization with an externally fixed embodiment is a restricted case of co\-design, recovered by takingE=\{e¯\}E=\\\{\\bar\{e\}\\\}, and can be considered a subproblem in our discussion, as it admits a contained feasible set\. The perspective helps identify which design choices a problem leaves open, which it fixes in advance, and how those decisions shape the policies and performance available to the agent\.

While fundamentally a part of a joint problem, embodiment and policy can be viewed as residing in distinct subspaces\. This is useful for creating optimizers with different properties for these subspaces, incorporating structural priors, or enabling hierarchy\. For instance, sampling from the policy subspace \(e\.g\. realtime action\) may be a lot faster than from the embodiment subspace \(e\.g\. making prototypes of novel hardware\)\. The former can benefit from a scalable optimizer that performs best when ample data is available, while the latter requires data\-efficient methods\. Hierarchical considerations can be accommodated as well\. As discussed above, temporal hierarchy can be respected by selecting embodiment first in each optimization trial, then rolling out policy actions\. Hence, the above co\-design formulation treats optimizing embodiment and policy as a joint problem, while still providing enough structure to accommodate their differences\.

Overall,Co\-Design Gymprovides a unified interface to connect policy and design optimization, offering the broader research community a view on co\-design as a general doubly non\-stationary optimization problem\.

## 4Co\-Design Gym

![Refer to caption](https://arxiv.org/html/2610.02366v1/figures/env_overview_v7.png)Figure 3:Overview of the available environments\. For additional illustrations of the environment presets, refer to[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1)\.Co\-design gym combines environments that span a wide range of domains, including but not limited to robotics, video games, and grid management\. Our environments include ‘from\-scratch’ setups that natively support efficient and flexible embodiment changes, co\-design extensions of the popular and OpenAI MuJoCo Control tasks ported to MuJoCo Warp, and lifts of existing RL environments from various fields as scalable co\-design problems\.

Each environment pairs an inner policy optimization loop with an outer embodiment problem, and ships with several presets of varying complexity\. All rigid\-body robotics tasks are built in MuJoCo\([Todorov et al\., 2012](https://arxiv.org/html/2610.02366#bib.bib43)\)and MuJoCo Warp\([DeepMind, 2025](https://arxiv.org/html/2610.02366#bib.bib44);[Macklin, 2022](https://arxiv.org/html/2610.02366#bib.bib45)\)\. Most of our environments are natively GPU\-accelerated\. Below is a brief summary of the main environment families\. For full descriptions, reward definitions, and additional presets, please refer to Appendix[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1)\.

Ball Catching\.A robot must catch a thrown ball in a glove while remaining upright\. The policy controls the robot, and the embodiment varies its limbs and torso\. Three presets modify the throw: fast and flat, high arc, and varied 3D fielding: testing co\-design under dynamic conditions inspired by[Morgan et al\. \(2026\)](https://arxiv.org/html/2610.02366#bib.bib63)\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/panel_cleaner_v6.png)Figure 4:Solar Panel Cleaning environment\.Solar Panel Cleaning\.A quadruped robot with a roller\-tipped arm must clean solar panels without any other body part touching them\. The policy controls body and arm, and the embodiment modifies all limbs\. Presets cover a variety of solar panel layouts and terrains: targeting practical deployments of co\-designed locomoting robots for intricate manipulation scenarios\.

Truck Unloading\.A mobile manipulator unloads boxes from a truck onto a conveyor, where another sorts and shelves or stacks them by colour\. A single policy controls all robots, and the embodiment modifies the arms of each robot\. Presets vary the number of unloaders and different box colours: presenting a coordination challenge at the embodiment and control levels\.

Nero\-LEAP Grasping\.A LEAP hand on a Nero arm must stably lift small tools and hardware from a table\. The policy controls the hand, and the embodiment replaces the thumb and index fingertip meshes\. Presets cover different groups of objects: testing high\-dimensional mesh\-based shape design coupled with fine, contact\-rich interactions\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/rotate_in_hand_v1.png)Figure 5:The MorphHand\. Colors show elevation at each vertex\.MorphHand Pick\-Up & Rotation\.A four\-fingered hand must either reorient an object in\-hand or pick up and pose a cube, ball, or hammer\. Supporting transition\-level embodiment changes, these environments enable the use of a new class of design optimization algorithms for complex single\-hand manipulation\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/soft_robot_v2.png)Figure 6:Soft Walkers environment\.Soft Walkers\.Following[Bhatia et al\. \(2022\)](https://arxiv.org/html/2610.02366#bib.bib54)and[Cheney et al\. \(2014b\)](https://arxiv.org/html/2610.02366#bib.bib9), a soft robot must traverse flat, stepped, or gapped terrain\. Each terrain is paired with five 2D/3D spring, MPM, beam, and FEM solvers \(15 presets\)\. Co\-designing soft robots presents a unique challenge due to their deformable bodies and the large and varied design space spanning shape, mass, material, and actuator properties\.

Car Racing\.A scaled F1Tenth car\([O’Kelly et al\., 2020](https://arxiv.org/html/2610.02366#bib.bib50)\)must minimise lap time on the Levine or Spielberg track\. The policy drives the car\. The embodiment sets the vehicle setup, such as tyre grip, cornering stiffness, and drive power, and each parameter trades off against the others\.

Linear\-Quadratic Control\.A mass\-spring\-damper chain must be regulated against disturbances\. The embodiment includes the masses, stiffnesses, damping, and actuator and sensor placement\. Three presets cover continuous design, combinatorial placement under noise, and budget\-constrained design with saturated inputs, so the environment serves as a lightweight sanity check for co\-design algorithms\.

Electrical Microgrid\.A microgrid must meet the randomised demand of one or more buildings over 24–72 hours, either grid\-connected or off\-grid\([Henri et al\., 2020](https://arxiv.org/html/2610.02366#bib.bib48)\)\. The policy dispatches energy, and the embodiment sizes solar, battery, and diesel capacity\. Design costs are charged in the reward, making this a budgeted problem\. A variety of presets test the idiosyncratic problem of grid control with careful resource allocation\.

Wireless Network\.Base stations must give strong, consistent coverage to moving mobile users\([Schneider et al\., 2022](https://arxiv.org/html/2610.02366#bib.bib49)\)\. The policy assigns users to stations, and the embodiment includes the number, placement, and transmit power of the stations\. Presets range from slow urban pedestrians to sparse, high\-speed motorway traffic: combining the challenges of control in networking with geographic layouts and resource allocation\.

Robotic Warehouse\.A robot fleet fetches requested shelves and delivers them to goals, with the aim of maximising throughput\([Bonnet et al\., 2024](https://arxiv.org/html/2610.02366#bib.bib51);[Christianos, 2020](https://arxiv.org/html/2610.02366#bib.bib52)\)\. The policy controls the fleet, and the embodiment lays out shelves on a fixed floor, trading route length against congestion\. Presets scale from 2 to 8 robots: representing the challenges of finding optimal layout patterns for fleets of uniform agents\.

Pokemon Showdown\.The agent must win single battles, or a gauntlet of five League opponents, in Pokemon Showdown\([University, 2011](https://arxiv.org/html/2610.02366#bib.bib46);[Sahovic, 2019](https://arxiv.org/html/2610.02366#bib.bib47)\)\. The policy chooses moves, and the embodiment is the six\-Pokemon team\. Opponents range from random players to the League teams of all nine game generations, with or without healing between battles\. With a combinatorially explosive discrete design space, and intricate environment dynamics, this tests sparse design space navigation in addition to complex environment rule\-learning\.

### 4\.1API

To facilitate ease of use we keep API changes minimal relative to the widely used Gymnasium\([Towers et al\., 2024](https://arxiv.org/html/2610.02366#bib.bib57)\)API\. That way, existing control implementations remain usable and co\-design methods differ only where necessary\.

Any Gymnasium environment becomes a co\-design environment by declaring anembodiment\_space, a map from𝐞∈𝐄\\mathbf\{e\}\\in\\mathbf\{E\}to a realised body, and by forwarding the new argument throughreset; the dynamics, reward, and termination logic are untouched\. The original environment is recovered exactly as the base embodiment𝐞0\\mathbf\{e\}\_\{0\}, so the conversion adds designs without altering the original task\.

Theembodiment\_spaceis a GymnasiumSpacerepresenting𝐄\\mathbf\{E\}\(see Section[3](https://arxiv.org/html/2610.02366#S3)\), alongside the usualobservation\_spaceandaction\_space\. It supportssample\(\)andcontains\(\)with the standard semantics, so a design space can be enumerated, sampled from, or checked for membership by the same machinery used for actions\.

Each environment declares a*base embodiment*𝐞0∈𝐄\\mathbf\{e\}\_\{0\}\\in\\mathbf\{E\}: the conventionally chosen body, with which it is realised until some other design is requested\. PassingNoneretains the embodiment currently in force, which for an environment never given an explicit one is𝐞0\\mathbf\{e\}\_\{0\}throughout; passing𝐞∈𝐄\\mathbf\{e\}\\in\\mathbf\{E\}requests that the agent be realised as𝐞\\mathbf\{e\}\. An environment called withembodiment=Noneeverywhere is therefore a standard Gymnasium environment with embodiment𝐞0\\mathbf\{e\}\_\{0\}\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/api_fig_short.png)Figure 7:Minimal usage ofcodesign\_gym\. A single entry point,make\_vec, returns a natively parallelized environment together with its base embodiment supporting GPU\-acceleration\. An outer design optimizer proposes an embodiment that is passed toreset, and an inner policy then acts against it\. The embodiment can optionally be re\-proposed and passed tostepat every timestep\.##### Within\-task modifications

Our API admits the embodiment argument instepas well asreset\. This choice allows the embodiment to change during an episode\. This is not physically meaningful for many tasks—bodies do not deform on command mid\-trajectory—but it is potentially useful during training, where it permits a single rollout to experiment with multiple designs\. Because within\-episode modification is physically infeasible for many tasks, results obtained with it are not directly interpretable as task performance\. A method that exploits it during evaluation would report returns no physical system could achieve\. Each environment therefore carries anis\_eval\_envflag, fixed at construction,env = CoDesignEnv\(\.\.\., is\_eval\_env=True\)\. If true, changes that alter the agent’s embodiment take effect only when supplied toreset\.

## 5Experiments with Representative Optimization Methods

To demonstrate our environments, we benchmark four co\-design algorithms on them\. Existing research in co\-design has not had a unified interface/setup and therefore makes state\-of\-the\-art algorithms difficult to compare\. Our selection of baselines represents different frontiers of co\-design algorithms from evolutionary approaches and reinforcement learning \(RL\) to transformer\-based search and hybrid approaches\.

We compare against four baselines covering evolutionary, reinforcement\-learning, alternating, and generative approaches to co\-design\. For all baselines, every 5% of training the current best embodiment and controller are evaluated in parallel environments on held\-out seeds\.

CMA\-ES\.CMA\-ES searches over controller and embodiment parameters jointly as a single vector\. The controller parameters are the weights of a neural\-network policy\.

FastTD3\([Seo et al\., 2025](https://arxiv.org/html/2610.02366#bib.bib68)\)\.We add the embodiment parameters to the action space of FastTD3, an off\-policy RL algorithm for continuous control\. The embodiment actions are applied at the first step of each rollout and the control actions at every later step\. Unused action components are zeroed in the replay buffer, so the policy learns only from actions that were actually taken\.

PPO\+NGOpt\([Schulman et al\., 2017](https://arxiv.org/html/2610.02366#bib.bib70);[Rapin and Teytaud, 2018](https://arxiv.org/html/2610.02366#bib.bib69)\)PPO trains the controller\. After each PPO update, NGOpt proposes one embodiment per training environment and scores it by its mean return in the rollouts where it was used\.

LOKI\.We generalise[Jeon et al\. \(2026\)](https://arxiv.org/html/2610.02366#bib.bib58)to arbitrary embodiment parametrisations by replacing its robot\-centric tokenization with a naive one\. We pre\-train its Transformer VAE on 500k randomly sampled valid embodiments\. We then run the original online loop with PPO\-trained Transformer controllers, using 6 clusters instead of 40 for comparability with the other baselines\. At evaluation, the controller of the best cluster is paired with that cluster’s top embodiments\.

For a detailed description of our baselines, we refer to Appendix[B](https://arxiv.org/html/2610.02366#A2)\.

We evaluate each of these methods on our co\-design gym tasks, reporting performance across evaluation environments, embodiment rejection rates, and training behaviour\. Each baseline and environment are trained under matched hardware conditions with 4\-hour wall\-clock time budgets other than LOKI\. For comparability, we perform online training for LOKI for 3\-hours, as an average of 1\-hour was consumed in data generation, VAE pre\-training, and cluster generation\. Results are reported across five seeds\.

Table 1:The Co\-Design Gym’s first scoreboard: Final evaluation return \(mean±\\pmstd over 5 seeds; higher is better\) of each baseline under a matched 4\-hour wall\-clock budget \(LOKI: 3 hours online after 1 hour of preparation\)\. FastTD3 on NeroGraspAll: every proposed design failed structural validation, so every episode received the rejection reward \(−250\-250\)\. Bold: best mean\.Figure 8:Leading baseline over trainingon the 16 presets run with all four baselines\. Each cell names the method with the highest mean eval return \(5 seeds\) at that point, using the latest evaluation so far\. CMA\-ES leads early on 9 presets but at the end on only 4; FastTD3 ends ahead on 6, PPO\+NGOpt and LOKI on 3 each\. The leader changes at least once on 12 of 16 presets, and no method leads on every domain group, so the ranking of co\-design baselines depends on both the task and the budget\.Contained Study: LLM Co\-Design\.Large language models \(LLMs\) have shown strong capabilities across a wide range of domains\([Minaee et al\., 2024](https://arxiv.org/html/2610.02366#bib.bib65)\), which raises the question of whether they can act as co\-design optimizers\. We examine this in a contained study in which an LLM \(Gemma 4 31B\) proposes robot embodiments in the outer loop while PPO learns the controller in the inner loop\. Because this setup mirrors PPO\+NGOpt with the LLM in place of NGOpt, it serves as an ablation of the outer\-loop design optimizer\. Interestingly, PPO\+LLM stays near zero return throughout training, even though its prompt provides additional task\-relevant information to the model\. We refer to Appendix[D](https://arxiv.org/html/2610.02366#A4)for a detailed description\.

## 6Discussion and Conclusion

No single dominating co\-design method\.Across the 18 presets on which all four baselines were run, no method leads, and the emerged ranking depends on both the task and the budget \(Figure[8](https://arxiv.org/html/2610.02366#S5.F8)\)\. The final winners also cluster by domain\. CMA\-ES performs best on most of the extended locomotion tasks, FastTD3 on the ball\-catching, racing and most microgrid presets, and PPO\+NGOpt on most of the warehouse, wireless and Pokémon presets\. Across the variety of domains represented in our benchmark, none of the evaluated co\-design methods can claim general capability\. This is made visible by the presentation of our unified benchmark; and fragmented evaluation settings of prior work would not reveal this\.

Substantial headroom remains\.Many presets are far from solved on which no co\-design algorithm approached the return ceiling\. For instance, on BallCatcher\-Pitch the best method completes only 16\.6% of catches\. Additionally, all methods score near zero on more challenging benchmarks such as SolarCleaner, WarehouseMedium and WarehouseCongested\. The benchmark leaves ample room for methods that explore the co\-optimization of embodiment and policy more efficiently and thoroughly than current approaches across continuous and discrete tasks\. We hope that this encourages exciting new development of co\-design algorithms to push the frontier on many fun, practical, and challenging domains\.

Limitations and future work\.For many proposed environments, the design space does not encompass all conceivable degrees of freedom; further extensions are possible, leaving room for richer parametrizations\. Moreover, due to the breadth of the proposed benchmark and the generality of our underlying simulators, our baselines cover only a subset of the available presets\. Furthermore, the fixed wall\-clock budget in our experiments favors faster implementations, so part of the performance gap between baselines may reflect engineering rather than algorithmic differences\. Finally, achieving high throughput with embodiment changes in parallelized, GPU\-accelerated physics simulation represents a substantial amount of work in structuring the compilation, caching, and workload scheduling\. Incorporating more complex phenomena, such as fluids, wind, or smoke, may require integrating new simulators, with comparably large engineering effort\.

Conclusion\.We presentedCo\-Design Gym, a unified benchmark for the joint optimization of embodiment and policy\. In our work we formalized co\-design problems and solutions focusing on pairs of design spaces and policy classes, and situated standard RL as a special case of co\-design with a single admissible design\. We realized this formalism as an extension of the Gymnasium API, and used it to build on a large variety of widely used benchmark environments as novel co\-design problems\. In addition, we built a number of new environments from scratch specifically to stress\-test the interplay between the embodiment and policy optimization, native and targeted to co\-design\. Finally, our experiments tested representative co\-design methods and analyzed their performance\. The appendix complements these results with full environment specifications and additional presets \(Appendix A\), baseline implementation details \(Appendix B\), per\-preset learning curves and final designs \(Appendix C\), a contained study in which an LLM given privileged task knowledge fails as an outer\-loop designer \(Appendix D\), an analysis of the design gap that fixed\-design methods cannot measure \(Appendix E\), and a detailed illustration of the API \(Appendix F\)\. Unified benchmarks helped carry RL from struggling on basic tasks to superhuman performance\. We hope Co\-Design Gym can play the same role for co\-design, advancing towards its broader promise: optimizing every dimension of a problem jointly to solve it as effectively as possible\.

## Acknowledgements and Funding Statement

The authors would like to acknowledge and thank Austin Yang, Hantao Zhong, Andy Zhou, Allan Zhao, and Ferran Hernandez Caralt for their insightful discussions\. We would further like to thank Taylor Whatley and Ruth Lu for assistance with reviewing code and preliminary testing\.

This project is funded by the Advanced Research \+ Invention Agency \(ARIA\) as part of its Robot Dexterity programme under grant numbers SMRB\-P01\-P20 and SMRB\-P01\-P14\.

## Appendix A: Environments

Unless stated otherwise, the tasks described below use MuJoCo\([Todorov et al\., 2012](https://arxiv.org/html/2610.02366#bib.bib43)\)and MuJoCo Warp\([DeepMind, 2025](https://arxiv.org/html/2610.02366#bib.bib44);[Macklin, 2022](https://arxiv.org/html/2610.02366#bib.bib45)\)as the physics engine\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/ball_catcher_v3.png)Figure 9:Ball Catching environment\.Ball Catching\.In this environment a robot inspired by Rocky from Project Hail Mary\([Weir, 2021](https://arxiv.org/html/2610.02366#bib.bib66);[Lord and Miller, 2026](https://arxiv.org/html/2610.02366#bib.bib67)\)is tasked with catching a ball that is thrown towards it\. At the end of the arm a ”glove” is attached, in which the ball must be caught\. The source of the ball is fixed but the target and speed are randomised\. We provide three presets\. 1\) BallCatcherPitch in which the ball is thrown fast, flat, and low\. 2\) BallCatcherArc in which the ball is thrown in a slower, high arc descending towards the quadruped\. 3\) BallCatcherFielding in which the ball is thrown with varied speed and direction toward a broad 3D catch region requiring versatility\. In these tasks, the control optimization loop is training a policy for the arm\-equipped quadruped, and the design problem is the configuration of the arm\. Partial rewards are obtained for catching and dropping the ball, but full reward \(10\) is accomplished when catching, retaining the ball, and with all three legs touching the ground\. This final condition is required to prevent unstable jumping policies\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/panel_cleaner_v6.png)Figure 10:Solar Panel Cleaning environment\.Solar Panel Cleaning\.In this environment a quadruped is equipped with an arm and tasked with carefully cleaning an assortment of solar panels\. At the end of the arm is a small roller which cleans a solar panel cell when in contact\. Since panels are delicate, no other part of the robot may come in contact with the panel cells\. We provide three presets\. 1\) SolarCleanerReach in which three panels are mounted higher up than the base configuration of the robot and laid out in a line\. 2\) SolarCleanerHillClimb in which four panels are distributed across a rocky and uneven terrain\. 3\) SolarCleanerTraverse in which a 3x4 grid of solar panels is tightly aligned forming narrow aisles for robot traversal\. In these tasks, the control optimization loop is training a policy to control the Ant robot and arm\. The design problem is to modify the legs and the arms of the robot to enable stable and careful locomotion alongside precise arm control\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/multi_agent_robot_v1.png)Figure 11:Truck Unloading environment\.Truck Unloading\.In this task mobile manipulator robots are tasked with unloading a truck packed with boxes\. One mobile manipulator collects boxes from the truck and places them on a conveyor belt, around which other robots are placed\. These must collect different boxes from the conveyor \(keyed by color\) and place them in a specified area\. We provide two presets\. 1\) TruckUnloadSingle in which there is a single mobile manipulator collecting the red boxes from the conveyor belt, and placing them on a shelf\. 2\) TruckUnloadMulti in which there is an additional mobile manipulator collecting the blue boxes from the conveyor, and stack these boxes into an area behind them\. In these tasks, the control optimization loop is training a single policy to control all of the robots\. The design problem is to modify the robot arms on the mobile manipulators to enable dexterous and quick unloading and packing of boxes\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/nerograsp_v7.png)Figure 12:Nero\-LEAP Grasping environment\.Nero\-LEAP Grasping\.A Nero robot arm with a LEAP hand is tasked with picking up small objects from a flat surface\. The LEAP hand has fingertips affixed\. The objects are randomized from a set of ten divided into three categories\. A success in this task is considered a sustained, controlled, and stable lift of the object\. We provide four main presets\. 1\) NeroGraspSmall in which the object is randomly selected from the ‘small‘ category\. 2\) NeroGraspTiny in which the object is randomly selected from the ‘tiny‘ category\. 3\) NeroGraspThin in which the object is a thin washer whose radius is randomly selected\. 4\) NeroGraspAll in which the object is randomly selected from all ten available objects\. There are additional presets for each object, which are further described in Appendix[A\.1](https://arxiv.org/html/2610.02366#Ax1.SS1)\. The trajectory of the Nero arm is fixed, and the control optimization loop is training a policy for the LEAP hand\. The design problem in this task is to modify the meshes of the LEAP fingertips on the thumb and index, to enable a stable and sustained lift\. The base vertices of the mesh are fixed to the robot, but the remainder of the mesh can be fully replaced by the user\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/rotate_in_hand_v1.png)Figure 13:MorphHand Rotation environment\.MorphHand Rotation\.In this environment an anthropomorphic robot hand is tasked with reorienting and rotating an object in its palm to a target orientation relative to the palm\. We provide a single preset in which an 8 cm cube is used, the goal tolerance is 0\.1 rad and new targets are generated 1 radian away from a met target\. The control optimization loop trains a policy for the hand and wrist, while the design problem is the modify the surface of the hand through height\-field displacements\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/hand_pick_up_v4.png)Figure 14:MorphHand Pick\-Up environment\.MorphHand Pick\-Up\.In this environment an anthropomorphic hand is tasked to pick up an object from a flat surface and place it in a certain position and orientation\. We provide three presets\. 1\) MorphHandPickUp\-Cube in which the hand must grasp an 8 cm cube\. 2\) MorphHandPickUp\-SquashBall in which the hand must grasp a 4 cm ball\. 3\) MorphHandPickUp\-Hammer in which the hand must grasp a hammer\. The control optimization loop is training a policy to control the hand and wrist, and the design problem is the modify the surface of the hand through height\-field displacements and finger configuration parameters \(spacing and width\)\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/soft_robot_v2.png)Figure 15:Soft Walkers environment\.Soft Walkers\.In this task a soft robot is tasked with efficiently traversing a given domain\. We provide three terrain presets: 1\) a flat terrain; 2\) a terrain with equidistant stairs; 3\) a terrain with gaps and holes\. For each of these three terrains, we provide five versions of the problem each supported by GPU\-accelerated implicit Euler integration: 1\) in 2D using a Spring\-solver\([Bhatia et al\., 2022](https://arxiv.org/html/2610.02366#bib.bib54)\); 2\) in 2D using an Material Point Method \(MPM\)\-solver\([Spielberg et al\., 2023](https://arxiv.org/html/2610.02366#bib.bib20)\); 3\) in 3D using an MPM\-solver\([Spielberg et al\., 2023](https://arxiv.org/html/2610.02366#bib.bib20)\); 4\) in 3D using a Beam\-solver\([Cheney et al\., 2014b](https://arxiv.org/html/2610.02366#bib.bib9)\); 5\) in 3D using an Finite Element Method \(FEM\)\-solver\([Ma et al\., 2021](https://arxiv.org/html/2610.02366#bib.bib31)\)\. This leads to a total of 15 unique presets\. In these tasks, the control optimization loop trains a policy to control the actuation of the soft robot walker\. The design problem specifies the shape, mass, actuation, and material properties of the robot\. This task was inspired by[Bhatia et al\. \(2022\)](https://arxiv.org/html/2610.02366#bib.bib54)and[Cheney et al\. \(2014b\)](https://arxiv.org/html/2610.02366#bib.bib9), and is built on NVIDIA Warp[Macklin \(2022\)](https://arxiv.org/html/2610.02366#bib.bib45)\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/pokenv.png)Figure 16:The Pokemon Showdown co\-design environment\.Pokemon Showdown\.In this task a Pokemon battle is conducted using a team of six Pokemon tasked with winning the battle or sequence of battles against an opponent trainer\. We provide four main presets\. 1\) PokenvRandom in which the opponent’s Pokemon team is randomly sampled and the opponent agent takes random moves in a Pokemon Generation 4 Single Battle\. 2\) PokenvMaxPower in which the opponent’s Pokemon team is randomly sampled and the opponent agent chooses moves with the highest base power in a Pokemon Generation 6 Single Battle\. 3\) PokenvHeuristic in which the opponent’s Pokemon team is randomly sampled and the opponent agent plays according to the SimpleHeuristicPlayer\([Sahovic, 2019](https://arxiv.org/html/2610.02366#bib.bib47)\)in a Pokemon Generation 9 Single Battle\. 4\) PokenvLeague in which the opponent is a sequence of 5\-opponents representing the Pokemon League, an elite set of trainers from the video games\. This preset has a number of variants, which we discuss in more detail in Appendix[A\.1](https://arxiv.org/html/2610.02366#Ax1.SS1)\. In this task, the control optimization loop is tasked with finding the best policy to play with a given Pokemon team, and the design problem is to build the optimal Pokemon team\. This task is built on Pokemon Showdown\([University, 2011](https://arxiv.org/html/2610.02366#bib.bib46)\)and Poke\-env\([Sahovic, 2019](https://arxiv.org/html/2610.02366#bib.bib47)\)\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/microgrid.png)Figure 17:Electrical Microgrid environment\.Electrical Microgrid\.In this budgeted optimization task an energy Microgrid is supplying power to a small collection of buildings, each with their own power demands and schedules which are randomised\. A success is considered when there is no unmet demand and there was little unnecessary spending over a given time frame\. We provide three presets where the Microgrid is connected to a larger grid\. 1\) MicrogridRandom \(24 hours\) in which there is a single random building connected to the grid\. 2\) MicrogridForecast \(24 hours\) in which there is a single random building connected to the grid and observations are augmented with noisy 12 hour forecasts\. 3\) MicrogridCampus \(24 hours\) in which there are several buildings\. We provide a further two presets where the Microgrid is not connected to a larger grid\. 1\) MicrogridOffGridSingle \(72 hours\) in which there is a single random building connected to the grid\. 2\) MicrogridOffGridCampus \(72 hours\) in which there are several buildings connected the grid\. In these tasks, the control optimization loop is training a policy for energy flow, and the design problem specifies solar capacity, battery size, and diesel generator capacities\. Each design choice incurs a cost which is paid in the task reward at the first step, forming a budgeted co\-optimization problem\. This task is built onpython\-microgrid\([Henri et al\., 2020](https://arxiv.org/html/2610.02366#bib.bib48)\)\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/mobile_env.png)Figure 18:Wireless Network environment\.Wireless Network\.In this budgeted optimization task a set of wireless stations are covering a high\-traffic area providing service to mobile\-phone users\. Success in this task is directly correlated with the strength and regularity of the signal provided across all users\. We provide three presets\. 1\) NetworkUrban in which pedestrian low\-velocity users are moving in random movement patterns\. 2\) NetworkSuburban in which there is a mix of velocities with some vehicular users in random movement patterns\. 3\) NetworkMotorway in which users are travelling at high velocities, and terrain requires sparse coverage\. In these tasks, the control optimization loop chooses which base station each mobile user is provided connection to as they move around the map\. The design problem specifies how many base stations to deploy \(under a maximum\), where they are placed, and what transmission power capacity each station is given\. This task is built on[Schneider et al\. \(2022\)](https://arxiv.org/html/2610.02366#bib.bib49)\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/f1tenth_v1.png)Figure 19:Car Racing environment\.Car Racing\.In this task a scaled racing car is driving around a track, trying to complete laps in the lowest possible time\. Each car is modelled with load sensitivity, yaw\-inertia, and friction trade\-offs\. We provide two presets\. 1\) RacingLevine in which the car is racing around the Levine track preset\. 2\) RacingSpielberg in which the car is racing around the Spielberg track preset\. In these tasks, the control optimization loop is training a policy for the car and the design problem specifies the vehicle setup\. This configuration includes tyre grip, cornering stiffness, drive power, and more which all possess trade\-offs\. This task is built on F1Tenth\([O’Kelly et al\., 2020](https://arxiv.org/html/2610.02366#bib.bib50)\)\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/robot_warehouse_v1.png)Figure 20:Robotic Warehouse environment\.Robotic Warehouse\.In this task a warehouse is equipped with multiple robotic agents with the goal of providing the highest throughput of request completion\. These requests are deliveries to goals, and they are randomized for each environment within a certain max load\. We provide three presets\. 1\) WarehouseSmall in which there is a 10x10 warehouse floor, two robots, and≥\\geq25 shelves\. 2\) WarehouseMedium in which there is a 16x16 warehouse floor, four robots, and≥\\geq115 shelves\. 3\) WarehouseCongested in which there is a 20x22 warehouse floor, eight robots, and≥\\geq264 shelves\. In these tasks, the control optimization loop is training a policy for the fleet of robots to jointly navigate them to retrieve requested shelves and deliver them to goals\. The design problem specifies the layout and placement of the shelves on a fixed floor trading off route length and congestion\. This task is built on Jumanji\([Bonnet et al\., 2024](https://arxiv.org/html/2610.02366#bib.bib51)\)and RWARE\([Christianos, 2020](https://arxiv.org/html/2610.02366#bib.bib52)\)\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/environment_screenshots/lq_structure.png)Figure 21:Linear\-Quadratic Control environment\.Linear\-Quadratic Control\.In this task a mass\-spring\-damper chain is regulated against environmental disturbances\. We provide three presets: 1\) continuous design with unconstrained, full\-state control; 2\) actuator and sensor placement with control under process and measurement noise; and 3\) actuator placement under a mass and actuator budget with saturated control inputs\. In these tasks, the control optimization loop trains a policy to regulate the structure against the disturbances, while the design problem specifies the masses, stiffnesses, damping, and the actuator/sensor placements\. Because the three presets span continuous, combinatorial, and budget\-constrained design, this environment is designed to serve as a lightweight sanity\-checker exercising the distinct facets of a co\-design algorithm\.

### A\.1Additional Environment Presets

Alongside the main presets described and presented in Section[4](https://arxiv.org/html/2610.02366#S4)\. There are additional environment configurations exposed by the gym for the Nero\-LEAP Grasping and Pokemon Showdown environments\.

#### A\.1\.1Nero\-LEAP Grasping

The NeroGrasp class of environment presets contains ten objects\. These are scissors, tweezers, spatula, stick, screw, long screw, headless screw, t\-nut, lock nut, and washer\. Each object is categorised in the main presets, but can be launched on its own as well\. This exposes nine additional presets\.

#### A\.1\.2Pokemon Showdown

The Pokenv class of environment presets exposes the PokenvLeague preset\. This environment is the most challenging of the Pokemon presets, and involves fighting 5 designed opponents in sequence, mimicking the Pokemon League from the video games\. Over the nine generations of Pokemon video games, there are nine iterations of the Pokemon League, who are typically challenged by the player towards the end of the main storyline\. In this environment preset, we therefore instantiate all nine of these leagues with their teams taken directly from the video games, each controlled by the SimpleHeuristicPlayer\([Sahovic, 2019](https://arxiv.org/html/2610.02366#bib.bib47)\)\. Each league exposes two presets\. One where the agent’s team is “healed” and reset to full strength between battles, and one where it is not\. The second class of presets is substantially harder, and requires the co\-design learner to design a team and play such that it can survive 5 successive opponents\. In addition to these 18 presets, we also expose two additional presets which randomly select the opponent league from all nine generations\.

## Appendix BBaseline Optimization Methods

![Refer to caption](https://arxiv.org/html/2610.02366v1/codesign_baselines_fig_v2.png)Figure 22:WIP: An overview of the space of co\-design algorithms highlighting our chosen baselines\.Baseline 1: CMA\-ES\. The CMA\-ES evolutionary global optimizer is deployed for co\-design by sampling embodiments and controller parameters from the algorithm\. This puts both the controller and the embodiment into the same joint search space from which the algorithm samples\. The controller parameters sampled are used in a neural\-network controller and the embodiment values are passed directly to the environment\. At evaluation boundaries \(every 5% of training\), the current best controller\-embodiment pair is evaluated in multiple parallel evaluation environments on held\-out seeds\.

Baseline 2: FastTD3\. FastTD3 is an off\-policy state\-of\-the\-artReinforcement Learning \(RL\)algorithm predominantly designed for continuous control tasks, such as robot control\. To instantiate FastTD3 for co\-design, we expand its action space to include both the embodiment parameters and the control actions\. At the first step of a FastTD3 rollout, the chosen embodiment values are used to instantiate/reset the environment and the control actions are discarded\. In subsequent steps, the embodiment values are discarded and the control actions are used to step through the environment\. The replay buffer is populated with the corrected samples \(after setting either the embodiment values or control actions to 0\) so that the policy learns from which actions were actually taken\. At evaluation boundaries \(every 5% of training\), the current policy generates a new embodiment and control actions for multiple parallel evaluation environments on held\-out seeds\.

Baseline 3: PPO and NGOpt\. In this baseline PPO and NGOpt are used in alternation to perform co\-design\. PPO is used as normal to train the controller\. After each PPO update iteration, NGOpt samples one embodiment per training environment\. Samples from NGOpt are scored using the mean return obtained by that embodiment in the rollouts in which it was used\. At evaluation boundaries \(every 5% of training\), the current best embodiment is used to instantiate multiple parallel evaluation environments on held\-out seeds and is controlled with the current PPO policy\.

Baseline 4: LOKI\([Jeon et al\., 2026](https://arxiv.org/html/2610.02366#bib.bib58)\)\. In this baseline algorithm, we generalise the approach of[Jeon et al\. \(2026\)](https://arxiv.org/html/2610.02366#bib.bib58)to arbitrary embodiment parametrisations\. For each set of embodiment parameters𝐞\\mathbf\{e\}, we construct a naive tokenization scheme, in place of the robot\-centric tokenization scheme from the original work\. Following this, we randomly sample500,000500,000valid embodiments and pre\-train the Transformer VAE on these embodiments following the original training recipe\. Afterwards, we run the online training loop of[Jeon et al\. \(2026\)](https://arxiv.org/html/2610.02366#bib.bib58)with 6 clusters \(as opposed to the original 40\) to ensure comparability with other baselines, using PPO to train Transformer controllers\. At evaluation boundaries \(every 5% of online training\), the controller from the best cluster is selected and the top performing embodiments from that cluster are used to instantiate multiple parallel evaluation environments on held\-out seeds\.

## Appendix CDetailed results

Table 2:Final evaluation return \(mean±\\pmstd over 5 seeds; higher is better\) on the remaining presets, run with CMA\-ES, FastTD3 and PPO\+NGOpt under a matched 4\-hour wall\-clock budget\. Bold: best mean\.†: runs still in progress\.\(n\):n≠5n\\neq 5seeds\.Figure 23:BallCatcher\-Pitch\.Only FastTD3, which emits the design as the first action of each episode, learns to catch \(16\.6% full catches vs\.≤\\leq1\.1% for the others; final return 3\.42 vs\.≤\\leq0\.44\)\. Its final return varies widely across seeds \(1\.35–5\.98\), so the joint design\-and\-policy agent is effective here but unreliable\. None of the outer\-loop methods reaches catching behaviour within the budget\.Figure 24:Hopper\.The four families trace four different learning curves on the same task, and the leader changes six times in four hours\. FastTD3 peaks early \(2,800 at 0\.6 h\) and decays to 897; PPO\+NGOpt jumps late \(761→\\rightarrow1,588 in the final hour, seeds 462–2,885\); LOKI improves steadily throughout; CMA\-ES plateaus at≈\\approx1,260 almost immediately\. A single budget checkpoint would rank the methods differently at almost every time\.Figure 25:RacingSpielberg\.PPO\+NGOpt finds a strong vehicle setup within the first hour \(eval 157 at 1 h\) and then loses it, ending at 64 after≈\\approx245 design rounds: more outer\-loop design updates are not monotonically better, and a good design found early is not retained\. Its train return \(≈\\approx13\) stays far below eval because most rounds train on exploratory setups\. FastTD3 is stable at≈\\approx77; CMA\-ES never leaves≈\\approx7\.Figure 26:RacingLevine\.The same pattern as on RacingSpielberg on a second track: PPO\+NGOpt peaks within the first hour \(63\) and declines to 40 over≈\\approx274 design rounds, while FastTD3 improves steadily, overtakes it and ends with the most completed laps \(37% vs\. 20%\)\. Early design\-search gains that are not retained recur across tracks, so they reflect the algorithm rather than a single environment\.![Refer to caption](https://arxiv.org/html/2610.02366v1/expr_figures/fig_embodiments_loki_ballcatcher_pitch_v1.png)Figure 27:Final designs on BallCatcher\-Pitch, one per run, taken from the first frame of the run’s final evaluation rollout and shown at a common scale; seeds are sorted by final eval return \(above each image\) and the best is framed thick\. The methods converge to visibly different bodies, and FastTD3 and LOKI also differ from seed to seed, whereas CMA\-ES’s designs look alike: similar returns can come from different morphologies\.Figure 28:Train vs\. eval return on every preset run with all four baselines\(mean over 5 seeds; one block of panels per preset, sharing a y\-axis\)\. Train return averages over the designs being trained or explored and eval return uses the selected design, so the gap measures each method’s cost of design exploration; for CMA\-ES on the Warehouse presets the train return also includes the penalty for rejected designs\. PPO\+NGOpt does not log a train return on HalfCheetah, Swimmer and HumanoidStandup, and its HumanoidStandup runs were still in progress when this figure was generated\.
## Appendix DLLM Co\-Design

In this experiment, we use Gemma 4 31B as the design optimizer in a PPO\+LLM co\-design loop on the BallCatcher\-Pitch preset\. The algorithm mirrors the PPO\+NGOpt baseline \(Section[5](https://arxiv.org/html/2610.02366#S5)\), with the LLM taking the place of NGOpt as the outer\-loop design optimizer\. The PPO controller is conditioned on the parameters of the embodiment it controls\.

We query the LLM for embodiment proposals using the prompt below, with a temperature of X\. Placeholders of the form\{VAR\}are replaced with the corresponding values at query time\.

SYSTEM INSTRUCTION:Propose exactly\{NUM\_TEMPLATES\}embodiment templates in one response\. Return only a JSON object containing anembodimentsarray with exactly\{NUM\_TEMPLATES\}lists\. Each list must contain exactly\{EMBODIMENT\_DIM\}numeric values in parameter\-index order\. Maximize the mean episodic return across random perturbations of each proposed template\. Respect all parameter bounds and integer constraints\. Copies of each template receive independent Gaussian perturbations scaled by the corresponding parameter ranges\. Perturbed values are clipped to their bounds and integer\-valued parameters are rounded\. Use the supplied template\-statistics history to explore the design space and improve robust designs\. The reported statistics describe perturbed copies rather than the exact templates\. Rejected copies receive the rejection penalty and are included when computing mean reward\. The PPO controller changes after every round, so account for the round labels when comparing results\. Return no code, explanation, or text outside the required JSON object\.INPUT:The user message is a JSON object with the following structure:[⬇](data:text/plain;base64,ewogICJ0YXNrIjogIkJhbGxDYXRjaGVyLVBpdGNoIiwKICAiZGVzY3JpcHRpb24iOiB7VEFTS19ERVNDUklQVElPTn0sCiAgInRhc2tfY29udGV4dCI6IHsKICAgICJwYXJhbWV0ZXJfZGVmYXVsdHMiOiB7UEFSQU1FVEVSX0RFRkFVTFRTfSwKICAgICJwYXJhbWV0ZXJfY29sdW1ucyI6IHtQQVJBTUVURVJfQ09MVU1OU30sCiAgICAicGFyYW1ldGVyX3Jvd3MiOiB7UEFSQU1FVEVSX1JPV1N9LAogICAgInBhcmFtZXRlcl9lbmNvZGluZyI6IHtQQVJBTUVURVJfRU5DT0RJTkd9CiAgfSwKICAicm91bmQiOiB7Uk9VTkR9LAogICJzZWFyY2hfc3RhdGUiOiB7CiAgICAiY29tcGxldGVkX3JvdW5kcyI6IHtDT01QTEVURURfUk9VTkRTfSwKICAgICJjb21wbGV0ZWRfcHBvX2l0ZXJhdGlvbnMiOiB7Q09NUExFVEVEX1BQT19JVEVSQVRJT05TfSwKICAgICJwb3B1bGF0aW9uX3NpemUiOiB7UE9QVUxBVElPTl9TSVpFfSwKICAgICJ0ZW1wbGF0ZV9jb3VudCI6IHtOVU1fVEVNUExBVEVTfSwKICAgICJjb3BpZXNfcGVyX3RlbXBsYXRlIjoge0NPUElFU19QRVJfVEVNUExBVEV9LAogICAgInJlbWFpbmluZ19zZWNvbmRzIjoge1JFTUFJTklOR19TRUNPTkRTfSwKICAgICJiZXN0X29ic2VydmVkX2Rlc2lnbiI6IHtCRVNUX0VNQk9ESU1FTlR9LAogICAgImJlc3Rfb2JzZXJ2ZWRfcmV3YXJkIjoge0JFU1RfUkVXQVJEfQogIH0sCiAgImJhc2VfZW1ib2RpbWVudCI6IHtCQVNFX0VNQk9ESU1FTlR9LAogICJwZXJ0dXJiYXRpb25fc3RkX2ZyYWN0aW9uIjoge1BFUlRVUkJBVElPTl9TVER9LAogICJ0ZW1wbGF0ZV9zdGF0aXN0aWNzX2NvbHVtbnMiOiB7U1RBVElTVElDU19DT0xVTU5TfSwKICAidGVtcGxhdGVfc3RhdGlzdGljc19yb3dzIjoge1NUQVRJU1RJQ1NfUk9XU30sCiAgImhpc3RvcnlfZW1ib2RpbWVudF9lbmNvZGluZyI6ICJFYWNoIGhpc3RvcmljYWwgZW1ib2RpbWVudCBjZWxsIGNvbnRhaW5zCiAgICAgIFtwYXJhbWV0ZXJfaW5kZXgsIHZhbHVlXSBwYWlyczsgb21pdHRlZCBpbmRpY2VzIGVxdWFsIGJhc2VfZW1ib2RpbWVudC4iLAogICJkaWFnbm9zdGljX2RlZmluaXRpb25zIjogIlJhdGVzIGFyZSBmcmFjdGlvbnMgb2YgZXZhbHVhdGVkIGNvcGllcyB3aXRoIHRoZQogICAgICBldmVudCBhdCBsZWFzdCBvbmNlIGR1cmluZyB0aGVpciBmaXJzdCBlcGlzb2RlOyByZWplY3RlZCBjb3BpZXMgY29udHJpYnV0ZQogICAgICB6ZXJvIGV2ZW50cyBhbmQgemVybyBsZW5ndGguIGNhdGNoX3JhdGUgcmVxdWlyZXMgYWNjdW11bGF0ZWQgZHdlbGwgYW5kCiAgICAgIHJldGVudGlvbjsgZ29hbF9lbnRyeV9yYXRlIGluY2x1ZGVzIGJyaWVmIGNvbnRhY3Qgd2l0aCB0aGUgcmV3YXJkIHJlZ2lvbi4KICAgICAgTWlzc2luZyBzaWduYWxzIGFyZSBvbWl0dGVkLiIKfQ==)\{"task":"BallCatcher\-Pitch","description":\{TASK\_DESCRIPTION\},"task\_context":\{"parameter\_defaults":\{PARAMETER\_DEFAULTS\},"parameter\_columns":\{PARAMETER\_COLUMNS\},"parameter\_rows":\{PARAMETER\_ROWS\},"parameter\_encoding":\{PARAMETER\_ENCODING\}\},"round":\{ROUND\},"search\_state":\{"completed\_rounds":\{COMPLETED\_ROUNDS\},"completed\_ppo\_iterations":\{COMPLETED\_PPO\_ITERATIONS\},"population\_size":\{POPULATION\_SIZE\},"template\_count":\{NUM\_TEMPLATES\},"copies\_per\_template":\{COPIES\_PER\_TEMPLATE\},"remaining\_seconds":\{REMAINING\_SECONDS\},"best\_observed\_design":\{BEST\_EMBODIMENT\},"best\_observed\_reward":\{BEST\_REWARD\}\},"base\_embodiment":\{BASE\_EMBODIMENT\},"perturbation\_std\_fraction":\{PERTURBATION\_STD\},"template\_statistics\_columns":\{STATISTICS\_COLUMNS\},"template\_statistics\_rows":\{STATISTICS\_ROWS\},"history\_embodiment\_encoding":"Eachhistoricalembodimentcellcontains\[parameter\_index,value\]pairs;omittedindicesequalbase\_embodiment\.","diagnostic\_definitions":"Ratesarefractionsofevaluatedcopieswiththeeventatleastonceduringtheirfirstepisode;rejectedcopiescontributezeroeventsandzerolength\.catch\_raterequiresaccumulateddwellandretention;goal\_entry\_rateincludesbriefcontactwiththerewardregion\.Missingsignalsareomitted\."\}TASK:Propose\{NUM\_TEMPLATES\}new embodiment templates that are likely to maximize mean episodic return after independent perturbation\. Use the task description, embodiment parameter semantics, current PPO training state, best result observed so far, historical template statistics, rejection statistics, and available episode diagnostics\.OUTPUT REQUIREMENTS:Return JSON only: no extra text, code fences, comments, or reasoning\. Return one JSON object matching this schema:[⬇](data:text/plain;base64,ewogICJlbWJvZGltZW50cyI6IFsKICAgIFs8ZXhhY3RseSB7RU1CT0RJTUVOVF9ESU19IG51bWVyaWMgdmFsdWVzPl0sCiAgICAuLi4KICAgIFs8ZXhhY3RseSB7RU1CT0RJTUVOVF9ESU19IG51bWVyaWMgdmFsdWVzPl0KICBdCn0=)\{"embodiments":\[\[<exactly\{EMBODIMENT\_DIM\}numericvalues\>\],\.\.\.\[<exactly\{EMBODIMENT\_DIM\}numericvalues\>\]\]\}IMPORTANT INFORMATION:•The output must contain exactly\{NUM\_TEMPLATES\}embodiment templates\.•Every template must contain exactly\{EMBODIMENT\_DIM\}values in parameter\-index order\.•Every value must satisfy its supplied bounds and integer constraint\.•The optimization target is mean episodic return across the perturbed copies of each template\.•Rejected embodiments receive the rejection penalty and reduce the template’s mean reward\.•Historical rewards come from different PPO training rounds; their round labels must therefore be considered\.•Only valid JSON matching the specified schema is accepted\.Figure 29:Prompt used by the PPO\+LLM outer loop on BallCatcher\-Pitch\. Placeholders of the form\{VAR\}are replaced with the corresponding values at query time\.This prompt conditions the LLM on far more information than any other baseline receives\. It conveys details of the environment that the other baselines can only learn through interaction, and it provides context on each embodiment parameter, such that a model could anticipate the relationship between provided embodiment parameters and design\.

To ensure a fair comparison, PPO\+LLM must train on as many embodiments per design round as the other baselines\. Otherwise, any difference in performance could simply reflect a narrower exploration of the design space\. PPO\+NGOpt, its direct counterpart, samples one embodiment for each of the 16,384 parallel environments\. Having the LLM generate that many designs directly, however, is infeasible under API rate limits and token costs\. We therefore split design generation into two stages\. In each design round, the LLM proposes 16*template*embodiments, and each template is perturbed with Gaussian noise \(standard deviation 0\.05\) to yield 1,024 variants, giving 16,384 embodiments in total\. The prompt informs the LLM of this perturbation scheme\. For evaluation runs, both methods evaluate the single best\-scoring design found so far\.

##### Results\.

Despite the informational advantage granted to the agentic designer, PPO\+LLM performs poorly in all runs, reaching low final return and showing little learning progress\. The embodiments proposed by the LLM were often impractical and over\-engineered, such that, even when the controller did learn meaningful behaviour, the given design stalled learning progress\.

Figure[30](https://arxiv.org/html/2610.02366#A4.F30)compares training and evaluation return against the number of PPO iterations\. Over a comparable number of iterations \(68 and 59 for PPO\+LLM, 68–71 for PPO\+NGOpt\), PPO\+NGOpt improves from≈\\approx0\.08 to 0\.43 eval return, while both PPO\+LLM runs stay at or near zero throughout training and at every evaluation\. Between 27% and 36% of LLM\-designs generated were unusable\. The best design was updated 3 and 4 times in the two PPO\+LLM runs, and between 1 and 15 times across the PPO\+NGOpt seeds\. Figure[31](https://arxiv.org/html/2610.02366#A4.F31)shows every distinct evaluated design for seed 0 of each method\.

Table 3:PPO\+LLM \(Gemma 4 31B, 16 templates per round, perturbation std 0\.05, 16,384 environments\) on BallCatcher\-Pitch, per seed, against the PPO\+NGOpt baseline\.Figure 30:PPO\+LLM vs\. PPO\+NGOpt on BallCatcher\-Pitch, plotted against PPO iterations because both algorithms run the same PPO inner loop and the PPO\+LLM runs stopped early \(×\{\\times\}\)\. PPO\+NGOpt’s return \(5\-seed mean±\\pmstd\) rises from≈\\approx0\.08 to 0\.43 at evaluation, whereas both PPO\+LLM runs stay at or near zero in training \(≤\\leq0\.005\) and at every evaluation \(≤\\leq0\.02\), over a comparable number of PPO iterations \(68 and 59 vs\. 68–71\)\.![Refer to caption](https://arxiv.org/html/2610.02366v1/fig_llm_designs_over_training.png)Figure 31:Every distinct evaluated design\(seed 0, illustrative\) for PPO\+LLM \(amber frames\) and PPO\+NGOpt \(aqua frames\): the initial design and the design after each time a better\-scoring candidate replaced it \(PPO\+LLM at 0\.21, 0\.64 and 2\.84 h; PPO\+NGOpt once, at 1\.91 h\), each taken from the first frame of the next evaluation video and shown at a common scale \(title: PPO iteration and eval return of that evaluation\)\.

## Appendix EAPI Illustration

Figure[1](https://arxiv.org/html/2610.02366#S0.F1)illustrates the inner workings of our API: It functions as an interface between co\-design algorithm and the environments while minimizing deviations to the established Gymnasium structure, thereby ensuring broad compatibility across established frameworks\.

![Refer to caption](https://arxiv.org/html/2610.02366v1/api_fig_full.png)Figure 32:Co\-design Gym API flowchart\.
## References

- Balciunaiteet al\.\(2026\)A\. Balciunaite, M\. Y\. Michelis, M\. Filippi, P\. Paniagua, O\. Yasa, and R\. K\. KatzschmannCo\-optimized volumetric muscle design for large dynamic deformations in versatile biohybrid robots\.Nature Communications\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p3.1)\.
- Berneret al\.\(2019\)C\. Berner, G\. Brockman, B\. Chan, V\. Cheung, P\. Debiak, C\. Dennison, D\. Farhi, Q\. Fischer, S\. Hashme, C\. Hesse, R\. Józefowicz, S\. Gray, C\. Olsson, J\. W\. Pachocki, M\. Petrov, H\. P\. de Oliveira Pinto, J\. Raiman, T\. Salimans, J\. Schlatter, J\. Schneider, S\. Sidor, I\. Sutskever, J\. Tang, F\. Wolski, and S\. ZhangDota 2 with large scale deep reinforcement learning\.ArXivabs/1912\.06680\.External Links:[Link](https://api.semanticscholar.org/CorpusID:209376771)Cited by:[§1](https://arxiv.org/html/2610.02366#S1.p4.1)\.
- Bhatiaet al\.\(2022\)J\. Bhatia, H\. Jackson, Y\. Tian, J\. Xu, and W\. MatusikEvolution gym: a large\-scale benchmark for evolving soft robots\.InNeural Information Processing Systems,External Links:[Link](https://api.semanticscholar.org/CorpusID:245011343)Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p8.1),[§2](https://arxiv.org/html/2610.02366#S2.p4.1),[§4](https://arxiv.org/html/2610.02366#S4.p8.1)\.
- Bongard \(2011\)J\. BongardMorphological change in machines accelerates the evolution of robust behavior\.Proceedings of the National Academy of Sciences108\(4\),pp\. 1234–1239\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Bonnetet al\.\(2024\)C\. Bonnet, D\. Luo, D\. Byrne, S\. Surana, S\. Abramowitz, P\. Duckworth, V\. Coyette, L\. I\. Midgley, E\. Tegegn, T\. Kalloniatis, O\. Mahjoub, M\. Macfarlane, A\. P\. Smit, N\. Grinsztajn, R\. Boige, C\. N\. Waters, M\. A\. Mimouni, U\. A\. M\. Sob, R\. de Kock, S\. Singh, D\. Furelos\-Blanco, V\. Le, A\. Pretorius, and A\. LaterreJumanji: a diverse suite of scalable reinforcement learning environments in jax\.External Links:2306\.09884,[Link](https://arxiv.org/abs/2306.09884)Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p13.1),[§4](https://arxiv.org/html/2610.02366#S4.p13.1)\.
- Brockmanet al\.\(2016\)G\. Brockman, V\. Cheung, L\. Pettersson, J\. Schneider, J\. Schulman, J\. Tang, and W\. ZarembaOpenai gym\.arXiv preprint arXiv:1606\.01540\.Cited by:[§1](https://arxiv.org/html/2610.02366#S1.p4.1),[§2](https://arxiv.org/html/2610.02366#S2.p4.1)\.
- Censi \(2015\)A\. CensiA mathematical theory of co\-design\.arXiv preprint arXiv:1512\.08055\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Chenet al\.\(2026\)C\. Chen, X\. Xu, X\. Wang, J\. Wang, and X\. HuangNatural selection via foundation models for soft robot evolution\.In2026 IEEE 9th International Conference on Soft Robotics \(RoboSoft\),pp\. 422–429\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Cheneyet al\.\(2014a\)N\. Cheney, J\. Clune, and H\. LipsonEvolved electrophysiological soft robots\.InArtificial life conference proceedings,pp\. 222–229\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Cheneyet al\.\(2014b\)N\. Cheney, R\. MacCurdy, J\. Clune, and H\. LipsonUnshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding\.ACM SIGEVOlution7\(1\),pp\. 11–23\.Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p8.1),[§2](https://arxiv.org/html/2610.02366#S2.p2.1),[§4](https://arxiv.org/html/2610.02366#S4.p8.1)\.
- Christianos \(2020\)F\. ChristianosMulti\-robot warehouse \(rware\)\.GitHub\.External Links:[Link](https://github.com/semitable/robotic-warehouse)Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p13.1),[§4](https://arxiv.org/html/2610.02366#S4.p13.1)\.
- Coroset al\.\(2013\)S\. Coros, B\. Thomaszewski, G\. Noris, S\. Sueda, M\. Forberg, R\. W\. Sumner, W\. Matusik, and B\. BickelComputational design of mechanical characters\.ACM Transactions on Graphics \(TOG\)32\(4\),pp\. 1–12\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Corucciet al\.\(2016\)F\. Corucci, N\. Cheney, H\. Lipson, C\. Laschi, and J\. BongardEvolving swimming soft\-bodied creatures\.InALIFE XV, the fifteenth international conference on the synthesis and simulation of living systems, late breaking proceedings,Vol\.6\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Daiet al\.\(2026\)Y\. Dai, Y\. Wang, D\. R\. Ashley, and J\. SchmidhuberEfficient morphology–control co\-design via stackelberg proximal policy optimization\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- De Michell and Gupta \(1997\)G\. De Michell and R\. K\. GuptaHardware/software co\-design\.Proceedings of the IEEE85\(3\),pp\. 349–365\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p1.1)\.
- DeepMind \(2025\)G\. DeepMindMujocowarp\.Note:GitHub repository\. GPU\-optimized version of the MuJoCo physics simulator, designed for NVIDIA hardware\.\. Accessed 2026\-09\-12External Links:[Link](https://github.com/google-deepmind/mujoco_warp)Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p1.1),[§4](https://arxiv.org/html/2610.02366#S4.p2.1)\.
- Degraveet al\.\(2022\)J\. Degrave, F\. Felici, J\. Buchli, M\. Neunert, B\. D\. Tracey, F\. Carpanese, T\. Ewalds, R\. Hafner, A\. Abdolmaleki, D\. de Las Casas, C\. Donner, L\. Fritz, C\. Galperti, A\. Huber, J\. Keeling, M\. Tsimpoukelli, J\. Kay, A\. Merle, J\-M\. Moret, S\. Noury, F\. Pesamosca, D\. Pfau, O\. Sauter, C\. Sommariva, S\. Coda, B\. P\. Duval, A\. Fasoli, P\. Kohli, K\. Kavukcuoglu, D\. Hassabis, and M\. A\. RiedmillerMagnetic control of tokamak plasmas through deep reinforcement learning\.Nature602,pp\. 414 – 419\.External Links:[Link](https://api.semanticscholar.org/CorpusID:246904229)Cited by:[§1](https://arxiv.org/html/2610.02366#S1.p4.1)\.
- Dowski Jr and Cathey \(1995\)E\. R\. Dowski Jr and W\. T\. CatheyExtended depth of field through wave\-front coding\.Applied optics34\(11\),pp\. 1859–1866\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p1.1)\.
- Fayet al\.\(2026\)K\. Fay, D\. A\. Djapri, A\. Zorin, J\. Clinton, A\. E\. Lahib, H\. Su, M\. T\. Tolley, S\. Yi, and X\. WangHouse of dextra: cross\-embodied co\-design for dexterous hands\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://api.semanticscholar.org/CorpusID:283466942)Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p3.1)\.
- Gaoet al\.\(2026\)G\. Gao, T\. Li, J\. Shi, Y\. Li, Z\. Zhang, N\. Figueroa, and D\. JayaramanVLMgineer: vision\-language models as robotic toolsmiths\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 91732–91763\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Geleset al\.\(2026\)I\. Geles, L\. Bauersfeld, M\. Wulfmeier, and D\. ScaramuzzaSuperhuman safe and agile racing through multi\-agent reinforcement learning\.External Links:2605\.22748,[Link](https://arxiv.org/abs/2605.22748)Cited by:[§1](https://arxiv.org/html/2610.02366#S1.p4.1)\.
- Haet al\.\(2026\)H\. Ha, C\. K\. Liu, and S\. SongTransformer transformer: a unified model for motion\-conditioned robot co\-design\.arXiv preprint arXiv:2607\.25798\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Haet al\.\(2017\)S\. Ha, S\. Coros, A\. Alspach, J\. Kim, and K\. YamaneJoint optimization of robot design and motion parameters using the implicit function theorem\.\.InRobotics: Science and systems,Vol\.13,pp\. 10–15607\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Hejna IIIet al\.\(2021\)D\. J\. Hejna III, P\. Abbeel, and L\. PintoTask\-agnostic morphology evolution\.arXiv preprint arXiv:2102\.13100\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Henriet al\.\(2020\)G\. Henri, T\. Levent, A\. Halev, R\. Alami, and P\. CordierPymgrid: an open\-source python microgrid simulator for applied artificial intelligence research\.ArXivabs/2011\.08004\.External Links:[Link](https://api.semanticscholar.org/CorpusID:226964357)Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p10.1),[§4](https://arxiv.org/html/2610.02366#S4.p11.1)\.
- Hiller and Lipson \(2011\)J\. Hiller and H\. LipsonAutomatic design and manufacture of soft robots\.IEEE Transactions on Robotics28\(2\),pp\. 457–466\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Jeonet al\.\(2026\)H\. Jeon, A\. Eftekhar, A\. Walsman, K\. Zeng, A\. Farhadi, and R\. KrishnaConvergent functions, divergent forms\.Advances in Neural Information Processing Systems38,pp\. 46049–46076\.Cited by:[Appendix B](https://arxiv.org/html/2610.02366#A2.p4.1),[Appendix B](https://arxiv.org/html/2610.02366#A2.p4.1.1),[§5](https://arxiv.org/html/2610.02366#S5.p6.1)\.
- Kriegmanet al\.\(2020\)S\. Kriegman, D\. Blackiston, M\. Levin, and J\. BongardA scalable pipeline for designing reconfigurable organisms\.Proceedings of the National Academy of Sciences117\(4\),pp\. 1853–1859\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p3.1)\.
- Lord and Miller \(2026\)P\. Lord and C\. MillerProject hail mary\.Amazon MGM Studios\.Note:FilmScreenplay by Drew Goddard, based on the novel by Andy Weir\. Starring Ryan GoslingCited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p2.1)\.
- Luyben and Floudas \(1994\)M\. Luyben and C\. FloudasAnalyzing the interaction of design and control—1\. a multiobjective framework and application to binary distillation synthesis\.Computers & chemical engineering18\(10\),pp\. 933–969\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p1.1)\.
- Maet al\.\(2021\)P\. Ma, T\. Du, J\. Z\. Zhang, K\. Wu, A\. Spielberg, R\. K\. Katzschmann, and W\. MatusikDiffaqua: a differentiable computational design pipeline for soft underwater swimmers with shape interpolation\.ACM Transactions on Graphics \(TOG\)40\(4\),pp\. 1–14\.Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p8.1),[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Macklin \(2022\)M\. MacklinWarp: A High\-performance Python Framework for GPU Simulation and Graphics\.External Links:[Link](https://github.com/nvidia/warp)Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p1.1),[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p8.1),[§4](https://arxiv.org/html/2610.02366#S4.p2.1)\.
- Margoliset al\.\(2021\)G\. B\. Margolis, T\. Chen, K\. Paigwar, X\. Fu, D\. Kim, S\. Kim, and P\. AgrawalLearning to jump from pixels\.arXiv preprint arXiv:2110\.15344\.Cited by:[§1](https://arxiv.org/html/2610.02366#S1.p4.1)\.
- Matthewset al\.\(2023\)D\. Matthews, A\. Spielberg, D\. Rus, S\. Kriegman, and J\. BongardEfficient automatic design of robots\.Proceedings of the National Academy of Sciences120\(41\),pp\. e2305180120\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Milderet al\.\(2012\)P\. Milder, F\. Franchetti, J\. C\. Hoe, and M\. PüschelComputer generation of hardware for linear digital signal processing transforms\.ACM Transactions on Design Automation of Electronic Systems \(TODAES\)17\(2\),pp\. 1–33\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p1.1)\.
- Minaeeet al\.\(2024\)S\. Minaee, T\. Mikolov, N\. Nikzad, M\. A\. Chenaghlu, R\. Socher, X\. Amatriain, and J\. GaoLarge language models: a survey\.ArXivabs/2402\.06196\.External Links:[Link](https://api.semanticscholar.org/CorpusID:267617032)Cited by:[§5](https://arxiv.org/html/2610.02366#S5.p9.1)\.
- Mnihet al\.\(2015\)V\. Mnih, K\. Kavukcuoglu, D\. Silver, A\. A\. Rusu, J\. Veness, M\. G\. Bellemare, A\. Graves, M\. Riedmiller, A\. K\. Fidjeland, G\. Ostrovski, S\. Petersen, C\. Beattie, A\. Sadik, I\. Antonoglou, H\. King, D\. Kumaran, D\. Wierstra, S\. Legg, and D\. HassabisHuman\-level control through deep reinforcement learning\.Nature518\(7540\),pp\. 529–533\.External Links:ISSN 00280836,[Link](http://dx.doi.org/10.1038/nature14236)Cited by:[§1](https://arxiv.org/html/2610.02366#S1.p4.1)\.
- Morganet al\.\(2026\)A\. S\. Morgan, G\. Xie, C\. Bass, R\. Thomasson, C\. Wang, E\. Shahriari, H\. Busa, O\. Araromi, J\. Aronov, M\. Burgess, V\. D\. Dimitrov, M\. A\. Estrada, F\. Hamdi, S\. Kendig, T\. Lee, J\. O\. Mur\-Miranda, E\. Sommers, P\. Titchener, M\. Wang, A\. Wilson, M\. Yeatman, O\. D\. Yirmibesoglu, A\. A\. Rizzi, A\. Mozeika, N\. Rojas, and L\. OdhnerAthenaZero: a low\-inertia, bimanual robot for dynamic manipulation\.Science Robotics11\(118\)\.External Links:ISSN 2470\-9476,[Link](http://dx.doi.org/10.1126/scirobotics.aee1868),[Document](https://dx.doi.org/10.1126/scirobotics.aee1868)Cited by:[§4](https://arxiv.org/html/2610.02366#S4.p3.1)\.
- O’Kellyet al\.\(2020\)M\. O’Kelly, H\. Zheng, D\. Karthik, and R\. MangharamF1TENTH: an open\-source evaluation environment for continuous control and reinforcement learning\.InNeurIPS 2019 Competition and Demonstration Track,pp\. 77–89\.Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p12.1),[§4](https://arxiv.org/html/2610.02366#S4.p9.1)\.
- Park and Asada \(1994\)J\. Park and H\. AsadaConcurrent design optimization of mechanical structure and control for high speed robots\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p3.1)\.
- Penget al\.\(2018\)X\. B\. Peng, P\. Abbeel, S\. Levine, and M\. Van de PanneDeepmimic: example\-guided deep reinforcement learning of physics\-based character skills\.ACM Transactions On Graphics \(TOG\)37\(4\),pp\. 1–14\.Cited by:[§1](https://arxiv.org/html/2610.02366#S1.p4.1)\.
- Penget al\.\(2017\)X\. B\. Peng, G\. Berseth, K\. Yin, and M\. Van De PanneDeeploco: dynamic locomotion skills using hierarchical deep reinforcement learning\.Acm transactions on graphics \(tog\)36\(4\),pp\. 1–13\.Cited by:[§1](https://arxiv.org/html/2610.02366#S1.p4.1)\.
- Rapin and Teytaud \(2018\)J\. Rapin and O\. TeytaudNevergrad \- A gradient\-free optimization platform\.GitHub\.Note:[https://GitHub\.com/FacebookResearch/Nevergrad](https://github.com/FacebookResearch/Nevergrad)Cited by:[§5](https://arxiv.org/html/2610.02366#S5.p5.1.1)\.
- Sahovic \(2019\)H\. SahovicPoke\-env\.Note:GitHub repository\. Poke\-env: Python Interface for Pokemon Showdown Bots\. Accessed 2026\-09\-23External Links:[Link](https://github.com/hsahovic/poke-env)Cited by:[§A\.1\.2](https://arxiv.org/html/2610.02366#Ax1.SS1.SSS2.p1.1),[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p9.1),[§4](https://arxiv.org/html/2610.02366#S4.p14.1)\.
- Salamaet al\.\(1989\)M\. Salama, R\. Ramaker, and M\. MilmanRecent experience in simultaneous control\-structure optimization\.NASA, Langley Research Center, NASA \(DOD Controls\-Structures Interaction Technology 1989\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p1.1)\.
- Schneideret al\.\(2022\)S\. Schneider, S\. Werner, R\. Khalili, A\. Hecker, and H\. KarlMobile\-env: an open platform for reinforcement learning in wireless mobile networks\.InNetwork Operations and Management Symposium \(NOMS\),Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p11.1),[§4](https://arxiv.org/html/2610.02366#S4.p12.1)\.
- Schulmanet al\.\(2017\)J\. D\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. KlimovProximal policy optimization algorithms\.ArXivabs/1707\.06347\.External Links:[Link](https://api.semanticscholar.org/CorpusID:28695052)Cited by:[§5](https://arxiv.org/html/2610.02366#S5.p5.1.1)\.
- Seoet al\.\(2025\)Y\. Seo, C\. Sferrazza, H\. Geng, M\. Nauman, Z\. Yin, and P\. AbbeelFastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control\.Note:Version Number: 3External Links:[Link](https://arxiv.org/abs/2505.22642),[Document](https://dx.doi.org/10.48550/ARXIV.2505.22642)Cited by:[§5](https://arxiv.org/html/2610.02366#S5.p4.1.1)\.
- Silveret al\.\(2016\)D\. Silver, A\. Huang, C\. J\. Maddison, A\. Guez, L\. Sifre, G\. van den Driessche, J\. Schrittwieser, I\. Antonoglou, V\. Panneershelvam, M\. Lanctot, S\. Dieleman, D\. Grewe, J\. Nham, N\. Kalchbrenner, I\. Sutskever, T\. P\. Lillicrap, M\. Leach, K\. Kavukcuoglu, T\. Graepel, and D\. HassabisMastering the game of go with deep neural networks and tree search\.Nature529,pp\. 484–489\.External Links:[Link](https://api.semanticscholar.org/CorpusID:515925)Cited by:[§1](https://arxiv.org/html/2610.02366#S1.p4.1)\.
- Sims \(1994\)K\. SimsEvolving virtual creatures\.InProceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques \(SIGGRAPH\),pp\. 15–22\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Spielberget al\.\(2017\)A\. Spielberg, B\. Araki, C\. Sung, R\. Tedrake, and D\. RusFunctional co\-optimization of articulated robots\.In2017 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 5035–5042\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Spielberget al\.\(2023\)A\. Spielberg, T\. Du, Y\. Hu, D\. Rus, and W\. MatusikAdvanced soft robot modeling in chainqueen\.Robotica41\(1\),pp\. 74–104\.Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p8.1),[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Spielberget al\.\(2019\)A\. Spielberg, A\. Zhao, Y\. Hu, T\. Du, W\. Matusik, and D\. RusLearning\-in\-the\-loop optimization: end\-to\-end control and co\-design of soft robots through learned deep latent representations\.Advances in Neural Information Processing Systems32\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Todorovet al\.\(2012\)E\. Todorov, T\. Erez, and Y\. TassaMuJoCo: A physics engine for model\-based control\.In2012 IEEE/RSJ International Conference on Intelligent Robots and Systems,pp\. 5026–5033\.External Links:[Document](https://dx.doi.org/10.1109/IROS.2012.6386109)Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p1.1),[§4](https://arxiv.org/html/2610.02366#S4.p2.1)\.
- Towerset al\.\(2024\)M\. Towers, A\. Kwiatkowski, J\. K\. Terry, J\. U\. Balis, G\. M\. D\. Cola, T\. Deleu, M\. Goulão, A\. Kallinteris, M\. Krimmel, K\. Arjun, R\. Perez\-Vicente, A\. Pierré, S\. Schulhoff, J\. J\. Tai, H\. Tan, and O\. G\. YounisGymnasium: a standard interface for reinforcement learning environments\.External Links:[Link](https://api.semanticscholar.org/CorpusID:290088643)Cited by:[§1](https://arxiv.org/html/2610.02366#S1.p4.1),[§2](https://arxiv.org/html/2610.02366#S2.p4.1),[§4\.1](https://arxiv.org/html/2610.02366#S4.SS1.p1.1)\.
- University \(2011\)S\. UniversityPokemon\-showdown\.Note:GitHub repository\. Pokémon battle simulator\.\. Accessed 2026\-09\-12External Links:[Link](https://github.com/smogon/pokemon-showdown)Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p9.1),[§4](https://arxiv.org/html/2610.02366#S4.p14.1)\.
- Ventrella \(1994\)J\. VentrellaExplorations in the emergence of morphology and locomotion behavior in animated characters\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Wampler and Popović \(2009\)K\. Wampler and Z\. PopovićOptimal gait and form for animal locomotion\.ACM Transactions on Graphics \(TOG\)28\(3\),pp\. 1–8\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Wanget al\.\(2026a\)H\. Wang, Z\. Fang, J\. Hanna, and X\. XiongEfficient and versatile quadrupedal skating: optimal co\-design via reinforcement learning and bayesian optimization\.arXiv preprint arXiv:2603\.18408\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p3.1)\.
- Wanget al\.\(2026b\)J\. Wang, L\. Qian, Z\. Wang, W\. Wu, J\. Zong, A\. Gupta, and D\. PathakBRIDGE: an open\-source humanoid platform via morphology\-control co\-design for physical ai\.arXiv preprint arXiv:2609\.03497\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p3.1)\.
- Wanget al\.\(2019\)T\. Wang, Y\. Zhou, S\. Fidler, and J\. BaNeural graph evolution: towards efficient automatic robot design\.arXiv preprint arXiv:1906\.05370\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Wanget al\.\(2023a\)T\. J\. Wang, J\. Zheng, P\. Ma, Y\. Du, B\. Kim, A\. Spielberg, J\. Tenenbaum, C\. Gan, and D\. RusDiffusebot: breeding soft robots with physics\-augmented generative diffusion models\.Advances in Neural Information Processing Systems36,pp\. 44398–44423\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Wanget al\.\(2023b\)T\. Wang, P\. Ma, A\. E\. Spielberg, Z\. Xian, H\. Zhang, J\. B\. Tenenbaum, D\. Rus, and C\. GanSoftZoo: a soft robot co\-design benchmark for locomotion in diverse environments\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Xyme9p1rpZw)Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p4.1)\.
- Weir \(2021\)A\. WeirProject hail mary\.Ballantine Books,New York\.External Links:ISBN 9780593135204Cited by:[Appendix A: Environments](https://arxiv.org/html/2610.02366#Ax1.p2.1)\.
- Xuet al\.\(2021\)J\. Xu, A\. Spielberg, A\. Zhao, D\. Rus, and W\. MatusikMulti\-objective graph heuristic search for terrestrial robot design\.In2021 IEEE international conference on robotics and automation \(ICRA\),pp\. 9863–9869\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Xuet al\.\(2024\)X\. Xu, H\. Ha, and S\. SongDynamics\-guided diffusion model for sensor\-less robot manipulator design\.arXiv preprint arXiv:2402\.15038\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p3.1)\.
- Yuet al\.\(2026\)C\. Yu, D\. Matthews, J\. Wang, J\. Gu, D\. Blackiston, M\. Rubenstein, and S\. KriegmanAgile legged locomotion in reconfigurable modular robots\.Proceedings of the National Academy of Sciences123\(10\),pp\. e2519129123\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p3.1)\.
- Yuanet al\.\(2021\)Y\. Yuan, Y\. Song, Z\. Luo, W\. Sun, and K\. KitaniTransform2act: learning a transform\-and\-control policy for efficient agent design\.arXiv preprint arXiv:2110\.03659\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Zardini \(2023\)G\. ZardiniCo\-design of complex systems: from autonomy to future mobility systems\.Ph\.D\. Thesis,ETH Zurich\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.
- Zhaoet al\.\(2020\)A\. Zhao, J\. Xu, M\. Konaković\-Luković, J\. Hughes, A\. Spielberg, D\. Rus, and W\. MatusikRobogrammar: graph grammar for terrain\-optimized robot design\.ACM Transactions on Graphics \(TOG\)39\(6\),pp\. 1–16\.Cited by:[§2](https://arxiv.org/html/2610.02366#S2.p2.1)\.