Impressive paper from Stanford on harness evolution for robotics.

They improve frozen robot policies from 9.7% to 43.3% success on unseen scenes without changing a single weight.

The gain comes from the harness.

RoboArchitect gives a coding model the robot’s rollouts. The coding model diagnoses failures and rewrites the control code around the frozen policy. That code decides when to call the policy, when to use vision tools and when to command the arm directly.

The harness is optimized on one set of episodes and tested on held-out scenes and tasks. Success on unseen tasks rises from 12.7% to 40.2%.

The largest gains appear where the benchmark is far from the policy’s training data. Inside an evolved harness, a general-purpose policy often beats the finetuned version of the same policy.

The search also produced control routines such as lifting a jammed arm clear, and it added a persistent memory that stores values like effective grasp depths.

Paper: https://arxiv.org/abs/2610.42141

Chat with Paper: https://academy.dair.ai/papers/roboarchitect-automating-the-design-of-robotic-agents-with-persistent-memory-2610.42141…

Automating the Design ofRobotic Agents with Persistent Memory

Source: https://arxiv.org/html/2610.42141

RoboArchitect: Automating the Design of Robotic Agents with Persistent Memory

Suning HuangCherie HoRia DoshiHao ZhuSerena Yeung-LevyAffiliation:Stanford UniversityAffiliation:https://roboarchitect-ai.github.io/

Abstract

Recent work shows that a frozen robot policy performs far better inside an agent harness, where control code decides when to query the policy, consult vision tools, or command the arm directly. Yet today these harnesses are built by hand and redesigned for each new robot or benchmark. We show that harness design can be automated, offering a strong alternative to finetuning on a novel setup. InRoboArchitect, a coding model observes a robot’s rollouts, diagnoses failures, and iteratively revises the harness around the frozen policy. The harness evolves on a set of episodes for its optimization and is evaluated on held-out scenes and tasks. Without updating any model weights,RoboArchitectraises average success from 9.7% to 43.3% on unseen scenes (4.5×\times) and from 12.7% to 40.2% on unseen tasks (3.2×\times), across multiple benchmarks and policies. The gains are largest where the benchmark is far from the policy’s training data: wrapping a general-purpose policy, the evolved harness yields a specialist that often outperforms even the finetuned version. Intuitive control strategies are automatically designed, such as lifting a jammed arm clear or reaching in horizontally when an obstacle blocks the way from above. The harness also evolves its own persistent memory, carrying learned experience, such as effective grasp depths, into later decisions. These results suggest that substantial headroom in robotic manipulation lies in the agent harness, and its design can be effectively automated and optimized without weight updates.

1Introduction

Language models have become far more useful inside an agent harness: surrounding code that mediates when and how to call the model, what context to supply, and how to handle its output. Robot policies are undergoing the same shift. This shift is particularly important for robotics, where adapting a policy to new tasks or hardware often requires collecting new demonstrations and finetuning model weights(Kim et al., 2024;Zang et al., 2025;Park et al., 2026). We posit thatharness optimization offers an effective alternative to finetuning, turning a general policy into a specialistfor a specific robot setup (Section4.2).

Recent agentic robot systems like VoLo(Chen et al., 2026d)and Harness VLA(Zhang et al., 2026e)show strong gains from wrapping frozen policies in hand-written harnesses(Xu et al., 2026;Zhao et al., 2026b;Gao et al., 2026;Hu et al., 2026). Today, these harnesses are still manually designed, often requiring substantial engineering effort for each task and embodiment. Building such a harness follows a familiar trial-and-error loop: run the agent on the robot, observe where it fails, diagnose the cause, modify the control code, and test again. This work is slow to perform by hand, and long recordings of robot rollouts are difficult for a human engineer to inspect exhaustively. We askwhether this complete engineering loop, from diagnosing failure traces to refining the harness design to fix them, can be automated.

In this work,RoboArchitectuses a coding model to automatically design and optimize the harness (Section3). Given access to a frozen policy, auxiliary vision and language tools, and step-by-step rollout recordings, the coding model diagnoses failure modes and proposes an updated harness to address them. Like a finetuned policy, a harness optimized on specific rollouts could overfit to them. We therefore split each benchmark into “training” (for harness optimization) and held-out episodes, evaluating onunseen scenes, where tasks are shared but scene arrangements differ, andunseen tasks, where held-out episodes contain entirely different tasks that require shared manipulation skills (Section4.1).

Figure 1:Wrapping a frozen policy in an evolved harness raises its held-out success rate from 9.7% to 43.3% on unseen scenes and from 12.7% to 40.2% on unseen tasks, often past the same policy finetuned on the benchmark(Section4.2). All scores are evaluated on the held-out episodes never seen during the harness’s optimization.*Unseen scenes:*held-out episodes evaluate the same tasks under varied scene arrangements.*Unseen tasks:*held-out episodes evaluate distinct tasks to test cross-task skill transfer (Section4.1).Across diverse benchmarks and policies, our auto-optimized harnesses raise the average success rate on unseen scenes from 9.7% to 43.3% (4.5×\times), and on unseen tasks from 12.7% to 40.2% (3.2×\times) (Fig.1). The strongest gains occur when policies are deployed in out-of-domain settings, whereRoboArchitectcan effectively substitute for new training data: wrapping a general policy yields a specialist that often outperforms the finetuned version of the same model. For example, in settings where the vision-language-action (VLA) policy contributes no useful signal, the harness learns to stop querying it and instead drives the arm from its own perception, effectively becoming a code-as-policy agent. While code-as-policy(Liang et al., 2023;Lu et al., 2026;Elmaaroufi et al., 2026)and learned VLA policies(Kim et al., 2024;Intelligence et al., 2025)have traditionally been treated as distinct paradigms,RoboArchitectunifies them empirically (Section4.2). Unlike weight updates, each harness revision is also fully interpretable: it targets specific failure modes with intuitive control logic (Figs.3and6), making subsequent revisions easy to inspect and build upon, and such iterative refinement proves to be key to generalization (Section4.2).

We also equip the harness withpersistent memory, allowing experience to accumulate across episodes. No schema or memory rules are specified;RoboArchitectis entirely free to decide what to store, in what form, and how to retrieve it. As a result, diverse forms of memories emerge from the harness optimization to record key pragmatic clues, such as effective grasp depths that inform later control decisions.

AsRoboArchitectproves highly effective in solving tasks on one robot embodiment at a time, we further use its harness evolution to builda unified, generalizable agent that operates across different robot platforms(Section4.2). In this single harness optimized on a training pool drawn from diverse benchmarks, shared control strategies and memory structures emerge naturally across embodiments.

Our findings suggest that substantial performance headroom in robotic manipulation lies in the agent harness, as an alternative to finetuning.RoboArchitectautomates its design and optimization, providing a practical way to adapt robot policies to new environments without updating weights.

2Related Work

Vision-language-action policies.VLA policies are pretrained vision-language models finetuned for robot control, mapping visual observations and language instructions directly to robot actions(Brohan et al., 2023;Kim et al., 2024;Black et al., 2026;Intelligence et al., 2025;Shukor et al., 2025;Zheng et al., 2026;Fang et al., 2026;Kim et al., 2026). In practice, their performance on new, out-of-distribution tasks can depend heavily on finetuning(Zang et al., 2025;Park et al., 2026), which requires collecting demonstrations for every new task or hardware configuration. Such finetuned manipulation skills often fail to generalize beyond initial training distributions(Zhou et al., 2025). Our held-out protocols test exactly this transfer (Section4.1), on which the evolved harnesses maintain substantially better performance (Section4.2). A complementary line of work keeps the policy frozen and adds inference-time mechanisms: retrieving demonstrations or successful action segments to condition the policy(Park et al., 2026;Zhao et al., 2026a), or selecting among the policy’s candidate actions with a learned value function(Xie et al., 2026b;Zhang et al., 2026c). In contrast,RoboArchitectevolves complete agent interfaces from scratch that combine various mechanisms, including persistent memory, programmatic control, and policy orchestration, all custom-fit to each setup (Sections4.3and4.4).

Robotic agents.Code-as-policy methods(Liang et al., 2023;Singh et al., 2022;Huang et al., 2023)bypass trained policies, and instead use executable programs as the primary robot control, composing perception, reasoning, and motion primitives. Recent coding agents go further, using iterative code generation to solve manipulation tasks(Tsui et al., 2026;Kumar, 2026;Xie et al., 2026a). However, CaP-X(Fu et al., 2026)shows that their success depends heavily on the hand-written primitives they are given. Other recent work retains trained VLAs but places them inside agent-level orchestration. VoLo(Chen et al., 2026d)treats the policy as an interruptible tool, guiding it alongside vision models and programmatic motions. Harness VLA(Zhang et al., 2026e)assigns high-level planning to an LLM agent and restricts the frozen policy to contact-level primitives. Several systems further decompose tasks into subproblems with memory or verification mechanisms(Xu et al., 2026;Gao et al., 2026;Zhao et al., 2026b).Hu et al. (2026)find that components in such hierarchical VLA architectures (planner, controllers, and switching logic) must be co-designed to maximize performance. These works establish that the interface around a VLA matters, but their agent architectures were specified manually in advance.RoboArchitectinstead automates harness design and optimization, including the VLA/code division of labor (Section4.2) and persistent memory (Section4.4).

Automated code optimization.Coding models are increasingly used in auto-research routines that evaluate agent outputs, analyze execution traces, and iteratively refine agent implementations(Hu et al., 2025;Zhang et al., 2026b;Lee et al., 2026;Zhang et al., 2026a;Wei et al., 2026;Zhou, 2026;Cai et al., 2026). Within robotics, most LLM-driven optimization loops generate artifacts that train policy weights: reward functions (Eureka(Ma et al., 2024)), training configurations revised from execution logs (ENPIRE(Xiao et al., 2026)), or VLM-decomposed exploration data distilled into policies via finetuning (EXIMO(Sukhija et al., 2026)). These systemsautomate finetuning, and are thus complementary toRoboArchitect, whichautomates the agent built around a frozen policy. Another line of work accumulates reusable code skills: repairing and archiving control functions(Lu et al., 2026), searching over repositories of control code(Elmaaroufi et al., 2026), assembling computation graphs from skill libraries(Chen et al., 2026b), collecting skills during self-directed play(Zhang et al., 2026d), or distilling language corrections into a skill library for in-context learning(Chen et al., 2026a). Crucially, these skill systems optimize programs thatentirely replace the robot policy. In contrast,RoboArchitectdevelops the harnessarounda frozen policy, allowing the code toautonomously divide control between the policy and its own direct commands(Section4.2).Berman et al. (2026)similarly find that frontier coding models are more capable when given higher-level access to pretrained robot policies than when controlling low-level motion directly, an arrangementRoboArchitectoften evolves into on its own.

Figure 2:RoboArchitectturns a frozen policy into a performant robotic agent without updating its weights: a coding model iteratively refines the harness from execution traces.Starting from a bare policy (VLA),RoboArchitectevolves a robotic agent equipped with auxiliary tools and persistent memory. The right side illustrates experiment conditions (Section4.1) that test how well the evolved harness generalizes.

3RoboArchitect

RoboArchitectoptimizes the harness around a frozen policy, as illustrated inFig.2. It starts from a harness that passes the policy’s actions through unchanged and repeats an empirical optimization loop: the harness runs on the training episodes, a coding model observes the recorded rollouts, diagnoses the failures and refines the harness. These “training” episodes serve only to optimize the harness:no weights are updated, and the final harness is evaluated on separate held-out episodes (Section4.1). Over iterations, the harness grows into a highly performant robotic agent that, as needed, follows the policy, consults auxiliary tools, or commands the arm directly, and it can carry experience in a persistent memory. Below, we describe its key components.

Harness interface.A robotic agent harness is a program that sits between environment observations and low-level robot execution commands. At each control step, the harness receives the current environment observation, including camera images, depth maps, gripper pose, and gripper status, along with the natural language task instruction. It outputs low-level action vectors executed by the robot controller. Initially, the harness is a simple pass-through of the bare VLA policy: it forwards observations to the frozen policy and returns the policy’s predicted actions directly. TheBare Policycolumns ofTable1are thus the standard evaluation of the policy alone.

Auxiliary tools.Beyond the base policy, the harness can query auxiliary pretrained tools (Section4.1), including language models, vision-language models, object detectors, segmentation models, pointing models, and grasp pose generators. The harness also has access to geometric utilities that project pixel-depth pairs into 3D robot base coordinates, and generate relative trajectory primitives. Together, these tools give the harness alternatives to relying on the frozen policy alone.

Optimization loop.The optimization follows the same trial-and-error loop as a human engineer (Section1). Every evaluation logs complete trajectory traces, including per-step camera frames, the robot’s readings, and all model and tool outputs. A coding model reviews these execution traces from the current harness, diagnoses failure modes, and proposes a candidate harness revision, where it may follow the policy, invoke any of the tools, or command the arm programmatically. The automated optimization loop thus settles the balance between policy and programmatic control (Section4.2). The candidate is then re-evaluated on the same training set (Section4.1) and accepted only if its performance surpasses that of the current harness, following the empirical validation in prior work on self-improving agents(Zhang et al., 2026b;Zhang et al., 2026a). We thus ensure a monotonic optimization trajectory on the training set, capped at a maximum number of iterations (Section4.1). We compare iterative refinement with a single-attempt automated agent design inSection4.2.

Persistent memory.We also provide the harness with a persistent memory mechanism that carries information across episodes and tasks. Unlike fixed hand-crafted memory schemes in recent robotic agents(Zhang et al., 2026e;Xu et al., 2026;Zhao et al., 2026b;Gao et al., 2026), our persistent memory structure isautonomously evolved based on accumulated rollout evidence, and fitted to each setup. As discussed inSection4.4,RoboArchitectdiscovers diverse forms of persistent memory.

4Experiments

4.1Setup

Benchmarks and policies.We evaluate on LIBERO-Pro-10, which is LIBERO-10 (*i.e.,*LIBERO-Long)(Liu et al., 2023)with object locations swapped as in LIBERO-Pro(Zhou et al., 2025), RoboCerebra(Han et al., 2025), VLABench(Zhang et al., 2025), RoboCasa-365(Nasiriany et al., 2024;Nasiriany et al., 2026)and SimplerEnv WidowX(Li et al., 2024). The unified harness (Section4.1) is also tested on SimplerEnv Google(Li et al., 2024).AppendixBdescribes each benchmark. The VLA policies are MolmoAct-2(Fang et al., 2026), OpenVLA-7B(Kim et al., 2024),π0.5\pi_{0.5}(Intelligence et al., 2025), RLinf-π0.5\pi_{0.5}(Chen et al., 2026c;Zang et al., 2025), SmolVLA(Shukor et al., 2025), X-VLA(Zheng et al., 2026), and RLDX-1(Kim et al., 2026).

Training-source distance.We characterize each setup by the distance between the policy’s training data and the evaluation benchmark (Table1): (1) in-distribution, where the policy was tuned on this exact benchmark; (2)swapped, with object locations permuted as in LIBERO-Pro; (3)cross-benchmark, a different benchmark on the same simulator and robot; and (4)out-of-domain, with only general robot pretraining and benchmark unseen. For comparison with finetuning, we also evaluate public checkpoints of the same policy finetuned on the target benchmark on the same held-out test set (rows marked↪\hookrightarrowinTable1).

Held-out evaluation protocols.Every harness is evaluated on held-out episodes never seen during the harness’s optimization (i.e.,“training”), under the following protocols:

(1)Unseen scenes. Training and held-out splits share identical tasks but differ in initial scene arrangements and layouts (Fig.9, left). The evolved harness must generalize beyond specific object positions or spatial configurations.

(2)Unseen tasks. Training and held-out splits contain different tasks that share underlying manipulation skills (Fig.9, right). The evolved harness must generalize at the skill level rather than memorizing task-specific routines.

(3)Unified harness across benchmarks. Under the protocols above, individual agents are evolved separately for each setup, allowing them to specialize to the challenges of specific embodiments. As an extension, we also evolve asingle unified harness that works across multiple benchmarks simultaneously, spanning different robot embodiments, simulators and action layouts, with the goal of discovering shared cross-embodiment agent mechanisms and persistent memory structures. We take the benchmarks LIBERO-Pro, VLABench, SimplerEnv WidowX and SimplerEnv Google as test bed (Franka Panda, WidowX and the Google mobile manipulator). We use the following policy configurations: (1) thegeneralX-VLA checkpoint, never tuned on any of these benchmarks and thus operating entirely out of distribution; (2)X-VLA specialists, one checkpoint finetuned per benchmark. To evolve the unified harness, we build a training pool ofprocedurally generated episodes with new scene arrangements, objects, instructions or tasksoutside standard benchmark evaluations (Fig.9, top), and evaluate on the benchmarks’ own standard episodes (Fig.9, bottom).

Evaluation metrics.We report standard benchmark metrics. On VLABench, we use the official progress score, which assigns partial credit for grasping the right object and for each goal condition met; on all other benchmarks, we report success rate averaged over tasks. For simplicity, we call all these scores success rates (%) in this paper, although the VLABench numbers are progress scores. All the scores we compare, including those of the finetuned references, use the same step limits. On SimplerEnv WidowX and SimplerEnv Google, episodes always run to the limit and are scored on their final state.

Harness evolution configurations.We run at most 5 optimization iterations, usingclaude-opus-5as the coding model, invoked via the Claude Code client at maximum effort. Theauxiliary toolsare Qwen3-30B-A3B-Instruct(Yang et al., 2025)as the language model, Molmo2-8B(Clark et al., 2026)or Qwen3-VL-8B-Instruct(Bai et al., 2025)as the vision-language model, Grounding-DINO(Liu et al., 2024)for object detection, SAM 3(Carion et al., 2026)for segmentation, Molmo2-8B for pointing, and GraspGen(Murali et al., 2025)for grasp pose generation. Further implementation details are provided inAppendixB.

Table 1:RoboArchitect’s iteratively evolved harness raises mean held-out success by +345% on unseen scenes and +216% on unseen tasks.Bare Policy: the VLA policy alone.Harness (ours): the same frozen policy inside the harnessRoboArchitectdesigned, after a single design attempt (One-Shot, iteration 1) or after iterative refinement over up to 5 iterations (Iterative).Δ\Delta: gain of the iterative harness over the bare policy. Rows marked↪\hookrightarrowshow the finetuned version of the same policy evaluated on the same held-out set. Training-source distance between the policy and the evaluation benchmark is color-coded: black = in-distribution;gold= spatially swapped;orange= cross-benchmark;dark red= out-of-domain. All scores are success rates in %.Table 2:A unified evolved harness serves different benchmarks: around the general X-VLA it substantially raises held-out success, and around the finetuned X-VLA specialists it preserves their competence.Columns and labels read as inTable1.

(a)MolmoAct-2 on LIBERO-Pro-10,“put both the cream cheese box and the butter in the basket”: the VLA policy keeps driving into the juice carton until it tips over; the evolved harness detects the jam and lifts clear.

(b)X-VLA on VLABench,“Please take the book contract_law_in_japan”: the earlier harness defaults to top-down motion and collides repeatedly with the shelf board from above; the evolved harness turns the wrist and reaches in from the side.

(c)RLDX-1 on RoboCasa-365,“Close the toaster oven door”: the VLA policy leans on the toaster-oven door too gently and it stays open; the evolved harness rescales the action magnitude and latches the door shut.

(d)RLinf-π0.5\pi_{0.5}on LIBERO-Pro-10,“turn on the stove and put the moka pot on it”: the VLA policy closes on the frying pan standing at the moka pot’s usual location memorized in the original LIBERO training data; the evolved harness locates the named object, moka pot.

Figure 3:Evolved control routines fix concrete failure patterns.Each row shows an episode under an earlier harness (starting with bare VLA policy) on the left, and under the revisedRoboArchitectharness on the right. Row (a) shows detecting and clearing arm jams, (b) synthesizing missing motion primitives, (c) rescaling contact forces, and (d) redirecting to target objects instead of memorized locations.Fig.5shows more examples. See discussions on these revisions inSection4.3andAppendixA.

(a)X-VLA on SimplerEnv WidowX,“put carrot on plate”(left and middle) and*“put eggplant into yellow basket”*(right): the memory records the fractions of object height that successfully lifted the object; closing at 18% (raised to a 0.8 cm minimum) on the carrot lifts nothing, while closing at 45% of the object height (1.4 cm on the carrot, 3.9 cm on the eggplant) lifts both.

(b)OpenVLA-7B on LIBERO-Pro-10,“put the white mug on the plate and put the chocolate pudding to the right of the plate”: the memory records how often each grip style (pose) has worked for each object; the white mug isspread from the insideand stays upright, while the pudding box ispinched.

(c)RLDX-1 on RoboCasa-365,“pressdown the lever to turn on the electric kettle”(left) and*“pickthe bell pepper from the plate and place it in the pan”(right): the memory records that “pressing” tasks rarelyhold*objects while “picking” tasks do; the harness keeps pressing on the lever with closed fingers, but backs off and retries when the fingers close on nothing over the pepper.

(d)General X-VLA with the unified harness on SimplerEnv Google,“pick coke can”(left and middle) and on LIBERO-Pro,“pick the akita black bowl on the cookie box and place it on the plate”(right): the memory records the direction in which each robot’s fingers close, and the harness turns the wrist accordingly; the Google robot closes its fingers across the can whether it lies sideways (left) or points toward the camera (middle), while the Franka arm grips the bowl’s rim with one finger inside and one outside, following its correct wrist orientation.

Figure 4:Persistent memories emerge from harness evolution and inform action decisions.Each row illustrates one form of memory mechanism; rows (a) and (b) show reusable execution experience, (c) task and environment context, and (d) embodiment-specific calibration. See discussions on these memories inSection4.4andAppendixA.

4.2Harness Optimization Unlocks Substantial Robot Manipulation Headroom

An evolved harness turns a general policy into a specialist.Without any weight updates,RoboArchitectyields substantial gains across nearly all setups (Table1), and converts poorly performing cross-benchmark and out-of-domain policies (Section4.1) into functional specialists.Table1compares harness design and finetuning directly: on VLABench under both protocols and on SimplerEnv WidowX under unseen scenes,the evolved harness around the general policy outperforms the finetuned weights, carrying the frozen policy past a full weight update; it falls short only on SimplerEnv WidowX under unseen tasks. The gainstransfer strongly to unseen scenes and unseen tasks: onunseen scenes, they generalize across novel scene arrangements, with the only minor exception of RLDX-1 on RoboCasa-365, which improved only on its optimization episodes (Fig.10); every setup also transfers positively tounseen tasks(Table1). This generalization reflects the structure of the revisions themselves: control routines work frommeasured geometry and the robot’s own sensing, not from static object names or positions written into the code (Section4.3), and memories are often indexed bymeasured physical dimensions(Section4.4). These results show that the domain gap traditionally closed through demonstration collection(Kim et al., 2024;Intelligence et al., 2025)can also be closed through automated harness design.

The harness dynamically balances policy and programmatic control.In several setups, the evolved harness bypasses the frozen policy entirely on benchmarks a general policy was never trained for. For example, the general X-VLA on SimplerEnv WidowX commands a fully open gripper at every step, so the harness learns todisregard the policy entirelyand evolves into a vision-based code-as-policy agent (Fig.6i; details inAppendixA). On its optimization episodes, this harness raised success from 0% to 100% (Figs.11and10), and achieved 87.5% on unseen scenes and 66.7% on unseen tasks. In contrast, the unified harness evolved on finetuned policies (Section4.2)keeps the VLA specialists in charge and intervenes only when they fail. As shown inFig.7a, if late in an episode the gripper under the VLA policy’s control has never approached the instructed object, the harness takes over via the pointing tool, and directly commands the arm to reach and grasp it. The harness also learns to hand control back to the VLA policy if it is already closing in on the object (Fig.7b). Most evolved harnesses settle between these two ends:the policy retains the motion it executes well, while the code takes over the decisions the policy gets wrong, such as lifting a jammed arm clear before re-querying the policy (Fig.3a), or carrying the gripper to the named object before the policy grasps it (Fig.3d). Crucially, we never instructed the coding model how to divide control; each harness arrived at its structure from rollout evidence alone.

A unified harness works across robots.We next test whether a single harness can generalize across embodiments (Section4.1; results inTable2, full evolution trajectories inFig.12). The general X-VLA model scores near zero across all benchmarks, as it is entirely out of distribution (Table2). The harness then evolves into a pure code-as-policy agent, replacing the general VLA entirely (as inSection4.2). It raises every benchmark from (near) zero, reaching 41.6% on the training pool and 28.8% on held-out evaluation. This unified harness reads everything that differs between robots at runtime from the action specification and proprioception, never hardcoding embodiment-specific constants (Sections4.3and4.4). It thusoperates across embodiments without domain-specific finetuning. Using finetuned specialists as base policies, the unified harness yields smaller gains. These policies already perform well (58.0% training, 45.9% held-out), and so our auto-evolved harnesscorrectly preserves their native competence. As such, the unified harness strikes the same balance as individual setups (Section4.2): it evolves into a pure code-as-policy agent when wrapping the general VLA, but correctly preserves native competence when wrapping finetuned specialists.

Iterative refinement further improves the automatically designed harness.A coding model can already design an agent from scratch in one attempt. But the agent itself generates new evidence through its own execution, and the same optimization process ofRoboArchitectcan use that evidence to refine it further. The harness after the first iteration (One-ShotinTables1and2) represents aone-shot agent design: the coding model sees the bare VLA policy’s rollout failures (iteration 0) and produces an agent in a single attempt (iteration 1). Subsequent iterations observe the behavior of this agent and refine its control logic based on newly exposed failure modes (Section3), up to iteration 5 (IterativeinTables1and2).Figs.10,11and12illustrate these optimization trajectories, showingsteady, mostly monotonic progressionover iterations. As shown inTable1, the iteratively evolved harness outperforms the one-shot design on the training set in every setup (Figs.10and11) and on held-out sets, except RLDX-1 on RoboCasa-365 under unseen scenes. For example, the code-as-policy agent on VLABench is built in its first attempt, and only its own rollouts reveal subtler failures, which later revisions repair (Fig.6g,h; details inAppendixA).Iterative refinement also improves generalization: while some one-shot designs fall below the bare VLA policy on unseen scenes or tasks, the iteratively evolved harness surpasses it on nearly every held-out set (Table1). The improvement is not limited to adding new tools: later revisions also refine the division of control between the policy and programmatic logic, as discussed inSection4.2. These results show that the harness is not merely automatically designed, but caniteratively self-improveby inspecting the execution traces of its own previous designs.

4.3Evolved Control Routines Driving the Performance Gains

The evolved harnesses address concrete failure modes with targeted control routines.Figs.3and6show representative revisions across benchmarks. The harnesses learn todetect and clear arm jamsthat the policy cannot recover from (Fig.3a), and tosynthesize missing motion primitives, such as a side approach under a shelf board or a handle pull on a cabinet (Fig.3b,Fig.5e). On RoboCasa-365, theyrescale contact forceswhen the policy pushes a door too gently (Fig.3c), andtether the mobile baseto active workspaces when it drifts away (Fig.5f). Other routines make perception more reliable: the harness redirects the arm to the target object when a policy overfit to its training scenes reaches for a memorized location (Fig.3d), andadapts to perception limitationsby working down ranked candidates and checking for firm grasps from depth (Fig.6g,h). The unified harness alsoadapts to each robot’s setup, such as its speed: it ends each move when the arm arrives, not on a fixed time budget (Fig.7c), which contributed to raising the training-pool success rate on the slower Google robot from 11.0% to 30.2%.AppendixAdetails each routine.

4.4Persistent Memory Emerges from Harness Evolution

Across nearly all setups,RoboArchitect’s evolved harnesses develop persistent memory structures that carry experience across episodes.Table3lists examples of memory entries and how often each held-out episode reads the memory. Rather than following any predefined memory schema,RoboArchitectdiscovers diverse forms of memories that capturereusable execution experience, task and environment context, and embodiment-specific calibration.

Table 3:Auto-designed persistent memories are frequently accessed and yield highly useful information to guide later actions.Reads per Episode: average frequency of memory reads in held-out episodes.Actionable Reads: the share of these reads that find stored info.Memorieslearn from the harness’s own grasps: one records thegrasp heightsthat lifted objects, relative to each object’s measured height, and consults them on new objects (Fig.4a); another remembers thegrip style(pinchorspread) that held each object (Fig.4b). Indexed by measured physical properties or object identity, this experience carries over to novel objects and new tasks. Memories also trackimplicit task and environment clues, such as which instructions needholdingobjects (Fig.4c) and the 150-step scene-reset period of RoboCerebra. The unified harnesscalibrates itself to each robot, recording which way the robot’s fingers close to orient later grasps (Fig.4d).AppendixAdetails each type of memory.

Importantly, these memories aredata structures within the designed robot agent, not the coding model’s own context: they accumulate information during the agent’s rollouts and persist across episodes at execution time. As shown inTable3, the harnessesactively consult their memoryseveral times per episode on average, andover 90% of these reads are actionable, that is, they successfully find a stored entry, even on held-out episodes unseen during harness optimization. Because every evaluation starts with an empty memory to be accumulated, these entries’ designs effectively carry over to held-out episodes and remain highly useful.

5Conclusion

We introducedRoboArchitectto automate the design and optimization of agent harnesses around frozen robot policies. Across diverse benchmarks and frozen policies, the evolved harnesses substantially improve performance on held-out scenes and tasks. Even when the policy is untuned for the target environment,RoboArchitectcan adapt it to a specialist in out-of-domain settings, and can even outperform its finetuned version. Crucially, the harness continues to self-improve beyond its initial design, using the behavior of previous versions as new evidence for subsequent refinements. Emergent from harness optimization, we observe intuitive control routines that resolve specific failure modes, and persistent memories that carry experience across episodes to inform later decisions.RoboArchitectcan also evolve a unified harness that works across multiple embodiments, adapting itself to each robot. Together, these results suggest that substantial headroom in robot manipulation resides in the agent harness, and that this headroom can be unlocked by automatically evolving the harness for novel setups.

6Limitations

Scaling cap of iterations.As shown inFigs.10,11and12, harness performance on the training episodes improves consistently across iterations. However, this work uses a fixed maximum of 5 optimization iterations (Section4.1). More iterations may yield further gains, but the point at which returns saturate remains to be explored. Nevertheless, even with a moderate budget of only 5 iterations (AppendixB),RoboArchitectachieves substantial gains that transfer to held-out settings.

More complex manipulation.A natural next step is extendingRoboArchitectto tasks with richer contact dynamics, such as dexterous in-hand manipulation or tool use, where the space of failure modes is larger than in pick-and-place. The range of control routines and memory structures that automated evolution can discover in these settings is an interesting open question.

Sim-to-real transfer.All evaluations run in simulation. Because our harness reads only the standard input space (*e.g.,*camera images, depth and the robot’s own readings), its interface should carry over to a real robot. Testing our simulation-evolved harnesses on real robots and investigating the number of further improvement iterations on a real deployment are future work.

References

• Baiet al.(2025)S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge,et al.Qwen3-vl technical report.arXiv preprint arXiv:2511.21631.Cited by:§4.1.
• Bermanet al.(2026)S. Berman, M. Ilie, J. Deng, and D. FreemanHow claude performs on robotics tasks.Note:Anthropic Research Blog,https://www.anthropic.com/research/claude-plays-roboticsCited by:§2.
• Blacket al.(2026)K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinskyπ0\pi_{0}: A vision-language-action flow model for general robot control.External Links:2410.24164,LinkCited by:§2.
• Brohanet al.(2023)A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn,et al.Rt-2: vision-language-action models transfer web knowledge to robotic control.arXiv preprint arXiv:2307.15818.Cited by:§2.
• Caiet al.(2026)Q. Cai, Y. Zhang, X. Jia, H. Zheng, W. Xue, J. Song, X. Tian, and Y. GuoMoss: self-evolution through source-level rewriting in autonomous agent systems.arXiv preprint arXiv:2605.22794.Cited by:§2.
• Carionet al.(2026)N. Carion, L. Gustafson, Y. Hu, S. Debnath, R. Hu, D. Suris Coll-Vinent, C. Ryali, K. V. Alwala, H. Khedr, A. Huang,et al.Sam 3: segment anything with concepts.InInternational conference on learning representations,Vol.2026,pp. 138846–138923.Cited by:§4.1.
• Chenet al.(2026a)D. Chen, A. R. Jain, E. Goossen, E. Romig, M. Murray, N. Walker, and M. CakmakA few words go a long way: language guided robot policy synthesis.arXiv preprint arXiv:2607.23784.Cited by:§2.
• Chenet al.(2026b)K. Chen, S. Xie, L. Fu, J. Yu, W. Pacini, S. Bajamahal, H. Kim, J. Drake, D. Kim, H. Xue,et al.GaP: a graph-as-policy multi-agent self-learning harness for variational automation tasks.arXiv preprint arXiv:2607.05369.Cited by:§2.
• Chenet al.(2026c)K. Chen, Z. Liu, T. Zhang, Z. Guo, S. Xu, H. Lin, H. Zang, X. Li, Q. Zhang, Z. Yu, G. Fan, T. Huang, Y. Wang, and C. YuπRL\pi_{\texttt{RL}}: Online rl fine-tuning for flow-based vision-language-action models.External Links:2510.25889,LinkCited by:§4.1.
• Chenet al.(2026d)S. Chen, H. Hadfield, A. Zook, M. A. Uy, C. H. Song, E. Coumans, X. Yang, F. Ladhak, Q. Qu, S. Birchfield,et al.VoLo: a physical orchestrator for open-vocabulary long-horizon manipulation.arXiv preprint arXiv:2606.07723.Cited by:§1,§2.
• Clarket al.(2026)C. Clark, J. Zhang, Z. Ma, J. S. Park, R. Tripathi, S. Lee, M. Salehi, J. Ren, C. D. Kim, Y. Yang,et al.Molmo2: open weights and data for vision-language models with video understanding and grounding.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp. 28652–28668.Cited by:§4.1.
• Elmaaroufiet al.(2026)K. Elmaaroufi, J. Svegliato, S. Kalade, G. Schelle, S. A. Seshia, and M. ZahariaRHO: your coding agent is secretly a roboticist.arXiv preprint arXiv:2606.16458.Cited by:§A.1,§1,§2.
• Fanget al.(2026)H. Fang, J. Duan, D. Clay, S. Wang, S. Liu, W. Huang, X. Fan, W. Tsai, S. Chen, Y. R. Wang,et al.Molmoact2: action reasoning models for real-world deployment.arXiv preprint arXiv:2605.02881.Cited by:§2,§4.1.
• Fuet al.(2026)L. Fu, J. Yu, K. El-Refai, E. Kou, H. Xue, H. Huang, W. Xiao, G. Wang, D. Niu, F. Li,et al.CaP-x: a framework for benchmarking and improving coding agents for robot manipulation.arXiv preprint arXiv:2603.22435.Cited by:§2.
• Gaoet al.(2026)H. Gao, Z. Ni, X. Wen, S. Xu, and R. LiAGM: achievement-grounded memory for closed-loop agents with frozen vla policies.arXiv preprint arXiv:2608.29537.Cited by:§1,§2,§3.
• Hanet al.(2025)S. Han, B. Qiu, Y. Liao, S. Huang, C. Gao, S. Yan, and S. LiuRoboCerebra: a large-scale benchmark for long-horizon robotic manipulation evaluation.External Links:2506.06677,LinkCited by:Appendix B,§4.1.
• Huet al.(2026)J. Hu, M. Shridhar, C. Lu, D. Shah, H. L. Chiang, J. Tan, and A. XieWhat matters in orchestrating robot policies: a systematic study of hierarchical vla agents.arXiv preprint arXiv:2606.10267.Cited by:§1,§2.
• Huet al.(2025)S. Hu, C. Lu, and J. CluneAutomated design of agentic systems.InInternational Conference on Learning Representations,Vol.2025,pp. 21344–21377.Cited by:§2.
• Huanget al.(2023)W. Huang, C. Wang, R. Zhang, Y. Li, J. Wu, and L. Fei-FeiVoxposer: composable 3d value maps for robotic manipulation with language models.arXiv preprint arXiv:2307.05973.Cited by:§2.
• Intelligenceet al.(2025)P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, M. Y. Galliker, D. Ghosh, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, D. LeBlanc, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, A. Z. Ren, L. X. Shi, L. Smith, J. T. Springenberg, K. Stachowicz, J. Tanner, Q. Vuong, H. Walke, A. Walling, H. Wang, L. Yu, and U. Zhilinskyπ0.5\pi_{0.5}: A vision-language-action model with open-world generalization.External Links:2504.16054,LinkCited by:§1,§2,§4.1,§4.2.
• Kimet al.(2026)D. Kim, H. Jang, M. Koo, S. Jang, T. Kim, B. Kim, B. Yoon, C. Jang, D. Choi, D. Han,et al.Rldx-1 technical report.arXiv preprint arXiv:2605.03269.Cited by:§2,§4.1.
• Kimet al.(2024)M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi,et al.Openvla: an open-source vision-language-action model.arXiv preprint arXiv:2406.09246.Cited by:§1,§1,§2,§4.1,§4.2.
• Kumar (2026)V. KumarAct-observe-rewrite: multimodal coding agents as in-context policy learners for robot manipulation.arXiv preprint arXiv:2603.04466.Cited by:§2.
• Leeet al.(2026)Y. Lee, R. Nair, Q. Zhang, K. Lee, O. Khattab, and C. FinnMeta-harness: end-to-end optimization of model harnesses.arXiv preprint arXiv:2603.28052.Cited by:§2.
• Liet al.(2024)X. Li, K. Hsu, J. Gu, K. Pertsch, O. Mees, H. R. Walke, C. Fu, I. Lunawat, I. Sieh, S. Kirmani,et al.Evaluating real-world robot manipulation policies in simulation.arXiv preprint arXiv:2405.05941.Cited by:Appendix B,§4.1.
• Lianget al.(2023)J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. ZengCode as policies: language model programs for embodied control.In2023 IEEE International conference on robotics and automation (ICRA),pp. 9493–9500.Cited by:§A.1,§1,§2.
• Liuet al.(2023)B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. StoneLibero: benchmarking knowledge transfer for lifelong robot learning.Advances in Neural Information Processing Systems36,pp. 44776–44791.Cited by:Appendix B,§4.1.
• Liuet al.(2024)S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su,et al.Grounding dino: marrying dino with grounded pre-training for open-set object detection.InEuropean conference on computer vision,pp. 38–55.Cited by:§4.1.
• Luet al.(2026)R. Lu, Y. Wu, E. Kou, L. Fu, W. Xiao, A. Mandlekar, Y. Xu, G. Shi, K. Goldberg, A. Chen, M. Chowdhury, Y. Zhu, L. “. Fan, and G. WangASPIRE: agentic /skills discovery for robotics.External Links:2607.00272,LinkCited by:§1,§2.
• Maet al.(2024)Y. J. Ma, W. Liang, G. Wang, D. Huang, O. Bastani, D. Jayaraman, Y. Zhu, J. Fan,et al.Eureka: human-level reward design via coding large language models.InInternational conference on learning Representations,Vol.2024,pp. 26516–26560.Cited by:§2.
• Muraliet al.(2025)A. Murali, B. Sundaralingam, Y. Chao, W. Yuan, J. Yamada, M. Carlson, F. Ramos, S. Birchfield, D. Fox, and C. EppnerGraspgen: a diffusion-based framework for 6-dof grasping with on-generator training.arXiv preprint arXiv:2507.13097.Cited by:§4.1.
• Nasirianyet al.(2024)S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, A. Joshi, A. Mandlekar, and Y. ZhuRobocasa: large-scale simulation of everyday tasks for generalist robots.arXiv preprint arXiv:2406.02523.Cited by:Appendix B,§4.1.
• Nasirianyet al.(2026)S. Nasiriany, S. Nasiriany, A. Maddukuri, and Y. ZhuRoboCasa365: a large-scale simulation framework for training and benchmarking generalist robots.External Links:2603.04356,LinkCited by:Appendix B,§4.1.
• Parket al.(2026)J. Park, J. Park, T. Kim, S. Choi, D. Han, and S. YunRetrieve, don’t retrain: extending vision language action models to new tasks at test time.arXiv preprint arXiv:2606.15631.Cited by:§1,§2.
• Shukoret al.(2025)M. Shukor, D. Aubakirova, F. Capuano, P. Kooijmans, S. Palma, A. Zouitine, M. Aractingi, C. Pascal, M. Russi, A. Marafioti,et al.Smolvla: a vision-language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844.Cited by:§2,§4.1.
• Singhet al.(2022)I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. GargProgprompt: generating situated robot task plans using large language models.arXiv preprint arXiv:2209.11302.Cited by:§2.
• Sukhijaet al.(2026)B. Sukhija, O. Groth, M. Shridhar, T. Hertweck, M. Bloesch, M. Wulfmeier, A. Abdolmaleki, and M. RiedmillerEximo: vlm guided exploration of vla policies.arXiv preprint arXiv:2608.19891.Cited by:§2.
• Tsuiet al.(2026)B. Y. Tsui, A. Y. Fang, and T. J. HwuDemonstration-free robotic control via llm agents.External Links:2601.20334,LinkCited by:§A.1,§2.
• Weiet al.(2026)T. Wei, Z. Shi, M. Lin, B. He, Z. Liu, Y. Sang, Y. Bei, X. Ning, J. Zou, T. Li,et al.Evo-harness: context-to-harness skill compilation for self-evolving agents.arXiv preprint arXiv:2608.15071.Cited by:§2.
• Xiaoet al.(2026)W. Xiao, J. Xie, T. Zhang, H. Lin, L. Fu, H. Xue, J. Lu, Y. Yang, C. Dai, Z. Wang,et al.ENPIRE: agentic robot policy self-improvement in the real world.arXiv preprint arXiv:2606.19980.Cited by:§2.
• Xieet al.(2026a)S. Xie, K. Chen, and K. GoldbergRevisiting the “push-t” robot manipulation task with agentic robotics.External Links:2608.18227,LinkCited by:§2.
• Xieet al.(2026b)X. Xie, Z. Hu, Z. Liu, Y. Dong, W. Wu, Z. Sun, H. Li, C. Chen, Z. Wang, and P. WangLook before you leap: distilling tree search into action evaluation for frozen vla models.arXiv preprint arXiv:2607.03751.Cited by:§2.
• Xuet al.(2026)B. Xu, Y. Shang, and E. FerraraDon’t drop the baton: long-horizon robot manipulation via agentic subtask exploration and transition-aware memory.arXiv preprint arXiv:2608.16889.Cited by:§1,§2,§3.
• Yanget al.(2025)A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv,et al.Qwen3 technical report.arXiv preprint arXiv:2505.09388.Cited by:§4.1.
• Zanget al.(2025)H. Zang, M. Wei, S. Xu, Y. Wu, Z. Guo, Y. Wang, H. Lin, P. Wang, L. Shi, Y. Xie,et al.RLinf-vla: a unified and efficient framework for reinforcement learning of vision-language-action models.arXiv preprint arXiv:2510.06710.Cited by:§1,§2,§4.1.
• Zhanget al.(2026a)H. Zhang, S. Zhang, K. Li, C. Zhang, Y. Chen, Y. Zhang, L. Bai, and S. HuSelf-harness: harnesses that improve themselves.arXiv preprint arXiv:2606.09498.Cited by:§2,§3.
• Zhanget al.(2026b)J. Zhang, S. Hu, C. Lu, R. Lange, and J. CluneDarwin gödel machine: open-ended evolution of self-improving agents.InInternational Conference on Learning Representations,Vol.2026,pp. 104223–104294.Cited by:§2,§3.
• Zhanget al.(2026c)J. Zhang, J. Nie, J. Lao, W. Cheng, C. Liu, J. Jiang, and S. HuangWhat frozen vlas already know about success: a probing study of value-like structure in foundation robot policies.arXiv preprint arXiv:2605.28527.Cited by:§2.
• Zhanget al.(2026d)J. Zhang, J. Ge, H. Yoo, L. Fu, Z. Yang, Y. Liu, R. Saravanan, S. Yin, J. Yu, D. Niu,et al.Playful agentic robot learning.arXiv preprint arXiv:2606.19419.Cited by:§2.
• Zhanget al.(2025)S. Zhang, Z. Xu, P. Liu, X. Yu, Y. Li, Q. Gao, Z. Fei, Z. Yin, Z. Wu, Y. Jiang,et al.Vlabench: a large-scale benchmark for language-conditioned robotics manipulation with long-horizon reasoning tasks.In2025 IEEE/CVF International Conference on Computer Vision (ICCV),pp. 11142–11152.Cited by:Appendix B,§4.1.
• Zhanget al.(2026e)Y. Zhang, H. Zhang, F. Gao, X. Li, Z. Liu, C. Zhu, J. Qiu, Y. Yan, J. Liu, W. Tang,et al.Harness vla: steering frozen vlas into reliable manipulation primitives via memory-guided agents.arXiv preprint arXiv:2607.08448.Cited by:§1,§2,§3.
• Zhaoet al.(2026a)J. Zhao, H. Yang, Y. Hu, Y. Gao, Q. Ou, C. Wan, S. Dong, Z. Ma, and Y. GongRetrieve-then-steer: online success memory for test-time adaptation of generative vlas.arXiv preprint arXiv:2605.10094.Cited by:§2.
• Zhaoet al.(2026b)Y. Zhao, Z. Ni, H. Chen, R. Zhang, and Q. ZhuSkills in weights, memory in code: hybrid learning for memory-dependent robot manipulation.arXiv preprint arXiv:2608.09410.Cited by:§1,§2,§3.
• Zhenget al.(2026)J. Zheng, J. Li, Z. Wang, D. Liu, X. Kang, Y. Feng, Y. Zheng, J. Zou, Y. Chen, J. Zeng,et al.X-vla: soft-prompted transformer as scalable cross-embodiment vision-language-action model.InInternational Conference on Learning Representations,Vol.2026,pp. 60580–60606.Cited by:§2,§4.1.
• Zhou (2026)T. ZhouHierarchical self-improvement: a framework for task-specific evolvable agent harnesses.arXiv preprint arXiv:2608.08466.Cited by:§2.
• Zhouet al.(2025)X. Zhou, Y. Xu, G. Tie, Y. Chen, G. Zhang, D. Chu, P. Zhou, and L. SunLibero-pro: towards robust and fair evaluation of vision-language-action models beyond memorization.arXiv preprint arXiv:2510.03827.Cited by:§A.1,Appendix B,§2,§4.1.

Appendix AEvolved Control Routines and Persistent Memories in Detail

This appendix detailsRoboArchitect’s auto-evolved control routines ofSection4.3and the persistent memories ofSection4.4;Figs.5,6and7show more of the revisions.

A.1Control Routines

Detecting and clearing arm jams.When the arm is jammed against an object, the bare policy may continue issuing similar commands because its visual observations remain nearly unchanged during a jam, so it tips objects over and cannot recover. The evolved harness learns to monitor discrepancies between commanded and actual gripper movements; when disagreement persists, it lifts the arm clear before re-querying the policy (Fig.3a).

Synthesizing missing motion primitives.Some tasks require motion primitives absent from the base policy or the earlier harness, such as approaching an object from the side rather than from above. For retrieving a book from the shelf on VLABench, the earlier harness defaults to top-down motion and collides with the overhead board. The evolved harness detects overhead obstructions via depth maps and computes horizontal approach vectors to reach beneath them (Fig.3b). The same pattern appears elsewhere: on RoboCerebra, the harness adds a wrist-rotation primitive for pouring withπ0.5\pi_{0.5}, while for cabinet opening with SmolVLA, it locates the cabinet with the pointing model, finds its handles in the depth image, then aligns and pulls (Fig.5e).

Rescaling contact forces.On RoboCasa-365, some failures arise frominsufficient contact forcerather than trajectory misalignment. For example, the toaster-oven door requires closure within 0.5% of its opening range, but the policy RLDX-1 pushed too gently. The evolved harness rescales action magnitudes while preserving direction, providing sufficient force to latch the door (Fig.3c).

Tethering mobile bases to active workspaces.On RoboCasa-365, the harness also learns to constrainmobile-base driftaround active workspaces. It defines workspace anchors as locations where the last gripper activations occurred, gives the base a driving budget, and retraces the base to the latest anchor once the budget is spent (Fig.5f).

Redirecting to target objects instead of memorized locations.For spatially shifted scenes such as LIBERO-Pro, policies overfit on original LIBERO consistently reach toward wrong items at memorized locations(Zhou et al., 2025).RoboArchitectindependently diagnosed this memorization, and evolved the harness to query the pointing model for the target item. With RLinf-π0.5\pi_{0.5}, it learns to locate the instructed moka pot rather than reaching to its memorized position (Fig.3d).

Replacing an unusable VLA policy.The base (un-finetuned) X-VLA on SimplerEnv WidowX commanded a fully open gripper at every step, producing no useful control signal.RoboArchitectlearned todisregard the policy entirely, as its commands barely change with the robot’s observations. Instead, the harness evolved into a complete vision-based picking agent, akin to code-as-policy(Liang et al., 2023;Tsui et al., 2026;Elmaaroufi et al., 2026): parsing instructions, detecting target objects via open-vocabulary detectors, estimating 3D poses from depth maps, computing grasp vectors, and executing trajectories directly (Fig.6i).

Adapting to perception limitations.When the VLA policy is unusable, the harness evolves into a pure code-as-policy agent (Section4.2). Crucially, such agents continue torefine their own perception and control mechanismsacross iterations (Section4.2). On VLABench, the vision tools could not reliably distinguish mahjong tile faces. Rather than relying on a single vision prediction, the harness maintains a ranked candidate list, sequentially interacting with candidate tiles until the task completes (Fig.6g). Vision-based grasp verification could also be unreliable, causing the agent to prematurely put down objects it was holding. The harness thus replacesvisual grasp checkswitha depth check under the fingertips, verifying grasps by the presence of a surface just below the fingers (Fig.6h).

Per-robot control-rate independence.The unified harness (Section4.2) evolves mechanisms that adapt control to each robot across platforms (seeFig.7). Each motion phase used to end on a fixed time budget, which on a slower platform (*e.g.,*the 3 Hz Google robot,Fig.7c) expires while the arm is still in transit. The evolved harness ends a move only when the arm arrives or stops making progress, measured by the distance between the arm’s position and its target rather than by elapsed time, and reaches handles on vertical surfaces from the front along the measured surface normal. The revision that introduced these changes raised the agent’s performance on the Google robot from 11.0% to 30.2% on the training pool.

(e)SmolVLA on RoboCerebra,“Open the wooden cabinet top region”: the VLA policy fails entirely, missing required motions; the evolved harness correctly locates the cabinet, closes on the handle and pulls the drawer out.

(f)RLDX-1 on RoboCasa-365,“Open the cabinet, pick up the bread from the cabinet and place it in the basket. Then move the basket to the dining counter.”: the VLA policy drifts the base away and never returns to the counter; the evolved harness detects the drift and retraces back to the counter.

Figure 5:More evolved control routines.Rows read as inFig.3. Row (e) shows synthesizing missing motion primitives, and (f) tethering mobile bases to active workspaces.

(g)X-VLA on VLABench,“Pick up the mahjong of 1 pin”: the earlier harness commits to just a single tile; the evolved harness works down its ranked candidate list of multiple tiles.

(h)X-VLA on VLABench,“It’s a hot day and I could use some refreshment”: the earlier harness judges its grip empty after lifting the drink and puts it down; the evolved harness verifies grasps from the depth image under the fingertips and keeps hold of the drink.

(i)X-VLA on SimplerEnv WidowX,“put the spoon on the towel”: the VLA policy (un-tuned and out of distribution) fails entirely;RoboArchitectreplaces it with a code-as-policy agent that picks and places the spoon correctly.

Figure 6:Evolved harnesses also enhance the robotic agent’s perception precision and reliability.Panels read as inFig.3, discussions inSection4.3. Rows (g) and (h) show adapting to perception limitations, and (i) replacing an unusable policy with a code-as-policy agent (Section4.2).

(a)Finetuned X-VLA on VLABench,“Please put the white in any hole.”: the VLA policy sweeps above the table and never descends; the revised harness borrows the arm late in the episode and brings the gripper down to the ball.

(b)Finetuned X-VLA on SimplerEnv Google,“open bottom drawer”: the earlier harness takes the arm over after the drawer is already open and pushes it shut again; the revised harness no longer takes the arm from a policy that has already reached its target, and the drawer stays open.

(c)General X-VLA on SimplerEnv Google,“open bottom drawer”: the earlier harness presses the hand onto the drawer front and times out on a clock set for a faster robot; the revised harness ends moves on arrival and reaches the handle from the front along the surface normal.

Figure 7:The unified harness learns to take over the arm command from a failing policy and to leave a progressing one alone, and calibrates itself to each robot’s configurations.Each row shows an episode under the harness before (left) and after (right) an iteration’s revision.

A.2Persistent Memories

Learning execution parameters from experience.The harnesses store empirical execution statistics that inform later control decisions. On SimplerEnv WidowX, the harness records the grasp heights that have successfully lifted an object, as a function of measured object height, so that effective grasp parameters transfer to novel objects (Fig.4a). On LIBERO-Pro-10, the MolmoAct-2 harness instead averages the depth of successful grasps below the object top, as a share of the object’s height, and uses this statistic to parameterize fallback grasps. The OpenVLA-7B harness similarly records the grip style,pinchorspread, that succeeds for each object, so future grasps select the proven pose when the object’s shape allows both (Fig.4b). Crucially, across these designs, memory is indexed byactual, measured physical properties or object identityrather than specific task names, so experience generalizes to novel objects across different tasks.

Tracking implicit task and environment clues.Persistent memory also captures recurring clues in the task instruction and environment. On RoboCasa-365, the harness associates instruction verbs with the need tohold (“pick up”)an object: once sufficient evidence accumulates for a“picking”task, it enables anempty-grasp recovery routinethat retries when fingers close on nothing, while leaving the routine disabled for*“pressing”*tasks where a closed empty gripper is intentional (Fig.4c). On RoboCerebra, the harness discovers and stores the environment’s 150-step scene-reset period, allowing later episodes to anticipate resets and take over a placement early enough to finish it before the reset. Memories are also used to cache inference outputs, such as VLM/LLM parses of multi-step instructions, so that repeated executions avoid redundant computation.

Persistent self-calibration across embodiments.In the unified harness (Section4.2), persistent memory additionally stores embodiment-specific calibration. Because finger motion differs across robots, the harness opens and closes the empty gripper, observes the resulting motion in the depth image, andstores the kinematic mapping in memory. This calibration guides grasp orientation across different embodiments (*e.g.,*on the Franka arm and on the Google robot) (Fig.4d). The same routine records the resting finger configuration, providing a reference for detecting empty grasps on robots reporting finger positions.

Appendix BImplementation Details

Benchmarks.LIBERO-Pro-10 evaluates long-horizon task sequences from LIBERO(Liu et al., 2023)with initial object locations swapped while preserving original natural language instructions. These spatial swaps follow the LIBERO-Pro benchmark protocol(Zhou et al., 2025)designed to distinguish genuine manipulation skill from spatial memorization via overfitting on LIBERO. RoboCerebra(Han et al., 2025)tests on a sequence of short manipulation tasks with scene resets every 150 control steps, challenging the ability to handle sudden environment changes. VLABench(Zhang et al., 2025)requires identifying specific target items among visually similar distractors with a focus on visual grounding. RoboCasa-365(Nasiriany et al., 2024;Nasiriany et al., 2026)places a mobile manipulator in varied kitchen environments, requiring coordinated mobile navigation and arm control. SimplerEnv WidowX(Li et al., 2024)deploys a WidowX manipulator within photorealistic scenes, testing cross-embodiment transfer under visual domain shift. SimplerEnv Google(Li et al., 2024)deploys the Google mobile manipulator under the same visual domain shift as SimplerEnv WidowX, and is included in the test bed of the unified harness (Section4.1).

Figure 8:Held-out episodes differ from the optimization episodes either in scene arrangement (unseen scenes) or in the task itself (unseen tasks)(Section4.1).*Left, unseen scenes:*the same tasks with scene configurations dealt out alternately.*Right, unseen tasks:*different tasks that require similar underlying skills.

Figure 9:The unified harness is evolved only on a training pool of procedurally generated episodes and evaluated on the benchmarks’ own standard episodes(Sections4.1and4.2).Top, optimized on:procedurally generated scenes, objects, instructions & tasks form the training pool.Bottom, held out:the benchmarks’ own standard episodes, unseen during harness evolution.Episode splits.Fig.9shows the split of a benchmark’s episodes under theunseen scenesandunseen tasksprotocols, andFig.9shows the training pool and the held-out episodes of the unified harness (Section4.1).Table4lists, for each benchmark, the tasks used for the harness evolution and the held-out tasks under theunseen tasksprotocol. The held-out tasks share underlying manipulation skills with the harness optimization tasks. The task split of a benchmark is shared by all the policies evaluated on it (Table1). Every task is run for the same number of trials (i.e.,“episodes”), as listed next to each benchmark. The unified harness (Section4.2) is evolved on a training pool of 2,201 procedurally generated episodes, and evaluated on the 2,160 standard episodes.

Table 4:Optimization and held-out tasks under theunseen tasksprotocol.On RoboCerebra a task is a chain of sub-task instructions stated here in brief. On VLABench and RoboCasa-365 the instruction of a task changes with the sampled objects.Trialsis the number of trials (*i.e.,*episodes) per task.Cost of automated harness optimization.A complete evolution of 5 iterations (Section4.1) takes under 6 hours of coding-model time and stays under $150 on average, with no demonstrations collected and no model weights updated. ThroughRoboArchitect, the engineering loop that is slow by hand (Section1) becomes an inexpensive routine easily repeatable for every new robot setup.

Appendix CHarness Evolution Trajectories

Figs.10,11and12show the complete optimization update chain for every experimental setting under both held-out evaluation protocols (Section4.1) and the unified agent design (Section4.2).

Figure 10:Underunseen scenes, the harness improves over iterations on the training set, and the final harness outperforms the one-shot harness on nearly every held-out set.Left of the dotted line is the training episode set, iteration by iteration; iteration 0 is the bare VLA policy, filled markers denote accepted harness revisions that improved task success over prior iterations, and open markers denote revisions that were not accepted. Right of the dotted line are the scores on the held-out set: the bare policy, the harness after iteration 1 (a single-attempt agent design) and the final iteratively refined harness. The held-out set evaluates identical tasks under different scene arrangements (Section4.1).Figure 11:Underunseen tasks, the harness improves over iterations on the training set, and the final harness outperforms the bare VLA policy and the one-shot harness on every held-out set.Labels and panels read as inFig.10. A cross marks a revision that could not be evaluated; when this is the first revision, no single-attempt harness is available for held-out evaluation. The held-out set evaluates unseen tasks requiring shared underlying skills (Section4.1).Figure 12:Around the general X-VLA, the unified harness improves on every benchmark of the training pool over iterations; around the finetuned specialists, it leaves their already competent scores intact.Left column, using the general X-VLA checkpoint as policy; right column, using benchmark-finetuned X-VLA specialists as policy. The first row is the mean over the benchmarks and each later row is one benchmark. Left of the dotted line shows the performance on the procedurally generated training pool, iteration by iteration (iteration 0 is the bare policy; filled markers denote accepted harness revisions). Right of it are the scores on the benchmark’s standard episodes (unseen in the training pool): the bare policy, the harness after iteration 1 (a single-attempt agent design) and the final iteratively refined harness.