From VLM Planners to Robotic Agents: Self-Evolving Harnesses for History-Dependent Manipulation

Hugging Face Daily Papers Papers
robotics vision-language-models vision-language-action long-horizon-manipulation agentic-planning embodied-ai self-evolution

Summary

论文提出 SERA（Self-Evolving Robotic Agent），通过可编辑的 harness 维护任务上下文并利用执行反馈自我进化规划，配合固定的 VLA 执行器来改进长时程机器人操作任务，在 RoboMME 上取得 51.38% 的成功率。

Vision-language-action (VLA) models achieve strong performance across embodied manipulation tasks by unifying visual understanding, language conditioning, and action generation. However, long-horizon tasks can involve non-Markovian dependencies: successful decisions may require retaining task progress and past interaction outcomes, while partial observability and execution errors compound over time. Existing work has begun to address these challenges by augmenting foundation-model policies with memory and by introducing hierarchical architectures that separate high-level planning from low-level execution. We organize the high-level VLM as a robotic agent through an editable harness to maintain task context and improve planning through execution experience. We instantiate this approach as SERA (Self-Evolving Robotic Agent), with a fixed VLA executor providing low-level control. The harness maintains and updates task context using observations and execution feedback to guide subsequent planning. Self-evolution uses environment rollouts to propose and evaluate harness revisions, with the selected configuration fixed before held-out testing. On RoboMME, SERA achieves 51.38% overall success with autonomous selection, compared with 44.38% without self-evolution. Our exploration suggests that agentic planning is a promising direction for history-dependent robotic manipulation.
Original Article
View Cached Full Text

Cached at: 10/22/26, 03:37 PM

Paper page - From VLM Planners to Robotic Agents: Self-Evolving Harnesses for History-Dependent Manipulation

Source: https://huggingface.co/papers/2610.29560 Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

SERA improves long-horizon robotic manipulation by using an editable harness to maintain task context and evolve planning through execution feedback, paired with a fixed low-level VLA executor.

Vision-language-action (VLA) models achieve strong performance across embodied manipulation tasks by unifying visual understanding, language conditioning, and action generation. However, long-horizon tasks can involvenon-Markovian dependencies: successful decisions may require retaining task progress and past interaction outcomes, while partial observability and execution errors compound over time. Existing work has begun to address these challenges by augmenting foundation-model policies with memory and by introducinghierarchical architecturesthat separate high-level planning from low-level execution. We organize the high-levelVLMas a robotic agent through an editable harness to maintaintask contextand improve planning through execution experience. We instantiate this approach asSERA(Self-Evolving Robotic Agent), with a fixed VLA executor providing low-level control. The harness maintains and updatestask contextusing observations andexecution feedbackto guide subsequent planning. Self-evolution usesenvironment rolloutsto propose and evaluate harness revisions, with the selected configuration fixed before held-out testing. On RoboMME,SERAachieves 51.38% overall success with autonomous selection, compared with 44.38% without self-evolution. Our exploration suggests thatagentic planningis a promising direction for history-dependent robotic manipulation.

View arXiv pageView PDFProject pageGitHub2Add to collection

Get this paper in your agent:

hf papers read 2610\.29560

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.29560 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.29560 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.29560 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Self-Evolving Agent Harness

Hugging Face Daily Papers

提出一个自我进化代理框架（Harness RSI），在保持模型权重固定的情况下，通过反馈驱动的修订与进化优化来持续改进代理的软件脚手架（提示、记忆、工作流和可执行组件），并借助持续 PAC 学习给出泛化与保留的理论保证。

EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents

arXiv cs.LG

Introduces EvoHarness-RL, a framework that learns runtime harness policies for long-horizon LLM agents, enabling them to construct and update external state (belief, progress, experience) during task execution. Using Qwen3-8B on ALFWorld, it achieves 96.9% success and reveals harness annealing and evolution dynamics.