Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation

Hugging Face Daily Papers Papers
sim-to-real robotics agentic-learning manipulation llm-agents policy-transfer

Summary

Skill2Real introduces an agentic skill learning framework using a Proposer-Verifier-Governor loop that learns executable robot skills in simulation via a shared API and transfers them zero-shot to real-world manipulation without fine-tuning, achieving strong success rates on LIBERO benchmarks and real-robot tasks.

Transferring robotic skills from simulation to reality requires task knowledge that remains usable across differences in perception, dynamics, and embodiment. We introduce Skill2Real, an agentic policy framework that learns executable skills through a shared application programming interface (API). A Proposer-Verifier-Governor (PVG) loop uses privileged simulation evidence to diagnose outcomes and validate updates, while keeping learned skills grounded in public observations and API semantics. The Cerebellum first acquires local manipulation skills; the Brain then learns task-level composition with the Cerebellum frozen. Both memories transfer to the real robot without task-policy fine-tuning or skill-memory updates. As GPT-5.6 Sol learns skills on LIBERO-90, evaluating each frozen checkpoint with GPT-6 Astra raises LIBERO-Pro Long success from 2.0% to 56.3%, without training on Pro Long. Independent Robosuite training reaches 85.1% and 89.4% mean success with Sol and Opus 5 across seven tasks, respectively. Frozen Sol-trained LIBERO-90 skills achieve 78.75% mean completion across four real-world manipulation tasks with Astra. Removing the Verifier or Governor during LIBERO-90 training lowers final Pro Long success by 17.3 and 13.3 percentage points, respectively. These results support learning and transferring a hierarchy of executable skills through a common robot interface.
Original Article
View Cached Full Text

Cached at: 10/05/26, 04:46 AM

Paper page - Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation

Source: https://huggingface.co/papers/2610.02788

Abstract

Transferringroboticskillsfromsimulationtorealityrequirestaskknowledgethatremainsusableacrossdifferencesinperception,dynamics,andembodiment.WeintroduceSkill2Real,anagenticpolicyframeworkthatlearnsexecutableskillsthroughasharedapplicationprogramminginterface(API).AProposer-Verifier-Governor(PVG)loopusesprivilegedsimulationevidencetodiagnoseoutcomesandvalidateupdates,whilekeepinglearnedskillsgroundedinpublicobservationsandAPIsemantics.TheCerebellumfirstacquireslocalmanipulationskills;theBrainthenlearnstask-levelcompositionwiththeCerebellumfrozen.Bothmemoriestransfertotherealrobotwithouttask-policyfine-tuningorskill-memoryupdates.AsGPT-5.6SollearnsskillsonLIBERO-90,evaluatingeachfrozencheckpointwithGPT-6AstraraisesLIBERO-ProLongsuccessfrom2.0%to56.3%,withouttrainingonProLong.IndependentRobosuitetrainingreaches85.1%and89.4%meansuccesswithSolandOpus5acrossseventasks,respectively.FrozenSol-trainedLIBERO-90skillsachieve78.75%meancompletionacrossfourreal-worldmanipulationtaskswithAstra.RemovingtheVerifierorGovernorduringLIBERO-90traininglowersfinalProLongsuccessby17.3and13.3percentagepoints,respectively.Theseresultssupportlearningandtransferringahierarchyofexecutableskillsthroughacommonrobotinterface.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2610\.02788

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.02788 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.02788 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.02788 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Real2Gym: Building Gyms from Videos, Bringing Skills to Robots

Hugging Face Daily Papers

Real2Gym is an agentic Real2Sim2Real framework that transforms human and robot video demonstrations into interactive simulation gyms, enabling skill transfer to physical robots with improved success rates over baselines.

Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching

Hugging Face Daily Papers

This paper introduces Domain-Randomized Instance Set (DRIS), a method that simultaneously represents multiple randomized instances to improve sim-to-real transfer for dexterous manipulation. It demonstrates zero-shot transfer on a challenging reactive catching task with a flat plate end-effector, requiring no real-world fine-tuning.

Playful Agentic Robot Learning

Hugging Face Daily Papers

Introduces Playful Agentic Robot Learning, where embodied coding agents use self-directed play to learn reusable skills, improving downstream task performance without additional training. The proposed RATs system achieves significant gains over baselines in simulation and real-world transfer.

MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation

Hugging Face Daily Papers

MotorMind introduces a harness that lets a general-purpose VLM control a robot directly by reasoning over observations and issuing mid-level actions with asynchronous feedback, achieving 66.7% success on LIBERO-PRO zero-shot (vs. up to 13.3% prior methods) and 95% on a real xArm6 robot without task-specific training, coding agents, or grounding tools.