Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching
Summary
This paper introduces Domain-Randomized Instance Set (DRIS), a method that simultaneously represents multiple randomized instances to improve sim-to-real transfer for dexterous manipulation. It demonstrates zero-shot transfer on a challenging reactive catching task with a flat plate end-effector, requiring no real-world fine-tuning.
View Cached Full Text
Cached at: 05/20/26, 06:39 PM
Paper page - Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching
Source: https://huggingface.co/papers/2605.09789
Abstract
Domain-Randomized Instance Set (DRIS) enables robust policy learning for dexterous manipulation tasks by simultaneously representing multiple randomized instances, achieving strong sim-to-real transfer without extensive real-world fine-tuning.
Dexterous manipulation is physics-intensive and highly sensitive to modeling errors and perception noise, makingsim-to-real transferprohibitively challenging.Domain randomization(DR) is commonly used to improve the robustness of learned policies for such tasks, but conventional DR randomizes one instance per episode, offering very limited exposure to the variability of real-world dynamics. To this end, we proposeDomain-Randomized Instance Set(DRIS), which represents and propagates a set of randomized instances simultaneously, providing richer approximation of uncertain dynamics and enabling policies to learn actions that account for multiple possible outcomes. Supported by theoretical analysis, we show that DRIS yields more robust policies and alleviates the need for real-world fine-tuning, even with a modest number of instances (e.g., 10). We demonstrate this on a challengingreactive catchingtask. Unlike traditional catching setups that use end-effectors designed to mechanically stabilize the object (e.g., curved or enclosing surfaces), our system uses a flat plate that offers no passive stabilization, making the task highly sensitive to noise and requiring rapid reactive motions. The learned policies exhibit strong robustness to uncertainties and achieve reliable zero-shotsim-to-real transfer.
View arXiv pageView PDFAdd to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.09789 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.09789 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.09789 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
DeVI: Physics-based Dexterous Human-Object Interaction via Synthetic Video Imitation
DeVI introduces a framework that turns text-conditioned synthetic videos into physically plausible dexterous robot control via a hybrid 3D-2D tracking reward, enabling zero-shot generalization to unseen objects.
Sim-to-real transfer of robotic control with dynamics randomization
OpenAI researchers demonstrate a method to bridge the reality gap in robotic control by training policies with randomized simulator dynamics, enabling robots trained purely in simulation to successfully transfer to real-world tasks like object manipulation without physical training.
Learning dexterity
OpenAI announces Dactyl, a system that learns robotic hand dexterity through simulation and reinforcement learning, using LSTMs to generalize across different physical environments and the Rapid PPO implementation to train policies that transfer to real-world manipulation tasks.
Transfer from simulation to real world through learning deep inverse dynamics model
This paper proposes a method to bridge the simulation-to-real-world gap in robotics by learning a deep inverse dynamics model that maps desired next states (from simulation) to appropriate real-world actions. The approach is evaluated against baselines like output error control and Gaussian dynamics adaptation.
Domain randomization and generative models for robotic grasping
Researchers explore a data generation pipeline using domain randomization and procedurally generated objects to train a deep neural network for robotic grasp planning. The proposed autoregressive model achieves >90% success on unseen objects in simulation and 80% in the real world, despite being trained only on random simulated objects.