Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA Enhancement

Hugging Face Daily Papers Papers
residual-rl sim-to-real object-centric reinforcement-learning vision-language-action robotics zero-shot

Summary

An object-centric residual reinforcement learning framework enhances zero-shot sim-to-real transfer for vision-language-action models, improving success rates from 42% to 76% on manipulation tasks without real-world training.

Vision-Language-Action (VLA) models can generalize across diverse manipulation tasks, but their imitation-learning-based policies remain brittle in precise physical interactions due to compounding execution errors; Can a reinforcement learning policy trained purely in simulation improve the robustness of real-world VLAs zero-shot? Residual RL, which learns a corrective policy on top of a frozen VLA, offers a natural framework, but existing approaches face a fundamental sim-to-real dilemma: privileged-state methods require lossy distillation for deployment; image-based methods suffer from the visual domain gap; and real-world RL is costly and unsafe. We propose an object-centric residual RL framework that refines VLA actions using object poses, enabling a compact observation space that transfers consistently between simulation and reality. To align the two domains, we additionally replay the same teleoperation demonstrations in simulation to train a sim counterpart of the real-world VLA. The residual RL policy is trained only in simulation with pose noise injection and dropout, and transfers zero-shot to the real robot. Across five manipulation tasks on a real Franka Research 3 (FR3) robot, our method improves the success rate from 42% to 76% zero-shot, and the improved rollouts can be further reused to retrain the base VLA for self-improvement without additional teleoperation. Project page: https://www.microsoft.com/en-us/research/articles/object-centric-residual-rl/
Original Article
View Cached Full Text

Cached at: 06/29/26, 10:02 AM

Paper page - Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA Enhancement

Source: https://huggingface.co/papers/2606.18953

Abstract

An object-centric residual reinforcement learning framework improves real-world vision-language-action model robustness through simulation-trained corrective policies that transfer zero-shot despite sim-to-real challenges.

Vision-Language-Action (VLA) models can generalize across diverse manipulation tasks, but their imitation-learning-based policies remain brittle in precise physical interactions due to compounding execution errors; Can areinforcement learningpolicy trained purely in simulation improve the robustness of real-world VLAs zero-shot?Residual RL, which learns a corrective policy on top of a frozen VLA, offers a natural framework, but existing approaches face a fundamentalsim-to-real dilemma: privileged-state methods require lossy distillation for deployment; image-based methods suffer from the visualdomain gap; and real-world RL is costly and unsafe. We propose an object-centricresidual RLframework that refines VLA actions using object poses, enabling a compact observation space that transfers consistently between simulation and reality. To align the two domains, we additionally replay the sameteleoperationdemonstrations in simulation to train a sim counterpart of the real-world VLA. Theresidual RLpolicy is trained only in simulation with pose noise injection and dropout, and transfers zero-shot to the real robot. Across five manipulation tasks on a real Franka Research 3 (FR3) robot, our method improves the success rate from 42% to 76% zero-shot, and the improved rollouts can be further reused to retrain the base VLA for self-improvement without additionalteleoperation. Project page: https://www.microsoft.com/en-us/research/articles/object-centric-residual-rl/

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2606\.18953

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.18953 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.18953 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.18953 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Learning Visual Feature-Based World Models via Residual Latent Action

Hugging Face Daily Papers

This paper introduces RLA-WM, a visual feature-based world model that leverages residual latent actions and flow matching to efficiently predict future visual states. The method outperforms existing video-diffusion and feature-based approaches while enabling novel robot learning techniques from offline, actionless demonstration videos.