DexPolicy: Scheduled Exploration for Trajectory-Guided Dexterous Manipulation

Hugging Face Daily Papers Papers
reinforcement-learning dexterous-manipulation robotics exploration-scheduling trajectory-guided-rl

Summary

DexPolicy introduces scheduled exploration for trajectory-guided dexterous manipulation, annealing exploration noise from broad to narrow during RL training and significantly boosting Target success on YCB objects and a RealMan RM75 robot arm across PPO, GRPO, and a flow-parameterized PPO variant.

Reinforcement learning (RL) for dexterous manipulation must discover finger-object contacts and then control the object precisely; the action noise that serves the first goal can interfere with the second. In trajectory-guided settings such as ViViDex, where RL refine hand-object trajectories from human video, our baseline PPO runs end near their initial action noise after 5M steps, motivating explicit control of exploration scale. DexPolicy makes that scale an explicit function of training steps, annealing from broad to narrow exploration while holding loss, architecture, reward, and optimizer settings fixed. We study three policy-optimization settings: PPO, critic-free GRPO continuation, and a flow-parameterized PPO variant (FPO). Across five YCB objects and three training seeds, mean deterministic Target success rises from 49.4% to 68.1% (FPO), 14.1% to 45.4% (GRPO), and 32.0% to 35.7% (PPO). On a RealMan RM75 arm with an Inspire/RH56 hand, 360 trials over three objects raise mean Target success from 25.0% to 85.0% (FPO), 10.0% to 63.3% (GRPO), and 8.3% to 43.3% (PPO), with one trained model per object-method condition. PPO component screening favors noise control over the tested optimizer contraction; the selected PPO schedule yields higher mean Target success than linear decay with the same endpoints on three tested objects. Training return, deterministic Target success, and tolerance to execution noise dissociate; schedules should therefore be judged by terminal task success under the intended execution conditions, per task and policy-optimization setting. Code: https://github.com/AIGeeksGroup/DexPolicy. Website: https://aigeeksgroup.github.io/DexPolicy.
Original Article
View Cached Full Text

Cached at: 10/02/26, 04:27 AM

Paper page - DexPolicy: Scheduled Exploration for Trajectory-Guided Dexterous Manipulation

Source: https://huggingface.co/papers/2610.00360

Abstract

Reinforcementlearning(RL)fordexterousmanipulationmustdiscoverfinger-objectcontactsandthencontroltheobjectprecisely;theactionnoisethatservesthefirstgoalcaninterferewiththesecond.Intrajectory-guidedsettingssuchasViViDex,whereRLrefinehand-objecttrajectoriesfromhumanvideo,ourbaselinePPOrunsendneartheirinitialactionnoiseafter5Msteps,motivatingexplicitcontrolofexplorationscale.DexPolicymakesthatscaleanexplicitfunctionoftrainingsteps,annealingfrombroadtonarrowexplorationwhileholdingloss,architecture,reward,andoptimizersettingsfixed.Westudythreepolicy-optimizationsettings:PPO,critic-freeGRPOcontinuation,andaflow-parameterizedPPOvariant(FPO).AcrossfiveYCBobjectsandthreetrainingseeds,meandeterministicTargetsuccessrisesfrom49.4%to68.1%(FPO),14.1%to45.4%(GRPO),and32.0%to35.7%(PPO).OnaRealManRM75armwithanInspire/RH56hand,360trialsoverthreeobjectsraisemeanTargetsuccessfrom25.0%to85.0%(FPO),10.0%to63.3%(GRPO),and8.3%to43.3%(PPO),withonetrainedmodelperobject-methodcondition.PPOcomponentscreeningfavorsnoisecontroloverthetestedoptimizercontraction;theselectedPPOscheduleyieldshighermeanTargetsuccessthanlineardecaywiththesameendpointsonthreetestedobjects.Trainingreturn,deterministicTargetsuccess,andtolerancetoexecutionnoisedissociate;schedulesshouldthereforebejudgedbyterminaltasksuccessundertheintendedexecutionconditions,pertaskandpolicy-optimizationsetting.Code:https://github.com/AIGeeksGroup/DexPolicy.Website:https://aigeeksgroup.github.io/DexPolicy.

View arXiv pageView PDFProject pageGitHub0Add to collection

Get this paper in your agent:

hf papers read 2610\.00360

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.00360 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.00360 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.00360 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

RLDX-1 Technical Report

Hugging Face Daily Papers

RLDX-1 is a general-purpose robotic policy for dexterous manipulation that uses a Multi-Stream Action Transformer architecture to integrate heterogeneous modalities, outperforming existing VLA models in real-world tasks.

Learning dexterity

OpenAI Blog

OpenAI announces Dactyl, a system that learns robotic hand dexterity through simulation and reinforcement learning, using LSTMs to generalize across different physical environments and the Rapid PPO implementation to train policies that transfer to real-world manipulation tasks.

Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching

Hugging Face Daily Papers

This paper introduces Domain-Randomized Instance Set (DRIS), a method that simultaneously represents multiple randomized instances to improve sim-to-real transfer for dexterous manipulation. It demonstrates zero-shot transfer on a challenging reactive catching task with a flat plate end-effector, requiring no real-world fine-tuning.