DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation

Hugging Face Daily Papers Papers
cross-embodiment robot-manipulation vision-language-action mixture-of-experts dynamics-priors robotics

Summary

DyPES-VLA is a cross-embodiment VLA model that learns shared dynamics priors via future prediction and uses an embodiment-specific Mixture-of-Experts action head to control robots in their native action spaces, achieving state-of-the-art results on LIBERO, RoboCasa, and RoboTwin benchmarks.

Vision-Language-Action (VLA) models have become a powerful paradigm for robot manipulation, but training a single generalist policy for heterogeneous robot embodiments remains an open problem. Existing methods have two main limitations. First, they underuse dynamics priors shared across diverse visual and interaction data, limiting cross-embodiment transfer. Second, they require extensive manual preprocessing to convert embodiment-specific actions into a common format. To overcome these limitations, we propose DyPES-VLA, a cross-embodiment VLA that learns shared Dynamics Priors and Embodiment-Specific control. First, we learn shared dynamics priors by training the vision-language model (VLM) with a future-prediction objective on cross-embodiment data, driving the shared query representation to capture object motion, contact, and interaction-induced scene changes. Second, an embodiment-specific Mixture-of-Experts (MoE) action head translates these shared dynamics priors into executable controls directly in each embodiment's native action space, without manually pre-aligning heterogeneous actions into a common format. This head shares attention layers to capture common temporal action structures, while its embodiment-specific feed-forward experts resolve the unique kinematic constraints and control semantics of distinct embodiments. As a generalist policy, our \ourmethod achieves state-of-the-art performance across simulation and real-world evaluations, reaching 98.0% success on LIBERO, 59.25% on RoboCasa-GR1, and 89.02% on RoboTwin~2.0.
Original Article
View Cached Full Text

Cached at: 08/07/26, 05:56 AM

Paper page - DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation

Source: https://huggingface.co/papers/2608.06374 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Vision-Language-Action(VLA)modelshavebecomeapowerfulparadigmforrobotmanipulation,buttrainingasinglegeneralistpolicyforheterogeneousrobotembodimentsremainsanopenproblem.Existingmethodshavetwomainlimitations.First,theyunderusedynamicspriorssharedacrossdiversevisualandinteractiondata,limitingcross-embodimenttransfer.Second,theyrequireextensivemanualpreprocessingtoconvertembodiment-specificactionsintoacommonformat.Toovercometheselimitations,weproposeDyPES-VLA,across-embodimentVLAthatlearnssharedDynamicsPriorsandEmbodiment-Specificcontrol.First,welearnshareddynamicspriorsbytrainingthevision-languagemodel(VLM)withafuture-predictionobjectiveoncross-embodimentdata,drivingthesharedqueryrepresentationtocaptureobjectmotion,contact,andinteraction-inducedscenechanges.Second,anembodiment-specificMixture-of-Experts(MoE)actionheadtranslatestheseshareddynamicspriorsintoexecutablecontrolsdirectlyineachembodiment’snativeactionspace,withoutmanuallypre-aligningheterogeneousactionsintoacommonformat.Thisheadsharesattentionlayerstocapturecommontemporalactionstructures,whileitsembodiment-specificfeed-forwardexpertsresolvetheuniquekinematicconstraintsandcontrolsemanticsofdistinctembodiments.Asageneralistpolicy,our\ourmethodachievesstate-of-the-artperformanceacrosssimulationandreal-worldevaluations,reaching98.0%successonLIBERO,59.25%onRoboCasa-GR1,and89.02%onRoboTwin~2.0.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2608\.06374

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.06374 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.06374 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.06374 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

HiVLA: A Visual-Grounded-Centric Hierarchical Embodied Manipulation System

Hugging Face Daily Papers

HiVLA introduces a hierarchical vision-language-action framework that decouples semantic planning from motor control using a diffusion transformer action expert for improved robotic manipulation. The system combines a VLM planner for task decomposition and visual grounding with a specialized DiT action expert using cascaded cross-attention, outperforming end-to-end baselines particularly in long-horizon tasks and fine-grained manipulation.