Paper page - iMaC: Translating Actions into Motion and Contact Images for Embodied World Models

Source: https://huggingface.co/papers/2606.09813 Authors:

,

,

,

,

,

,

,

,

,

Abstract

iMac presents a unified control paradigm that uses raw visual images as native action representations for embodied world models, enabling more expressive and generalized robotic control through image-based action tokens.

Embodied world modelshave emerged as a pivotal paradigm forvisual robotic decision-makingand interactive environment simulation. However, conventional embodied frameworks rely on low-dimensional structured action vectors (e.g., joint angles and end-effector poses), which suffer from limited expressive capacity, poor generalization across diverse embodiments, and unnatural dynamic modeling for complex physical interactions. To address these limitations, this paper proposesiMac (Image as Action Control), a novel unified control paradigm that treats raw visual images as native action representations forembodied world models. Departing from traditional explicit kinematic action encoding, iMac formulatescontinuous visual manipulationasimage-based action tokens, which inherently encapsulate spatial motion intentions, interactive geometric constraints and subtle physical dynamics. We construct a dual-branch embodied architecture consisting of animage-action encoderand adynamic world predictor: the encoder compressestarget-driven visual imagesinto compactaction embeddings, while the predictor learnsenvironment transition rulesconditioned on image actions to achieve high-fidelity future state prediction andclosed-loop embodied control. Extensive experiments are conducted on public embodied manipulation benchmarks and real-world robotic scenarios. The results demonstrate that iMac outperforms vector-based action control baselines in prediction accuracy, task success rate andcross-scene generalizationability. Moreover, our image-action design eliminates the reliance on manually defined action spaces, realizing flexible and universal control for heterogeneous embodied agents. This work provides an innovative visual-action perspective forembodied world models, offering a simple yet effective paradigm for scalable robotic perception and manipulation.

View arXiv pageView PDFProject pageGitHub14Add to collection

Get this paper in your agent:

hf papers read 2606\.09813

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.09813 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.09813 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.09813 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.