Paper page - SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models

Source: https://huggingface.co/papers/2610.06598 Published on Oct 5

·

Submitted byhttps://huggingface.co/Xiaodong

Wangon Oct 16

Abstract

SimForcing improves action-conditioned robot world models by distilling motion knowledge from simulation and using controllable simulation conditioning to generate realistic videos across robot datasets.

Action-conditioned robot world modelsmust respond precisely to robot trajectories while preserving realistic visual dynamics, yet learning both from heterogeneous robot videos remains challenging. Simulation offers structured motion supervision, but appearance differences hinder direct transfer, and inaccurate simulation predictions can misguide real-video generation. We present SimForcing, a simulation-guided framework that uses simulation both as a source of transferable motion knowledge and as a controllable reference for prediction. First, we transfer motion knowledge from a simulation teacher throughlatent-motion distillation, aligning temporal changes in latent space to internalize motion priors while mitigating the influence of appearance differences. Second, we introducemulti-block simulation conditioningwithcondition dropoutto exploit predicted simulation trajectories without relying excessively on their accuracy. Our simulation-conditioningclassifier-free guidancescheme unifies these two ideas by balancing predictions based on internalized motion knowledge with those additionally guided by simulation latents. The jointly trained student generates both simulation conditions and real-domain videos, requiring no additional world model at inference. On Bridge, SimForcing achieves the best PSNR, SSIM, LPIPS, and FVD among the compared methods without external embodied pretraining. Evaluation on InternData-A1 further supports its applicability across robot datasets. Moreover, using our trained world model to initialize avision-language-action modelimproves LIBERO success, suggesting its utility for downstream policy learning. https://github.com/Wang-Xiaodong1899/SimForcing

View arXiv pageView PDFProject pageGitHub21Add to collection

Get this paper in your agent:

hf papers read 2610\.06598

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.06598 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.06598 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.06598 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.