Paper page - Holo-World: Unified Camera, Object and Weather Control for Video World Model

Source: https://huggingface.co/papers/2606.20083

Abstract

A unified controllable video world model generates videos from a single image while preserving scene structure and transferring to target weather states through specialized parameterization and conditioning techniques.

Video world modelsare moving toward preserving an observed world under controllable camera andobject motionwhile allowing its environmental state to change. Yet these controls remain isolated, andweather generationtypically relies on a source video or reconstructed scene that already specifies future structure. We study a first-frame-anchoredsource-to-state setting, where the model starts from a single image and follows explicit camera and object controls and an optional weather instruction, then generates a video that either preserves the source world or transfers it to a target weather state. To address these challenges, we first buildHoloStateData, a state video dataset that turns diverse videos into unified control samples for camera, object, and weather supervision. Second, we introduceHolo-World, a unified controllable video world model that jointly controls scene from a single image. ItsUnified Scene Adapterfactorizes world preservation andweather transferinto distinct parameter subspaces, usingrendered background,geometry buffers, and object controls to maintain controlled scene structure while modeling weather-dependent appearance and particle effects. Additionally,Scene-Weather Decomposed CFGguides scene and weather residuals separately, strengthening target weather effects without over-amplifying the full condition. Quantitative and qualitative experiments demonstrate thatHolo-Worldmaintains precise camera and object control with consistent scene structure while transferring scenes into diverse target weather state, outperformingvideo-to-video weather editingbaselines on weather-state generation. Our project page is available at https://xiangchenyin.github.io/Holo-World/.

View arXiv pageView PDFProject pageGitHub4Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.20083 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.20083 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.20083 in a Space README.md to link it from this page.

Collections including this paper1