WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory

Hugging Face Daily Papers Papers
video-generation world-model controllable-simulation persistent-memory llm 3d-trajectories

Summary

WorldDirector introduces a framework for controllable video world simulation that decouples semantic motion planning from visual rendering using an LLM to coordinate 3D trajectories and camera movements, enabling persistent object memory and stable appearance even after objects re-enter the scene.

We present WorldDirector, a highly controllable video world model framework designed for persistent dynamic object memory and unrestricted viewpoint exploration. Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, our framework explicitly decouples semantic motion orchestration from visual generation. By leveraging an LLM to coordinate 3D trajectories with camera movements and subsequently employing these orchestrated trajectories as control signals for video generation, our approach ensures strict physical logic and appearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view. Experimental results demonstrate that our method supports the synthesis of complex and extended events with unprecedented controllability and persistent dynamic object memory. Project Page: https://worlddirector.github.io/
Original Article
View Cached Full Text

Cached at: 07/03/26, 03:52 AM

Paper page - WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory

Source: https://huggingface.co/papers/2607.02517 Authors:

,

,

,

,

,

,

,

,

,

,

,

Abstract

WorldDirector enables controllable video generation with persistent object memory by decoupling semantic motion planning from visual rendering through LLM coordination of 3D trajectories and camera movements.

We present WorldDirector, a highly controllablevideo world modelframework designed forpersistent dynamic object memoryand unrestrictedviewpoint exploration. Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, our framework explicitly decouplessemantic motion orchestrationfromvisual generation. By leveraging anLLMto coordinate3D trajectorieswithcamera movementsand subsequently employing these orchestrated trajectories as control signals for video generation, our approach ensures strictphysical logicandappearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view. Experimental results demonstrate that our method supports the synthesis of complex and extended events with unprecedented controllability andpersistent dynamic object memory. Project Page: https://worlddirector.github.io/

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2607\.02517

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.02517 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.02517 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.02517 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Latent Spatial Memory for Video World Models

Hugging Face Daily Papers

This paper introduces latent spatial memory for video world models, storing 3D scene information directly in diffusion latent space to avoid costly pixel-space reconstruction. The proposed Mirage framework achieves up to 10.57x faster generation and 55x memory reduction while achieving state-of-the-art performance on WorldScore and RealEstate10K.