Paper page - WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models

Source: https://huggingface.co/papers/2605.25077 Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

WorldCraft extends interactive video world models to enable object-level trajectory control while maintaining camera navigation capabilities through specialized control pipelines.

Recentvideo-based world modelshave made pixel-space environments interactive at the camera level: users can navigate viewpoints while the model generates coherent visual continuations. Yet their action spaces remain incomplete: users can move the camera, but cannot act on individual objects. Since real-world interaction is inherently object-centric, such models remain closer to passive scene observers than truly manipulable environments. We present WorldCraft, a framework that expands interactive videoworld modelsfromcamera navigationtoobject-level trajectory actions. Given a user click and a sketched path, WorldCraft generates future frames in which the selected object follows the prescribed trajectory while the camera continues to navigate the scene. WorldCraft achieves this through a trajectory-centric control pipeline: First,Normalized World Trajectory(NWT) represents user-drawn motion in acamera-invariant world coordinate systemand dynamically re-projects it under the current camera pose, separating object motion from camera-inducedscreen-space displacement;Spatial-Pathway LoRA(SP-LoRA) then injects this world-space signal through the model’sspatial-control pathway, adding object manipulation capability while preserving the pretrained camera controller; finally,Trajectory-Anchored State Persistence(TASP) treats the world trajectory as a persistent spatial state and refreshesautoregressive memoryaftertrajectory-conditioned generation, allowing moved objects to reappear at their updated positions after leaving the camera view. Experiments show that WorldCraft enables accurate object control, preserves the video-based world model’s camera fidelity under camera-only evaluation, and maintains object state across long autoregressive rollouts with off-camera excursions.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2605\.25077

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.25077 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.25077 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.25077 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.