WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes

Hugging Face Daily Papers Papers
3d-world-modeling scene-decomposition interactive-scenes generative-scene-synthesis object-level-editing embodied-simulation

Summary

WorldAct is a framework that converts static 3D generated environments into editable and interactive object-centric scenes using multimodal agents and geometric reconstruction, enabling object-level editing and embodied task execution.

Recent 3D world modeling systems based on generative scene synthesis, such as Marble, can create coherent and explorable 3D environments, yet their outputs are typically static monolithic assets with limited editability and physical interaction. This restricts their use in immersive content creation and embodied simulation, where generated worlds must be actively modified and manipulated. To tackle this challenge, we present WorldAct, a framework that converts static generated 3D worlds into editable and interaction-ready scenes. WorldAct uses a multimodal agent to guide scene decomposition, identify actionable objects, reconstruct geometrically aligned object-level meshes for interaction, and restore the residual background via 3D inpainting. The resulting scenes support object-level editing, collision-aware manipulation, and embodied task execution while preserving global scene coherence. Experiments show that WorldAct enables richer interaction scenarios than the original generated scenes, suggesting a practical path toward editable and interactive 3D world models.
Original Article
View Cached Full Text

Cached at: 05/18/26, 06:24 AM

Paper page - WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes

Source: https://huggingface.co/papers/2605.15843

Abstract

WorldAct framework transforms static 3D generated environments into editable and interactive scenes through multimodal agents and geometric reconstruction techniques.

Recent3D world modelingsystems based ongenerative scene synthesis, such as Marble, can create coherent and explorable 3D environments, yet their outputs are typically static monolithic assets with limited editability and physical interaction. This restricts their use in immersive content creation andembodied simulation, where generated worlds must be actively modified and manipulated. To tackle this challenge, we present WorldAct, a framework that converts static generated 3D worlds into editable and interaction-ready scenes. WorldAct uses a multimodal agent to guidescene decomposition, identifyactionable objects, reconstruct geometrically alignedobject-level meshesfor interaction, and restore the residual background via3D inpainting. The resulting scenes supportobject-level editing,collision-aware manipulation, and embodied task execution while preserving global scene coherence. Experiments show that WorldAct enables richer interaction scenarios than the original generated scenes, suggesting a practical path toward editable and interactive 3D world models.

View arXiv pageView PDFProject pageGitHub3Add to collection

Get this paper in your agent:

hf papers read 2605\.15843

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.15843 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.15843 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.15843 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles