Paper page - Chat-Edit-3D++: Interactive 3D and 4D Scene Editing via Large Language Models

Source: https://huggingface.co/papers/2608.29137

Abstract

CE3D++ enables flexible dialogue-driven 3D and 4D scene editing by decoupling 2D editing from 3D reconstruction via a Hash-Atlas network and using a large language model to autonomously invoke visual tools.

Recent work on image content manipulation based onvision-language pre-trainingmodels has been effectively extended to text-driven3D scene editing. However, existing schemes for3D scene editingstill have certain shortcomings, hindering their further development as interactive design tools. Such schemes typically adhere to fixed input patterns, limiting flexibility in text input. Furthermore, their editing capabilities are constrained by a single or a few 2Dvisual modelsand require intricate pipeline design to integrate these models into 3D reconstruction processes. To address the aforementioned issues, we propose theHash-Atlas network, which reformulates3D scene editingas operations on 2D atlas images, thereby achieving a workflow decoupling of the 2D editing and 3D reconstruction processes. Building on this foundation, we introduce a dialogue-based3D scene editingapproach, termed CE3D++, which is centered on alarge language model(LLM) that allows arbitrary textual input from users and interprets their intentions, subsequently facilitating the autonomous invocation of the correspondingvisual models. Additionally, we extend CE3D++ tomonocular 4D scenesby imposingmotion constraintson moving objects and further fine-tuning the LLM by creating atrajectory datasetrelated to editing tasks, which enables the smaller LLM to schedule up to 30 different visual tools accurately. Experimental results demonstrate that CE3D++ effectively integrates multiplevisual modelsto achieve diverse visual editing effects, possessing strong scene comprehension and multi-round dialog capabilities. The source codes and trained models are available at https://github.com/Fangkang515/CE3D.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2608\.29137

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.29137 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.29137 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.29137 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.