Direct 3D-Aware Object Insertion via Decomposed Visual Proxies
Summary
This paper introduces DIRECT, a framework for pose-controllable 3D-aware object insertion that decomposes conditions into appearance, geometry, and context guidance to achieve high-fidelity compositing with explicit 3D pose control.
View Cached Full Text
Cached at: 06/08/26, 11:15 AM
Paper page - Direct 3D-Aware Object Insertion via Decomposed Visual Proxies
Source: https://huggingface.co/papers/2606.06601
Abstract
DIFFUSION-BASED OBJECT INSERTION FRAMEWORK WITH POSE CONTROL THROUGH DECOMPOSED GUIDANCE COMPONENTS
Object insertionaims to seamlessly composite a reference object into a specified region of a background image. Recentdiffusion-based methodsachieve highvisual qualitybut formulate insertion as a simple 2Dinpainting task, providing no explicit control over the object’s3D poseand limiting their practical applicability. We propose DIRECT (Decomposed Injection for Reference Composition and Target-integration), a novel framework that integratesinteractive pose manipulationwith high-fidelity2D image synthesisto enable pose-controllableobject insertion. Our method decomposes the insertion conditions into three complementary components:appearance guidancecapturing visual details from the reference object,geometry guidancederived from the user-adjusted3D proxy, andcontext guidancefrom the target background. By injecting them through separate pathways, DIRECT avoidsfeature entanglementand simultaneously preserves reference appearance, follows the user-specified pose, and adapts the object to the target scene. We also introduce an automated data construction pipeline to improve the diversity and quality of training data. Experiments show that DIRECT outperforms previous methods in bothgeometric controllabilityandvisual quality.
View arXiv pageView PDFProject pageGitHub23Add to collection
Get this paper in your agent:
hf papers read 2606\.06601
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.06601 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.06601 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.06601 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
DeVI: Physics-based Dexterous Human-Object Interaction via Synthetic Video Imitation
DeVI introduces a framework that turns text-conditioned synthetic videos into physically plausible dexterous robot control via a hybrid 3D-2D tracking reward, enabling zero-shot generalization to unseen objects.
Scenes as Objects, Not Primitives: Instance-Structured 3D Tokenization from Unposed Views
This paper proposes a feed-forward framework that decomposes 3D scenes into instance-structured token groups from unposed multi-view images, enabling direct object-level reconstruction, segmentation, and manipulation without 3D annotations.
Explicit Layer Modeling for Video Object Insertion and Layer Decomposition
This paper introduces TriLayer, a large-scale video dataset with foreground-background-composite triplets, and DBL-Diffusion, a dual-branch diffusion framework for explicit layered video representation, enabling high-fidelity object insertion and layer decomposition.
Pantheon360: Taming Digital Twin Generation via 3D-Aware 360° Video Diffusion
Pantheon360 introduces a 3D-aware 360° video diffusion framework that uses an explicit 3D cache to enforce geometric consistency, enabling high-fidelity digital twin generation from sparse 360° inputs.
CoInteract: Physically-Consistent Human-Object Interaction Video Synthesis via Spatially-Structured Co-Generation
CoInteract introduces an end-to-end Diffusion Transformer framework that jointly models RGB appearance and HOI geometry to generate physically-plausible human-object interaction videos with stable hands/faces and zero inference overhead.