Explicit Layer Modeling for Video Object Insertion and Layer Decomposition

Hugging Face Daily Papers Papers
video-editing layer-decomposition object-insertion diffusion-model dataset computer-vision

Summary

This paper introduces TriLayer, a large-scale video dataset with foreground-background-composite triplets, and DBL-Diffusion, a dual-branch diffusion framework for explicit layered video representation, enabling high-fidelity object insertion and layer decomposition.

Most video editing systems still lack explicit layered video representations, limiting their ability to perform realistic compositing, object reuse, and consistent manipulation. This limitation is especially pronounced in video object insertion and video layer decomposition, where existing methods rely on implicit inference or per-scene optimization due to the absence of explicit foreground-layer supervision. We introduce TriLayer, a large-scale triplet video dataset containing aligned composite, background, and foreground videos, where the foreground layers include both object appearance and associated visual effects. This explicit supervision enables models to learn layered video representations directly rather than inferring them implicitly. Building on this dataset, we propose DBL-Diffusion, a dual-branch diffusion framework that jointly models RGB composites and RGBA foreground layers through shared denoising and cross-branch interaction. We instantiate the framework in two tasks: DBL-Insert for layered object insertion, which generates explicit RGBA layers for realistic compositing and flexible post-editing, and DBL-Decompose for video layer decomposition, which recovers foreground and background layers using triplet supervision. Experiments demonstrate that explicit layer modeling substantially improves both insertion fidelity and decomposition quality.
Original Article
View Cached Full Text

Cached at: 07/30/26, 01:45 AM

Paper page - Explicit Layer Modeling for Video Object Insertion and Layer Decomposition

Source: https://huggingface.co/papers/2607.25802

Abstract

Mostvideoeditingsystemsstilllackexplicitlayeredvideorepresentations,limitingtheirabilitytoperformrealisticcompositing,objectreuse,andconsistentmanipulation.Thislimitationisespeciallypronouncedinvideoobjectinsertionandvideolayerdecomposition,whereexistingmethodsrelyonimplicitinferenceorper-sceneoptimizationduetotheabsenceofexplicitforeground-layersupervision.WeintroduceTriLayer,alarge-scaletripletvideodatasetcontainingalignedcomposite,background,andforegroundvideos,wheretheforegroundlayersincludebothobjectappearanceandassociatedvisualeffects.Thisexplicitsupervisionenablesmodelstolearnlayeredvideorepresentationsdirectlyratherthaninferringthemimplicitly.Buildingonthisdataset,weproposeDBL-Diffusion,adual-branchdiffusionframeworkthatjointlymodelsRGBcompositesandRGBAforegroundlayersthroughshareddenoisingandcross-branchinteraction.Weinstantiatetheframeworkintwotasks:DBL-Insertforlayeredobjectinsertion,whichgeneratesexplicitRGBAlayersforrealisticcompositingandflexiblepost-editing,andDBL-Decomposeforvideolayerdecomposition,whichrecoversforegroundandbackgroundlayersusingtripletsupervision.Experimentsdemonstratethatexplicitlayermodelingsubstantiallyimprovesbothinsertionfidelityanddecompositionquality.

View arXiv pageView PDFGitHub2Add to collection

Get this paper in your agent:

hf papers read 2607\.25802

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.25802 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.25802 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.25802 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

DreamTraj: Generating 6-DoF Object Trajectories by Reading Unrendered Video Diffusion Latents

Hugging Face Daily Papers

DreamTraj predicts 6-DoF object trajectories from a single RGB image and a language instruction by decoding internal video diffusion latents, eliminating the need for video, depth, or CAD models at inference. It introduces the MOVEdataset with fine-grained language-to-motion annotations and achieves state-of-the-art performance while running 4.6x faster than generate-then-extract pipelines.

Beyond Pixels: From Video Priors to 4D Worlds

Hugging Face Daily Papers

This paper introduces Latent-to-4D, a method for direct 4D scene generation from video diffusion latents without retraining across generators, achieving better geometry and temporal stability than cascaded approaches.

VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding

Papers with Code Trending

VideoChat3 is a fully open, efficient, and generalist video-centric multimodal large language model that introduces Inflated 3D Vision Transformer (I3D-ViT) and Adaptive Frame Resolution for streaming video perception, along with scalable video data synthesis pipelines, achieving superior performance with only 4B parameters.