Paper page - Realiz3D: 3D Generation Made Photorealistic via Domain-Aware Learning

Source: https://huggingface.co/papers/2605.13852

Abstract

Realiz3D addresses the domain gap between synthetic renders and real images in 3D-consistent image generation by decoupling visual domain from control signals through residual adapters and layer-specific denoising strategies.

We often aim to generate images that are bothphotorealisticand3D-consistent, adhering to precise geometry, material, and viewpoint controls. Typically, this is achieved by fine-tuning animage generator, pre-trained on billions of real images, using renders ofsynthetic 3D assets, where annotations forcontrol signalsare available. While this approach can learn the desired controls, it often compromises the realism of the images due todomain gapbetween photographs and renders. We observe that this issue largely arises from the model learning an unintended association between the presence ofcontrol signalsand the synthetic appearance of the images. To address this, we introduce Realiz3D, a lightweight framework for trainingdiffusion models, that decouples controls andvisual domain. The key idea is to explicitly learnvisual domain, real or synthetic, separately from othercontrol signalsby introducing a co-variate that, fed into smallresidual adapters, shifts the domain. Then, the generator can be trained to gain controllability, without fitting to specificvisual domain. In this way, the model can be guided to produce realistic images even when controls are applied. We enhance control transferability to the real domain by leveraging insights about roles of different layers anddenoising stepsin diffusion-based generators, informing new training and inference strategies that further mitigate the gap. We demonstrate the advantages of Realiz3D in tasks astext-to-multiview generationandtexturingfrom 3D inputs, producing outputs that are3D-consistentandphotorealistic.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2605\.13852

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.13852 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.13852 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.13852 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.