Does Native 3D Texture Generation Necessarily Require 3D Assets for Training?

Hugging Face Daily Papers Papers
3d-texture-generation diffusion-transformers data-efficiency multimodal-learning generative-models 3d-graphics

Summary

The paper proposes Tex-Zero, a native 3D texture generation framework trained entirely on constructed 2D images rather than real 3D assets, by converting high-quality 2D images into 3D training samples via plane representations and patch-wise random rotations, showing high-fidelity texture synthesis is possible without 3D asset data.

Native 3D texture generation synthesizes colors directly in 3D space for a given geometry, conditioned on multi-view reference images. It is generally believed that training such models requires large-scale, high-quality real 3D asset data, whose acquisition remains a long-standing and challenging problem. In this work, we propose Tex-Zero, demonstrating that a high-fidelity native 3D texture generation framework can be trained without 3D assets. Our key observation is that only high-quality and fine-grained color information is essential for 3D texture training, while the required geometric information is less critical and can be manually constructed rather than obtained from real 3D assets. This finding makes it possible to transform abundant, high-quality 2D images into effective training samples for 3D texture generation. Specifically, we convert high-quality 2D images into 3D training samples by representing each image as a plane in 3D space and applying patch-wise random rotations and aggregation to construct complex geometric structures. Using these constructed image data, we train the Tex-Zero VAE, which can reconstruct real 3D assets with high quality despite never observing them during training. Building upon the Tex-Zero VAE, we train the Tex-Zero DiT also exclusively on the constructed image data, where the conditioning 2D multi-view images are transformed into planes in 3D space and also encoded by the Tex-Zero VAE, thereby reducing the representation gap and improving generation quality. Extensive experiments show that Tex-Zero generates high-fidelity 3D textures with fine-grained details solely using images as training data, offering a promising perspective on the data paradigm for scaling 3D texture generation.
Original Article
View Cached Full Text

Cached at: 10/02/26, 08:28 AM

Paper page - Does Native 3D Texture Generation Necessarily Require 3D Assets for Training?

Source: https://huggingface.co/papers/2609.34621

Abstract

Native3Dtexturegenerationsynthesizescolorsdirectlyin3Dspaceforagivengeometry,conditionedonmulti-viewreferenceimages.Itisgenerallybelievedthattrainingsuchmodelsrequireslarge-scale,high-qualityreal3Dassetdata,whoseacquisitionremainsalong-standingandchallengingproblem.Inthiswork,weproposeTex-Zero,demonstratingthatahigh-fidelitynative3Dtexturegenerationframeworkcanbetrainedwithout3Dassets.Ourkeyobservationisthatonlyhigh-qualityandfine-grainedcolorinformationisessentialfor3Dtexturetraining,whiletherequiredgeometricinformationislesscriticalandcanbemanuallyconstructedratherthanobtainedfromreal3Dassets.Thisfindingmakesitpossibletotransformabundant,high-quality2Dimagesintoeffectivetrainingsamplesfor3Dtexturegeneration.Specifically,weconverthigh-quality2Dimagesinto3Dtrainingsamplesbyrepresentingeachimageasaplanein3Dspaceandapplyingpatch-wiserandomrotationsandaggregationtoconstructcomplexgeometricstructures.Usingtheseconstructedimagedata,wetraintheTex-ZeroVAE,whichcanreconstructreal3Dassetswithhighqualitydespiteneverobservingthemduringtraining.BuildingupontheTex-ZeroVAE,wetraintheTex-ZeroDiTalsoexclusivelyontheconstructedimagedata,wheretheconditioning2Dmulti-viewimagesaretransformedintoplanesin3DspaceandalsoencodedbytheTex-ZeroVAE,therebyreducingtherepresentationgapandimprovinggenerationquality.ExtensiveexperimentsshowthatTex-Zerogenerateshigh-fidelity3Dtextureswithfine-graineddetailssolelyusingimagesastrainingdata,offeringapromisingperspectiveonthedataparadigmforscaling3Dtexturegeneration.

View arXiv pageView PDFProject pageGitHub4Add to collection

Get this paper in your agent:

hf papers read 2609\.34621

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.34621 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.34621 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.34621 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

JanusMesh: Fast and Zero-Shot 3D Visual Illusion Generation via Cross-Space Denoising

Hugging Face Daily Papers

JanusMesh is a fast, training-free framework that generates text-driven 3D visual illusions—a single mesh revealing different semantics from different viewing angles—by decoupling generation into cross-space dual-branch denoising and view-conditioned texture synthesis, achieving high realism in just 3-5 minutes.