Luce: Relightable Gaussians for 3D Asset Generation

Hugging Face Daily Papers Papers
3d-generation gaussian-splats pbr-materials variational-autoencoder image-to-3d relighting

Summary

Luce unifies geometry and PBR materials in a voxelized Gaussian cloud to generate relightable 3D assets from single images, achieving state-of-the-art results on benchmarks like Toys4K.

High-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance. To support relighting and integration into standard rendering pipelines, the representation should include physically based rendering (PBR) modalities such as albedo, metallic-roughness, and surface normals. We propose Luce, a 3D representation that unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud, using dedicated Gaussian primitives for each modality. A variational autoencoder compresses this representation into a unified material-aware latent space. A rectified-flow transformer generates this latent from a single image, conditioned on multi-layer features from a pretrained image encoder that preserve both semantic context and fine spatial detail. The latent then decodes into relightable PBR Gaussians and an optional textured mesh with a tangent-space normal map. On Toys4K, Luce achieves state-of-the-art single-image-to-3D generation, improving FID by 28% over the strongest baseline. We further introduce a benchmark of AI-generated images, on which Luce improves the CLIP image-alignment score over the best baseline (0.8519 vs. 0.8299). Luce generates relightable, geometrically accurate, and materially faithful assets that preserve fine details such as text, logos, and inscriptions.
Original Article
View Cached Full Text

Cached at: 08/29/26, 03:30 AM

Paper page - Luce: Relightable Gaussians for 3D Asset Generation

Source: https://huggingface.co/papers/2608.23943

Abstract

Luce unifies geometry and PBR materials in a voxelized Gaussian cloud, using a variational autoencoder and rectified-flow transformer to generate relightable 3D assets from single images.

High-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance. To support relighting and integration into standard rendering pipelines, the representation should include physically based rendering (PBR) modalities such as albedo, metallic-roughness, and surface normals. We propose Luce, a 3D representation that unifies geometry andPBR materialswithin avoxelized multimodal Gaussian cloud, using dedicated Gaussian primitives for each modality. Avariational autoencodercompresses this representation into a unified material-aware latent space. Arectified-flow transformergenerates this latent from a single image, conditioned onmulti-layer featuresfrom a pretrained image encoder that preserve both semantic context and fine spatial detail. The latent then decodes into relightable PBR Gaussians and an optional textured mesh with atangent-space normal map. On Toys4K, Luce achieves state-of-the-art single-image-to-3D generation, improving FID by 28% over the strongest baseline. We further introduce a benchmark of AI-generated images, on which Luce improves the CLIP image-alignment score over the best baseline (0.8519 vs. 0.8299). Luce generates relightable, geometrically accurate, and materially faithful assets that preserve fine details such as text, logos, and inscriptions.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2608\.23943

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.23943 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.23943 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.23943 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Structured 3D Latents for Scalable and Versatile 3D Generation

Papers with Code Trending

微软提出统一的结构化潜在表示（SLAT），结合 rectified flow transformers 实现高质量、多格式（Radiance Fields、3D Gaussians、网格）的 3D 资产生成，支持文本/图像条件及局部编辑，在 500K 3D 数据集上训练的 2B 参数模型显著超越现有方法。