Beyond Prompts: Unconditional 3D Inversion for Out-of-Distribution Shapes
Summary
This paper identifies and addresses 'latent sink traps' in text-to-3D generative models where they become insensitive to text prompts, proposing a framework that decouples geometric representation from linguistic sensitivity to enable robust text-based 3D shape editing of out-of-distribution shapes.
View Cached Full Text
Cached at: 04/20/26, 08:27 AM
Paper page - Beyond Prompts: Unconditional 3D Inversion for Out-of-Distribution Shapes
Source: https://huggingface.co/papers/2604.14914
Abstract
State-of-the-art text-to-3D generative models suffer from latent sink traps where they lose sensitivity to text prompts, but a robust framework can overcome this by decoupling geometric representation from linguistic sensitivity.
Text-driven inversion of generative models is a core paradigm for manipulating 2D or 3D content, unlocking numerous applications such as text-based editing, style transfer, or inverse problems. However, it relies on the assumption that generative models remain sensitive to natural language prompts. We demonstrate that for state-of-the-art nativetext-to-3D generative models (https://huggingface.co/papers?q=text-to-3D%20generative%20models), this assumption often collapses. We identify a critical failure mode where generation trajectories are drawn into latent ``sink traps’’: regions where the model becomes insensitive toprompt modifications (https://huggingface.co/papers?q=prompt%20modifications). In these regimes, changes to the input text fail to alter internal representations in a way that alters the output geometry. Crucially, we observe that this is not a limitation of the model’s geometric expressivity; the same generative models possess the ability to produce a vast diversity of shapes but, as we demonstrate, become insensitive toout-of-distribution text guidance (https://huggingface.co/papers?q=out-of-distribution%20text%20guidance). We investigate this behavior by analyzing the sampling trajectories of the generative model, and find that complex geometries can still be represented and produced by leveraging the model’s unconditionalgenerative prior (https://huggingface.co/papers?q=generative%20prior). This leads to a more robust framework for text-based 3D shape editing that bypasses latent sinks by decoupling a model’sgeometric representation (https://huggingface.co/papers?q=geometric%20representation)power from itslinguistic sensitivity (https://huggingface.co/papers?q=linguistic%20sensitivity). Our approach addresses the limitations of current 3D pipelines and enables high-fidelitysemantic manipulation (https://huggingface.co/papers?q=semantic%20manipulation)of out-of-distribution 3D shapes. Project webpage: https://daidedou.sorpi.fr/publication/beyondprompts
View arXiv page (https://arxiv.org/abs/2604.14914)View PDF (https://arxiv.org/pdf/2604.14914)Project page (https://daidedou.sorpi.fr/publication/beyondprompts)Add to collection (https://huggingface.co/login?next=%2Fpapers%2F2604.14914)
Get this paper in your agent:
hf papers read 2604\.14914
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2604.14914 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2604.14914 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2604.14914 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollection (https://huggingface.co/new-collection)to link it from this page.
Similar Articles
Zero-shot rib design: merging training-free generative prior with topology optimization
This paper presents a novel framework that integrates a frozen text-to-image diffusion model as a training-free prior into topology optimization, enabling engineers to express design intent through natural language prompts and achieving significant compliance reductions in structural design.
Scaling Properties of Text Conditioning in Visual Generation
This paper studies empirical scaling properties for text conditioning in visual generation, showing that converged diffusion loss scales with structured language in prompts, and introduces methods to improve diffusability and promptability.
Learning 3D Editing without Paired Supervision via Generative Prior Distillation
This paper introduces a feed-forward 3D editing framework that uses generative prior distillation from foundation models to avoid paired training data, improving instruction fidelity and cross-view consistency.
EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers
EVA01 is a unified framework that integrates 3D mesh as a native modality into multimodal language models via a Mixture-of-Transformers architecture, enabling state-of-the-art text-to-3D generation and long-context multi-turn geometric editing.
Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning
This paper proposes GASP, a framework that injects geometric priors into vision-language models via deep supervision with contrastive and depth consistency losses, achieving significant improvements on 3D spatial reasoning benchmarks without using 3D VQA data.