Beyond Prompts: Unconditional 3D Inversion for Out-of-Distribution Shapes
Summary
This paper identifies and addresses 'latent sink traps' in text-to-3D generative models where they become insensitive to text prompts, proposing a framework that decouples geometric representation from linguistic sensitivity to enable robust text-based 3D shape editing of out-of-distribution shapes.
View Cached Full Text
Cached at: 04/20/26, 08:27 AM
Paper page - Beyond Prompts: Unconditional 3D Inversion for Out-of-Distribution Shapes
Source: https://huggingface.co/papers/2604.14914
Abstract
State-of-the-art text-to-3D generative models suffer from latent sink traps where they lose sensitivity to text prompts, but a robust framework can overcome this by decoupling geometric representation from linguistic sensitivity.
Text-driven inversion of generative models is a core paradigm for manipulating 2D or 3D content, unlocking numerous applications such as text-based editing, style transfer, or inverse problems. However, it relies on the assumption that generative models remain sensitive to natural language prompts. We demonstrate that for state-of-the-art nativetext-to-3D generative models (https://huggingface.co/papers?q=text-to-3D%20generative%20models), this assumption often collapses. We identify a critical failure mode where generation trajectories are drawn into latent ``sink traps’’: regions where the model becomes insensitive toprompt modifications (https://huggingface.co/papers?q=prompt%20modifications). In these regimes, changes to the input text fail to alter internal representations in a way that alters the output geometry. Crucially, we observe that this is not a limitation of the model’s geometric expressivity; the same generative models possess the ability to produce a vast diversity of shapes but, as we demonstrate, become insensitive toout-of-distribution text guidance (https://huggingface.co/papers?q=out-of-distribution%20text%20guidance). We investigate this behavior by analyzing the sampling trajectories of the generative model, and find that complex geometries can still be represented and produced by leveraging the model’s unconditionalgenerative prior (https://huggingface.co/papers?q=generative%20prior). This leads to a more robust framework for text-based 3D shape editing that bypasses latent sinks by decoupling a model’sgeometric representation (https://huggingface.co/papers?q=geometric%20representation)power from itslinguistic sensitivity (https://huggingface.co/papers?q=linguistic%20sensitivity). Our approach addresses the limitations of current 3D pipelines and enables high-fidelitysemantic manipulation (https://huggingface.co/papers?q=semantic%20manipulation)of out-of-distribution 3D shapes. Project webpage: https://daidedou.sorpi.fr/publication/beyondprompts
View arXiv page (https://arxiv.org/abs/2604.14914)View PDF (https://arxiv.org/pdf/2604.14914)Project page (https://daidedou.sorpi.fr/publication/beyondprompts)Add to collection (https://huggingface.co/login?next=%2Fpapers%2F2604.14914)
Get this paper in your agent:
hf papers read 2604\.14914
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2604.14914 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2604.14914 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2604.14914 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollection (https://huggingface.co/new-collection)to link it from this page.
Similar Articles
Scaling Properties of Text Conditioning in Visual Generation
This paper studies empirical scaling properties for text conditioning in visual generation, showing that converged diffusion loss scales with structured language in prompts, and introduces methods to improve diffusability and promptability.
EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers
EVA01 is a unified framework that integrates 3D mesh as a native modality into multimodal language models via a Mixture-of-Transformers architecture, enabling state-of-the-art text-to-3D generation and long-context multi-turn geometric editing.
Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning
This paper proposes GASP, a framework that injects geometric priors into vision-language models via deep supervision with contrastive and depth consistency losses, achieving significant improvements on 3D spatial reasoning benchmarks without using 3D VQA data.
Decomposing how prompting steers behavior
This paper introduces a nested geometric decomposition framework to analyze how prompting reorganizes internal representations in large language and vision-language models. The authors show that affine transformations, particularly cross-dimensional linear mixing, are key to explaining prompt-induced behavioral changes.
VGGT-Edit: Feed-forward Native 3D Scene Editing with Residual Field Prediction
VGGT-Edit proposes a feed-forward framework for text-conditioned native 3D scene editing using depth-synchronized text injection and residual field prediction, achieving superior quality and efficiency over 2D-lifting approaches.