Geometric and Semantic Coupling for Interaction Understanding in 3D Scenes
Summary
This paper introduces Segment-Snap, a method that combines geometric and semantic cues to improve interaction understanding in 3D scenes, achieving significant gains in motion-gated AP and handle detection metrics.
View Cached Full Text
Cached at: 09/23/26, 03:30 AM
Paper page - Geometric and Semantic Coupling for Interaction Understanding in 3D Scenes
Source: https://huggingface.co/papers/2609.25247 Published on Sep 21
·
Submitted byhttps://huggingface.co/imsuperkong
Kongon Sep 23
Abstract
Interactionunderstandingin3Dscenesrequiresajointdescriptionofmovableparts,theirmotion,andtheregionsthroughwhichtheycanbeoperated.WepresentSegment-Snap,whichconnectstheseoutputsthroughthephysicalrelationshipbetweenpartsandhandles.Learnedpredictorsidentifybroadpartsurfacesandsmallhandles.Ageometricdecoderusesplanaranduprightpriorstoconstrainmotion,thenselectshingelinesusingpredictedhandlelocations,withouttrainingamotionregressor.Conversely,ajointpart-and-handlepredictorsuppliesadditionalhandlecandidates,whosemotionclassesarerefinedusingcontainingparts.Eachinformationtransferisappliedonce,withoutiterativefeedback.OnArticulate3Dvalidation,handleguidanceraisesmotion-gatedAPfrom13.74%to40.98%atfixedmasksandaxes.AdditionalhandlecandidatesraisehandleAPfrom24.63%to29.65%;part-basedclasscorrectionadds0.98points,andfullcontextreaches30.99%.Repeatedtraining,learned-decodercontrolsandpairedvisualizationsestablishthebenefitsandlimitationsofcombininggeometricandsemanticevidenceforinteractionunderstanding.
View arXiv pageView PDFProject pageGitHub1Add to collection
Get this paper in your agent:
hf papers read 2609\.25247
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.25247 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.25247 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.25247 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence
This paper introduces a post-training framework that leverages 3D priors from SAM3D to improve semantic correspondence in 2D foundation features, addressing issues like left-right confusion and repeated parts. The method uses instance-specific 3D reconstruction without pose annotations or spherical geometry shortcuts.
One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models
Introduces MultiDepth-3k, a benchmark to evaluate depth-layer preferences in monocular depth foundation models, and shows Laplacian Visual Prompting can alter reported depth layers, suggesting complementary geometric hypotheses exist across models.
Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning
This paper proposes GASP, a framework that injects geometric priors into vision-language models via deep supervision with contrastive and depth consistency losses, achieving significant improvements on 3D spatial reasoning benchmarks without using 3D VQA data.
Anchor Divergence for Semantic Geometry in Contrastive Learning
This paper introduces Anchor Divergences, a method that leverages the interplay between contrastive learning, exponential families, and information geometry to define context-specific semantic geometries on fixed CLIP-like representations, allowing similarity to reflect task-relevant attributes (e.g., dog breed vs. background) instead of a single fixed cosine geometry. Retrieval experiments show the approach is both effective and efficient for specifying context-dependent semantic similarity.
SAM 3: Segment Anything with Concepts
SAM 3 introduces a unified model for promptable concept segmentation and tracking, achieving state-of-the-art performance with a decoupled recognition and localization architecture and a scalable data engine.