Geometric and Semantic Coupling for Interaction Understanding in 3D Scenes

Hugging Face Daily Papers Papers

Summary

This paper introduces Segment-Snap, a method that combines geometric and semantic cues to improve interaction understanding in 3D scenes, achieving significant gains in motion-gated AP and handle detection metrics.

Interaction understanding in 3D scenes requires a joint description of movable parts, their motion, and the regions through which they can be operated. We present Segment-Snap, which connects these outputs through the physical relationship between parts and handles. Learned predictors identify broad part surfaces and small handles. A geometric decoder uses planar and upright priors to constrain motion, then selects hinge lines using predicted handle locations, without training a motion regressor. Conversely, a joint part-and-handle predictor supplies additional handle candidates, whose motion classes are refined using containing parts. Each information transfer is applied once, without iterative feedback. On Articulate3D validation, handle guidance raises motion-gated AP from 13.74% to 40.98% at fixed masks and axes. Additional handle candidates raise handle AP from 24.63% to 29.65%; part-based class correction adds 0.98 points, and full context reaches 30.99%. Repeated training, learned-decoder controls and paired visualizations establish the benefits and limitations of combining geometric and semantic evidence for interaction understanding.
Original Article
View Cached Full Text

Cached at: 09/23/26, 03:30 AM

Paper page - Geometric and Semantic Coupling for Interaction Understanding in 3D Scenes

Source: https://huggingface.co/papers/2609.25247 Published on Sep 21

·

Submitted byhttps://huggingface.co/imsuperkong

Kongon Sep 23

Abstract

Interactionunderstandingin3Dscenesrequiresajointdescriptionofmovableparts,theirmotion,andtheregionsthroughwhichtheycanbeoperated.WepresentSegment-Snap,whichconnectstheseoutputsthroughthephysicalrelationshipbetweenpartsandhandles.Learnedpredictorsidentifybroadpartsurfacesandsmallhandles.Ageometricdecoderusesplanaranduprightpriorstoconstrainmotion,thenselectshingelinesusingpredictedhandlelocations,withouttrainingamotionregressor.Conversely,ajointpart-and-handlepredictorsuppliesadditionalhandlecandidates,whosemotionclassesarerefinedusingcontainingparts.Eachinformationtransferisappliedonce,withoutiterativefeedback.OnArticulate3Dvalidation,handleguidanceraisesmotion-gatedAPfrom13.74%to40.98%atfixedmasksandaxes.AdditionalhandlecandidatesraisehandleAPfrom24.63%to29.65%;part-basedclasscorrectionadds0.98points,andfullcontextreaches30.99%.Repeatedtraining,learned-decodercontrolsandpairedvisualizationsestablishthebenefitsandlimitationsofcombininggeometricandsemanticevidenceforinteractionunderstanding.

View arXiv pageView PDFProject pageGitHub1Add to collection

Get this paper in your agent:

hf papers read 2609\.25247

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.25247 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.25247 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.25247 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence

Hugging Face Daily Papers

This paper introduces a post-training framework that leverages 3D priors from SAM3D to improve semantic correspondence in 2D foundation features, addressing issues like left-right confusion and repeated parts. The method uses instance-specific 3D reconstruction without pose annotations or spherical geometry shortcuts.

Anchor Divergence for Semantic Geometry in Contrastive Learning

arXiv cs.AI

This paper introduces Anchor Divergences, a method that leverages the interplay between contrastive learning, exponential families, and information geometry to define context-specific semantic geometries on fixed CLIP-like representations, allowing similarity to reflect task-relevant attributes (e.g., dog breed vs. background) instead of a single fixed cosine geometry. Retrieval experiments show the approach is both effective and efficient for specifying context-dependent semantic similarity.

SAM 3: Segment Anything with Concepts

Papers with Code Trending

SAM 3 introduces a unified model for promptable concept segmentation and tracking, achieving state-of-the-art performance with a decoupled recognition and localization architecture and a scalable data engine.