PAMI: Part Anchored Motion for Text to Human-Object Interaction Generation

Hugging Face Daily Papers Papers
human-object-interaction text-to-motion generative-models vae computer-graphics diffusion-alternatives

Summary

PAMI 提出一种 Part-Anchored Motion 框架，通过身体部位锚点投票机制和粗到细的生成层级（PamiGen 与 PamiRefiner）实现文本条件下的全身人-物交互生成，在 InterAct 数据集上较此前 SOTA 提升 14.5% 的接触召回率。

Text-conditioned full-body human-object interaction (HOI) generation requires synthesizing human motion and object trajectories that match the input text while remaining precisely coordinated over time. Most methods represent the human and object as separate trajectories and predict the global human-object couplings. Learning this complex, dynamically changing relationship implicitly, however, often yields object drift, missed contact, and penetration. We introduce PAMI, a Part-Anchored Motion framework for Interaction generation. Inspired by the classic Hough Transform, our key idea is to localize object motion by letting body-part anchors vote for it: we express object motion relative to multiple body-part anchors and use PamiVAE to learn an interaction latent space, decoding frame-wise weights that aggregate these part-specific votes. Building on this representation, PAMI generates interactions in a coarse-to-fine hierarchy. PamiGen first generates a coarse human-object interaction from text in this structured latent space, and PamiRefiner then recursively resolves fine-grained contact geometry using a hybrid surface-sensing representation, combining long-range probes that capture overall body-part influence with short-range sensors that resolve detailed contacts near the object surface. Experiments on InterAct show that PAMI generates more faithful interactions and more accurate human-relative object motion than previous methods, achieving 14.5% higher contact recall than the previous state of the art. Extensive ablations validate the contributions of both the part-anchored voting representation and hybrid surface-sensing refinement.
Original Article
View Cached Full Text

Cached at: 10/08/26, 03:58 PM

Paper page - PAMI: Part Anchored Motion for Text to Human-Object Interaction Generation

Source: https://huggingface.co/papers/2609.38466

Abstract

Text-conditionedfull-bodyhuman-objectinteraction(HOI)generationrequiressynthesizinghumanmotionandobjecttrajectoriesthatmatchtheinputtextwhileremainingpreciselycoordinatedovertime.Mostmethodsrepresentthehumanandobjectasseparatetrajectoriesandpredicttheglobalhuman-objectcouplings.Learningthiscomplex,dynamicallychangingrelationshipimplicitly,however,oftenyieldsobjectdrift,missedcontact,andpenetration.WeintroducePAMI,aPart-AnchoredMotionframeworkforInteractiongeneration.InspiredbytheclassicHoughTransform,ourkeyideaistolocalizeobjectmotionbylettingbody-partanchorsvoteforit:weexpressobjectmotionrelativetomultiplebody-partanchorsandusePamiVAEtolearnaninteractionlatentspace,decodingframe-wiseweightsthataggregatethesepart-specificvotes.Buildingonthisrepresentation,PAMIgeneratesinteractionsinacoarse-to-finehierarchy.PamiGenfirstgeneratesacoarsehuman-objectinteractionfromtextinthisstructuredlatentspace,andPamiRefinerthenrecursivelyresolvesfine-grainedcontactgeometryusingahybridsurface-sensingrepresentation,combininglong-rangeprobesthatcaptureoverallbody-partinfluencewithshort-rangesensorsthatresolvedetailedcontactsneartheobjectsurface.ExperimentsonInterActshowthatPAMIgeneratesmorefaithfulinteractionsandmoreaccuratehuman-relativeobjectmotionthanpreviousmethods,achieving14.5%highercontactrecallthanthepreviousstateoftheart.Extensiveablationsvalidatethecontributionsofboththepart-anchoredvotingrepresentationandhybridsurface-sensingrefinement.

View arXiv pageView PDFProject pageGitHubAdd to collection

Get this paper in your agent:

hf papers read 2609\.38466

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.38466 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.38466 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.38466 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Semantic Motion Anchors: Bridging Motion and Meaning in Co-Speech Gestures

arXiv cs.CL

This paper proposes semantic motion anchors, natural-language abstractions of gesture motion for co-speech gesture retrieval and synthesis. The method discretizes 3D gestures into body-hand motion primitives and grounds them in transcripts, achieving significant improvements in text-to-gesture retrieval and user preference in generation.

PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation

Hugging Face Daily Papers

PINT 提出了一种统一模型，通过可学习的预测意图令牌（Predictive Intent Tokens）协调视觉语言推理、视频世界建模与动作生成，在无需具身预训练的情况下显著提升了机器人操作的分布外泛化能力，在 RoboTwin 2.0-Hard 和 LIBERO-Plus 等基准上大幅超越 Fast-WAM 和 StarVLA。

MAMHOI: Factorizing Scene-Aware Human-Object Interaction through Affordances

Hugging Face Daily Papers

MAMHOI 通过动作可供性（affordances）对场景感知的人-物交互生成进行解耦，利用场景条件模型预测可供性，再由可供性条件生成模型合成协调的人-物运动，从而无需配对的人-物-场景训练数据即可生成更符合场景几何、减少穿透的交互运动。

AffordanceWAM: Affordance-Aware Joint World-Action Modeling for Robot Manipulation

Hugging Face Daily Papers

AffordanceWAM 提出一种感知可供性（affordance）的生成式世界-动作模型，利用人类视频中的可供性表征联合预测未来场景与机器人动作，无需人类动作标注即可实现人类到机器人的迁移，并在 RoboCasa、CALVIN 等基准上取得一致提升。