MAMHOI: Factorizing Scene-Aware Human-Object Interaction through Affordances

Hugging Face Daily Papers Papers

Summary

MAMHOI 通过动作可供性(affordances)对场景感知的人-物交互生成进行解耦,利用场景条件模型预测可供性,再由可供性条件生成模型合成协调的人-物运动,从而无需配对的人-物-场景训练数据即可生成更符合场景几何、减少穿透的交互运动。

Generating realistic human-object interactions (HOI) in complex 3D scenes requires two complementary capabilities: reasoning about interaction feasibility in the environment and synthesizing realistic human-object motion. However, supervision for these capabilities is rarely available jointly at scale. Human-scene datasets provide rich information about environment-aware motion, while human-object datasets capture detailed interaction dynamics, yet paired human-object-scene data remain scarce. We present MAMHOI, an affordance-mediated factorization for scene-aware human-object interaction generation. MAMHOI factorizes scene-aware HOI generation through an explicit motion-affordance interface between scene understanding and motion synthesis: a scene-conditioned model first predicts where and how an interaction can be feasibly executed, and an affordance-conditioned HOI model then generates the corresponding human-object motion. This factorization allows scene understanding and interaction dynamics to be learned from complementary sources of supervision without requiring paired human-object-scene data. Experiments in complex indoor environments show that MAMHOI reduces object--scene penetration while better preserving human--object interaction quality, yielding more realistic and physically feasible scene-aware interactions. Project page: https://leimingyuan.github.io/MAMHOI-project-page/
Original Article
View Cached Full Text

Cached at: 10/12/26, 02:40 PM

Paper page - MAMHOI: Factorizing Scene-Aware Human-Object Interaction through Affordances

Source: https://huggingface.co/papers/2610.12416 We introduce MAMHOI, a framework for generating scene-aware human–object interactions in complex 3D environments. Our key observation is that this task requires two complementary capabilities: understanding where an interaction can be feasibly performed within a scene and synthesizing how the human and object should move together. However, paired human–object–scene motion data providing both forms of supervision remains scarce.

MAMHOI addresses this challenge by factorizing scene-aware HOI generation through motion affordances. A scene-conditioned model learns to predict task-relevant affordances from diverse indoor human–scene interaction data, while an affordance-conditioned generative model learns coordinated human–object dynamics from scene-free HOI data. The predicted affordance provides a shared representation that connects scene understanding with interaction synthesis.

This factorization allows MAMHOI to adapt learned human–object interactions to complex environments without requiring paired human–object–scene training data. Our experiments show that the resulting motions better respect scene geometry, substantially reduce object–scene penetration, and preserve the quality of human–object interactions.

We welcome comments, questions, and feedback from the community!

Similar Articles

AffordanceWAM: Affordance-Aware Joint World-Action Modeling for Robot Manipulation

Hugging Face Daily Papers

AffordanceWAM 提出一种感知可供性(affordance)的生成式世界-动作模型,利用人类视频中的可供性表征联合预测未来场景与机器人动作,无需人类动作标注即可实现人类到机器人的迁移,并在 RoboCasa、CALVIN 等基准上取得一致提升。

AFUN: Towards an Affordance Foundation Model for Functionality Understanding

Hugging Face Daily Papers

AFUN proposes an affordance foundation model that predicts functional masks and 3D motion curves from RGB-D observations and language descriptions, enabling generalizable robot manipulation across diverse environments. The model outperforms baselines on multiple benchmarks and can be deployed for real-world tasks without fine-tuning.