PhyGenHOI: Physically-Aware 4D Generation of Dynamic Human-Object Interactions
Summary
PhyGenHOI is a novel framework that generates physically accurate 4D human-object interactions by coupling motion diffusion models with material point method simulations using 3D Gaussian representations.
View Cached Full Text
Cached at: 05/29/26, 11:01 AM
Paper page - PhyGenHOI: Physically-Aware 4D Generation of Dynamic Human-Object Interactions
Source: https://huggingface.co/papers/2605.30268
Abstract
PhyGenHOI synthesizes physically accurate 4D human-object interactions by combining motion diffusion models with material point method simulations using 3D Gaussian representations.
We address the task of generating physically accurate and visually faithful 4D Human-Object Interaction (HOI). Given a static 3D human and target object represented as3D Gaussian Splats(3DGS), our goal is to synthesize dynamic scenes where the human actively engages with the object through actions, such as punching or kicking, in accordance with a given input text. To this end, we introduce PhyGenHOI, a novel framework that couples generative human motion with an explicit physical object simulation. We model the human as a semantic agent driven by aMotion Diffusion Model(MDM) and the object as a physical agent simulated via theMaterial Point Method(MPM), utilizing 3D Gaussians as a unified, differentiable representation. We supervise their interaction through three coupled mechanisms: (1) AWindowed Attraction Lossthat temporally synchronizes generative motion to intercept the object; (2) AContact-Driven Re-simulationstep that triggers physically consistent momentum transfer upon impact; and (3) AMasked Video-SDSobjective that injects video-based priors to enhance contact fidelity. Experiments show PhyGenHOI generates physically consistent 4D HOI across diverse actions, humans, and objects, outperforming baselines. Project page and videos: https://omerbenishu.github.io/PhyGenHOI/
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2605\.30268
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.30268 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.30268 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.30268 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
OneHOI: Unifying Human-Object Interaction Generation and Editing
OneHOI is a unified diffusion transformer framework that consolidates human-object interaction (HOI) generation and editing into a single conditional denoising process using relational modeling and structured attention mechanisms. The approach achieves state-of-the-art results across both HOI generation and editing tasks with support for multiple control modalities.
CoInteract: Physically-Consistent Human-Object Interaction Video Synthesis via Spatially-Structured Co-Generation
CoInteract introduces an end-to-end Diffusion Transformer framework that jointly models RGB appearance and HOI geometry to generate physically-plausible human-object interaction videos with stable hands/faces and zero inference overhead.
HarmoHOI: Harmonizing Appearance and 3D Motion for Multi-view Hand-Object Interaction Synthesis
HarmoHOI is a unified diffusion framework that jointly generates synchronized multi-view hand-object interaction videos and globally aligned 3D point tracks, achieving state-of-the-art performance in visual quality, motion plausibility, and multi-view geometric consistency.
PhysiFormer: Learning to Simulate Mechanics in World Space
PhysiFormer uses coordinate-space diffusion to generate physically-plausible 3D object motions without explicit inductive biases, enabling efficient multi-object reasoning and generalization to complex materials and geometries.
MAMHOI: Factorizing Scene-Aware Human-Object Interaction through Affordances
MAMHOI 通过动作可供性（affordances）对场景感知的人-物交互生成进行解耦，利用场景条件模型预测可供性，再由可供性条件生成模型合成协调的人-物运动，从而无需配对的人-物-场景训练数据即可生成更符合场景几何、减少穿透的交互运动。