A Plug-and-Play 2D Motion Interface for Real-World Motion Language Models
Summary
This paper introduces a plug-and-play 2D motion interface that enables pretrained motion language models to process 2D inputs from monocular videos without retraining, improving real-world applicability.
View Cached Full Text
Cached at: 08/18/26, 11:52 AM
Paper page - A Plug-and-Play 2D Motion Interface for Real-World Motion Language Models
Source: https://huggingface.co/papers/2608.15984
Abstract
A plug-and-play 2D motion interface allows pretrained motion language models to process 2D inputs without retraining, improving real-world applicability.
Motion Language Models(MoLMs) typically understand human motions by tokenizing 3D motion and processing the resulting tokens using a language model. However, obtaining accurate 3D motions frommonocular videos is challenging, limiting their real-world applicability. To address this issue, we introduce a plug-and-play2D Motion Interfacethat enables 3D-pretrained MoLMs to accept 2D motion inputs without modifying or fine-tuning the original models. Experiments on public datasets show that our method achieves performance comparable to 3D motion inputs across multiple MoLMs and outperforms training MoLMs from scratch on 2D motions. We further construct a monocular real-world video motion evaluation dataset and introduce areal-video adapter, demonstrating the usefulness of 2D motions over 3D motions under the evaluated monocular pose-estimation setting. These results suggest that 2D motion provides a practical interface for deploying MoLMs in real-world motion understanding settings. Code is available at https://github.com/irajisamurai/2D-Motion-Interface.
View arXiv pageView PDFGitHub1Add to collection
Models citing this paper1
#### KanameYOkoYAMA/2d-motion-interface Other• Updatedabout 10 hours ago
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.15984 in a dataset README.md to link it from this page.
Spaces citing this paper1
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
VideoMDM: Towards 3D Human Motion Generation From 2D Supervision
VideoMDM trains 3D human motion priors from 2D poses using a diffusion framework with 2D reprojection loss and 3D motion regularizers, achieving near-3D supervised performance without requiring 3D ground truth.
MotionVLA: Vision-Language-Action Model for Humanoid Motion
Proposes MotionVLA, a vision-language-action model for humanoid motion generation using a dual-stream frequency tokenizer that separately encodes pose and physical dynamics, achieving better diversity and consistency.
MolmoMotion: Language-guided 3D motion forecasting
MolmoMotion is a new language-guided 3D motion forecasting model that predicts future 3D point trajectories from video frames and action descriptions, achieving stronger performance than existing methods. Alongside the model, a large dataset (MolmoMotion-1M) and a benchmark (PointMotionBench) are released.
Sim-to-Real Transfer of Vision-Language Navigation in Continuous Environments Using an Ackermann-Steered Mobile Robot
This paper presents a sim-to-real transfer approach for Vision-Language Navigation in continuous environments, using a Cross-Modal Attention architecture fine-tuned with real-world data from a custom Ackermann-steered robot equipped with camera and LiDAR, achieving effective offline navigation without relying on navigation graphs or panoramic views.
SimpleMemVLA: A Simple but Effective Native-Video Memory for Vision-Language-Action Models
SimpleMemVLA introduces a simple memory mechanism for Vision-Language-Action models by feeding intact timestamped video history into a pretrained VLM backbone, achieving state-of-the-art results on long-horizon manipulation tasks without dedicated memory modules.