Training-Free Dense Hand Contact Estimation with Multi-Modal Large Language Models

Hugging Face Daily Papers Papers
hand-contact-estimation multi-modal large-language-models training-free zero-shot computer-vision

Summary

The paper presents ContactPrompt, a training-free and zero-shot approach for dense hand contact estimation using multi-modal large language models, outperforming supervised methods without training.

Dense hand contact estimation requires both high-level semantic understanding and fine-grained geometric reasoning of human interaction to accurately localize contact regions. Recently, multi-modal large language models (MLLMs) have demonstrated strong capabilities in understanding visual semantics, enabled by vision-language priors learned from large-scale data. However, leveraging MLLMs for dense hand contact estimation remains underexplored. There are two major challenges in applying MLLMs to dense hand contact estimation. First, encoding explicit 3D hand geometry is difficult, as MLLMs primarily operate on vision and language modalities. Second, capturing fine-grained vertex-level contact remains challenging, as MLLMs tend to focus on high-level semantics rather than detailed geometric reasoning. To address these challenges, we propose ContactPrompt, a training-free and zero-shot approach for dense hand contact estimation using MLLMs. To effectively encode 3D hand geometry, we introduce a detailed hand-part segmentation and a part-wise vertex-grid representation that provides structured, localized geometric information. To enable accurate and efficient dense contact prediction, we develop a multi-stage structured contact reasoning with part conditioning, progressively bridging global semantics and fine-grained geometry. Therefore, our method effectively leverages the reasoning capabilities of MLLMs while enabling precise dense hand contact estimation. Surprisingly, the proposed approach outperforms previous supervised methods trained on large-scale dense contact datasets without requiring any training. The codes will be released.
Original Article
View Cached Full Text

Cached at: 05/12/26, 10:52 AM

Paper page - Training-Free Dense Hand Contact Estimation with Multi-Modal Large Language Models

Source: https://huggingface.co/papers/2605.05886

Abstract

ContactPrompt enables zero-shot dense hand contact estimation using multi-modal large language models through structured 3D hand geometry encoding and multi-stage contact reasoning.

Dense hand contact estimationrequires both high-level semantic understanding and fine-grained geometric reasoning of human interaction to accurately localize contact regions. Recently,multi-modal large language models(MLLMs) have demonstrated strong capabilities in understanding visual semantics, enabled byvision-language priorslearned from large-scale data. However, leveraging MLLMs fordense hand contact estimationremains underexplored. There are two major challenges in applying MLLMs todense hand contact estimation. First, encoding explicit 3D hand geometry is difficult, as MLLMs primarily operate on vision and language modalities. Second, capturing fine-grained vertex-level contact remains challenging, as MLLMs tend to focus on high-level semantics rather than detailed geometric reasoning. To address these challenges, we propose ContactPrompt, a training-free and zero-shot approach fordense hand contact estimationusing MLLMs. To effectively encode 3D hand geometry, we introduce a detailedhand-part segmentationand a part-wisevertex-grid representationthat provides structured, localized geometric information. To enable accurate and efficient dense contact prediction, we develop amulti-stage structured contact reasoningwithpart conditioning, progressively bridging global semantics and fine-grained geometry. Therefore, our method effectively leverages the reasoning capabilities of MLLMs while enabling precisedense hand contact estimation. Surprisingly, the proposed approach outperforms previous supervised methods trained on large-scale dense contact datasets without requiring any training. The codes will be released.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2605\.05886

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.05886 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.05886 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.05886 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles