HumanCLAW: Can Vision-Language Models Act Through a Body?

Hugging Face Daily Papers Papers
vision-language-models embodied-ai evaluation-framework benchmark physical-action

Summary

This paper introduces HumanCLAW, an evaluation framework that decouples action decision-making from low-level execution to assess whether vision-language models can act through a physical body. Testing nine state-of-the-art VLMs on 1,218 episodes across 41 scenes, the best model achieves only a 16.8% success rate, revealing that current VLMs lack embodied self-awareness.

Evaluating whether a vision-language model (VLM) can act through a physical body is challenging. The outcome of an action couples the VLM's decision with motor control. When a task fails, it is hard to tell whether the VLM made a bad choice or the motor controller simply failed to execute it, e.g., losing balance and falling. In this work, we introduce HumanCLAW, an evaluation framework that decouples action decision-making from low-level execution. At every step, a harnessed, off-the-shelf VLM issues an atomic skill command, and the command is translated into a sub-second chunk of continuous full-body motion with real physical consequences, including gravity and collisions. The body can therefore act freely in the physical world, while execution-side disturbances, balance and motor errors, are factored out. What remains measurable is the model's action intelligence: its moment-to-moment choice of what the body should execute next. Based on this framework, we build HumanCLAW-Bench: 1,218 long-horizon, egocentric find-navigate-interact episodes across 41 indoor scenes. We test nine state-of-the-art VLMs and find that none solves the benchmark; the best model reaches only a 16.8% success rate. Recognizing the target is not the bottleneck. What current VLMs lack is embodied self-awareness: they lose track of their own body, failing to tell where it is, whether it has reached the goal, or whether it has hit an obstacle.
Original Article
View Cached Full Text

Cached at: 07/30/26, 05:46 AM

Paper page - HumanCLAW: Can Vision-Language Models Act Through a Body?

Source: https://huggingface.co/papers/2607.27180 Published on Jul 29

#2 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Evaluatingwhetheravision-languagemodel(VLM)canactthroughaphysicalbodyischallenging.TheoutcomeofanactioncouplestheVLM’sdecisionwithmotorcontrol.Whenataskfails,itishardtotellwhethertheVLMmadeabadchoiceorthemotorcontrollersimplyfailedtoexecuteit,e.g.,losingbalanceandfalling.Inthiswork,weintroduceHumanCLAW,anevaluationframeworkthatdecouplesactiondecision-makingfromlow-levelexecution.Ateverystep,aharnessed,off-the-shelfVLMissuesanatomicskillcommand,andthecommandistranslatedintoasub-secondchunkofcontinuousfull-bodymotionwithrealphysicalconsequences,includinggravityandcollisions.Thebodycanthereforeactfreelyinthephysicalworld,whileexecution-sidedisturbances,balanceandmotorerrors,arefactoredout.Whatremainsmeasurableisthemodel’sactionintelligence:itsmoment-to-momentchoiceofwhatthebodyshouldexecutenext.Basedonthisframework,webuildHumanCLAW-Bench:1,218long-horizon,egocentricfind-navigate-interactepisodesacross41indoorscenes.Wetestninestate-of-the-artVLMsandfindthatnonesolvesthebenchmark;thebestmodelreachesonlya16.8%successrate.Recognizingthetargetisnotthebottleneck.WhatcurrentVLMslackisembodiedself-awareness:theylosetrackoftheirownbody,failingtotellwhereitis,whetherithasreachedthegoal,orwhetherithashitanobstacle.

View arXiv pageView PDFProject pageGitHub0Add to collection

Get this paper in your agent:

hf papers read 2607\.27180

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.27180 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.27180 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.27180 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding

arXiv cs.AI

CLAP proposes a method to convert pretrained vision-language models (VLMs) into vision-language-action models (VLAs) by prepending natural-language action descriptions to action token sequences, preserving semantic capabilities without architectural changes. It achieves 90.8% on LIBERO and improves robustness.

MotionVLA: Vision-Language-Action Model for Humanoid Motion

Hugging Face Daily Papers

Proposes MotionVLA, a vision-language-action model for humanoid motion generation using a dual-stream frequency tokenizer that separately encodes pose and physical dynamics, achieving better diversity and consistency.

LabVLA: Grounding Vision-Language-Action Models in Scientific Laboratories

Hugging Face Daily Papers

LabVLA is a vision-language-action model for scientific laboratory automation, trained with a two-stage approach combining action token pretraining and flow matching. It achieves state-of-the-art success rates on the LabUtopia benchmark by leveraging simulated data to bridge the gap between household demonstrations and lab-specific tasks.

Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence

arXiv cs.AI

This paper presents Capek 0.5, an embodied vision-language model built around an execution-centric capability taxonomy, using specialist training via reinforcement learning followed by weight-space merging and routed policy-space distillation. It introduces a new state verification benchmark and demonstrates strong performance across embodied and general capabilities at 2B and 35B-A3B scales.