VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control

Hugging Face Daily Papers Papers
embodied-ai spatial-intelligence benchmark evaluation visual-demonstrations active-perception metric-control

Summary

VABench introduces a benchmark to evaluate embodied spatial intelligence in models by testing their ability to observe, reason, and act through visual demonstrations and active perception. It shows that active camera control improves task success, but no model completes long-horizon episodes.

Spatial intelligence requires more than describing object locations. Under incomplete observation, models must identify and acquire missing evidence, interpret it in a common spatial frame, and act on it. We introduce VA-Bench to evaluate the complete observe-reason-act-revise loop. General-purpose MLLMs learn procedural context from RGB-only demonstrations, actively select camera viewpoints, issue metric Cartesian commands, and revise them from execution feedback. Models receive no privileged object poses, oracle trajectories, or learned action heads. A fixed model-agnostic controller executes only model-specified targets. VA-Bench contains 14 base task families (11 single-arm and three dual-arm), seven held-out geometry/layout variants, and a long-horizon five-object composition track. We evaluate 12 primary model conditions in three independent runs over the same 20 physically verified seeds per base task, reporting terminal success, nine trajectory-level behavioral diagnostics, and subtask progress. First, the best-performing model scores 100.0% on target localization and 78.9% on spatial relations in the annotated run. Its three-run macro-average task success is only 53.93+/-3.17%. Second, active camera control significantly improves task success over passive multi-view observation. In one matched comparison, success rises from 27.86% to 57.50%. Third, held-out geometric transfer can reduce task success by over 30 percentage points. No model completes a strict long-horizon episode, despite substantial partial progress. VA-Bench thus tests whether general-purpose MLLMs can turn visual demonstrations and actively acquired evidence into successful embodied action.
Original Article
View Cached Full Text

Cached at: 09/18/26, 03:01 AM

Paper page - VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control

Source: https://huggingface.co/papers/2609.19554

Abstract

Spatialintelligencerequiresmorethandescribingobjectlocations.Underincompleteobservation,modelsmustidentifyandacquiremissingevidence,interpretitinacommonspatialframe,andactonit.WeintroduceVA-Benchtoevaluatethecompleteobserve-reason-act-reviseloop.General-purposeMLLMslearnproceduralcontextfromRGB-onlydemonstrations,activelyselectcameraviewpoints,issuemetricCartesiancommands,andrevisethemfromexecutionfeedback.Modelsreceivenoprivilegedobjectposes,oracletrajectories,orlearnedactionheads.Afixedmodel-agnosticcontrollerexecutesonlymodel-specifiedtargets.VA-Benchcontains14basetaskfamilies(11single-armandthreedual-arm),sevenheld-outgeometry/layoutvariants,andalong-horizonfive-objectcompositiontrack.Weevaluate12primarymodelconditionsinthreeindependentrunsoverthesame20physicallyverifiedseedsperbasetask,reportingterminalsuccess,ninetrajectory-levelbehavioraldiagnostics,andsubtaskprogress.First,thebest-performingmodelscores100.0%ontargetlocalizationand78.9%onspatialrelationsintheannotatedrun.Itsthree-runmacro-averagetasksuccessisonly53.93+/-3.17%.Second,activecameracontrolsignificantlyimprovestasksuccessoverpassivemulti-viewobservation.Inonematchedcomparison,successrisesfrom27.86%to57.50%.Third,held-outgeometrictransfercanreducetasksuccessbyover30percentagepoints.Nomodelcompletesastrictlong-horizonepisode,despitesubstantialpartialprogress.VA-Benchthustestswhethergeneral-purposeMLLMscanturnvisualdemonstrationsandactivelyacquiredevidenceintosuccessfulembodiedaction.

View arXiv pageView PDFGitHub3Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.19554 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.19554 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.19554 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop

Hugging Face Daily Papers

Introduces ESI-BENCH, a comprehensive benchmark for embodied spatial intelligence built on OmniGibson, covering 10 task categories and 29 subcategories. Experiments show active exploration substantially outperforms passive approaches, with failures mainly due to action blindness rather than perception, revealing a metacognitive gap in models compared to humans.

SVI-Bench: A Dynamic Microworld for Strategic Video Intelligence

Hugging Face Daily Papers

Introduces SVI-Bench, a large-scale benchmark for strategic video intelligence using team sports, designed to evaluate models on dynamic scene understanding, causal reasoning, strategic simulation, and agentic synthesis. The benchmark reveals a capability cliff where models perform well on perceptual tasks but sharply degrade on higher-level strategic reasoning.