SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them

Hugging Face Daily Papers Papers
spatial-reasoning vision-language-models embodied-agents tool-use benchmark fine-tuning reinforcement-learning

Summary

SpatialCLI presents a framework that trains vision-language models to use specialist spatial tools and then internalize those capabilities, boosting Qwen3-VL-8B-Instruct on the MindCube benchmark from 29.3% to 84.6% with tools.

Vision-language models (VLMs) are increasingly used in embodied agents to interpret visual inputs, reason about spatial relationships, and make task-level decisions based on that reasoning. However, a fundamental capability mismatch remains: general VLMs can reason about the overall task but often miss the visual details that determine success, while specialist vision models can capture those details but cannot translate them into task-level decisions. In this work, we propose SpatialCLI, a framework that teaches VLMs to reason with spatial tools and progressively internalize the specialist perceptual capabilities they provide. SpatialCLI proceeds in three stages: (1) Call exposes specialist vision models as spatial tools to augment the VLM's perception; (2) Learn uses Cold-Start SFT and agentic RL to improve tool use; and (3) Internalize verbalizes successful tool-use trajectories to internalize specialist perceptual capabilities. We further introduce SpatialCLI-Bench, a 516-example benchmark for compositional perception across localization, segmentation, depth, and pose. On MindCube, SpatialCLI raises Qwen3-VL-8B-Instruct from 29.3% to 84.6% with tools, surpassing GPT-5.6 Sol with tools (72.1%), while retaining 73.8% without tools after internalization.
Original Article
View Cached Full Text

Cached at: 07/31/26, 01:51 AM

Paper page - SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them

Source: https://huggingface.co/papers/2607.27703 Authors:

,

,

,

,

,

,

,

,

,

,

,

Abstract

Vision-languagemodels(VLMs)areincreasinglyusedinembodiedagentstointerpretvisualinputs,reasonaboutspatialrelationships,andmaketask-leveldecisionsbasedonthatreasoning.However,afundamentalcapabilitymismatchremains:generalVLMscanreasonabouttheoveralltaskbutoftenmissthevisualdetailsthatdeterminesuccess,whilespecialistvisionmodelscancapturethosedetailsbutcannottranslatethemintotask-leveldecisions.Inthiswork,weproposeSpatialCLI,aframeworkthatteachesVLMstoreasonwithspatialtoolsandprogressivelyinternalizethespecialistperceptualcapabilitiestheyprovide.SpatialCLIproceedsinthreestages:(1)CallexposesspecialistvisionmodelsasspatialtoolstoaugmenttheVLM’sperception;(2)LearnusesCold-StartSFTandagenticRLtoimprovetooluse;and(3)Internalizeverbalizessuccessfultool-usetrajectoriestointernalizespecialistperceptualcapabilities.WefurtherintroduceSpatialCLI-Bench,a516-examplebenchmarkforcompositionalperceptionacrosslocalization,segmentation,depth,andpose.OnMindCube,SpatialCLIraisesQwen3-VL-8B-Instructfrom29.3%to84.6%withtools,surpassingGPT-5.6Solwithtools(72.1%),whileretaining73.8%withouttoolsafterinternalization.

View arXiv pageView PDFGitHubAdd to collection

Get this paper in your agent:

hf papers read 2607\.27703

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.27703 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.27703 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.27703 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes

Hugging Face Daily Papers

SpatialAct is a new simulator-grounded benchmark that probes whether VLM agents can perform coherent spatial reasoning and translate it into actions in 3D environments across multi-turn feedback settings. Experiments reveal a significant reasoning-to-action gap, with current VLMs struggling to maintain spatial beliefs and produce reliable actions despite performing well on isolated reasoning tasks.

Reinforcing Dual-Path Reasoning in Spatial Vision Language Models

Hugging Face Daily Papers

This paper introduces SR-REAL, a unified framework for spatial vision-language models that combines linguistic deduction and 3D geometric reasoning via reinforcement learning, enabling robust multi-step spatial reasoning across diverse tasks.