Self-Evolving Neuro-Symbolic Skills for Tool-Augmented Spatial Reasoning

arXiv cs.AI Papers

Summary

Presents NeSy-Spatial, a neuro-symbolic framework that self-evolves spatial reasoning skills by composing tool-use and geometry skills, improving accuracy on spatial reasoning benchmarks.

arXiv:2608.07955v1 Announce Type: new Abstract: Large vision-language models have achieved strong performance in multimodal reasoning, but they remain unreliable on fine-grained spatial tasks that demand both precise spatial perception and fine-grained geometric computation beyond end-to-end generation. Tool augmentation offers a natural solution, while existing methods either plan tool calls from scratch without explicit dependency constraints or rely on fixed pipelines that are redundant and generalize poorly across spatial tasks. An effective spatial reasoning agent should instead accumulate reusable experience and adaptively compose it for new problems. To this end, we propose NeSy-Spatial, a neuro-symbolic framework for self-evolving spatial skills. NeSy-Spatial abstracts tool interactions and geometric operations into typed executable atomic instructions and composes them into two complementary skill types: Tool-Use Skills for organizing tool execution and Geometry Skills for structured geometric reasoning. During inference, NeSy-Spatial retrieves and executes relevant skills in a closed-loop process. During evolution, it analyzes buffered successful and failed trajectories to refine skill structures and prune unreliable or inactive entries. Experiments on three spatial reasoning benchmarks show that NeSy-Spatial consistently improves reasoning accuracy with more precise tool utilization.
Original Article
View Cached Full Text

Cached at: 08/11/26, 08:04 AM

# Self-Evolving Neuro-Symbolic Skills for Tool-Augmented Spatial Reasoning
Source: [https://arxiv.org/html/2608.07955](https://arxiv.org/html/2608.07955)
Shi\-Yu Tian1,2\\equalcontrib, Zhuo\-Xia Wang1,2\\equalcontrib, Xuan\-Yi Zhu1,2, Zhi Zhou1, Xinwei Yang3, Kun\-Yang Yu1,2, Ming Yang1,2, Yang Chen1, Yu\-Feng Li1,2\\corresponding

###### Abstract

Large vision\-language models have achieved strong performance in multimodal reasoning, but they remain unreliable on fine\-grained spatial tasks that demand both precise spatial perception and fine\-grained geometric computation beyond end\-to\-end generation\. Tool augmentation offers a natural solution, while existing methods either plan tool calls from scratch without explicit dependency constraints or rely on fixed pipelines that are redundant and generalize poorly across spatial tasks\. An effective spatial reasoning agent should instead accumulate reusable experience and adaptively compose it for new problems\. To this end, we proposeNeSy\-Spatial, a neuro\-symbolic framework for self\-evolving spatial skills\.NeSy\-Spatialabstracts tool interactions and geometric operations into typed executable atomic instructions and composes them into two complementary skill types:Tool\-Use Skillsfor organizing tool execution andGeometry Skillsfor structured geometric reasoning\. During inference,NeSy\-Spatialretrieves and executes relevant skills in a closed\-loop process\. During evolution, it analyzes buffered successful and failed trajectories to refine skill structures and prune unreliable or inactive entries\. Experiments on three spatial reasoning benchmarks show thatNeSy\-Spatialconsistently improves reasoning accuracy with more precise tool utilization\.

## 1Introduction

Large vision\-language models \(LVLMs\) have made substantial progress in general visual understanding\(OpenAI[2023](https://arxiv.org/html/2608.07955#bib.bib14)\), yet they remain unreliable for fine\-grained spatial reasoning\(Yanget al\.[2025a](https://arxiv.org/html/2608.07955#bib.bib105); Chenet al\.[2025c](https://arxiv.org/html/2608.07955#bib.bib129)\)\. Unlike generic visual understanding, spatial reasoning often requires explicit geometric reasoning and intermediate verification beyond end\-to\-end generation\. Such reasoning is difficult to accomplish using LVLMs alone, as it depends on precise visual perception and structured geometric computation\. Tool augmentation has therefore emerged as an effective solution\(Zhanget al\.[2025](https://arxiv.org/html/2608.07955#bib.bib114); Yuet al\.[2026](https://arxiv.org/html/2608.07955#bib.bib158); Tianet al\.[2026](https://arxiv.org/html/2608.07955#bib.bib171)\), enabling LVLMs to leverage specialized vision models and external executors such as depth estimators\(Yanget al\.[2024](https://arxiv.org/html/2608.07955#bib.bib137)\)or reconstruction models\(Wanget al\.[2025a](https://arxiv.org/html/2608.07955#bib.bib139)\)to obtain reliable intermediate evidence for reasoning\.

![Refer to caption](https://arxiv.org/html/2608.07955v1/Figures/intro_v1.png)Figure 1:Limitations of tool\-augmented spatial reasoning agents: dynamic planning may violate tool dependencies, while fixed pipelines can be redundant and generalize poorly\.However, existing tool\-augmented methods mainly follow two paradigms\. One line of work dynamically plans tool calls from a general\-purpose toolbox for each query\(Tianet al\.[2026](https://arxiv.org/html/2608.07955#bib.bib171); Wuet al\.[2026](https://arxiv.org/html/2608.07955#bib.bib166)\), while another executes a manually designed pipeline tailored to a specific class of spatial reasoning tasks\(Choet al\.[2026](https://arxiv.org/html/2608.07955#bib.bib167); Chenet al\.[2026a](https://arxiv.org/html/2608.07955#bib.bib168)\)\. Although dynamic planning is highly flexible, it lacks explicit constraints on tool dependencies and execution order\. As a result, agents may generate invalid tool sequences—for example, invoking a Python interpreter before acquiring prerequisite information, such as grounded object coordinates or point cloud representations, causing subsequent reasoning to fail\. In contrast, fixed pipelines enforce a valid execution order but often invoke unnecessary tools regardless of task complexity\. Even simple queries may trigger the entire workflow, leading to redundant computation and reduced efficiency, while handcrafted pipelines are difficult to generalize across diverse spatial reasoning scenarios\.

Therefore, a spatial reasoning agent should abstract retrievable and composable skills from historical trajectories, rather than planning from scratch or relying on fixed templates\. However, this goal presents two challenges\. First, spatial skills involve both tool invocation and geometric processing\. Existing methods typically rely on a Python interpreter to perform ad hoc computations over tool outputs, resulting in limited robustness and success rates\. Second, skills are difficult to extract directly from trajectories, as successful trajectories may contain redundant steps, while failed ones may still include locally effective operations\. The key challenge is therefore to identify reusable atomic instructions and compose them into high\-level skills that jointly support tool execution and geometric reasoning\.

To overcome this, we proposeNeSy\-Spatial, a neuro\-symbolic framework for self\-evolving skills in spatial reasoning\. Rather than simply expanding the available toolset,NeSy\-Spatialextracts atomic instructions from spatial reasoning trajectories and further composes them into reusable high\-level skills\. Specifically, we define two complementary types of skills:Tool\-Use SkillandGeometry Skill\.Tool\-Use Skillsorganize the execution of external tools, where atomic instructions correspond to individual tool calls and high\-level skills encode their execution order and dependency structure\.Geometry Skillsstructure geometric reasoning, where atomic instructions correspond to executable functional code blocks and high\-level skills abstract reusable operations such as coordinate transformation, distance computation, and consistency checking\.

NeSy\-Spatialcontinuously learns and abstracts skills during online interaction, and further leverages the evolving skill library to guide subsequent reasoning\. This process consists of two stages: inference and evolution\. During inference,NeSy\-Spatialretrieves, instantiates, and executes relevant skills in a closed\-loop process, dynamically adapting subsequent reasoning based on intermediate results\. During evolution, the model mines new atomic instructions and local rules from both successful and failed execution trajectories, integrates them with existing skills, and removes unreliable or inactive entries according to verification results and usage patterns\. Experiments on three datasets show thatNeSy\-Spatialconsistently improves spatial reasoning accuracy while achieving more precise tool utilization\.

Our contributions are summarized as follows:

- •We introduce a neuro\-symbolic definition of spatial skills that unifies tool invocation and geometric computation through typed executable atomic instructions\.
- •We proposeNeSy\-Spatial, a skill self\-evolution framework that updates atomic instructions and high\-level Tool\-Use and Geometry Skills from buffered trajectories through analysis, fusion, and pruning\.
- •We evaluateNeSy\-Spatialon multiple spatial reasoning benchmarks, showing improvements over base static tool\-use agents and fixed\-pipeline methods in performance\.

![Refer to caption](https://arxiv.org/html/2608.07955v1/Figures/frame_v2.png)Figure 2:Overview ofNeSy\-Spatial\.Left:the agent retrieves and executes skills for closed\-loop spatial reasoning\.Center top:the Skill Library organizes atomic instructions into reusable high\-level skills\.Center bottom:execution trajectories record the shared reasoning process\.Right:trajectory analysis, skill fusion, and pruning enable continual self\-evolution\.![Refer to caption](https://arxiv.org/html/2608.07955v1/Figures/example_v2.png)

Figure 3:Qualitative example of skill\-guided spatial reasoning and skill evolution\.Left:NeSy\-Spatialcomposes visual and geometric tools to solve a perspective\-taking question\.Right:failure diagnosis identifies missing grounding dependencies, updates the Tool\-Use Skill, and enables the revised pipeline to produce the correct answer on a subsequent sample\.
## 2Related Work

### 2\.1Spatial Reasoning

Spatial reasoning is the ability to interpret geometric relationships between objects and their environment\(Chenet al\.[2024a](https://arxiv.org/html/2608.07955#bib.bib110); Liuet al\.[2023](https://arxiv.org/html/2608.07955#bib.bib113); Yanget al\.[2025a](https://arxiv.org/html/2608.07955#bib.bib105)\)\. It is fundamental to robotic perception and embodied intelligence\(Liuet al\.[2025](https://arxiv.org/html/2608.07955#bib.bib116)\), as well as autonomous driving\(Chenet al\.[2024b](https://arxiv.org/html/2608.07955#bib.bib127)\)\. Recent benchmarks extend earlier static settings to multi\-view observations, viewpoint changes, scale estimation, and dynamic scenes, including BLINK\(Fuet al\.[2024](https://arxiv.org/html/2608.07955#bib.bib134)\), CVBench\(Tonget al\.[2024](https://arxiv.org/html/2608.07955#bib.bib133)\), VSI\-Bench\(Yanget al\.[2025a](https://arxiv.org/html/2608.07955#bib.bib105)\), and STI\-Bench\(Liet al\.[2025b](https://arxiv.org/html/2608.07955#bib.bib128)\)\. Our evaluation covers MMSI\(Yanget al\.[2025b](https://arxiv.org/html/2608.07955#bib.bib172)\), MindCube\(Chenet al\.[2026b](https://arxiv.org/html/2608.07955#bib.bib170)\), and OmniSpatial\(Jiaet al\.[2025](https://arxiv.org/html/2608.07955#bib.bib169)\)\. Despite advances in general perception, vision\-language models still struggle with multi\-step spatial inference, precise scale estimation\(Chenet al\.[2025a](https://arxiv.org/html/2608.07955#bib.bib130)\), and viewpoint\-dependent reasoning\(Liet al\.[2025a](https://arxiv.org/html/2608.07955#bib.bib131)\)\.

Existing solutions include task\-specific fine\-tuning\(Chenget al\.[2024](https://arxiv.org/html/2608.07955#bib.bib111)\), progressive training, 3D\-aware representations\(Xuet al\.[2024](https://arxiv.org/html/2608.07955#bib.bib135)\), and modular systems that invoke visual tools for detection, depth estimation, reconstruction, or geometric measurement\(Chenet al\.[2025b](https://arxiv.org/html/2608.07955#bib.bib107); Hanet al\.[2025](https://arxiv.org/html/2608.07955#bib.bib120)\)\. Recent tool\-augmented agents include SpatialScore’s SpaAge\-PE and SpaAge\-ReAct\(Wuet al\.[2026](https://arxiv.org/html/2608.07955#bib.bib166)\), SpatialCLAW\(Choet al\.[2026](https://arxiv.org/html/2608.07955#bib.bib167)\), GCA\(Chenet al\.[2026a](https://arxiv.org/html/2608.07955#bib.bib168)\), and LAST\(Tianet al\.[2026](https://arxiv.org/html/2608.07955#bib.bib171)\)\. Although explicit visual and geometric operations improve spatial reasoning, these methods primarily optimize individual reasoning pipelines rather than accumulating, reusing, and refining procedures across tasks\.

### 2\.2Skill Self\-evolution

Skill self\-evolution enables agents to abstract reusable knowledge from past interactions to improve future task solving\. Reflection\-based methods revise reasoning through feedback, self\-critique, or external validation\(Shinnet al\.[2023](https://arxiv.org/html/2608.07955#bib.bib159); Madaanet al\.[2023](https://arxiv.org/html/2608.07955#bib.bib160); Gouet al\.[2024](https://arxiv.org/html/2608.07955#bib.bib161)\), while experience\-driven methods retrieve knowledge abstracted from historical trajectories\(Zhaoet al\.[2024](https://arxiv.org/html/2608.07955#bib.bib162); Shaoet al\.[2026](https://arxiv.org/html/2608.07955#bib.bib173)\)\. Skill\-memory systems further maintain reusable skills or workflows to guide future actions\(Wanget al\.[2023](https://arxiv.org/html/2608.07955#bib.bib163),[2024](https://arxiv.org/html/2608.07955#bib.bib164),[2025b](https://arxiv.org/html/2608.07955#bib.bib165)\)\. Together, these studies show that agent experience can be accumulated and reused\. However, prior studies largely address text reasoning, web navigation, coding, or games, representing skills as instructions, workflows, or programs\. Tool\-augmented spatial reasoning instead requires coordination among visual perception, geometric computation, and multi\-step inference\. It therefore calls for reusable procedures that specify when to invoke tools, how to interpret their outputs, and how to transfer successful geometric reasoning across spatial tasks\.

## 3Method

### 3\.1Problem Setup

We study online tool\-augmented spatial reasoning\. At episodett, the agent receivesxt=\(It,qt\)∈ℐ×𝒬x\_\{t\}=\(I\_\{t\},q\_\{t\}\)\\in\\mathcal\{I\}\\times\\mathcal\{Q\}and predictsy^t∈𝒴\\hat\{y\}\_\{t\}\\in\\mathcal\{Y\}using a tool set𝒯=𝒯vis∪𝒯py\\mathcal\{T\}=\\mathcal\{T\}\_\{\\mathrm\{vis\}\}\\cup\\mathcal\{T\}\_\{\\mathrm\{py\}\}and a skill libraryLibt=\(𝒜t,𝒮t\)\\mathrm\{Lib\}\_\{t\}=\(\\mathcal\{A\}\_\{t\},\\mathcal\{S\}\_\{t\}\)\. The Python executor is exposed as a single tool\-level atomic instruction, while its internal geometric computation is constructed from reusable Geometry Skills\. At reasoning stepkk, the working memoryMt,kM\_\{t,k\}maintains the accumulated evidence, symbolic variables, execution history, verifier outcomes, and answer hypothesis\.

During an episode,Libt\\mathrm\{Lib\}\_\{t\}is fixed\. Under our test\-then\-update protocol, each prediction is recorded exactly once before its ground\-truth answer is revealed; the label is then used only to evaluate that prediction and assess the completed trajectoryτt\\tau\_\{t\}, which is appended to the trajectory bufferℬt\\mathcal\{B\}\_\{t\}\. Once the buffer reachesNbufN\_\{\\mathrm\{buf\}\}, self\-evolution extracts and integrates reusable structures and prunes unreliable or inactive entries to obtainLibt\+1\\mathrm\{Lib\}\_\{t\+1\}; otherwise, the library remains unchanged\. Thus, all library updates occur between episodes, as illustrated in Figure[2](https://arxiv.org/html/2608.07955#S1.F2)\.

### 3\.2Neuro\-Symbolic Skill Formulation

Spatial reasoning must couple learned perception with verifiable geometric computation\. At a fixed episode, we omit the time index and write the skill library as

Lib=\(𝒜,𝒮\),𝒮=𝒮tool∪𝒮geo,\\mathrm\{Lib\}=\\bigl\(\\mathcal\{A\},\\mathcal\{S\}\\bigr\),\\qquad\\mathcal\{S\}=\\mathcal\{S\}\_\{\\mathrm\{tool\}\}\\cup\\mathcal\{S\}\_\{\\mathrm\{geo\}\},\(1\)where𝒜\\mathcal\{A\}is a set of executable atomic instructions, while𝒮tool\\mathcal\{S\}\_\{\\mathrm\{tool\}\}and𝒮geo\\mathcal\{S\}\_\{\\mathrm\{geo\}\}contain Tool\-Use and Geometry Skills, respectively\. Both skill types are represented by atomic\-action nodes, ordering edges, and an instantiation context\.

#### Atomic Instructions

An atomic instruction is a typed executable operation with a local verifier:

a=\(ea,va\),ea:𝒰a→𝒪a,va:𝒪a→\{0,1\}\.a=\\bigl\(e\_\{a\},v\_\{a\}\\bigr\),\\qquad e\_\{a\}:\\mathcal\{U\}\_\{a\}\\rightarrow\\mathcal\{O\}\_\{a\},\\qquad v\_\{a\}:\\mathcal\{O\}\_\{a\}\\rightarrow\\\{0,1\\\}\.\(2\)Hereeae\_\{a\}maps atom\-specific typed inputs𝒰a\\mathcal\{U\}\_\{a\}to outputs𝒪a\\mathcal\{O\}\_\{a\}, andvav\_\{a\}validates the output\. We partition𝒜\\mathcal\{A\}into controller\-visible tool atoms𝒜tool\\mathcal\{A\}\_\{\\mathrm\{tool\}\}and internal geometry atoms𝒜geo\\mathcal\{A\}\_\{\\mathrm\{geo\}\}\. The former wraps visual tools and the Python executor, each as one atomic instruction; the latter contains reusable code blocks for geometric transformation, numerical comparison, and relation verification\. Geometry atoms are executed inside the Python tool rather than selected as independent controller actions\.

#### High\-Level Skills

We represent each high\-level skill as a node–edge–context structure:

s=\(V,E,κ\),V⊆𝒜,E⊆V×V,s=\\bigl\(V,E,\\kappa\\bigr\),\\qquad V\\subseteq\\mathcal\{A\},\\quad E\\subseteq V\\times V,\(3\)whereVVcontains atomic instructions,EEspecifies their execution dependencies, andκ\\kappastores the context required for instantiation\. Tool\-Use and Geometry Skills share this representation but differ in their node types and execution roles\.

Tool\-Use Skill: Camera\-Pose RelationUse when:A target camera must be located relative to a reference camera from two images\.VGGT\.reconstruct Input: images, question→\\rightarrowOutput: reconstruction⇓\\DownarrowPythonTool\.code Input: reconstruction, question→\\rightarrowOutput: answerStop when:Both camera poses are available for computing their relation in the reference camera frame\.

Geometry Skill: Camera\-Frame Direction MatchingUse when:Camera extrinsics, grounded 3D points, and directional options are available\.RobustCenter\(points\_w\)→\\rightarrowcenter\_wWorldToCamera\(center\_w, E\_ref\)→\\rightarrowcenter\_cNormalize\(center\_c\[x,z\]\)→\\rightarrowdir\_xzMatchOption\(dir\_xz, options\)→\\rightarrowchoiceOutput:The matched direction

Figure 4:Examples of Tool\-Use and Geometry Skills\.ATool\-Use Skills∈𝒮tools\\in\\mathcal\{S\}\_\{\\mathrm\{tool\}\}organizes controller\-visible tool atoms into a reusable workflow\. Its nodes satisfyV⊆𝒜toolV\\subseteq\\mathcal\{A\}\_\{\\mathrm\{tool\}\}, its edges define execution dependencies, and its context stores object bindings, tool arguments, and available evidence\. Python is treated as an atomic tool node to keep the controller action space compact, while its internal computation is handled separately byGeometry Skills\.

AGeometry Skills∈𝒮geos\\in\\mathcal\{S\}\_\{\\mathrm\{geo\}\}provides a reusable and verifiable implementation for a Python tool call\. Its nodes satisfyV⊆𝒜geoV\\subseteq\\mathcal\{A\}\_\{\\mathrm\{geo\}\}, its edges encode data dependencies, and its context records reference frames, units, predicates, and numerical evidence\. When invoking a Python tool, the model retrieves relevant Geometry Skills to guide code construction before submitting the generated code to the interpreter\. Figure[4](https://arxiv.org/html/2608.07955#S3.F4)illustrates representative skills and their organizational structure\.

### 3\.3Skill\-Guided Spatial Reasoning

During inference episodett,NeSy\-Spatialperforms state\-adaptive spatial reasoning over\(It,qt\)\(I\_\{t\},q\_\{t\}\)\. As shown in the left panel of Figure[2](https://arxiv.org/html/2608.07955#S1.F2), each iteration follows four stages: Skill Retrieval, Action Selection, Tool/Code Execution, and State Update\. Figure[3](https://arxiv.org/html/2608.07955#S1.F3)illustrates how these skills organize tool execution and are subsequently refined through failure diagnosis\. WithLibt\\mathrm\{Lib\}\_\{t\}fixed, the loop repeats until the accumulated state is sufficient to produce an answer\.

#### Skill Retrieval

At each iteration, a retrieval interface converts the question and current state into a structured decision context and queries the skill library for a small candidate set:

𝒞t,k=LLMRetrieve​\(It,qt,Mt,k;𝒮t\)\.\\mathcal\{C\}\_\{t,k\}=\\mathrm\{LLM\}\_\{\\mathrm\{Retrieve\}\}\(I\_\{t\},q\_\{t\},M\_\{t,k\};\\mathcal\{S\}\_\{t\}\)\.\(4\)Here𝒞t,k⊆𝒮t\\mathcal\{C\}\_\{t,k\}\\subseteq\\mathcal\{S\}\_\{t\}may contain Tool\-Use or Geometry Skills\. The interface is implemented with task\-specific instruction templates that expose the target and reference objects, relation, available evidence, and required geometric operation\.

#### Action Selection

Given the retrieved candidates and current state, the controller selects the next action:

ut,k=LLMSelect​\(It,qt,Mt,k,𝒞t,k\),ut,k∈\{stop\}∪\(𝒞t,k∩𝒮tool,t\)∪𝒜tool,t\.\\begin\{gathered\}u\_\{t,k\}=\\mathrm\{LLM\}\_\{\\mathrm\{Select\}\}\(I\_\{t\},q\_\{t\},M\_\{t,k\},\\mathcal\{C\}\_\{t,k\}\),\\\\\[\-2\.0pt\] u\_\{t,k\}\\in\\\{\\mathrm\{stop\}\\\}\\cup\\bigl\(\\mathcal\{C\}\_\{t,k\}\\cap\\mathcal\{S\}\_\{\\mathrm\{tool\},t\}\\bigr\)\\cup\\mathcal\{A\}\_\{\\mathrm\{tool\},t\}\.\\end\{gathered\}\(5\)The controller may terminate with the current answer, activate or resume a retrieved Tool\-Use Skill, or directly call a compatible tool atom\. Geometry Skills are not independent controller actions; they are instantiated within the Python atom during execution\.

Table 1:Average accuracy \(%\) over the online stream, broken down by dataset\-specific subtasks and capability groups\. The best and second\-best results within each backbone block are highlighted in bold and underlined, respectively\.#### Tool/Code Execution

The selected action executes either a single tool atom or the next dependency\-ready node of an active Tool\-Use Skill\. The resulting observation is returned immediately for state update, so the skill can be interrupted after any node\. Visual tool atoms return perceptual evidence directly\. When execution reaches the Python atom, the agent binds the required artifacts fromMt,kM\_\{t,k\}and instantiates a compatible retrieved Geometry Skill\. Its code blocks execute in dependency order and return both numerical outputs and verifier outcomes to the Python node\. This preserves a compact tool\-level action space while making geometric computation structured and verifiable\.

#### State Update

After execution, the controller incorporates the new observation and verifier outcomes into its working memory\. For a non\-terminal action, the complete iteration is

\(It,qt,Mt,k\)\\displaystyle\\bigl\(I\_\{t\},q\_\{t\},M\_\{t,k\}\\bigr\)→Retrieve𝒞t,k→Selectut,k,\\displaystyle\\xrightarrow\{\\mathrm\{Retrieve\}\}\\mathcal\{C\}\_\{t,k\}\\xrightarrow\{\\mathrm\{Select\}\}u\_\{t,k\},\(6\)ut,k\\displaystyle u\_\{t,k\}→Exec\(ot,k,νt,k\)→UpdateMt,k\+1\.\\displaystyle\\xrightarrow\{\\mathrm\{Exec\}\}\(o\_\{t,k\},\\nu\_\{t,k\}\)\\xrightarrow\{\\mathrm\{Update\}\}M\_\{t,k\+1\}\.Hereot,ko\_\{t,k\}is the execution output, whileνt,k∈\{0,1\}\\nu\_\{t,k\}\\in\\\{0,1\\\}indicates whether execution and local verification succeed\. Unless the controller selectsstop\\mathrm\{stop\}, the updated state initiates the next retrieval round; otherwise, the current answer hypothesis is returned as the final answer\. After final\-answer evaluation, the completed trace is stored for later batch\-wise self\-evolution\.

### 3\.4Skill Self\-Evolution

NeSy\-Spatialevolves a single skill library from batches of completed trajectories\. The trajectory produced by episodettis

τt\\displaystyle\\tau\_\{t\}=\(\(\(Mt,k,ut,k,ot,k,νt,k\)\)k=1Kt,y^t\),\\displaystyle=\\left\(\\bigl\(\(M\_\{t,k\},u\_\{t,k\},o\_\{t,k\},\\nu\_\{t,k\}\)\\bigr\)\_\{k=1\}^\{K\_\{t\}\},\\hat\{y\}\_\{t\}\\right\),\(7\)ℬ¯t\+1\\displaystyle\\overline\{\\mathcal\{B\}\}\_\{t\+1\}=ℬt∪\{τt\}\.\\displaystyle=\\mathcal\{B\}\_\{t\}\\cup\\\{\\tau\_\{t\}\\\}\.whereKtK\_\{t\}is the number of executed node\-level steps,y^t\\hat\{y\}\_\{t\}is the final answer, andℬ¯t\+1\\overline\{\\mathcal\{B\}\}\_\{t\+1\}is the buffer before evolution\. Self\-evolution is triggered only when\|ℬ¯t\+1\|≥Nbuf\|\\overline\{\\mathcal\{B\}\}\_\{t\+1\}\|\\geq N\_\{\\mathrm\{buf\}\}; otherwise,Libt\+1=Libt\\mathrm\{Lib\}\_\{t\+1\}=\\mathrm\{Lib\}\_\{t\}andℬt\+1=ℬ¯t\+1\\mathcal\{B\}\_\{t\+1\}=\\overline\{\\mathcal\{B\}\}\_\{t\+1\}\. Once triggered,NeSy\-Spatialanalyzes the buffered trajectories, fuses the extracted structures with the current library, and prunes unreliable or inactive entries\. The processed buffer is then cleared, while its aggregated invocation statistics are retained\.

![Refer to caption](https://arxiv.org/html/2608.07955v1/Figures/skill_evolution_over_samples.png)

Figure 5:Left:FTCS and Python success rate across spatial reasoning benchmarks; the best and second\-best results within each block are highlighted in bold and underlined, respectively\.Right:the evolution of the number of retained skills throughout the online stream\.#### Trajectory Analysis

A single trajectory may contain incidental failures or task\-specific choices, soNeSy\-Spatialextracts reusable structures only after accumulating sufficient evidence in the buffer\. Each buffered trajectory is decomposed into executed operations, arguments, produced artifacts, consumed variables, and local outcomes\. The analyzer then aggregates their dependency patterns and returns atomic\-instruction updates together with supported and invalid skill fragments:

\(Δ​𝒜t,𝒢^t\+,𝒢^t−\)=LLMAnalyze​\(ℬ¯t\+1\)\.\\bigl\(\\Delta\\mathcal\{A\}\_\{t\},\\widehat\{\\mathcal\{G\}\}^\{\+\}\_\{t\},\\widehat\{\\mathcal\{G\}\}^\{\-\}\_\{t\}\\bigr\)=\\mathrm\{LLM\}\_\{\\mathrm\{Analyze\}\}\(\\overline\{\\mathcal\{B\}\}\_\{t\+1\}\)\.\(8\)HereΔ​𝒜t\\Delta\\mathcal\{A\}\_\{t\}contains newly required or revised atomic instructions, while𝒢^t\+\\widehat\{\\mathcal\{G\}\}^\{\+\}\_\{t\}and𝒢^t−\\widehat\{\\mathcal\{G\}\}^\{\-\}\_\{t\}contain supported and invalid skill fragments, respectively\. For example, a supported fragment may require invoking atomaia\_\{i\}beforeaja\_\{j\}, whereas an invalid fragment records that this transition repeatedly causes execution or verifier failures\. Aggregating multiple trajectories reduces the influence of isolated errors and exposes dependencies that recur across tasks\.

#### Skill Fusion

Extracted atoms and fragments may duplicate, extend, or conflict with existing skills\. The fusion module therefore compares them with the unified library and produces a structurally revised library:

Libt\+1fuse=LLMFusion​\(Libt,Δ​𝒜t,𝒢^t\+,𝒢^t−\)\.\\mathrm\{Lib\}^\{\\mathrm\{fuse\}\}\_\{t\+1\}=\\mathrm\{LLM\}\_\{\\mathrm\{Fusion\}\}\\bigl\(\\mathrm\{Lib\}\_\{t\},\\Delta\\mathcal\{A\}\_\{t\},\\widehat\{\\mathcal\{G\}\}^\{\+\}\_\{t\},\\widehat\{\\mathcal\{G\}\}^\{\-\}\_\{t\}\\bigr\)\.\(9\)We writeLibt\+1fuse=\(𝒜t\+1fuse,𝒮t\+1fuse\)\\mathrm\{Lib\}^\{\\mathrm\{fuse\}\}\_\{t\+1\}=\(\\mathcal\{A\}^\{\\mathrm\{fuse\}\}\_\{t\+1\},\\mathcal\{S\}^\{\\mathrm\{fuse\}\}\_\{t\+1\}\)\. Fusion first reconcilesΔ​𝒜t\\Delta\\mathcal\{A\}\_\{t\}with𝒜t\\mathcal\{A\}\_\{t\}by adding a novel typed instruction or replacing an existing implementation when the new version better satisfies its local verifier\. It then edits high\-level skills through three operations: adding a supported fragment, deleting a redundant or incompatible dependency, or replacing an affected subgraph while preserving the remainder of the skill\. Tool\-organizing fragments update Tool\-Use Skills, whereas fragments that transform or verify numerical evidence update Geometry Skills\. All edits are applied directly to the unified library\.

#### Skill Pruning

Fusion revises the library structure, while pruning removes unreliable or inactive entries according to their invocation history\. For eachz∈𝒜t\+1fuse∪𝒮t\+1fusez\\in\\mathcal\{A\}^\{\\mathrm\{fuse\}\}\_\{t\+1\}\\cup\\mathcal\{S\}^\{\\mathrm\{fuse\}\}\_\{t\+1\}, letnt​\(z\)n\_\{t\}\(z\)be its number of invocations andERt​\(z\)\\mathrm\{ER\}\_\{t\}\(z\)the fraction of invocations that end in an execution or local\-verifier failure\. We further define its inactivity age asAget​\(z\)=t−tlast​\(z\)\\mathrm\{Age\}\_\{t\}\(z\)=t\-t\_\{\\mathrm\{last\}\}\(z\)and its tenure asTenuret​\(z\)=t−tadd​\(z\)\\mathrm\{Tenure\}\_\{t\}\(z\)=t\-t\_\{\\mathrm\{add\}\}\(z\), wheretlast​\(z\)t\_\{\\mathrm\{last\}\}\(z\)andtadd​\(z\)t\_\{\\mathrm\{add\}\}\(z\)denote the most recent invocation and insertion episodes, respectively\. Before the first invocation, we settlast​\(z\)=tadd​\(z\)t\_\{\\mathrm\{last\}\}\(z\)=t\_\{\\mathrm\{add\}\}\(z\)\.

Given an error\-rate thresholdρerr\\rho\_\{\\mathrm\{err\}\}, an inactivity limitTidleT\_\{\\mathrm\{idle\}\}, and a minimum support thresholdnminn\_\{\\min\}, we prunezzwhen

Prunet​\(z\)=\\displaystyle\\mathrm\{Prune\}\_\{t\}\(z\)=\{\}\[nt​\(z\)≥nmin∧ERt​\(z\)\>wt​\(z\)​ρerr\]\\displaystyle\\bigl\[n\_\{t\}\(z\)\\geq n\_\{\\min\}\\wedge\\mathrm\{ER\}\_\{t\}\(z\)\>w\_\{t\}\(z\)\\rho\_\{\\mathrm\{err\}\}\\bigr\]\(10\)∨\[Aget​\(z\)\>wt​\(z\)​Tidle\]\.\\displaystyle\{\}\\vee\\bigl\[\\mathrm\{Age\}\_\{t\}\(z\)\>w\_\{t\}\(z\)T\_\{\\mathrm\{idle\}\}\\bigr\]\.Both thresholds are adapted by the tenure\-dependent retention factor

wt​\(z\)=1\+λ​Tenuret​\(z\)1\+Tenuret​\(z\),w\_\{t\}\(z\)=1\+\\lambda\\frac\{\\mathrm\{Tenure\}\_\{t\}\(z\)\}\{1\+\\mathrm\{Tenure\}\_\{t\}\(z\)\},whereλ≥0\\lambda\\geq 0is chosen such that\(1\+λ\)​ρerr≤1\(1\+\\lambda\)\\rho\_\{\\mathrm\{err\}\}\\leq 1\. Thus,wt​\(z\)w\_\{t\}\(z\)grows from11toward1\+λ1\+\\lambda, providing longer\-lived entries with bounded protection, whilenminn\_\{\\min\}prevents pruning after only a few noisy failures\. Entries without any invocation history are evaluated only by inactivity\. After removing all marked atoms and skills, we repair or discard skills that depend on a pruned atom, yielding the updated libraryLibt\+1\\mathrm\{Lib\}\_\{t\+1\}\.

## 4Experiments

### 4\.1Experimental Setup

Datasets and Models\.We evaluateNeSy\-Spatialon MMSI\(Yanget al\.[2025b](https://arxiv.org/html/2608.07955#bib.bib172)\), MindCube\(Chenet al\.[2026b](https://arxiv.org/html/2608.07955#bib.bib170)\), and OmniSpatial\(Jiaet al\.[2025](https://arxiv.org/html/2608.07955#bib.bib169)\)using GPT\-5\.4 and Gemini 2\.5 Pro as the two API backbones\. For MMSI, we use all four subsets: Attribute, Motion, Positional Relation, and Multi\-Step Reasoning \(MSR\)\. For MindCube, we use all three subsets: Rotation, Around, and Among\. Following GCA\(Chenet al\.[2026a](https://arxiv.org/html/2608.07955#bib.bib168)\), we evaluate OmniSpatial on two subsets, Dynamic Reasoning and Perspective Taking\.

Comparison Methods\.We compare against the general inductive agents AWM\(Wanget al\.[2024](https://arxiv.org/html/2608.07955#bib.bib164)\)and ASI\(Wanget al\.[2025b](https://arxiv.org/html/2608.07955#bib.bib165)\), as well as domain\-specific spatial reasoning agents with different tool\-use mechanisms\. SpaAge\-PE and SpaAge\-ReAct from SpatialScore\(Wuet al\.[2026](https://arxiv.org/html/2608.07955#bib.bib166)\)let the LVLM plan tool calls directly over a provided toolbox, whereas GCA\(Chenet al\.[2026a](https://arxiv.org/html/2608.07955#bib.bib168)\)first maps each problem into a unified geometric formalization before invoking tools\. All methods are evaluated with the same tool set and interface configuration; LAST\(Tianet al\.[2026](https://arxiv.org/html/2608.07955#bib.bib171)\)differs in that its spatial skills are predefined\. For all methods evaluated in the online setting, we report the average accuracy over all samples in the online stream\.

Evaluation Metrics\.We report prequential answer accuracy, where each prediction is evaluated before its label is used for skill evolution\. Tool execution is measured by First\-Pass Compositional Tool\-Chain Success Rate \(FTCS\) and Python success rate\. For samples𝒮pipe\\mathcal\{S\}\_\{\\mathrm\{pipe\}\}invoking at least two distinct task tools,

FTCS=1\|𝒮pipe\|​∑s∈𝒮pipecs,\\mathrm\{FTCS\}=\\frac\{1\}\{\|\\mathcal\{S\}\_\{\\mathrm\{pipe\}\}\|\}\\sum\_\{s\\in\\mathcal\{S\}\_\{\\mathrm\{pipe\}\}\}c\_\{s\},wherecs=1c\_\{s\}=1if all task\-tool calls succeed without a failed attempt, and0otherwise\. Python success rate is the fraction of successfulPythonTool\.codecalls, counting retries separately\.

Implementation Details\.We initialize the skill library withNseed=3N\_\{\\mathrm\{seed\}\}=3seed skills for MMSI and MindCube, andNseed=4N\_\{\\mathrm\{seed\}\}=4for OmniSpatial\. Skill evolution is triggered after everyNbuf=20N\_\{\\mathrm\{buf\}\}=20completed trajectories\. For skill pruning, we set the minimum invocation support tonmin=5n\_\{\\min\}=5, the base error\-rate threshold toρerr=0\.4\\rho\_\{\\mathrm\{err\}\}=0\.4, the inactivity limit toTidle=100T\_\{\\mathrm\{idle\}\}=100episodes, and the tenure\-dependent retention coefficient toλ=0\.5\\lambda=0\.5, satisfying\(1\+λ\)​ρerr≤1\(1\+\\lambda\)\\rho\_\{\\mathrm\{err\}\}\\leq 1\. The same hyperparameters are used across all datasets and LVLM backbones\. We provide all prompts used in the paper, the complete set of evolved rules, and the remaining hyperparameter settings in the appendix\.

### 4\.2Main Results

NeSy\-Spatialimproves spatial reasoning performance and achieves state\-of\-the\-art results\.As shown in Table[3\.3](https://arxiv.org/html/2608.07955#S3.SS3.SSSx2),NeSy\-Spatialachieves the best Overall accuracy in five of the six backbone–dataset settings\. With GPT\-5\.4,NeSy\-Spatialreaches 49\.50 on MMSI and 71\.50 on MindCube, outperforming the strongest baselines by 1\.50 and 0\.49 percentage points, respectively, while remaining competitive on OmniSpatial \(60\.27 vs\. 61\.00\)\. With Gemini 2\.5 Pro, it achieves the highest Overall accuracy on all three benchmarks, outperforming the strongest baselines by 0\.50, 10\.12, and 3\.97 percentage points on MMSI, MindCube, and OmniSpatial, respectively\. These consistent gains across different backbones and benchmarks demonstrate the effectiveness ofNeSy\-Spatialfor spatial reasoning\.

NeSy\-Spatialimproves tool utilization efficiency and execution accuracy\.The left panel of Figure[3\.4](https://arxiv.org/html/2608.07955#S3.SS4)reports SpaAge\-PE and SpaAge\-ReAct jointly as SpaAge\. With GPT\-5\.4,NeSy\-Spatialachieves the highest average FTCS of 94\.56 and Python success rate of 95\.45, outperforming the strongest baseline averages by 7\.17 and 6\.67 percentage points, respectively\. Across datasets, it leads both metrics on MMSI and OmniSpatial and remains close to the best on MindCube, demonstrating consistent gains beyond the averages\. With Gemini 2\.5 Pro,NeSy\-Spatialagain achieves the best average FTCS and Python success rate, exceeding the strongest baselines by 0\.21 and 7\.76 points, respectively\. It attains the highest Python success rate on all three benchmarks and the highest FTCS on MindCube and OmniSpatial, while trailing LAST by 3\.47 points in FTCS on MMSI\. Together, these results show thatNeSy\-Spatialmore reliably completes compositional tool chains on the first pass while improving the accuracy of individual Python executions\.

### 4\.3Further Analysis

![Refer to caption](https://arxiv.org/html/2608.07955v1/x1.png)Figure 6:Cumulative prequential accuracy ofNeSy\-Spatialand Frozen throughout the online stream\. Frozen keeps the initial skill library fixed without online evolution\.Online skill evolution yields consistent overall gains in spatial reasoning performance\.Figure[6](https://arxiv.org/html/2608.07955#S4.F6)compares the cumulative prequential accuracy ofNeSy\-Spatialwith Frozen, whose initial skill library \(i\.e\., the knowledge base\) remains fixed throughout the stream, corresponding to the setting without evolution\. Because cumulative estimates based on only a few observations are highly volatile, the curves are displayed from 5% stream progress, after 10 samples have been processed; each point nevertheless includes all observations from the start of the stream\. The curves become more stable as evaluation proceeds, andNeSy\-Spatialestablishes sustained gains over Frozen across all three benchmarks\. By the end of the stream,NeSy\-Spatialfinishes 12\.5, 3\.5, and 2\.5 percentage points higher on MindCube, MMSI, and OmniSpatial, respectively\. The macro\-average accuracy similarly improves from 54\.2% to 60\.3%, showing that online skill evolution provides consistent benefits across the three benchmarks\. Together, the endpoint margins and full\-stream trajectories show that these gains persist as evaluation continues rather than arising from a brief early advantage\.

Skill evolution selectively expands the library rather than accumulating skills indiscriminately\.The right panel of Figure[3\.4](https://arxiv.org/html/2608.07955#S3.SS4)tracks how the number of retained skills changes throughout online evolution\. Overall, the library expands from 3 to 17 skills on MindCube, from 3 to 9 on MMSI, and from 4 to 20 on OmniSpatial\. However, this growth is not monotonic: the library size decreases at multiple stages as unreliable or inactive skills are removed\. These downward steps indicate that the pruning strategy actively controls redundancy and prevents unbounded accumulation\. The resulting trajectories reflect a dynamic balance between incorporating reusable skills from new experience and retaining only those that remain useful\.

Table 2:Ablation study of skill evolution, Tool\-Use Skills, and Geometry Skills using GPT\-5\.4\.Each component plays a distinct and complementary role in the full framework\.As shown in Table[2](https://arxiv.org/html/2608.07955#S4.T2), disabling skill evolution while retaining both skill types reduces accuracy by 12\.50, 3\.50, and 2\.78 percentage points on MindCube, MMSI, and OmniSpatial, respectively\. Removing Tool\-Use Skills causes corresponding drops of 11\.50, 6\.00, and 2\.68 points, whereas removing Geometry Skills lowers accuracy by 2\.00, 2\.50, and 2\.27 points\. The full configuration consistently performs best across all three benchmarks, demonstrating that online evolution provides substantial gains while Tool\-Use and Geometry Skills contribute complementary execution and reasoning capabilities\.

## 5Conclusion

This paper presentedNeSy\-Spatial, a self\-evolving neuro\-symbolic framework for tool\-augmented spatial reasoning\. It organizes typed atomic instructions for tool interaction and geometric computation into interruptible Tool\-Use Skills and reusable Geometry Skills\. Across three benchmarks and two LVLM backbones,NeSy\-Spatialachieves the best aggregate accuracy in five of six settings and the highest average FTCS and Python success rate\. Online results show sustained accuracy gains with experience, while library\-size changes indicate that pruning removes unreliable or inactive skills instead of allowing indiscriminate growth\. Evaluation is limited to selected benchmarks and tool environments; future work will examine broader tasks and transfer to unseen settings\.

## References

- Spatialvlm: endowing vision\-language models with spatial reasoning capabilities\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 14455–14465\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1)\.
- L\. Chen, P\. Wu, K\. Chitta, B\. Jaeger, A\. Geiger, and H\. Li \(2024b\)End\-to\-end autonomous driving: challenges and frontiers\.IEEE Transactions on Pattern Analysis and Machine Intelligence\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1)\.
- P\. Chen, Y\. Lou, S\. Cao, J\. Guo, L\. Fan, Y\. Wu, L\. Yang, L\. Ma, and J\. Ye \(2025a\)SD\-vlm: spatial measuring and understanding with depth\-encoded vision\-language models\.arXiv preprint arXiv:2509\.17664\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1)\.
- S\. Chen, M\. A\. Uy, C\. H\. Song, F\. Ladhak, A\. Murali, Q\. Qu, S\. Birchfield, V\. Blukis, and J\. Tremblay \(2025b\)SpaceTools: tool\-augmented spatial reasoning via double interactive rl\.arXiv preprint arXiv:2512\.04069\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p2.1)\.
- Z\. Chen, X\. Lu, Z\. Zheng, P\. Li, L\. He, Y\. Zhou, J\. Shao, B\. Zhuang, and L\. Sheng \(2026a\)Geometrically\-constrained agent for spatial reasoning\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 38689–38699\.Cited by:[§1](https://arxiv.org/html/2608.07955#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2608.07955#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07955#S4.SS1.p2.1)\.
- Z\. Chen, M\. Zhang, X\. Yu, X\. Luo, M\. Sun, Z\. Pan, X\. An, Y\. Feng, P\. Pei, X\. Cai,et al\.\(2026b\)Think with 3d: geometric imagination grounded spatial reasoning from limited views\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 2613–2624\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07955#S4.SS1.p1.1)\.
- Z\. Chen, M\. Zhang, X\. Yu, X\. Luo, M\. Sun, Z\. Pan, Y\. Feng, P\. Pei, X\. Cai, and R\. Huang \(2025c\)Think with 3d: geometric imagination grounded spatial reasoning from limited views\.arXiv preprint arXiv:2510\.18632\.Cited by:[§1](https://arxiv.org/html/2608.07955#S1.p1.1)\.
- A\. Cheng, H\. Yin, Y\. Fu, Q\. Guo, R\. Yang, J\. Kautz, X\. Wang, and S\. Liu \(2024\)Spatialrgpt: grounded spatial reasoning in vision\-language models\.Advances in Neural Information Processing Systems37,pp\. 135062–135093\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p2.1)\.
- S\. Cho, R\. Hachiuma, A\. Badki, H\. Su, B\. Lee, C\. H\. Song, S\. Liu, S\. Radhakrishnan, S\. Kim, Y\. F\. Wang,et al\.\(2026\)SpatialClaw: rethinking action interface for agentic spatial reasoning\.arXiv preprint arXiv:2606\.13673\.Cited by:[§1](https://arxiv.org/html/2608.07955#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p2.1)\.
- X\. Fu, Y\. Hu, B\. Li, Y\. Feng, H\. Wang, X\. Lin, D\. Roth, N\. A\. Smith, W\. Ma, and R\. Krishna \(2024\)Blink: multimodal large language models can see but not perceive\.InEuropean Conference on Computer Vision,pp\. 148–166\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1)\.
- Z\. Gou, Z\. Shao, Y\. Gong, y\. shen, Y\. Yang, N\. Duan, and W\. Chen \(2024\)CRITIC: large language models can self\-correct with tool\-interactive critiquing\.InInternational Conference on Learning Representations,B\. Kim, Y\. Yue, S\. Chaudhuri, K\. Fragkiadaki, M\. Khan, and Y\. Sun \(Eds\.\),Vol\.2024,pp\. 57734–57811\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/file/fef126561bbf9d4467dbb8d27334b8fe-Paper-Conference.pdf)Cited by:[§2\.2](https://arxiv.org/html/2608.07955#S2.SS2.p1.1)\.
- Y\. Han, C\. Chi, E\. Zhou, S\. Rong, J\. An, P\. Wang, Z\. Wang, L\. Sheng, and S\. Zhang \(2025\)TIGeR: tool\-integrated geometric reasoning in vision\-language models for robotics\.arXiv preprint arXiv:2510\.07181\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p2.1)\.
- M\. Jia, Z\. Qi, S\. Zhang, W\. Zhang, X\. Yu, J\. He, H\. Wang, and L\. Yi \(2025\)Omnispatial: towards comprehensive spatial reasoning benchmark for vision language models\.arXiv preprint arXiv:2506\.03135\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07955#S4.SS1.p1.1)\.
- D\. Li, H\. Li, Z\. Wang, Y\. Yan, H\. Zhang, S\. Chen, G\. Hou, S\. Jiang, W\. Zhang, Y\. Shen,et al\.\(2025a\)ViewSpatial\-bench: evaluating multi\-perspective spatial localization in vision\-language models\.arXiv preprint arXiv:2505\.21500\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1)\.
- Y\. Li, Y\. Zhang, T\. Lin, X\. Liu, W\. Cai, Z\. Liu, and B\. Zhao \(2025b\)Sti\-bench: are mllms ready for precise spatial\-temporal world understanding?\.arXiv preprint arXiv:2503\.23765\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1)\.
- F\. Liu, G\. Emerson, and N\. Collier \(2023\)Visual spatial reasoning\.Transactions of the Association for Computational Linguistics11,pp\. 635–651\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1)\.
- Y\. Liu, W\. Chen, Y\. Bai, X\. Liang, G\. Li, W\. Gao, and L\. Lin \(2025\)Aligning cyber space with physical world: a comprehensive survey on embodied ai\.IEEE/ASME Transactions on Mechatronics\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. Clark \(2023\)Self\-refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 46534–46594\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/91edff07232fb1b55a505a9e9f6c0ff3-Paper-Conference.pdf)Cited by:[§2\.2](https://arxiv.org/html/2608.07955#S2.SS2.p1.1)\.
- OpenAI \(2023\)GPT\-4\.Technical reportCited by:[§1](https://arxiv.org/html/2608.07955#S1.p1.1)\.
- J\. Shao, H\. Yin, Y\. Lyu, X\. Yu, L\. Guo, I\. Tsang, J\. Kwok, and Y\. Li \(2026\)Lifting traces to logic: programmatic skill induction with neuro\-symbolic learning for long\-horizon agentic tasks\.InForty\-third International Conference on Machine Learning,Cited by:[§2\.2](https://arxiv.org/html/2608.07955#S2.SS2.p1.1)\.
- N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 8634–8652\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/1b44b878bb782e6954cd888628510e90-Paper-Conference.pdf)Cited by:[§2\.2](https://arxiv.org/html/2608.07955#S2.SS2.p1.1)\.
- S\. Tian, Z\. Zhou, K\. Yu, M\. Yang, Y\. Chen, Z\. Shang, L\. Guo, and Y\. Li \(2026\)Last: leveraging tools as hints to enhance spatial reasoning for multimodal large language models\.arXiv preprint arXiv:2604\.09712\.Cited by:[§1](https://arxiv.org/html/2608.07955#S1.p1.1),[§1](https://arxiv.org/html/2608.07955#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2608.07955#S4.SS1.p2.1)\.
- P\. Tong, E\. Brown, P\. Wu, S\. Woo, A\. J\. V\. IYER, S\. C\. Akula, S\. Yang, J\. Yang, M\. Middepogu, Z\. Wang,et al\.\(2024\)Cambrian\-1: a fully open, vision\-centric exploration of multimodal llms\.Advances in Neural Information Processing Systems37,pp\. 87310–87356\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1)\.
- G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. Anandkumar \(2023\)Voyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§2\.2](https://arxiv.org/html/2608.07955#S2.SS2.p1.1)\.
- J\. Wang, M\. Chen, N\. Karaev, A\. Vedaldi, C\. Rupprecht, and D\. Novotny \(2025a\)VGGT: visual geometry grounded transformer\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 5294–5306\.Cited by:[§1](https://arxiv.org/html/2608.07955#S1.p1.1)\.
- Z\. Z\. Wang, A\. Gandhi, G\. Neubig, and D\. Fried \(2025b\)Inducing programmatic skills for agentic tasks\.arXiv preprint arXiv:2504\.06821\.Cited by:[§2\.2](https://arxiv.org/html/2608.07955#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2608.07955#S4.SS1.p2.1)\.
- Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. Neubig \(2024\)Agent workflow memory\.arXiv preprint arXiv:2409\.07429\.Cited by:[§2\.2](https://arxiv.org/html/2608.07955#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2608.07955#S4.SS1.p2.1)\.
- H\. Wu, X\. Huang, Y\. Chen, Y\. Zhang, Y\. Wang, and W\. Xie \(2026\)Spatialscore: towards comprehensive evaluation for spatial intelligence\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 31029–31041\.Cited by:[§1](https://arxiv.org/html/2608.07955#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2608.07955#S4.SS1.p2.1)\.
- R\. Xu, X\. Wang, T\. Wang, Y\. Chen, J\. Pang, and D\. Lin \(2024\)Pointllm: empowering large language models to understand point clouds\.InEuropean Conference on Computer Vision,pp\. 131–147\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p2.1)\.
- J\. Yang, S\. Yang, A\. W\. Gupta, R\. Han, L\. Fei\-Fei, and S\. Xie \(2025a\)Thinking in space: how multimodal large language models see, remember, and recall spaces\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 10632–10643\.Cited by:[§1](https://arxiv.org/html/2608.07955#S1.p1.1),[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1)\.
- L\. Yang, B\. Kang, Z\. Huang, Z\. Zhao, X\. Xu, J\. Feng, and H\. Zhao \(2024\)Depth anything v2\.External Links:2406\.09414Cited by:[§1](https://arxiv.org/html/2608.07955#S1.p1.1)\.
- S\. Yang, R\. Xu, Y\. Xie, S\. Yang, M\. Li, J\. Lin, C\. Zhu, X\. Chen, H\. Duan, X\. Yue,et al\.\(2025b\)Mmsi\-bench: a benchmark for multi\-image spatial intelligence\.arXiv preprint arXiv:2505\.23764\.Cited by:[§2\.1](https://arxiv.org/html/2608.07955#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.07955#S4.SS1.p1.1)\.
- K\. Yu, Z\. Zhou, S\. Tian, X\. Yang, Z\. Jia, M\. Yang, Z\. Cheng, L\. Guo, and Y\. Li \(2026\)Thinking with tables: enhancing multi\-modal tabular understanding via neuro\-symbolic reasoning\.arXiv preprint arXiv:2603\.24004\.Cited by:[§1](https://arxiv.org/html/2608.07955#S1.p1.1)\.
- Y\. Zhang, X\. Lu, S\. Yin, C\. Fu, W\. Chen, X\. Hu, B\. Wen, K\. Jiang, C\. Liu, T\. Zhang,et al\.\(2025\)Thyme: think beyond images\.arXiv preprint arXiv:2508\.11630\.Cited by:[§1](https://arxiv.org/html/2608.07955#S1.p1.1)\.
- A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. Huang \(2024\)Expel: llm agents are experiential learners\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19632–19642\.Cited by:[§2\.2](https://arxiv.org/html/2608.07955#S2.SS2.p1.1)\.

## Supplementary Material

Section[A](https://arxiv.org/html/2608.07955#A1)reports implementation settings, native tools, and skill representations\. Section[B](https://arxiv.org/html/2608.07955#A2)presents end\-to\-end inference and online\-evolution cases\. Section[C](https://arxiv.org/html/2608.07955#A3)summarizes the retained learned skills, and Section[D](https://arxiv.org/html/2608.07955#A4)provides the complete prompt templates used for inference and evolution\.

## Appendix AImplementation Details

### A\.1Hyperparameter Settings

We report the implementation settings used for skill initialization, online induction, candidate fusion, empirical skill maintenance, and inference\. Most settings are shared across datasets and LVLM backbones, while dataset\-specific settings are listed separately\.

Skill Initialization\.We initialize the online library with a small set of dataset\-specific Tool\-Use seed skills\. These seeds provide initial pipeline structures, while the mutable skill library is subsequently updated from online interaction trajectories\.

Online Induction and Fusion\.Online induction starts after an initial warm\-up period\. Interaction traces are organized by routing family before being summarized into candidate skills\. Candidate skills are subsequently fused with the mutable skill library\.

Skill Selection and Library Capacity\.At inference time, the selector first observes compact descriptions of the available skills and retrieves a small subset relevant to the current problem\. Complete executable definitions are exposed only for the selected skills, reducing prompt length and irrelevant context\.

Empirical Skill Maintenance\.Newly induced skills enter a probationary state\. Their invocation statistics and task outcomes are accumulated during subsequent episodes\. A skill can be promoted after obtaining sufficient empirical support, whereas consistently unsuccessful skills are removed from the mutable library\. The initial seed skills are protected from this retirement procedure\.

Table 3:Shared settings for initialization, online induction, selection, and empirical maintenance\.Runtime and Inference Settings\.All experiments use the simple agent runtime\. Queries are processed in waves of five\. We allow a larger reasoning budget for MMSI because its questions typically require longer multi\-view geometric pipelines\. GroundingDINO is fixed as the object\-detection backend across LVLM backbones, preventing changes in the reasoning model from simultaneously changing the visual grounding component\.

Table 4:Dataset\-specific inference settings\.
### A\.2Native Tool Library

The agent is equipped with a fixed library of native neuro\-symbolic tools shared across all three benchmarks\. These tools provide visual grounding, 3D reconstruction, pose estimation, metric\-scale recovery, text recognition, symbolic camera\-layout analysis, and executable numerical reasoning\. A Tool\-Use skill organizes calls to these native tools into a task\-specific pipeline\.

Table 5:Native tool atoms available to the agent\.Tool outputs are stored as typed workspace variables and can be consumed by later steps\. For example, a detector output can be passed to 3D projection, the projected points can be combined with camera extrinsics, and a geometry skill can use these variables to compute the final spatial decision\.

The visual foundation models supporting these tools are treated as internal backends rather than planner\-visible tool atoms\. Specifically, GroundingDINO supports open\-vocabulary detection, VGGT supports multi\-view reconstruction, MoGe supports metric\-scale estimation, and the object\-pose backend provides orientation evidence\. Fixing these backends ensures that changing the LVLM backbone does not simultaneously change the underlying visual tool implementation\.

We additionally usePlanner\.reasonas an internal semantic transition for role binding, option interpretation, and skill retrieval\. Since it does not invoke an external perception or computation module, it is not counted as a native tool execution\.

### A\.3Skill Representation and Retrieval

The skill library contains two complementary types of reusable experience:*Tool\-Use skills*and*geometry skills*\. A Tool\-Use skill represents a structured pipeline composed of executable tool atoms, whereas a geometry skill stores a reusable computational kernel together with its applicability conditions\. Geometry skills are internally represented aspythoncode lines\.

Table 6:Overview of the two reusable skill types\.#### Tool\-Use Skill Representation

A Tool\-Use skill is stored as a high\-level pipeline\. Itsdocstringdescribes when the skill should be retrieved, whilestepsspecify the ordered tool atoms and their semantic responsibilities\. Thestopping\_conditiondefines when the pipeline has collected sufficient evidence to return control to the planner\.

Listing[A\.3](https://arxiv.org/html/2608.07955#A1.SS3.SSSx1)presents an abridged pipeline skill for a single\-anchor spatial reasoning problem\.

`Listing 1: Tool\-Use Skill Representation The pipeline representation separates reusable structure from instance\-specific bindings\. For example, the skill specifies that a target must be detected and projected, while the target name, source view, selected bounding box, reference camera, and answer\-option mapping are bound from the current question at execution time\.`

`Geometry Skill Representation A geometry skill is represented as a self\-contained computational kernel\. Internally, the kernel may contain multiple helper routines, each implementing an atomic geometric operation, such as robust point aggregation, coordinate transformation, option parsing, or directional scoring\. These routines are composed into a high\-level decision function and stored as a single Geometry skill\. Listing A\.3 shows the internal composition of a representative geometry skill\. For readability, metadata are rendered as comments and the executable kernel is shown as ordinary Python; the actual record serializes the same lines in its code\_lines field\. Function bodies are abbreviated, while the decomposition follows the stored rule\. Listing 2: Geometry Skill Kernel The helper routines provide a modular decomposition of the geometry kernel: • \_extract\_points\_and\_weights binds projection outputs to the expected numerical inputs; • \_robust\_center aggregates noisy 3D observations; • \_world\_to\_camera enforces the requested reference frame; • \_parse\_option\_direction converts answer semantics into comparable geometric vectors; and • reusable\_decision composes these operations into an auditable high\-level decision kernel\. These helper routines are not independently retrieved by the skill selector\. Instead, the selector retrieves the high\-level geometry skill through its docstring, after which the complete kernel and its usage constraints are exposed to the reasoning model\. The empirical\_stats field is updated from subsequent trajectories\. Here, matched denotes retrieval by the selector, whereas used indicates that the skill was actually executed\. The outcome counters record whether samples answered after using the skill were judged correct or incorrect\. These statistics support empirical promotion and retirement, but are not exposed to the first\-stage selector\. Two\-Stage Skill Retrieval Providing every complete skill to the reasoning model would introduce substantial context length and irrelevant executable details\. We therefore use two\-stage retrieval\. In the first stage, each pipeline is reduced to a compact selection payload containing its identifier, type, level, docstring, and a bounded structural hint\. Listing 3: Compact Selector Payload Given the question, a compact workspace summary, and these selection payloads, the selector returns only the identifiers of relevant skills: Listing 4: Skill\-Selector Output In the second stage, the complete stored representation is expanded only for the selected skills\. For a Tool\-Use skill, the planner receives the ordered atoms, step instructions, and stopping condition\. For a geometry skill, it receives the usage constraints and reusable code kernel\. This design preserves semantic and executable information after selection while keeping the initial retrieval context compact\. Appendix B Case Studies B\.1 End\-to\-End Tool Execution We present representative examples to illustrate how the agent retrieves skills, executes native tools, reuses geometry kernels, and derives its final answer from explicit tool outputs\. Case 1: Camera\-Motion Reasoning on MindCube This case illustrates how a Tool\-Use skill and a Geometry skill work together to solve a camera\-motion question\. The input consists of two views of the same scene, and the agent must determine the motion of the camera from the first view to the second\. Input\. The question asks: \(a\) First input view \(b\) Second input view Figure 7: Original input images for the MindCube camera\-motion case\. The task asks for the camera motion from the first view to the second view\. Question\. Based on these two views in Figure 7 showing the same scene, in which direction did I move from the first view to the second view? Options\. \(A\) diagonally forward and right; \(B\) directly right; \(C\) diagonally forward and left; and \(D\) directly left\. Skill Retrieval\. The selector first examines the compact descriptions of the available Tool\-Use skills\. Since the question requires reasoning about the relative motion between two cameras, it retrieves mindcube\_recon\_then\_python\_skeleton\. After selection, the complete pipeline is exposed to the agent\. Retrieved Tool\-Use Skill Internal ID\. mindcube\_recon\_then\_python\_skeleton Applicability\. Use when a multi\-view reconstruction directly contains the camera geometry required by the question and no explicit object grounding is needed\. Step 1: Reconstruct the shared scene\. Call GeometricReconstructor\.reconstruct on all input images to recover their shared geometry and camera poses\. ↓\\boldsymbol\{\\downarrow\} Step 2: Resolve the spatial relation\. Pass the reconstruction output to PythonTool\.code and use a Geometry skill to compute the camera\-frame motion and map it to one of the answer choices\. Stopping condition\. Stop when the recovered geometry supports a unique benchmark\-formatted answer without additional object grounding\. The retrieved skill determines the high\-level execution order, while leaving the benchmark\-specific geometric computation to the subsequent Geometry skill\. Scene Reconstruction\. Following the retrieved pipeline, the agent invokes GeometricReconstructor\.reconstruct on the two input images\. The tool successfully produces a shared reconstruction and estimates the extrinsic parameters of both cameras\. This operation converts the original visual question into a structured geometric problem\. The camera motion can now be computed in a common coordinate system instead of being inferred directly from appearance\. Geometry Reasoning\. Once the camera poses are available, the selector retrieves camera0\_motion\_option\_from\_extrinsics\. The skill defines the first camera as the reference frame and interprets the displacement of the second camera relative to it\. The following listing shows the core structure of the retrieved Geometry skill\. Low\-level validation and diagnostic statements are omitted for clarity\. CAMERA\-MOTION GEOMETRY KERNEL The helper routines implement atomic geometric operations, including extrinsic normalization, camera\-center recovery, reference\-frame conversion, option parsing, and direction comparison\. The high\-level Geometry skill composes these operations into a reusable decision kernel\. For this instance, the kernel identifies both a leftward component and a forward component in the second camera’s motion\. Final Decision\. Result\. The recovered motion is diagonally forward\-left, corresponding to option C, which agrees with the ground\-truth answer\. Execution path: two\-view input →\\rightarrow Tool\-Use skill retrieval →\\rightarrow scene reconstruction →\\rightarrow Geometry skill retrieval →\\rightarrow numerical execution →\\rightarrow C\. Case 2: Grounded Object\-Direction Reasoning on MMSI This case requires both visual grounding and camera\-frame geometric reasoning\. Unlike Case 1, the target location cannot be obtained from the camera poses alone\. The agent must first identify the specified object and lift its image\-space location into the reconstructed 3D scene\. Input\. The question asks: \(a\) First input view \(b\) Second input view Figure 8: Original input images for the MMSI grounded\-direction case\. The painting is grounded in Image 1, while its direction is evaluated relative to the observer in Image 2\. Question\. When you took the photo in Image 2 in Figure 8, where was the painting with a black frame and white background in relation to you? Options\. \(A\) rear right; \(B\) rear left; \(C\) front left; and \(D\) directly to the left\. Skill Retrieval\. The question names one target object and asks for its position relative to the observer in a specified image\. The selector therefore retrieves mmsi\_single\_anchor\_grounding\_skeleton\. Retrieved Tool\-Use Skill Internal ID\. mmsi\_single\_anchor\_grounding\_skeleton Applicability\. Use when a question asks where one named object or scene region is located relative to an observer or a camera\-defined reference frame\. Step 1: Reconstruct the scene\. Call GeometricReconstructor\.reconstruct to recover a shared 3D scene and the camera poses of the input images\. ↓\\boldsymbol\{\\downarrow\} Step 2: Ground the target\. Call SemanticDetector\.detect to localize the named object in an input image\. ↓\\boldsymbol\{\\downarrow\} Step 3: Lift the target into 3D\. Call GeometricReconstructor\.project\_box\_to\_3d\_points to project the selected detection into the shared 3D scene\. ↓\\boldsymbol\{\\downarrow\} Step 4: Resolve the camera\-frame relation\. Pass the reconstruction and grounded target points to PythonTool\.code\. Apply a Geometry skill to express the target relative to the requested camera and select the corresponding option\. Stopping condition\. Stop when the target has been grounded in 3D and its direction in the requested camera frame maps to one unique answer\. The retrieved skill determines both the required tool sequence and the data flow between the tools: the detector supplies a bounding box to the projection tool, and the projected points are subsequently consumed by the geometric decision kernel\. Scene Reconstruction\. Following the retrieved pipeline, the agent first invokes GeometricReconstructor\.reconstruct on the two images\. The operation successfully recovers a common 3D scene and estimates the poses of both cameras\. The resulting reconstruction provides the transformation required to express an object observed in Image 1 relative to the observer in Image 2\. Target Grounding\. The agent invokes SemanticDetector\.detect with the description “the painting with a black frame and white background\.” The detector returns multiple candidate boxes\. By inspecting their visual locations, the agent selects the small framed artwork near the bar opening in Image 1\. The selected box is then passed to GeometricReconstructor\.project\_box\_to\_3d\_points\. This operation lifts the painting from its two\-dimensional bounding box into a set of points in the shared 3D coordinate frame\. At this stage, the workspace contains both pieces of evidence required for geometric reasoning: the Image 2 camera pose and the 3D points associated with the painting\. Geometry Reasoning\. The selector retrieves camera\_frame\_object\_quadrant\_choice\. This skill determines the direction of a grounded object relative to a selected camera and maps that direction to the semantic labels in the answer choices\. The core structure of the retrieved Geometry skill is shown below\. Input validation and diagnostic branches are omitted for clarity\. CAMERA\-FRAME OBJECT\-QUADRANT KERNEL For this instance, the agent binds the reference camera to Image 2 and passes the projected painting points as the target\. The helper operations estimate a robust target center, transform it from the shared world frame into the Image 2 camera frame, and compare its planar direction with the four candidate relations\. The resulting camera\-frame direction is front\-left\. Final Decision\. Result\. The grounded painting is front\-left in the Image 2 camera frame, corresponding to option C\. Execution path: two\-image input →\\rightarrow Tool\-Use skill retrieval →\\rightarrow scene reconstruction →\\rightarrow target detection →\\rightarrow 3D projection →\\rightarrow Geometry skill retrieval →\\rightarrow camera\-frame reasoning →\\rightarrow C\. Case 3: Allocentric Relation Reasoning on OmniSpatial This case demonstrates allocentric spatial reasoning, in which the required reference frame belongs to a visible entity rather than to the camera\. The agent must estimate the orientation of a projection screen and determine the position of an electric fan from the screen’s viewpoint\. Input\. The question asks: Figure 9: Original input image for the OmniSpatial allocentric case\. The electric fan is localized relative to the projection screen’s object\-centric viewpoint\. Question\. In Figure 9, where is the electric fan, seen from the projection screen’s viewpoint? Options\. \(A\) right and \(B\) left\. Skill Retrieval\. The selector identifies the projection screen as the reference entity, the electric fan as the target entity, and left–right as the requested relation\. It retrieves omnispatial\_allocentric\_pairwise\_role\_completion\_skeleton\. Retrieved Tool\-Use Skill Internal ID\. omnispatial\_allocentric\_pairwise\_role\_completion\_skeleton Applicability\. Use for a visible\-entity viewpoint question containing one oriented reference entity and one target entity, where the target must be classified in the reference entity’s coordinate frame\. Step 1: Bind the semantic roles\. Identify the reference entity, target entity, requested relation, and option meanings separately\. ↓\\boldsymbol\{\\downarrow\} Step 2: Reconstruct the scene\. Call GeometricReconstructor\.reconstruct to establish a shared 3D coordinate frame\. ↓\\boldsymbol\{\\downarrow\} Step 3: Ground both entities\. Call SemanticDetector\.detect separately for the reference and target\. One aggregate detection cannot fill both roles\. ↓\\boldsymbol\{\\downarrow\} Step 4: Estimate the reference orientation\. Call ObjPoseEstimator\.predict\_obj\_pose on the selected reference box to establish its object\-centric frame\. ↓\\boldsymbol\{\\downarrow\} Step 5: Lift both entities into 3D\. Call GeometricReconstructor\.project\_box\_to\_3d\_points for both selected boxes in the same reconstruction\. ↓\\boldsymbol\{\\downarrow\} Step 6: Resolve the allocentric relation\. Pass the reference pose and both 3D anchors to PythonTool\.code, then apply a Geometry skill to classify the target in the reference frame\. Stopping condition\. Stop when the reference identity and orientation, target identity, same\-frame 3D anchors, and option mapping jointly support one answer\. This skill is more constrained than a simple visual left–right heuristic\. In particular, it requires explicit orientation evidence for the reference entity and does not permit the camera frame to be silently substituted for the reference frame\. Scene Reconstruction\. The agent invokes GeometricReconstructor\.reconstruct on the input image\. The tool recovers a 3D representation of the scene that will be shared by the subsequent grounding and pose\-estimation steps\. Role\-Specific Grounding\. The agent invokes SemanticDetector\.detect twice with role\-specific prompts: once for the projection screen and once for the electric fan\. Both calls return one valid detection\. Keeping these calls separate preserves the semantic distinction between the reference and target\. The screen detection defines the entity whose viewpoint is requested, while the fan detection defines the entity whose relative position must be determined\. Reference\-Frame Construction\. The selected screen box is passed to ObjPoseEstimator\.predict\_obj\_pose\. The resulting semantic 6D pose provides the orientation of the screen and defines its local left–right frame\. The agent then projects both the screen box and the fan box through GeometricReconstructor\.project\_box\_to\_3d\_points\. This produces two point sets in the same reconstructed world frame: one for the reference screen and one for the target fan\. Geometry Reasoning\. With the reference pose and both 3D anchors available, the selector retrieves allocentric\_left\_right\_from\_reference\_pose\. The following listing shows the central computation of the retrieved Geometry skill\. Validation and diagnostic branches are omitted for clarity\. ALLOCENTRIC LEFT–RIGHT GEOMETRY KERNEL The skill first estimates robust centers for the screen and fan\. It then combines the screen center with its estimated orientation to construct a screen\-centric coordinate frame\. The fan center is transformed into this frame, and the sign of its local lateral component determines whether it lies to the screen’s left or right\. For this instance, the transformed fan position has a positive rightward component in the screen\-centric frame\. Final Decision\. Result\. The fan lies to the screen’s right in the screen\-centric frame, corresponding to option A\. Execution path: single\-image input →\\rightarrow Tool\-Use skill retrieval →\\rightarrow scene reconstruction →\\rightarrow reference and target detection →\\rightarrow reference pose estimation →\\rightarrow paired 3D projection →\\rightarrow Geometry skill retrieval →\\rightarrow allocentric transformation →\\rightarrow A\. B\.2 Online Skill Evolution The preceding cases demonstrate how existing skills guide inference\. We next show how the skill library itself changes during online interaction\. We separately examine the evolution of a Tool\-Use skill and a Geometry skill\. Case 4: Evolution of a Tool\-Use Skill \(a\) Front \(b\) Left \(c\) Back \(d\) Right Figure 10: Original four\-view input for the evolved Tool\-Use skill\. In Figure 10, this case concerns four\-view MindCube questions in which an observer starts from a specified viewpoint, turns left or right, moves forward, and must decide whether the movement approaches a named destination object\. Initial Skill\. For this task family, the initial library contains four\_view\_turn\_move\_symbolic\_layout\_root\. The skill reconstructs the shared scene and resolves the observer’s orientation after the specified turn\. Initial Tool\-Use Skill Internal ID\. four\_view\_turn\_move\_symbolic\_layout\_root Applicability\. Use for same\-position multi\-view questions that provide an ordered camera layout and ask about turning left or right from a named view\. Pipeline\. GeometricReconstructor\.reconstruct ↓\\boldsymbol\{\\downarrow\} LanguageToCamera\.visualize\_camera\_layout Stopping condition\. Stop when the shared scene and the observer’s post\-turn orientation have been recovered\. This pipeline is sufficient for questions that only ask which direction the observer faces after turning\. However, it does not fully solve questions that additionally ask whether moving forward approaches a specific destination object\. Observed Failure\. In unsuccessful trajectories, the initial pipeline correctly reconstructed the scene and resolved the post\-turn viewpoint, but then stopped\. Consequently, the workspace contained the observer’s final orientation but not the location of the named destination\. For example, after resolving a left turn from Image 2, the agent could determine that the observer would face the direction represented by Image 3\. However, it had not yet grounded the target object in Image 3 and therefore could not determine whether forward movement would reduce the distance to that object\. The missing evidence was thus not another camera\-layout operation\. It was a destination\-grounding and movement\-comparison stage after the symbolic turn had been resolved\. Residual Induction\. The online induction process groups successful and unsuccessful trajectories from the same task family\. Both groups share the following parent prefix: scene reconstruction →\\rightarrow symbolic turn resolution\. Successful trajectories continue with two additional semantic operations: ground the named destination in the turned\-facing view →\\rightarrow evaluate whether forward movement approaches it\. In contrast, the unsuccessful trajectories terminate after the shared prefix\. This comparison identifies the latter two operations as a reusable residual extension of the parent pipeline\. Importantly, the induction process uses more than the atom sequence\. The trace records specify that the detector must ground the destination entity in the turned\-facing image, and that the numerical step must evaluate distance change in the post\-turn observer frame\. Branch Fusion\. Fusion preserves the original pipeline and creates a specialized branch for turn\-then\-move questions that mention a destination object\. The new branch is stored as four\_view\_turn\_move\_target\_grounding\_branch\. Tool\-Use Skill After Online Fusion Internal ID\. four\_view\_turn\_move\_target\_grounding\_branch Parent skill\. four\_view\_turn\_move\_symbolic\_layout\_root Retained parent prefix\. GeometricReconstructor\.reconstruct ↓\\boldsymbol\{\\downarrow\} LanguageToCamera\.visualize\_camera\_layout Online extension\. SemanticDetector\.detect ↓\\boldsymbol\{\\downarrow\} PythonTool\.code The detector grounds the named destination in the image corresponding to the resolved post\-turn orientation\. The numerical step then determines whether forward motion in that frame decreases the distance to the grounded destination\. New stopping condition\. Stop only after the turned\-facing view, destination grounding, and forward\-motion comparison jointly produce a yes–no answer\. The fusion operation therefore does not overwrite the more general parent skill\. The original pipeline remains applicable to pure viewpoint questions, while the new branch handles the narrower turn\-then\-move task family\. Later Reuse\. In a subsequent query, the observer starts from Image 2, turns left, moves forward, and asks whether the movement gets closer to a smoking machine\. The selector retrieves the evolved branch and instantiates its open semantic slots using the current question\. Instantiated Evolved Tool\-Use Skill Internal ID\. four\_view\_turn\_move\_target\_grounding\_branch Current semantic bindings\. Starting viewpoint: Image 2 Turn operation: left turn Resolved facing view: Image 3 Destination entity: smoking machine Requested decision: whether forward movement approaches the destination Instantiated execution\. 1\. Shared\-scene reconstruction GeometricReconstructor\.reconstruct Recover the shared geometry and camera poses from the four ordered views\. ↓\\boldsymbol\{\\downarrow\} 2\. Post\-turn viewpoint resolution LanguageToCamera\.visualize\_camera\_layout Bind Image 2 as the starting viewpoint and apply a left turn\. The resulting forward direction is aligned with Image 3\. ↓\\boldsymbol\{\\downarrow\} 3\. Destination grounding SemanticDetector\.detect Detect the smoking machine specifically in Image 3, which is the resolved post\-turn reference view\. ↓\\boldsymbol\{\\downarrow\} 4\. Forward\-motion comparison PythonTool\.code Use the reconstruction, resolved camera layout, and grounded destination to determine whether moving forward decreases the distance to the smoking machine\. Output\. The forward movement approaches the destination: A\. Yes\. The first generated numerical program contains an execution\-level syntax error\. The agent regenerates the computation while preserving the retrieved pipeline, semantic bindings, and reference frame\. The corrected execution returns A\. Yes\. Evolution path: partial parent pipeline →\\rightarrow recurring missing destination evidence →\\rightarrow trace\-level residual induction →\\rightarrow candidate branch fusion →\\rightarrow instantiated evolved skill →\\rightarrow successful execution\. Case 5: Evolution of a Geometry Skill This case shows how repeated sample\-specific computations are converted into a reusable Geometry skill\. In figure 7, the task family consists of two\-view camera\-motion questions that ask whether the second camera moved left, right, diagonally forward\-left, or diagonally forward\-right relative to the first camera\. Initial Execution\. The initial Tool\-Use skill already provides the appropriate high\-level pipeline: scene reconstruction →\\rightarrow numerical reasoning over camera extrinsics\. However, the initial library does not contain a specialized Geometry skill for converting two world\-to\-camera extrinsics into one of the benchmark motion choices\. Consequently, PythonTool\.code must generate this computation independently for each sample\. Different trajectories implement the same underlying operations in slightly different ways: invert camera extrinsics, recover camera centers, compute the displacement between the views, and compare its lateral and forward components with the answer choices\. Recurring Computation\. The online trace buffer contains multiple two\-view motion trajectories with the same semantic objective and data requirements\. A successful trajectory correctly recovers the second camera center, expresses its displacement relative to the first camera, and selects the corresponding diagonal motion option\. Another trajectory attempts the same computation but uses brittle option parsing and fallback behavior\. When the current options are not extracted correctly, its sample\-specific program can return an unrelated or incorrectly mapped label\. The two trajectories therefore reveal both the reusable computation and the behavior that must not be retained: Recurring Geometry Pattern Required inputs\. A stack containing at least two world\-to\-camera extrinsic matrices and a set of answer choices describing left, right, or diagonal\-forward camera motion\. Shared computation\. recover camera centers ↓\\boldsymbol\{\\downarrow\} express camera\-1 displacement in the camera\-0 frame ↓\\boldsymbol\{\\downarrow\} retain the lateral and forward components ↓\\boldsymbol\{\\downarrow\} parse the current option semantics ↓\\boldsymbol\{\\downarrow\} select the best\-supported motion option Observed failure mode\. A sample\-specific implementation may silently use hard\-coded option labels or a default answer when option parsing, extrinsic validation, or motion estimation fails\. Because the traces share the same reference frame, input schema, geometric transformation, and output semantics, they form one code\-control family rather than unrelated Python programs\. Atomic\-Operation Induction\. The induction process decomposes the recurring programs into a small set of reusable geometric operations: • normalize and validate the extrinsic stack; • recover a camera center from a world\-to\-camera transform; • express camera displacement in the first\-camera frame; • parse direction semantics from the current answer choices; • score direct and diagonal motion candidates; and • return a diagnostic instead of fabricating an answer when the required evidence is invalid\. These operations are not stored as independently retrievable skills\. They are composed as helper routines inside one self\-contained high\-level Geometry kernel\. Geometry\-Skill Fusion\. Fusion produces the candidate Geometry skill camera0\_motion\_option\_from\_extrinsics\. The skill records not only the reusable code but also its applicability conditions, input bindings, reference\-frame convention, and failure guards\. Evolved Geometry Skill Internal ID\. camera0\_motion\_option\_from\_extrinsics Function\. Choose among direct\-left, direct\-right, diagonal\-forward\-left, and diagonal\-forward\-right options by expressing the second camera’s motion in the first camera’s coordinate frame\. Input contract\. The input must contain at least two valid world\-to\-camera extrinsics\. The first camera defines the reference frame, with positive horizontal motion pointing right and positive depth motion pointing forward\. Option contract\. The answer choices must contain recognizable left, right, and forward\-motion semantics\. Their option letters are parsed from the current question rather than fixed in the skill\. Failure guards\. The skill returns a diagnostic when the extrinsic stack is missing or malformed, fewer than two cameras are available, the motion magnitude is degenerate, or the options cannot be parsed\. The core structure of the evolved kernel is shown below\. Detailed validation messages and scoring adjustments are omitted for clarity\. EVOLVED CAMERA\-MOTION GEOMETRY KERNEL Compared with the original sample\-specific programs, the evolved skill makes the reference frame explicit, parses the current option mapping, and replaces unsafe default answers with observable diagnostics\. Later Retrieval and Reuse\. The later query presented in Case 1 again asks for the motion between two camera views\. Its Tool\-Use pipeline first invokes GeometricReconstructor\.reconstruct, producing the required camera extrinsics\. At the subsequent Python step, the Geometry selector initially sees the compact description of camera0\_motion\_option\_from\_extrinsics\. After selecting it, the runtime exposes the complete usage conditions and reusable kernel\. The current reconstruction and answer choices are then bound to the kernel’s inputs\. Instantiated Evolved Geometry Skill Internal ID\. camera0\_motion\_option\_from\_extrinsics Current bindings\. Extrinsic input: camera poses returned by the current reconstruction Reference frame: the first input camera Compared camera: the second input camera Current options: direct\-left, direct\-right, diagonal\-forward\-left, and diagonal\-forward\-right Execution\. The kernel recovers both camera centers, expresses the displacement of the second camera in the first\-camera frame, and compares its lateral and forward components with the current option semantics\. Output\. The recovered motion is diagonally forward and left, producing option C for this instance\. The result illustrates the full Geometry\-skill evolution cycle: sample\-specific computation is observed in traces, recurring atomic operations are induced, fusion creates a guarded reusable kernel, and the resulting skill is retrieved to guide later numerical execution\. Evolution path: independent generated programs →\\rightarrow recurring geometric pattern →\\rightarrow atomic\-operation induction →\\rightarrow guarded Geometry kernel →\\rightarrow later retrieval and reuse\. Appendix C Evolved Skill Inventory The following tables list the skills retained after online evolution\. Initial seed skills are excluded, and readable names replace internal identifiers\. Table 7: Evolved skill inventory for MindCube\. Skill type Skill name Tool\-Use Four\-View Behind\-Me Camera Layout Explicit Turn\-View Symbolic Layout Four\-View Turn\-and\-Move Symbolic Layout Behind\-Me Camera Layout with Behind\-View Option Grounding Behind\-Me Resolved\-View Option Grounding Query\-View Candidate Grounding with a Target\-Pose Frame Four\-View Turn\-and\-Move Target Grounding Query\-View Candidate Grounding with a Query\-View Target Pose Behind\-Me Symbolic Layout followed by Option Grounding Turned\-View Target Grounding Turned\-View Target Grounding V2 Geometry Camera\-0 Motion from Extrinsics Right\-Turn Distance Change to a Target Left\-Turn Proximity in the Camera Frame Table 8: Evolved skill inventory for MMSI\. Skill type Skill name Tool\-Use Single\-Anchor Grounding with Camera\-Layout Binding Reconstruction and Camera\-Layout Reasoning for Egomotion Dual\-Landmark Grounding for Observer\-Relative Comparison Geometry Object\-Side Reasoning from Pose Bearing Camera\-Frame Object\-Quadrant Selection Absolute Compass\-Side Reasoning from a Reference View Table 9: Evolved skill inventory for OmniSpatial\. Skill type Skill name Tool\-Use Refined Entity Redetection after Grounding Review Camera\-Arrival Grounding Review Detection and Grounding Ambiguity Review Allocentric Pairwise Grounding\-Readiness Review Egocentric Relation Failure Review Allocentric Egocentric\-Relation Review Reference\-Frame Relation Counting Grounding\-Consistency Review Viewpoint\-Conditioned Region Membership Allocentric Axial\-Relation Review Geometry Camera\-Relative Travel\-Time Selection Allocentric Left–Right Reasoning from Reference Pose Appendix D Prompt Templates This section presents the core prompts used during inference and online skill evolution\. To improve readability, we organize each prompt into four parts: Purpose, Inputs, Instructions, and Output format\. Full prompts will be available after the code release\. SKILL RETRIEVAL PROMPT Purpose\. Select a small set of reusable rules that are relevant to the current task and workspace context\. The selector sees only compact rule descriptions at this stage\. Inputs\. • <TASK\_INSTRUCTION\> • <CONTEXT\_SUMMARY\> • <RULE\_TYPE\> • <TARGET\_ACTION\_IF\_APPLICABLE\> • <SELECTION\_CONTEXT\> • <CANDIDATE\_RULES\_JSON\> • <MAX\_RULES\> • optional <ERROR\_FEEDBACK\> Instructions\. 1\. Select only rules whose docstring fits the current task and context\. 2\. Use structure\_hint only to distinguish otherwise similar rules\. 3\. Do not select a rule merely because one word or one tool overlaps\. 4\. It is valid to select no rules if none are specific enough\. 5\. Return at most <MAX\_RULES\> rule ids\. Strict output format\. ⬇ 1\{ 2 "selected\_rule\_ids": \[ 3 "rule\_id\_if\_useful" 4 \], 5 "rationale": "brief reason for the selection, or why none apply" 6\} If no rule is useful, return: ⬇ 1\{ 2 "selected\_rule\_ids": \[\], 3 "rationale": "no candidate rule is specific enough for this task" 4\} INFERENCE PLANNER PROMPT Purpose\. Act as the planner for a tool\-using spatial reasoning agent\. At each turn, choose exactly one top\-level decision: use one complete Tool\-Use skill, execute one immediate tool action, or return the final answer\. Inputs\. • <TASK\_INSTRUCTION\> • <WORKSPACE\_SUMMARY\> • <RECENT\_OBSERVATIONS\> • <CANDIDATE\_BLOCK\_RULES\> • <AVAILABLE\_TOOL\_SCHEMAS\> • optional <ERROR\_FEEDBACK\> Instructions\. 1\. Choose exactly one decision type: • decision\_type="rule": select one reusable high\-level rule by id; • decision\_type="action": emit one immediate tool action or the final answer\. 2\. A block rule is a semantically complete reusable procedure, not an arbitrary tool\-call fragment\. 3\. The executor does not decide final answers; return a final answer only when workspace evidence is sufficient\. 4\. Use missing\_information to list unresolved semantic roles, such as the reference frame, reference entity, target, candidate set, requested relation, or option mapping\. 5\. One aggregate detection cannot fill multiple semantic roles\. 6\. Prefer one small tool action at a time unless multiple calls are clearly independent\. 7\. For spatial direction, distance, or orientation, prefer explicit geometry and deterministic computation\. 8\. For an object\-centric viewpoint, use object\-pose evidence when a visible entity defines the reference orientation\. Strict output format\. Rule decision: ⬇ 1\{ 2 "decision\_type": "rule", 3 "use\_rule": true, 4 "selected\_rule\_id": "exact\_rule\_id", 5 "selected\_rule\_reason": "why this complete rule matches the current need", 6 "missing\_information": \[ 7 "unresolved semantic role" 8 \], 9 "tool\_calls": \[\], 10 "final\_answer": null 11\} Immediate action: ⬇ 1\{ 2 "decision\_type": "action", 3 "use\_rule": false, 4 "selected\_rule\_id": null, 5 "selected\_rule\_reason": "why an immediate action is needed", 6 "missing\_information": \[ 7 "unresolved semantic role" 8 \], 9 "tool\_calls": \[ 10 \{ 11 "tool\_name": "Tool\.method", 12 "output\_variable": "workspace\_name", 13 "args": \{\} 14 \} 15 \], 16 "final\_answer": null 17\} Final answer: ⬇ 1\{ 2 "decision\_type": "action", 3 "use\_rule": false, 4 "selected\_rule\_id": null, 5 "selected\_rule\_reason": "workspace already satisfies the task", 6 "missing\_information": \[\], 7 "tool\_calls": \[\], 8 "final\_answer": "answer derived from workspace evidence" 9\} TOOL\-USE SKILL INDUCTION PROMPT Purpose\. Infer one reusable pipeline extension from a batch of compact traces\. The goal is to grow a narrower child pipeline from an existing reusable trunk whenever possible\. Inputs\. • <ATOM\_CONTRACTS\_JSON\> • <CURRENT\_PIPELINE\_KB\_JSON\> • <TRACE\_BATCH\_JSON\> • optional <ERROR\_FEEDBACK\> Instructions\. 1\. Group traces by task family and required computation\. 2\. A valid family must contain at least two traces with the same semantic roles, evidence needs, and decision pattern, including at least one correct trace\. 3\. Generate at most one candidate from one supported family\. 4\. Use operation="branch" when an existing parent pipeline supplies a reusable trunk\. 5\. Record only the missing or replaced semantic operations in extension\_steps; fusion will materialize the complete child\. 6\. Use operation="new\_root" only when no reusable parent exists\. 7\. Do not generate incomplete fragments such as reconstruction\-only, retry\-only, or isolated detection/projection steps\. 8\. Do not invent tools, and do not place Geometry skill ids inside a Tool\-Use pipeline\. 9\. The stopping condition must describe a semantically complete and observable workspace state\. Strict output format\. ⬇ 1\{ 2 "candidate\_rules": \[ 3 \{ 4 "id": "short\_snake\_case\_extension", 5 "rule\_type": "pipeline\_delta", 6 "operation": "branch", 7 "parent\_rule\_id": "existing\_pipeline\_id", 8 "docstring": "narrow task family and added evidence need", 9 "extension\_steps": \[ 10 \{ 11 "atom": "Tool\.method", 12 "instruction": "missing semantic operation", 13 "bindings": \{ 14 "required\_arg": "$semantic\_workspace\_slot" 15 \}, 16 "output": "semantic\_output\_name" 17 \} 18 \], 19 "stopping\_condition": "semantically complete child result" 20 \} 21 \], 22 "rationale": "supporting trace family and recurring gap" 23\} If no supported reusable program exists, return: ⬇ 1\{ 2 "candidate\_rules": \[\], 3 "rationale": "no supported reusable program exists" 4\} GEOMETRY SKILL INDUCTION PROMPT Purpose\. Induce a reusable geometry kernel for PythonTool\.code from execution records\. The induced rule should capture a complete recurring computation, together with its applicability conditions and failure guards\. Inputs\. • <CURRENT\_CODE\_CONTROL\_KB\_JSON\> • <PYTHON\_TRACE\_BATCH\_JSON\> • optional <ERROR\_FEEDBACK\> Instructions\. 1\. Group records by the same computation, required variables, reference frame, and output decision\. 2\. A valid family must contain at least two traces, including at least one correct execution\. 3\. Prefer families where correct and incorrect traces expose a concrete reusable difference\. 4\. Generalize the complete decision computation rather than a trivial helper or a shared code substring\. 5\. usage must specify required variables, shapes, reference\-frame assumptions, and per\-call bindings\. 6\. code\_lines must define one self\-contained reusable kernel, possibly with helper functions and one decision function\. 7\. Missing or malformed evidence must return a diagnostic rather than a fabricated answer\. 8\. Do not encode sample ids, one\-off object names, option letters, or sample\-specific answers\. Strict output format\. ⬇ 1\{ 2 "candidate\_rules": \[ 3 \{ 4 "id": "short\_snake\_case\_kernel", 5 "rule\_type": "code\_control", 6 "level": "high\_level", 7 "docstring": "narrow coding scenario and evidence conditions", 8 "usage": \[ 9 "required inputs, frames, modes, and bindings", 10 "conditions under which the kernel must not be used" 11 \], 12 "code\_lines": \[ 13 "def reusable\_decision\(required\_values, mode\):", 14 " \# complete reusable computation", 15 " return result" 16 \] 17 \} 18 \], 19 "rationale": "why the traces share one reusable computation" 20\} If unsupported, return: ⬇ 1\{ 2 "candidate\_rules": \[\], 3 "rationale": "no supported reusable computation exists" 4\} SKILL FUSION PROMPT Purpose\. Fuse the mutable main library and the candidate library into a bounded active library\. For Tool\-Use skills, fusion materializes complete child pipelines; for Geometry skills, it merges reusable kernels\. Inputs\. • <MAX\_RULES\> • <READ\_ONLY\_PARENT\_RULES\_JSON\> • <MUTABLE\_MAIN\_KB\_JSON\> • <CANDIDATE\_KB\_JSON\> • <TRACE\_PROOF\_RESERVOIR\_JSON\> • optional <ERROR\_FEEDBACK\> Instructions\. 1\. Preserve the schema of Tool\-Use pipelines and Geometry kernels\. 2\. Parent pipelines are read\-only references and must not be rewritten\. 3\. For pipeline\_delta candidates with operation="branch", preserve the reusable parent trunk and insert the extension where it semantically belongs\. 4\. Never emit pipeline\_delta directly into the active library; always materialize a complete child\. 5\. Merge duplicate children and discard unsupported, overly broad, or sample\-specific records\. 6\. Discard children that still fail to reach the evidence state claimed by their docstring\. 7\. Tool\-Use pipelines may reference only native tool atoms and planner reasoning transitions\. 8\. When trace\-supported fusion is enabled, provide one pipeline\_proofs entry for every new or materially changed pipeline\. Strict output format\. ⬇ 1\{ 2 "rules": \[ 3 \{ 4 "id": "complete\_reusable\_skill", 5 "rule\_type": "pipeline\_or\_code\_control", 6 "\.\.\.": "fields from the corresponding schema" 7 \} 8 \], 9 "pipeline\_proofs": \[ 10 \{ 11 "rule\_id": "materialized\_pipeline", 12 "segments": \[ 13 \{ 14 "steps": \[ 15 \{ 16 "atom": "Tool\.method", 17 "input\_keys": \["required\_arg"\], 18 "semantic\_roles": \["target", "camera"\] 19 \} 20 \], 21 "prototype\_trace\_ids": \[ 22 "correct\_trace\_1", 23 "correct\_trace\_2" 24 \], 25 "counterexample\_trace\_ids": \[ 26 "incorrect\_trace\_1", 27 "incorrect\_trace\_2" 28 \] 29 \} 30 \] 31 \} 32 \] 33\} GEOMETRY\-GUIDED CODE GENERATION PROMPT Purpose\. Generate the executable wrapper for PythonTool\.code\. The code generator receives the current variables, their schemas, the reference\-frame requirements, and optionally a retrieved Geometry skill\. Inputs\. • <USER\_REQUEST\> • <REFERENCE\_FRAME\_DESCRIPTION\> • <COMPUTATIONAL\_OBJECTIVE\> • <CONTEXT\_DESCRIPTION\> • <VARIABLE\_SCHEMAS\_AND\_DOCUMENTATION\> • <SELECTED\_CODE\_CONTROL\_DOCSTRING\> • <SELECTED\_CODE\_CONTROL\_USAGE\> • <SELECTED\_REUSABLE\_KERNEL\> • optional <PREVIOUS\_EXECUTION\_ERROR\> Instructions\. 1\. Write one Python function that implements the requested computation\. 2\. Use only a reusable kernel whose input contract matches the current variables and objective\. 3\. Bind kernel inputs to current workspace values\. 4\. Do not redefine the selected helper block\. 5\. Adapt only the wrapper, current option mapping, indices, thresholds, and return format\. 6\. Verify the computation using the supplied variable documentation\. 7\. For multiple\-choice questions, evaluate the current options rather than relying on fixed option letters\. 8\. Missing or incomparable evidence should result in a diagnostic rather than a fabricated answer\. 9\. The returned result must be serializable\. Strict output format\. When a reusable Geometry kernel is selected: ⬇ 1\{ 2 "use\_code\_blocks": \[ 3 "exact\_geometry\_skill\.kernel" 4 \], 5 "execute\_code": 6"def execute\(current\_variables\):\\n\\ 7 result = reusable\_decision\(\.\.\.\)\\n\\ 8 return result" 9\} Otherwise, return exactly one Python code block defining: ⬇ 1def execute\(current\_variables\): 2 import \.\.\. 3 \.\.\. 4 return serializable\_value`

Similar Articles