Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation

Hugging Face Daily Papers Papers
long-horizon-navigation foundation-models embodied-ai scaffolding agents

Summary

NavMCP is an agentic scaffolding framework that integrates vision-language models with navigation foundation models to enable persistent long-horizon physical-world exploration, achieving state-of-the-art results on benchmarks and real-robot tasks.

Long-horizon physical-world agents must reason over distant goals while grounding decisions in reliable closed-loop behavior. Today's foundation models split these capabilities: vision-language models (VLMs) infer missing information and adapt high-level plans but remain brittle and inefficient at repeated navigation grounding, while navigation foundation models (NFMs) robustly execute semantic goals but operate as bounded episodes without persistent task-level reasoning. We introduce NavMCP, an agentic scaffolding framework that couples a VLM reasoning agent with an NFM executor for long-horizon exploration. The VLM decides what evidence to seek, where to search, and when to stop, while the NFM grounds each semantic sub-goal into closed-loop navigation. Three channels structure their collaboration: intent translates evidence needs into navigation calls, observation converts rollouts into source-grounded trajectory evidence, and memory accumulates findings, negative evidence, and unresolved goals across calls. This design turns isolated navigation rollouts into persistent embodied interaction without retraining either model. On Embodied Question Answering, NavMCP achieves state-of-the-art results on HM-EQA, MT-HM3D, and EXPRESS-Bench. Under matched agent and executor backbones, it outperforms an episodic interface by 14.9 percentage points on HM-EQA. On a Unitree Go2, NavMCP reaches 78.3% success, with its margin over the strongest baseline growing from 10 to 45 points as the task horizon increases. These results demonstrate the potential of scaffolding complementary foundation models into long-horizon physical-world agents.
Original Article
View Cached Full Text

Cached at: 09/01/26, 11:51 AM

Paper page - Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation

Source: https://huggingface.co/papers/2608.30396

Abstract

NavMCP integrates vision-language reasoning with navigation execution via structured collaboration channels to enable persistent long-horizon embodied exploration without retraining.

Long-horizon physical-world agents must reason over distant goals while grounding decisions in reliable closed-loop behavior. Today’s foundation models split these capabilities:vision-language models(VLMs) infer missing information and adapt high-level plans but remain brittle and inefficient at repeated navigation grounding, whilenavigation foundation models(NFMs) robustly execute semantic goals but operate as bounded episodes without persistent task-level reasoning. We introduceNavMCP, an agentic scaffolding framework that couples a VLM reasoning agent with an NFM executor for long-horizon exploration. The VLM decides what evidence to seek, where to search, and when to stop, while the NFM grounds each semantic sub-goal intoclosed-loop navigation. Three channels structure their collaboration: intent translates evidence needs into navigation calls, observation converts rollouts into source-grounded trajectory evidence, and memory accumulates findings, negative evidence, and unresolved goals across calls. This design turns isolated navigation rollouts into persistent embodied interaction without retraining either model. OnEmbodied Question Answering,NavMCPachieves state-of-the-art results on HM-EQA, MT-HM3D, and EXPRESS-Bench. Under matched agent and executor backbones, it outperforms an episodic interface by 14.9 percentage points on HM-EQA. On a Unitree Go2,NavMCPreaches 78.3% success, with its margin over the strongest baseline growing from 10 to 45 points as the task horizon increases. These results demonstrate the potential of scaffolding complementary foundation models into long-horizon physical-world agents.

View arXiv pageView PDFAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.30396 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.30396 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.30396 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles