UniNav: A Unified World-Action Diffusion Model for Visual Navigation

# UniNav: A Unified World-Action Diffusion Model for Visual Navigation
Source: [https://arxiv.org/html/2608.03244](https://arxiv.org/html/2608.03244)
###### Abstract

Image\-goal visual navigation is a fundamental capability for embodied agents\. Existing navigation policies efficiently predict waypoint trajectories but lack visual foresight, while navigation world models can anticipate future observations but often require costly planning rollouts\. We presentUniNav, a unified world\-action model that generates future visual observations and continuous waypoint trajectories through a single diffusion process\. Given history frames and a goal image, UniNav jointly denoises visual and waypoint tokens within a single transformer, unifying future prediction and action generation in a shared framework\. To improve spatial grounding, we incorporate geometry\-aware camera tokens\. We also train on both trajectory\-labeled navigation data and video\-only data, enabling the model to benefit from diverse videos without waypoint annotations\. Based on this unified framework, we introduce two variants:UniNav\-Fulljointly predicts interpretable future observations and their corresponding trajectories, whileUniNav\-Fastremoves future\-image tokens at inference for efficient trajectory prediction\. Experiments on navigation benchmarks show that UniNav outperforms the strongest baseline in ATE across all datasets\. With one\-step inference, UniNav\-Fast achieves a latency of0\.10\.1s without a substantial accuracy drop\. Code will be released\.

\\raisebox\{\-0\.43057pt\}\{\\scalebox\{1\.05\}\{\\Letter\}\}\\raisebox\{\-0\.43057pt\}\{\\scalebox\{1\.05\}\{\\Letter\}\}footnotetext:Corresponding author\.## 1Introduction

![Refer to caption](https://arxiv.org/html/2608.03244v1/x1.png)Figure 1:Comparison of visual navigation paradigms\.\(a\) Policy\-only methods directly map the observation history and goal image to a waypoint trajectory, without explicitly modeling future observations\. \(b\) Action\-conditioned navigation world models generate visual rollouts for candidate action sequences and select actions by comparing the imagined outcomes with the goal\. \(c\) Our unified world\-action diffusion model jointly predicts executable waypoint trajectories and their corresponding egocentric visual futures within a single architecture, while supporting efficient action\-only inference when visual prediction is not required\.Visual navigation is a fundamental capability for embodied agents, requiring robots to reach visually goals from egocentric observations while generalizing across environments and embodiments\. Recent general\-purpose navigation policy \(action\) models, such as GNM\(Shahet al\.[2023a](https://arxiv.org/html/2608.03244#bib.bib15)\), ViNT\(Shahet al\.[2023b](https://arxiv.org/html/2608.03244#bib.bib14)\), and NoMaD\(Sridharet al\.[2024](https://arxiv.org/html/2608.03244#bib.bib13)\), learn transferable goal\-reaching behaviors from large and diverse robot trajectory datasets\. These models effectively predict local waypoint trajectories, establishing a promising foundation\-model paradigm for robot visual navigation\. However, they formulate navigation primarily as direct observation\-to\-waypoint prediction\.Human navigation, in contrast, is naturally accompanied by visual foresight: before moving toward a destination, we can anticipate how both the surrounding scene and the target will appear from future viewpoints\.Such anticipation provides an interpretable basis for action selection and enables reasoning about unseen parts of the environment\. Waypoint\-only policies do not explicitly model these future visual states, limiting both their interpretability and their ability to ground trajectory prediction in anticipated visual and geometric changes\. Navigation world models address this limitation by predicting future observations conditioned on candidate motion sequences\(Baret al\.[2025](https://arxiv.org/html/2608.03244#bib.bib19); Zhanget al\.[2026](https://arxiv.org/html/2608.03244#bib.bib17); Kimet al\.[2026a](https://arxiv.org/html/2608.03244#bib.bib16)\)\. NWM\(Baret al\.[2025](https://arxiv.org/html/2608.03244#bib.bib19)\), for example, generates video rollouts for candidate trajectories and evaluates whether the imagined futures reach the goal\. Subsequent works improve rollout quality using dense visual representations\(Zhanget al\.[2026](https://arxiv.org/html/2608.03244#bib.bib17)\)or compact latent representations for efficient planning\(Kimet al\.[2026a](https://arxiv.org/html/2608.03244#bib.bib16)\)\. Although these approaches provide explicit visual foresight, their generate\-then\-evaluate paradigm is computationally expensive because each candidate trajectory requires an independent rollout\. This makes real\-time closed\-loop navigation increasingly difficult as the number of candidate trajectories grows\.

World\-action models offer a more efficient alternative by jointly modeling future world states and robot actions within a single generative process\. Recent robotic manipulation systems demonstrate that unified video\-action representations can effectively couple visual prediction with control generation\(Yeet al\.[2026b](https://arxiv.org/html/2608.03244#bib.bib28); Liet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib35); Zhuet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib36); Biet al\.[2026](https://arxiv.org/html/2608.03244#bib.bib37); Liet al\.[2026b](https://arxiv.org/html/2608.03244#bib.bib47); Paiet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib48); Liaoet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib49); Guoet al\.[2026](https://arxiv.org/html/2608.03244#bib.bib50); Liet al\.[2026e](https://arxiv.org/html/2608.03244#bib.bib51)\)\. Extending this paradigm to visual navigation, however, is challenging due to large\-scale egocentric motion, long\-horizon planning, and the need to jointly model future observations and continuous waypoint trajectories under geometric constraints\.

Recent navigation world\-action models have begun to address this challenge\. UniWM\(Donget al\.[2025](https://arxiv.org/html/2608.03244#bib.bib53)\)interleaves action prediction and future\-view generation in a memory\-augmented autoregressive model, while WAM\-Nav\(Yanget al\.[2026](https://arxiv.org/html/2608.03244#bib.bib52)\)jointly generates trajectories and latent visual foresight with a diffusion transformer\. Nevertheless, existing methods either rely on costly visual rollouts, employ autoregressive architectures that scale poorly, or lack explicit metric geometric supervision\. Furthermore, visual navigation is inherently multimodal, with multiple feasible trajectories leading to distinct future observations\. Diffusion models are well suited to capturing such joint world\-action distributions\. Combined with the strong priors of pretrained video diffusion models, they provide a scalable foundation for jointly predicting future observations and navigation actions\. This motivates a geometry\-aware unified world\-action diffusion model that offers both interpretable visual foresight and efficient real\-time waypoint prediction\.

To this end, we propose UniNav, a unified world\-action diffusion model for predictable image\-goal visual navigation\. Built upon the pretrained Wan2\.1\-1\.3B video diffusion model\(Wanget al\.[2025](https://arxiv.org/html/2608.03244#bib.bib40)\), UniNav jointly predicts future egocentric observations and continuous waypoint trajectories\. Given historical observations and a goal image, future visual latents, waypoint tokens, geometry\-aware camera tokens\(Linet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib45)\), and register tokens\(Darcetet al\.[2024](https://arxiv.org/html/2608.03244#bib.bib54); Zhuet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib36); Oquabet al\.[2023](https://arxiv.org/html/2608.03244#bib.bib56); Wanget al\.[2026](https://arxiv.org/html/2608.03244#bib.bib55)\)are unified as token sequences and optimized under a shared flow\-matching objective\(Lipmanet al\.[2022](https://arxiv.org/html/2608.03244#bib.bib46)\), coupling visual foresight with executable navigation\. To improve spatial grounding, we introduce geometry\-aware camera\-token denoising, where camera tokens extracted by a frozen geometry foundation model\(Linet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib45)\)provide 3D supervision for the shared world\-action representation\. To improve scalability, UniNav jointly leverages trajectory\-labeled navigation data and video\-only data, allowing abundant unlabeled videos to provide additional visual and geometric supervision without requiring waypoint annotations\. Finally, we develop two complementary variants\. UniNav\-Full jointly generates future observations and waypoint trajectories for interpretable visual foresight, while UniNav\-Fast employs asymmetric attention\(Yuanet al\.[2026](https://arxiv.org/html/2608.03244#bib.bib31); Yeet al\.[2026a](https://arxiv.org/html/2608.03244#bib.bib32); Liet al\.[2026a](https://arxiv.org/html/2608.03244#bib.bib72)\)to decouple waypoint prediction from noisy image tokens, enabling efficient inference by discarding image tokens at test time\.

Our contributions are summarized as follows:

- •We propose a unified world\-action diffusion framework that jointly models future egocentric observations and continuous waypoint trajectories using a pretrained video diffusion transformer\.
- •We introduce geometry\-aware camera\-token denoising to transfer 3D structure from a frozen geometry foundation model into the shared world\-action representation\.
- •We incorporate video\-only data, enabling additional visual and geometric supervision without requiring waypoint annotations for every sequence\.
- •We develop two complementary variants: UniNav\-Full for joint visual and trajectory generation, and UniNav\-Fast for efficient waypoint prediction\. Extensive experiments on multiple public visual navigation benchmarks demonstrate consistent performance improvements over existing methods\.

## 2Related Work

### 2\.1Image\-Goal Conditioned Visual Navigation

Learning\-based visual navigation has evolved from dataset\-specific policies toward general\-purpose models trained on diverse multi\-robot trajectories\. GNM\(Shahet al\.[2023a](https://arxiv.org/html/2608.03244#bib.bib15)\)established a shared goal\-conditioned policy across robot platforms and environments, while ViNT\(Shahet al\.[2023b](https://arxiv.org/html/2608.03244#bib.bib14)\)introduced transformer\-based temporal modeling for improved zero\-shot transfer\. NoMaD\(Sridharet al\.[2024](https://arxiv.org/html/2608.03244#bib.bib13)\)further replaced deterministic action prediction with a diffusion policy, improving trajectory generation under multimodal navigation behaviors\. Subsequent work has focused on improving safety, efficiency, and geometric reasoning\. NavDP\(Caiet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib18)\)and NaviDiffusor\(Zenget al\.[2025](https://arxiv.org/html/2608.03244#bib.bib20)\)incorporate privileged information or cost guidance to improve obstacle avoidance\. FlowNav\(Godeet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib21)\)adopts conditional flow matching for faster inference and introduces monocular depth priors, while LoGoPlanner\(Penget al\.[2025](https://arxiv.org/html/2608.03244#bib.bib22)\)integrates state estimation and metric 3D reasoning into a diffusion policy\. Other approaches explicitly construct geometric representations, such as the 3D Gaussian Splatting maps used by BEINGS\(Menget al\.[2025](https://arxiv.org/html/2608.03244#bib.bib23)\)\. These methods primarily optimize action prediction and do not explicitly model the future visual consequences of the predicted trajectory\. In contrast, UniNav retains an efficient waypoint\-prediction path while jointly learning future visual dynamics through world\-action training\.

### 2\.2Navigation World Models

Navigation world models predict future egocentric observations conditioned on candidate actions, enabling planning through imagined rollouts\. NWM\(Baret al\.[2025](https://arxiv.org/html/2608.03244#bib.bib19)\)follows this paradigm by generating future video frames for action sequences and selecting actions through model predictive control\. Although such visual rollouts provide interpretable predictions, evaluating multiple candidate trajectories requires repeated video generation and results in substantial computational cost\(Kimet al\.[2026a](https://arxiv.org/html/2608.03244#bib.bib16)\)\. Recent work improves the efficiency or consistency of this planning paradigm\. MWM\(Yanet al\.[2026](https://arxiv.org/html/2608.03244#bib.bib38)\)separates structure learning from action\-consistency post\-training, while CompACT\(Kimet al\.[2026a](https://arxiv.org/html/2608.03244#bib.bib16)\)compresses observations into a small set of discrete latent tokens for faster planning\. ReL\-NWM\(Zhanget al\.[2025](https://arxiv.org/html/2608.03244#bib.bib27)\)and RAE\-NWM\(Zhanget al\.[2026](https://arxiv.org/html/2608.03244#bib.bib17)\)move world modeling from pixel space to DINOv2 feature space, reducing generation cost while preserving semantic and spatial structure\. NavDreamer\(Huanget al\.[2026](https://arxiv.org/html/2608.03244#bib.bib24)\)instead generates navigation videos and extracts 3D waypoints from the imagined observations\. Despite these advances, action\-conditioned world models generally require separate rollouts for candidate trajectories\. UniNav avoids this bottleneck by directly modeling future observations and waypoint trajectories within a shared generative process, with a fast inference mode that predicts actions without generating future images\.

### 2\.3Unified World\-Action Models

Unified world\-action models jointly learn visual dynamics and robot actions within a shared generative architecture\. UVA\(Liet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib35)\)jointly models latent video and actions with decoupled decoding, while UWM\(Zhuet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib36)\)independently noises video and action tokens to support policy learning, dynamics modeling, and video generation\. Fast\-WAM\(Yuanet al\.[2026](https://arxiv.org/html/2608.03244#bib.bib31)\)and GigaWorld\-Policy\(Yeet al\.[2026a](https://arxiv.org/html/2608.03244#bib.bib32)\)further show that visual prediction during training can improve action generation without requiring video synthesis at deployment\. This paradigm has been explored through diverse backbone and representation designs\. Motus\(Biet al\.[2026](https://arxiv.org/html/2608.03244#bib.bib37)\)combines language, video, and action experts in a mixture\-of\-transformers architecture, while DreamZero\(Yeet al\.[2026b](https://arxiv.org/html/2608.03244#bib.bib28)\), Cosmos Policy\(Kimet al\.[2026b](https://arxiv.org/html/2608.03244#bib.bib29)\), and LingBot\-VA\(Liet al\.[2026c](https://arxiv.org/html/2608.03244#bib.bib30)\)adapt pretrained video\-generation models for robot control\. AIM\(Fanet al\.[2026](https://arxiv.org/html/2608.03244#bib.bib33)\)and Multi\-View VDP\(Liet al\.[2026d](https://arxiv.org/html/2608.03244#bib.bib34)\)further introduce structured representations based on spatial value maps and multi\-view 3D\-aware video latents, respectively\. Recent navigation methods have begun to adopt similar formulations\. UniWM\(Donget al\.[2025](https://arxiv.org/html/2608.03244#bib.bib53)\)interleaves discrete action prediction and future\-view generation in an autoregressive multimodal model with hierarchical memory, whereas WAM\-Nav\(Yanget al\.[2026](https://arxiv.org/html/2608.03244#bib.bib52)\)jointly models long\-horizon actions and short\-horizon latent visual foresight using a diffusion transformer\. In contrast, UniNav adapts a pretrained video diffusion transformer to image\-goal navigation and jointly performs flow matching over continuous future\-video latents, waypoint tokens, and camera\-geometry tokens\. This enables decoded RGB foresight, explicit geometric grounding, and efficient image\-free waypoint inference within a single architecture\.

## 3Method

![Refer to caption](https://arxiv.org/html/2608.03244v1/x2.png)Figure 2:Overview of UniNav training and inference\. During training, visual tokens from a pretrained VAE, camera\-geometry tokens extracted by a frozen geometry model, trajectory tokens, history and goal tokens, and learnable register tokens are jointly processed by the world\-action model\. The model jointly denoises future\-video, camera, and trajectory tokens using the corresponding objectivesℒvid\\mathcal\{L\}\_\{\\mathrm\{vid\}\},ℒcam\\mathcal\{L\}\_\{\\mathrm\{cam\}\}, andℒtraj\\mathcal\{L\}\_\{\\mathrm\{traj\}\}\. At inference, noisy future\-image and trajectory tokens are initialized from random noise and iteratively denoised, conditioned on the history and goal observations, to jointly produce future visual observations and waypoint trajectories\.### 3\.1Overview

We present UniNav, a geometry\-aware world\-action diffusion model for image\-goal visual navigation\. GivenCChistory framesIt−C\+1:t=\{It−C\+1,…,It\}\{I\}\_\{t\-C\+1:t\}=\\\{I\_\{t\-C\+1\},\\ldots,I\_\{t\}\\\}and a goal imageIgI\_\{g\}, UniNav predicts aKK\-step local waypoint trajectory𝐚t\+1:t\+K∈ℝK×3\\mathbf\{a\}\_\{t\+1:t\+K\}\\in\\mathbb\{R\}^\{K\\times 3\}and, optionally, the corresponding future egocentric observationsIt\+1:t\+KI\_\{t\+1:t\+K\}\. Each waypoint𝐚^t\+k=\(Δ​xt\+k,Δ​yt\+k,Δ​ψt\+k\)\\hat\{\\mathbf\{a\}\}\_\{t\+k\}=\(\\Delta x\_\{t\+k\},\\Delta y\_\{t\+k\},\\Delta\\psi\_\{t\+k\}\)specifies the relative translation and yaw change in the robot’s local coordinate frame\.

As illustrated in[Figure˜2](https://arxiv.org/html/2608.03244#S3.F2), UniNav formulates navigation as a joint diffusion process over future visual observations, waypoint trajectories, and camera geometry\. During training, noisy visual, trajectory, and camera representations are jointly denoised under a unified flow\-matching objective\(Lipmanet al\.[2022](https://arxiv.org/html/2608.03244#bib.bib46)\), coupling visual foresight, geometric understanding, and executable action prediction within a single model\. At inference, the model starts from randomly initialized future\-image and trajectories and iteratively refines them while conditioning on the history observations and goal image\. This unified formulation enables UniNav to jointly generate future observations and waypoint trajectories, while also supporting efficient action\-only inference by omitting future\-image tokens\.

### 3\.2Geometry\-Aware Token Representation

To instantiate the joint diffusion process described above, UniNav represents the navigation context and prediction variables as a heterogeneous token sequence\. Specifically, the sequence consists of visual tokens, trajectory tokens, camera\-geometry tokens, history and goal condition tokens, and learnable register tokens\.

#### Visual tokens\.

The history frames, future frames, and goal image are encoded by a frozen video VAE\(Wanget al\.[2025](https://arxiv.org/html/2608.03244#bib.bib40)\)and patchified into spatiotemporal latent tokens\. The history tokens𝐡hist\\mathbf\{h\}\_\{\\mathrm\{hist\}\}and goal tokens𝐡goal\\mathbf\{h\}\_\{\\mathrm\{goal\}\}provide visual context, while the future\-video tokens𝐡pred\\mathbf\{h\}\_\{\\mathrm\{pred\}\}are initialized from noised future latents and progressively denoised by the transformer\.

#### Waypoint tokens\.

We represent theKK\-step local trajectory usingKKcontinuous waypoint tokens, each corresponding to one waypoint in the robot’s local coordinate frame\. During training, each waypoint𝐚k\\mathbf\{a\}\_\{k\}is perturbed according to the flow\-matching noise schedule, producing a noised trajectory state𝐳traj,k\\mathbf\{z\}\_\{\\mathrm\{traj\},k\}\. The noised waypoint is projected into the transformer hidden space using a lightweight MLP:

𝐡traj,k=MLPtraj​\(𝐳traj,k\)\.\\mathbf\{h\}\_\{\\mathrm\{traj\},k\}=\\mathrm\{MLP\}\_\{\\mathrm\{traj\}\}\(\\mathbf\{z\}\_\{\\mathrm\{traj\},k\}\)\.\(1\)The resulting waypoint features are processed by the shared transformer and decoded into trajectory flow vectors\(Lipmanet al\.[2022](https://arxiv.org/html/2608.03244#bib.bib46)\)by a lightweight trajectory head\.

#### Camera tokens\.

Image\-goal navigation requires not only appearance matching but also an understanding of how the egocentric view changes under camera motion\. Camera pose and 3D scene geometry therefore provide important cues for both future\-view prediction and waypoint trajectory prediction\(Teed and Deng[2021](https://arxiv.org/html/2608.03244#bib.bib69); Wanget al\.[2026](https://arxiv.org/html/2608.03244#bib.bib55)\)\. To incorporate such geometric information, we use a frozen geometry foundation modelGξG\_\{\\xi\}\(Linet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib45)\)to extract camera\-aware target tokens for theKKfuture views:

𝐱cam,t\+1:t\+K=Norm​\(Gξ​\(\[𝐨t−C\+1:t,𝐨t\+1:t\+K\]\)t\+1:t\+K\),\\mathbf\{x\}\_\{\\mathrm\{cam\},t\+1:t\+K\}=\\mathrm\{Norm\}\\left\(G\_\{\\xi\}\(\[\\mathbf\{o\}\_\{t\-C\+1:t\},\\mathbf\{o\}\_\{t\+1:t\+K\}\]\)\_\{t\+1:t\+K\}\\right\),\(2\)where𝐱cam,t\+1:t\+K=\{𝐱cam,k\}k=t\+1t\+K\\mathbf\{x\}\_\{\\mathrm\{cam\},t\+1:t\+K\}=\\\{\\mathbf\{x\}\_\{\\mathrm\{cam\},k\}\\\}\_\{k=t\+1\}^\{t\+K\}denotes the normalized camera tokens\. Each token provides a compact geometric target for its corresponding future view\. Following the unified flow\-matching formulation, each clean camera token𝐱cam,k\\mathbf\{x\}\_\{\\mathrm\{cam\},k\}is perturbed with noise at a sampled timestep to obtain the noised camera state𝐳cam,k\\mathbf\{z\}\_\{\\mathrm\{cam\},k\}\. The noised camera state is then projected into the transformer hidden space as𝐡cam,k=MLPc​\(𝐳cam,k\)\\mathbf\{h\}\_\{\\mathrm\{cam\},k\}=\\mathrm\{MLP\}\_\{c\}\(\\mathbf\{z\}\_\{\\mathrm\{cam\},k\}\)\. Denoising these camera tokens alongside the visual and waypoint representations encourages the model to capture metric scene structure\.

#### Unified token sequence\.

As illustrated in[Figure˜2](https://arxiv.org/html/2608.03244#S3.F2), the complete token sequence is organized as

\[𝐡cam​∣𝐡pred∣​𝐡traj​∣𝐡hist∣​𝐡goal∣𝐡reg\]\.\[\\mathbf\{h\}\_\{\\mathrm\{cam\}\}\\mid\\mathbf\{h\}\_\{\\mathrm\{pred\}\}\\mid\\mathbf\{h\}\_\{\\mathrm\{traj\}\}\\mid\\mathbf\{h\}\_\{\\mathrm\{hist\}\}\\mid\\mathbf\{h\}\_\{\\mathrm\{goal\}\}\\mid\\mathbf\{h\}\_\{\\mathrm\{reg\}\}\]\.\(3\)All token types are processed by the same denoising transformer, enabling geometric reasoning, future visual prediction, and waypoint prediction to interact within a shared representation space\. We further introduce a small set of learnable register tokens𝐡reg\\mathbf\{h\}\_\{\\mathrm\{reg\}\}\(Darcetet al\.[2024](https://arxiv.org/html/2608.03244#bib.bib54); Liet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib35); Oquabet al\.[2023](https://arxiv.org/html/2608.03244#bib.bib56)\)to facilitate global information exchange across heterogeneous token types\. Visual tokens preserve the positional encoding of the pretrained video model\(Suet al\.[2024](https://arxiv.org/html/2608.03244#bib.bib71)\), as they correspond to locations on the spatiotemporal latent grid\. In contrast, trajectory, camera, and register tokens have no direct correspondence to image\-grid locations and are therefore assigned identity positional encodings\. This design avoids imposing artificial spatial coordinates on non\-visual tokens while maintaining compatibility with the pretrained transformer\.

### 3\.3Unified Flow\-Matching Objective

We train UniNav with a unified flow\-matching objective\(Lipmanet al\.[2022](https://arxiv.org/html/2608.03244#bib.bib46)\)over three target modalities: future\-video latents, waypoint trajectories, and camera tokens\. For each modalitym∈\{vid,traj,cam\}m\\in\\\{\\mathrm\{vid\},\\mathrm\{traj\},\\mathrm\{cam\}\\\}, let𝐱m\\mathbf\{x\}\_\{m\}denote the clean target,ϵm\\boldsymbol\{\\epsilon\}\_\{m\}Gaussian noise, andσm\\sigma\_\{m\}a sampled noise level\. The noised state and target velocity are defined as

𝐳m=\(1−σm\)​𝐱m\+σm​ϵm,𝐮m=ϵm−𝐱m\.\\mathbf\{z\}\_\{m\}=\(1\-\\sigma\_\{m\}\)\\mathbf\{x\}\_\{m\}\+\\sigma\_\{m\}\\boldsymbol\{\\epsilon\}\_\{m\},\\qquad\\mathbf\{u\}\_\{m\}=\\boldsymbol\{\\epsilon\}\_\{m\}\-\\mathbf\{x\}\_\{m\}\.\(4\)The shared transformer processes all modalities and predicts their velocities through lightweight modality\-specific heads\.

#### Video loss\.

Let𝐱vid\\mathbf\{x\}\_\{\\mathrm\{vid\}\}denote the clean future\-video latents encoded by the frozen video VAE\. We optimize

ℒvid=‖vθ,vid​\(𝐳,σ\)−𝐮vid‖22\.\\mathcal\{L\}\_\{\\mathrm\{vid\}\}=\\left\\\|v\_\{\\theta,\\mathrm\{vid\}\}\(\\mathbf\{z\},\\sigma\)\-\\mathbf\{u\}\_\{\\mathrm\{vid\}\}\\right\\\|\_\{2\}^\{2\}\.\(5\)

#### Trajectory loss\.

For waypoint prediction, the clean target is the local trajectory𝐱traj=𝐚t\+1:t\+K\\mathbf\{x\}\_\{\\mathrm\{traj\}\}=\\mathbf\{a\}\_\{t\+1:t\+K\}\. We supervise the predicted trajectory velocity using a SmoothL1 loss:

ℒtraj=∑k=1K∑d=13SmoothL1​\(vθ,traj,k,d​\(𝐳,σ\),𝐮traj,k,d\),\\mathcal\{L\}\_\{\\mathrm\{traj\}\}=\\sum\_\{k=1\}^\{K\}\\sum\_\{d=1\}^\{3\}\\mathrm\{SmoothL1\}\\left\(v\_\{\\theta,\\mathrm\{traj\},k,d\}\(\\mathbf\{z\},\\sigma\),\\mathbf\{u\}\_\{\\mathrm\{traj\},k,d\}\\right\),\(6\)where𝐮traj\\mathbf\{u\}\_\{\\mathrm\{traj\}\}denotes the target flow velocity of the trajectory tokens\.

#### Camera loss\.

For geometric supervision,𝐱cam=𝐱cam,t\+1:t\+K\\mathbf\{x\}\_\{\\mathrm\{cam\}\}=\\mathbf\{x\}\_\{\\mathrm\{cam\},t\+1:t\+K\}denotes the normalized camera tokens extracted by the frozen geometry foundation modelGξG\_\{\\xi\}\. The camera\-token loss is

ℒcam=‖vθ,cam​\(𝐳,σ\)−𝐮cam‖22\.\\mathcal\{L\}\_\{\\mathrm\{cam\}\}=\\left\\\|v\_\{\\theta,\\mathrm\{cam\}\}\(\\mathbf\{z\},\\sigma\)\-\\mathbf\{u\}\_\{\\mathrm\{cam\}\}\\right\\\|\_\{2\}^\{2\}\.\(7\)This auxiliary objective encourages the shared representation to preserve camera\-aware geometric structure while keepingGξG\_\{\\xi\}frozen\.

#### Total objective\.

The overall training objective is

ℒ=λvid​mvid​ℒvid\+λtraj​mtraj​ℒtraj\+λcam​mcam​ℒcam,\\mathcal\{L\}=\\lambda\_\{\\mathrm\{vid\}\}m\_\{\\mathrm\{vid\}\}\\mathcal\{L\}\_\{\\mathrm\{vid\}\}\+\\lambda\_\{\\mathrm\{traj\}\}m\_\{\\mathrm\{traj\}\}\\mathcal\{L\}\_\{\\mathrm\{traj\}\}\+\\lambda\_\{\\mathrm\{cam\}\}m\_\{\\mathrm\{cam\}\}\\mathcal\{L\}\_\{\\mathrm\{cam\}\},\(8\)wheremvidm\_\{\\mathrm\{vid\}\},mtrajm\_\{\\mathrm\{traj\}\}, andmcamm\_\{\\mathrm\{cam\}\}indicate whether the corresponding supervision is available for each sample\. Our training data consist of both trajectory\-labeled navigation sequences and video\-only sequences without action annotations\. For navigation data, all three objectives are enabled, allowing the model to jointly learn visual dynamics, camera geometry, and executable waypoint prediction\. For video\-only data, we setmtraj=0m\_\{\\mathrm\{traj\}\}=0and optimize only the video and camera\-token objectives\. This mixed training strategy enables UniNav to benefit from large\-scale and diverse video data without requiring additional waypoint annotations or introducing pseudo\-trajectories\.

### 3\.4UniNav\-Full and UniNav\-Fast

We instantiate UniNav as two model variants with different training and inference designs\.

#### UniNav\-Full\.

UniNav\-Full uses the complete world\-action token sequence defined in[Section˜3\.2](https://arxiv.org/html/2608.03244#S3.SS2)to jointly denoise future\-video latents, waypoint trajectories, and camera tokens\. As illustrated in[Figure˜2](https://arxiv.org/html/2608.03244#S3.F2), at each denoising step, the transformer predicts updates for the current noisy tokens conditioned on the history and goal observations\. We iteratively apply this denoising process over multiple steps, ultimately obtaining an executableKK\-step trajectory together with its corresponding visual future\.

#### UniNav\-Fast\.

UniNav\-Fast is trained as a separate, trajectory\-efficient variant\. It uses the complete token sequence during training but applies an asymmetric attention mask\(Yuanet al\.[2026](https://arxiv.org/html/2608.03244#bib.bib31); Yeet al\.[2026a](https://arxiv.org/html/2608.03244#bib.bib32); Liet al\.[2026a](https://arxiv.org/html/2608.03244#bib.bib72)\)that prevents all non\-future tokens from attending to the noisy future\-video tokens\. The future\-video tokens can still attend to the remaining tokens, retaining visual\-foresight supervision without making trajectory denoising dependent on future\-video representations\. At inference time, UniNav\-Fast removes the future\-video tokens and directly samples the waypoint trajectory, avoiding both future\-video denoising and VAE decoding\. This design reduces the sequence length and inference latency while retaining the same flow\-matching formulation for trajectory prediction\.

## 4Experiments

Table 1:Action prediction comparison on four navigation benchmarks\. We report ATE and RPE, where lower values indicate better performance\.MethodRECONSACSoNGO StanfordSCANDAvg\.ATE↓\\downarrowRPE↓\\downarrowATE↓\\downarrowRPE↓\\downarrowATE↓\\downarrowRPE↓\\downarrowATE↓\\downarrowRPE↓\\downarrowATE↓\\downarrowRPE↓\\downarrow\\rowcolorblack\!8Action OnlyViNT\(Shahet al\.[2023b](https://arxiv.org/html/2608.03244#bib.bib14)\)0\.6590\.2150\.6160\.1690\.8540\.1100\.5070\.2860\.6590\.195NoMaD\(Sridharet al\.[2024](https://arxiv.org/html/2608.03244#bib.bib13)\)1\.2200\.2621\.2740\.2861\.1580\.1331\.0250\.3451\.1690\.257NavDP\(Caiet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib18)\)0\.9520\.2590\.7380\.4390\.8400\.2420\.6490\.4130\.7950\.338FlowNav\(Godeet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib21)\)1\.0210\.3511\.0910\.4351\.0580\.1410\.9500\.5271\.0300\.364\\rowcolorblack\!8World ModelNWM\(Baret al\.[2025](https://arxiv.org/html/2608.03244#bib.bib19)\)0\.3260\.1150\.6530\.3250\.8950\.1200\.4970\.2580\.5930\.205UniNav\-Fast \(Ours\)0\.3160\.1220\.4900\.1660\.6870\.1030\.4740\.2500\.4920\.160UniNav\-Full \(Ours\)0\.3140\.1210\.4640\.1560\.7220\.1100\.4630\.2310\.4910\.155

![Refer to caption](https://arxiv.org/html/2608.03244v1/x3.png)Figure 3:Qualitative joint prediction of future observations and waypoint trajectories\. The predicted trajectories are shown alongside the corresponding ground truth\.
### 4\.1Experimental Setup

#### Datasets\.

We evaluate UniNav on four real\-world image\-goal navigation datasets: RECON\(Shahet al\.[2021](https://arxiv.org/html/2608.03244#bib.bib41)\), SaCSoN\(Hiroseet al\.[2023](https://arxiv.org/html/2608.03244#bib.bib42)\), GO Stanford\(Hiroseet al\.[2019](https://arxiv.org/html/2608.03244#bib.bib43)\), and SCAND\(Karnanet al\.[2022](https://arxiv.org/html/2608.03244#bib.bib44)\)\. Each navigation sample consists of a short history of egocentric observations, a goal image, future observations, and local waypoint annotations\. For mixed\-data training, we additionally use video\-only 3D datasets, including ScanNet\(Daiet al\.[2017](https://arxiv.org/html/2608.03244#bib.bib59)\), DL3DV\(Linget al\.[2024](https://arxiv.org/html/2608.03244#bib.bib60)\), CityWalker\(Liuet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib61)\), and LAVN\(Johnsonet al\.[2024](https://arxiv.org/html/2608.03244#bib.bib62)\)\. These sequences supervise future\-video and camera\-token denoising but do not contribute to the action loss\. Unless otherwise specified, we useC=3C=3history frames, predictK=6K=6future frames and waypoints, and resize all frames to256×192256\\times 192\.

#### Metrics\.

For waypoint prediction, we report Absolute Trajectory Error \(ATE\) and Relative Pose Error \(RPE\)\(Sturmet al\.[2012](https://arxiv.org/html/2608.03244#bib.bib64)\), where lower values indicate better performance\. Future\-frame quality is evaluated using PSNR, SSIM\(Wanget al\.[2004](https://arxiv.org/html/2608.03244#bib.bib66)\), and LPIPS\(Zhanget al\.[2018](https://arxiv.org/html/2608.03244#bib.bib67)\)\. For inference efficiency, we report model latency and the number of function evaluations \(NFE\) under the same hardware setting as shown in[Table˜4](https://arxiv.org/html/2608.03244#S4.T4)\.

#### Implementation details\.

We initialize UniNav from the pretrained Wan2\.1\-T2V\-1\.3B video diffusion transformer\(Wanget al\.[2025](https://arxiv.org/html/2608.03244#bib.bib40)\)\. The video VAE remains frozen, while the diffusion transformer and newly introduced modules are fine\-tuned\. We use AdamW\(Loshchilov and Hutter[2017](https://arxiv.org/html/2608.03244#bib.bib65)\)with learning rates of5×10−65\\times 10^\{\-6\}for the pretrained transformer and5×10−55\\times 10^\{\-5\}for the newly introduced modules\. Training is performed on four NVIDIA RTX 4090 GPUs for two epochs, with a batch size of 1 per GPU and gradient accumulation over 4 iterations\. The loss weights are set toλvid=1\\lambda\_\{\\mathrm\{vid\}\}=1,λact=5\\lambda\_\{\\mathrm\{act\}\}=5, andλcam=0\.01\\lambda\_\{\\mathrm\{cam\}\}=0\.01\. Unless otherwise specified, inference uses 2 sampling steps\.

### 4\.2Main Results

We compare UniNav with two groups of methods: action\-only policies, including ViNT\(Shahet al\.[2023b](https://arxiv.org/html/2608.03244#bib.bib14)\), NoMaD\(Sridharet al\.[2024](https://arxiv.org/html/2608.03244#bib.bib13)\), NavDP\(Caiet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib18)\), and FlowNav\(Godeet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib21)\); and the navigation world\-model baseline NWM\(Baret al\.[2025](https://arxiv.org/html/2608.03244#bib.bib19)\)\. We evaluate two variants of our model\.UniNav\-Fullkeeps future\-image tokens and jointly predicts future frames and waypoint actions, whileUniNav\-Fastremoves future\-image tokens and skips future\-frame decoding during inference\.

To ensure a fair comparison with prior navigation methods, all results in the main comparison are obtained usingonly trajectory\-labeled navigation data, without additional video\-only training data\.[Section˜4](https://arxiv.org/html/2608.03244#S4)reports waypoint prediction results on four real\-world navigation benchmarks\. All results of UniNav use the default two\-step sampler\. UniNav achieves the best ATE on all datasets, with reductions of3\.7%3\.7\\%on RECON,24\.7%24\.7\\%on SaCSoN,18\.2%18\.2\\%on GO Stanford, and6\.8%6\.8\\%on SCAND compared with the strongest baseline on each benchmark\. UniNav also delivers competitive RPE across datasets, showing that the improvement in trajectory accuracy is consistent across different evaluation metrics\. These results indicate that jointly modeling future visual dynamics and waypoint actions improves waypoint prediction across diverse real\-world navigation settings\. Moreover, UniNav\-Fast maintains strong accuracy while removing future\-frame decoding, providing a more efficient inference mode for visual navigation\.

### 4\.3Ablation Studies

![Refer to caption](https://arxiv.org/html/2608.03244v1/x4.png)Figure 4:Qualitative future\-frame predictions on video\-only datasets trained without waypoint annotations\.We conduct ablation studies on RECON\(Shahet al\.[2021](https://arxiv.org/html/2608.03244#bib.bib41)\)to isolate the key design choices of UniNav\. In the main paper, we investigate the effects of camera\-token geometry supervision and mixed training, as well as the accuracy\-efficiency trade\-offs of different model configurations\. Additional analyses are provided in the Appendix, including visual prediction performance, alternative world\-action prediction designs, and zero\-shot evaluation on real world robot\-collected data\.

Table 2:Effect of camera\-token geometry supervision on RECON\. Clean camera\-token conditioning is an oracle setting that extracts camera tokens from ground\-truth future frames and is therefore unavailable in practical inference\.Geometry designATE↓\\downarrowRPE↓\\downarrowWithout camera tokens0\.3210\.125Clean camera\-token conditioning0\.2990\.102Camera\-token denoising \(ours\)0\.3140\.121#### Effect of Camera\-Token Geometry Supervision\.

[Table˜2](https://arxiv.org/html/2608.03244#S4.T2)compares three ways of incorporating camera geometry: removing camera tokens, conditioning on clean camera tokens extracted by DA3\(Linet al\.[2025](https://arxiv.org/html/2608.03244#bib.bib45)\), and treating camera tokens as an auxiliary denoising target\. Clean camera\-token conditioning achieves the lowest ATE and RPE, confirming that future\-view geometry provides useful guidance for trajectory prediction\. However, this result represents an oracle setting: the camera tokens are extracted from ground\-truth future frames, which are unavailable during practical inference\. In contrast, our camera\-token denoising design does not require access to future observations at test time and still improves over the model without camera tokens\. These results demonstrate that explicitly learning to denoise camera tokens provides effective geometric supervision while preserving a fully deployable inference pipeline\.

Table 3:Effect of video\-only data and camera\-token supervision on RECON\.Cam\. denoisingDataATE↓\\downarrowRPE↓\\downarrowNoNavigation0\.3210\.125\+ Video\-only0\.3300\.125\+0\.0090\.000YesNavigation0\.3140\.121\+ Video\-only0\.3020\.109\-0\.012\-0\.012
#### Effect of Mixed Navigation and Video\-Only Training\.

[Table˜3](https://arxiv.org/html/2608.03244#S4.T3)examines how the proposed camera\-token geometry supervision affects the use of video\-only training data\. Trajectory\-labeled navigation samples supervise video, trajectory, and camera\-token denoising, whereas video\-only samples provide only video and camera\-token supervision\. Without camera\-token denoising, adding video\-only data increases ATE by0\.0090\.009, suggesting that visual prediction supervision alone does not effectively transfer to waypoint prediction\. In contrast, when camera\-token denoising is enabled, adding video\-only data reduces both ATE and RPE by0\.0120\.012, yielding the best overall performance of0\.302/0\.1090\.302/0\.109\. These results demonstrate that our camera\-token geometry supervision provides an effective geometric connection between visual dynamics and navigation actions, allowing UniNav to benefit from video\-only data\.

Table 4:Accuracy and efficiency profiles under different denoising steps on RECON\. Latency is measured on a single NVIDIA RTX 4090 GPU\.ModelStepsATE↓\\downarrowRPE↓\\downarrowLatency \(s\)↓\\downarrowNFEViNT–0\.6590\.2150\.0151NoMaD101\.2200\.2620\.05110FlowNav101\.0210\.3510\.05810NWM500\.3260\.115∼\\sim90012,000UniNav\-Full10\.3120\.1210\.192120\.3140\.1210\.389250\.3100\.1050\.8105100\.3080\.0991\.81510200\.2990\.0973\.35320500\.3120\.1036\.11250UniNav\-Fast10\.3180\.1230\.109120\.3160\.1220\.212250\.3120\.1200\.5355100\.3000\.1051\.04010200\.3130\.1032\.11020500\.3310\.1075\.36050

#### Model Profile\.

[Table˜4](https://arxiv.org/html/2608.03244#S4.T4)reports the accuracy–efficiency profiles of UniNav under different numbers of denoising steps\. With the default two\-step sampler, UniNav\-Full and UniNav\-Fast achieve similar trajectory accuracy, while UniNav\-Fast reduces latency from0\.3890\.389s to0\.2120\.212s by removing future\-image tokens and video decoding\. With one\-step sampling, UniNav\-Fast further reduces latency to0\.1090\.109s while retaining strong accuracy, providing a practical operating point for online waypoint prediction\. Increasing the number of denoising steps can moderately improve accuracy, but substantially increases latency and does not yield monotonic gains\. Compared with the planning\-based NWM\(Baret al\.[2025](https://arxiv.org/html/2608.03244#bib.bib19)\), both variants require far fewer function evaluations and complete inference within a much shorter time, making UniNav more suitable for efficient navigation deployment\.

### 4\.4Qualitative Results

[Figure˜3](https://arxiv.org/html/2608.03244#S4.F3)shows coherent future observations and waypoint trajectories across diverse navigation environments\.[Figure˜4](https://arxiv.org/html/2608.03244#S4.F4)presents plausible future predictions on video\-only datasets without waypoint annotations\. The Appendix provides additional visualizations for both settings, together with zero\-shot results on our robot\-collected real\-world data\.

## 5Conclusion

We presented UniNav, a geometry\-aware world\-action diffusion model that unifies future observation prediction and waypoint generation for image\-goal visual navigation\. By jointly denoising visual, trajectory, and camera\-geometry tokens, UniNav couples visual foresight with executable action prediction in a single model\. Its Full variant provides interpretable visual predictions, while the Fast variant enables efficient action inference by removing future\-image tokens\. Moreover, mixed training with trajectory\-labeled navigation data and video\-only data allows the model to benefit from heterogeneous visual experience without requiring action annotations for every sequence\.

Limitations and Future Work\.Our current model focuses on short\-horizon local waypoint prediction and does not explicitly address long\-horizon planning\. In addition, our evaluation is primarily conducted in offline settings, and broader closed\-loop deployment remains to be explored\. Future work will investigate integration with long\-horizon planners and extensive closed\-loop evaluation across diverse robots and real\-world environments\.

## References

- A\. Bar, G\. Zhou, D\. Tran, T\. Darrell, and Y\. LeCun \(2025\)Navigation world models\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 15791–15801\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p1.1),[§2\.2](https://arxiv.org/html/2608.03244#S2.SS2.p1.1),[§4](https://arxiv.org/html/2608.03244#S4.10.10.10.18.1),[§4\.2](https://arxiv.org/html/2608.03244#S4.SS2.p1.1),[§4\.3](https://arxiv.org/html/2608.03244#S4.SS3.SSS0.Px3.p1.3)\.
- H\. Bi, H\. Tan, S\. Xie, Z\. Wang, S\. Huang, H\. Liu, R\. Zhao, Y\. Feng, C\. Xiang, Y\. Rong,et al\.\(2026\)Motus: a unified latent action world model\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 35101–35113\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p2.1),[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1)\.
- W\. Cai, J\. Peng, Y\. Yang, Y\. Zhang, M\. Wei, H\. Wang, Y\. Chen, T\. Wang, and J\. Pang \(2025\)Navdp: learning sim\-to\-real navigation diffusion policy with privileged information guidance\.arXiv preprint arXiv:2505\.08712\.Cited by:[§2\.1](https://arxiv.org/html/2608.03244#S2.SS1.p1.1),[§4](https://arxiv.org/html/2608.03244#S4.10.10.10.15.1),[§4\.2](https://arxiv.org/html/2608.03244#S4.SS2.p1.1)\.
- A\. Dai, A\. X\. Chang, M\. Savva, M\. Halber, T\. Funkhouser, and M\. Nießner \(2017\)Scannet: richly\-annotated 3d reconstructions of indoor scenes\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 5828–5839\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px1.p1.3)\.
- T\. Darcet, M\. Oquab, J\. Mairal, and P\. Bojanowski \(2024\)Vision transformers need registers\.InInternational conference on learning representations,Vol\.2024,pp\. 2632–2652\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p4.1),[§3\.2](https://arxiv.org/html/2608.03244#S3.SS2.SSS0.Px4.p1.1)\.
- Y\. Dong, F\. Wu, G\. Chen, Z\. Cheng, Q\. Hu, Y\. Zhou, J\. Sun, J\. He, Q\. Dai, and A\. G\. Hauptmann \(2025\)Unified world models: memory\-augmented planning and foresight for visual navigation\.arXiv preprint arXiv:2510\.08713\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p3.1),[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1)\.
- L\. Fan, Z\. Xu, C\. Cao, W\. Zhang, M\. Yuan, and J\. Chen \(2026\)AIM: intent\-aware unified world action modeling with spatial value maps\.arXiv preprint arXiv:2604\.11135\.Cited by:[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1)\.
- S\. Gode, A\. Nayak, D\. N\. Oliveira, M\. Krawez, C\. Schmid, and W\. Burgard \(2025\)Flownav: combining flow matching and depth priors for efficient navigation\.In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems \(IROS\),pp\. 17762–17768\.Cited by:[§2\.1](https://arxiv.org/html/2608.03244#S2.SS1.p1.1),[§4](https://arxiv.org/html/2608.03244#S4.10.10.10.16.1),[§4\.2](https://arxiv.org/html/2608.03244#S4.SS2.p1.1)\.
- J\. Guo, Q\. Li, P\. Li, Z\. Chen, N\. Sun, Y\. Su, H\. Wang, Y\. Zhang, X\. Li, and H\. Liu \(2026\)Unified 4d world action modeling from video priors with asynchronous denoising\.arXiv preprint arXiv:2604\.26694\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p2.1)\.
- N\. Hirose, D\. Shah, A\. Sridhar, and S\. Levine \(2023\)Sacson: scalable autonomous control for social navigation\.IEEE Robotics and Automation Letters9\(1\),pp\. 49–56\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px1.p1.3)\.
- N\. Hirose, F\. Xia, R\. Martín\-Martín, A\. Sadeghian, and S\. Savarese \(2019\)Deep visual mpc\-policy learning for navigation\.IEEE Robotics and Automation Letters4\(4\),pp\. 3184–3191\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px1.p1.3)\.
- X\. Huang, W\. Gai, T\. Wu, C\. Wang, Z\. Liu, X\. Zhou, Y\. Wu, and F\. Gao \(2026\)NavDreamer: video models as zero\-shot 3d navigators\.arXiv preprint arXiv:2602\.09765\.Cited by:[§2\.2](https://arxiv.org/html/2608.03244#S2.SS2.p1.1)\.
- F\. Johnson, B\. B\. Cao, K\. Dana, S\. Jain, and A\. Ashok \(2024\)A landmark\-aware visual navigation dataset\.arXiv preprint arXiv:2402\.14281\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px1.p1.3)\.
- H\. Karnan, A\. Nair, X\. Xiao, G\. Warnell, S\. Pirk, A\. Toshev, J\. Hart, J\. Biswas, and P\. Stone \(2022\)Socially compliant navigation dataset \(scand\): a large\-scale dataset of demonstrations for social navigation\.IEEE Robotics and Automation Letters7\(4\),pp\. 11807–11814\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px1.p1.3)\.
- D\. Kim, G\. Seo, J\. Lee, M\. Cho, and S\. Kwak \(2026a\)Planning in 8 tokens: a compact discrete tokenizer for latent world model\.arXiv preprint arXiv:2603\.05438\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p1.1),[§2\.2](https://arxiv.org/html/2608.03244#S2.SS2.p1.1)\.
- M\. J\. Kim, Y\. Gao, T\. Lin, Y\. Lin, Y\. Ge, G\. Lam, P\. Liang, S\. Song, M\. Liu, C\. Finn,et al\.\(2026b\)Cosmos policy: fine\-tuning video models for visuomotor control and planning\.arXiv preprint arXiv:2601\.16163\.Cited by:[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1)\.
- J\. Li, Z\. Liu, D\. Hu, J\. Wu, Z\. Ma, W\. Wu, C\. Han, Z\. Hao, Z\. Liu, K\. Zhan,et al\.\(2026a\)Metis: a generalizable and efficient world\-action model for autonomous driving and urban navigation\.arXiv preprint arXiv:2606\.15869\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p4.1),[§3\.4](https://arxiv.org/html/2608.03244#S3.SS4.SSS0.Px2.p1.1)\.
- L\. Li, Q\. Zhang, Y\. Luo, S\. Yang, R\. Wang, F\. Han, M\. Yu, Z\. Gao, N\. Xue, X\. Zhu,et al\.\(2026b\)Causal world modeling for robot control\.arXiv preprint arXiv:2601\.21998\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p2.1)\.
- L\. Li, Q\. Zhang, Y\. Luo, S\. Yang, R\. Wang, F\. Han, M\. Yu, Z\. Gao, N\. Xue, X\. Zhu,et al\.\(2026c\)Causal world modeling for robot control\.arXiv preprint arXiv:2601\.21998\.Cited by:[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1)\.
- P\. Li, Y\. Chen, Y\. Xu, J\. Yang, X\. Wu, J\. Guo, N\. Sun, L\. Qian, X\. Li, X\. Xiao,et al\.\(2026d\)Multi\-view video diffusion policy: a 3d spatio\-temporal\-aware video action model\.arXiv preprint arXiv:2604\.03181\.Cited by:[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1)\.
- S\. Li, Y\. Gao, D\. Sadigh, and S\. Song \(2025\)Unified video action model\.arXiv preprint arXiv:2503\.00200\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p2.1),[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1),[§3\.2](https://arxiv.org/html/2608.03244#S3.SS2.SSS0.Px4.p1.1)\.
- Z\. Li, D\. Cheng, Y\. Wang, S\. Wang, X\. Xu, L\. Weng, J\. Wang, and J\. Wang \(2026e\)Light\-wam: efficient world action models with state\-fusion action decoding\.External Links:2606\.08242,[Link](https://arxiv.org/abs/2606.08242)Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p2.1)\.
- Y\. Liao, P\. Zhou, S\. Huang, D\. Yang, S\. Chen, Y\. Jiang, Y\. Hu, J\. Cai, S\. Liu, J\. Luo,et al\.\(2025\)Genie envisioner: a unified world foundation platform for robotic manipulation\.arXiv preprint arXiv:2508\.05635\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p2.1)\.
- H\. Lin, S\. Chen, J\. Liew, D\. Y\. Chen, Z\. Li, G\. Shi, J\. Feng, and B\. Kang \(2025\)Depth anything 3: recovering the visual space from any views\.arXiv preprint arXiv:2511\.10647\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p4.1),[§3\.2](https://arxiv.org/html/2608.03244#S3.SS2.SSS0.Px3.p1.2),[§4\.3](https://arxiv.org/html/2608.03244#S4.SS3.SSS0.Px1.p1.1)\.
- L\. Ling, Y\. Sheng, Z\. Tu, W\. Zhao, C\. Xin, K\. Wan, L\. Yu, Q\. Guo, Z\. Yu, Y\. Lu,et al\.\(2024\)Dl3dv\-10k: a large\-scale scene dataset for deep learning\-based 3d vision\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 22160–22169\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px1.p1.3)\.
- Y\. Lipman, R\. T\. Chen, H\. Ben\-Hamu, M\. Nickel, and M\. Le \(2022\)Flow matching for generative modeling\.arXiv preprint arXiv:2210\.02747\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p4.1),[§3\.1](https://arxiv.org/html/2608.03244#S3.SS1.p2.1),[§3\.2](https://arxiv.org/html/2608.03244#S3.SS2.SSS0.Px2.p1.5),[§3\.3](https://arxiv.org/html/2608.03244#S3.SS3.p1.4)\.
- X\. Liu, J\. Li, Y\. Jiang, N\. Sujay, Z\. Yang, J\. Zhang, J\. Abanes, J\. Zhang, and C\. Feng \(2025\)Citywalker: learning embodied urban navigation from web\-scale videos\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 6875–6885\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px1.p1.3)\.
- I\. Loshchilov and F\. Hutter \(2017\)Decoupled weight decay regularization\.arXiv preprint arXiv:1711\.05101\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px3.p1.5)\.
- W\. Meng, T\. Wu, H\. Yin, and F\. Zhang \(2025\)Beings: bayesian embodied image\-goal navigation with gaussian splatting\.In2025 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 5252–5258\.Cited by:[§2\.1](https://arxiv.org/html/2608.03244#S2.SS1.p1.1)\.
- M\. Oquab, T\. Darcet, T\. Moutakanni, H\. Vo, M\. Szafraniec, V\. Khalidov, P\. Fernandez, D\. Haziza, F\. Massa, A\. El\-Nouby,et al\.\(2023\)Dinov2: learning robust visual features without supervision\.arXiv preprint arXiv:2304\.07193\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p4.1),[§3\.2](https://arxiv.org/html/2608.03244#S3.SS2.SSS0.Px4.p1.1)\.
- J\. Pai, L\. Achenbach, V\. Montesinos, B\. Forrai, O\. Mees, and E\. Nava \(2025\)Mimic\-video: video\-action models for generalizable robot control beyond vlas\.arXiv preprint arXiv:2512\.15692\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p2.1)\.
- J\. Peng, W\. Cai, Y\. Yang, T\. Wang, Y\. Shen, and J\. Pang \(2025\)Logoplanner: localization grounded navigation policy with metric\-aware visual geometry\.arXiv preprint arXiv:2512\.19629\.Cited by:[§2\.1](https://arxiv.org/html/2608.03244#S2.SS1.p1.1)\.
- D\. Shah, B\. Eysenbach, G\. Kahn, N\. Rhinehart, and S\. Levine \(2021\)Rapid exploration for open\-world navigation with latent goal models\.arXiv preprint arXiv:2104\.05859\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px1.p1.3),[§4\.3](https://arxiv.org/html/2608.03244#S4.SS3.p1.1)\.
- D\. Shah, A\. Sridhar, A\. Bhorkar, N\. Hirose, and S\. Levine \(2023a\)Gnm: a general navigation model to drive any robot\.In2023 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 7226–7233\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p1.1),[§2\.1](https://arxiv.org/html/2608.03244#S2.SS1.p1.1)\.
- D\. Shah, A\. Sridhar, N\. Dashora, K\. Stachowicz, K\. Black, N\. Hirose, and S\. Levine \(2023b\)ViNT: a foundation model for visual navigation\.arXiv preprint arXiv:2306\.14846\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p1.1),[§2\.1](https://arxiv.org/html/2608.03244#S2.SS1.p1.1),[§4](https://arxiv.org/html/2608.03244#S4.10.10.10.13.1),[§4\.2](https://arxiv.org/html/2608.03244#S4.SS2.p1.1)\.
- A\. Sridhar, D\. Shah, C\. Glossop, and S\. Levine \(2024\)Nomad: goal masked diffusion policies for navigation and exploration\.In2024 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 63–70\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p1.1),[§2\.1](https://arxiv.org/html/2608.03244#S2.SS1.p1.1),[§4](https://arxiv.org/html/2608.03244#S4.10.10.10.14.1),[§4\.2](https://arxiv.org/html/2608.03244#S4.SS2.p1.1)\.
- J\. Sturm, W\. Burgard, and D\. Cremers \(2012\)Evaluating egomotion and structure\-from\-motion approaches using the tum rgb\-d benchmark\.InProc\. of the Workshop on Color\-Depth Camera Fusion in Robotics at the IEEE/RJS International Conference on Intelligent Robot Systems \(IROS\),Vol\.13,pp\. 6\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px2.p1.1)\.
- J\. Su, M\. Ahmed, Y\. Lu, S\. Pan, W\. Bo, and Y\. Liu \(2024\)Roformer: enhanced transformer with rotary position embedding\.Neurocomputing568,pp\. 127063\.Cited by:[§3\.2](https://arxiv.org/html/2608.03244#S3.SS2.SSS0.Px4.p1.1)\.
- Z\. Teed and J\. Deng \(2021\)Droid\-slam: deep visual slam for monocular, stereo, and rgb\-d cameras\.Advances in neural information processing systems34,pp\. 16558–16569\.Cited by:[§3\.2](https://arxiv.org/html/2608.03244#S3.SS2.SSS0.Px3.p1.2)\.
- A\. Wang, B\. Ai, B\. Wen, C\. Mao, C\. Xie, D\. Chen, F\. Yu, H\. Zhao, J\. Yang, J\. Zeng,et al\.\(2025\)Wan: open and advanced large\-scale video generative models\.arXiv preprint arXiv:2503\.203143\(4\),pp\. 6\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p4.1),[§3\.2](https://arxiv.org/html/2608.03244#S3.SS2.SSS0.Px1.p1.3),[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px3.p1.5)\.
- J\. Wang, M\. Chen, S\. Zhang, N\. Karaev, J\. Schönberger, P\. Labatut, P\. Bojanowski, D\. Novotny, A\. Vedaldi, and C\. Rupprecht \(2026\)VGGT\-Ω\\Omega\.arXiv preprint arXiv:2605\.15195\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p4.1),[§3\.2](https://arxiv.org/html/2608.03244#S3.SS2.SSS0.Px3.p1.2)\.
- Z\. Wang, A\. C\. Bovik, H\. R\. Sheikh, and E\. P\. Simoncelli \(2004\)Image quality assessment: from error visibility to structural similarity\.IEEE transactions on image processing13\(4\),pp\. 600–612\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px2.p1.1)\.
- H\. Yan, Z\. Xiang, Z\. Zhang, and H\. Tang \(2026\)MWM: mobile world models for action\-conditioned consistent prediction\.arXiv preprint arXiv:2603\.07799\.Cited by:[§2\.2](https://arxiv.org/html/2608.03244#S2.SS2.p1.1)\.
- N\. Yang, Y\. Huang, K\. Peng, Z\. He, K\. Wang, C\. Miao, K\. Lyu, G\. Li, X\. Wang, Z\. Zhu,et al\.\(2026\)WAM\-nav: asymmetric latent world\-action modeling for unified visual navigation\.arXiv preprint arXiv:2606\.04907\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p3.1),[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1)\.
- A\. Ye, B\. Wang, C\. Ni, G\. Huang, G\. Zhao, H\. Li, H\. Li, J\. Li, J\. Lv, J\. Liu,et al\.\(2026a\)GigaWorld\-policy: an efficient action\-centered world–action model\.arXiv preprint arXiv:2603\.17240\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p4.1),[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1),[§3\.4](https://arxiv.org/html/2608.03244#S3.SS4.SSS0.Px2.p1.1)\.
- S\. Ye, Y\. Ge, K\. Zheng, S\. Gao, S\. Yu, G\. Kurian, S\. Indupuru, Y\. L\. Tan, C\. Zhu, J\. Xiang,et al\.\(2026b\)World action models are zero\-shot policies\.arXiv preprint arXiv:2602\.15922\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p2.1),[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1)\.
- T\. Yuan, Z\. Dong, Y\. Liu, and H\. Zhao \(2026\)Fast\-wam: do world action models need test\-time future imagination?\.arXiv preprint arXiv:2603\.16666\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p4.1),[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1),[§3\.4](https://arxiv.org/html/2608.03244#S3.SS4.SSS0.Px2.p1.1)\.
- Y\. Zeng, H\. Ren, S\. Wang, J\. Huang, and H\. Cheng \(2025\)Navidiffusor: cost\-guided diffusion model for visual navigation\.In2025 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 11994–12001\.Cited by:[§2\.1](https://arxiv.org/html/2608.03244#S2.SS1.p1.1)\.
- M\. Zhang, W\. Shen, F\. Zhang, H\. Qin, Z\. Pei, and Z\. Meng \(2026\)RAE\-nwm: navigation world model in dense visual representation space\.arXiv preprint arXiv:2603\.09241\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p1.1),[§2\.2](https://arxiv.org/html/2608.03244#S2.SS2.p1.1)\.
- R\. Zhang, P\. Isola, A\. A\. Efros, E\. Shechtman, and O\. Wang \(2018\)The unreasonable effectiveness of deep features as a perceptual metric\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 586–595\.Cited by:[§4\.1](https://arxiv.org/html/2608.03244#S4.SS1.SSS0.Px2.p1.1)\.
- Z\. Zhang, H\. Zhang, K\. Huang, C\. Shi, and H\. Lu \(2025\)Efficient image\-goal navigation with representative latent world model\.arXiv preprint arXiv:2511\.11011\.Cited by:[§2\.2](https://arxiv.org/html/2608.03244#S2.SS2.p1.1)\.
- C\. Zhu, R\. Yu, S\. Feng, B\. Burchfiel, P\. Shah, and A\. Gupta \(2025\)Unified world models: coupling video and action diffusion for pretraining on large robotic datasets\.arXiv preprint arXiv:2504\.02792\.Cited by:[§1](https://arxiv.org/html/2608.03244#S1.p2.1),[§1](https://arxiv.org/html/2608.03244#S1.p4.1),[§2\.3](https://arxiv.org/html/2608.03244#S2.SS3.p1.1)\.