Valerant: An Automatic Navigable Game Map Generator via Action-Conditioned World Model Exploration

# Valerant: An Automatic NaVigAbLE Game Map GeneratoR via Action-CoNditioned World Model ExploraTion
Source: [https://arxiv.org/html/2609.09418](https://arxiv.org/html/2609.09418)
Feng WangAffiliation:Johns Hopkins University\{yxq350,jxm1384\}@case\.edu,wangf3014@gmail\.comJing Ma††thanks:Corresponding author\.Affiliation:Case Western Reserve University

###### Abstract

World Action Models \(WAMs\) couple predictive world modeling with action generation, allowing anticipated future states to guide agent behavior\. Although WAMs are rapidly advancing embodied AI, general\-purpose counterparts remain largely unexplored in games\. Existing game\-oriented approaches often combine action\-conditioned world models with external policies and reward functions to realize WAM\-like decision\-making, yet they operate mainly in 2D visual observation space and do not instantiate persistent 3D geometry\. Extending this paradigm to 3D games introduces a distinct challenge\. In autonomous driving and robotics, the physical environment exists independently of the model, providing a persistent 3D world in which selected actions can be executed\. Games have no such external substrate; the virtual world itself must be instantiated\. Most playable games require a persistent and navigable space, while 3D games additionally require explicit geometry that supports movement and interaction\. Action\-conditioned video rollouts provide visual observations but not this spatial representation\. We presentValerant, a training\-free framework that transforms a pretrained action\-conditioned world model into a WAM for exploring and constructing 3D game maps\. By coupling predictive visual rollouts with SLAM\-based spatial reconstruction and exploration\-driven action selection,Valerantprogressively transforms a single image into a persistent 3D game map\. This framework extends WAM\-based interaction beyond 2D visual simulation and offers a new approach to reducing manual effort in 3D game\-map creation\.

## 1Introduction

Rapid advances in large\-scale generative modeling have established content synthesis as one of the most active areas of modern artificial intelligence, giving rise to the broad paradigm of AI\-generated content \(AIGC\)\([Peebles and Xie, 2023](https://arxiv.org/html/2609.09418#bib.bib1);[Podell et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib2);[Wu et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib3);[Wan et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib18)\)\. Modern generative models can synthesize high\-quality images, videos, and multimodal sequences under increasingly flexible forms of conditioning\. Among their dominant technical foundations, diffusion models generate data by learning to reverse a progressive corruption process\([Ho et al\., 2020](https://arxiv.org/html/2609.09418#bib.bib4);[Rombach et al\., 2022](https://arxiv.org/html/2609.09418#bib.bib5)\), whereas flow\-matching models learn a continuous velocity field that transports samples from a simple prior distribution to the data distribution\([Lipman et al\., 2022](https://arxiv.org/html/2609.09418#bib.bib6);[Liu et al\., 2022](https://arxiv.org/html/2609.09418#bib.bib7)\)\. Building on increasingly coherent video generation, recent work has begun to formulate video generators as observation\-space world models \(WMs\)\([Brooks et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib8);[Wang et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib9);[Alhaija et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib10);[Qin et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib11)\)\. By predicting temporally coherent visual trajectories, these models capture regularities in object motion, scene evolution, and the observable outcomes of physical interactions, thereby approximating how a world evolves without requiring an explicit representation of its underlying 3D state\. When actions are introduced as conditioning signals, the resulting action\-conditioned world models predict future observations under a prescribed action or action sequence\([Yang et al\., 2023](https://arxiv.org/html/2609.09418#bib.bib12);[Valevski et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib13);[Che et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib14);[Bar et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib15);[Parker\-Holder et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib16);[Ball et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib17)\)\. Crucially, predicting the consequence of a given action does not by itself determine which action should be taken\. World Action Models \(WAMs\) take a further step by coupling predictive world modeling with action generation or selection\([Wang et al\., 2026a](https://arxiv.org/html/2609.09418#bib.bib53);[Zhang et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib54)\)\. This coupling can be realized through a cascaded pipeline that derives actions from predicted futures or through a unified model that jointly models future states and actions\([Du et al\., 2023](https://arxiv.org/html/2609.09418#bib.bib55);[Zhao et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib56)\)\. Rather than merely simulating what may happen under an externally specified action, WAMs use anticipated world evolution to inform what an agent should do\. Taken together, this progression extends generative modeling from passive content synthesis, through controllable prediction, to predictive decision\-making in dynamic environments\.

The practical value of this shift is particularly evident in autonomous driving and robotics, where WAMs use predicted world evolution to guide motion planning and policy generation\([Xia et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib19);[Zhao et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib56);[Kim et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib37)\)\. The motivation is straightforward: collecting closed\-loop experience and evaluating policies in the physical world are expensive, time\-consuming, and difficult to scale, while failed trials may damage hardware or create unacceptable safety risks\([Li et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib21);[Quevedo et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib22)\)\. By coupling prospective world prediction with action generation, WAMs allow possible behaviors and their consequences to inform decisions before costly physical deployment\. Game creation presents a fundamentally different setting\. General\-purpose WAMs remain largely unexplored in games, particularly for applications that require the underlying 3D world itself to be instantiated\. Existing approaches instead follow two related routes: model\-based agents combine learned WMs with external policies or reward functions to optimize behavior, while interactive game WMs generate action\-conditioned visual observations from user inputs\([Alonso et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib23);[Hafner et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib32);[Valevski et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib13);[Li et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib24);[Wang et al\., 2026b](https://arxiv.org/html/2609.09418#bib.bib25)\)\. Although these systems can realize WAM\-like interaction or decision\-making at the pipeline level, their learned environments remain represented by 2D frames or latent observation sequences rather than persistent, explicit 3D geometry\. Comparable modular WM–policy systems are also used in autonomous driving and robotics\([Wang et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib9);[Bar et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib15);[Gao et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib20)\)\. Importantly, however, the worlds modeled in these physical domains already exist independently of the learned system\. A vehicle or robot operates in a persistent physical 3D environment, which provides the spatial substrate against which model predictions can be grounded and from which new observations can be obtained after execution\. In game creation, by contrast, the world being modeled is also the artifact that must be generated\. There is no independently existing physical environment that can provide a persistent 3D scene behind the synthesized observations\. Most first\-person shooters require a persistent, navigable 3D space in which camera motion, collision, visibility, and object interactions can be resolved consistently over time\. Observation\-space WMs and existing WM–policy systems therefore cannot directly provide the explicit 3D maps required for constructing such games\. Bridging WAM\-based exploration with explicit 3D map construction is thus necessary to extend these models from interactive visual simulation to practical game creation\.

To address this limitation, we introduceValerant, an automatic game\-map generator that transforms a single input image into a navigable 3D environment\. Without additional training,Valerantconverts a pretrained action\-conditioned WM into a WAM by coupling its predictive rollouts with exploration\-driven action selection\.Valerantinstantiates a virtual agent that queries the WM with a finite set of candidate actions at each exploration step, producing counterfactual video rollouts from a common pre\-action state\. Each rollout is processed by visual SLAM\([Teed and Deng, 2021](https://arxiv.org/html/2609.09418#bib.bib26);[Matsuki et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib27);[Murai et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib28)\)to estimate the camera trajectory and reconstruct the observed 3D structure\. An exploration policy scores the resulting candidates according to their mapping utility, after which the agent returns to the pre\-action state and executes only the highest\-scoring action\. The selected observations are integrated into a persistent map, and this process is repeated until the exploration budget is exhausted, yielding a consolidated 3D point\-cloud game map\. Our contributions are threefold:

- •We introduceValerant, a unified training\-free framework that converts a pretrained action\-conditioned WM into a WAM and couples its exploration with SLAM\-based reconstruction for automatic game\-map generation\.
- •We develop a counterfactual exploration procedure in which alternative WM rollouts guide action selection and are converted into a persistent 3D environment, bridging the gap between visual world simulation and the explicit spatial instantiation required by games\.
- •We establish a new direction for game\-map authoring in which a single visual concept can be expanded into navigable 3D geometry, thereby reducing game developers’ reliance on labor\-intensive and cumbersome map\-construction workflows\.

## 2Related Work

![Refer to caption](https://arxiv.org/html/2609.09418v1/Figure1_2.png)Figure 1:Overview ofValerant\. Starting from a single input image, a virtual agent queries the WM with candidate actions to generate alternative what\-if video rollouts\. Visual SLAM reconstructs the corresponding 3D geometry, and an exploration policy selects the branch with the highest mapping utility\. The selected observations are committed to a persistent point\-cloud map, which is iteratively expanded into the final 3D game map\. The upper panel provides a schematic illustration of the pipeline, while the lower panel demonstrates the process using an example scene\.### 2\.1World Model and World Action Model

World models learn predictive representations of environment dynamics to support planning and control\. Early approaches compressed high\-dimensional observations into latent states and learned action\-conditioned transitions within these compact spaces\([Ha and Schmidhuber, 2018](https://arxiv.org/html/2609.09418#bib.bib29);[Hafner et al\., 2019b](https://arxiv.org/html/2609.09418#bib.bib30);[Hafner et al\., 2019a](https://arxiv.org/html/2609.09418#bib.bib31)\)\. Subsequent methods improved the scalability, robustness, and generality of latent world modeling, enabling agents to learn across increasingly diverse control tasks through imagined trajectories\([Hansen et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib44);[Hafner et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib32)\)\. These models are primarily optimized to capture task\-relevant dynamics for decision\-making, rather than to synthesize photorealistic and spatially persistent environments\. Advances in video generation have shifted world modeling toward the direct prediction of visual futures\. iVideoGPT\([Wu et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib33)\)represents observations, actions, and rewards as interleaved tokens for scalable interactive prediction, while AVID\([Rigter et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib34)\)adapts pretrained video diffusion models into action\-conditioned simulators using lightweight domain\-specific adapters\. In robotics, IRASim\([Zhu et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib35)\)generates fine\-grained robot–object interactions conditioned on action trajectories\. These systems are action\-conditioned WMs: actions are supplied as conditions, and the models predict their observational consequences rather than generating the actions themselves\. Following recent formalizations\([Wang et al\., 2026a](https://arxiv.org/html/2609.09418#bib.bib53);[Zhang et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib54)\), we reserve the term World Action Model \(WAM\) for systems that couple forward world prediction with action generation or selection\. This coupling can be implemented through cascaded architectures that derive actions from predicted futures or joint architectures that model future states and actions within a unified framework\([Du et al\., 2023](https://arxiv.org/html/2609.09418#bib.bib55);[Zhao et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib56);[Bi et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib36);[Kim et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib37)\)\. Games provide a natural testbed for these models because both observations and actions can be collected at scale\. WHAM\([Kanervisto et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib38)\)jointly models gameplay visuals and controller actions, representing a game\-specific joint WAM for gameplay ideation\. By contrast, Oasis\([Decart et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib39)\), MineWorld\([Guo et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib40)\), and Matrix\-Game\([Wang et al\., 2026b](https://arxiv.org/html/2609.09418#bib.bib25)\)are action\-conditioned WMs that generate future gameplay observations from user controls\. Recent work has further extended action\-conditioned generation to environments containing reactive agents and complex multi\-agent behavior\([Agarwal et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib41)\)\. Despite their increasing visual fidelity and controllability, these methods primarily realize their generated worlds as autoregressive streams of visual observations\. Geometry\-aware WMs have begun to address spatial forgetting by introducing persistent point\-cloud memory or latent 3D scene states\([Wu et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib42);[Garcin et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib43)\)\. Their primary objective, however, is to improve the spatial consistency and long\-horizon stability of visual generation rather than to construct a reusable game map\. In contrast,Valerantstarts from a pretrained action\-conditioned WM and, without additional training, couples its counterfactual rollouts to an exploration policy, thereby forming a cascaded WAM whose selected observations are reconstructed into an explicit 3D point\-cloud game map\.

### 2\.2Visual SLAM and Online Mapping

Simultaneous localization and mapping \(SLAM\) jointly estimates an agent’s trajectory and incrementally constructs a spatial representation of its surroundings from sequential observations\. Classical visual SLAM systems rely on geometric correspondences, bundle adjustment, and loop closure to maintain globally consistent camera poses and maps, as exemplified by ORB\-SLAM3\([Campos et al\., 2021](https://arxiv.org/html/2609.09418#bib.bib45)\)\. Learning\-based approaches have since incorporated learned components into this pipeline: DROID\-SLAM\([Teed and Deng, 2021](https://arxiv.org/html/2609.09418#bib.bib26)\)performs recurrent optimization over camera poses and dense depth, while recent systems such as MASt3R\-SLAM\([Murai et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib28)\), SLAM3R\([Liu et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib46)\), and VGGT\-SLAM\([Maggio et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib47)\)exploit learned 3D reconstruction priors to recover dense geometry from monocular image streams\. SLAM has also been tightly coupled with embodied exploration, where an agent repeatedly selects an action, moves through its environment, acquires new observations, and updates its map online\([Chaplot et al\., 2020](https://arxiv.org/html/2609.09418#bib.bib48)\)\. Consequently, both robotic SLAM andValerantfollow an incremental mapping process in which exploration continually reveals new parts of the environment\. Their central distinction lies instead in the source of these observations\. Conventional embodied systems obtain sensor measurements by moving through an existing real or simulated 3D environment\. InValerant, no such environment is initially available\. A frozen action\-conditioned video WM generates the predicted visual consequences of externally specified candidate actions, while an exploration policy evaluates the resulting what\-if trajectories and selects the action to execute\. SLAM then estimates the corresponding camera motion and integrates the selected observations, progressively transforming transient WM rollouts into a persistent and navigable 3D point\-cloud map\.

### 2\.3Counterfactual Reasoning and What\-If Rollouts

Counterfactual reasoning asks how the future would change under an alternative action while the preceding context remains fixed\([Rubin, 1974](https://arxiv.org/html/2609.09418#bib.bib49);[Holland, 1986](https://arxiv.org/html/2609.09418#bib.bib50);[Pearl and others, 2003](https://arxiv.org/html/2609.09418#bib.bib51)\)\. In the physical world, however, only the outcome of the executed action can be observed; the mutually exclusive futures associated with unchosen actions remain inaccessible\. Learned world models provide a practical means of exploring these alternatives by simulating candidate actions before execution, an idea widely used in model\-based planning\([Hansen et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib44)\)\. Action\-conditioned visual WMs further render these alternatives as predicted video trajectories, supporting the comparison of possible maneuvers in embodied navigation and autonomous driving\([Yu et al\., 2022](https://arxiv.org/html/2609.09418#bib.bib52);[Wang et al\., 2024](https://arxiv.org/html/2609.09418#bib.bib9);[Bar et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib15)\)\. Such WMs are naturally suited to counterfactual\-style reasoning because they can preserve a common observation history while varying the candidate action\. This predictive capability becomes part of a WAM when the resulting futures are coupled to action generation or selection\.Valerantrealizes this coupling without additional training: it evaluates alternative WM rollouts according to their mapping utility and selects the action that most effectively reveals unobserved 3D space\.

## 3Method

### 3\.1Preliminaries

##### World Models\.

A world model \(WM\) approximates the transition dynamics of an environment by predicting how its observations evolve under externally specified actions\([Wang et al\., 2026a](https://arxiv.org/html/2609.09418#bib.bib53)\)\. Let𝐨t\\mathbf\{o\}\_\{t\}denote the visual observation at environment steptt\. We define the visual context as𝐎tc=\(𝐨t−K\+1,…,𝐨t\)\\mathbf\{O\}^\{\\mathrm\{c\}\}\_\{t\}=\(\\mathbf\{o\}\_\{t\-K\+1\},\\ldots,\\mathbf\{o\}\_\{t\}\), a candidate action sequence as𝐀t=\(𝐚t,…,𝐚t\+H−1\)\\mathbf\{A\}\_\{t\}=\(\\mathbf\{a\}\_\{t\},\\ldots,\\mathbf\{a\}\_\{t\+H\-1\}\), and the corresponding future observations as𝐎tf=\(𝐨t\+1,…,𝐨t\+H\)\\mathbf\{O\}^\{\\mathrm\{f\}\}\_\{t\}=\(\\mathbf\{o\}\_\{t\+1\},\\ldots,\\mathbf\{o\}\_\{t\+H\}\), whereKKandHHdenote the context length and prediction horizon, respectively\. An observation\-space WM parameterized byθ\\thetagenerates a possible future rollout according to

𝐎tf∼pθ\(⋅∣𝐎tc,𝐀t\),\{\\mathbf\{O\}\}^\{\\mathrm\{f\}\}\_\{t\}\\sim p\_\{\\theta\}\\\!\\left\(\\,\\cdot\\,\\mid\\mathbf\{O\}^\{\\mathrm\{c\}\}\_\{t\},\\mathbf\{A\}\_\{t\}\\right\),\(1\)wherepθp\_\{\\theta\}denotes the conditional distribution over future visual observations learned by the WM\. Modeling a distribution allows the WM to represent multiple plausible futures that may follow the same observation history and action sequence\. Importantly, the action sequence is provided as a condition: the WM predicts its possible visual consequences but does not determine which action should be executed\.

##### Action\-Conditioned Video Diffusion Models\.

An action\-conditioned video diffusion WM instantiates the conditional distribution in equation[1](https://arxiv.org/html/2609.09418#S3.E1)with a latent video generator\([Wang et al\., 2026b](https://arxiv.org/html/2609.09418#bib.bib25)\)\. Letℰ\\mathcal\{E\}and𝒟\\mathcal\{D\}denote a video encoder and decoder, respectively\. We encode the visual context as𝐡t=ℰ⁡\(𝐎tc\)\\mathbf\{h\}\_\{t\}=\\mathcal\{E\}\(\\mathbf\{O\}^\{\\mathrm\{c\}\}\_\{t\}\)and the target future video as𝐳0=ℰ⁡\(𝐎tf\)\\mathbf\{z\}\_\{0\}=\\mathcal\{E\}\(\\mathbf\{O\}^\{\\mathrm\{f\}\}\_\{t\}\)\. Following the flow\-matching \(FM\) formulation, a noisy latent at diffusion timeτ∈\[0,1\]\\tau\\in\[0,1\]is constructed as

𝐳τ=\(1−τ\)​𝐳0\+τ​ϵ,ϵ∼𝒩⁡\(𝟎,𝐈\),\\mathbf\{z\}\_\{\\tau\}=\(1\-\\tau\)\\mathbf\{z\}\_\{0\}\+\\tau\\bm\{\\epsilon\},\\qquad\\bm\{\\epsilon\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},\\mathbf\{I\}\),\(2\)and the conditional velocity fieldvθv\_\{\\theta\}is learned using

ℒFM=𝔼\[‖vθ\(𝐳τ,τ∣𝐡t,𝐀t\)−\(ϵ−𝐳0\)‖22\]\.\\mathcal\{L\}\_\{\\mathrm\{FM\}\}=\\mathbb\{E\}\\\!\\left\[\\left\\\|v\_\{\\theta\}\\\!\\left\(\\mathbf\{z\}\_\{\\tau\},\\tau\\mid\\mathbf\{h\}\_\{t\},\\mathbf\{A\}\_\{t\}\\right\)\-\\left\(\\bm\{\\epsilon\}\-\\mathbf\{z\}\_\{0\}\\right\)\\right\\\|\_\{2\}^\{2\}\\right\]\.\(3\)At inference time, the learned flow is integrated from Gaussian noise toward the video latent and decoded by𝒟\\mathcal\{D\}to obtain𝐎tf\{\\mathbf\{O\}\}^\{\\mathrm\{f\}\}\_\{t\}\. yr19, different candidate action sequences conditioned on the same visual context produce alternative future video rollouts\. InValerant, the video WM remains frozen and is used only for action\-conditioned rollout generation\.

##### Visual Simultaneous Localization and Mapping\.

Visual simultaneous localization and mapping \(SLAM\) jointly estimates camera motion and reconstructs 3D scene structure from visual observations\([Teed and Deng, 2021](https://arxiv.org/html/2609.09418#bib.bib26);[Murai et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib28)\)\. Given an observation sequence𝐎1:L=\(𝐨1,…,𝐨L\)\\mathbf\{O\}\_\{1:L\}=\(\\mathbf\{o\}\_\{1\},\\ldots,\\mathbf\{o\}\_\{L\}\), visual SLAM estimates the camera trajectory𝐓1:L\\mathbf\{T\}\_\{1:L\}, where𝐓i∈SE⁡\(3\)\\mathbf\{T\}\_\{i\}\\in\\mathrm\{SE\}\(3\)is the camera pose at frameii, together with a spatial mapℳ\\mathcal\{M\}\. This process can be expressed abstractly as

\(𝐓^1:L,ℳ^\)=arg⁡min𝐓1:L,ℳ𝒥SLAM\(𝐎1:L;𝐓1:L,ℳ\),\\left\(\\hat\{\\mathbf\{T\}\}\_\{1:L\},\\hat\{\\mathcal\{M\}\}\\right\)=\\underset\{\\mathbf\{T\}\_\{1:L\},\\,\\mathcal\{M\}\}\{\\arg\\min\}\\;\\mathcal\{J\}\_\{\\mathrm\{SLAM\}\}\\\!\\left\(\\mathbf\{O\}\_\{1:L\};\\mathbf\{T\}\_\{1:L\},\\mathcal\{M\}\\right\),\(4\)where𝒥SLAM\\mathcal\{J\}\_\{\\mathrm\{SLAM\}\}measures the visual and geometric consistency between the observations, camera poses, and reconstructed scene\. InValerant, each generated rollout𝐎^t,mf\\hat\{\\mathbf\{O\}\}^\{\\mathrm\{f\}\}\_\{t,m\}is processed by visual SLAM to recover its camera trajectory and register its observed geometry in a common 3D coordinate system\.

### 3\.2Overview

Given a single input image𝐨0\\mathbf\{o\}\_\{0\},Valerantprogressively explores the synthesized environment and constructs a navigable 3D game map\. At exploration steptt, the system maintains a visual context𝐎tc\\mathbf\{O\}^\{\\mathrm\{c\}\}\_\{t\}, a current camera pose𝐓t\\mathbf\{T\}\_\{t\}, and a global mapℳt\\mathcal\{M\}\_\{t\}\. The pretrained video WM remains frozen throughout this process, while visual SLAM and the exploration policy are applied entirely at inference time\. By coupling the WM with geometric reconstruction and action selection,Valerantturns its passive visual predictions into a closed\-loop exploration process without additional training\.

At each step, the system checkpoints the current WM and SLAM states and evaluates five candidate action sequences:

𝒜t=\{𝐀t\(1\),…,𝐀t\(5\)\}=\{𝚞𝚠,𝚓𝚠,𝚕𝚠,𝚓𝚚,𝚕𝚚\}\.\\mathcal\{A\}\_\{t\}=\\left\\\{\\mathbf\{A\}^\{\(1\)\}\_\{t\},\\ldots,\\mathbf\{A\}^\{\(5\)\}\_\{t\}\\right\\\}=\\left\\\{\\mathtt\{uw\},\\mathtt\{jw\},\\mathtt\{lw\},\\mathtt\{jq\},\\mathtt\{lq\}\\right\\\}\.\(5\)Here,𝚞𝚠\\mathtt\{uw\}denotes moving straight forward without turning, while𝚓𝚠\\mathtt\{jw\}and𝚕𝚠\\mathtt\{lw\}denote turning left and right, respectively, followed by forward movement\. The remaining actions,𝚓𝚚\\mathtt\{jq\}and𝚕𝚚\\mathtt\{lq\}, denote in\-place rotations to the left and right\. The resulting action space therefore contains three forward\-moving candidates and two rotation\-only candidates\.

Starting from the same pre\-action context𝐎tc\\mathbf\{O\}^\{\\mathrm\{c\}\}\_\{t\}, the WM generates a separate future rollout𝐎t,mf\\mathbf\{O\}^\{\\mathrm\{f\}\}\_\{t,m\}for each candidate𝐀t\(m\)\\mathbf\{A\}^\{\(m\)\}\_\{t\}following equation[1](https://arxiv.org/html/2609.09418#S3.E1)\. Because all candidates branch from an identical visual context, their generated rollouts provide directly comparable what\-if outcomes of the available actions\. Rather than estimating their consequences using an additional learned predictor,Valerantexplicitly generates and reconstructs every candidate future\.

Visual SLAM processes each generated rollout𝐎t,mf\\mathbf\{O\}^\{\\mathrm\{f\}\}\_\{t,m\}according to equation[4](https://arxiv.org/html/2609.09418#S3.E4), producing an estimated camera trajectory𝐓^t,m\\hat\{\\mathbf\{T\}\}\_\{t,m\}and candidate scene geometryℳ^t,m\\hat\{\\mathcal\{M\}\}\_\{t,m\}\. Each candidate is therefore evaluated using its reconstructed motion and observed 3D structure rather than its visual appearance alone\. We define its mapping utility using exploration coverage𝒞t,m\\mathcal\{C\}\_\{t,m\}and forward progress𝒫t,m\\mathcal\{P\}\_\{t,m\}, whileℬt,m∈\{0,1\}\\mathcal\{B\}\_\{t,m\}\\in\\\{0,1\\\}indicates whether the candidate violates the collision\-free constraint\. The selected branch is

m⋆=arg⁡maxm∈\{1,…,5\}:ℬt,m=0\(λcov𝒞t,m\+λprog𝒫t,m\),m^\{\\star\}=\\underset\{m\\in\\\{1,\\ldots,5\\\}:\\mathcal\{B\}\_\{t,m\}=0\}\{\\arg\\max\}\\left\(\\lambda\_\{\\mathrm\{cov\}\}\\mathcal\{C\}\_\{t,m\}\+\\lambda\_\{\\mathrm\{prog\}\}\\mathcal\{P\}\_\{t,m\}\\right\),\(6\)whereλcov\\lambda\_\{\\mathrm\{cov\}\}andλprog\\lambda\_\{\\mathrm\{prog\}\}balance the two exploration objectives\.

After selection,Valerantrestores the pre\-action state and commits only𝐀t\(m⋆\)\\mathbf\{A\}^\{\(m^\{\\star\}\)\}\_\{t\}\. The selected action is replayed from the restored state, while all unsuccessful branches are discarded without modifying the persistent world state\. The selected observations and reconstructed geometry are then incorporated into the next visual context and global map:

𝐎t\+1c\\displaystyle\\mathbf\{O\}^\{\\mathrm\{c\}\}\_\{t\+1\}=Update⁡\(𝐎tc,𝐎t,m⋆f\),\\displaystyle=\\operatorname\{Update\}\\\!\\left\(\\mathbf\{O\}^\{\\mathrm\{c\}\}\_\{t\},\\mathbf\{O\}^\{\\mathrm\{f\}\}\_\{t,m^\{\\star\}\}\\right\),\(7\)ℳt\+1\\displaystyle\\mathcal\{M\}\_\{t\+1\}=Fuse⁡\(ℳt,ℳ^t,m⋆\)\.\\displaystyle=\\operatorname\{Fuse\}\\\!\\left\(\\mathcal\{M\}\_\{t\},\\hat\{\\mathcal\{M\}\}\_\{t,m^\{\\star\}\}\\right\)\.This branch–reconstruct–evaluate–commit loop is repeated until the exploration budget is exhausted, progressively consolidating the selected observations into a 3D point\-cloud game map\.

### 3\.3Robust Exploration under Geometric Drift and Visual Hallucinations

Generated rollouts may contain visual hallucinations, while monocular SLAM can accumulate pose and scale drift, leading to unreliable geometry and unsafe exploration decisions\. We define robust exploration as the ability to maintain geometrically consistent mapping and collision\-aware progress under these errors, while recovering from locally trapped states\.

#### 3\.3\.1Robust Floor Estimation

Reliable collision reasoning requires a stable estimate of the floor\. In particular, reconstructed points must be distinguished as traversable ground, body\-level obstacles, or overhead structures according to their height above the local supporting surface\. A floor reference calibrated only once is unreliable under monocular SLAM, whose estimated scale and vertical position may drift over time\. As a result, an otherwise valid ground surface can gradually shift into the obstacle height range, producing false collision detections even in open space\.

To address this issue, we introduce a robust floor estimator that replaces the fixed global reference with a locally adaptive one\. For each SLAM keyframe, we identify candidate support points near the lower envelope of its local reconstruction and fit a dominant plane\. The estimate is accepted only when the plane has sufficient geometric support and its normal is consistent with the current vertical direction\. When the floor cannot be reliably observed, such as when the camera faces a nearby wall, we propagate the most recent reliable camera\-to\-floor relation instead of reverting to the initial global calibration\. This allows the floor estimate to follow gradual SLAM drift while preventing non\-ground surfaces from being adopted as the new reference\.

Let𝐪i\\mathbf\{q\}\_\{i\}be a point on the estimated floor plane of keyframeii,𝐧i\\mathbf\{n\}\_\{i\}its unit normal oriented toward the camera center𝐜i\\mathbf\{c\}\_\{i\}, and𝐩\\mathbf\{p\}a reconstructed 3D point\. We define its normalized floor\-relative height as

h¯i​\(𝐩\)=𝐧i⊤​\(𝐩−𝐪i\)𝐧i⊤​\(𝐜i−𝐪i\)\.\\bar\{h\}\_\{i\}\(\\mathbf\{p\}\)=\\frac\{\\mathbf\{n\}\_\{i\}^\{\\top\}\(\\mathbf\{p\}\-\\mathbf\{q\}\_\{i\}\)\}\{\\mathbf\{n\}\_\{i\}^\{\\top\}\(\\mathbf\{c\}\_\{i\}\-\\mathbf\{q\}\_\{i\}\)\}\.\(8\)Under this normalization, the local floor has height zero and the camera center has height one\. A common translation or scale change in the local reconstruction therefore affects the floor, camera, and scene points together and is canceled by the relative representation\. All subsequent collision tests operate onh¯i\\bar\{h\}\_\{i\}rather than absolute global height\. Consequently, the ground and body\-level occupancy bands remain aligned under reconstruction drift, substantially reducing false obstacles caused by inconsistent floor elevations\.

#### 3\.3\.2Complementary Collision Detection

##### Point\-Cloud Collision\.

Action\-conditioned video WMs do not explicitly enforce collision constraints and may generate trajectories that pass through scene geometry\. We detect such failures directly from the branch\-local geometry reconstructed for each candidate rollout, without relying on a learned occupancy predictor\. We treat the candidate mapℳ^t,m\\hat\{\\mathcal\{M\}\}\_\{t,m\}as a set of reconstructed 3D points and use𝐩\\mathbf\{p\}to denote an individual point\. Each point is assigned a normalized floor\-relative heighth¯​\(𝐩\)\\bar\{h\}\(\\mathbf\{p\}\)using the local floor estimate of its source keyframe from equation[8](https://arxiv.org/html/2609.09418#S3.E8)\. Let𝒮t,m\\mathcal\{S\}\_\{t,m\}denote the set of trajectory samples used for geometric testing, with𝐓^t,m,i\\hat\{\\mathbf\{T\}\}\_\{t,m,i\}representing the pose of samplei∈𝒮t,mi\\in\\mathcal\{S\}\_\{t,m\}andκt,m,i\\kappa\_\{t,m,i\}its associated camera\-to\-floor distance\. We detect a point\-cloud collision by testing whether the reconstructed geometry intersects a normalized cylindrical body volume along the sampled trajectory:

ℬt,mpc=𝕀\[maxi∈𝒮t,m∑𝐩∈ℳ^t,m𝕀\(dist⟂⁡\(𝐩,𝐓^t,m,i\)κt,m,i≤ρ∧ηlow≤h¯\(𝐩\)≤ηhigh\)≥Nocc\],\\mathcal\{B\}^\{\\mathrm\{pc\}\}\_\{t,m\}=\\mathbb\{I\}\\\!\\left\[\\max\_\{i\\in\\mathcal\{S\}\_\{t,m\}\}\\sum\_\{\\mathbf\{p\}\\in\\hat\{\\mathcal\{M\}\}\_\{t,m\}\}\\mathbb\{I\}\\\!\\left\(\\frac\{\\operatorname\{dist\}\_\{\\perp\}\(\\mathbf\{p\},\\hat\{\\mathbf\{T\}\}\_\{t,m,i\}\)\}\{\\kappa\_\{t,m,i\}\}\\leq\\rho\\;\\land\\;\\eta\_\{\\mathrm\{low\}\}\\leq\\bar\{h\}\(\\mathbf\{p\}\)\\leq\\eta\_\{\\mathrm\{high\}\}\\right\)\\geq N\_\{\\mathrm\{occ\}\}\\right\],\(9\)wheredist⟂\\operatorname\{dist\}\_\{\\perp\}measures the horizontal distance from a point to the agent’s vertical body axis,ρ\\rhois the normalized body radius, and\[ηlow,ηhigh\]\[\\eta\_\{\\mathrm\{low\}\},\\eta\_\{\\mathrm\{high\}\}\]specifies the body\-level height range\. The thresholdNoccN\_\{\\mathrm\{occ\}\}suppresses isolated reconstruction noise, while the height restriction excludes traversable ground and overhead structures\. Rotation\-only candidates are exempt because they introduce no additional translational penetration\.

##### Depth\-Based Collision\.

Point\-cloud collision detection may become unreliable for textureless or reflective walls, whose correspondences can receive low confidence and be removed during SLAM reconstruction\. We therefore introduce a complementary detector that operates directly on the unfiltered depth estimates of all rollout frames, including those not selected as SLAM keyframes\. Let𝜹t,m,ℓ\\bm\{\\delta\}\_\{t,m,\\ell\}denote the raw depth map at rollout frameℓ\\ell,Ωctr\\Omega\_\{\\mathrm\{ctr\}\}rectangular region centered in the image, andκt,m,ℓ\\kappa\_\{t,m,\\ell\}the camera\-to\-floor distance associated with that frame\. A penetration event is identified when a frontal surface appears within a normalized near\-field distance but the estimated trajectory nevertheless continues to advance:

ℬt,mdep=𝕀\[∃ℓ:Avg𝐮∈Ωctr⁡𝜹t,m,ℓ​\(𝐮\)κt,m,ℓ≤dnear∧Fwd⁡\(𝐓^t,m,ℓ,L\)κt,m,ℓ≥smin\(ℓ\)\],\\mathcal\{B\}^\{\\mathrm\{dep\}\}\_\{t,m\}=\\mathbb\{I\}\\\!\\left\[\\exists\\,\\ell:\\frac\{\\operatorname\{Avg\}\_\{\\mathbf\{u\}\\in\\Omega\_\{\\mathrm\{ctr\}\}\}\\bm\{\\delta\}\_\{t,m,\\ell\}\(\\mathbf\{u\}\)\}\{\\kappa\_\{t,m,\\ell\}\}\\leq d\_\{\\mathrm\{near\}\}\\;\\land\\;\\frac\{\\operatorname\{Fwd\}\\\!\\left\(\\hat\{\\mathbf\{T\}\}\_\{t,m\};\\ell,L\\right\)\}\{\\kappa\_\{t,m,\\ell\}\}\\geq s\_\{\\mathrm\{min\}\}\(\\ell\)\\right\],\(10\)whereFwd⁡\(𝐓^t,m,ℓ,L\)\\operatorname\{Fwd\}\(\\hat\{\\mathbf\{T\}\}\_\{t,m\};\\ell,L\)measures the forward displacement along the estimated trajectory over the video interval from frameℓ\\ellto frameLL, andsmin​\(ℓ\)s\_\{\\mathrm\{min\}\}\(\\ell\)accounts for the remaining rollout horizon\. The central average provides a robust signature of a nearby frontal surface, while the subsequent motion distinguishes penetration from merely observing a close wall\. The collision indicatorℬt,m\\mathcal\{B\}\_\{t,m\}in equation[6](https://arxiv.org/html/2609.09418#S3.E6)is activated whenever eitherℬt,mpc\\mathcal\{B\}^\{\\mathrm\{pc\}\}\_\{t,m\}orℬt,mdep\\mathcal\{B\}^\{\\mathrm\{dep\}\}\_\{t,m\}is positive\.

#### 3\.3\.3Dead\-Pocket Rewind

Receding\-horizon exploration can lead the agent into a dead pocket in which every translational candidate either violates the collision\-free constraint or provides negligible forward progress\. Continuing local action selection in this state produces repeated rotations or stagnation rather than revealing new space\. We address this failure through a checkpoint\-based rewind mechanism that augments local exploration with backtracking\. Along the committed trajectory,Valerantmaintains synchronized checkpoints of the WM and SLAM states\. When insufficient progress persists, the system selects an earlier checkpointr<tr<tpreceding the stalled segment and restores the visual context, camera pose, and reconstructed map to that state\. To prevent the agent from immediately re\-entering the same pocket, the spatial cells traversed along the discarded segment are retained in a failed\-region memory\. Let𝚽t\\bm\{\\Phi\}\_\{t\}denote the joint WM–SLAM checkpoint at steptt, letℱt\\mathcal\{F\}\_\{t\}denote the accumulated set of rejected cells, and letcell⁡\(𝐓k\)\\operatorname\{cell\}\(\\mathbf\{T\}\_\{k\}\)map a camera pose to its spatial cell\. The rewind operation is summarized as

𝚽t\+←𝚽r,ℱt\+←ℱt∪\{cell⁡\(𝐓k\)\|r<k≤t\},\\bm\{\\Phi\}\_\{t^\{\+\}\}\\leftarrow\\bm\{\\Phi\}\_\{r\},\\qquad\\mathcal\{F\}\_\{t^\{\+\}\}\\leftarrow\\mathcal\{F\}\_\{t\}\\cup\\left\\\{\\operatorname\{cell\}\(\\mathbf\{T\}\_\{k\}\)\\,\\middle\|\\,r<k\\leq t\\right\\\},\(11\)wheret\+t^\{\+\}denotes the state immediately after rewinding\. Subsequent candidate trajectories that enterℱt\+\\mathcal\{F\}\_\{t^\{\+\}\}are treated as inadmissible, encouraging exploration from the restored anchor toward an alternative route\. Because the WM and SLAM states are restored jointly, the generated visual context and reconstructed geometry remain synchronized after backtracking\. If the restored state remains trapped, the procedure can be repeated with an earlier checkpoint, turning the otherwise greedy exploration process into a spatial search with recoverable decisions\.

## 4Experiments

![Refer to caption](https://arxiv.org/html/2609.09418v1/Res1.png)Figure 2:Qualitative Results ofValerantfor Single\-Image 3D Game\-Map Generation\. Each pair of rows presents one scene\. The leftmost column shows the input first frame, the middle columns show six generated frames sampled along the agent’s trajectory, and the rightmost column presents a top\-down view of the final reconstructed map with the agent’s trajectory overlaid\.![Refer to caption](https://arxiv.org/html/2609.09418v1/Ablation.png)Figure 3:Qualitative ablation of the robust exploration modules\. Each pair of rows presents one scene and isolates the effect of a single module by comparing generated frames without \(top\) and with \(bottom\) that module\. RF, COL, and DP denote robust floor estimation, complementary collision detection, and dead\-pocket rewind, respectively\.Table 1:Quantitative comparison\. AS denotes aesthetic score, and UP denotes user preference\.### 4\.1Experiment Settings

##### Dataset & Baseline

All input images used in our experiments are captured directly as screenshots fromVALORANT\. Each screenshot serves as the initial frame and single\-view condition\. We evaluate all methods using the same set of initial frames\. We compareValerantwith HY\-World 2\.0\([HY\-World et al\., 2026](https://arxiv.org/html/2609.09418#bib.bib57)\), which likewise generates a navigable 3D world from a single image but adopts a substantially different technical approach\.

##### Metric

Our task is inherently open\-ended and generative: a single input image can correspond to many plausible 3D environments, and no unique ground\-truth map is available for reference\-based evaluation\. We therefore adopt two perceptual metrics:Authenticity ScoreandUser Preference\. The former uses a VLM as an evaluator to assess the perceptual authenticity of the generated results, while the latter reports the proportion of pairwise comparisons in which human evaluators prefer the output of each method\.

##### Implementation Details

We use Matrix\-Game 3\.0\([Wang et al\., 2026b](https://arxiv.org/html/2609.09418#bib.bib25)\)as the action\-conditioned video WM and MASt3R\-SLAM\([Murai et al\., 2025](https://arxiv.org/html/2609.09418#bib.bib28)\)for visual localization and 3D reconstruction\. We optionally apply video2world\([Höllein and Nießner, 2026](https://arxiv.org/html/2609.09418#bib.bib58)\)as a post\-processing stage to globally refine the reconstructed point cloud\. All experiments are conducted on a single NVIDIA H100 GPU\. Detailed hyperparameter settings are provided in Appendix\.

### 4\.2Experiment Results

##### Main Results

Figure[2](https://arxiv.org/html/2609.09418#S4.F2)presents the qualitative results ofValerantacross four scenes\. Starting from only the first frame, the entire exploration and reconstruction process proceeds automatically, without manually specified trajectories or intermediate intervention\. The generated frames sampled along each trajectory show that the agent moves smoothly through corridors, turns, and open regions while avoiding collisions\. Meanwhile, visual SLAM continuously integrates the selected observations into a persistent 3D point\-cloud map, with the final top\-down views demonstrating substantial spatial coverage along the executed trajectories\. These results show thatValerantcan jointly achieve autonomous collision\-free exploration and incremental 3D game\-map construction from a single image\.

##### Ablation Study

Figure[3](https://arxiv.org/html/2609.09418#S4.F3)qualitatively isolates the contributions of the three robustness components\. In Heaven, removing RF destabilizes the estimated ground reference under geometric drift, causing the agent to remain near the surrounding walls and make limited exploration progress; RF restores reliable floor\-relative reasoning and enables smooth forward traversal into the open area\. In Icebox, the absence of COL allows generated trajectories to penetrate the orange wall, whereas COL rejects geometrically infeasible candidates and keeps exploration within free space\. In Split, without DP, the agent becomes trapped in a dead pocket and repeatedly alternates between left and right turns without making spatial progress; DP restores an earlier exploration state and redirects the agent toward an alternative route\. These comparisons demonstrate that RF, COL, and DP address complementary failure modes arising from floor\-estimation drift, wall penetration, and local exploration traps, respectively\.

## 5Conclusion

We presentedValerant, a training\-free framework that constructs navigable 3D game maps from a single image\. By coupling a frozen action\-conditioned video world model with visual SLAM and an exploration policy,Valerantevaluates alternative action rollouts and incrementally integrates selected observations into a persistent point\-cloud map\. Experiments show that robust floor estimation, collision detection, and dead\-pocket rewind support reliable autonomous exploration under geometric drift and visual hallucinations\. Future work will improve geometric consistency and convert the reconstructed point clouds into richer, editable game assets\.

## References

- Agarwalet al\.\(2026\)A\. Agarwal, P\. Meshram, S\. Singh, S\. Suman, A\. Lapp, S\. Matiana, L\. Castricato, and S\. FrazierCOMBAT: conditional world models for behavioral agent training\.arXiv preprint arXiv:2603\.00825\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Alhaijaet al\.\(2025\)H\. A\. Alhaija, J\. Alvarez, M\. Bala, T\. Cai, T\. Cao, L\. Cha, J\. Chen, M\. Chen, F\. Ferroni, S\. Fidler,et al\.Cosmos\-transfer1: conditional world generation with adaptive multimodal control\.arXiv preprint arXiv:2503\.14492\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Alonsoet al\.\(2024\)E\. Alonso, A\. Jelley, V\. Micheli, A\. Kanervisto, A\. Storkey, T\. Pearce, and F\. FleuretDiffusion for world modeling: visual details matter in atari\.Advances in Neural Information Processing Systems37,pp\. 58757–58791\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p2.1)\.
- Ballet al\.\(2025\)P\. J\. Ball, J\. Bauer, F\. Belletti, B\. Brownfield, A\. Ephrat, S\. Fruchter, A\. Gupta, K\. Holsheimer, A\. Holynski, J\. Hron, C\. Kaplanis, M\. Limont, M\. McGill, Y\. Oliveira, J\. Parker\-Holder, F\. Perbet, G\. Scully, J\. Shar, S\. Spencer, O\. Tov, R\. Villegas, E\. Wang, J\. Yung, C\. Baetu, J\. Berbel, D\. Bridson, J\. Bruce, G\. Buttimore, S\. Chakera, B\. Chandra, P\. Collins, A\. Cullum, B\. Damoc, V\. Dasagi, M\. Gazeau, C\. Gbadamosi, W\. Han, E\. Hirst, A\. Kachra, L\. Kerley, K\. Kjems, E\. Knoepfel, V\. Koriakin, J\. Lo, C\. Lu, Z\. Mehring, A\. Moufarek, H\. Nandwani, V\. Oliveira, F\. Pardo, J\. Park, A\. Pierson, B\. Poole, H\. Ran, T\. Salimans, M\. Sanchez, I\. Saprykin, A\. Shen, S\. Sidhwani, D\. Smith, J\. Stanton, H\. Tomlinson, D\. Vijaykumar, L\. Wang, P\. Wingfield, N\. Wong, K\. Xu, C\. Yew, N\. Young, V\. Zubov, D\. Eck, D\. Erhan, K\. Kavukcuoglu, D\. Hassabis, Z\. Gharamani, R\. Hadsell, A\. van den Oord, I\. Mosseri, A\. Bolton, S\. Singh, and T\. RocktäschelGenie 3: a new frontier for world models\.External Links:Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Baret al\.\(2025\)A\. Bar, G\. Zhou, D\. Tran, T\. Darrell, and Y\. LeCunNavigation world models\.In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 15791–15801\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1),[§1](https://arxiv.org/html/2609.09418#S1.p2.1),[§2\.3](https://arxiv.org/html/2609.09418#S2.SS3.p1.1)\.
- Biet al\.\(2026\)H\. Bi, H\. Tan, S\. Xie, Z\. Wang, S\. Huang, H\. Liu, R\. Zhao, Y\. Feng, C\. Xiang, Y\. Rong,et al\.Motus: a unified latent action world model\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 35101–35113\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Brookset al\.\(2024\)T\. Brooks, B\. Peebles, C\. Holmes, W\. DePue, Y\. Guo, L\. Jing, D\. Schnurr, J\. Taylor, T\. Luhman, E\. Luhman,et al\.Video generation models as world simulators\.OpenAI Blog1\(8\),pp\. 1\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Camposet al\.\(2021\)C\. Campos, R\. Elvira, J\. J\. G\. Rodríguez, J\. M\. Montiel, and J\. D\. TardósOrb\-slam3: an accurate open\-source library for visual, visual–inertial, and multimap slam\.IEEE transactions on robotics37\(6\),pp\. 1874–1890\.Cited by:[§2\.2](https://arxiv.org/html/2609.09418#S2.SS2.p1.1)\.
- Chaplotet al\.\(2020\)D\. S\. Chaplot, D\. Gandhi, S\. Gupta, A\. Gupta, and R\. SalakhutdinovLearning to explore using active neural slam\.arXiv preprint arXiv:2004\.05155\.Cited by:[§2\.2](https://arxiv.org/html/2609.09418#S2.SS2.p1.1)\.
- Cheet al\.\(2025\)H\. Che, X\. He, Q\. Liu, C\. Jin, and H\. ChenGamegen\-x: interactive open\-world game video generation\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 37546–37593\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Decartet al\.\(2024\)E\. Decart, Q\. McIntyre, S\. Campbell, X\. Chen, and R\. WachenOasis: a universe in a transformer\.URL: https://oasis\-model\. github\. io2\(3\),pp\. 6\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Duet al\.\(2023\)Y\. Du, S\. Yang, B\. Dai, H\. Dai, O\. Nachum, J\. Tenenbaum, D\. Schuurmans, and P\. AbbeelLearning universal policies via text\-guided video generation\.Advances in neural information processing systems36,pp\. 9156–9172\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Gaoet al\.\(2026\)S\. Gao, W\. Liang, K\. Zheng, A\. Malik, S\. Ye, S\. Yu, W\. Tseng, Y\. Dong, K\. Mo, C\. Lin,et al\.Dreamdojo: a generalist robot world model from large\-scale human videos\.arXiv preprint arXiv:2602\.06949\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p2.1)\.
- Garcinet al\.\(2026\)S\. Garcin, T\. Walker, S\. McDonagh, T\. Pearce, H\. Bilen, T\. He, K\. Wang, and J\. BianBeyond pixel histories: world models with persistent 3d state\.arXiv preprint arXiv:2603\.03482\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Guoet al\.\(2025\)J\. Guo, Y\. Ye, T\. He, H\. Wu, Y\. Jiang, T\. Pearce, and J\. BianMineworld: a real\-time and open\-source interactive world model on minecraft\.arXiv preprint arXiv:2504\.08388\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Ha and Schmidhuber \(2018\)D\. Ha and J\. SchmidhuberWorld models\.arXiv preprint arXiv:1803\.101222\(3\),pp\. 440\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Hafneret al\.\(2019a\)D\. Hafner, T\. Lillicrap, J\. Ba, and M\. NorouziDream to control: learning behaviors by latent imagination\.arXiv preprint arXiv:1912\.01603\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Hafneret al\.\(2019b\)D\. Hafner, T\. Lillicrap, I\. Fischer, R\. Villegas, D\. Ha, H\. Lee, and J\. DavidsonLearning latent dynamics for planning from pixels\.InInternational conference on machine learning,pp\. 2555–2565\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Hafneret al\.\(2025\)D\. Hafner, J\. Pasukonis, J\. Ba, and T\. LillicrapMastering diverse control tasks through world models\.Nature640\(8059\),pp\. 647–653\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Hansenet al\.\(2024\)N\. Hansen, H\. Su, and X\. WangTd\-mpc2: scalable, robust world models for continuous control\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 47376–47405\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1),[§2\.3](https://arxiv.org/html/2609.09418#S2.SS3.p1.1)\.
- Hoet al\.\(2020\)J\. Ho, A\. Jain, and P\. AbbeelDenoising diffusion probabilistic models\.Advances in neural information processing systems33,pp\. 6840–6851\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Holland \(1986\)P\. W\. HollandStatistics and causal inference\.Journal of the American statistical Association81\(396\),pp\. 945–960\.Cited by:[§2\.3](https://arxiv.org/html/2609.09418#S2.SS3.p1.1)\.
- Höllein and Nießner \(2026\)L\. Höllein and M\. NießnerWorld reconstruction from inconsistent views\.arXiv preprint arXiv:2603\.16736\.Cited by:[§4\.1](https://arxiv.org/html/2609.09418#S4.SS1.SSS0.Px3.p1.1)\.
- HY\-Worldet al\.\(2026\)T\. HY\-World, C\. Cao, X\. Zuo, Z\. Wang, Y\. Zhang, J\. Wu, Z\. Liu, Y\. Gong, Y\. Liu, B\. Yuan,et al\.HY\-world 2\.0: a multi\-modal world model for reconstructing, generating, and simulating 3d worlds\.arXiv preprint arXiv:2604\.14268\.Cited by:[§4\.1](https://arxiv.org/html/2609.09418#S4.SS1.SSS0.Px1.p1.1)\.
- Kanervistoet al\.\(2025\)A\. Kanervisto, D\. Bignell, L\. Y\. Wen, M\. Grayson, R\. Georgescu, S\. Valcarcel Macua, S\. Z\. Tan, T\. Rashid, T\. Pearce, Y\. Cao,et al\.World and human action models towards gameplay ideation\.Nature638\(8051\),pp\. 656–663\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Kimet al\.\(2026\)M\. J\. Kim, Y\. Gao, T\. Lin, Y\. Lin, Y\. Ge, G\. Lam, P\. Liang, S\. Song, M\. Liu, C\. Finn,et al\.Cosmos policy: fine\-tuning video models for visuomotor control and planning\.arXiv preprint arXiv:2601\.16163\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Liet al\.\(2025\)J\. Li, J\. Tang, Z\. Xu, L\. Wu, Y\. Zhou, S\. Shao, T\. Yu, Z\. Cao, and Q\. LuHunyuan\-gamecraft: high\-dynamic interactive game video generation with hybrid history condition\.arXiv preprint arXiv:2506\.172012\(3\),pp\. 6\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p2.1)\.
- Liet al\.\(2024\)X\. Li, K\. Hsu, J\. Gu, K\. Pertsch, O\. Mees, H\. R\. Walke, C\. Fu, I\. Lunawat, I\. Sieh, S\. Kirmani,et al\.Evaluating real\-world robot manipulation policies in simulation\.arXiv preprint arXiv:2405\.05941\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p2.1)\.
- Lipmanet al\.\(2022\)Y\. Lipman, R\. T\. Chen, H\. Ben\-Hamu, M\. Nickel, and M\. LeFlow matching for generative modeling\.arXiv preprint arXiv:2210\.02747\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Liuet al\.\(2022\)X\. Liu, C\. Gong, and Q\. LiuFlow straight and fast: learning to generate and transfer data with rectified flow\.arXiv preprint arXiv:2209\.03003\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Liuet al\.\(2025\)Y\. Liu, S\. Dong, S\. Wang, Y\. Yin, Y\. Yang, Q\. Fan, and B\. ChenSlam3r: real\-time dense scene reconstruction from monocular rgb videos\.In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 16651–16662\.Cited by:[§2\.2](https://arxiv.org/html/2609.09418#S2.SS2.p1.1)\.
- Maggioet al\.\(2026\)D\. Maggio, H\. Lim, and L\. CarloneVggt\-slam: dense rgb slam optimized on the sl \(4\) manifold\.Advances in Neural Information Processing Systems38,pp\. 129839–129867\.Cited by:[§2\.2](https://arxiv.org/html/2609.09418#S2.SS2.p1.1)\.
- Matsukiet al\.\(2024\)H\. Matsuki, R\. Murai, P\. H\. Kelly, and A\. J\. DavisonGaussian splatting slam\.In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 18039–18048\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p3.1)\.
- Muraiet al\.\(2025\)R\. Murai, E\. Dexheimer, and A\. J\. DavisonMast3r\-slam: real\-time dense slam with 3d reconstruction priors\.In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 16695–16705\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p3.1),[§2\.2](https://arxiv.org/html/2609.09418#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2609.09418#S3.SS1.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2609.09418#S4.SS1.SSS0.Px3.p1.1)\.
- Parker\-Holderet al\.\(2024\)J\. Parker\-Holder, P\. Ball, J\. Bruce, V\. Dasagi, K\. Holsheimer, C\. Kaplanis, A\. Moufarek, G\. Scully, J\. Shar, J\. Shi, S\. Spencer, J\. Yung, M\. Dennis, S\. Kenjeyev, S\. Long, V\. Mnih, H\. Chan, M\. Gazeau, B\. Li, F\. Pardo, L\. Wang, L\. Zhang, F\. Besse, T\. Harley, A\. Mitenkova, J\. Wang, J\. Clune, D\. Hassabis, R\. Hadsell, A\. Bolton, S\. Singh, and T\. RocktäschelGenie 2: a large\-scale foundation world model\.External Links:[Link](https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/)Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Pearlet al\.\(2003\)J\. Pearlet al\.Causality: models, reasoning, and inference\.Econometric Theory19\(675\-685\),pp\. 46\.Cited by:[§2\.3](https://arxiv.org/html/2609.09418#S2.SS3.p1.1)\.
- Peebles and Xie \(2023\)W\. Peebles and S\. XieScalable diffusion models with transformers\.In2023 IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 4172–4182\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Podellet al\.\(2024\)D\. Podell, Z\. English, K\. Lacey, A\. Blattmann, T\. Dockhorn, J\. Müller, J\. Penna, and R\. RombachSdxl: improving latent diffusion models for high\-resolution image synthesis\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 1862–1874\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Qinet al\.\(2024\)Y\. Qin, Z\. Shi, J\. Yu, X\. Wang, E\. Zhou, L\. Li, Z\. Yin, X\. Liu, L\. Sheng, J\. Shao,et al\.Worldsimbench: towards video generation models as world simulators\.arXiv preprint arXiv:2410\.18072\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Quevedoet al\.\(2026\)J\. Quevedo, A\. K\. Sharma, Y\. Sun, V\. Suryavanshi, P\. Liang, and S\. YangWorldgym: world model as an environment for policy evaluation\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 78932–78957\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p2.1)\.
- Rigteret al\.\(2024\)M\. Rigter, T\. Gupta, A\. Hilmkil, and C\. MaAvid: adapting video diffusion models to world models\.arXiv preprint arXiv:2410\.12822\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Rombachet al\.\(2022\)R\. Rombach, A\. Blattmann, D\. Lorenz, P\. Esser, and B\. OmmerHigh\-resolution image synthesis with latent diffusion models\.In2022 IEEE/CVF conference on computer vision and pattern recognition \(CVPR\),pp\. 10674–10685\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Rubin \(1974\)D\. B\. RubinEstimating causal effects of treatments in randomized and nonrandomized studies\.\.Journal of educational Psychology66\(5\),pp\. 688\.Cited by:[§2\.3](https://arxiv.org/html/2609.09418#S2.SS3.p1.1)\.
- Teed and Deng \(2021\)Z\. Teed and J\. DengDroid\-slam: deep visual slam for monocular, stereo, and rgb\-d cameras\.Advances in neural information processing systems34,pp\. 16558–16569\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p3.1),[§2\.2](https://arxiv.org/html/2609.09418#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2609.09418#S3.SS1.SSS0.Px3.p1.1)\.
- Valevskiet al\.\(2025\)D\. Valevski, Y\. Leviathan, M\. Arar, and S\. FruchterDiffusion models are real\-time game engines\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 73754–73776\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1),[§1](https://arxiv.org/html/2609.09418#S1.p2.1)\.
- Wanet al\.\(2025\)T\. Wan, A\. Wang, B\. Ai, B\. Wen, C\. Mao, C\. Xie, D\. Chen, F\. Yu, H\. Zhao, J\. Yang,et al\.Wan: open and advanced large\-scale video generative models\.arXiv preprint arXiv:2503\.20314\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Wanget al\.\(2026a\)S\. Wang, J\. Shi, Z\. Fu, X\. He, F\. Liu, C\. Yang, Y\. Zhou, Z\. Fei, J\. Gong, J\. Fu,et al\.World action models: the next frontier in embodied ai\.arXiv preprint arXiv:2605\.12090\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1),[§3\.1](https://arxiv.org/html/2609.09418#S3.SS1.SSS0.Px1.p1.1)\.
- Wanget al\.\(2024\)Y\. Wang, J\. He, L\. Fan, H\. Li, Y\. Chen, and Z\. ZhangDriving into the future: multiview visual forecasting and planning with world model for autonomous driving\.In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 14749–14759\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1),[§1](https://arxiv.org/html/2609.09418#S1.p2.1),[§2\.3](https://arxiv.org/html/2609.09418#S2.SS3.p1.1)\.
- Wanget al\.\(2026b\)Z\. Wang, Z\. Liu, J\. Li, K\. Huang, B\. Xu, F\. Kang, M\. An, P\. Wang, B\. Jiang, Y\. Wei,et al\.Matrix\-game 3\.0: real\-time and streaming interactive world model with long\-horizon memory\.arXiv preprint arXiv:2604\.08995\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1),[§3\.1](https://arxiv.org/html/2609.09418#S3.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2609.09418#S4.SS1.SSS0.Px3.p1.1)\.
- Wuet al\.\(2024\)J\. Wu, S\. Yin, N\. Feng, X\. He, D\. Li, J\. Hao, and M\. LongIvideogpt: interactive videogpts are scalable world models\.Advances in Neural Information Processing Systems37,pp\. 68082–68119\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Wuet al\.\(2026\)T\. Wu, S\. Yang, R\. Po, Y\. Xu, Z\. Liu, D\. Lin, and G\. WetzsteinVideo world models with long\-term spatial memory\.Advances in Neural Information Processing Systems38,pp\. 49371–49393\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Wuet al\.\(2025\)Y\. Wu, Z\. Zhang, J\. Chen, H\. Tang, D\. Li, Y\. Fang, L\. Zhu, E\. Xie, H\. Yin, L\. Yi,et al\.Vila\-u: a unified foundation model integrating visual understanding and generation\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 93620–93638\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Xiaet al\.\(2026\)T\. Xia, Y\. Li, L\. Zhou, J\. Yao, K\. Xiong, H\. Sun, B\. Wang, K\. Ma, G\. Chen, H\. Ye,et al\.Drivelaw: unifying planning and video generation in a latent driving world\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 39701–39712\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p2.1)\.
- Yanget al\.\(2023\)S\. Yang, Y\. Du, K\. Ghasemipour, J\. Tompson, L\. Kaelbling, D\. Schuurmans, and P\. AbbeelLearning interactive real\-world simulators\.arXiv preprint arXiv:2310\.06114\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1)\.
- Yuet al\.\(2022\)W\. Yu, W\. Chen, S\. Yin, S\. Easterbrook, and A\. GargModular action concept grounding in semantic video prediction\.In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 3595–3604\.Cited by:[§2\.3](https://arxiv.org/html/2609.09418#S2.SS3.p1.1)\.
- Zhanget al\.\(2026\)X\. Zhang, X\. Zeng, and W\. ZhangFrom world models to world action models: a concise tutorial for robotics\.arXiv preprint arXiv:2607\.00836\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Zhaoet al\.\(2025\)Q\. Zhao, Y\. Lu, M\. J\. Kim, Z\. Fu, Z\. Zhang, Y\. Wu, Z\. Li, Q\. Ma, S\. Han, C\. Finn,et al\.Cot\-vla: visual chain\-of\-thought reasoning for vision\-language\-action models\.In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 1702–1713\.Cited by:[§1](https://arxiv.org/html/2609.09418#S1.p1.1),[§1](https://arxiv.org/html/2609.09418#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.
- Zhuet al\.\(2025\)F\. Zhu, H\. Wu, S\. Guo, Y\. Liu, C\. Cheang, and T\. KongIrasim: a fine\-grained world model for robot manipulation\.In2025 IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 9834–9844\.Cited by:[§2\.1](https://arxiv.org/html/2609.09418#S2.SS1.p1.1)\.