具身图:面向长时程任务的关系世界模型
摘要
该论文提出了具身图(Embodied Graph)和关系世界模型(Relational World Model,RWM),它们在稀疏的图结构状态抽象上进行想象推演,以支持具身环境和网页环境中的长时程规划,并取得了显著成果,例如在预算范围内于《我的世界》中获取钻石。
arXiv:2611.24115v1 Announce Type: new
Abstract: Long-horizon tasks in embodied and web environments involve extended decision sequences and delayed feedback, making accurate planning crucial for success. World models tackle this by learning latent dynamics for imagination-based planning. However, existing approaches merely rely on dense pixel-level observations, which forces the model to absorb redundant information while overlooking relations among task-relevant entities. Consequently, small errors may accumulate over long imagined rollouts and gradually corrupt the predictions. In this paper, we explore world models that imagine over sparse state abstractions, which poses two key challenges: 1) how to remove redundant information and form a compact state abstraction; 2) how to incorporate the relational structure to learn reliable long-horizon dynamics. To address these challenges, we propose Embodied Graph, a relational state abstraction that supports structured dynamics modeling and long-horizon imagination, and the Relational World Model (RWM), which performs imagination directly over graph states. RWM consists of (i) a Task-Aware Embodied Graph Constructor that filters redundant details and maintains graph memory through online updates, and (ii) an Embodied Graph Recurrent State Space Model that learns node-wise latent dynamics while capturing inter-node dependencies. The imagined graph states provide predictions of action consequences to guide decision-making. Experiments on Minecraft and Crafter demonstrate the effectiveness of RWM in embodied environments. Notably, in Minecraft, RWM is the only method that obtains diamonds within the given budget. Experiments on web interaction further illustrate the applicability of relational state prediction to the decision making process of web agents.
查看缓存全文
缓存时间: 2026/11/20 09:14
# Embodied Graph: A Relational World Model for Long-Horizon Tasks
Source: [https://arxiv.org/html/2611.24115](https://arxiv.org/html/2611.24115)
Conference:;DOI:[;](https://doi.org/;)CCS:Computing methodologies Machine learningCCS:Computing methodologies Artificial intelligence2027
###### Abstract\.
Long\-horizon tasks in embodied and web environments involve extended decision sequences and delayed feedback, making accurate planning crucial for success\. World models tackle this by learning latent dynamics for imagination\-based planning\. However, existing approaches merely rely ondensepixel\-level observations, which forces the model to absorb redundant information while overlooking relations among task\-relevant entities\. Consequently, small errors may accumulate over long imagined rollouts and gradually corrupt the predictions\. In this paper, we explore world models that imagine over sparse state abstractions, which poses two key challenges: 1\) how to remove redundant information and form a compact state abstraction; 2\) how to incorporate the relational structure to learn reliable long\-horizon dynamics\. To address these challenges, we proposeEmbodied Graph, a relational state abstraction that supports structured dynamics modeling and long\-horizon imagination, and theRelationalWorldModel \(RWM\), which performs imagination directly over graph states\. RWM consists of \(i\) a*Task\-Aware Embodied Graph Constructor*that filters redundant details and maintains graph memory through online updates, and \(ii\) an*Embodied Graph Recurrent State Space Model*that learns node\-wise latent dynamics while capturing inter\-node dependencies\. The imagined graph states provide predictions of action consequences to guide decision\-making\. Experiments on Minecraft and Crafter demonstrate the effectiveness of RWM in embodied environments\. Notably, in Minecraft, RWM is the only method that obtains diamonds within the given budget\. Experiments on web interaction further illustrate the applicability of relational state prediction to the decision making process of web agents\.
###### Keywords:
Embodied Graph, World Model, Long\-Horizon Task, Web Agents
## 1\.Introduction
Embodied long\-horizon tasks\([Feng et al\., 2025a](https://arxiv.org/html/2611.24115#bib.bib59);[Feng et al\., 2025c](https://arxiv.org/html/2611.24115#bib.bib26);[Guo et al\., 2024](https://arxiv.org/html/2611.24115#bib.bib25);[Shen et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib24)\)are interactive tasks in which agents must maintain coherent planning and decision\-making over an extended interaction horizon to achieve the final goal\. Despite recent progress, these tasks remain challenging in open dynamic environments\([Feng et al\., 2026](https://arxiv.org/html/2611.24115#bib.bib60)\), ranging from manipulation\([Ma et al\., 2024](https://arxiv.org/html/2611.24115#bib.bib17);[Yang et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib18)\), navigation\([Liu et al\., 2024a](https://arxiv.org/html/2611.24115#bib.bib19);[Hong et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib20)\)to task planning\([Wang et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib21);[Fei et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib22)\), largely due to sparse rewards and delayed credit assignment over long time spans\([Pignatelli et al\., 2023](https://arxiv.org/html/2611.24115#bib.bib23)\)\. World models have emerged as a promising approach for such tasks by learning internal representations and predicting future states of the environment for imagination\-based planning and control\([Shang et al\., 2026](https://arxiv.org/html/2611.24115#bib.bib61);[Li et al\., 2025b](https://arxiv.org/html/2611.24115#bib.bib12);[Hafner and et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib4);[Ren et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib13);[Feng et al\., 2025b](https://arxiv.org/html/2611.24115#bib.bib57)\)\.
Nevertheless, existing world models are typically trained withdensesupervision from high\-dimensional observations, which injects massive redundant details\([Kim et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib37);[Deng et al\., 2021](https://arxiv.org/html/2611.24115#bib.bib36);[Burchi and Timofte, 2024](https://arxiv.org/html/2611.24115#bib.bib35)\)and obscures the spatio\-temporal relations that matter for long\-horizon prediction and planning\([Kipf et al\., 2020](https://arxiv.org/html/2611.24115#bib.bib38)\)\. The amount of visual change does not necessarily reflect its importance for decision\-making\. A visual change can alter many pixels while leaving the underlying object states unchanged, whereas acquiring a needed item may produce only a small visual difference\. Dense prediction objectives therefore devote capacity to appearance changes while overlooking state transitions that determine subsequent actions, which can lead to errors in predicting task\-relevant state changes\. Small errors may then accumulate over long rollouts\([Talvitie, 2017](https://arxiv.org/html/2611.24115#bib.bib39)\), forming imagination drift to corrupt imagined trajectories and shortening the effective imagination horizon\. In Fig\.[1](https://arxiv.org/html/2611.24115#S1.F1), a smoothed sandbar compounds into a hallucinated oasis, and a policy trained on such rollouts is steered toward states that the real environment never offers\.
Figure 1\.Imagination drift in a pixel\-level world model\. A minor early inaccuracy \(a smoothed sandbar\) can accumulate into a severe deviation \(a hallucinated oasis\) that would lead the agent toward an implausible state\.For this reason, we seek to use sparse relational graphs as state abstractions, alleviating error accumulation through a compact state representation\. Unlike dense representations, graphs offer a structured and compact abstraction of objects and their relations, which can naturally filter out irrelevant details and focus on critical interactions\. For instance, on interactive webpages, such an abstraction can capture interactive page elements and their relations, supporting predictions of changes in field values and control states that inform subsequent action choices\. However, leveraging graph representations for long\-horizon tasks introduces several challenges: \(1\) since environments are open and dynamic, objects and relations may appear, disappear, or alter their attributes, it is challenging to distill informative structure and update the graph online while maintaining consistency and stability\. \(2\) It is also challenging to learn stable graph dynamics whose rollouts incur less compounding error, thereby extending the effective imagination horizon, i\.e\., how far the model can be rolled out before its error grows substantially\.
To tackle these challenges, we introduceEmbodied Graph, a sparse relational state abstraction for open dynamic environments\. In Embodied Graph, nodes represent the agent and environment entities, edges encode the relations along which the effects of interactions propagate, and an interaction is an agent action applied to a target node\. On top of it, we build theRelationalWorldModel \(RWM\), a*graph world model*with two learned components: \(i\) a*Task\-Aware Embodied Graph Constructor \(TA\-EGC\)*that maintains the embodied graph as a bounded global memory updated from online observations, and \(ii\) an*Embodied Graph Recurrent State Space Model \(EG\-RSSM\)*that learns node\-wise latent dynamics with relational propagation and imagines over graph states\. Experiments in Embodied environments \(Minecraft\([Hill et al\., 2023](https://arxiv.org/html/2611.24115#bib.bib14)\)and Crafter\([Hafner, 2022](https://arxiv.org/html/2611.24115#bib.bib27)\)\) show that RWM outperforms state\-of\-the\-art baselines, with the largest gains on long\-horizon targets, and its open\-loop prediction error grows much more slowly than that of DreamerV3\. And further experiments on web agents\([Zhou et al\., 2024](https://arxiv.org/html/2611.24115#bib.bib53)\)illustrate the effectiveness of RWM on web agents\. Our contributions are:
- •We proposeEmbodied Graph, a graph\-based paradigm for modeling environment dynamics in long\-horizon tasks\.
- •We proposeRelational World Model\(RWM\), a graph\-based world model framework consists of a*Task\-Aware Embodied Graph Constructor*for maintaining a sparse global embodied graph, and an*Embodied Graph Recurrent State Space Model*that performs node\-wise latent dynamics learning while capturing inter\-node dependencies\.
- •Through extensive experiments, we demonstrate that the proposedRWMachieves superior performance over strong baselines in challenging long\-horizon tasks on both of the embodied control and web agents\.
## 2\.Embodied Graph
As structured abstractions of complex relational patterns, graphs are well\-suited for modeling the intricate relationships between agents and entities in their environments\. However, traditional graph approaches such as 3D scene graphs cannot adequately model dynamic interactions\. To address this limitation, we introduce theEmbodied Graph\(EG\), an interactive graph formalism designed to capture environmental relational dynamics\. We formally define the embodied graph and the associated task formulation as follows\.
Embodied Graph\.We denote the embodied graph at timettasGt\(𝒱t,ℰt,ℐt\)G^\{t\}\(\\mathcal\{V\}^\{t\},\\mathcal\{E\}^\{t\},\\mathcal\{I\}^\{t\}\), where𝒱t\\mathcal\{V\}^\{t\},ℰt\\mathcal\{E\}^\{t\}, andℐt\\mathcal\{I\}^\{t\}represent the sets of nodes, edges, and interactions, respectively\. The complete embodied graph is expressed as𝒢\(𝒱,ℰ,ℐ\)=\{Gt\(𝒱t,ℰt,ℐt\)\}t=1T\\mathcal\{G\}\(\\mathcal\{V\},\\mathcal\{E\},\\mathcal\{I\}\)=\\\{G^\{t\}\(\\mathcal\{V\}^\{t\},\\mathcal\{E\}^\{t\},\\mathcal\{I\}^\{t\}\)\\\}\_\{t=1\}^\{T\}, where𝒱=⋃t=1T𝒱t\\mathcal\{V\}=\\bigcup\_\{t=1\}^\{T\}\\mathcal\{V\}^\{t\},ℰ=⋃t=1Tℰt\\mathcal\{E\}=\\bigcup\_\{t=1\}^\{T\}\\mathcal\{E\}^\{t\}, andℐ=⋃t=1Tℐt\\mathcal\{I\}=\\bigcup\_\{t=1\}^\{T\}\\mathcal\{I\}^\{t\}\.
Node, Edge, and Interaction Sets\.Nodes in the embodied graph encompass both agents and environment entities, where agents are capable of actively interacting with entities\. This heterogeneous node structure is represented as𝒱=\{𝒱a,𝒱e\}\\mathcal\{V\}=\\\{\\mathcal\{V\}\_\{a\},\\mathcal\{V\}\_\{e\}\\\}, with𝒱a\\mathcal\{V\}\_\{a\}and𝒱e\\mathcal\{V\}\_\{e\}corresponding to the agent and entity nodes, respectively\. Therefore, edges are also heterogeneous and categorized into entity\-entity, entity\-agent, and agent\-agent relationships, denoted asℰ=\{ℰee,ℰea,ℰaa\}\\mathcal\{E\}=\\\{\\mathcal\{E\}\_\{ee\},\\mathcal\{E\}\_\{ea\},\\mathcal\{E\}\_\{aa\}\\\}\(agent\-agent edges exist only in multi\-agent scenarios\)\. Both nodes and edges incorporate the corresponding feature sets𝒳=\{𝒳a,𝒳e\}\\mathcal\{X\}=\\\{\\mathcal\{X\}\_\{a\},\\mathcal\{X\}\_\{e\}\\\}and𝒮=\{𝒮ee,𝒮ea,𝒮aa\}\\mathcal\{S\}=\\\{\\mathcal\{S\}\_\{ee\},\\mathcal\{S\}\_\{ea\},\\mathcal\{S\}\_\{aa\}\\\}\. Edges are directed, where\(u,v\)∈ℰt\(u,v\)\\in\\mathcal\{E\}^\{t\}means that the state ofuuinforms the change ofvv, a symmetric relation is represented by two opposite edges, and𝒩\(v\)=\{u:\(u,v\)∈ℰt\}\\mathcal\{N\}\(v\)=\\\{u:\(u,v\)\\in\\mathcal\{E\}^\{t\}\\\}denotes the neighbors ofvv\. Unlike traditional graphs, embodied graphs involve interactions between different nodes\. An interactionA=\(i,v\)A=\(i,v\)consists of an interaction typeiiand a target nodevv\. Due to the heterogeneity of the nodes, similar to the edges, the interactions can be divided asℐ=\{ℐee,ℐea,ℐaa\}\\mathcal\{I\}=\\\{\\mathcal\{I\}\_\{ee\},\\mathcal\{I\}\_\{ea\},\\mathcal\{I\}\_\{aa\}\\\}\. The corresponding interaction features are denoted asℛ=\{ℛee,ℛea,ℛaa\}\\mathcal\{R\}=\\\{\\mathcal\{R\}\_\{ee\},\\mathcal\{R\}\_\{ea\},\\mathcal\{R\}\_\{aa\}\\\}\.
Prediction Tasks\.We consider a partially observable environment in which the agent receives observationsoto\_\{t\}and aims to maximize its expected return\. The embodied graph is maintained from the observations,Gt=Cϕ\(Gt−1,ot\)G^\{t\}=C\_\{\\phi\}\(G^\{t\-1\},o\_\{t\}\), and provides a structured state abstraction to predict environment evolution under interactions:
\(1\)pΘ\(Gt\+1∣G≤t,A≤t\),At=\(it,vtart\)∈ℐt,p\_\{\\Theta\}\\big\(G^\{t\+1\}\\mid G^\{\\leq t\},A^\{\\leq t\}\\big\),\\qquad A^\{t\}=\(i^\{t\},v\_\{tar\}^\{t\}\)\\in\\mathcal\{I\}^\{t\},wherepΘp\_\{\\Theta\}is the learnable dynamics model andAtA^\{t\}is the chosen interaction with its target node\. The model is learned by maximizing the likelihood of the observed next graphs\.
\(2\)minΘℒ\(Θ\)=−∑tlogpΘ\(Gt\+1∣G≤t,A≤t\),\\min\_\{\\Theta\}\\ \\mathcal\{L\}\(\\Theta\)=\-\\sum\_\{t\}\\log p\_\{\\Theta\}\\big\(G^\{t\+1\}\\mid G^\{\\leq t\},A^\{\\leq t\}\\big\),
## 3\.Relational World Model
This section presents theRelational World Model \(RWM\)\. RWM consists of the Task\-Aware Embodied Graph Constructor \(Section[3\.1](https://arxiv.org/html/2611.24115#S3.SS1)\), which implementsCϕC\_\{\\phi\}, and the Embodied Graph Recurrent State Space Model \(Section[3\.2](https://arxiv.org/html/2611.24115#S3.SS2)\), which implementspΘp\_\{\\Theta\}and performs imagination\. Section[3\.3](https://arxiv.org/html/2611.24115#S3.SS3)describes how RWM is used for decision making in embodied control and web agents\.
Figure 2\.Overall framework of RWM\. A Task\-Aware Embodied Graph Constructor builds and continuously updates a compact embodied graph, while an Embodied Graph Recurrent State Space Model learns action\-conditioned, node\-wise dynamics through relational propagation\. The graph imagination model predicts future node attributes and structural outcomes, providing forecasts of action consequences to guide decision\-making in dynamic environments\.### 3\.1\.Task\-Aware Embodied Graph Constructor \(TA\-EGC\)
TA\-EGC implementsCϕC\_\{\\phi\}and maintains the embodied graph as a bounded global memory of at mostNNnodes\. At each step, a front\-end converts the observationoto\_\{t\}into candidate nodes𝒱^t=\{v^it\}i=1M\\hat\{\\mathcal\{V\}\}^\{t\}=\\\{\\hat\{v\}\_\{i\}^\{t\}\\\}\_\{i=1\}^\{M\}with attribute vectorso^it\\hat\{o\}\_\{i\}^\{t\}\. The update\(Gt−1,𝒱^t\)→Gt\(G^\{t\-1\},\\hat\{\\mathcal\{V\}\}^\{t\}\)\\rightarrow G^\{t\}consists of task\-aware node filtering and dynamic graph fusion\.
Task\-aware node filtering\.Inserting all candidates would fill the graph with redundant fragments and transient noise\. Therefore, an admission scorer rates each candidate by task relevance, novelty, and coverage,sit=αtsi,taskt\+αnsi,novelt\+αcsi,coverts\_\{i\}^\{t\}=\\alpha\_\{t\}s\_\{i,task\}^\{t\}\+\\alpha\_\{n\}s\_\{i,novel\}^\{t\}\+\\alpha\_\{c\}s\_\{i,cover\}^\{t\}, and the Top\-PPcandidates form the insertion set𝒰t\\mathcal\{U\}^\{t\}\. Each score is predicted by a scorerSθkS\_\{\\theta\_\{k\}\}, wherek∈𝒦=\{task,novel,cover\}k\\in\\mathcal\{K\}=\\\{task,novel,cover\\\}, and trained with labelsyi,kt∈\[0,1\]y\_\{i,k\}^\{t\}\\in\[0,1\]:
\(3\)ℒscore=−∑k∈𝒦λk∑i∈𝒟twi,ktlogpθk\(yi,kt∣o^it\)∑i∈𝒟twi,kt,\\mathcal\{L\}\_\{score\}=\-\\sum\_\{k\\in\\mathcal\{K\}\}\\lambda\_\{k\}\\frac\{\\sum\_\{i\\in\\mathcal\{D\}^\{t\}\}w\_\{i,k\}^\{t\}\\log p\_\{\\theta\_\{k\}\}\(y\_\{i,k\}^\{t\}\\mid\\hat\{o\}\_\{i\}^\{t\}\)\}\{\\sum\_\{i\\in\\mathcal\{D\}^\{t\}\}w\_\{i,k\}^\{t\}\},where𝒟t\\mathcal\{D\}^\{t\}indexes the candidates at timettandλk\\lambda\_\{k\}are loss weights\. The labels are defined in hindsight: a candidate is task\-relevant if it becomes the target of an interaction within the nextWWsteps, novelty reflects how rare its semantic category is among the current candidates, and coverage favors candidates that spread insertions over the observation instead of concentrating them in one region\. The weightswi,ktw\_\{i,k\}^\{t\}are uniform, except that the task term is reweighted by the policy advantageAdvit\\text\{Adv\}\_\{i\}^\{t\}to emphasize important steps\.
Dynamic graph fusion\.Each existing nodevjt−1v\_\{j\}^\{t\-1\}is associated with at most one admitted candidate through an identity test\. The best\-matching candidate updates the node’s attributes and its last\-seen stepljl\_\{j\}, and unmatched candidates are inserted as new nodes\. When the graph reaches its capacity, the nodes unseen for the longest time are evicted\. Finally, the edge set is rebuilt by a relation function,ℰt=ℛ\(𝒱t\)\\mathcal\{E\}^\{t\}=\\mathcal\{R\}\(\\mathcal\{V\}^\{t\}\), and the messages exchanged along the edges are learned by EG\-RSSM\.
### 3\.2\.Embodied Graph Recurrent State Space Model \(EG\-RSSM\)
Most world models learn dynamics in an entangled latent state, where objects, background, and noise are mixed and inter\-object relations are not modeled explicitly, which makes the learned dynamics brittle to small variations\. Since our embodied graph already filters redundancy, we predict and update the environment state directly in the graph\. EG\-RSSM maintains a latent state for each node and learns relation\-aware node\-wise dynamics, propagating interaction effects along edges\.
Node\-wise latent state\.We represent the latent state of each nodevjtv\_\{j\}^\{t\}as\{hjt,zjt\}\\\{h\_\{j\}^\{t\},z\_\{j\}^\{t\}\\\}, wherehjth\_\{j\}^\{t\}is the deterministic latent state andzjtz\_\{j\}^\{t\}is the stochastic latent state\. This node\-wise factorization encourages the dynamics modelfdynf\_\{dyn\}to operate on the retained, task\-relevant objects inGtG^\{t\}, avoiding entangled modeling of dense background content and thereby reducing redundancy in long\-horizon prediction\.
Graph\-conditioned recurrent state transition\.For each nodevjtv\_\{j\}^\{t\}, conditioned on the graph\-level actionAt−1A^\{t\-1\}and current node attributesojto\_\{j\}^\{t\}, we obtain node embeddings viaxjt=fenc\(ojt\)x\_\{j\}^\{t\}=f\_\{enc\}\(o\_\{j\}^\{t\}\), and further incorporate relational information by computing a context representationcjtc\_\{j\}^\{t\}through message aggregation over neighborhood𝒩\(j\)\\mathcal\{N\}\(j\):
\(4\)cjt=AGG\(\{ϕ\(xjt,xkt\)\|k∈𝒩\(j\)\}\),c\_\{j\}^\{t\}=AGG\(\\\{\\phi\(x\_\{j\}^\{t\},x\_\{k\}^\{t\}\)\|k\\in\\mathcal\{N\}\(j\)\\\}\),whereϕ\(⋅\)\\phi\(\\cdot\)is a learnable message function that produces neighbor messages\. The deterministic transition is then calculated as follows:
\(5\)hjt=fdyn\(hjt−1,zjt−1,cjt−1,At−1\)\.h\_\{j\}^\{t\}=f\_\{dyn\}\(h\_\{j\}^\{t\-1\},z\_\{j\}^\{t\-1\},c\_\{j\}^\{t\-1\},A^\{t\-1\}\)\.We learn the node\-wise recurrent state transition functionfdynf\_\{dyn\}with two distributions over the stochastic state: a posterior distributionq\(zjt\|hjt,xjt,cjt\)q\(z\_\{j\}^\{t\}\|h\_\{j\}^\{t\},x\_\{j\}^\{t\},c\_\{j\}^\{t\}\)relying on the current observation\(xjt,cjt\)\(x\_\{j\}^\{t\},c\_\{j\}^\{t\}\), and a prior distributionp\(z^jt\|hjt\)p\(\\hat\{z\}\_\{j\}^\{t\}\|h\_\{j\}^\{t\}\)that depends only on the recurrent state\. Two complementary KL regularizers are applied to align the prior and posterior\. Specifically, the dynamic loss trains the prior distribution to match the inference posterior:
\(6\)ℒdyn=∑t∑j=1NKL\[sg\(qω\(zjt\|hjt,xjt,cjt\)\)\|\|pω\(z^jt\|hjt\)\],\\mathcal\{L\}\_\{dyn\}=\\sum\_\{t\}\\sum\_\{j=1\}^\{N\}KL\[sg\(q\_\{\\omega\}\(z\_\{j\}^\{t\}\|h\_\{j\}^\{t\},x\_\{j\}^\{t\},c\_\{j\}^\{t\}\)\)\|\|p\_\{\\omega\}\(\\hat\{z\}\_\{j\}^\{t\}\|h\_\{j\}^\{t\}\)\],
wheresg\(⋅\)sg\(\\cdot\)represents the stop gradient operator andω\\omegais the dynamic model parameters\. Conversely, the representation loss regularizes the posterior to remain consistent with the predictive dynamics along real trajectories:
\(7\)ℒrep=∑t∑j=1NKL\[qω\(zjt\|hjt,xjt,cjt\)\|\|sg\(pω\(z^jt\|hjt\)\)\]\.\\mathcal\{L\}\_\{rep\}=\\sum\_\{t\}\\sum\_\{j=1\}^\{N\}KL\[q\_\{\\omega\}\(z\_\{j\}^\{t\}\|h\_\{j\}^\{t\},x\_\{j\}^\{t\},c\_\{j\}^\{t\}\)\|\|sg\(p\_\{\\omega\}\(\\hat\{z\}\_\{j\}^\{t\}\|h\_\{j\}^\{t\}\)\)\]\.
Althoughℒdyn\\mathcal\{L\}\_\{dyn\}andℒrep\\mathcal\{L\}\_\{rep\}regularize node\-wise latent transitions by aligning the prior and posterior distributions, they do not constrain interaction\-induced structural changes\. Therefore, we introduce an auxiliary topology objective that targets such structural events, complementing the previous losses with direct structural supervision\. Specifically, we predict the structural outcomebtb^\{t\}of each interaction, and optimize
\(8\)ℒtopo\(ρ\)=−∑tlogpρ\(bt∣hkt−1,zkt−1,At−1\),\\mathcal\{L\}\_\{topo\}\(\\rho\)=\-\\sum\_\{t\}\\log p\_\{\\rho\}\\\!\\left\(b^\{t\}\\mid h\_\{k\}^\{t\-1\},z\_\{k\}^\{t\-1\},A^\{t\-1\}\\right\),
wherekkdenotes the node being interacted with at timet−1t\-1\. We instantiate the structural outcome as node birth in the embodied domains \(Appendix[A\.1](https://arxiv.org/html/2611.24115#A1.SS1)\) and the page reached after the interaction on the Web \(Appendix[B](https://arxiv.org/html/2611.24115#A2)\)\.
Dual Prediction\.To ensure that the imagined rollouts remain interpretable and predictive, the latent state should be decodable into a compact graph state\. However, the previous objectives primarily regularize latent evolution and do not guarantee that the latent variables preserve task\-relevant semantics\. We therefore introduce a*dual prediction loss*that supervises both \(i\) environment attributes and \(ii\) agent attributes\. Concretely, given the node\-wise latent statehjt,zjth\_\{j\}^\{t\},z\_\{j\}^\{t\}, we predict the attributes of environment and agent nodes \(Appendices[A\.1](https://arxiv.org/html/2611.24115#A1.SS1)and[B](https://arxiv.org/html/2611.24115#A2)\)\. Denote environment node attributes asojto\_\{j\}^\{t\}and agent attributes asoato\_\{a\}^\{t\}, the decoders produceo^jt∼pψe\(ojt\|hjt,zjt\)\\hat\{o\}\_\{j\}^\{t\}\\sim p\_\{\\psi\_\{e\}\}\(o\_\{j\}^\{t\}\|h\_\{j\}^\{t\},z\_\{j\}^\{t\}\)ando^at∼pψa\(oat\|hat,zat\)\\hat\{o\}\_\{a\}^\{t\}\\sim p\_\{\\psi\_\{a\}\}\(o\_\{a\}^\{t\}\|h\_\{a\}^\{t\},z\_\{a\}^\{t\}\), where\(hat,zat\)\(h\_\{a\}^\{t\},z\_\{a\}^\{t\}\)are the agents’ deterministic and stochastic latent states,ψe\\psi\_\{e\}andψa\\psi\_\{a\}denote the learnable parameters of the environment node decoder and the agent decoder, respectively\. We train the decoders by maximizing the likelihood of the corresponding targets, yielding:
\(9\)ℒpred,e\(ψe\)=∑t∑j=1N−logpψe\(ojt\|hjt,zjt\),\\mathcal\{L\}\_\{pred,e\}\(\\psi\_\{e\}\)=\\sum\_\{t\}\\sum\_\{j=1\}^\{N\}\-\\log p\_\{\\psi\_\{e\}\}\(o\_\{j\}^\{t\}\|h\_\{j\}^\{t\},z\_\{j\}^\{t\}\),
\(10\)ℒpred,a\(ψa\)=∑t−logpψa\(oat\|hat,zat\)\.\\mathcal\{L\}\_\{pred,a\}\(\\psi\_\{a\}\)=\\sum\_\{t\}\-\\log p\_\{\\psi\_\{a\}\}\(o\_\{a\}^\{t\}\|h\_\{a\}^\{t\},z\_\{a\}^\{t\}\)\.\(11\)ℒpred=βeℒpred,e\+βaℒpred,a\\mathcal\{L\}\_\{pred\}=\\beta\_\{e\}\\mathcal\{L\}\_\{pred,e\}\+\\beta\_\{a\}\\mathcal\{L\}\_\{pred,a\}Finally, the total loss is the weighted combination of the aforementioned losses:
\(12\)ℒeg=∑q∈𝒬λqℒq,\\mathcal\{L\}\_\{eg\}=\\sum\_\{q\\in\\mathcal\{Q\}\}\\lambda\_\{q\}\\mathcal\{L\}\_\{q\},where𝒬=\{dyn,rep,topo,pred\}\\mathcal\{Q\}=\\\{dyn,rep,topo,pred\\\}, andλq\\lambda\_\{q\}is the corresponding coefficient\. In this way, we jointly stabilize node\-wise latent dynamics, supervise interaction\-induced structural events, and align node representations with their corresponding attributes in an end\-to\-end manner\.
### 3\.3\.Decision Making with RWM
In embodied control, imagined graph rollouts train a policy over graph\-level macro\-actions\. While in Web agents, the imagination helps a verifier compare candidate browser actions\. The graph construction in each setting is given in Appendices[A\.1](https://arxiv.org/html/2611.24115#A1.SS1)and[B](https://arxiv.org/html/2611.24115#A2)\.
#### 3\.3\.1\.Embodied Control
In embodied control, the embodied graph is built from the segmented objects of the current observation, and an interaction is one of two types, moving to or acting on a target object \(Appendix[A\.1](https://arxiv.org/html/2611.24115#A1.SS1)\)\. Given the learned EG\-RSSM and the imagination horizonKK, we roll out graph latents\{ht,zt\}t=tsts\+K\\\{h^\{t\},z^\{t\}\\\}\_\{t=t\_\{s\}\}^\{t\_\{s\}\+K\}from the posterior at timetst\_\{s\}\. As in Dreamer\([Hafner and et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib4)\), a graph policyπg\(At∣ht,zt\)\\pi\_\{\\text\{g\}\}\(A^\{t\}\\mid h^\{t\},z^\{t\}\)is trained with actor\-critic learning on these imagined trajectories to maximize the predicted return \(details in Appendix[A\.5](https://arxiv.org/html/2611.24115#A1.SS5)\)\. EveryHHenvironment steps, a macro\-actionAt=\(it,vtart\)A^\{t\}=\(i^\{t\},v\_\{tar\}^\{t\}\)is selected underπg\\pi\_\{\\text\{g\}\}\. After that, an executor produces primitive actions\{at,…,at\+H−1\}\\\{a\_\{t\},\\ldots,a\_\{t\+H\-1\}\\\}in the original environment action space to achieve the proposed macro\-actionAtA^\{t\}\. It keeps the primitive action space unchanged and can be trained or replaced independently\. In this case, we use a 200M\-parameter DreamerV3 as the executor\.
#### 3\.3\.2\.Web Agents
On webpages, the embodied graph is built from the DOM elements of the current page, and an interaction is a native browser action \(click, fill, press, scroll, or select\) on a target element \(Appendix[B](https://arxiv.org/html/2611.24115#A2)\)\. At decision time, a frozen web agent, WebRL\-Llama\-3\.1\-8B\([Qi et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib48)\), proposes several candidate actions\. From the common posterior, RWM starts the corresponding independent imagined branch with each candidate action\. A verifier, Qwen2\.5\-VL\-32B\-Instruct\([Bai et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib49)\), receives the imagined rollouts to make the final decision\.
## 4\.Experiments
### 4\.1\.Experiments on Embodied Control
#### 4\.1\.1\.Setup
We evaluate on two long\-horizon embodied benchmarks:Minecraft\([Hill et al\., 2023](https://arxiv.org/html/2611.24115#bib.bib14)\)andCrafter\([Hafner, 2022](https://arxiv.org/html/2611.24115#bib.bib27)\)\.
Minecraftis a procedurally generated open\-world*3D*sandbox where the agent must navigate large biomes and cave systems, manage survival signals \(e\.g\., health and hunger\), and interact with the environment by breaking blocks and fighting hostile mobs\. We use*diamond acquisition*as the target task, where agents need to explore, mine, and obtain diamonds under sparse and delayed rewards, and report the intermediate targets stone, iron and gold\. The agent starts with the required tools so that it can focus on discovering and mining the targets\. Following prior work\([Hafner and et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib4)\), we increase the block breaking speed to accelerate training and form the observation and action space\. Additionally, RWM obtains its graph candidates from the observed images with a learned object\-segmentation front\-end \(Appendix[A\.2](https://arxiv.org/html/2611.24115#A1.SS2)\)\.
Crafter\([Hafner, 2022](https://arxiv.org/html/2611.24115#bib.bib27)\)distills similar mechanics into a compact*2D*setting in which the agent must gather materials, craft increasingly advanced tools, and handle hazards to unlock 22 achievements\. Its setup is given in Appendix[A\.7](https://arxiv.org/html/2611.24115#A1.SS7)\.
Baselines\.We compare with DreamerV3\([Hafner and et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib4)\), a state\-of\-the\-art model\-based agent that imagines trajectories in a latent space learned from high\-dimensional observations and was the first to obtain diamonds in Minecraft without human data\. LS\-Imagine\([Li et al\., 2025a](https://arxiv.org/html/2611.24115#bib.bib28)\), which augments imagination with long\-horizon “jumpy” transitions for open\-world exploration\. And the model\-free PPO\([Schulman et al\., 2017](https://arxiv.org/html/2611.24115#bib.bib16)\)and Rainbow\([Hessel et al\., 2018](https://arxiv.org/html/2611.24115#bib.bib15)\)\.
Implementation details\.A macro\-action is selected everyH=50H=50environment steps\. RWM has 4M parameters, and the executor is a 200M\-parameter DreamerV3\([Hafner and et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib4)\)\. The complete hyperparameters are provided in Table[5](https://arxiv.org/html/2611.24115#A1.T5)\. The computational complexity and measured computational cost are reported in Appendix[A\.6](https://arxiv.org/html/2611.24115#A1.SS6)\.
For Minecraft, all agents are trained for 5M environment steps with a maximum episode length of 36,000 steps, and we report results after 1M, 2M, 3M, 4M and 5M steps\. For Crafter, we follow the default setting in\([Hafner, 2022](https://arxiv.org/html/2611.24115#bib.bib27)\), training agents for 1M steps with a maximum episode length of 10,000\. Unless otherwise stated, we apply the same environment configuration to all baselines\. Results are the mean±\\pmstandard deviation over 3 training seeds\.
Metrics\.For Minecraft, we report the Success Rate \(SR\), the fraction of episodes in which the agent obtains the target item, and the Success Length \(SL\), the earliest time step at which the target is obtained, averaged over successful episodes \(“–” when no episode succeeds\)\. For Crafter, we follow the standard evaluation protocol and report reward and the aggregated Crafter score, defined as the geometric mean over achievement success rates to emphasize broad and compositional skill acquisition\.
#### 4\.1\.2\.Results
Table 1\.Minecraft results after 1M–5M environment steps: success rate \(SR↑\\uparrow, %\) and success length \(SL↓\\downarrow, steps, “– if none\) are reported mean±\\pmstd over 3 seeds\. Best in bold, second best underlined\. Ranks are computed among methods with non\-zero SR\.Table[1](https://arxiv.org/html/2611.24115#S4.T1)shows that RWM outperforms the baselines, with the largest gains on long\-horizon targets\.
For relatively simple tasks, such as collecting stone and iron, their performances remain similar since these tasks can be solved through simple random exploration without requiring a deep understanding of environment dynamics\. The performance gap becomes more evident on challenging tasks, such as collecting gold and diamonds\. These materials are buried deeper and associated with sparse rewards\. Among the strong baselines, RWM outperforms the others by capturing abstract object relations and leveraging sparse feedback more effectively, resulting in continued gains and shorter average success lengths\. Notably, it is the only method that finds diamonds within the given 5M training budget\.
To further assess the generality of our approach beyond*3D*scenarios, we evaluate RWM on Crafter\. The results are provided in Table[3](https://arxiv.org/html/2611.24115#S4.T3), which exhibit a similar trend to those on Minecraft\. Compared with the baselines, RWM achieves the best performance in terms of both Crafter score and episodic reward\. Since the Crafter score is defined as the geometric mean over achievement success rates\([Hafner, 2022](https://arxiv.org/html/2611.24115#bib.bib27)\), the improvement in both reward and score suggests that RWM not only improves the overall episode performance, but also helps the agent make broader progress across diverse achievements\. A further analysis of the learned agent skills is provided in Appendix[A\.7](https://arxiv.org/html/2611.24115#A1.SS7), where RWM achieves comparable performance on simple early\-stage achievements and obtains clear gains on harder long\-horizon achievements that require multi\-step resource collection and crafting\.
Table 2\.Ablation study in the Minecraft diamond task\. Variant 1: w/o admission scorer\. Variant 2: w/o relational propagation\. Success rate \(SR, %\) and success length \(SL, steps\) are reported by mean±\\pmstd over 3 seeds\.Table 3\.Experimental results in Crafter after 1M environment steps \(mean±\\pmstd\)\.
#### 4\.1\.3\.Imagination Drift Analysis
To directly examine whether RWM mitigates compounding prediction errors during long\-horizon imagination, we evaluate the accumulating imagination drift in Fig\.[3](https://arxiv.org/html/2611.24115#S4.F3)\. Starting from the same posterior state, each model performs open\-loop imagination for up to 500 environment steps, and we measure the discrepancy between the imagined state and the corresponding ground\-truth future state through the decoded predictions\. Specifically, since the decoders and prediction targets vary across methods, the raw MSE values are not directly comparable\. For a fair comparison, we normalize the MSE at each imagination step by the MSE at the first imagination step \(Normalized MSE\), so that the metric reflects the relative growth of prediction error, i\.e\., the magnitude of imagination drift\. As shown in Fig\.[3](https://arxiv.org/html/2611.24115#S4.F3), the drift increases as the imagination horizon becomes longer, confirming that long\-horizon rollout is inherently vulnerable to error accumulation\. However, RWM exhibits much slower drift growth than the dense\-observation baseline \(DreamerV3\)\. This indicates that the proposed embodied graph abstraction and relation\-aware EG\-RSSM provide a more stable predictive structure, thereby extending the effective imagination horizon and supporting more reliable macro\-action selection\. In addition, the smaller variance of RWM further suggests that our method is more stable across rollout samples\.
Figure 3\.Open\-loop imagination drift under different imagination lengths\. Compared with DreamerV3, RWM maintains a nearly stable prediction error over long rollouts, showing that our proposed method effectively reduces compounding imagination drift\.This is consistent with where RWM helps in Table[1](https://arxiv.org/html/2611.24115#S4.T1): short rollouts suffice for simple targets, whereas diamond acquisition requires coordinating many interdependent interactions, where small rollout errors lead to wrong macro\-action choices\.
#### 4\.1\.4\.Ablation Study
To isolate the contributions of key design choices inRWM, we conduct ablation studies by constructing two variants:
1. \(1\)w/o admission scorer in TA\-EGC\.We remove the admission scorer in task\-aware embodied graph constructor\. Instead of selecting task\-relevant nodes, we update the embodied graph with all nodes extracted from the local observation at every step\. This variant tests whether the proposed sparse, task\-aware embodied graph constructor is necessary to filter redundancy and preserve the informative structure\.
2. \(2\)w/o relational propagation in EG\-RSSM\.We remove relational propagation in EG\-RSSM by disabling message passing across edges such that each node latent is updated independently without relational context\. This variant keeps the same object\-level graph state, constructor, and node\-wise latent dynamics, and removes only the relations\. It therefore separates the value of explicit relations from the value of moving from pixels to an object\-level state\.
We evaluate the two ablations on the diamond task\. As shown in Table[2](https://arxiv.org/html/2611.24115#S4.T2), removing either component consistently degrades performance, indicating that both of them are crucial for RWM\. Variant 1 \(without the task\-aware admission scorer\) completely fails to obtain the diamond\. This suggests that without controlled admissions, embodied graph may becomes unstable and changes frequently, making it difficult to learn consistent transition dynamics\. Variant 2 \(without relational propagation\) exhibits a substantially lower SR, indicating that relational context contributes beyond the object\-level state representation\. The gap between Variant 2 and the full model widens as training progresses\. Their mean success rates are close at 2M steps, but by 4M steps, the full model reaches 5\.72%, while Variant 2 remains at 1\.66%\. At 5M steps, their success rates reach 6\.94% and 2\.48%, respectively\. Both models can discover successful trajectories, but the full model shows larger improvements with continued training\. This pattern suggests that relational propagation contributes beyond initial task discovery, helping the agent make better use of additional training experience\.
### 4\.2\.Evaluation on Interactive Webpages
Beyond embodied control, we test whether the same relational world model carries over to Web agents, where page elements and their DOM relations form the graph\.
#### 4\.2\.1\.Benchmark\.
We evaluate 35 content\-management tasks \(CMS35\) of WebArena\-Lite\([Liu et al\., 2024b](https://arxiv.org/html/2611.24115#bib.bib46);[Zhou et al\., 2024](https://arxiv.org/html/2611.24115#bib.bib53)\), which are performed on the administration back end of an e\-commerce website \(Magento\)\. Each episode starts from a freshly deployed site and is limited to 30 steps\. We use the official observation and action spaces and the official evaluators\.
#### 4\.2\.2\.Training data\.
RWM is trained offline on 94 recorded browser trajectories collected by a frozen A3\-Qwen3\.5\-9B Web agent\([Lù and Reddy, 2026](https://arxiv.org/html/2611.24115#bib.bib47)\)on 12 WebArena CMS tasks\. None of these tasks, nor their task templates appears in CMS35\.
#### 4\.2\.3\.Baselines\.
We compare RWM with a pixel\-based world model under the decision procedure of Section[3\.3\.2](https://arxiv.org/html/2611.24115#S3.SS3.SSS2): WebRL\-Llama\-3\.1\-8B\([Qi et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib48)\)proposes the candidate actions, a world model forecasts the consequence of each candidate, and another verifier \(Qwen2\.5\-VL\-32B\-Instruct\([Bai et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib49)\)\) selects one of them\. The three variants differ only in what the verifier receives for each candidate\. Verifier\-only branch receives the candidates without any forecast\. Verifier \+ image RSSM additionally receives the next screen predicted by a DreamerV3\-style pixel world model on 288×512 screenshots, trained on the same trajectories with the same number of updates and history length\. Verifier \+ RWM receives the element\-level effects predicted by RWM\. Each task is run once per variant from a freshly deployed site\. For reference, we also report the published WebArena\-Lite results of WebRL and other agents\.
#### 4\.2\.4\.Decision\-making on CMS35\.
Table[4](https://arxiv.org/html/2611.24115#S4.T4)reports task success on CMS35\. Without forecasts, the verifier\-only branch solves 45\.7% of the tasks, even worse than the pure WebRL\-Llama\-3\.1\-8B\. Pixel forecasts raise this to 51\.4%, and RWM further improves it to 60\.0%\. This indicates that without proper evidence on the consequences of the candidate actions, a general LLM struggles to tell which action is correct\. The predicted screen conveys the consequence of an action only implicitly\. Changes in the state of a single element are hard to read from it\. Whereas RWM explicitly indicates which elements change and how, which the verifier can relate directly to the task\.
#### 4\.2\.5\.Case Study
We further examine some cases to verify how RWM takes effect to increase the success rate and how other methods fails\.
Case 1: searching customers\.The task is “Find the customer name and email with phone number 8015551212”, and the agent is on the customer list \(Fig\.[4](https://arxiv.org/html/2611.24115#S4.F4)\)\. WebRL proposes the first action to click the*Filters*button, for which RWM forecasts that the search box stays empty\. Therefore, the verifier instead selects typing8015551212into the*Search by keyword*box, for which RWM forecasts that the box receives focus and its value becomes8015551212\. This helps the RWM branch to success in this task\. Instead, the image RSSM predicts a blurred page for both candidates\. The search box shows no text, and the two predicted screens are nearly indistinguishable\. This makes the verifier fails to choose the right action\.
Case 2: configuring a report\.The task asks for the monthly count of successful orders in 2022, and the agent is on the Orders Report page\. WebRL proposes a wrong action to type12/31/2022into the*From*field\. The verifier instead selectsmonthin the*Period*drop\-down, for which RWM forecasts that Period changes fromdaytomonthwith all other fields unchanged, again matching the actual next state\. The predicted screens of the image RSSM do not show this change and are again nearly identical across the two candidates\.
In both cases the initial WebRL proposal is not good enough, and the action consequence is a field value that covers a small portion of the screen, which a pixel forecast does not resolve but an element\-level forecast states directly\. And this is how RWM help with the higher success rate\.
Figure 4\.Case study on CMS35\. Top: Case 1, typing a phone number into the search box\. Bottom: Case 2, selecting the report period\. Each case is enlarged around the changed element and shows \(a\) the current screen, \(b\) the actual next screen after the verifier’s choice, \(c\) the image RSSM’s prediction for the verifier’s choice, and \(d\) its prediction for WebRL’s proposal\. RWM forecasts the new values exactly \(8015551212in the search box andmonthin the Period drop\-down\), whereas the image RSSM’s predictions are blurred and nearly identical for the two candidates\.Table 4\.Success rate \(%\) on WebArena\-Lite CMS35\. Upper block: published results\([Qi et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib48)\)\. Lower block: our runs, in which WebRL\-Llama\-3\.1\-8B proposes the candidates and the same verifier selects one\. The variants differ only in the forecasts given to the verifier\.
## 5\.Related Work
World Models\.World models learn internal representations\([Bruce et al\., 2024](https://arxiv.org/html/2611.24115#bib.bib7);[Chen et al\., 2022](https://arxiv.org/html/2611.24115#bib.bib8);[Robine et al\., 2023](https://arxiv.org/html/2611.24115#bib.bib11);[Wang et al\., 2024](https://arxiv.org/html/2611.24115#bib.bib9)\)and predictive dynamics\([Hafner et al\., 2020](https://arxiv.org/html/2611.24115#bib.bib2);[Hafner and et al\., 2021](https://arxiv.org/html/2611.24115#bib.bib3);[Okada and Taniguchi, 2022](https://arxiv.org/html/2611.24115#bib.bib10);[Wu et al\., 2022](https://arxiv.org/html/2611.24115#bib.bib6);[Zhan et al\., 2026](https://arxiv.org/html/2611.24115#bib.bib58)\)to enable imagination\-based reasoning and planning\. Early work\([Ha and Schmidhuber, 2018](https://arxiv.org/html/2611.24115#bib.bib1)\)simulates future observations with latent generative RNNs, while recent methods scale capacity and generalization\. PWM\([Georgiev et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib33)\)leverages large pre\-trained world models for policy learning across continuous\-control tasks, STORM\([Zhang et al\., 2023](https://arxiv.org/html/2611.24115#bib.bib5)\)combines stochastic latent dynamics with transformers for strong multi\-step prediction on Atari, and Dreamer\-style agents\([Hafner et al\., 2020](https://arxiv.org/html/2611.24115#bib.bib2);[Hafner and et al\., 2021](https://arxiv.org/html/2611.24115#bib.bib3);[Hafner and et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib4)\)learn from imagined trajectories for sample\-efficient long\-horizon credit assignment\. LS\-Imagine\([Li et al\., 2025a](https://arxiv.org/html/2611.24115#bib.bib28)\)further extends imagination in open\-world settings via long\-horizon “jumpy” transitions\. Object\-centric world models, such as C\-SWM\([Kipf et al\., 2020](https://arxiv.org/html/2611.24115#bib.bib38)\), OP3\([Veerapaneni et al\., 2020](https://arxiv.org/html/2611.24115#bib.bib50)\)and SlotFormer\([Wu et al\., 2023](https://arxiv.org/html/2611.24115#bib.bib51)\), factorize the state into object slots discovered from pixels, mostly in small synthetic scenes with a fixed number of objects\. Most world models still rely on dense pixel\-level or entangled latent representations and lack explicit relational structure, which can limit rollout stability and the effective imagination horizon\. Open\-world tasks further require the set of modeled objects to change online as the agent explores\.
Graphs in Embodied AI\.Graphs provide structured representations of entities and relations for embodied perception and reasoning\. Scene\-graph methods such as SG\-Nav\([Yin et al\., 2024](https://arxiv.org/html/2611.24115#bib.bib29)\), ESCA\([Huang et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib30)\), and GraphEQA\([Saxena et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib34)\)support navigation, grounding, and question answering with 3D semantic context\. EmbodiedVSR\([Zhang et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib52)\)updates a dynamic scene graph after each action to guide the chain\-of\-thought reasoning of a multimodal language model, but the graph serves as input for reasoning rather than as the state of a learned dynamics model\. Graphs are also used as supervision, e\.g\., Scene Graph Contrastive loss\([Singh et al\., 2022](https://arxiv.org/html/2611.24115#bib.bib42)\), and as knowledge memories for instruction following, e\.g\., Optimus\-1\([Li et al\., 2024](https://arxiv.org/html/2611.24115#bib.bib31)\)and ExRAP\([Yoo et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib32)\)\. In model\-based and hierarchical RL, graph abstractions have been used for planning with skills, including Deep Skill Graphs\([Bagaria et al\., 2021](https://arxiv.org/html/2611.24115#bib.bib40)\)and intrinsically motivated temporally abstract graph\-based world models\([Bagaria et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib41)\)\. In these methods, nodes represent subgoals or abstract states, and edges represent skills that move the agent between them through sequences of primitive actions\. The graph records which subgoals can be reached through skill execution, allowing the planner to combine several skills into a longer plan\. Despite these advances, these methods primarily use graphs for perception, reasoning, or skill\-level planning, rather than modeling interaction\-driven changes in entity states over long imagined rollouts\.
World Models for Web Agents\.Web agents act on realistic websites through browser actions such as clicking, typing and selecting\. WebArena\([Zhou et al\., 2024](https://arxiv.org/html/2611.24115#bib.bib53)\)provides self\-hosted websites for e\-commerce, content management, forums, code hosting and maps, and WebArena\-Lite\([Liu et al\., 2024b](https://arxiv.org/html/2611.24115#bib.bib46)\)is a human\-verified subset of its tasks\. WebRL\([Qi et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib48)\)trains LLM agents on WebArena\-Lite with online reinforcement learning, using a self\-evolving curriculum and an outcome\-supervised reward model\. To anticipate the consequences of actions before committing to them, recent work equips Web agents with world models\. WMA\([Chae et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib43)\)trains an LLM to predict the change of the observation caused by an action in natural language and uses these predictions to select actions at inference time, and WebDreamer\([Gu et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib44)\)prompts an LLM to simulate the outcomes of candidate actions and plans by scoring the simulated outcomes\. However, these world models describe the next state as free\-form text and do not explicitly represent page elements and their relations, which makes it difficult to tell which elements an action changes and to verify a prediction against the actual page\.
## 6\.Limitations
RWM relies on an entity\-level front\-end: object\-segmentation models trained with object labels in Minecraft and Crafter, and the DOM on the Web\. The result may be affected by the quality of the front\-end, and transfer to real\-world settings requires a detector for the relevant objects\. Moreover, predefined relations \(KnnK\_\{nn\}spatial neighbors, DOM links\) are adopted as edges, which may be unavailable in some cases\. Learning the edges is left for future works\.
## 7\.Conclusion
Existing world models often struggle with long\-horizon tasks because prediction errors accumulate during imagination, leading to drift in long rollouts\. We proposed theRelational World Model \(RWM\), a graph world model that imagines overEmbodied Graph, a sparse relational state abstraction\. RWM combines a*Task\-Aware Embodied Graph Constructor*with an*Embodied Graph Recurrent State Space Model*, and is instantiated for embodied control and Web agents\. In Minecraft, RWM’s open\-loop prediction error grows much more slowly than that of a pixel world model, and the resulting agent is the only one in our comparison that obtains diamonds within the given budget\. On webpages, RWM forecasts element\-level action consequences\. Given these forecasts, a verifier selects better actions, and solves more tasks on WebArena\-Lite\. These findings suggest that sparse relational abstractions are a useful way to extend the effective horizon of world models, and their value grows with the difficulty of the task\.
## References
- A\. Bagaria, A\. D\. M\. Koch, R\. Rodriguez\-Sanchez, S\. Lobel, and G\. KonidarisIntrinsically motivated discovery of temporally abstract graph\-based models of the world\.Reinforcement Learning Journal6,pp\. 2115–2134\.Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p2.1)\.
- Bagariaet al\.\(2021\)A\. Bagaria, J\. K\. Senthil, and G\. KonidarisSkill discovery for exploration and planning using deep skill graphs\.InProceedings of the 38th International Conference on Machine Learning,M\. Meila and T\. Zhang \(Eds\.\),Proceedings of Machine Learning Research, Vol\.139,pp\. 521–531\.External Links:[Link](https://proceedings.mlr.press/v139/bagaria21a.html)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p2.1)\.
- Baiet al\.\(2025\)S\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge, S\. Song, K\. Dang, P\. Wang, S\. Wang, J\. Tang, H\. Zhong, Y\. Zhu, M\. Yang, Z\. Li, J\. Wan, P\. Wang, W\. Ding, Z\. Fu, Y\. Xu, J\. Ye, X\. Zhang, T\. Xie, Z\. Cheng, H\. Zhang, Z\. Yang, H\. Xu, and J\. LinQwen2\.5\-vl technical report\.External Links:2502\.13923,[Link](https://arxiv.org/abs/2502.13923)Cited by:[§3\.3\.2](https://arxiv.org/html/2611.24115#S3.SS3.SSS2.p1.1),[§4\.2\.3](https://arxiv.org/html/2611.24115#S4.SS2.SSS3.p1.1)\.
- Bruceet al\.\(2024\)J\. Bruce, M\. Dennis, A\. Edwards, J\. Parker\-Holder, Y\. Shi, E\. Hughes, M\. Lai, A\. Mavalankar, R\. Steigerwald, C\. Apps, Y\. Aytar, S\. Bechtle, F\. Behbahani, S\. Chan, N\. Heess, L\. Gonzalez, S\. Osindero, S\. Ozair, S\. Reed, J\. Zhang, K\. Zolna, J\. Clune, N\. de Freitas, S\. Singh, and T\. RocktäschelGenie: generative interactive environments\.arXiv preprint arXiv:2402\.15391\.Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Burchi and Timofte \(2024\)M\. Burchi and R\. TimofteMuDreamer: learning predictive world models without reconstruction\.External Links:2405\.15083,[Link](https://arxiv.org/abs/2405.15083)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p2.1)\.
- Chaeet al\.\(2025\)H\. Chae, N\. Kim, K\. T\. Ong, M\. Gwak, G\. Song, J\. Kim, S\. Kim, D\. Lee, and J\. YeoWeb agents with world models: learning and leveraging environment dynamics in web navigation\.External Links:2410\.13232,[Link](https://arxiv.org/abs/2410.13232)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p3.1)\.
- Chenet al\.\(2022\)C\. Chen, Y\. Wu, J\. Yoon, and S\. AhnTransDreamer: reinforcement learning with transformer world models\.arXiv preprint arXiv:2202\.09481\.Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Chenet al\.\(2018\)L\. Chen, Y\. Zhu, G\. Papandreou, F\. Schroff, and H\. AdamEncoder\-decoder with atrous separable convolution for semantic image segmentation\.External Links:1802\.02611,[Link](https://arxiv.org/abs/1802.02611)Cited by:[§A\.2](https://arxiv.org/html/2611.24115#A1.SS2.p1.1)\.
- Chenet al\.\(2026\)S\. Chen, H\. Guo, S\. Zhu, F\. Zhang, Z\. Huang, J\. Feng, and B\. KangVideo depth anything: consistent depth estimation for super\-long videos\.External Links:2501\.12375,[Link](https://arxiv.org/abs/2501.12375)Cited by:[§A\.2](https://arxiv.org/html/2611.24115#A1.SS2.p1.1)\.
- Denget al\.\(2021\)F\. Deng, I\. Jang, and S\. AhnDreamerPro: reconstruction\-free model\-based reinforcement learning with prototypical representations\.External Links:2110\.14565,[Link](https://arxiv.org/abs/2110.14565)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p2.1)\.
- Feiet al\.\(2025\)Z\. Fei, L\. Ji, S\. Wang, J\. Shi, J\. Gong, and X\. QiuUnleashing embodied task planning ability in llms via reinforcement learning\.arXiv preprint arXiv:2506\.23127\.External Links:[Link](https://arxiv.org/abs/2506.23127)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Fenget al\.\(2025a\)T\. Feng, X\. Wang, F\. Han, L\. Zhang, and W\. ZhuU2UData\+: a scalable swarm uavs autonomous flight dataset for embodied long\-horizon tasks\.External Links:2509\.00055,[Link](https://arxiv.org/abs/2509.00055)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Fenget al\.\(2025b\)T\. Feng, X\. Wang, Y\. Jiang, and W\. ZhuEmbodied ai: from llms to world models\.External Links:2509\.20021,[Link](https://arxiv.org/abs/2509.20021)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Fenget al\.\(2025c\)T\. Feng, X\. Wang, Z\. Zhou, R\. Wang, Y\. Zhan, G\. Li, Q\. Li, and W\. ZhuEvoAgent: self\-evolving agent with continual world model for long\-horizon tasks\.External Links:2502\.05907,[Link](https://arxiv.org/abs/2502.05907)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Fenget al\.\(2026\)T\. Feng, X\. Wang, and W\. ZhuSelf\-evolving embodied ai\.External Links:2602\.04411,[Link](https://arxiv.org/abs/2602.04411)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Georgievet al\.\(2025\)I\. Georgiev, V\. Giridhar, N\. Hansen, and A\. GargPWM: policy learning with multi\-task world models\.External Links:2407\.02466,[Link](https://arxiv.org/abs/2407.02466)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Guet al\.\(2025\)Y\. Gu, K\. Zhang, Y\. Ning, B\. Zheng, B\. Gou, T\. Xue, C\. Chang, S\. Srivastava, Y\. Xie, P\. Qi, H\. Sun, and Y\. SuIs your llm secretly a world model of the internet? model\-based planning for web agents\.External Links:2411\.06559,[Link](https://arxiv.org/abs/2411.06559)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p3.1)\.
- Guoet al\.\(2024\)W\. Guo, Z\. Kingston, and L\. E\. KavrakiCaStL: constraints as specifications through llm translation for long\-horizon task and motion planning\.External Links:2410\.22225,[Link](https://arxiv.org/abs/2410.22225)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Ha and Schmidhuber \(2018\)D\. Ha and J\. SchmidhuberWorld models\.arXiv preprint arXiv:1803\.10122\.Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Hafner and et al\. \(2021\)D\. Hafner and et al\.Mastering atari with discrete world models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Hafner and et al\. \(2025\)D\. Hafner and et al\.Mastering diverse control tasks through world models\.Nature640,pp\. 647–653\.Cited by:[§A\.3](https://arxiv.org/html/2611.24115#A1.SS3.p1.1),[Table 5](https://arxiv.org/html/2611.24115#A1.T5),[Table 5](https://arxiv.org/html/2611.24115#A1.T5.4),[§1](https://arxiv.org/html/2611.24115#S1.p1.1),[§3\.3\.1](https://arxiv.org/html/2611.24115#S3.SS3.SSS1.p1.1),[§4\.1\.1](https://arxiv.org/html/2611.24115#S4.SS1.SSS1.p2.1),[§4\.1\.1](https://arxiv.org/html/2611.24115#S4.SS1.SSS1.p4.1),[§4\.1\.1](https://arxiv.org/html/2611.24115#S4.SS1.SSS1.p5.1),[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Hafneret al\.\(2020\)D\. Hafner, T\. Lillicrap, J\. Ba, and M\. NorouziDream to control: learning behaviors by latent imagination\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Hafner \(2022\)D\. HafnerBenchmarking the spectrum of agent capabilities\.International Conference on Learning Representations \(ICLR\)\.Cited by:[§A\.7](https://arxiv.org/html/2611.24115#A1.SS7.p1.1),[§1](https://arxiv.org/html/2611.24115#S1.p4.1),[§4\.1\.1](https://arxiv.org/html/2611.24115#S4.SS1.SSS1.p1.1),[§4\.1\.1](https://arxiv.org/html/2611.24115#S4.SS1.SSS1.p3.1),[§4\.1\.1](https://arxiv.org/html/2611.24115#S4.SS1.SSS1.p6.1),[§4\.1\.2](https://arxiv.org/html/2611.24115#S4.SS1.SSS2.p3.1)\.
- Heet al\.\(2015\)K\. He, X\. Zhang, S\. Ren, and J\. SunDeep residual learning for image recognition\.External Links:1512\.03385,[Link](https://arxiv.org/abs/1512.03385)Cited by:[§A\.2](https://arxiv.org/html/2611.24115#A1.SS2.p1.1)\.
- Hesselet al\.\(2018\)M\. Hessel, J\. Modayil, H\. Van Hasselt, T\. Schaul, G\. Ostrovski, W\. Dabney, D\. Horgan, B\. Piot, M\. Azar, and D\. SilverRainbow: combining improvements in deep reinforcement learning\.InProceedings of the AAAI Conference on Artificial Intelligence,External Links:[Link](https://arxiv.org/abs/1710.02298)Cited by:[§4\.1\.1](https://arxiv.org/html/2611.24115#S4.SS1.SSS1.p4.1)\.
- Hillet al\.\(2023\)W\. Hill, I\. Liu, A\. D\. M\. Koch, D\. Harvey, N\. Kumar, G\. Konidaris, and S\. JamesMineplanner: a benchmark for long\-horizon planning in large minecraft worlds\.arXiv preprint arXiv:2312\.12891\.Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p4.1),[§4\.1\.1](https://arxiv.org/html/2611.24115#S4.SS1.SSS1.p1.1)\.
- Honget al\.\(2025\)H\. Hong, Y\. Qiao, S\. Wang, J\. Liu, and Q\. WuGeneral scene adaptation for vision\-and\-language navigation\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Huanget al\.\(2025\)J\. Huang, A\. Sethi, M\. Kuo, M\. Keoliya, N\. Velingker, J\. Jung, S\. Lim, Z\. Li, and M\. NaikESCA: contextualizing embodied agents via scene\-graph generation\.External Links:2510\.15963,[Link](https://arxiv.org/abs/2510.15963)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p2.1)\.
- Jocheret al\.\(2023\)Ultralytics YOLOv8External Links:[Link](https://github.com/ultralytics/ultralytics)Cited by:[§A\.2](https://arxiv.org/html/2611.24115#A1.SS2.p1.1)\.
- Kimet al\.\(2025\)K\. Kim, J\. Lanier, and R\. FoxMake the pertinent salient: Task\-relevant reconstruction for visual control with distractions\.Reinforcement Learning Journal6,pp\. 1334–1364\.Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p2.1)\.
- Kipfet al\.\(2020\)T\. Kipf, E\. van der Pol, and M\. WellingContrastive learning of structured world models\.External Links:1911\.12247,[Link](https://arxiv.org/abs/1911.12247)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p2.1),[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Liet al\.\(2025a\)J\. Li, Q\. Wang, Y\. Wang, X\. Jin, Y\. Li, W\. Zeng, and X\. YangOpen\-world reinforcement learning over long short\-term imagination\.International Conference on Learning Representations \(ICLR\)\.Cited by:[§4\.1\.1](https://arxiv.org/html/2611.24115#S4.SS1.SSS1.p4.1),[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Liet al\.\(2024\)Z\. Li, Y\. Xie, R\. Shao, G\. Chen, D\. Jiang, and L\. NieOptimus\-1: hybrid multimodal memory empowered agents excel in long\-horizon tasks\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 49881–49913\.External Links:[Document](https://dx.doi.org/10.52202/079017-1580),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/5949a8750a110ce1f0631b1776c500a2-Paper-Conference.pdf)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p2.1)\.
- Liet al\.\(2025b\)Z\. Li, Y\. Xie, R\. Shao, G\. Chen, D\. Jiang, and L\. NieOptimus\-2: multimodal minecraft agent with goal\-observation\-action conditioned policy\.InProceedings of the Computer Vision and Pattern Recognition Conference \(CVPR\),pp\. 9039–9049\.Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Liuet al\.\(2024a\)R\. Liu, W\. Wang, and Y\. YangVolumetric environment representation for vision\-language navigation\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Liuet al\.\(2024b\)X\. Liu, T\. Zhang, Y\. Gu, I\. L\. Iong, Y\. Xu, X\. Song, S\. Zhang, H\. Lai, X\. Liu, H\. Zhao, J\. Sun, X\. Yang, Y\. Yang, Z\. Qi, S\. Yao, X\. Sun, S\. Cheng, Q\. Zheng, H\. Yu, H\. Zhang, W\. Hong, M\. Ding, L\. Pan, X\. Gu, A\. Zeng, Z\. Du, C\. H\. Song, Y\. Su, Y\. Dong, and J\. TangVisualAgentBench: towards large multimodal models as visual foundation agents\.External Links:2408\.06327,[Link](https://arxiv.org/abs/2408.06327)Cited by:[§B\.1](https://arxiv.org/html/2611.24115#A2.SS1.p1.1),[§4\.2\.1](https://arxiv.org/html/2611.24115#S4.SS2.SSS1.p1.1),[§5](https://arxiv.org/html/2611.24115#S5.p3.1)\.
- Lù and Reddy \(2026\)X\. H\. Lù and S\. ReddyStructured distillation of web agent capabilities enables generalization\.External Links:2604\.07776,[Link](https://arxiv.org/abs/2604.07776)Cited by:[§4\.2\.2](https://arxiv.org/html/2611.24115#S4.SS2.SSS2.p1.1)\.
- Maet al\.\(2024\)X\. Ma, S\. Patidar, I\. Haughton, and S\. JamesHierarchical diffusion policy for kinematics\-aware multi\-task robotic manipulation\.CVPR\.Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Okada and Taniguchi \(2022\)M\. Okada and T\. TaniguchiDreamingV2: reinforcement learning with discrete world models without reconstruction\.arXiv preprint arXiv:2203\.00494\.Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Pignatelliet al\.\(2023\)E\. Pignatelli, J\. Ferret, M\. Geist, T\. Mesnard, H\. van Hasselt, and L\. ToniA survey of temporal credit assignment in deep reinforcement learning\.arXiv preprint arXiv:2312\.01072\.Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Qiet al\.\(2025\)Z\. Qi, X\. Liu, I\. L\. Iong, H\. Lai, X\. Sun, W\. Zhao, Y\. Yang, X\. Yang, J\. Sun, S\. Yao, T\. Zhang, W\. Xu, J\. Tang, and Y\. DongWebRL: training llm web agents via self\-evolving online curriculum reinforcement learning\.External Links:2411\.02337,[Link](https://arxiv.org/abs/2411.02337)Cited by:[§3\.3\.2](https://arxiv.org/html/2611.24115#S3.SS3.SSS2.p1.1),[§4\.2\.3](https://arxiv.org/html/2611.24115#S4.SS2.SSS3.p1.1),[Table 4](https://arxiv.org/html/2611.24115#S4.T4),[Table 4](https://arxiv.org/html/2611.24115#S4.T4.4),[§5](https://arxiv.org/html/2611.24115#S5.p3.1)\.
- Renet al\.\(2025\)P\. Ren, K\. Zhang, H\. Zheng, Z\. Li, Y\. Wen, F\. Zhu, S\. Ma, and X\. LiangSurfer: a world model\-based framework for vision\-language robot manipulation\.IEEE Transactions on Neural Networks and Learning Systems\.Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Robineet al\.\(2023\)J\. Robine, M\. Höftmann, T\. Uelwer, and S\. HarmelingTransformer\-based world models are happy with 100k interactions\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=TdBaDGCpjly)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Saxenaet al\.\(2025\)S\. Saxena, B\. Buchanan, C\. Paxton, P\. Liu, B\. Chen, N\. Vaskevicius, L\. Palmieri, J\. Francis, and O\. KroemerGraphEQA: using 3d semantic scene graphs for real\-time embodied question answering\.CoRL\.Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p2.1)\.
- Schulmanet al\.\(2017\)J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. KlimovProximal policy optimization algorithms\.External Links:1707\.06347,[Link](https://arxiv.org/abs/1707.06347)Cited by:[§4\.1\.1](https://arxiv.org/html/2611.24115#S4.SS1.SSS1.p4.1)\.
- Shanget al\.\(2026\)Y\. Shang, Z\. Li, Y\. Ma, W\. Su, X\. Jin, Z\. Wang, L\. Jin, X\. Zhang, Y\. Tang, H\. Su, C\. Gao, W\. Wu, X\. Liu, D\. Shah, Z\. Zhang, Z\. Chen, J\. Zhu, Y\. Tian, T\. Chua, W\. Zhu, and Y\. LiWorldArena: a unified benchmark for evaluating perception and functional utility of embodied world models\.External Links:2602\.08971,[Link](https://arxiv.org/abs/2602.08971)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Shenet al\.\(2025\)Y\. Shen, H\. Liu, P\. Liu, R\. Xia, T\. Yao, Y\. Sun, and T\. FengDETACH: cross\-domain learning for long\-horizon tasks via mixture of disentangled experts\.ArXivabs/2508\.07842\.External Links:[Link](https://api.semanticscholar.org/CorpusID:280567182)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Singhet al\.\(2022\)K\. P\. Singh, J\. Salvador, L\. Weihs, and A\. KembhaviA general purpose supervisory signal for embodied agents\.External Links:2212\.01186,[Link](https://arxiv.org/abs/2212.01186)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p2.1)\.
- Talvitie \(2017\)E\. TalvitieSelf\-correcting models for model\-based reinforcement learning\.External Links:1612\.06018,[Link](https://arxiv.org/abs/1612.06018)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p2.1)\.
- Veerapaneniet al\.\(2020\)R\. Veerapaneni, J\. D\. Co\-Reyes, M\. Chang, M\. Janner, C\. Finn, J\. Wu, J\. B\. Tenenbaum, and S\. LevineEntity abstraction in visual model\-based reinforcement learning\.External Links:1910\.12827,[Link](https://arxiv.org/abs/1910.12827)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Wanget al\.\(2025\)S\. Wang, Z\. Fei, Q\. Cheng, S\. Zhang, P\. Cai, J\. Fu, and X\. QiuWorld modeling makes a better planner: dual preference optimization for embodied task planning\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(ACL\),External Links:[Link](https://aclanthology.org/2025.acl-long.1044/)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Wanget al\.\(2024\)X\. Wang, Z\. Zhu, G\. Huang, B\. Wang, X\. Chen, and J\. LuWorldDreamer: towards general world models for video generation via predicting masked tokens\.arXiv preprint arXiv:2401\.09985\.Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Wuet al\.\(2022\)P\. Wu, A\. Escontrela, D\. Hafner, K\. Goldberg, and P\. AbbeelDayDreamer: world models for physical robot learning\.InConference on Robot Learning \(CoRL\),Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Wuet al\.\(2023\)Z\. Wu, N\. Dvornik, K\. Greff, T\. Kipf, and A\. GargSlotFormer: unsupervised visual dynamics simulation with object\-centric models\.External Links:2210\.05861,[Link](https://arxiv.org/abs/2210.05861)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Yanget al\.\(2025\)Y\. Yang, J\. Sun, S\. Kou, Y\. Wang, and Z\. DengLoHoVLA: a unified vision\-language\-action model for long\-horizon embodied tasks\.arXiv preprint arXiv:2506\.00411\.External Links:[Link](https://arxiv.org/abs/2506.00411)Cited by:[§1](https://arxiv.org/html/2611.24115#S1.p1.1)\.
- Yinet al\.\(2024\)H\. Yin, X\. Xu, Z\. Wu, J\. Zhou, and J\. LuSG\-nav: online 3d scene graph prompting for llm\-based zero\-shot object navigation\.arXiv preprint arXiv:2410\.08189\.Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p2.1)\.
- Yooet al\.\(2025\)M\. Yoo, J\. Jang, W\. Park, and H\. WooExploratory retrieval\-augmented planning for continual embodied instruction following\.External Links:2509\.08222,[Link](https://arxiv.org/abs/2509.08222)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p2.1)\.
- Zhanet al\.\(2026\)Y\. Zhan, X\. Wang, P\. Mao, T\. Feng, R\. Wang, and W\. ZhuModularAgent: a task\-aware modular framework for joint optimization of multimodal large language models and world models\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 8087–8096\.Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Zhanget al\.\(2023\)W\. Zhang, G\. Wang, J\. Sun, Y\. Yuan, and G\. HuangSTORM: efficient stochastic transformer based world models for reinforcement learning\.External Links:2310\.09615,[Link](https://arxiv.org/abs/2310.09615)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p1.1)\.
- Zhanget al\.\(2025\)Y\. Zhang, Q\. Zhang, X\. Ju, Z\. Liu, J\. Mao, J\. Sun, J\. Wu, S\. Gao, S\. Cai, Z\. Qin, L\. Liang, J\. Wang, Y\. Duan, J\. Cao, R\. Xu, and J\. TangEmbodiedVSR: dynamic scene graph\-guided chain\-of\-thought reasoning for visual spatial tasks\.External Links:2503\.11089,[Link](https://arxiv.org/abs/2503.11089)Cited by:[§5](https://arxiv.org/html/2611.24115#S5.p2.1)\.
- Zhouet al\.\(2024\)S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. NeubigWebArena: a realistic web environment for building autonomous agents\.External Links:2307\.13854,[Link](https://arxiv.org/abs/2307.13854)Cited by:[§B\.1](https://arxiv.org/html/2611.24115#A2.SS1.p1.1),[§1](https://arxiv.org/html/2611.24115#S1.p4.1),[§4\.2\.1](https://arxiv.org/html/2611.24115#S4.SS2.SSS1.p1.1),[§5](https://arxiv.org/html/2611.24115#S5.p3.1)\.
## Appendix AImplementation Details and More Results on Embodied Control
### A\.1\.Graph Construction
In Minecraft and Crafter, candidates are produced by an object\-segmentation front\-end applied to the image observation \(Appendix[A\.2](https://arxiv.org/html/2611.24115#A1.SS2)\)\. An interaction is one of two types, moving to or acting on a target object\. The graph has a fixed capacity ofNNnodes, where node zero is the agent\. Each object node carries its positionp∈ℝ3p\\in\\mathbb\{R\}^\{3\}, typeτ\\tau, sizemmand last\-seen stepll, and the agent node carries its position and inventory\. A candidate matches an existing node if both have the same type and their squared distance is belowδ2\\delta^\{2\}, and the closest such candidate is selected\. Edges connect each node to itsKnnK\_\{nn\}nearest neighbors in space\. The contextcjtc\_\{j\}^\{t\}of EG\-RSSM is computed by an MPNN over these edges with the same number of layers and hidden size as the dynamics model\. All scorer labels are computed from the front\-end’s detections and the agent’s own decisions\. A candidate receives a positive task label if it lies within a distance threshold of the target of a macro\-action chosen within the nextWWsteps, and the task loss is weighted by the policy advantage\. Novelty and coverage labels use type rarity and distance from the agent\. The prediction heads of EG\-RSSM decode node position, type and size, and the agent’s position and inventory\. The structural outcome inℒtopo\\mathcal\{L\}\_\{topo\}is node birth, a binary indicator of whether the interaction creates new nodes\.
### A\.2\.Perception Front\-End and Node Candidates
Minecraft\.The front\-end has two branches applied to the same frame\. An instance branch, a YOLOv8m segmentation model\([Jocher et al\., 2023](https://arxiv.org/html/2611.24115#bib.bib54)\), detects and segments objects in nine categories: log, tree, crafting table, furnace, gold ore, diamond ore, iron ore, coal ore and zombie\. A semantic branch, DeepLabV3\+\([Chen et al\., 2018](https://arxiv.org/html/2611.24115#bib.bib55)\)with a ResNet\-50 encoder\([He et al\., 2015](https://arxiv.org/html/2611.24115#bib.bib45)\), labels every pixel as background, water, grass, sand, lava or stone\. Both branches are trained on frames collected from Minecraft episodes with random exploration, which we annotated manually with the category labels above, and are frozen during RL\. Instance masks take precedence over semantic labels where they overlap\. Each instance yields a candidate with its type, mask centroid, bounding box and area\. The 2D outputs are lifted to 3D with a monocular depth model \(Video Depth Anything\([Chen et al\., 2026](https://arxiv.org/html/2611.24115#bib.bib56)\)\), which predicts depth\. The 3D position is used to match candidates across steps\.
Figure[5](https://arxiv.org/html/2611.24115#A1.F5)shows the front\-end output on the agent’s64×6464\\times 64observations in three scenes\. Large regions such as grass, water and sand are segmented as contiguous areas, and small objects such as single ore blocks are kept as separate instances\. The areas and instantces form the node candidates\.
Figure 5\.Perception front\-end in Minecraft\. Each row corresponds to a64×6464\\times 64observation \(left\), the segmentation \(middle\), and the resulting label map \(right\)\.Crafter\.As the observation image in Crafter is a grid of different objects, segmentation is naturally provided\. Node candidates are produced by an object semantic classifier MLP head trained on Crafter observations\.
### A\.3\.Observation and Action space
For Minecraft and Crafter, we follow the observation and action spaces of DreamerV3\([Hafner and et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib4)\)\.
### A\.4\.Goal\-Conditioned Executor and Macro\-Action
The executor is a 200M\-parameter DreamerV3\. It takes dense observations together with the macro\-action and outputs primitive actions\. The macro\-action consists of an interaction type and a target node\. The executor receives the interaction type and the target’s position as its goal, which is embedded and concatenated with the policy inputs, without modifying the observation or action spaces\. A maximum ofH=50H=50environment steps are given to the executor to achieve the goal\.
### A\.5\.Optimization
RWM is trained with the total loss of Eq\. \([12](https://arxiv.org/html/2611.24115#S3.E12)\) \(weights in Table[5](https://arxiv.org/html/2611.24115#A1.T5)\) using DreamerV3’s optimizer settings unless otherwise stated\. For policy learning in embodied control, we summarize the graph by a readoutgtg^\{t\}that concatenates the latent state of the agent node with pooled latent states of the object nodes\. Reward and continuation heads predictr^t\\hat\{r\}^\{t\}andc^t\\hat\{c\}^\{t\}fromgtg^\{t\}\. The graph policy assigns a score to each admissible interaction based on the latent state of its target node and selects an interaction by a softmax over these scores\. A value function ongtg^\{t\}is trained onλ\\lambda\-return targets, as in DreamerV3\.
### A\.6\.Online Update Cost and Graph Size
The graph has fixed capacity, so the per\-step cost and memory of RWM do not grow with episode length\. Only the current graphGtG^\{t\}is kept online \(at mostNNnodes andKnnNK\_\{nn\}Ndirected edges\. Each step of TA\-EGC has four parts\. First, three small MLP heads score at mostMMcandidates, which costsO\(M\)O\(M\)\. Second, the Top\-PPcandidates are selected inO\(MlogP\)O\(M\\log P\)\. Third, each of thePPadmitted candidates is compared with theNNexisting nodes of the same type, which costsO\(PN\)O\(PN\), and the nodes unseen for the longest time are evicted when the graph is full\. Finally, theKnnK\_\{nn\}\-nearest\-neighbor edges are rebuilt\. Our implementation computes all pairwise distances for this, which costsO\(N2\)O\(N^\{2\}\)and is cheap forN=128N=128\. Additionally, EG\-RSSM’s message passing costsO\(\(N\+KnnN\)d\)O\(\(N\+K\_\{nn\}N\)\\,d\)per step for hidden sizedd\. In Minecraft \(N=128N=128,M=256M=256,Knn=4K\_\{nn\}=4\) the full agent runs at 63\.28±\\pm0\.88 policy steps per second, compared with 70\.76±\\pm1\.53 for DreamerV3 with the same hyperparameters \(AMD EPYC 7543 CPU, single NVIDIA A100 GPU\)\.
Table 5\.Key hyperparameters for RWM\. Unless otherwise specified, shared hyperparameters follow the default DreamerV3 configuration\([Hafner and et al\., 2025](https://arxiv.org/html/2611.24115#bib.bib4)\)\.
### A\.7\.Crafter Setup and More Results
Setup\.We use the standard Crafter environment\([Hafner, 2022](https://arxiv.org/html/2611.24115#bib.bib27)\)with64×6464\\times 64image observations, the 17 discrete actions, and the default episode length of 10,000 steps\. We train every agent for 1M environment steps\. The macro\-action interval, graph capacity and other RWM hyperparameters are provided in Table[5](https://arxiv.org/html/2611.24115#A1.T5)\.
Figure 6\.Agent ability spectrum for Crafter \(ordered left to right by number of prerequisites for an achievement\)\. The y\-axis is shown in log scale\. RWM achieves comparable performance on common early\-stage achievements and stronger performance on harder long\-horizon achievements involving resource collection and tool crafting\.Per\-achievement Results\.Figure[6](https://arxiv.org/html/2611.24115#A1.F6)shows that RWM is similar to DreamerV3 on early achievements with few prerequisites \(wood, drinking, sapling, waking up, placing a table\), while higher on achievements with longer dependency chains \(coal, furnace, stone tools, iron\), which is consistent with the Minecraft results\.
## Appendix BImplementation Details on the Web
### B\.1\.Web embodied graph\.
We follow the official observation and action spaces\([Liu et al\., 2024b](https://arxiv.org/html/2611.24115#bib.bib46);[Zhou et al\., 2024](https://arxiv.org/html/2611.24115#bib.bib53)\)\. Candidate nodes are the DOM elements of the current page, with their tag, ARIA role, input type, text, value and instruction strings, normalized bounding box, and corresponding flags\. A virtual page node replaces the agent node\. Edges connect parent and child elements in both directions and link native labels to the controls they activate\. An interactionA=\(i,vtar\)A=\(i,v\_\{tar\}\)is one of click, fill, press, scroll, or select applied to a target node, together with its payload \(typed text, key, option, or scroll amount\)\.
### B\.2\.Construction and Learning
TA\-EGC scores the DOM candidates with the same task, novelty and coverage heads as in embodied control and admits the Top\-PPinto the graph\. Instead of spatial proximity, a candidate is matched to an existing node by its document, element reference and tag\. The task label marks the targets of the nextW=16W\{=\}16recorded interactions\. Novelty and coverage use the rarity of the node type and the rarity of its position on the screen, respectively\. All labels are weighted uniformly, since the Web policy is fixed and provides no advantage\. The prediction heads decode the state flags, input values and bounding boxes of the elements, and TA\-EGC and EG\-RSSM are trained jointly\.
### B\.3\.Navigation Head
On the Web, the structural outcome inℒtopo\\mathcal\{L\}\_\{topo\}is the page that an interaction leads to: a click either stays on the current page, where changes of elements are captured by the predicted flags, or loads a new page\. We predict this outcome with a navigation head, a softmax classifier on the frozen EG\-RSSM latents\. Its classes arestayor the site routes of the pages seen in training\.
### B\.4\.Decision procedure\.
At each step, WebRL proposes 3 candidate actions from the official prompt\. The world model forecasts each candidate from the shared posterior\. The verifier \(Qwen2\.5\-VL\-32B\-Instruct\) receives the task, the current screenshot, the observed action history, the candidates and their forecasts, and returns one candidate\. Unparsable replies fall back to WebRL’s top candidate\. The verifier is greedily decoded with at most 768 output tokens\. All three variants use the same procedure, prompts, and budgets\. For the variant without forecasts, the candidates are shown without predicted consequences\.
### B\.5\.Environment and scoring\.
Each episode starts from a newly launched CMS container with a fresh login\. The tasks follow the official configuration and the early\-stop rules, and the browser window is set to1280×7201280\\times 720pixels\. Judge\-graded tasks are scored with the official judge prompt\. Episodes ending with an environment error count as failures\.相似文章
GraphThink:面向长时程具身任务规划的图增强LLM思维
GraphThink 是一个将任务图与场景图相结合的框架,旨在增强基于LLM的长时程具身任务规划,在ALFRED基准上取得了最先进的结果,并提升了泛化能力和闭环重规划能力。
GAVEL:基于图世界模型的验证高效长时域LLM任务规划
本文介绍了GAVEL,这是一个使用图世界模型来验证和修复机器人任务长时域LLM规划的框架,显著提高了仿真中的成功率和效率。
WorldLines:长周期有状态具身智能体的基准测试与建模
WorldLines 提出了一个针对长周期具身家务辅助的基准,包含记忆问答和部分可观察条件下的具身任务规划,并提出了 ObsMem,一个可见性感知的记忆框架。
关系学习即世界建模
论文提出 TabWM(表格世界模型)与 WorldRel 生成器,通过在上下文中观察执行样本来推断关系型系统的动态,预测动作导致的稀疏结构化编辑,从而将关系学习从状态建模扩展到世界建模与动作规划。
ReWorld:一个具有长期记忆的交互式世界模型
ReWorld 提出了一个交互式世界模型,该模型使用混合注意力窗口和分布匹配的LoRA蒸馏,将短期控制与长期记忆解耦,实现了具有高动作保真度和回忆能力的实时视频生成。