World Action Critic (WAC) Model: Joint Foresight and Criticism for Policy Learning

# World Action Critic (WAC) Model: Joint Foresight and Criticism for Policy Learning
Source: [https://arxiv.org/html/2610.22825](https://arxiv.org/html/2610.22825)
Xiangteng Zhang††thanks:Equal contribution\.Haoyuan Liu11footnotemark:1Affiliation:School of Vehicle and Mobility, Tsinghua UniversityYang GuanAffiliation:School of Vehicle and Mobility, Tsinghua UniversityJingliang DuanAffiliation:School of Vehicle and Mobility, Tsinghua UniversityYao LyuAffiliation:School of Vehicle and Mobility, Tsinghua UniversityTao ZhangAffiliation:SunRisingAI LabShuai ZhangAffiliation:Didi Voyager Labs, Didi Autonomous DrivingKehua ShengAffiliation:Didi Voyager Labs, Didi Autonomous DrivingBo ZhangAffiliation:Didi Voyager Labs, Didi Autonomous DrivingShengbo Eben Li††thanks:Corresponding author\.Affiliation:School of Vehicle and Mobility, Tsinghua UniversityAffiliation:College of AI, Tsinghua University

###### Abstract

World models have shown considerable promise in sequential decision\-making by anticipating the consequences of actions and providing predictive foresight for policy learning\. However, learning to reason about counterfactual actions remains challenging, since predictive objectives may underemphasize information critical for action evaluation, while unexecuted actions lack observed futures for direct supervision\. To address these challenges, we propose the World Action Critic \(WAC\) Model, which proposes multimodal action candidates and jointly learns candidate\-conditioned future prediction and consequence evaluation\. In our proposed framework, critic supervision directly shapes intermediate predictive representations during rollout, encouraging them to retain information needed to distinguish action quality\. Observed futures supervise prediction for executed actions, while external task\-quality targets provide evaluation signals for counterfactual candidates\. Through joint optimization, critic signals from these candidates propagate into predictive representations without requiring their observed futures\. Our theoretical analysis characterizes critic\-guided feature retention and bounds candidate\-selection regret\. Applied to autonomous driving, WAC achieves 93\.4 PDMS on NAVSIM\-v1, demonstrating the effectiveness of jointly learning predictive foresight and consequence evaluation\.

## 1Introduction

World models enable agents to anticipate how their environments evolve under different actions, supporting policy learning in reinforcement learning\[[56](https://arxiv.org/html/2610.22825#bib.bib31),[42](https://arxiv.org/html/2610.22825#bib.bib26),[17](https://arxiv.org/html/2610.22825#bib.bib10),[13](https://arxiv.org/html/2610.22825#bib.bib66),[20](https://arxiv.org/html/2610.22825#bib.bib13)\], robot control\[[58](https://arxiv.org/html/2610.22825#bib.bib61),[55](https://arxiv.org/html/2610.22825#bib.bib62)\], and autonomous driving\[[11](https://arxiv.org/html/2610.22825#bib.bib9),[28](https://arxiv.org/html/2610.22825#bib.bib63),[44](https://arxiv.org/html/2610.22825#bib.bib25)\]\. By predicting future states from current observations and actions, they provide a basis for reasoning about the consequences of alternative actions before execution\. This capability also supports efforts to build physics\-native intelligence, as explored in models such as Phi\-WM\[[40](https://arxiv.org/html/2610.22825#bib.bib1)\]\. Effective decision making therefore requires predictive representations that preserve the information needed to compare these consequences\.

Existing approaches integrate world models into policy learning through three main approaches \(Figure[1](https://arxiv.org/html/2610.22825#S1.F1)\)\. First, predictive representation learning uses future\-state prediction as an auxiliary objective to enrich the features consumed by a policy\[[33](https://arxiv.org/html/2610.22825#bib.bib20),[29](https://arxiv.org/html/2610.22825#bib.bib17),[4](https://arxiv.org/html/2610.22825#bib.bib44),[25](https://arxiv.org/html/2610.22825#bib.bib15)\]\. Second, simulator\-based methods use action\-conditioned imagined transitions for reward\-based policy optimization or model\-based planning\[[19](https://arxiv.org/html/2610.22825#bib.bib11),[18](https://arxiv.org/html/2610.22825#bib.bib12),[47](https://arxiv.org/html/2610.22825#bib.bib29),[26](https://arxiv.org/html/2610.22825#bib.bib18),[34](https://arxiv.org/html/2610.22825#bib.bib21)\]\. Predicted states are evaluated using rewards or values\[[21](https://arxiv.org/html/2610.22825#bib.bib14)\], while the transition model is primarily learned from predictive objectives\. Evaluation therefore provides a downstream criterion for imagined consequences, with limited control over the information retained in the predictive representations themselves\. Third, future\-conditioned policies and World Action Models \(WAMs\) bring prediction into action generation, with recent WAMs jointly modeling future states and actions\[[8](https://arxiv.org/html/2610.22825#bib.bib38),[65](https://arxiv.org/html/2610.22825#bib.bib36),[3](https://arxiv.org/html/2610.22825#bib.bib3),[54](https://arxiv.org/html/2610.22825#bib.bib30),[38](https://arxiv.org/html/2610.22825#bib.bib23)\]\. Their joint modeling of future states and actions can capture multimodal behavioral alternatives, while the relative task quality of these alternatives remains implicit\. Counterfactual action comparison requires an explicit criterion for evaluating the consequences associated with each candidate\.

![Refer to caption](https://arxiv.org/html/2610.22825v1/figures/fig1.png)Figure 1:Overview of world models for decision making\.\(a\) Future prediction provides an auxiliary learning signal for policy features\. \(b\) An action\-conditioned simulator supplies imagined states to a planner for reward\-based evaluation and search\. \(c\) WAM jointly models future states and actions for action generation\. \(d\) WAC jointly learns candidate\-conditioned future prediction and consequence evaluation\. Factual future targets ground prediction, while candidate\-quality supervision shapes predictive representations for counterfactual action comparison\.Learning world models for counterfactual action evaluation faces two challenges\. First, predictive representations can be mismatched with the information needed for decision making\. Under limited representation capacity, future prediction can favor dominant environmental variation over subtle details that distinguish action quality, making predictive accuracy an insufficient criterion for decision value\[[9](https://arxiv.org/html/2610.22825#bib.bib7),[12](https://arxiv.org/html/2610.22825#bib.bib37),[41](https://arxiv.org/html/2610.22825#bib.bib24),[11](https://arxiv.org/html/2610.22825#bib.bib9)\]\. Second, counterfactual actions lack direct future supervision\. A recorded transition provides an observed future only for the executed action, while decisions require comparing unexecuted alternatives at the same decision point\[[59](https://arxiv.org/html/2610.22825#bib.bib32)\]\. External evaluators can supply task\-quality targets through simulation and scoring, but these targets leave complete counterfactual future states unspecified\. The challenge is to use this evaluation supervision to learn decision\-relevant predictions beyond actions with observed futures\.

To address these challenges, we propose theWorld Action Critic \(WAC\) Model, which jointly learns candidate\-conditioned future prediction and consequence evaluation\. A multimodal policy first proposes diverse action candidates\. Each candidate conditions a recurrent transition–critic Transformer that predicts future latent states and accumulates evidence for evaluating the candidate over the full horizon\. Asymmetric attention allows the critic to read intermediate predictive features while keeping critic tokens hidden from the transition stream\. Critic supervision propagates through these features during joint training, encouraging them to preserve information needed to distinguish action quality\.

WAC uses complementary supervision to extend this learning to counterfactual actions\. Observed futures supervise prediction for executed actions, while an external evaluator supplies candidate\-level quality targets for both executed and counterfactual actions\. Critic gradients optimize the predictive features along each candidate’s rollout, allowing unexecuted alternatives to guide representation learning even when their future observations are unavailable\. Building on existing consequence\-based trajectory evaluation\[[61](https://arxiv.org/html/2610.22825#bib.bib34),[31](https://arxiv.org/html/2610.22825#bib.bib19)\], WAC couples evaluation to intermediate features throughout recurrent prediction under this complementary supervision\. The resulting scores provide a common criterion for comparing multimodal candidates based on their predicted consequences\.

Our theoretical analyses address these two learning challenges: they characterize how critic supervision promotes decision\-relevant feature retention in a linear setting and bound candidate\-selection regret in terms of critic fitting error, evaluator error, and candidate coverage\. We instantiate WAC for autonomous driving and evaluate it on NAVSIM\[[7](https://arxiv.org/html/2610.22825#bib.bib6)\], achieving 93\.4 PDMS on NAVSIM\-v1\. Controlled ablations examine factual future supervision, recurrent rollout, and critic gradients into predictive features, supporting the complementary roles of future prediction and consequence evaluation\.

Our main contributions can be summarized as follows:

- •We propose WAC, which jointly learns future prediction and consequence evaluation for counterfactual action reasoning, using critic supervision to shape predictive representations for decision making\.
- •We develop a recurrent transition–critic architecture with asymmetric attention and complementary supervision, grounding prediction in factual futures while allowing candidate\-quality targets to guide learning from counterfactual actions\.
- •We provide theoretical analyses of critic\-guided predictive feature retention and candidate selection, showing how evaluation supervision promotes action\-relevant future representations and establishing a regret bound under evaluator accuracy and candidate coverage assumptions\.

## 2Related Work

### 2\.1World Models for Planning and Control

World models provide predictive representations through which agents can reason about actions before execution\. Latent dynamics support imagination\-based policy optimization, trajectory optimization, and visual planning, as exemplified by Dreamer, TD\-MPC2, DINO\-WM, and BOOM\[[18](https://arxiv.org/html/2610.22825#bib.bib12),[21](https://arxiv.org/html/2610.22825#bib.bib14),[62](https://arxiv.org/html/2610.22825#bib.bib35),[56](https://arxiv.org/html/2610.22825#bib.bib31)\]\. World models also support exploration augmentation by synthesizing transitions for off\-policy learning\[[42](https://arxiv.org/html/2610.22825#bib.bib26)\]\. This predictive interface extends to physical robot control and autonomous driving, where action\-conditioned futures inform behavior learning and planning\[[47](https://arxiv.org/html/2610.22825#bib.bib29),[44](https://arxiv.org/html/2610.22825#bib.bib25),[6](https://arxiv.org/html/2610.22825#bib.bib5),[14](https://arxiv.org/html/2610.22825#bib.bib64),[15](https://arxiv.org/html/2610.22825#bib.bib65)\]\. Across these settings, learned dynamics provide a substrate for decision making: policies or planners use predicted states together with rewards, values, or task costs\[[27](https://arxiv.org/html/2610.22825#bib.bib16)\]\. Prediction and evaluation often remain identifiable components, even when trained jointly\. This leaves an important design question beyond whether imagination improves control: how should an evaluation objective shape the information retained in the predictive representation itself?

### 2\.2World Action Models

World Action Models \(WAMs\) bring future prediction into action generation, extending action\-conditioned dynamics toward joint modeling of environment evolution and behavior\. UniPi illustrates an early future\-to\-action interface by translating generated visual plans into actions through inverse dynamics\[[8](https://arxiv.org/html/2610.22825#bib.bib38)\]\. More recent models, including WorldVLA, Motus, and DriveWAM, couple future and action modeling through shared generative or temporal representations\[[5](https://arxiv.org/html/2610.22825#bib.bib4),[3](https://arxiv.org/html/2610.22825#bib.bib3),[38](https://arxiv.org/html/2610.22825#bib.bib23)\]\. Their common interface uses predictive knowledge to generate or refine behavior, with action preferences potentially encoded in the learned generation distribution\. Such preferences do not necessarily provide an explicit task\-quality criterion for comparing externally proposed candidates\. WAC addresses this complementary problem: evaluating candidate actions supplied by a proposal policy through their predicted consequences, and using evaluation supervision to shape those predictions\.

### 2\.3World Models for Value Estimation and Candidate Evaluation

Prediction and evaluation have been connected at several levels\. Value\-aware model learning constrains predictive dynamics by downstream decision objectives, recognizing that uniform transition accuracy need not preserve the information most useful for control\[[9](https://arxiv.org/html/2610.22825#bib.bib7),[12](https://arxiv.org/html/2610.22825#bib.bib37),[41](https://arxiv.org/html/2610.22825#bib.bib24)\]\. At the representation level, World Value Models and World Critic Models connect world modeling with value learning to assess task progress or support policy improvement\[[45](https://arxiv.org/html/2610.22825#bib.bib28),[10](https://arxiv.org/html/2610.22825#bib.bib8)\]\. At the candidate level, driving methods use predicted consequences to compare proposed behaviors: World4Drive and WoTE connect future modeling to trajectory selection or scoring\[[61](https://arxiv.org/html/2610.22825#bib.bib34),[31](https://arxiv.org/html/2610.22825#bib.bib19)\]\. Together, these research directions connect predictive modeling with decision objectives through value learning and consequence\-based evaluation\. For candidate\-conditioned decision making, this relationship raises two linked questions: which predictive features evaluation should preserve, and what candidate labels can support when future observations are missing\. Building on these connections, WAC focuses on jointly learning candidate\-conditioned foresight and consequence criticism under asymmetric supervision: observed future targets supervise factual transitions, while critic labels supervise a broader set of candidate actions\. Our analysis examines how these signals influence feature retention and candidate selection without observed futures for every candidate\. A recurrent dual\-stream realization lets criticism read intermediate predictive features and, through joint optimization, shape the representations used for future prediction\.

## 3World Action Critic \(WAC\) Model

In this section, we present the World Action Critic \(WAC\) Model, which unifies action proposal, future prediction, and consequence evaluation for candidate selection\. We formulate their coupling within a joint model and develop a recurrent architecture that learns predictive representations from factual futures and candidate\-level quality supervision\.

### 3\.1From Action Generation to Candidate Evaluation

Letz=E⁡\(o≤t\)z=E\(o\_\{\\leq t\}\)denote the latent representation of the available observations and task context\. We useaato denote an action or action sequence andz′z^\{\\prime\}to denote the corresponding future latent state or sequence\. The coupling between future prediction and action generation in a WAM can be expressed as

p⁡\(z′,a∣z\)=p⁡\(z′∣z\)​p​\(a∣z,z′\),p\(z^\{\\prime\},a\\mid z\)=p\(z^\{\\prime\}\\mid z\)\\,p\(a\\mid z,z^\{\\prime\}\),\(1\)where predictive future context informs action generation\.

WAC extends this joint formulation with an explicit critic that evaluates the consequences of proposed actions\. Withqqdenoting the task\-quality score predicted by the critic, the model factorizes as

p⁡\(z′,a,q∣z\)=π⁡\(a∣z\)⏟actor​p⁡\(z′∣z,a\)⏟world​p⁡\(q∣z,a,z′\)⏟critic\.p\(z^\{\\prime\},a,q\\mid z\)=\\underbrace\{\\pi\(a\\mid z\)\}\_\{\\text\{actor\}\}\\,\\underbrace\{p\(z^\{\\prime\}\\mid z,a\)\}\_\{\\text\{world\}\}\\underbrace\{p\(q\\mid z,a,z^\{\\prime\}\)\}\_\{\\text\{critic\}\}\.\(2\)The policyπ\\piproposes a candidate set𝒜N=\{a\(i\)\}i=1N\\mathcal\{A\}\_\{N\}=\\\{a^\{\(i\)\}\\\}\_\{i=1\}^\{N\}\. For each candidatea\(i\)a^\{\(i\)\}, the world model predicts its corresponding futurez^′\(i\)\\widehat\{z\}^\{\\prime\(i\)\}, and the critic produces a quality estimateq^\(i\)\\widehat\{q\}^\{\(i\)\}conditioned on the current latent state, the candidate, and its predicted consequences\. The parenthesized superscript\(i\)\(i\)indexes candidates, so each action is paired with its own future and evaluation\. These factors describe the functional roles within a unified WAC, with the critic drawing on predictive features as the candidate\-conditioned future is formed\.

The resulting scores provide a common criterion for comparing the candidates\. WAC selects the action with the highest predicted quality,

a^=a\(i⋆\),i⋆∈arg⁡max1≤i≤N⁡q^\(i\)\.\\widehat\{a\}=a^\{\(i^\{\\star\}\)\},\\qquad i^\{\\star\}\\in\\arg\\max\_\{1\\leq i\\leq N\}\\widehat\{q\}^\{\(i\)\}\.\(3\)

### 3\.2Joint Prediction and Criticism

The candidate evaluation in Section[3\.1](https://arxiv.org/html/2610.22825#S3.SS1)relies on predicting the consequences of an action from the current latent state\. We implement this dependency with a joint prediction–critic module that predicts future latent states and extracts evidence for action evaluation during their generation\. As illustrated in Figure[2](https://arxiv.org/html/2610.22825#S3.F2), asymmetric attention connects future prediction to criticism within each layer, while recurrent application extends this interaction over the prediction horizon\.

![Refer to caption](https://arxiv.org/html/2610.22825v1/figures/fig2.png)Figure 2:Joint prediction and criticism in WAC\.Given latent statezzand candidate actionaa, WAC jointly predicts future latent statez′z^\{\\prime\}and quality scoreqq\. Asymmetric attention lets the critic read predictive features while keeping critic tokens hidden from latent prediction\. Conditions are injected through adaptive layer normalization \(AdaLN\)\.#### Joint prediction–critic module\.

WAC couples latent prediction and action evaluation through a shared sequence of joint Transformer layers\. The action encoderEaE\_\{a\}maps the complete candidate actionaato a tokenEa​\(a\)E\_\{a\}\(a\)for attention; this embedding is also combined with the task condition for AdaLN\. Starting from the current latent state, the module recurrently predicts action\-conditioned futures and accumulates critic evidence\. Each candidatea\(i\)a^\{\(i\)\}induces a separate prediction under shared model parameters\.

We omit the candidate index below and usekkfor rollout steps\. The history memoryMkM\_\{k\}contains latent statesz0:kz\_\{0:k\}and critic tokens𝐪0:k\\mathbf\{q\}\_\{0:k\}\. At each step, latent prediction uses learned queries conditioned on the candidate action and task, while the critic query is initialized from𝐪k\\mathbf\{q\}\_\{k\}\. The initial critic token𝐪0\\mathbf\{q\}\_\{0\}is learned\. A stack ofLLjoint layers updates the prediction and critic tokens, with the same stack and complete candidate action reused across rollout steps\.

#### Asymmetric joint attention\.

Within a joint layer, latent prediction first attends to the latent history together with the action token, followed by a residual feed\-forward update\. The critic then attends to the updated predictive features, the full history, the action token, and its own current representation\. Writing the layer\-ℓ\\ellprediction and critic tokens as𝐳ℓ\\mathbf\{z\}^\{\\ell\}and𝐪ℓ\\mathbf\{q\}^\{\\ell\}, their visible key/value sources are

𝒮zℓ\\displaystyle\\mathcal\{S\}\_\{z\}^\{\\ell\}=\[Mkz,Ea​\(a\)\],\\displaystyle=\[M\_\{k\}^\{z\},E\_\{a\}\(a\)\],\(4\)𝒮qℓ\\displaystyle\\mathcal\{S\}\_\{q\}^\{\\ell\}=\[Mk,Ea​\(a\),𝐳ℓ\+1,𝐪ℓ\],\\displaystyle=\[M\_\{k\},E\_\{a\}\(a\),\\mathbf\{z\}^\{\\ell\+1\},\\mathbf\{q\}^\{\\ell\}\],whereMkzM\_\{k\}^\{z\}contains the latent states, and bold symbols denote intermediate token representations\. An asymmetric mask excludes historical critic tokens from latent prediction\. The critic can therefore evaluate emerging future features at every layer, while latent prediction depends on state history and the candidate action\. Critic gradients propagate through the predictive features used for evaluation, allowing quality supervision to shape their representations\.

Prediction and criticism use separate attention projections and feed\-forward networks\. Action and task conditions are injected through adaptive layer normalization \(AdaLN\)\[[35](https://arxiv.org/html/2610.22825#bib.bib22)\], which generates feature\-wise scale and shift parameters for the normalized attention queries and feed\-forward inputs\. Rotary position embeddings encode temporal and spatial coordinates\.

After the final layer, normalized prediction tokens produce a residual update to the latent state, while the normalized critic token becomes𝐪k\+1\\mathbf\{q\}\_\{k\+1\}\. These outputs are appended to the history for the next step\. AfterHHsteps, a prediction head maps the final critic token𝐪H\(i\)\\mathbf\{q\}\_\{H\}^\{\(i\)\}to the scalar scoreq^\(i\)\\widehat\{q\}^\{\(i\)\}and auxiliary quality components\. Candidates are evaluated in parallel and selected using \([3](https://arxiv.org/html/2610.22825#S3.E3)\)\.

### 3\.3Learning from Factual Futures and Candidate Critiques

We train WAC through complementary supervision for future prediction and critic learning\. The recorded action provides an observed future for learning latent dynamics, while an external evaluator supplies critic targets for candidate actions\. This difference in supervision allows the model to learn from factual transitions and extend evaluation to a broader set of proposed actions\.

#### Factual future supervision\.

Letafa^\{\\mathrm\{f\}\}denote the recorded action andzkfz\_\{k\}^\{\\mathrm\{f\}\}its future latent target at stepk=1,…,Hk=1,\\ldots,H\. Conditioned onafa^\{\\mathrm\{f\}\}and the initial latent state, WAC recurrently predictsz^kf\\widehat\{z\}\_\{k\}^\{\\mathrm\{f\}\}\. Future observations provide the corresponding latent targets, which are treated as fixed supervision through a stop\-gradient operation\. The prediction objective is

ℒz=1H​∑k=1Hℓz​\(z^kf,sg⁡\(zkf\)\),\\mathcal\{L\}\_\{z\}=\\frac\{1\}\{H\}\\sum\_\{k=1\}^\{H\}\\ell\_\{z\}\\\!\\left\(\\widehat\{z\}\_\{k\}^\{\\mathrm\{f\}\},\\operatorname\{sg\}\(z\_\{k\}^\{\\mathrm\{f\}\}\)\\right\),\(5\)whereℓz\\ell\_\{z\}measures the discrepancy between predicted and target latent states, andsg\\operatorname\{sg\}stops gradients through the target representation\. This objective grounds the learned dynamics in the consequences of the recorded action\. The same predictive model is applied to other candidates, whose evaluation is supervised through the critic\.

#### Critic learning\.

For each candidatea\(i\)a^\{\(i\)\}, letq\(i\)q^\{\(i\)\}denote its evaluator\-provided target andq^\(i\)\\widehat\{q\}^\{\(i\)\}the critic prediction\. During training, we include the recorded action asa\(N\+1\)=afa^\{\(N\+1\)\}=a^\{\\mathrm\{f\}\}alongside theNNproposed candidates\. The critic objective is

ℒq=1N\+1​∑i=1N\+1ℓq​\(q^\(i\),q\(i\)\),\\mathcal\{L\}\_\{q\}=\\frac\{1\}\{N\+1\}\\sum\_\{i=1\}^\{N\+1\}\\ell\_\{q\}\\\!\\left\(\\widehat\{q\}^\{\(i\)\},q^\{\(i\)\}\\right\),\(6\)whereℓq\\ell\_\{q\}measures the discrepancy between the critic prediction and its supervision target\. Candidate\-level targets therefore extend learning beyond the recorded action, even when the corresponding future latent targets are unavailable\.

#### Joint optimization\.

The proposal policyπ\\piis supervised by the recorded actionafa^\{\\mathrm\{f\}\}using a best\-of\-NNregression objective\. Among theNNgenerated candidates, we select the one closest to the ground truth and apply the action loss to that candidate:

iπ∈arg⁡min1≤i≤N​ℓa​\(a\(i\),af\),ℒπ=ℓa​\(a\(iπ\),af\),i\_\{\\pi\}\\in\\arg\\min\_\{1\\leq i\\leq N\}\\ell\_\{a\}\\\!\\left\(a^\{\(i\)\},a^\{\\mathrm\{f\}\}\\right\),\\qquad\\mathcal\{L\}\_\{\\pi\}=\\ell\_\{a\}\\\!\\left\(a^\{\(i\_\{\\pi\}\)\},a^\{\\mathrm\{f\}\}\\right\),\(7\)whereℓa\\ell\_\{a\}is the distance over the action sequence\. This objective encourages the proposal policy to generate a candidate that matches the demonstrated action\. Together with future prediction and critic learning, the joint objective is

ℒ=λπ​ℒπ\+λz​ℒz\+λq​ℒq,\\mathcal\{L\}=\\lambda\_\{\\pi\}\\mathcal\{L\}\_\{\\pi\}\+\\lambda\_\{z\}\\mathcal\{L\}\_\{z\}\+\\lambda\_\{q\}\\mathcal\{L\}\_\{q\},\(8\)where the nonnegative weights balance the three objectives\. The losses above are defined for a training instance and averaged over the training data\. The choices ofℓz\\ell\_\{z\}andℓq\\ell\_\{q\}depend on the prediction targets and critic supervision of the task\.

The joint architecture connects these learning signals through the predictive features used by the critic\. Factual future supervision constrains latent dynamics, while critic supervision propagates through asymmetric attention and shapes the predictive representations used to compare candidates\. Each evaluated candidate thus contributes a learning signal to its predicted future representation\. At inference, the proposal policy supplies candidate actions, and WAC predicts their consequences and selects an action using its learned critic according to \([3](https://arxiv.org/html/2610.22825#S3.E3)\)\.

### 3\.4Why Joint Prediction and Criticism Help

Our analysis addresses the mismatch between predictive representation learning and decision\-oriented evaluation, and incomplete counterfactual future supervision for candidate actions\. We first show how critic supervision reweights predictive representations toward decision\-relevant future information\. We then establish how candidate\-level critic supervision supports action selection under coverage of the proposed actions, including those with unobserved futures\. The two results explain the roles of criticism in representation learning and decision making\. Full assumptions and proofs are provided in Appendix[A](https://arxiv.org/html/2610.22825#A1)\.

#### Learning predictive representations for criticism\.

Future prediction naturally prioritizes features that explain the variation of future states, although these features need not be the most informative for evaluating actions\. Joint learning of prediction and criticism introduces an additional learning signal that can promote future features according to their relevance to action evaluation\. We formalize this effect in a simple linear setting\. Let the centered future latent representation bez′∈ℝDz^\{\\prime\}\\in\\mathbb\{R\}^\{D\}with covarianceΣ≻0\\Sigma\\succ 0, and let the critic target depend on the future representation as

q=b⊤​z′\+ξ,q=b^\{\\top\}z^\{\\prime\}\+\\xi,\(9\)where𝔼⁡\[ξ∣z′\]=0\\mathbb\{E\}\[\\xi\\mid z^\{\\prime\}\]=0and𝔼⁡\[ξ2\]<∞\\mathbb\{E\}\[\\xi^\{2\}\]<\\infty\. We whiten the future representation asz~=Σ−1/2z′\\widetilde\{z\}=\\Sigma^\{\-1/2\}z^\{\\prime\}and consider add\-dimensional predictive representationU⊤​z~U^\{\\top\}\\widetilde\{z\}, where1≤d<D1\\leq d<DandU⊤​U=IdU^\{\\top\}U=I\_\{d\}\. Prediction and criticism are modeled by optimal linear heads with squared losses, and their relative weight is controlled byλ≥0\\lambda\\geq 0\. Define the critic\-relevant future direction as

v=Σ1/2​b\.v=\\Sigma^\{1/2\}b\.\(10\)
###### Theorem 1\(Critic\-guided predictive importance\)\.

Under the setting above, joint learning induces the importance matrix

Kλ=Σ⏟predictive importance\+λ​v​v⊤⏟decision importance\.K\_\{\\lambda\}=\\underbrace\{\\Sigma\}\_\{\\text\{predictive importance\}\}\+\\underbrace\{\\lambda vv^\{\\top\}\}\_\{\\text\{decision importance\}\}\.\(11\)An optimaldd\-dimensional predictive subspace is given by

span⁡\(Uλ\)=TopEigd⁡\(Kλ\)\.\\operatorname\{span\}\(U\_\{\\lambda\}\)=\\operatorname\{TopEig\}\_\{d\}\(K\_\{\\lambda\}\)\.\(12\)whereTopEigd\\operatorname\{TopEig\}\_\{d\}denotes a subspace spanned by eigenvectors associated with theddlargest eigenvalues; ties admit any maximizing choice\.

Theorem[1](https://arxiv.org/html/2610.22825#Thmtheorem1)shows that the prediction objective favors future directions with high predictive importance, while critic supervision additionally promotes directions that are informative for action evaluation\. Consequently, a decision\-relevant future feature can be retained even when it is not dominant under prediction alone\. In WAC, this mechanism is realized by allowing gradients fromℒq\\mathcal\{L\}\_\{q\}to propagate through the predictive features consumed by the critic\. The complete derivation and a two\-dimensional example are provided in Appendix[A\.1](https://arxiv.org/html/2610.22825#A1.SS1)\.

#### Action selection under candidate supervision\.

The second issue arises from the asymmetric supervision available during training\. The recorded action provides an observed future latent target, while proposed candidate actions generally have no corresponding counterfactual future observations\. Their external critic targets can nevertheless provide direct supervision for candidate comparison\. Letμ\\mudenote the joint distribution of critic\-supervised\(z,a\)\(z,a\)pairs\. Letν\\nudenote the distribution obtained by sampling a test latent state and its proposed candidate set and then choosing one of theNNcandidates uniformly\. We writeq⋆​\(z,a\)q^\{\\star\}\(z,a\)for the intended task score,qext\(z,a\)=𝔼\[q∣z,a\]q\_\{\\mathrm\{ext\}\}\(z,a\)=\\mathbb\{E\}\[q\\mid z,a\]for the evaluator’s conditional mean, andq^​\(z,a\)\\widehat\{q\}\(z,a\)for WAC’s predicted score\. Define

ϵfit\\displaystyle\\epsilon\_\{\\mathrm\{fit\}\}=𝔼μ​\(q^−qext\)2,\\displaystyle=\\mathbb\{E\}\_\{\\mu\}\\left\(\\widehat\{q\}\-q\_\{\\mathrm\{ext\}\}\\right\)^\{2\},\(13\)ϵext\\displaystyle\\epsilon\_\{\\mathrm\{ext\}\}=𝔼μ​\(qext−q⋆\)2,\\displaystyle=\\mathbb\{E\}\_\{\\mu\}\\left\(q\_\{\\mathrm\{ext\}\}\-q^\{\\star\}\\right\)^\{2\},\(14\)whereϵfit\\epsilon\_\{\\mathrm\{fit\}\}measures critic fitting error andϵext\\epsilon\_\{\\mathrm\{ext\}\}measures the discrepancy between the external evaluator and the intended task score\.

###### Theorem 2\(Candidate selection under critic supervision\)\.

Assumeν≪μ\\nu\\ll\\muwithd​ν/d​μ≤κ<∞d\\nu/d\\mu\\leq\\kappa<\\inftyμ\\mu\-almost surely, and that the errors in \([14](https://arxiv.org/html/2610.22825#S3.E14)\) are finite\. Leta⋆a^\{\\star\}maximizeq⋆q^\{\\star\}over the proposed candidate set𝒜N\\mathcal\{A\}\_\{N\}, and leta^\\widehat\{a\}be selected by \([3](https://arxiv.org/html/2610.22825#S3.E3)\)\. Then

𝔼⁡\[q⋆​\(z,a⋆\)−q⋆​\(z,a^\)\]\\displaystyle\\mathbb\{E\}\\left\[q^\{\\star\}\(z,a^\{\\star\}\)\-q^\{\\star\}\(z,\\widehat\{a\}\)\\right\]\(15\)≤2​N​κ​\(ϵfit\+ϵext\)\.\\displaystyle\\leq\\sqrt\{2N\\kappa\}\\left\(\\sqrt\{\\epsilon\_\{\\mathrm\{fit\}\}\}\+\\sqrt\{\\epsilon\_\{\\mathrm\{ext\}\}\}\\right\)\.The expectation includes the joint distribution of the proposed candidates and allows dependence among them\.

The bound shows that candidate selection depends on the accuracy of WAC’s critic predictions against the evaluator’s targets, the alignment of that evaluator with the intended task objective, and the coverage of test candidates by the critic\-supervised distribution\. Candidate\-level critic targets can therefore support reliable comparison without requiring counterfactual future observations for every proposed action\. Together, Theorems[1](https://arxiv.org/html/2610.22825#Thmtheorem1)and[2](https://arxiv.org/html/2610.22825#Thmtheorem2)show how joint criticism can influence both the learned future representation and the subsequent selection among candidate actions\.

## 4Experiments

### 4\.1Dataset and Evaluation Metrics

#### Dataset\.

We conduct our experiments on the NAVSIM benchmark, which is built upon OpenScene and ultimately derived from the nuPlan dataset\. It covers diverse urban driving scenarios\. Following prior work, we train on thenavtrainsplit and evaluate on the completenavtestsplit\.

#### Evaluation Metrics\.

We evaluate planning on NAVSIM\-v1 with the PDM Score \(PDMS\)\. It consists of five sub\-metrics: No at\-fault Collisions \(NC\), Drivable Area Compliance \(DAC\), Time to Collision \(TTC\), Ego Progress \(EP\), and Comfort \(C\)\. NC and DAC serve as multiplicative penalties, while TTC, EP, and C are combined as a weighted average\. The overall PDMS is defined as:

PDMS=NC⋅DAC⋅5​E​P\+5​T​T​C\+2​C12\.\\mathrm\{PDMS\}=\\mathrm\{NC\}\\cdot\\mathrm\{DAC\}\\cdot\\frac\{5\\mathrm\{EP\}\+5\\mathrm\{TTC\}\+2\\mathrm\{C\}\}\{12\}\.\(16\)
Table 1:Comparison with end\-to\-end, vision\-language\-action, and world\-model/world\-action planners on NAVSIM\-v1\. We report overall PDMS and its safety, road\-compliance, comfort, and progress components; higher is better for all metrics\. Baseline results follow their reported sensor and training configurations\. Using only front\-camera observations and ego status, WAC achieves the highest overall PDMS and ego progress among the compared methods\.

### 4\.2Implementation Details

Our visual backbone is based on V\-JEPA 2\[[2](https://arxiv.org/html/2610.22825#bib.bib2)\]\. We initialize the ViT\-L encoder from the driving\-video\-pretrained Drive\-JEPA checkpoint\[[43](https://arxiv.org/html/2610.22825#bib.bib27)\]and jointly fine\-tune it with the planning modules using RADAR\[[36](https://arxiv.org/html/2610.22825#bib.bib45)\], with mixed precision, and constant learning rates\. We summarize the architecture and training hyperparameters in Appendix[D](https://arxiv.org/html/2610.22825#A4)\.

Each input consists of two consecutive front\-view observations\. The model proposes 32 candidate trajectories, each containing eight future poses over 4s, and evaluates them in parallel with shared WAC parameters\. Each recurrent prediction step advances by four poses, yielding two prediction steps over the planning horizon\. At each step, the complete candidate trajectory and driving command condition future\-state prediction and critic updates\.

In the driving instantiation, the statezkz\_\{k\}in Section[3\.2](https://arxiv.org/html/2610.22825#S3.SS2)comprises scene latents and ego dynamics, both retained in the history memory\. The future\-prediction objectiveℒz\\mathcal\{L\}\_\{z\}combines their respective Smooth L1 losses against factual targets, while NAVSIM PDM evaluations supervise six critic outputs: NC, DAC, EP, TTC, comfort, and aggregate PDMS\. At inference, we select the candidate with the highest predicted aggregate score\. The focal\-weighted variant is defined in Appendix[D](https://arxiv.org/html/2610.22825#A4)\. The ablations use the same NAVSIM\-v1 evaluation split and report PDMS in points\.

### 4\.3Main Results

Table[1](https://arxiv.org/html/2610.22825#S4.T1)compares WAC with end\-to\-end, vision\-language\-action \(VLA\), and world\-model/world\-action planners on NAVSIM\-v1\. Our front\-camera planner achieves93\.493\.4PDMS, the highest overall score among the compared methods, including DrivoR\[[24](https://arxiv.org/html/2610.22825#bib.bib39)\]and the full Drive\-JEPA planner\[[43](https://arxiv.org/html/2610.22825#bib.bib27)\]\. The results show that WAC achieves the highest ego progress while maintaining competitive safety and compliance scores, indicating a favorable balance between planning efficiency, safety, and compliance\. The following scoring analysis and controlled ablations examine how this behavior relates to candidate evaluation and joint predictive learning\.

### 4\.4Candidate Scoring Quality

Candidate selection depends on the relative quality assigned to alternative trajectories\. We therefore evaluate every proposed candidate with the NAVSIM\-v1 evaluator and examine two aspects of the learned scores: whether they preserve the ordering of candidate quality, measured by Spearman correlation and pairwise ranking accuracy, and whether they overrate poor alternatives, measured by low\-score overestimation\. The analysis covers the complete test split and uses each planner’s own candidate pool\. Metric definitions are given in Appendix[C](https://arxiv.org/html/2610.22825#A3)\.

Table 2:Candidate scoring quality on NAVSIM\-v1\. PA denotes pairwise ranking accuracy\.Table[2](https://arxiv.org/html/2610.22825#S4.T2)shows that WAC improves candidate ordering and reduces overestimation of low\-quality alternatives\. Consistent gains in Spearman correlation and pairwise ranking accuracy indicate that its scores more reliably distinguish preferable candidates within a scene\. Reduced low\-score overestimation addresses inflated quality estimates that can promote poor trajectories during selection\. The following ablations examine the contributions of joint prediction and evaluation\.

![Refer to caption](https://arxiv.org/html/2610.22825v1/wacm_clean_urban_following.png)

![Refer to caption](https://arxiv.org/html/2610.22825v1/wacm_clean_urban_junction.png)

![Refer to caption](https://arxiv.org/html/2610.22825v1/wacm_clean_road_compliance.png)

Figure 3:Selected trajectories in urban car\-following, a signalized urban junction, and a wet urban junction \(left to right\)\. Each column pairs the front\-camera trajectory projection with its BEV\. Red denotes Drive\-JEPA, green WAC \(Ours\), and blue dashed lines the recorded GT\. Thin lines show the 32 candidates per method; thick lines show their selections\. Endpoint dots in BEV mark four\-second predictions\.
### 4\.5Ablation Studies

#### Model structure and supervision\.

Table[3](https://arxiv.org/html/2610.22825#S4.T3)compares WAC variants trained without focal weighting and with the same four\-pose prediction interval\. This isolates the effect of recurrent prediction, factual future supervision, and critic\-to\-transition gradients within the same planner\.

Table 3:WAC structure and supervision ablations on NAVSIM\-v1\.Autoregressive prediction outperforms one\-shot future prediction\. The one\-shot variant predicts future latent states in parallel from the initial context, without feeding an imagined state into the next prediction\. WAC instead propagates its predicted latent and critic states between steps, allowing later consequence estimates to depend on earlier imagined states\. The comparison supports maintaining this sequential dependence when evaluating a candidate over the planning horizon\.

Removing factual future\-latent and ego\-state supervision weakens performance, indicating that future targets contribute beyond trajectory\-quality labels alone\. Blocking critic gradients from the predictive features causes the largest degradation among these variants\. The critic can still learn to score candidates in that setting, but its quality supervision cannot adapt the transition features toward action evaluation\. This comparison supports the central joint\-learning mechanism: predictive features benefit from being optimized for the consequences that matter to candidate selection\. Factual future supervision and critic feedback thus play complementary roles, grounding prediction in observed dynamics and making the learned features useful for decisions\.

\(a\)Scoring and selection\(b\)Low\-score overestimation\(c\)Score agreement
Figure 4:Candidate scoring in three selected improvement examples\. Each subfigure overlays all 32 Drive\-JEPA candidates \(red triangles\) and all 32 WAC candidates \(green circles\)\. Stars mark selections: red outlines for Drive\-JEPA and green fills for WAC\. The dashed line indicates equality and the shaded band is±5\\pm 5PDMS points\. Coincident candidates overlap\. Each model generates its own candidates\.
#### Prediction interval and focal weighting\.

Table[4](https://arxiv.org/html/2610.22825#S4.T4)varies the prediction interval while keeping the planning horizon fixed\. With focal weighting disabled, predicting four poses per step over two rollout steps achieves 93\.3 PDMS, compared with 92\.8 for two poses per step over four rollout steps\. The final model uses the four\-pose prediction interval\.

Table 4:Prediction interval and focal\-loss ablation on NAVSIM\-v1\. Every trajectory has eight future poses\.Poses per stepRollout stepsFocalPDMS↑\\uparrow24×\\times92\.842×\\times93\.342✓\\checkmark93\.4With the architecture and rollout length fixed, focal weighting further increases PDMS from 93\.3 to 93\.4\. It emphasizes difficult component predictions, while progress and aggregate PDMS retain ordinary binary cross\-entropy\.

### 4\.6Qualitative Analysis

Figure[3](https://arxiv.org/html/2610.22825#S4.F3)compares WAC with the Drive\-JEPA baseline in urban driving scenes\. The candidate sets and selected paths show how the planners resolve alternatives, while the NAVSIM evaluator assesses the consequences of their choices\.

The car\-following example separates immediate collision avoidance from temporal safety: both selections pass the collision check, but only WAC passes TTC\. At the signalized junction, the baseline selection violates drivable\-area compliance, whereas WAC preserves compliance while maintaining progress\. The wet\-junction example illustrates a similar distinction in road compliance\. These cases highlight improvements in temporal safety and drivable\-area compliance\. Appendix[B](https://arxiv.org/html/2610.22825#A2)connects further planning examples with their candidate rankings\.

#### Candidate\-level scoring and selection\.

Figure[4](https://arxiv.org/html/2610.22825#S4.F4)connects candidate\-level scoring with the final selection\. In \(a\), better agreement with evaluator scores accompanies lower selection regret\. In \(b\), the baseline promotes a poor candidate to its highest predicted score, whereas WAC selects a high\-quality alternative\. In \(c\), WAC’s candidate scores show closer agreement with the evaluator\.

## 5Conclusion

We presented the World Action Critic \(WAC\) Model, which combines candidate proposal, action\-conditioned future prediction, and explicit evaluation in one decision framework\. Its recurrent transition and critic streams extract evaluation evidence as each candidate’s future is formed\. Directed attention preserves a prediction\-to\-criticism dependency, while joint optimization lets candidate\-quality supervision shape the transition features\. Observed futures supervise logged actions, and external critic labels supervise additional candidates\. Our analysis explains decision\-relevant feature retention in a restricted capacity model and bounds candidate\-selection error under explicit coverage and evaluator assumptions\. NAVSIM experiments demonstrate the framework’s application to autonomous driving\. Extending this empirical validation beyond driving remains future work\.

## References

- \[1\]S\. Ang, Y\. Yang, C\. Dang, C\. Chen, C\. Chi, H\. Liu, X\. Mao, J\. Bao, Xuliang, B\. Sun, and Y\. Wang\(2026\)From representational complementarity to dual systems: synergizing vlm and vision\-only backbones for end\-to\-end driving\.Note:arXiv preprint arXiv:2602\.10719Accepted at NeurIPS 2026External Links:2602\.10719,[Link](https://arxiv.org/abs/2602.10719)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.15.1.1)\.
- \[2\]M\. Assran, A\. Bardes, D\. Fan, Q\. Garrido, R\. Howes, M\. Komeili, M\. Muckley, A\. Rizvi, C\. Roberts, K\. Sinha, A\. Zholus, S\. Arnaud, A\. Gejji, A\. Martin, F\. R\. Hogan, D\. Dugas, P\. Bojanowski, V\. Khalidov, P\. Labatut, F\. Massa, M\. Szafraniec, K\. Krishnakumar, Y\. Li, X\. Ma, S\. Chandar, F\. Meier, Y\. LeCun, M\. Rabbat, and N\. Ballas\(2025\)V\-JEPA 2: Self\-Supervised Video Models Enable Understanding, Prediction and Planning\.arXiv\.Note:arXiv preprint arXiv:2506\.09985External Links:2506\.09985,[Document](https://dx.doi.org/10.48550/arXiv.2506.09985)Cited by:[§4\.2](https://arxiv.org/html/2610.22825#S4.SS2.p1.1)\.
- \[3\]H\. Bi, H\. Tan, S\. Xie, Z\. Wang, S\. Huang, H\. Liu, R\. Zhao, Y\. Feng, C\. Xiang, Y\. Rong, H\. Zhao, H\. Liu, Z\. Su, L\. Ma, H\. Su, and J\. Zhu\(2026\)Motus: A Unified Latent Action World Model\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 35101–35113\.External Links:[Link](https://openaccess.thecvf.com/content/CVPR2026/html/Bi_Motus_A_Unified_Latent_Action_World_Model_CVPR_2026_paper.html)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.22825#S2.SS2.p1.1)\.
- \[4\]K\. Bousmalis, G\. Vezzani, D\. Rao, C\. Devin, A\. X\. Lee, M\. Bauza, T\. Davchev, Y\. Zhou, A\. Gupta, A\. Raju, A\. Laurens, C\. Fantacci, V\. Dalibard, M\. Zambelli, M\. Martins, R\. Pevceviciute, M\. Blokzijl, M\. Denil, N\. Batchelor, T\. Lampe, E\. Parisotto, K\. Żołna, S\. Reed, S\. G\. Colmenarejo, J\. Scholz, A\. Abdolmaleki, O\. Groth, J\. Regli, O\. Sushkov, T\. Rothörl, J\. E\. Chen, Y\. Aytar, D\. Barker, J\. Ortiz, M\. Riedmiller, J\. T\. Springenberg, R\. Hadsell, F\. Nori, and N\. Heess\(2024\)RoboCat: A Self\-Improving Generalist Agent for Robotic Manipulation\.Transactions on Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=vsCpLiIWHu)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1)\.
- \[5\]J\. Cen, C\. Yu, H\. Yuan, Y\. Jiang, S\. Huang, J\. Guo, X\. Li, Y\. Song, H\. Luo, F\. Wang, D\. Zhao, and H\. Chen\(2026\)WorldVLA: Towards Autoregressive Action World Model\.InAdvances in Neural Information Processing Systems,Note:Accepted at NeurIPS 2026External Links:[Link](https://cen-jun.com/)Cited by:[§2\.2](https://arxiv.org/html/2610.22825#S2.SS2.p1.1)\.
- \[6\]J\. Chen, S\. E\. Li, and M\. Tomizuka\(2022\)Interpretable end\-to\-end urban autonomous driving with latent deep reinforcement learning\.IEEE Transactions on Intelligent Transportation Systems23\(6\),pp\. 5068–5078\.External Links:[Document](https://dx.doi.org/10.1109/TITS.2020.3046646)Cited by:[§2\.1](https://arxiv.org/html/2610.22825#S2.SS1.p1.1)\.
- \[7\]D\. Dauner, M\. Hallgarten, T\. Li, X\. Weng, Z\. Huang, Z\. Yang, H\. Li, I\. Gilitschenski, B\. Ivanovic, M\. Pavone, A\. Geiger, and K\. Chitta\(2024\)NAVSIM: data\-driven non\-reactive autonomous vehicle simulation and benchmarking\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 28706–28719\.External Links:[Document](https://dx.doi.org/10.52202/079017-0902),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/32768f7faf1995026ef9821c696f3404-Paper-Datasets_and_Benchmarks_Track.pdf)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p6.1)\.
- \[8\]Y\. Du, S\. Yang, B\. Dai, H\. Dai, O\. Nachum, J\. Tenenbaum, D\. Schuurmans, and P\. Abbeel\(2023\)Learning universal policies via text\-guided video generation\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/1d5b9233ad716a43be5c0d3023cb82d0-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.22825#S2.SS2.p1.1)\.
- \[9\]A\. Farahmand, A\. Barreto, and D\. Nikovski\(2017\)Value\-Aware Loss Function for Model\-based Reinforcement Learning\.InProceedings of the 20th International Conference on Artificial Intelligence and Statistics,pp\. 1486–1494\.External Links:ISSN 2640\-3498Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p3.1),[§2\.3](https://arxiv.org/html/2610.22825#S2.SS3.p1.1)\.
- \[10\]S\. Fei, X\. Yu, S\. Wang, X\. Zhao, J\. Gong, and X\. Qiu\(2026\)WCM: A World Critic Model for Vision\-Language\-Action Reinforcement Learning\.arXiv\.Note:arXiv preprint arXiv:2607\.29613External Links:2607\.29613,[Document](https://dx.doi.org/10.48550/arXiv.2607.29613)Cited by:[§2\.3](https://arxiv.org/html/2610.22825#S2.SS3.p1.1)\.
- \[11\]Z\. Gao, Y\. Mu, C\. Chen, J\. Duan, P\. Luo, Y\. Lu, and S\. E\. Li\(2024\)Enhance sample efficiency and robustness of end\-to\-end urban autonomous driving via semantic masked world model\.IEEE Transactions on Intelligent Transportation Systems25\(10\),pp\. 13067–13079\.External Links:[Document](https://dx.doi.org/10.1109/TITS.2024.3400227)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p1.1),[§1](https://arxiv.org/html/2610.22825#S1.p3.1)\.
- \[12\]C\. Grimm, A\. Barreto, S\. Singh, and D\. Silver\(2020\)The value equivalence principle for model\-based reinforcement learning\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 5541–5552\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/file/3bb585ea00014b0e3ebe4c6dd165a358-Paper.pdf)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p3.1),[§2\.3](https://arxiv.org/html/2610.22825#S2.SS3.p1.1)\.
- \[13\]Y\. Guan, S\. E\. Li, J\. Duan, J\. Li, Y\. Ren, Q\. Sun, and B\. Cheng\(2021\)Direct and indirect reinforcement learning\.International Journal of Intelligent Systems36\(8\),pp\. 4439–4467\.External Links:[Document](https://dx.doi.org/10.1002/int.22466)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p1.1)\.
- \[14\]Y\. Guan, L\. Tang, C\. Li, S\. E\. Li, Y\. Ren, J\. Wei, B\. Zhang, and K\. Li\(2022\)Integrated decision and control for high\-level automated vehicles by mixed policy gradient and its experiment verification\.Note:arXiv preprint arXiv:2210\.10613External Links:2210\.10613,[Link](https://arxiv.org/abs/2210.10613)Cited by:[§2\.1](https://arxiv.org/html/2610.22825#S2.SS1.p1.1)\.
- \[15\]Y\. Guan, L\. Tang, Y\. Lyu, S\. E\. Li, Y\. Yang, K\. Sheng, B\. Zhang, L\. He, and K\. Li\(2026\)Enhanced integrated decision and control for high\-level automated vehicles and its experiment verification\.IEEE Transactions on Automation Science and Engineering23,pp\. 11578–11595\.External Links:[Document](https://dx.doi.org/10.1109/TASE.2026.3695533)Cited by:[§2\.1](https://arxiv.org/html/2610.22825#S2.SS1.p1.1)\.
- \[16\]K\. Guo, H\. Liu, X\. Wu, J\. Pan, and C\. Lv\(2025\)IPad: iterative proposal\-centric end\-to\-end autonomous driving\.Note:arXiv preprint arXiv:2505\.15111External Links:2505\.15111,[Link](https://arxiv.org/abs/2505.15111)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.4.1.1)\.
- \[17\]D\. Ha and J\. Schmidhuber\(2018\)Recurrent world models facilitate policy evolution\.InAdvances in Neural Information Processing Systems,S\. Bengio, H\. Wallach, H\. Larochelle, K\. Grauman, N\. Cesa\-Bianchi, and R\. Garnett \(Eds\.\),Vol\.31,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2018/file/2de5d16682c3c35007e4e92982f1a2ba-Paper.pdf)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p1.1)\.
- \[18\]D\. Hafner, T\. Lillicrap, J\. Ba, and M\. Norouzi\(2020\)Dream to Control: Learning Behaviors by Latent Imagination\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=S1lOTC4tDS)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.22825#S2.SS1.p1.1)\.
- \[19\]D\. Hafner, T\. Lillicrap, I\. Fischer, R\. Villegas, D\. Ha, H\. Lee, and J\. Davidson\(2019\)Learning latent dynamics for planning from pixels\.InInternational Conference on Machine Learning,pp\. 2555–2565\.Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1)\.
- \[20\]D\. Hafner, J\. Pasukonis, J\. Ba, and T\. Lillicrap\(2025\)Mastering diverse control tasks through world models\.Nature640\(8059\),pp\. 647–653\.External Links:ISSN 1476\-4687,[Document](https://dx.doi.org/10.1038/s41586-025-08744-2)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p1.1)\.
- \[21\]N\. Hansen, H\. Su, and X\. Wang\(2024\)TD\-MPC2: Scalable, Robust World Models for Continuous Control\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 47376–47405\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/cf73d57b6dcda32b293df7c2d5341f49-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.22825#S2.SS1.p1.1)\.
- \[22\]Y\. Hong, X\. Zhou, Y\. Li, X\. Zhou, L\. Liu, Y\. Luo, S\. Xu, L\. Yang, and Z\. Song\(2026\)DriveFuture: Future\-Aware Latent World Models for Autonomous Driving\.arXiv\.Note:arXiv preprint arXiv:2605\.09701External Links:2605\.09701,[Document](https://dx.doi.org/10.48550/arXiv.2605.09701)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.26.1.1)\.
- \[23\]F\. Jia, L\. Liu, Z\. Song, C\. Jia, H\. Ye, X\. Hao, and L\. Chen\(2026\)DriveWorld\-vla: unified latent\-space world modeling with vision\-language\-action for autonomous driving\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.306,pp\. 51836–51856\.External Links:[Link](https://proceedings.mlr.press/v306/jia26f.html)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.13.1.1)\.
- \[24\]E\. Kirby, A\. Boulch, Y\. Xu, Y\. Yin, G\. Puy, É\. Zablocki, A\. Bursuc, S\. Gidaris, R\. Marlet, F\. Bartoccioni, A\. Cao, N\. Samet, T\. Vu, and M\. Cord\(2026\)Driving on registers\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 32058–32069\.External Links:[Link](https://openaccess.thecvf.com/content/CVPR2026/html/Kirby_Driving_on_Registers_CVPR_2026_paper.html)Cited by:[§4\.3](https://arxiv.org/html/2610.22825#S4.SS3.p1.1),[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.7.1.1)\.
- \[25\]Z\. Lan, Y\. Jiang, Y\. Mu, C\. Chen, and S\. Li\(2024\)SEPT: towards efficient scene representation learning for motion prediction\.InInternational Conference on Learning Representations,B\. Kim, Y\. Yue, S\. Chaudhuri, K\. Fragkiadaki, M\. Khan, and Y\. Sun \(Eds\.\),Vol\.2024,pp\. 33541–33554\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/file/8eff4196f50c43eda7bcf0f0cf87a0d0-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1)\.
- \[26\]Q\. Li, X\. Jia, S\. Wang, and J\. Yan\(2024\)Think2Drive: efficient reinforcement learning by thinking with latent world model for autonomous driving \(in carla\-v2\)\.InComputer Vision – ECCV 2024,pp\. 142–158\.External Links:[Document](https://dx.doi.org/10.1007/978-3-031-72995-9%5F9)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1)\.
- \[27\]S\. E\. Li\(2023\)Reinforcement learning for sequential decision and optimal control\.Springer,Singapore\.External Links:ISBN 978\-981\-19\-7784\-8Cited by:[§2\.1](https://arxiv.org/html/2610.22825#S2.SS1.p1.1)\.
- \[28\]T\. Li, L\. Chen, C\. Wang, H\. Liu, K\. Chitta, Z\. Yang, Y\. Lu, N\. Ye, Y\. Qiu, Y\. Wang, L\. Zou, J\. Peng, J\. Pan, Z\. Su, A\. Bursuc, S\. E\. Li, A\. Geiger, P\. Su, and H\. Li\(2026\)World Engine: towards the era of post\-training for autonomous driving\.Note:arXiv preprint arXiv:2606\.19836External Links:2606\.19836,[Link](https://arxiv.org/abs/2606.19836)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p1.1)\.
- \[29\]Y\. Li, L\. Fan, J\. He, Y\. Wang, Y\. Chen, Z\. Zhang, and T\. Tan\(2025\)Enhancing end\-to\-end autonomous driving with latent world model\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 42942–42959\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/6aa4967920e495e90aeeaa3acf18d019-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1)\.
- \[30\]Y\. Li, S\. Shang, W\. Liu, B\. Zhan, H\. Wang, Y\. Wang, Y\. Chen, X\. Wang, Y\. An, C\. Tang, L\. Hou, L\. Fan, and Z\. Zhang\(2025\)DriveVLA\-w0: world models amplify data scaling law in autonomous driving\.Note:arXiv preprint arXiv:2510\.12796External Links:2510\.12796,[Link](https://arxiv.org/abs/2510.12796)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.11.1.1)\.
- \[31\]Y\. Li, Y\. Wang, Y\. Liu, J\. He, L\. Fan, and Z\. Zhang\(2025\)End\-to\-End Driving with Online Trajectory Evaluation via BEV World Model\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 27137–27146\.External Links:[Link](https://openaccess.thecvf.com/content/ICCV2025/html/Li_End-to-End_Driving_with_Online_Trajectory_Evaluation_via_BEV_World_Model_ICCV_2025_paper.html)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p5.1),[§2\.3](https://arxiv.org/html/2610.22825#S2.SS3.p1.1),[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.21.1.1)\.
- \[32\]Y\. Li, K\. Xiong, X\. Guo, F\. Li, S\. Yan, G\. Xu, L\. Zhou, L\. Chen, H\. Sun, B\. Wang, K\. Ma, G\. Chen, H\. Ye, W\. Liu, and X\. Wang\(2025\)ReCogDrive: a reinforced cognitive framework for end\-to\-end autonomous driving\.Note:arXiv preprint arXiv:2506\.08052v2External Links:[Link](https://arxiv.org/abs/2506.08052v2),2506\.08052v2Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.12.1.1)\.
- \[33\]C\. Min, D\. Zhao, L\. Xiao, J\. Zhao, X\. Xu, Z\. Zhu, L\. Jin, J\. Li, Y\. Guo, J\. Xing, L\. Jing, Y\. Nie, and B\. Dai\(2024\)DriveWorld: 4D Pre\-trained Scene Understanding via World Models for Autonomous Driving\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 15522–15533\.External Links:[Link](https://openaccess.thecvf.com/content/CVPR2024/html/Min_DriveWorld_4D_Pre-trained_Scene_Understanding_via_World_Models_for_Autonomous_CVPR_2024_paper.html)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1)\.
- \[34\]Y\. Mu, Y\. Zhuang, B\. Wang, G\. Zhu, W\. Liu, J\. Chen, P\. Luo, S\. Li, C\. Zhang, and J\. Hao\(2021\)Model\-Based Reinforcement Learning via Imagination with Derived Memory\.InAdvances in Neural Information Processing Systems,Vol\.34,pp\. 9493–9505\.Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1)\.
- \[35\]W\. Peebles and S\. Xie\(2023\)Scalable Diffusion Models with Transformers\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 4195–4205\.External Links:[Link](https://openaccess.thecvf.com/content/ICCV2023/html/Peebles_Scalable_Diffusion_Models_with_Transformers_ICCV_2023_paper.html)Cited by:[§3\.2](https://arxiv.org/html/2610.22825#S3.SS2.SSS0.Px2.p2.1)\.
- \[36\]Z\. Ren, Y\. Lyu, C\. Li, L\. Zhang, and S\. E\. Li\(2026\)Momentum as residual\-driven multiplier correction for deep learning optimization\.arXiv preprint arXiv:2608\.12925\.Cited by:[§4\.2](https://arxiv.org/html/2610.22825#S4.SS2.p1.1)\.
- \[37\]S\. Shang, B\. Zhan, Y\. Yan, Y\. Wang, Y\. Li, Y\. An, X\. Wang, J\. Liu, L\. Hou, L\. Fan, Z\. Zhang, and T\. Tan\(2026\)DynVLA: learning world dynamics for action reasoning in autonomous driving\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.306,pp\. 110028–110045\.External Links:[Link](https://proceedings.mlr.press/v306/shang26f.html)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.14.1.1)\.
- \[38\]C\. Shi, J\. Xu, S\. Shi, K\. Sheng, B\. Zhang, and L\. Jiang\(2026\)DriveWAM: Video Generative Priors Enable Scalable World\-Action Modeling for Autonomous Driving\.InAdvances in Neural Information Processing Systems,Note:Accepted at NeurIPS 2026External Links:[Link](https://github.com/chenshi3/DriveWAM)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.22825#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.24.1.1)\.
- \[39\]W\. Sun, X\. Lin, K\. Chen, Z\. Pei, X\. Li, Y\. Shi, and S\. Zheng\(2026\)SparseDriveV2: scoring is all you need for end\-to\-end autonomous driving\.Note:arXiv preprint arXiv:2603\.29163External Links:2603\.29163,[Link](https://arxiv.org/abs/2603.29163)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.5.1.1)\.
- \[40\]SunRisingAI Lab\(2026\)Phi\-WM 1\.0 ActEffect: from predictive foresight to consequence feedback in robot learning\.Note:Technical reportExternal Links:[Link](https://www.sunrisingai.com/upload/202608/e597763b7e870a6ec5951d487b4b3e7f.pdf)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p1.1)\.
- \[41\]C\. Voelcker, V\. Liao, A\. Garg, and A\. Farahmand\(2022\)Value Gradient weighted Model\-Based Reinforcement Learning\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=4-D6CZkRXxI)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p3.1),[§2\.3](https://arxiv.org/html/2610.22825#S2.SS3.p1.1)\.
- \[42\]L\. Wang, X\. Zhang, Y\. Wang, G\. Zhan, W\. Wang, H\. Gao, J\. Duan, and S\. E\. Li\(2025\)Off\-policy Reinforcement Learning with Model\-based Exploration Augmentation\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 92121–92151\.Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.22825#S2.SS1.p1.1)\.
- \[43\]L\. Wang, Z\. Yang, C\. Bai, G\. Zhang, X\. Liu, X\. Zheng, X\. Long, C\. Lu, and C\. Lu\(2026\)Drive\-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End\-to\-End Driving\.arXiv\.Note:NAVSIM\-v1 results cited from arXiv version 1arXiv preprint arXiv:2601\.22032External Links:2601\.22032,[Document](https://dx.doi.org/10.48550/arXiv.2601.22032),[Link](https://arxiv.org/pdf/2601.22032v1)Cited by:[§4\.2](https://arxiv.org/html/2610.22825#S4.SS2.p1.1),[§4\.3](https://arxiv.org/html/2610.22825#S4.SS3.p1.1),[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.8.1.1)\.
- \[44\]Y\. Wang, J\. He, L\. Fan, H\. Li, Y\. Chen, and Z\. Zhang\(2024\)Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 14749–14759\.Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.22825#S2.SS1.p1.1)\.
- \[45\]Z\. Wang, J\. Li, Y\. Cui, Y\. Gao, X\. Zhan, J\. Yu, and X\. Ma\(2026\)World Value Models for Robotic Manipulation\.arXiv\.Note:arXiv preprint arXiv:2606\.24742External Links:2606\.24742,[Document](https://dx.doi.org/10.48550/arXiv.2606.24742)Cited by:[§2\.3](https://arxiv.org/html/2610.22825#S2.SS3.p1.1)\.
- \[46\]H\. Wu, M\. Steiner, M\. Schmidt, A\. Marcos\-Ramiro, and C\. Stiller\(2025\)NaviHydra: controllable navigation\-guided end\-to\-end autonomous driving with hydra\-distillation\.Note:arXiv preprint arXiv:2512\.10660External Links:2512\.10660,[Link](https://arxiv.org/pdf/2512.10660v1)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.6.1.1)\.
- \[47\]P\. Wu, A\. Escontrela, D\. Hafner, P\. Abbeel, and K\. Goldberg\(2023\)DayDreamer: World Models for Physical Robot Learning\.InProceedings of The 6th Conference on Robot Learning,Proceedings of Machine Learning Research, Vol\.205,pp\. 2226–2240\.External Links:[Link](https://proceedings.mlr.press/v205/wu23c.html)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.22825#S2.SS1.p1.1)\.
- \[48\]T\. Xia, Y\. Li, L\. Zhou, J\. Yao, K\. Xiong, H\. Sun, B\. Wang, K\. Ma, G\. Chen, H\. Ye, W\. Liu, and X\. Wang\(2026\)DriveLaW: unifying planning and video generation in a latent driving world\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 39701–39712\.External Links:[Link](https://openaccess.thecvf.com/content/CVPR2026/html/Xia_DriveLaW_Unifying_Planning_and_Video_Generation_in_a_Latent_Driving_CVPR_2026_paper.html)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.22.1.1)\.
- \[49\]T\. Xia, L\. Zhou, K\. Xiong, J\. Yao, Z\. Zhu, H\. Sun, B\. Wang, G\. Chen, W\. Liu, H\. Ye, and X\. Wang\(2026\)ReWorld: representation learning for world action models\.Note:arXiv preprint arXiv:2606\.27504External Links:2606\.27504,[Link](https://arxiv.org/abs/2606.27504)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.25.1.1)\.
- \[50\]C\. Xie, B\. Sun, T\. Li, J\. Wu, Z\. Hao, X\. Lang, and H\. Li\(2026\)LatentVLA: efficient vision\-language models for autonomous driving via latent action prediction\.Note:arXiv preprint arXiv:2601\.05611v1Version 1; results cited for LatentVLA \(iPad\)External Links:[Link](https://arxiv.org/pdf/2601.05611v1),2601\.05611v1Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.16.1.1)\.
- \[51\]X\. Yan, Z\. Lian, N\. Zhang, S\. Fang, H\. Wang, C\. Lv, J\. Hu, and B\. Song\(2026\)WCog\-vla: a dual\-level world\-cognitive vision\-language\-action model for end\-to\-end autonomous driving\.Note:arXiv preprint arXiv:2607\.08375External Links:2607\.08375,[Link](https://arxiv.org/abs/2607.08375)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.17.1.1)\.
- \[52\]J\. Yang, Z\. Chen, C\. Huang, and J\. Li\(2026\)Auto\-jepa: a latent world model of continuous intent for end\-to\-end autonomous driving\.Note:arXiv preprint arXiv:2607\.29031External Links:2607\.29031,[Link](https://arxiv.org/abs/2607.29031)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.27.1.1)\.
- \[53\]P\. Yang, B\. Lu, Z\. Xia, C\. Han, Y\. Gao, T\. Zhang, K\. Zhan, X\. Lang, Y\. Zheng, and Q\. Zhang\(2026\)WorldRFT: latent world model planning with reinforcement fine\-tuning for autonomous driving\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 11649–11657\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v40i14.38149)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.20.1.1)\.
- \[54\]S\. Ye, Y\. Ge, K\. Zheng, S\. Gao, S\. Yu, G\. Kurian, S\. Indupuru, Y\. L\. Tan, C\. Zhu, J\. Xiang, A\. Malik, K\. Lee, W\. Liang, N\. Ranawaka, J\. Gu, Y\. Xu, G\. Wang, F\. Hu, A\. Narayan, J\. Bjorck, J\. Wang, G\. Kim, D\. Niu, R\. Zheng, Y\. Xie, J\. Wu, Q\. Wang, R\. Julian, D\. Xu, Y\. Du, Y\. Chebotar, S\. Reed, J\. Kautz, Y\. Zhu, L\. ”\. Fan, and J\. Jang\(2026\)World Action Models are Zero\-shot Policies\.arXiv\.Note:arXiv preprint arXiv:2602\.15922External Links:2602\.15922,[Document](https://dx.doi.org/10.48550/arXiv.2602.15922)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1)\.
- \[55\]G\. Zhan, L\. Wang, F\. Zhang, Y\. Guan, and S\. E\. Li\(2026\)Harmonized dual policy improvement for modelic reinforcement learning\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.306,pp\. 154025–154043\.External Links:[Link](https://proceedings.mlr.press/v306/zhan26e.html)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p1.1)\.
- \[56\]G\. Zhan, L\. Wang, X\. Zhang, J\. Gao, M\. TOMIZUKA, and S\. E\. Li\(2025\)Bootstrap Off\-policy with World Model\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 134093–134121\.Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.22825#S2.SS1.p1.1)\.
- \[57\]K\. Zhang, Z\. Tang, X\. Hu, X\. Pan, X\. Guo, Y\. Liu, J\. Huang, L\. Yuan, Q\. Zhang, X\. Long, X\. Cao, and W\. Yin\(2025\)Epona: Autoregressive Diffusion World Model for Autonomous Driving\.arXiv\.Note:arXiv preprint arXiv:2506\.24113External Links:2506\.24113,[Document](https://dx.doi.org/10.48550/arXiv.2506.24113)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.19.1.1)\.
- \[58\]T\. Zhang, L\. Wang, G\. Zhan, F\. Zhang, Y\. Guan, Y\. Lyu, and S\. E\. Li\(2026\)Langevin rollout optimization for modelic reinforcement learning\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.306,pp\. 159138–159159\.External Links:[Link](https://proceedings.mlr.press/v306/zhang26gi.html)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p1.1)\.
- \[59\]X\. Zhang, Y\. Guan, B\. Zhang, H\. Li, Y\. Zhang, and S\. E\. Li\(2026\)On the Identifiability of Controlled World Models\.arXiv\.Note:arXiv preprint arXiv:2607\.22430External Links:2607\.22430,[Document](https://dx.doi.org/10.48550/arXiv.2607.22430)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p3.1)\.
- \[60\]Z\. Zhao, X\. Zhou, T\. Xu, Z\. Sun, K\. Zhou, H\. Li, D\. Liang, and X\. Bai\(2026\)SimWAM: A Simple World Action Model for End\-to\-End Autonomous Driving\.Note:arXiv preprint arXiv:2608\.07468Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.28.1.1)\.
- \[61\]Y\. Zheng, P\. Yang, Z\. Xing, Q\. Zhang, Y\. Zheng, Y\. Gao, P\. Li, T\. Zhang, Z\. Xia, P\. Jia, X\. Lang, and D\. Zhao\(2025\)World4Drive: End\-to\-End Autonomous Driving via Intention\-aware Physical Latent World Model\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 28632–28642\.External Links:[Link](https://openaccess.thecvf.com/content/ICCV2025/html/Zheng_World4Drive_End-to-End_Autonomous_Driving_via_Intention-aware_Physical_Latent_World_Model_ICCV_2025_paper.html)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p5.1),[§2\.3](https://arxiv.org/html/2610.22825#S2.SS3.p1.1)\.
- \[62\]G\. Zhou, H\. Pan, Y\. LeCun, and L\. Pinto\(2025\)DINO\-WM: World Models on Pre\-trained Visual Features enable Zero\-shot Planning\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 79115–79135\.External Links:[Link](https://proceedings.mlr.press/v267/zhou25t.html)Cited by:[§2\.1](https://arxiv.org/html/2610.22825#S2.SS1.p1.1)\.
- \[63\]Y\. Zhou, X\. Wang, H\. Shao, L\. Wang, G\. Zhao, J\. Shao, J\. Zhu, T\. Yu, Z\. Zhu, G\. Huang, and S\. L\. Waslander\(2026\)DriveDreamer\-policy: a geometry\-grounded world\-action model for unified generation and planning\.Note:arXiv preprint arXiv:2604\.01765External Links:2604\.01765,[Link](https://arxiv.org/abs/2604.01765)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.23.1.1)\.
- \[64\]Z\. Zhou, T\. Cai, S\. Z\. Zhao, Y\. Zhang, Z\. Huang, B\. Zhou, and J\. Ma\(2025\)AutoVLA: a vision\-language\-action model for end\-to\-end autonomous driving with adaptive reasoning and reinforcement fine\-tuning\.InAdvances in Neural Information Processing Systems,Vol\.38\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2025/hash/2843fccca5bedd369a4764848b9bd546-Abstract-Conference.html)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.10.1.1)\.
- \[65\]C\. Zhu, R\. Yu, S\. Feng, B\. Burchfiel, P\. Shah, and A\. Gupta\(2025\)Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets\.InProceedings of Robotics: Science and Systems,External Links:[Document](https://dx.doi.org/10.15607/RSS.2025.XXI.015),[Link](https://www.roboticsproceedings.org/rss21/p015.html)Cited by:[§1](https://arxiv.org/html/2610.22825#S1.p2.1)\.
- \[66\]J\. Zou, S\. Chen, B\. Liao, Z\. Zheng, Y\. Song, L\. Zhang, Q\. Zhang, W\. Liu, and X\. Wang\(2025\)DiffusionDriveV2: reinforcement learning\-constrained truncated diffusion modeling in end\-to\-end autonomous driving\.Note:arXiv preprint arXiv:2512\.07745External Links:2512\.07745,[Link](https://arxiv.org/abs/2512.07745)Cited by:[Table 1](https://arxiv.org/html/2610.22825#S4.T1.5.3.1.1)\.

## Appendix ATheoretical Analysis

### A\.1Critic\-Guided Predictive Representations

Letz′∈ℝDz^\{\\prime\}\\in\\mathbb\{R\}^\{D\}satisfy𝔼⁡\[z′\]=0\\mathbb\{E\}\[z^\{\\prime\}\]=0andCov⁡\(z′\)=Σ≻0\\operatorname\{Cov\}\(z^\{\\prime\}\)=\\Sigma\\succ 0\. The critic target isq=b⊤​z′\+ξq=b^\{\\top\}z^\{\\prime\}\+\\xi, where𝔼⁡\[ξ∣z′\]=0\\mathbb\{E\}\[\\xi\\mid z^\{\\prime\}\]=0andσξ2=𝔼⁡\[ξ2\]<∞\\sigma\_\{\\xi\}^\{2\}=\\mathbb\{E\}\[\\xi^\{2\}\]<\\infty\. Withz~=Σ−1/2z′\\widetilde\{z\}=\\Sigma^\{\-1/2\}z^\{\\prime\}andv=Σ1/2​bv=\\Sigma^\{1/2\}b, we haveCov⁡\(z~\)=I\\operatorname\{Cov\}\(\\widetilde\{z\}\)=Iandq=v⊤​z~\+ξq=v^\{\\top\}\\widetilde\{z\}\+\\xi\.

For1≤d<D1\\leq d<D, consider the representationU⊤​z~U^\{\\top\}\\widetilde\{z\}withU⊤​U=IdU^\{\\top\}U=I\_\{d\}\. Its optimal linear prediction and critic objectives are

ℛz​\(U\)\\displaystyle\\mathcal\{R\}\_\{z\}\(U\)=minA⁡𝔼​‖z′−A​U⊤​z~‖22,\\displaystyle=\\min\_\{A\}\\mathbb\{E\}\\\|z^\{\\prime\}\-AU^\{\\top\}\\widetilde\{z\}\\\|\_\{2\}^\{2\},\(17\)ℛq​\(U\)\\displaystyle\\mathcal\{R\}\_\{q\}\(U\)=minw⁡𝔼​\(q−w⊤​U⊤​z~\)2,\\displaystyle=\\min\_\{w\}\\mathbb\{E\}\(q\-w^\{\\top\}U^\{\\top\}\\widetilde\{z\}\)^\{2\},\(18\)whereA∈ℝD×dA\\in\\mathbb\{R\}^\{D\\times d\}andw∈ℝdw\\in\\mathbb\{R\}^\{d\}\. Joint learning minimizesℛz​\(U\)\+λ​ℛq​\(U\)\\mathcal\{R\}\_\{z\}\(U\)\+\\lambda\\mathcal\{R\}\_\{q\}\(U\)forλ≥0\\lambda\\geq 0\.

###### Proof of Theorem[1](https://arxiv.org/html/2610.22825#Thmtheorem1)\.

Sincez~\\widetilde\{z\}has identity covariance and the noise is orthogonal to its linear functions, least squares gives the optimal headsA=Σ1/2​UA=\\Sigma^\{1/2\}Uandw=U⊤​vw=U^\{\\top\}v\. Substituting them yields

ℛz​\(U\)\\displaystyle\\mathcal\{R\}\_\{z\}\(U\)=tr⁡\(Σ\)−tr⁡\(U⊤​Σ​U\),\\displaystyle=\\operatorname\{tr\}\(\\Sigma\)\-\\operatorname\{tr\}\(U^\{\\top\}\\Sigma U\),\(19\)ℛq​\(U\)\\displaystyle\\mathcal\{R\}\_\{q\}\(U\)=σξ2\+‖v‖22−‖U⊤​v‖22\.\\displaystyle=\\sigma\_\{\\xi\}^\{2\}\+\\\|v\\\|\_\{2\}^\{2\}\-\\\|U^\{\\top\}v\\\|\_\{2\}^\{2\}\.\(20\)Consequently,

ℛz​\(U\)\+λ​ℛq​\(U\)\\displaystyle\\mathcal\{R\}\_\{z\}\(U\)\+\\lambda\\mathcal\{R\}\_\{q\}\(U\)=tr⁡\(Σ\)\+λ⁡\(σξ2\+‖v‖22\)\\displaystyle=\\operatorname\{tr\}\(\\Sigma\)\+\\lambda\(\\sigma\_\{\\xi\}^\{2\}\+\\\|v\\\|\_\{2\}^\{2\}\)\(21\)−tr⁡\(U⊤​Kλ​U\),\\displaystyle\-\\operatorname\{tr\}\(U^\{\\top\}K\_\{\\lambda\}U\),whereKλ=Σ\+λ​v​v⊤K\_\{\\lambda\}=\\Sigma\+\\lambda vv^\{\\top\}\. The first two terms are independent ofUU, so joint learning maximizestr⁡\(U⊤​Kλ​U\)\\operatorname\{tr\}\(U^\{\\top\}K\_\{\\lambda\}U\)\.

Letη1≥⋯≥ηD\\eta\_\{1\}\\geq\\cdots\\geq\\eta\_\{D\}andr1,…,rDr\_\{1\},\\ldots,r\_\{D\}be the eigenvalues and orthonormal eigenvectors ofKλK\_\{\\lambda\}\. Then

tr⁡\(U⊤​Kλ​U\)=∑j=1Dηj​‖U⊤​rj‖22≤∑j=1dηj,\\operatorname\{tr\}\(U^\{\\top\}K\_\{\\lambda\}U\)=\\sum\_\{j=1\}^\{D\}\\eta\_\{j\}\\\|U^\{\\top\}r\_\{j\}\\\|\_\{2\}^\{2\}\\leq\\sum\_\{j=1\}^\{d\}\\eta\_\{j\},\(22\)because the weights‖U⊤​rj‖22\\\|U^\{\\top\}r\_\{j\}\\\|\_\{2\}^\{2\}lie in\[0,1\]\[0,1\]and sum todd\. Equality is attained by a leadingdd\-dimensional eigenspace, with any maximizing choice admissible at a tied eigenvalue\. This proves \([12](https://arxiv.org/html/2610.22825#S3.E12)\)\. ∎

#### Example of retaining a low\-variance feature\.

TakeD=2D=2,d=1d=1,Σ=diag⁡\(1,4\)\\Sigma=\\operatorname\{diag\}\(1,4\), andq=2​z1′q=2z^\{\\prime\}\_\{1\}with zero noise\. Prediction alone retains the higher\-variance featurez2′z^\{\\prime\}\_\{2\}, while the critic target depends entirely onz1′z^\{\\prime\}\_\{1\}\. Herev=\(2,0\)⊤v=\(2,0\)^\{\\top\}, soKλ=diag⁡\(1\+4​λ,4\)K\_\{\\lambda\}=\\operatorname\{diag\}\(1\+4\\lambda,4\)\. Forλ\>3/4\\lambda\>3/4, the leading direction switches toz1′z^\{\\prime\}\_\{1\}\. The critic term thus promotes a low\-variance future feature according to its relevance to action evaluation\.

Whitening provides a parameterization of rank\-ddlinear representations, with changes of basis absorbed into the output heads\.

### A\.2Candidate Selection under Critic Supervision

###### Proof of Theorem[2](https://arxiv.org/html/2610.22825#Thmtheorem2)\.

Fix the learned proposal policy and critic, and writee⁡\(z,a\)=q^​\(z,a\)−q⋆​\(z,a\)e\(z,a\)=\\widehat\{q\}\(z,a\)\-q^\{\\star\}\(z,a\)\. For a realized candidate set, leti⋆i^\{\\star\}andi^\\widehat\{i\}index the task\-optimal and selected candidates\. Substitutingq⋆=q^−eq^\{\\star\}=\\widehat\{q\}\-einto the regret gives

R\\displaystyle R:=q⋆​\(z,a\(i⋆\)\)−q⋆​\(z,a\(i^\)\)\\displaystyle:=q^\{\\star\}\(z,a^\{\(i^\{\\star\}\)\}\)\-q^\{\\star\}\(z,a^\{\(\\widehat\{i\}\)\}\)\(23\)=q^​\(z,a\(i⋆\)\)−q^​\(z,a\(i^\)\)\\displaystyle=\\widehat\{q\}\(z,a^\{\(i^\{\\star\}\)\}\)\-\\widehat\{q\}\(z,a^\{\(\\widehat\{i\}\)\}\)\+e⁡\(z,a\(i^\)\)−e⁡\(z,a\(i⋆\)\)\\displaystyle\+e\(z,a^\{\(\\widehat\{i\}\)\}\)\-e\(z,a^\{\(i^\{\\star\}\)\}\)≤e⁡\(z,a\(i^\)\)−e⁡\(z,a\(i⋆\)\)\.\\displaystyle\\leq e\(z,a^\{\(\\widehat\{i\}\)\}\)\-e\(z,a^\{\(i^\{\\star\}\)\}\)\.The last inequality follows becausei^\\widehat\{i\}maximizes the predicted score, making the difference of predicted scores nonpositive\. Thus regret is controlled by the errors on the selected and task\-optimal candidates\.

If the indices agree,R=0R=0\. Otherwise, usingR≥0R\\geq 0and\(x−y\)2≤2​\(x2\+y2\)\(x\-y\)^\{2\}\\leq 2\(x^\{2\}\+y^\{2\}\)yields

R2\\displaystyle R^\{2\}≤2​\[e​\(z,a\(i^\)\)2\+e​\(z,a\(i⋆\)\)2\]\\displaystyle\\leq 2\\bigl\[e\(z,a^\{\(\\widehat\{i\}\)\}\)^\{2\}\+e\(z,a^\{\(i^\{\\star\}\)\}\)^\{2\}\\bigr\]\(24\)≤2​∑i=1Ne​\(z,a\(i\)\)2\.\\displaystyle\\leq 2\\sum\_\{i=1\}^\{N\}e\(z,a^\{\(i\)\}\)^\{2\}\.To average this bound over test instances, recall thatν\\nusamples one candidate uniformly from each proposed set\. Hence

𝔼ν​e2=𝔼⁡\[1N​∑i=1Ne​\(z,a\(i\)\)2\]\.\\mathbb\{E\}\_\{\\nu\}e^\{2\}=\\mathbb\{E\}\\\!\\left\[\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}e\(z,a^\{\(i\)\}\)^\{2\}\\right\]\.\(25\)This identity holds for any dependence among the candidates\. Coverage then transfers the error to the critic\-supervised distribution,

𝔼ν​e2=𝔼μ​\[d​νd​μ​e2\]≤κ​𝔼μ​e2\.\\mathbb\{E\}\_\{\\nu\}e^\{2\}=\\mathbb\{E\}\_\{\\mu\}\\\!\\left\[\\frac\{d\\nu\}\{d\\mu\}e^\{2\}\\right\]\\leq\\kappa\\mathbb\{E\}\_\{\\mu\}e^\{2\}\.\(26\)Combining these steps and applying Cauchy–Schwarz inequality, we have

𝔼​R≤𝔼​R2≤2​N​𝔼ν​e2≤2​N​κ​‖e‖L2​\(μ\)\.\\mathbb\{E\}R\\leq\\sqrt\{\\mathbb\{E\}R^\{2\}\}\\leq\\sqrt\{2N\\mathbb\{E\}\_\{\\nu\}e^\{2\}\}\\leq\\sqrt\{2N\\kappa\}\\,\\\|e\\\|\_\{L\_\{2\}\(\\mu\)\}\.\(27\)Finally, decomposee=\(q^−qext\)\+\(qext−q⋆\)e=\(\\widehat\{q\}\-q\_\{\\mathrm\{ext\}\}\)\+\(q\_\{\\mathrm\{ext\}\}\-q^\{\\star\}\)into critic fitting error and evaluator discrepancy\. The triangle inequality inL2​\(μ\)L\_\{2\}\(\\mu\)gives

‖e‖L2​\(μ\)\\displaystyle\\\|e\\\|\_\{L\_\{2\}\(\\mu\)\}≤‖q^−qext‖L2​\(μ\)\+‖qext−q⋆‖L2​\(μ\)\\displaystyle\\leq\\\|\\widehat\{q\}\-q\_\{\\mathrm\{ext\}\}\\\|\_\{L\_\{2\}\(\\mu\)\}\+\\\|q\_\{\\mathrm\{ext\}\}\-q^\{\\star\}\\\|\_\{L\_\{2\}\(\\mu\)\}\(28\)=ϵfit\+ϵext\.\\displaystyle=\\sqrt\{\\epsilon\_\{\\mathrm\{fit\}\}\}\+\\sqrt\{\\epsilon\_\{\\mathrm\{ext\}\}\}\.Substitution proves \([15](https://arxiv.org/html/2610.22825#S3.E15)\)\. ∎

\(a\)Safety\-aware ordering![Refer to caption](https://arxiv.org/html/2610.22825v1/wacm_clean_truck_following.png)

\(b\)Avoiding an overrated selection![Refer to caption](https://arxiv.org/html/2610.22825v1/wacm_clean_sunny_boulevard.png)

\(c\)Progress–safety tradeoff![Refer to caption](https://arxiv.org/html/2610.22825v1/wacm_clean_rainy_road.png)

Figure 5:Candidate evaluation and planning decisions\. Thin trajectories show candidates and thick trajectories their selections\. Below each scene, candidates are sorted by descending predicted PDMS within each method\. Curves show predictions, dots evaluator scores, and stars the evaluator score of the selected candidate\. Scores use the 0–100 scale\.#### Connecting critic learning to selection\.

For squared loss and square\-integrable evaluator targets, conditional expectation gives

𝔼​\(q^​\(z,a\)−q\)2=ϵfit\+𝔼μ​Var⁡\(q∣z,a\)\.\\mathbb\{E\}\(\\widehat\{q\}\(z,a\)\-q\)^\{2\}=\\epsilon\_\{\\mathrm\{fit\}\}\+\\mathbb\{E\}\_\{\\mu\}\\operatorname\{Var\}\(q\\mid z,a\)\.\(29\)Thusϵfit\\epsilon\_\{\\mathrm\{fit\}\}equals the excess population loss over the optimal measurable predictor\. The evaluator discrepancyϵext\\epsilon\_\{\\mathrm\{ext\}\}separately accounts for alignment between the supervision and the intended task score\.

#### Coverage and the proposed action set\.

The coverage condition applies to the joint test latent\-state–action distribution\. It transfers critic accuracy on supervised pairs to the candidates encountered during selection\. The theorem bounds regret relative to the best proposed action\. Comparing with the optimum over all feasible actions additionally includes the proposal gap, separating the role of the policy in generating candidates from the role of the critic in selecting among them\.

## Appendix BAdditional Qualitative Results

Figure[5](https://arxiv.org/html/2610.22825#A1.F5)pairs planning trajectories with candidate rankings in three urban driving scenes\.

#### Safety\-aware ordering\.

In Fig\.[5](https://arxiv.org/html/2610.22825#A1.F5)\(a\), WAC selects a shorter trajectory behind the truck that passes TTC while retaining the baseline’s evaluator progress reward\. The baseline’s preferred candidates mix TTC failures with safer alternatives, whereas WAC ranks high\-quality candidates first\.

#### Avoiding an overrated selection\.

In Fig\.[5](https://arxiv.org/html/2610.22825#A1.F5)\(b\), the baseline’s highest\-scored trajectory fails the simulated collision check, while WAC’s selection passes both collision and TTC checks\. The selected paths have similar lateral endpoints but different outcomes\.

#### Progress–safety tradeoff\.

In Fig\.[5](https://arxiv.org/html/2610.22825#A1.F5)\(c\), WAC achieves TTC compliance with a shorter forward trajectory and a small reduction in evaluator progress relative to the baseline\. The baseline favors greater progress despite TTC failures, while WAC’s leading candidates combine substantial progress with better temporal safety\.

## Appendix CCandidate Scoring Evaluation Details

### C\.1Evaluation Protocol

We compare the complete planners on the NAVSIM\-v1 test split, evaluating each on its own full candidate pool and excluding recorded trajectories\. Predictions are the aggregate PDMS scores used for selection, without test\-set calibration\. The official simulator and scorer evaluate each candidate together with the PDM reference, preserving the official progress normalization independently of other proposals\.

### C\.2Ranking and Overestimation Metrics

#### Candidate ordering\.

Spearman correlation measures agreement between predicted and evaluator rankings, using average ranks for ties and excluding scenes where either ranking is constant\. Pairwise ranking accuracy measures whether candidates with distinct evaluator scores are ordered correctly, with half credit for tied predictions\. Each metric is computed within a scene and averaged equally across valid scenes\. These metrics measure ordering rather than absolute calibration\.

#### Low\-score overestimation\.

For candidates with evaluator PDMS below 50 on the 0–100 scale, we measure the positive prediction errormax⁡\(p−y,0\)\\max\(p\-y,0\), whereppandyyare the predicted and evaluator scores\. We average within each scene and then equally across scenes where both methods produce such candidates\.

### C\.3Uncertainty Estimation

Uncertainty is assessed by paired bootstrap resampling of driving logs, keeping scenes from the same log together to account for their dependence\.

## Appendix DImplementation and Training Details

Table[5](https://arxiv.org/html/2610.22825#A4.T5)summarizes the architecture and training hyperparameters\.

Factual future targets are computed with the shared visual and scene encoders under stop\-gradient\. The BCE configuration weights the six critic outputs equally\. For the focal\-weighted variant, letyc∈\[0,1\]y\_\{c\}\\in\[0,1\]denote an evaluator target,pcp\_\{c\}its predicted probability, andpt,c=yc​pc\+\(1−yc\)​\(1−pc\)p\_\{t,c\}=y\_\{c\}p\_\{c\}\+\(1\-y\_\{c\}\)\(1\-p\_\{c\}\)\. For NC, DAC, TTC, and comfort, the component loss is

ℒfocal,c=\(1−pt,c\)γ​BCE⁡\(pc,yc\),γ=2\.\\mathcal\{L\}\_\{\\mathrm\{focal\},c\}=\(1\-p\_\{t,c\}\)^\{\\gamma\}\\operatorname\{BCE\}\(p\_\{c\},y\_\{c\}\),\\qquad\\gamma=2\.\(30\)EP and aggregate PDMS retain ordinary BCE\.

Table 5:Key architecture and training hyperparameters\.
## Appendix EAcknowledgements

This work is supported by Didi Voyager Research Labs, and also supported by NSF China with 92582205, NSF China with 52602738\.