Latent Energy Action Planning with World Models

# Latent Energy Action Planning with World Models
Source: [https://arxiv.org/html/2609.03294](https://arxiv.org/html/2609.03294)
## Latent Energy Action Planning with World ModelsThanks:The authors are with the Department of Computer Science, Purdue University, USA,\{phupham,aniketbera\}@purdue\.edu

###### Abstract

Latent world models support efficient model predictive control from high\-dimensional observations, yet optimizing a single learned latent objective can favor action sequences whose decoder\-predicted terminal descriptor does not match the goal descriptor\. We introduce Latent Energy Action Planning \(LEAP\), which treats the complete action horizon as a differentiable variable and optimizes it through a frozen LeWorldModel \(LeWM\)\. LEAP couples terminal latent goal matching with a terminal\-window state energy\. Low energy requires the predicted terminal latent to agree with the goal latent and the decoder\-predicted terminal descriptor to agree with the goal descriptor\. A frozen goal\-conditioned proposal initializes the search, a quasi\-Newton solver refines actions through the autoregressive rollout, and post\-optimization projection enforces the admissible action range\. Across four control domains using the officially released LeWM checkpoints, the complete LEAP planning system raises mean success from 77\.5% for LeWM planned with the cross\-entropy method \(LeWM\+CEM\) to 94\.8% under a matched protocol, a 17\.3\-percentage\-point improvement, while retaining the frozen LeWM representation and predictor\.

###### Index Terms:

Deep Learning Methods, Motion and Path Planning, Learning from Demonstration, Model Predictive Control, Robot Learning

## IIntroduction

Latent world models compress visual observations into predictive states, enabling receding\-horizon control without reconstructing pixels\. PlaNet\[[1](https://arxiv.org/html/2609.03294#bib.bib4)\]and Dreamer\[[2](https://arxiv.org/html/2609.03294#bib.bib5)\]established that compact latent dynamics can support online planning and imagined policy learning\. More recent systems scale these ideas across diverse domains\. DreamerV3\[[3](https://arxiv.org/html/2609.03294#bib.bib6)\]learns behavior through imagined experience with one configuration, while TD\-MPC2\[[4](https://arxiv.org/html/2609.03294#bib.bib8)\]performs local trajectory optimization in a learned decoder\-free latent model\. Joint\-embedding predictive architectures replace pixel prediction with feature prediction\[[5](https://arxiv.org/html/2609.03294#bib.bib1)\]\. I\-JEPA\[[6](https://arxiv.org/html/2609.03294#bib.bib2)\]and V\-JEPA\[[7](https://arxiv.org/html/2609.03294#bib.bib3)\]instantiate this principle for images and video\. DINO\-WM\[[8](https://arxiv.org/html/2609.03294#bib.bib9)\]uses pretrained patch features for offline world modeling and test\-time goal\-conditioned action optimization\. LeWorldModel \(LeWM\)\[[9](https://arxiv.org/html/2609.03294#bib.bib21)\]applies joint\-embedding prediction to efficient visual control with a small Vision Transformer and autoregressive latent predictor\.

Efficiency does not guarantee that the learned planning objective remains reliable under optimization\. A planner actively searches for action sequences on which model error is favorable\. This search can lower predicted goal cost outside the action and rollout distributions represented by offline data\. MOReL\[[10](https://arxiv.org/html/2609.03294#bib.bib13)\]and MOPO\[[11](https://arxiv.org/html/2609.03294#bib.bib14)\]motivate pessimistic or uncertainty\-penalized model\-based offline RL, while BEAR\[[12](https://arxiv.org/html/2609.03294#bib.bib12)\]identifies instability caused by backing up actions outside the data distribution\. Our concern is narrower and occurs inside the planner itself\. This*optimization\-loop loophole*is distinct from ordinary prediction error because average predictive accuracy does not establish accuracy on the trajectories selected by optimization\.

Figure[1](https://arxiv.org/html/2609.03294#S1.F1)shows a representative Push\-T latent\-objective mismatch during cross\-entropy method \(CEM\)\[[13](https://arxiv.org/html/2609.03294#bib.bib16)\]refinement\. Latent\-only CEM lowers the learned latent cost from the expert reference of 4\.75 to 4\.40\. We define magnitude scale as the candidate maximum absolute action divided by the expert\-reference maximum absolute action\. Over the same refinement process, theℓ2\\ell\_\{2\}distance to the expert controls reaches 11\.19 and the magnitude scale reaches2\.32×2\.32\\times\. The opposing trends illustrate a failure mode of latent\-only refinement\. The improved latent objective coincides with increasing divergence from the demonstrated controls\. This example motivates LEAP’s optimization\-aware planner design\.

Fig\. 1:A representative Push\-T latent\-objective mismatch\. Left: latent\-only CEM lowers latent cost below the expert reference\. Right: the same refinement increases physical action divergence and raises the maximum action magnitude to2\.32×2\.32\\timesthe expert reference\.We address terminal ambiguity at planning time without changing LeWM’s representation or predictor\. Latent Energy Action Planning \(LEAP\) treats the complete action horizon as a differentiable variable and backpropagates through the frozen autoregressive rollout\. LEAP evaluates the same rollout in LeWM’s latent representation and through a decoder\-predicted terminal descriptor\. The terminal latent goal cost preserves LeWM’s native visual objective\. The terminal\-window state cost requires this low latent cost to be corroborated over the final two predictions\. A frozen goal\-conditioned proposal centers refinement around a data\-fitted action prediction, and post\-optimization projection prevents inadmissible controls from being selected\. These design choices separate predicted terminal\-descriptor disambiguation from action feasibility\. The loss equation, decoder architecture, normalization rule, terminal window, and energy weights remain shared across domains\.

We evaluate action\-gradient planning on Push\-T, OGBench\-Cube, Reacher, and TwoRooms\. All comparisons retain the official LeWM encoder and autoregressive predictor while varying the planner\. Paired\-start ablations isolate the two energy terms\. Under the same protocol, LEAP reaches 94\.8% mean success, compared with 77\.5% for native CEM\.

Our contributions are:

- •We introduce LEAP, a differentiable planner that optimizes complete action horizons through a frozen latent world model using quasi\-Newton refinement and post\-optimization action projection\.
- •We formulate a terminal energy that couples latent\-goal matching with decoder\-predicted terminal\-state matching, requiring agreement across two complementary representations without retraining the world model\.
- •We conduct an extensive controlled evaluation across four domains and show that LEAP outperforms native LeWM\+CEM by 17\.3 percentage points while adding only 0\.08 s of planning time per trial\.

## IIRelated Work

#### Latent world models\.

Latent dynamics reduce the cost of prediction and planning from images\. PlaNet\[[1](https://arxiv.org/html/2609.03294#bib.bib4)\]plans online in a learned latent model, Dreamer\[[2](https://arxiv.org/html/2609.03294#bib.bib5)\]differentiates through imagined trajectories, and DreamerV3\[[3](https://arxiv.org/html/2609.03294#bib.bib6)\]extends imagination\-based learning across more than 150 tasks with one configuration\. TD\-MPC\[[14](https://arxiv.org/html/2609.03294#bib.bib7)\]learns a task\-oriented latent model for model predictive control\. TD\-MPC2\[[4](https://arxiv.org/html/2609.03294#bib.bib8)\]subsequently scales local latent trajectory optimization across a broader task suite\. LeWM\[[9](https://arxiv.org/html/2609.03294#bib.bib21)\]uses joint\-embedding prediction and CEM to obtain efficient visual control\. LEAP retains LeWM’s learned representation and predictor while redesigning test\-time action selection\.

#### Joint\-embedding prediction\.

Joint\-embedding predictive architectures \(JEPAs\)\[[5](https://arxiv.org/html/2609.03294#bib.bib1)\]predict target features rather than reconstructing pixels\. I\-JEPA\[[6](https://arxiv.org/html/2609.03294#bib.bib2)\]applies masked feature prediction to images, and V\-JEPA\[[7](https://arxiv.org/html/2609.03294#bib.bib3)\]extends the principle to video\. DINO\-WM\[[8](https://arxiv.org/html/2609.03294#bib.bib9)\]predicts pretrained DINOv2 patch features from offline trajectories and optimizes action sequences toward visual goals at test time\. LeWM\[[9](https://arxiv.org/html/2609.03294#bib.bib21)\]instead learns a compact joint\-embedding predictor end to end\. In both cases the feature space is also the planner’s objective\. LEAP preserves this latent objective while adding decoder\-predicted terminal\-descriptor matching during action optimization\.

#### Sampling and gradient planning\.

CEM\[[13](https://arxiv.org/html/2609.03294#bib.bib16)\]is widely used because it does not require differentiable dynamics\. Sampling\-based planning with learned models appears in PETS\[[15](https://arxiv.org/html/2609.03294#bib.bib10)\]and visual foresight\[[16](https://arxiv.org/html/2609.03294#bib.bib15)\]\. Differentiable MPC\[[17](https://arxiv.org/html/2609.03294#bib.bib11)\]exposes structured controllers to end\-to\-end gradients through implicit differentiation\. Broad search can, however, amplify model error by selecting candidates precisely where the learned objective extrapolates\. LEAP instead differentiates a frozen rollout energy directly with respect to the open\-loop action tensor and projects the optimized plan into the admissible action range\.

#### Offline support and model exploitation\.

Offline control methods explicitly address decisions that leave the data distribution\. MOPO\[[11](https://arxiv.org/html/2609.03294#bib.bib14)\]penalizes uncertain model rollouts, whereas MOReL\[[10](https://arxiv.org/html/2609.03294#bib.bib13)\]constructs a pessimistic model for policy optimization\. In model\-free offline RL, BEAR\[[12](https://arxiv.org/html/2609.03294#bib.bib12)\]constrains action selection to reduce bootstrapping error from out\-of\-distribution actions\. LEAP is complementary\. It does not fit a policy or value function and does not modify the frozen world model\. Instead, it regularizes and projects the action sequence selected by test\-time planning\.

#### Energy\-based control\.

Energy\-based policies and trajectory objectives provide soft mechanisms for combining complementary goal signals\. Implicit Behavioral Cloning\[[18](https://arxiv.org/html/2609.03294#bib.bib18)\]models multimodal expert actions with an energy, Composable Energy Policies\[[19](https://arxiv.org/html/2609.03294#bib.bib19)\]combine policy factors, and Diffusion Policy\[[20](https://arxiv.org/html/2609.03294#bib.bib20)\]performs iterative action refinement through a learned score\. LEAP composes latent\-goal and terminal\-descriptor energies over a multi\-step action tensor and differentiates both through a frozen predictive world model\.

## IIIProblem Formulation

Letxtx\_\{t\}denote the current observation,gga goal observation, andygy\_\{g\}its numerical terminal descriptor\. The horizon\-HHaction sequence is𝐚=\(at,…,at\+H−1\)∈ℝH×da\\mathbf\{a\}=\(a\_\{t\},\\ldots,a\_\{t\+H\-1\}\)\\in\\mathbb\{R\}^\{H\\times d\_\{a\}\}, wheredad\_\{a\}is the action\-block dimension\. A latent world model encodesxtx\_\{t\}intozt=ΦLeWM​\(xt\)z\_\{t\}=\\Phi\_\{\\mathrm\{LeWM\}\}\(x\_\{t\}\)and predicts future latents

z^t\+k=Fθ\(z^t\+k−1,e\(at\+k−1\)\),k=1,…,H,\\hat\{z\}\_\{t\+k\}=F\_\{\\theta\}\\\!\\left\(\\hat\{z\}\_\{t\+k\-1\},e\(a\_\{t\+k\-1\}\)\\right\),\\quad k=1,\\dots,H,\(1\)wherez^t=zt\\hat\{z\}\_\{t\}=z\_\{t\},eeis the action\-block embedding,FθF\_\{\\theta\}is the frozen autoregressive predictor, andΦLeWM\\Phi\_\{\\mathrm\{LeWM\}\}is the frozen observation encoder\. Classical latent planning minimizes a learned task cost

𝐚⋆=argmin𝐚Clatent\(z^t\+1:t\+H,g\)\.\\mathbf\{a\}^\{\\star\}=\\arg\\min\_\{\\mathbf\{a\}\}C\_\{\\mathrm\{latent\}\}\(\\hat\{z\}\_\{t\+1:t\+H\},g\)\.\(2\)For compactness, subsequent costs are written as functions of𝐚\\mathbf\{a\}through the predicted rollout𝐳^\(𝐚\)=z^t\+1:t\+H\\hat\{\\mathbf\{z\}\}\(\\mathbf\{a\}\)=\\hat\{z\}\_\{t\+1:t\+H\}\.

### III\-AMotivating Failure of Latent\-Only Optimization

Prediction error describes disagreement on sampled inputs\. Latent\-only planning creates a distinct risk because the optimizer repeatedly concentrates evaluation on action sequences for which learned error is favorable\. Optimization can therefore lower latent cost while moving away from the actions and rollouts represented during training\. Figure[1](https://arxiv.org/html/2609.03294#S1.F1)shows this mismatch in a representative Push\-T trace\. The support analysis below formalizes how this failure can arise\. It does not assert that every optimized plan leaves the training distribution\.

Let𝒜data\\mathcal\{A\}\_\{\\mathrm\{data\}\}denote the action\-sequence support represented in the offline data, and let𝒵data\\mathcal\{Z\}\_\{\\mathrm\{data\}\}denote the corresponding latent\-rollout support\. Minimizing only terminal latent error imposes neither𝐚∈𝒜data\\mathbf\{a\}\\in\\mathcal\{A\}\_\{\\mathrm\{data\}\}nor𝐳^​\(𝐚\)∈𝒵data\\hat\{\\mathbf\{z\}\}\(\\mathbf\{a\}\)\\in\\mathcal\{Z\}\_\{\\mathrm\{data\}\}\. The planner can therefore select a sequence for which

Clatent​\(𝐚⋆\)\\displaystyle C\_\{\\mathrm\{latent\}\}\(\\mathbf\{a\}^\{\\star\}\)<Clatent​\(𝐚data\),\\displaystyle<C\_\{\\mathrm\{latent\}\}\(\\mathbf\{a\}\_\{\\mathrm\{data\}\}\),\(3\)𝐚⋆\\displaystyle\\mathbf\{a\}^\{\\star\}∉𝒜dataor𝐳^\(𝐚⋆\)∉𝒵data\.\\displaystyle\\notin\\mathcal\{A\}\_\{\\mathrm\{data\}\}\\quad\\text\{or\}\\quad\\hat\{\\mathbf\{z\}\}\(\\mathbf\{a\}^\{\\star\}\)\\notin\\mathcal\{Z\}\_\{\\mathrm\{data\}\}\.\(4\)Here,𝐚data∈𝒜data\\mathbf\{a\}\_\{\\mathrm\{data\}\}\\in\\mathcal\{A\}\_\{\\mathrm\{data\}\}is a reference sequence\. This condition does not require a particular optimizer\. It arises whenever optimization concentrates evaluation on model errors that are rare under the data distribution\. Low predicted goal error alone is therefore insufficient evidence that an action plan reaches the intended physical configuration\. This motivates complementing the visual latent objective with decoder\-predicted terminal\-descriptor matching and enforcing admissible controls through post\-optimization action projection\.

### III\-BProjected Energy\-Based Planning Objective

LEAP augments the terminal latent objective with terminal\-state matching\. GivenKKproposal\-seeded initializations\{𝐚0\(k\)\}k=1K\\\{\\mathbf\{a\}\_\{0\}^\{\(k\)\}\\\}\_\{k=1\}^\{K\}, it refines and then projects each candidate before selection:

𝐚~\(k\)=LBFGS⁡\(Etotal,𝐚0\(k\)\),𝐚\(k\)=Π𝒜\(𝐚~\(k\)\),k⋆=argminkEtotal\(𝐚\(k\)\)\.\\begin\{split\}\\tilde\{\\mathbf\{a\}\}^\{\(k\)\}&=\\operatorname\{LBFGS\}\\\!\\left\(E\_\{\\mathrm\{total\}\},\\mathbf\{a\}\_\{0\}^\{\(k\)\}\\right\),\\\\ \\mathbf\{a\}^\{\(k\)\}&=\\Pi\_\{\\mathcal\{A\}\}\\\!\\left\(\\tilde\{\\mathbf\{a\}\}^\{\(k\)\}\\right\),\\qquad k^\{\\star\}=\\arg\\min\_\{k\}E\_\{\\mathrm\{total\}\}\\\!\\left\(\\mathbf\{a\}^\{\(k\)\}\\right\)\.\\end\{split\}\(5\)Here,LBFGS\\operatorname\{LBFGS\}denotes the L\-BFGS solve,𝒜\\mathcal\{A\}is the normalized environment action domain, andΠ𝒜\\Pi\_\{\\mathcal\{A\}\}denotes Euclidean projection onto that domain\. The selected plan is𝐚⋆=𝐚\(k⋆\)\\mathbf\{a\}^\{\\star\}=\\mathbf\{a\}^\{\(k^\{\\star\}\)\}\. LetDψD\_\{\\psi\}be a frozen decoder from LeWM latent space to a normalized state descriptor\. LEAP uses

Etotal​\(𝐚\)=Clatent​\(𝐚\)\+λs​Eterminal​\(𝐚\),E\_\{\\mathrm\{total\}\}\(\\mathbf\{a\}\)=C\_\{\\mathrm\{latent\}\}\(\\mathbf\{a\}\)\+\\lambda\_\{s\}E\_\{\\mathrm\{terminal\}\}\(\\mathbf\{a\}\),\(6\)whereλs\>0\\lambda\_\{s\}\>0is the terminal\-state weight\. This energy combines the native visual goal with decoder\-predicted terminal\-descriptor matching\.

Both costs are nonnegative\. Thus, for anyϵ≥0\\epsilon\\geq 0,Etotal​\(𝐚\)≤ϵE\_\{\\mathrm\{total\}\}\(\\mathbf\{a\}\)\\leq\\epsilonimpliesClatent​\(𝐚\)≤ϵC\_\{\\mathrm\{latent\}\}\(\\mathbf\{a\}\)\\leq\\epsilonandEterminal​\(𝐚\)≤ϵ/λsE\_\{\\mathrm\{terminal\}\}\(\\mathbf\{a\}\)\\leq\\epsilon/\\lambda\_\{s\}\. Because neither term is negative, a low total cannot mask a large disagreement in either representation\.

The two terms address predicted terminal\-descriptor ambiguity through complementary goal representations\. The latent goal cost asks whether the terminal prediction resembles the goal in LeWM’s representation\. The terminal\-state cost asks whether the decoder\-predicted terminal descriptor matches the goal descriptor under a normalized metric\. A low latent score must therefore be corroborated by the decoder prediction instead of serving as the planner’s sole evidence of goal attainment\. Terminal\-state matching does not itself impose membership in𝒵data\\mathcal\{Z\}\_\{\\mathrm\{data\}\}\. Neither energy term constrains action magnitude\. Post\-optimization projection separately enforces admissible controls at the environment interface\. The resulting objective combines terminal disambiguation with action feasibility without treating either as a certificate of offline\-data support\.

## IVMethodology

### IV\-ADifferentiable Native\-LeWM Rollout

LEAP changes action selection while leaving LeWorldModel \(LeWM\) unchanged\. The native visual encoder maps the current image and goal image to latent vectors,

zt=ΦLeWM​\(xt\),zg=ΦLeWM​\(g\),z\_\{t\}=\\Phi\_\{\\mathrm\{LeWM\}\}\(x\_\{t\}\),\\qquad z\_\{g\}=\\Phi\_\{\\mathrm\{LeWM\}\}\(g\),\(7\)and the frozen autoregressive predictor rolls forward under a candidate action sequence,

z^t\+k=Fθ\(z^t\+k−1,e\(at\+k−1\)\),k=1,…,H\.\\hat\{z\}\_\{t\+k\}=F\_\{\\theta\}\\\!\\left\(\\hat\{z\}\_\{t\+k\-1\},e\(a\_\{t\+k\-1\}\)\\right\),\\qquad k=1,\\ldots,H\.\(8\)The encoder and predictor parameters remain frozen\. Unlike native CEM, LEAP retains the computation graph from the predicted latents to the candidate actions\. This graph supplies gradients for updating the entire horizon instead of fitting a sampling distribution to ranked candidates\.

Native LeWM\+CEM treats the frozen rollout model as a black\-box evaluator and updates a sampling distribution from ranked action sequences\. LEAP instead preserves the rollout computation graph and directly refines the complete action horizon using gradients of the combined terminal energy\. Because both planners retain the same LeWM representation and dynamics predictor, their distinction lies in how candidate actions are initialized, refined, and constrained\. LEAP therefore converts the pretrained world model from a sampling\-based scorer into a differentiable action\-planning model\.

Figure[2](https://arxiv.org/html/2609.03294#S4.F2)summarizes the planner\. Both energy terms act on the same frozen rollout, and their sum supplies a single action gradient\.

![Refer to caption](https://arxiv.org/html/2609.03294v1/figures/leap_overview.png)Fig\. 2:Overview of LEAP\. Frozen LeWM encodes the current observation and unrolls each ofKKcandidate action sequences through its autoregressive latent predictor\. Candidates are initialized around a goal\-conditioned proposal and optimized over the complete horizon, with each action block conditioning its corresponding rollout step\. The terminal latent\-goal term compares the predicted and goal latents, while a frozen decoder provides normalized descriptors for terminal\-state matching\. Their sum forms the total energy, whose gradient propagates through the rollout to refine the candidate actions\.
### IV\-BEnergy\-Based Latent Action Planning

For candidate sequence𝐚∈ℝH×da\\mathbf\{a\}\\in\\mathbb\{R\}^\{H\\times d\_\{a\}\}, the final LEAP design minimizes

Etotal​\(𝐚\)=Clatent​\(𝐚\)\+λs​Eterminal​\(𝐚\),λs=10\.E\_\{\\mathrm\{total\}\}\(\\mathbf\{a\}\)=C\_\{\\mathrm\{latent\}\}\(\\mathbf\{a\}\)\+\\lambda\_\{s\}E\_\{\\mathrm\{terminal\}\}\(\\mathbf\{a\}\),\\qquad\\lambda\_\{s\}=10\.\(9\)The terminal latent goal cost is LeWM’s native planning objective,

Clatent​\(𝐚\)=‖z^t\+H​\(𝐚\)−zg‖22\.C\_\{\\mathrm\{latent\}\}\(\\mathbf\{a\}\)=\\left\\\|\\hat\{z\}\_\{t\+H\}\(\\mathbf\{a\}\)\-z\_\{g\}\\right\\\|\_\{2\}^\{2\}\.\(10\)It supplies semantic goal direction in the representation on which LeWM was trained\.

To predict a complementary terminal descriptor, a frozen decoderDψD\_\{\\psi\}maps each predicted latent to a standardized state descriptor,

y^t\+k=Dψ​\(z^t\+k\)\.\\hat\{y\}\_\{t\+k\}=D\_\{\\psi\}\\\!\\left\(\\hat\{z\}\_\{t\+k\}\\right\)\.\(11\)The decoder is fitted from an offline training subset using frozen LeWM embeddings and benchmark state descriptors\. Success labels are not used\. Each decoder is trained for 50 epochs on 9,000 examples, with 1,000 held\-out examples, using the same two\-hidden\-layer MLP with widths 256–256\. Each coordinate is standardized with training\-set statistics\. The architecture, fitting procedure, coordinate\-wise normalization, and loss definition are identical across domains\. Only the fitted parameters and dimensionality follow the corresponding frozen checkpoint and descriptor schema\.

Letygy\_\{g\}be the standardized goal descriptor and letL=2L=2denote the terminal window\. The shared terminal\-window state energy is

Eterminal​\(𝐚\)=1L​D​∑k=H−L\+1H‖Dψ​\(z^t\+k​\(𝐚\)\)−yg‖22,L=2,E\_\{\\mathrm\{terminal\}\}\(\\mathbf\{a\}\)=\\frac\{1\}\{LD\}\\sum\_\{k=H\-L\+1\}^\{H\}\\left\\\|D\_\{\\psi\}\\\!\\left\(\\hat\{z\}\_\{t\+k\}\(\\mathbf\{a\}\)\\right\)\-y\_\{g\}\\right\\\|\_\{2\}^\{2\},\\qquad L=2,\(12\)whereDDis the descriptor dimension\. The combined energy turns goal evaluation into cross\-representation terminal agreement\. A candidate that enters a low latent\-cost region but predicts a terminal descriptor inconsistent withygy\_\{g\}remains penalized\. Scoring both final predictions further discourages plans that only cross the goal at a single terminal instant\. Equation \([9](https://arxiv.org/html/2609.03294#S4.E9)\), includingL=2L=2andλs=10\\lambda\_\{s\}=10, is unchanged across Push\-T, OGBench\-Cube, Reacher, and TwoRooms\. The algebraic objective contains no domain\-specific branches, coordinate masks, or success thresholds\.

Fig\.[3](https://arxiv.org/html/2609.03294#S4.F3)provides a geometric analysis of a local 2\-D Push\-T action slice around an expert action\. The asymmetric level sets, secondary basins, and curved descent paths expose the nonconvex geometry faced by action optimization\. In the left panel, a misleading latent basin has low cost but is inconsistent with the decoder\-predicted terminal descriptor\. In the right panel, terminal\-state matching raises that basin and redirects descent toward cross\-representation agreement\. The second term therefore changes the preferred action directions rather than merely rescaling the latent objective\.

Fig\. 3:2\-D Push\-T action\-slice energy surface around an expert action\. Left:ClatentC\_\{\\mathrm\{latent\}\}contains a misleading off\-center basin whose decoder\-predicted terminal descriptor is inconsistent with the goal \(cyan cross\)\. Right: the total LEAP energy raises this basin and redirects descent toward joint latent and terminal\-state agreement \(yellow star\)\. Both panels use the same normalized color scale\. Irregular local geometry shows that terminal\-state matching reshapes rather than simply rescales the action landscape\.
### IV\-CInitialization, Optimization, and Projection

LEAP initializes candidates with an auxiliary goal\-conditioned action proposal trained on offline trajectories and frozen for evaluation\. The proposal is a three\-hidden\-layer MLP with SiLU activations\. It maps benchmark\-provided planning context to a normalized five\-block action sequence\. Training minimizes mean\-squared error to the corresponding expert sequence using a 95/5 training/validation split, and we retain the minimum\-validation\-loss checkpoint\.

The proposal supplies only the search center and has zero weight in Eq\. \([9](https://arxiv.org/html/2609.03294#S4.E9)\)\. Its deterministic output defines the first candidate\. WhenK\>1K\>1, seeded perturbations around this center provide additional local starts\. This separation keeps the optimized energy compact while avoiding a cold start in the nonconvex action landscape\.

For each candidate, LEAP computes

∇𝐚Etotal=∂Etotal∂𝐳^​∂𝐳^∂𝐚\.\\nabla\_\{\\mathbf\{a\}\}E\_\{\\mathrm\{total\}\}=\\frac\{\\partial E\_\{\\mathrm\{total\}\}\}\{\\partial\\hat\{\\mathbf\{z\}\}\}\\frac\{\\partial\\hat\{\\mathbf\{z\}\}\}\{\\partial\\mathbf\{a\}\}\.\(13\)The final planner uses L\-BFGS\[[21](https://arxiv.org/html/2609.03294#bib.bib17)\]with learning rate 1\.0, ten internal iterations, history size 100, default gradient and parameter\-change tolerances of10−710^\{\-7\}and10−910^\{\-9\}, and strong\-Wolfe line search\. After the internal solve, post\-optimization action projection clamps every candidate to the normalized environment action range\. Energy is recomputed on the projected candidates, and selection uses the recomputed values\. This enforces admissible controls without changing the internal L\-BFGS trajectory\.

ForQQobjective\-and\-gradient evaluations,KKcandidate sequences, and rollout horizonHH, planning requires𝒪⁡\(Q​K​H\)\\mathcal\{O\}\(QKH\)predictor steps at fixed model width\. L\-BFGS line search can invoke more closures than its nominal iteration budget, so we measure synchronized planner time directly rather than treating iteration count as compute\. Differentiation retains candidate\-specific rollout activations\. Memory therefore grows withKK,HH, and model width\.

After selecting the lowest\-energy candidate independently for each environment, LEAP executes five action blocks \(25 primitive environment steps\) before replanning\.

Algorithm 1LEAP action\-gradient optimizationInput:Observation

xtx\_\{t\}, goal

\(g,yg\)\(g,y\_\{g\}\), horizon

HH, starts

KK, L\-BFGS budget

MM, weight

λs\\lambda\_\{s\}
Output:Configured receding\-horizon prefix

1Encode

zt←ΦLeWM​\(xt\)z\_\{t\}\\leftarrow\\Phi\_\{\\mathrm\{LeWM\}\}\(x\_\{t\}\)and

zg←ΦLeWM​\(g\)z\_\{g\}\\leftarrow\\Phi\_\{\\mathrm\{LeWM\}\}\(g\)with native LeWM

2Initialize

KKcandidates from the frozen action proposal and seeded perturbations

3Roll out candidates and compute Eqs\. \([10](https://arxiv.org/html/2609.03294#S4.E10)\) and \([12](https://arxiv.org/html/2609.03294#S4.E12)\)

4

𝐚~\(k\)←L−BFGSM⁡\(Etotal,𝐚0\(k\)\)\\tilde\{\\mathbf\{a\}\}^\{\(k\)\}\\leftarrow\\operatorname\{L\-BFGS\}\_\{M\}\\\!\\left\(E\_\{\\mathrm\{total\}\},\\mathbf\{a\}\_\{0\}^\{\(k\)\}\\right\),

k=1,…,Kk=1,\\ldots,K
5

𝐚\(k\)←Π𝒜​\(𝐚~\(k\)\)\\mathbf\{a\}^\{\(k\)\}\\leftarrow\\Pi\_\{\\mathcal\{A\}\}\\\!\\left\(\\tilde\{\\mathbf\{a\}\}^\{\(k\)\}\\right\)
6

sk←Etotal​\(𝐚\(k\)\)s\_\{k\}\\leftarrow E\_\{\\mathrm\{total\}\}\\\!\\left\(\\mathbf\{a\}^\{\(k\)\}\\right\)
7Reject the plan if

\{k:sk<∞\}=∅\\\{k:s\_\{k\}<\\infty\\\}=\\varnothing
8

k⋆←argmink:sk<∞skk^\{\\star\}\\leftarrow\\arg\\min\_\{k:s\_\{k\}<\\infty\}s\_\{k\}
9return

𝐚t:t\+4\(k⋆\)\\mathbf\{a\}\_\{t:t\+4\}^\{\(k^\{\\star\}\)\}

## VExperimental Evaluation

Fig\. 4:Task success rates \(%\)\. LEAP and LeWM\+CEM use our matched controlled 100\-trial protocol\. Black error bars show one binomial standard error for controlled evaluations and reported variation for contextual baselines\.### V\-ADomains and Evaluation Protocol

We evaluate Push\-T, OGBench\-Cube, Reacher, and TwoRooms using the officially released LeWM checkpoints\. The visual encoder and autoregressive predictor remain frozen\. Current and goal observations are sampled 25 environment steps apart within the same offline trajectory\. Each planner uses a five\-transition latent horizon, executes five action blocks \(25 primitive environment steps\) before replanning, and operates under a 50\-step interaction budget\. Candidate counts are 32 for Push\-T, one for OGBench\-Cube, and four for Reacher and TwoRooms\. Actions are restricted to\[−2,2\]\[\-2,2\], except in TwoRooms, which uses\[−1\.6,1\.6\]\[\-1\.6,1\.6\]\.

We select the shared terminal windowL=2L=2, terminal\-state weightλs=10\\lambda\_\{s\}=10, and optimizer configuration using the paired 50\-trial sweeps\. These settings are fixed across domains before the final 100\-trial evaluation, which uses a different start schedule\. No setting is adjusted per task\. We also report the corresponding 50\-trial final\-system results to assess evaluation stability\. The LEAP–CEM comparison uses matched sampled starts, task configurations, interaction budgets, official evaluation code, and checkpoints\.

Success is binary\. Both planners are timed one environment at a time on the same task\-assigned GPU\. Planning time includes candidate initialization and action optimization but excludes model loading, environment stepping, and visualization\. Budgeted steps per trial average all trials, using the first successful primitive step for successful trials and the full interaction budget for failures\.

### V\-BOverall Task Performance

Figure[4](https://arxiv.org/html/2609.03294#S5.F4)compares LEAP with our controlled recomputation of native LeWM\+CEM\. We reproduce the PLDM, DINO\-WM, and Random rates reported by LeWM\[[9](https://arxiv.org/html/2609.03294#bib.bib21)\]for context because we did not rerun those methods\. We omit baselines that LeWM did not evaluate on all four tasks, giving every displayed method the same domain coverage\.

LEAP reaches 90\.0% on Push\-T, 100\.0% on OGBench\-Cube, 89\.0% on Reacher, and 100\.0% on TwoRooms, yielding a 94\.8% mean\. Under the same protocol, LeWM\+CEM reaches 88\.0%, 62\.0%, 75\.0%, and 85\.0%, respectively, for a 77\.5% mean\. LEAP changes the planner and adds the terminal\-state readout while retaining the frozen LeWM representation and predictor\. This controlled comparison evaluates complete planning systems and does not isolate the effects of all planner components individually\.

Table[I](https://arxiv.org/html/2609.03294#S5.T1)reports both protocols\. The 50\-trial evaluation yields a 96\.5% mean for LEAP and 77\.0% for LeWM\+CEM\. LEAP’s unweighted mean changes by 1\.8 percentage points at 100 trials, and LeWM\+CEM changes by 0\.5 point\. No task\-level rate changes by more than four points, leaving every ranking unchanged\.

TABLE I:Results under the 50\- and 100\-trial protocols\. Entries are success rate±\\pmone binomial standard error \(%\)\. Mean is unweighted across tasks\. Bold identifies LEAP\.Table[II](https://arxiv.org/html/2609.03294#S5.T2)reports matched planning time and environment interaction\. Steps per trial average all trials and assign failures the full interaction budget\.

TABLE II:Reported success is in percent\. Planning time and environment steps are per trial\. Bold identifies LEAP\.LEAP improves mean success by 17\.3 percentage points and uses 5\.35 fewer budgeted steps per trial\. Differentiable L\-BFGS refinement adds 0\.08 s of planning time per trial \(1\.28 versus 1\.20 s\)\. This modest increase accompanies higher success and fewer interactions\.

### V\-CComplementarity of the Two Energies

We compare the complete objective with both single\-term variants while holding the checkpoint, proposal, optimizer, candidate count, projection, and paired starts fixed\. The latent\-goal\-only variant differentiates LeWM’s native objective through the same planner, while the terminal\-state\-only variant removes the visual latent term\.

TABLE III:LEAP energy ablation\. Paired 50\-trial protocol\. Task entries and Mean are success rates \(%\)\. Mean is unweighted\. Superscripts show percentage\-point changes from Full LEAP\. Bold marks the best mean\.The decomposition reveals an asymmetric complementarity\. Latent goal matching is already strong, reaching 90\.0% on Push\-T, 94\.0% on OGBench\-Cube, 86\.0% on Reacher, and 94\.0% on TwoRooms for a 91\.0% mean\. Terminal\-state matching alone averages 52\.0%\. It is a complementary signal, not a replacement for visual goal identity\. Within this paired ablation matrix, their combination is best on every domain and raises the task mean to 96\.5%\. With optimization and initialization unchanged, the comparison isolates the objective: semantic latent matching supplies broad goal direction, while terminal\-state matching resolves residual decoded error and prevents latent similarity from acting as the sole selection criterion\.

TABLE IV:First\-decision cross\-representation agreement\. Ratios compare LEAP with latent\-goal\-only error\. Paired lower denotes improved starts\. Mean is unweighted across tasks\.Table[IV](https://arxiv.org/html/2609.03294#S5.T4)compares the first selected plans on paired starts\. LEAP lowers mean decoder\-predicted terminal\-state error in every domain, delivering a 6\.7% task\-average reduction over latent\-goal\-only planning\. It selects a lower\-error plan on 89% of starts\. These consistent improvements show that terminal\-state matching reliably corrects plans that latent\-only optimization ranks favorably despite decoded terminal mismatch\. With optimizer and initialization fixed, the comparison isolates this contribution and demonstrates stronger cross\-representation goal agreement\.

The selected\-plan analysis complements task success by showing across domains that the additional energy systematically changes action selection rather than benefiting only a small subset of trials\. Together with the component ablation, this result identifies cross\-representation terminal agreement as the source of the improvement over latent\-goal\-only planning\.

### V\-DTerminal Regularization Ablations

We test whether suppressing decoded motion near the horizon improves latent\-goal optimization\. Fory^t\+k=Dψ​\(z^t\+k\)\\hat\{y\}\_\{t\+k\}=D\_\{\\psi\}\(\\hat\{z\}\_\{t\+k\}\), define

Edamp​\(𝐚\)=1\(L−1\)​D​∑k=H−L\+2H‖y^t\+k−y^t\+k−1‖22,E\_\{\\mathrm\{damp\}\}\(\\mathbf\{a\}\)=\\frac\{1\}\{\(L\-1\)D\}\\sum\_\{k=H\-L\+2\}^\{H\}\\left\\\|\\hat\{y\}\_\{t\+k\}\-\\hat\{y\}\_\{t\+k\-1\}\\right\\\|\_\{2\}^\{2\},\(14\)which penalizes terminal\-window descriptor motion\. This ablation\-only term has coefficientλv\\lambda\_\{v\}\.

TABLE V:Damping\-energy weight ablation withλs=0\\lambda\_\{s\}=0\. Paired 50\-trial protocol\. Task entries and Mean are success rates \(%\)\. Mean is unweighted\. Superscripts show percentage\-point changes from latent goal only\. Bold marks the best mean\.Table[V](https://arxiv.org/html/2609.03294#S5.T5)reports the complete four\-domain damping\-weight sweep with terminal\-state matching disabled\. The settingλv=3\\lambda\_\{v\}=3is the only tested weight that improves every task over latent goal only and raises the mean from 91\.0% to the best damping\-only mean of 94\.5%\. Smaller and larger weights regress Reacher, showing that the penalty requires appropriate scaling\. We useλv=3\\lambda\_\{v\}=3in the composition study of Table[VII](https://arxiv.org/html/2609.03294#S5.T7)\.

TABLE VI:Terminal\-state weight ablation withλv=0\\lambda\_\{v\}=0\. Paired 50\-trial protocol\. Task entries and Mean are success rates \(%\)\. Mean is unweighted\. Superscripts show percentage\-point changes from latent goal only\. Bold marks the best mean\.Table[VI](https://arxiv.org/html/2609.03294#S5.T6)disables damping energy and variesλs∈\{0,0\.1,0\.3,1,3,10\}\\lambda\_\{s\}\\in\\\{0,0\.1,0\.3,1,3,10\\\}\. Intermediate weights of 1 and 3 reduce mean performance because Reacher drops to 68%, whereasλs=10\\lambda\_\{s\}=10reaches 96\.5% and improves every task over latent goal only\. Reacher is particularly sensitive because success requires precise terminal end\-effector placement: at intermediate weights, comparably influential latent and terminal\-state gradients can steer refinement toward different local basins, whereasλs=10\\lambda\_\{s\}=10makes terminal grounding decisive enough to select actions consistent with the decoded goal geometry\. This non\-monotonic optimization geometry motivates the shared final setting\.

TABLE VII:Terminal energy composition\. Paired 50\-trial protocol\. Task entries and Mean are success rates \(%\)\. Mean is unweighted\. Superscripts show percentage\-point changes from latent goal only\. Bold marks the best mean\.Finally, Table[VII](https://arxiv.org/html/2609.03294#S5.T7)tests whether the two best auxiliary settings compose\. Damping energy alone improves the mean to 94\.5%, but addingλv=3\\lambda\_\{v\}=3to the terminal\-state term atλs=10\\lambda\_\{s\}=10reduces OGBench\-Cube from 100% to 96%, Reacher from 92% to 86%, and the mean from 96\.5% to 94\.0%\. The measured regression shows that the two auxiliary terms are not additive under this optimizer\. Terminal\-state matching penalizes both terminal predictions against the goal, while damping penalizes their separation regardless of goal error\. The latter introduces a stationary\-state preference that competes with goal matching when the decoder\-predicted terminal descriptor is inaccurate\. We therefore retain latent goal plus terminal\-state matching and exclude damping energy from final LEAP\.

### V\-EOptimizer Efficiency

Table[VIII](https://arxiv.org/html/2609.03294#S5.T8)compares optimizer choices under the final LEAP energy\. Task columns and Mean are success rates\. Mean superscripts give percentage\-point changes from the L\-BFGS/10 reference\. The L\-BFGS/10 row reports the final planner configuration\. The remaining rows report the optimizer sweep\.

TABLE VIII:Optimizer ablation\. Paired 50\-trial protocol\. Task entries and Mean are success rates \(%\)\. Mean is unweighted\. Superscripts show percentage\-point changes from L\-BFGS/10\. Bold marks the best mean\.Within the paired optimizer study, L\-BFGS/10 gives the highest mean success at 96\.5%\. The other tested configurations range from 91\.0% to 94\.0%, so L\-BFGS/10 remains the final design\.

![Refer to caption](https://arxiv.org/html/2609.03294v1/rollout_state_trajectories.png)Fig\. 5:Paired qualitative rollout comparison across four domains\. Red crosses denote states before goal attainment, green check marks denote states at or after goal attainment, and the final column shows the goal\.
### V\-FQualitative Rollout Analysis

Fig\.[5](https://arxiv.org/html/2609.03294#S5.F5)analyzes paired rollouts for all four domains\. The upper row for each task is LeWM\+CEM and the lower row is LEAP under the same displayed target\. Here,TTdenotes the primitive environment step\. LEAP reaches the goal byT=23T=23on OGBench\-Cube and byT=28T=28on Reacher, while LeWM\+CEM stays away from the target throughout the displayed sequence\. Push\-T and TwoRooms instead show paired starts where both planners succeed and LEAP finishes sooner\. LEAP requires 23 versus 25 steps on Push\-T and 9 versus 24 steps on TwoRooms\. For display times after a rollout terminates, the terminal frame is repeated\. The status marks make the first successful displayed state explicit\. Aggregate success rates across the evaluation schedule appear in Fig\.[4](https://arxiv.org/html/2609.03294#S5.F4)\.

## VIDiscussion and Conclusion

We present LEAP, a differentiable latent\-energy planner for action refinement through a frozen world model\. Its energy combines latent\-goal agreement with decoder\-predicted terminal\-state matching, while post\-optimization projection enforces admissible controls\. This formulation defines a differentiable energy over action sequences evaluated through pretrained world\-model rollouts without retraining the representation or dynamics model\.

Across four manipulation and navigation domains, LEAP raises mean success from 77\.5% with native CEM to 94\.8% while adding 0\.08 s of planning time per trial\. Because the official LeWM encoder and predictor remain frozen, the improvement reflects the planning\-system design rather than changes to the learned representation or dynamics model\. This complete\-system comparison does not isolate every planner component\.

Limitations and Future Directions\.LEAP inherits the frozen world model’s limitations on long\-horizon and out\-of\-distribution rollouts\. LEAP also uses a decoder trained to predict state descriptors from frozen LeWM embeddings and currently assumes a numerical goal descriptorygy\_\{g\}, which may not be available in uninstrumented domains\. A promising extension is to replaceygy\_\{g\}with a geometry\-aware representationG⁡\(g\)G\(g\)inferred directly from goal images, such as object\-centric keypoints or dense visual correspondences learned from unlabeled video\. Future work could evaluate this visually grounded formulation under occlusion and domain shift, alongside broader physical and long\-horizon evaluation\.

## References

- \[1\]D\. Hafner, T\. Lillicrap, I\. Fischer, R\. Villegas, D\. Ha, H\. Lee, and J\. Davidson\(2019\)Learning latent dynamics for planning from pixels\.InProceedings of the 36th International Conference on Machine Learning,K\. Chaudhuri and R\. Salakhutdinov \(Eds\.\),Proceedings of Machine Learning Research, Vol\.97,pp\. 2555–2565\.Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p1.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px1.p1.1)\.
- \[2\]D\. Hafner, T\. Lillicrap, J\. Ba, and M\. Norouzi\(2020\)Dream to control: learning behaviors by latent imagination\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=S1lOTC4tDS)Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p1.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px1.p1.1)\.
- \[3\]D\. Hafner, J\. Pasukonis, J\. Ba, and T\. Lillicrap\(2025\)Mastering diverse control tasks through world models\.Nature640,pp\. 647–653\.External Links:[Document](https://dx.doi.org/10.1038/s41586-025-08744-2)Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p1.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px1.p1.1)\.
- \[4\]N\. Hansen, H\. Su, and X\. Wang\(2024\)TD\-MPC2: scalable, robust world models for continuous control\.InThe Twelfth International Conference on Learning Representations,Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p1.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px1.p1.1)\.
- \[5\]Y\. LeCun\(2022\)A path towards autonomous machine intelligence\.OpenReview\.Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p1.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px2.p1.1)\.
- \[6\]M\. Assran, Q\. Duval, I\. Misra, P\. Bojanowski, P\. Vincent, M\. Rabbat, Y\. LeCun, and N\. Ballas\(2023\)Self\-supervised learning from images with a joint\-embedding predictive architecture\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 15619–15629\.Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p1.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px2.p1.1)\.
- \[7\]A\. Bardes, Q\. Garrido, J\. Ponce, X\. Chen, M\. Rabbat, Y\. LeCun, M\. Assran, and N\. Ballas\(2024\)Revisiting feature prediction for learning visual representations from video\.Transactions on Machine Learning Research\.Note:Featured CertificationExternal Links:ISSN 2835\-8856Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p1.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px2.p1.1)\.
- \[8\]G\. Zhou, H\. Pan, Y\. LeCun, and L\. Pinto\(2025\)DINO\-WM: world models on pre\-trained visual features enable zero\-shot planning\.InForty\-second International Conference on Machine Learning,Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p1.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px2.p1.1)\.
- \[9\]L\. Maes, Q\. Le Lidec, D\. Scieur, Y\. LeCun, and R\. Balestriero\(2026\)LeWorldModel: stable end\-to\-end joint\-embedding predictive architecture from pixels\.arXiv preprint arXiv:2603\.19312\.Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p1.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px1.p1.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px2.p1.1),[§V\-B](https://arxiv.org/html/2609.03294#S5.SS2.p1.1)\.
- \[10\]R\. Kidambi, A\. Rajeswaran, P\. Netrapalli, and T\. Joachims\(2020\)MOReL: model\-based offline reinforcement learning\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 21810–21823\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/file/f7efa4f864ae9b88d43527f4b14f750f-Paper.pdf)Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p2.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px4.p1.1)\.
- \[11\]T\. Yu, G\. Thomas, L\. Yu, S\. Ermon, J\. Zou, S\. Levine, C\. Finn, and T\. Ma\(2020\)MOPO: model\-based offline policy optimization\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 14129–14142\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/file/a322852ce0df73e204b7e67cbbef0d0a-Paper.pdf)Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p2.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px4.p1.1)\.
- \[12\]A\. Kumar, J\. Fu, G\. Tucker, and S\. Levine\(2019\)Stabilizing off\-policy q\-learning via bootstrapping error reduction\.InProceedings of the 33rd International Conference on Neural Information Processing Systems,Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p2.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px4.p1.1)\.
- \[13\]R\. Rubinstein\(1999\)The cross\-entropy method for combinatorial and continuous optimization\.Methodology and Computing in Applied Probability1\(2\),pp\. 127–190\.Cited by:[§I](https://arxiv.org/html/2609.03294#S1.p3.1),[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px3.p1.1)\.
- \[14\]N\. A\. Hansen, H\. Su, and X\. Wang\(2022\)Temporal difference learning for model predictive control\.InProceedings of the 39th International Conference on Machine Learning,K\. Chaudhuri, S\. Jegelka, L\. Song, C\. Szepesvari, G\. Niu, and S\. Sabato \(Eds\.\),Proceedings of Machine Learning Research, Vol\.162,pp\. 8387–8406\.Cited by:[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px1.p1.1)\.
- \[15\]K\. Chua, R\. Calandra, R\. McAllister, and S\. Levine\(2018\)Deep reinforcement learning in a handful of trials using probabilistic dynamics models\.InProceedings of the 32nd International Conference on Neural Information Processing Systems,NIPS’18,Red Hook, NY, USA,pp\. 4759–4770\.Cited by:[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px3.p1.1)\.
- \[16\]F\. Ebert, C\. Finn, S\. Dasari, A\. Xie, A\. Lee, and S\. Levine\(2018\)Visual foresight: model\-based deep reinforcement learning for vision\-based robotic control\.arXiv preprint arXiv:1812\.00568\.Cited by:[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px3.p1.1)\.
- \[17\]B\. Amos, I\. D\. J\. Rodriguez, J\. Sacks, B\. Boots, and J\. Z\. Kolter\(2018\)Differentiable mpc for end\-to\-end planning and control\.InProceedings of the 32nd International Conference on Neural Information Processing Systems,NIPS’18,Red Hook, NY, USA,pp\. 8299–8310\.Cited by:[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px3.p1.1)\.
- \[18\]P\. Florence, C\. Lynch, A\. Zeng, O\. A\. Ramirez, A\. Wahid, L\. Downs, A\. Wong, J\. Lee, I\. Mordatch, and J\. Tompson\(2021\)Implicit behavioral cloning\.In5th Annual Conference on Robot Learning,Cited by:[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px5.p1.1)\.
- \[19\]J\. Urain, A\. Li, P\. Liu, C\. D’Eramo, and J\. Peters\(2023\)Composable energy policies for reactive motion generation and reinforcement learning\.Int\. J\. Rob\. Res\.42\(10\),pp\. 827–858\.External Links:ISSN 0278\-3649,[Link](https://doi.org/10.1177/02783649231179499),[Document](https://dx.doi.org/10.1177/02783649231179499)Cited by:[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px5.p1.1)\.
- \[20\]C\. Chi, Z\. Xu, S\. Feng, E\. Cousineau, Y\. Du, B\. Burchfiel, R\. Tedrake, and S\. Song\(2025\)Diffusion policy: visuomotor policy learning via action diffusion\.Int\. J\. Rob\. Res\.44\(10\-11\),pp\. 1684–1704\.External Links:ISSN 0278\-3649,[Link](https://doi.org/10.1177/02783649241273668),[Document](https://dx.doi.org/10.1177/02783649241273668)Cited by:[§II](https://arxiv.org/html/2609.03294#S2.SS0.SSS0.Px5.p1.1)\.
- \[21\]D\. C\. Liu and J\. Nocedal\(1989\)On the limited memory BFGS method for large scale optimization\.Mathematical Programming45,pp\. 503–528\.Cited by:[§IV\-C](https://arxiv.org/html/2609.03294#S4.SS3.p3.2)\.