面向离线视觉控制的有向时序表示

# Directed Temporal Representations for Offline Visual Control
Source: [https://arxiv.org/html/2610.08960](https://arxiv.org/html/2610.08960)
Chenyang YuanHaoyu WangUniversity of SheffieldShanghai Jiao Tong Universitycyuan7@sheffield\.ac\.ukcreate\_arc@sjtu\.edu\.cnZhuo SunXiaoyuan ChengShanghai University of Finance and EconomicsUniversity College Londonzhuosunreid@outlook\.comucesxc4@ucl\.ac\.uk

###### Abstract

Predictive world models provide compact visual representations for control\. Control requires a latent geometry aligned with temporal reachability rather than predictive similarity alone\. We introduce Directed Temporal Representations for Control \(DTRC\), which learns such a geometry from offline visual trajectories on top of frozen LeWorldModel \(LeWM\) features\. DTRC constructs a directed temporal quasimetric over the learned control representation\. Short\-range temporal offsets calibrate the distance scale\. Bootstrapped targets extend temporal reachability across longer horizons\. Action\-conditioned consistency aligns the representation with local transition dynamics\. The resulting distance estimates temporal reaching cost, and its change across a transition defines goal\-relative temporal progress\. We use this progress signal as a temporal critic for direct goal\-conditioned policy learning\. Model\-assisted targets provide an additional training\-time refinement under behavior\-support and dynamics\-agreement constraints\. Across ten visual control tasks, DTRC achieves strong goal\-conditioned control performance relative to planning and direct\-policy baselines\. Held\-out diagnostics on the four LeWM tasks show consistent short\-range temporal calibration, task\-dependent long\-range and directional structure, and positive transition\-level progress\. Temporal supervision improves the same flow\-policy parameterization across all four LeWM tasks, while the resulting policy acts directly without iterative trajectory search at test time\. Our project website provides code and visualizations:[https://chenyang\-yuan\.github\.io/dtrc\-project/](https://chenyang-yuan.github.io/dtrc-project/)\.

## 1Introduction

Latent world models compress visual observations into predictive representations and learn action\-conditioned dynamics in latent space, enabling control through imagined rollouts\([Hafner et al\., 2019](https://arxiv.org/html/2610.08960#bib.bib26);[Hafner et al\., 2020](https://arxiv.org/html/2610.08960#bib.bib4);[Hansen et al\., 2022](https://arxiv.org/html/2610.08960#bib.bib13);[Hansen et al\., 2024](https://arxiv.org/html/2610.08960#bib.bib5)\)\. Visual goal\-reaching planners evaluate candidate actions using distances between predicted latent states and goal representations\([Zhou et al\., 2025](https://arxiv.org/html/2610.08960#bib.bib27);[Maes et al\., 2026](https://arxiv.org/html/2610.08960#bib.bib3)\)\. Latent geometry therefore directly influences the control objective\. Recent work reshapes latent spaces to align Euclidean distance with feasible transition structure and improve planning\([Wang et al\., 2026](https://arxiv.org/html/2610.08960#bib.bib25)\)\. Temporal reachability provides a complementary control structure based on ordered state–goal relations\. These relations are associated with interaction cost and may be asymmetric across opposite transition directions\([Wang et al\., 2023](https://arxiv.org/html/2610.08960#bib.bib8)\)\. These properties motivate a control representation aligned with temporal reachability rather than predictive similarity alone\.

Offline trajectories provide direct supervision for this temporal structure\. Future states serve as achieved goals for learning from offline experience\([Andrychowicz et al\., 2017](https://arxiv.org/html/2610.08960#bib.bib16);[Ghosh et al\., 2021](https://arxiv.org/html/2610.08960#bib.bib17)\)\. Temporal relations between states have also been used to construct goal\-conditioned representations and value structure\([Eysenbach et al\., 2022](https://arxiv.org/html/2610.08960#bib.bib28);[Park et al\., 2023](https://arxiv.org/html/2610.08960#bib.bib20);[Park et al\., 2024](https://arxiv.org/html/2610.08960#bib.bib1);[Wang et al\., 2023](https://arxiv.org/html/2610.08960#bib.bib8)\)\. For control, temporal reachability requires local scale, directional structure, and long\-horizon consistency\. Short\-range temporal offsets establish the local distance scale\. Ordered state–goal relations preserve directional information\. Multistep relations extend temporal reachability across longer horizons\([Zheng et al\., 2026](https://arxiv.org/html/2610.08960#bib.bib9)\)\. Changes in temporal cost across individual transitions provide a local measure of progress\. A control representation should preserve these properties in a common temporal geometry\.

We introduce Directed Temporal Representations for Control \(DTRC\) to learn this geometry from offline visual trajectories\. Starting from frozen LeWorldModel \(LeWM\) features\([Maes et al\., 2026](https://arxiv.org/html/2610.08960#bib.bib3)\), DTRC learns a control representation equipped with a directed temporal quasimetric\. A symmetric metric component defines the base geometry\. An order\-sensitive residual introduces directional variation in state–goal reaching cost\([Liu et al\., 2023](https://arxiv.org/html/2610.08960#bib.bib21)\)\. Short\-range temporal offsets calibrate the distance scale\. Bootstrapped targets extend temporal reachability across longer horizons\. Action\-conditioned consistency aligns the representation with local transition dynamics\. The resulting distance estimates temporal reaching cost\. Its reduction across a transition defines goal\-relative temporal progress\.

The directed temporal distance provides a critic for policy learning\. Advantage\-weighted methods use value estimates to weight recorded actions during policy learning\([Peng et al\., 2019](https://arxiv.org/html/2610.08960#bib.bib15);[Kostrikov et al\., 2022](https://arxiv.org/html/2610.08960#bib.bib19)\)\. DTRC uses goal\-relative temporal progress for the same purpose\. Recorded actions receive larger weights for transitions with greater temporal progress\. Policy\-generated actions receive the same temporal evaluation through their predicted successor states\. Additional model\-assisted targets are restricted by behavior support and dynamics agreement\([Yu et al\., 2020](https://arxiv.org/html/2610.08960#bib.bib6);[Yu et al\., 2021](https://arxiv.org/html/2610.08960#bib.bib7);[Sikchi et al\., 2022](https://arxiv.org/html/2610.08960#bib.bib14)\)\. Only supported candidates with lower predicted temporal cost than the recorded continuation contribute to the additional policy objective\. Model\-assisted targets therefore provide an additional training\-time refinement\. At test time, the goal\-conditioned policy acts directly without iterative trajectory search\.

We evaluate DTRC on ten visual goal\-conditioned control tasks spanning navigation, reaching, and manipulation\. We first evaluate end\-to\-end goal\-conditioned control performance\. We then study temporal structure beyond the directly supervised local range, including task\-dependent directionality and transition\-level progress\. Controlled comparisons isolate the effect of temporal supervision on the same flow\-policy parameterization\. Finally, qualitative visualizations examine the organization of the learned control representation across tasks\. Our main contributions are threefold:

- •We introduce DTRC, a framework for learning a directed temporal control representation on frozen predictive visual features\. The representation uses a directed quasimetric to represent ordered state–goal relations through temporal reaching cost\.
- •We develop an offline learning objective for temporal representation learning\. Short\-range trajectory offsets establish the distance scale, bootstrapped targets propagate temporal reachability across longer horizons, and action\-conditioned consistency aligns the representation with local transition dynamics\. Distance reduction then defines goal\-relative temporal progress for policy supervision\.
- •Across ten visual goal\-conditioned control tasks, DTRC achieves strong performance relative to predictive\-model planning and goal\-conditioned policy baselines\. Held\-out diagnostics on the four LeWM tasks show consistent local temporal calibration, task\-dependent long\-range and directional structure, and positive transition\-level progress\. Temporal supervision further improves the same flow\-policy parameterization across four LeWM tasks\.

## 2Preliminaries

#### Offline goal\-conditioned control\.

We consider finite\-horizon goal\-conditioned control from visual observations\. At timett, the agent observesot∈𝒪o\_\{t\}\\in\\mathcal\{O\}, aims to reach a goal observationog∈𝒪o\_\{g\}\\in\\mathcal\{O\}within a remaining horizonht∈\{1,…,H\}h\_\{t\}\\in\\\{1,\\ldots,H\\\}, and selects an actionat∈𝒜a\_\{t\}\\in\\mathcal\{A\}\. The learning setting provides a fixed offline dataset𝒟\\mathcal\{D\}of unlabeled raw\-pixel trajectories, where each trajectory is given byτ=\(o0,a0,o1,a1,…,oT\)\\tau=\(o\_\{0\},a\_\{0\},o\_\{1\},a\_\{1\},\\ldots,o\_\{T\}\)\. The goal\-conditioned control problem is to learn a policy

π⁡\(at∣ot,og,ht\)\\pi\(a\_\{t\}\\mid o\_\{t\},o\_\{g\},h\_\{t\}\)\(1\)that maps the current observation, goal observation, and remaining horizon to actions that progressively steer the system toward the goal\. At test time, control is closed\-loop: after executingata\_\{t\}, the agent receives the next observation and recomputes its action conditioned on the updated state and the same goal\.

#### Frozen predictive visual features\.

We encode the current and goal observations using a pretrained LeWM encoderEE\([Maes et al\., 2026](https://arxiv.org/html/2610.08960#bib.bib3)\),

zt=E⁡\(ot\),zg=E⁡\(og\),z\_\{t\}=E\(o\_\{t\}\),\\qquad z\_\{g\}=E\(o\_\{g\}\),\(2\)wherezt,zg∈ℝdzz\_\{t\},z\_\{g\}\\in\\mathbb\{R\}^\{d\_\{z\}\}denote predictive visual features\. LeWM is a joint\-embedding predictive world model that supports action\-conditioned prediction and latent\-space planning without pixel reconstruction\. We keep the pretrained encoder fixed throughout training\. DTRC learns the control representation and temporal geometry on top of these frozen features\.

## 3Directed Temporal Representations for Control

![Refer to caption](https://arxiv.org/html/2610.08960v1/H3_training_planning_V3.png)Figure 1:Overview of directed temporal representation learning and policy training\.\(a\)Offline visual trajectories provide temporal supervision on top of frozen LeWM features\.\(b\)DTRC learns a control representation with a directed temporal quasimetric\. A symmetric metric component defines the base geometry, and an order\-sensitive residual represents directional state–goal cost\.\(c\)Temporal distance reduction measures goal\-relative progress\. The resulting temporal advantage weights recorded continuations for direct policy learning\.We introduce Directed Temporal Representations for Control \(DTRC\), an offline framework for visual goal\-conditioned control\. DTRC learns a directed temporal geometry on top of frozen predictive visual features\. The method contains three stages\. First, we learn a control representation equipped with a directed temporal quasimetric \(Section[3\.1](https://arxiv.org/html/2610.08960#S3.SS1)\)\. Second, offline trajectory supervision calibrates the local temporal scale and propagates temporal reachability across longer horizons\. Action\-conditioned consistency further aligns the representation with local transition dynamics \(Section[3\.2](https://arxiv.org/html/2610.08960#S3.SS2)\)\. Third, temporal distance reduction provides a goal\-relative signal for direct policy learning \(Section[3\.3](https://arxiv.org/html/2610.08960#S3.SS3)\)\. Model\-assisted targets provide an additional training\-time refinement\.

### 3\.1Directed Temporal Representation

Temporal reaching cost has directional and compositional structure\. Opposite transition directions can have different reaching costs\. A path through an intermediate state also constrains the direct state–goal cost\([Wang et al\., 2023](https://arxiv.org/html/2610.08960#bib.bib8);[Liu et al\., 2023](https://arxiv.org/html/2610.08960#bib.bib21)\)\. We therefore represent temporal reachability with a directed quasimetric\.

Given a frozen LeWM featurezz, a representation headψ:ℝdz→ℝde\\psi:\\mathbb\{R\}^\{d\_\{z\}\}\\rightarrow\\mathbb\{R\}^\{d\_\{e\}\}produces the control representation

A symmetric metric headS:ℝde→ℝdϕS:\\mathbb\{R\}^\{d\_\{e\}\}\\rightarrow\\mathbb\{R\}^\{d\_\{\\phi\}\}defines the base geometry\([Park et al\., 2024](https://arxiv.org/html/2610.08960#bib.bib1)\)\. We augment this geometry with an order\-sensitive mapAθ:ℝde→ℝKA\_\{\\theta\}:\\mathbb\{R\}^\{d\_\{e\}\}\\rightarrow\\mathbb\{R\}^\{K\}using the metric–residual construction\([Liu et al\., 2023](https://arxiv.org/html/2610.08960#bib.bib21)\)\. Foret=ψ⁡\(zt\)e\_\{t\}=\\psi\(z\_\{t\}\)andeg=ψ⁡\(zg\)e\_\{g\}=\\psi\(z\_\{g\}\), the directed temporal distance is

dθ​\(et,eg\)=α⁡\[‖S⁡\(et\)−S⁡\(eg\)‖2\+maxj∈\[K\]⁡\[Aθ,j​\(eg\)−Aθ,j​\(et\)\]\+\],d\_\{\\theta\}\(e\_\{t\},e\_\{g\}\)=\\alpha\\left\[\\left\\\|S\(e\_\{t\}\)\-S\(e\_\{g\}\)\\right\\\|\_\{2\}\+\\max\_\{j\\in\[K\]\}\\left\[A\_\{\\theta,j\}\(e\_\{g\}\)\-A\_\{\\theta,j\}\(e\_\{t\}\)\\right\]\_\{\+\}\\right\],\(4\)where\[u\]\+=max⁡\(u,0\)\[u\]\_\{\+\}=\\max\(u,0\),Aθ:ℝde→ℝKA\_\{\\theta\}:\\mathbb\{R\}^\{d\_\{e\}\}\\rightarrow\\mathbb\{R\}^\{K\}defines the directed residual coordinates\. The first term is symmetric in the state–goal pair\. The residual depends on the ordering of the pair\. Reversing the state–goal order changes the residual and can therefore producedθ​\(et,eg\)≠dθ​\(eg,et\)d\_\{\\theta\}\(e\_\{t\},e\_\{g\}\)\\neq d\_\{\\theta\}\(e\_\{g\},e\_\{t\}\)\. The parameterα=exp⁡\(η\)\>0\\alpha=\\exp\(\\eta\)\>0is a learned positive scale\.

The metric–residual construction also satisfies the triangle inequality\. For any intermediate representationeme\_\{m\}, the direct state–goal cost is bounded by the cost of reachingeme\_\{m\}and continuing fromeme\_\{m\}to the goal:

dθ​\(et,eg\)≤dθ​\(et,em\)\+dθ​\(em,eg\)\.d\_\{\\theta\}\(e\_\{t\},e\_\{g\}\)\\leq d\_\{\\theta\}\(e\_\{t\},e\_\{m\}\)\+d\_\{\\theta\}\(e\_\{m\},e\_\{g\}\)\.\(5\)This property gives the distance a compositional structure across state–goal pairs while retaining directional asymmetry\. The following objectives assign temporal semantics to this structured distance\.

### 3\.2Learning Temporal Geometry from Offline Trajectories

The directed quasimetric defines the structural form of the temporal geometry\. Offline trajectories assign temporal semantics to this geometry\. Future states provide achieved goals, and their recorded offsets provide direct temporal supervision\([Andrychowicz et al\., 2017](https://arxiv.org/html/2610.08960#bib.bib16);[Ghosh et al\., 2021](https://arxiv.org/html/2610.08960#bib.bib17)\)\. We first calibrate the local distance scale and then propagate temporal reachability across longer horizons\.

#### Local temporal calibration\.

Short\-range state pairs provide direct temporal anchors\. For a recorded offsetΔ∈\{1,…,J\}\\Delta\\in\\\{1,\\ldots,J\\\}, we supervisedθ​\(et,et\+Δ\)d\_\{\\theta\}\(e\_\{t\},e\_\{t\+\\Delta\}\)using the observed separationΔ\\Delta\. We minimize

ℒlocal=𝔼t,Δ​\[ℓτe2​\(Δ−dθ​\(et,et\+Δ\)\)\],\\mathcal\{L\}\_\{\\mathrm\{local\}\}=\\mathbb\{E\}\_\{t,\\Delta\}\\left\[\\ell\_\{\\tau\_\{e\}\}^\{2\}\\left\(\\Delta\-d\_\{\\theta\}\(e\_\{t\},e\_\{t\+\\Delta\}\)\\right\)\\right\],\(6\)whereJJdenotes the largest directly supervised temporal offset\. This objective establishes the local temporal scale of the learned distance\.

#### Bootstrapped long\-horizon targets\.

For a current representationete\_\{t\}and goal representationege\_\{g\}, letΔt,g\\Delta\_\{t,g\}denote their recorded temporal separation for state–goal pairs from the same trajectory\. Direct temporal supervision is available for short offsets\. Longer\-range and cross\-trajectory goal pairs use a bootstrapped target\. We combine an observednn\-step prefix with the estimated remaining temporal cost:

yt,g=\{Δt,g,0<Δt,g≤n,n\+dθ¯​\(e¯t\+n,e¯g\),otherwise\.y\_\{t,g\}=\\begin\{cases\}\\Delta\_\{t,g\},&0<\\Delta\_\{t,g\}\\leq n,\\\\\[3\.0pt\] n\+d\_\{\\bar\{\\theta\}\}\(\\bar\{e\}\_\{t\+n\},\\bar\{e\}\_\{g\}\),&\\text\{otherwise\}\.\\end\{cases\}\(7\)Heree¯=ψ¯​\(z\)\\bar\{e\}=\\bar\{\\psi\}\(z\)denotes the target representation anddθ¯d\_\{\\bar\{\\theta\}\}denotes the corresponding target temporal distance\. The first branch anchors the geometry to observed temporal separation\. The second branch propagates the same scale to longer state–goal relations by combining an observed prefix with the estimated remaining cost\. The bootstrap horizonnncontrols the length of the observed prefix\.

Multiple trajectories and bootstrapped targets can provide different temporal costs for related state–goal pairs\. We use lower\-expectile regression to emphasize lower costs supported by the temporal targets\([Newey and Powell, 1987](https://arxiv.org/html/2610.08960#bib.bib18);[Kostrikov et al\., 2022](https://arxiv.org/html/2610.08960#bib.bib19)\)\. The goal\-conditioned temporal objective is

ℒgoal=𝔼⁡\[ℓτe2​\(yt,g−dθ​\(et,eg\)\)\],\\mathcal\{L\}\_\{\\mathrm\{goal\}\}=\\mathbb\{E\}\\left\[\\ell\_\{\\tau\_\{e\}\}^\{2\}\\left\(y\_\{t,g\}\-d\_\{\\theta\}\(e\_\{t\},e\_\{g\}\)\\right\)\\right\],\(8\)whereℓτe2​\(u\)=\|τe−𝟏​\(u<0\)\|​u2\\ell\_\{\\tau\_\{e\}\}^\{2\}\(u\)=\\left\|\\tau\_\{e\}\-\\mathbf\{1\}\(u<0\)\\right\|u^\{2\}denotes the expectile loss andu=yt,g−dθ​\(et,eg\)u=y\_\{t,g\}\-d\_\{\\theta\}\(e\_\{t\},e\_\{g\}\)is the temporal prediction residual\. We useτe<12\\tau\_\{e\}<\\frac\{1\}\{2\}\. This choice assigns greater weight to overestimation relative to the temporal target and shifts the learned distance toward lower supported costs\.

#### Action\-conditioned consistency\.

The temporal objectives constrain state–goal relations\. We additionally require the control representation to remain compatible with local action\-conditioned transitions\. A transition headTξT\_\{\\xi\}predicts the next control representation through the residual update

e^t\+1=et\+Tξ​\(et,at\)\.\\hat\{e\}\_\{t\+1\}=e\_\{t\}\+T\_\{\\xi\}\(e\_\{t\},a\_\{t\}\)\.\(9\)We optimize the representation and transition head with

ℒcons=𝔼⁡\[‖e^t\+1−e¯t\+1‖22\],\\mathcal\{L\}\_\{\\mathrm\{cons\}\}=\\mathbb\{E\}\\left\[\\left\\\|\\hat\{e\}\_\{t\+1\}\-\\bar\{e\}\_\{t\+1\}\\right\\\|\_\{2\}^\{2\}\\right\],\(10\)where gradients are stopped through the target representatione¯t\+1\\bar\{e\}\_\{t\+1\}\. This loss aligns the learned control representation with local transition dynamics\. The transition headTξT\_\{\\xi\}is used for representation learning\. It is separate from the latent dynamics ensemble used for model\-assisted policy augmentation\.

#### Overall temporal objective\.

We jointly optimize the control representation, directed temporal distance, and transition\-consistency head with

ℒtemp=ℒgoal\+λlocal​ℒlocal\+λcons​ℒcons\.\\mathcal\{L\}\_\{\\mathrm\{temp\}\}=\\mathcal\{L\}\_\{\\mathrm\{goal\}\}\+\\lambda\_\{\\mathrm\{local\}\}\\mathcal\{L\}\_\{\\mathrm\{local\}\}\+\\lambda\_\{\\mathrm\{cons\}\}\\mathcal\{L\}\_\{\\mathrm\{cons\}\}\.\(11\)The coefficientsλlocal\\lambda\_\{\\mathrm\{local\}\}andλcons\\lambda\_\{\\mathrm\{cons\}\}control local temporal calibration and action\-conditioned consistency, respectively\.

### 3\.3Policy Learning from Temporal Progress

The learned temporal distance provides a goal\-relative signal for policy learning\. We first use recorded continuations to train a direct goal\-conditioned policy\. Model\-assisted alternatives provide additional training targets\.

#### Temporal progress and advantage\.

For a recordedkk\-step continuation fromtttot\+kt\+k, we define goal\-relative temporal progress as the reduction in temporal distance:

Pt,g\(k\)=dθ¯​\(e¯t,e¯g\)−dθ¯​\(e¯t\+k,e¯g\)\.P\_\{t,g\}^\{\(k\)\}=d\_\{\\bar\{\\theta\}\}\(\\bar\{e\}\_\{t\},\\bar\{e\}\_\{g\}\)\-d\_\{\\bar\{\\theta\}\}\(\\bar\{e\}\_\{t\+k\},\\bar\{e\}\_\{g\}\)\.\(12\)Positive values indicate a reduction in the estimated remaining temporal cost\. The recorded continuation incurs a temporal cost ofkktransitions\. We therefore define its temporal advantage as

𝒜t,gdata=Pt,g\(k\)−k=dθ¯​\(e¯t,e¯g\)−\[k\+dθ¯​\(e¯t\+k,e¯g\)\]\.\\mathcal\{A\}\_\{t,g\}^\{\\mathrm\{data\}\}=P\_\{t,g\}^\{\(k\)\}\-k=d\_\{\\bar\{\\theta\}\}\(\\bar\{e\}\_\{t\},\\bar\{e\}\_\{g\}\)\-\\left\[k\+d\_\{\\bar\{\\theta\}\}\(\\bar\{e\}\_\{t\+k\},\\bar\{e\}\_\{g\}\)\\right\]\.\(13\)The subtraction ofkkaccounts for the temporal cost of the recorded action sequence\. A larger advantage indicates greater temporal progress relative to the cost of the recorded continuation\.

#### Advantage\-weighted policy training\.

We parameterize the goal\-conditioned policy with conditional flow matching\([Lipman et al\., 2023](https://arxiv.org/html/2610.08960#bib.bib10);[Park et al\., 2025b](https://arxiv.org/html/2610.08960#bib.bib11)\)\. This parameterization supports multimodal action distributions under a fixed state–goal condition\. For akk\-step action sequence𝐚t=\(at,…,at\+k−1\)\\mathbf\{a\}\_\{t\}=\(a\_\{t\},\\ldots,a\_\{t\+k\-1\}\), the policy condition is

ct=\[zt,zg,et,eg,Emb⁡\(ht/H\)\],c\_\{t\}=\[z\_\{t\},z\_\{g\},e\_\{t\},e\_\{g\},\\operatorname\{Emb\}\(h\_\{t\}/H\)\],\(14\)whereEmb\\operatorname\{Emb\}denotes the horizon embedding\.

The temporal advantage determines the contribution of each recorded continuation:

wt,gdata=min⁡\{wmax,exp⁡\(𝒜t,gdataβ\)\},w\_\{t,g\}^\{\\mathrm\{data\}\}=\\min\\left\\\{w\_\{\\max\},\\exp\\left\(\\frac\{\\mathcal\{A\}\_\{t,g\}^\{\\mathrm\{data\}\}\}\{\\beta\}\\right\)\\right\\\},\(15\)whereβ\\betacontrols the selectivity of advantage weighting andwmaxw\_\{\\max\}limits large weights\. The recorded\-data policy objective is

ℒpolicydata=𝔼⁡\[wt,gdata​ℓFM​\(ω,𝐚t,ct\)\],\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\\mathrm\{data\}\}=\\mathbb\{E\}\\left\[w\_\{t,g\}^\{\\mathrm\{data\}\}\\,\\ell\_\{\\mathrm\{FM\}\}\\left\(\\omega;\\mathbf\{a\}\_\{t\},c\_\{t\}\\right\)\\right\],\(16\)whereℓFM\\ell\_\{\\mathrm\{FM\}\}denotes the standard conditional flow\-matching loss\.

#### Model\-assisted refinement\.

The same temporal distance can also evaluate policy\-generated alternatives through their predicted successor states\. We retain additional targets that satisfy the behavior\-support and dynamics\-agreement criteria and improve the predicted temporal cost relative to the recorded continuation\. The resulting model\-assisted policy objective is denoted byℒpolicyim\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\\mathrm\{im\}\}\. We provide the candidate construction and filtering procedure in Appendix[B](https://arxiv.org/html/2610.08960#A2)\.

The total policy objective is

ℒpolicy=ℒpolicydata\+λim​ℒpolicyim,\\mathcal\{L\}\_\{\\mathrm\{policy\}\}=\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\\mathrm\{data\}\}\+\\lambda\_\{\\mathrm\{im\}\}\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\\mathrm\{im\}\},\(17\)whereλim\\lambda\_\{\\mathrm\{im\}\}controls the contribution of the model\-assisted targets\.

## 4Experiments

### 4\.1Experimental Setup

#### Environments and datasets\.

We evaluate DTRC on ten visual goal\-conditioned control tasks spanning navigation, reaching, and manipulation\. We use the four offline datasets adopted by LeWM\([Maes et al\., 2026](https://arxiv.org/html/2610.08960#bib.bib3)\): PushT, TwoRoom, Reacher, and OGBench\-Cube\. We additionally evaluate on six visual OGBench tasks: PointMaze, Cube\-Double, Scene, AntMaze\-Medium, HumanoidMaze\-Medium, and Puzzle\-3x3\. All methods receive current and goal images encoded by pretrained LeWM encoders\.

#### Baselines\.

We compare against LeWM\([Maes et al\., 2026](https://arxiv.org/html/2610.08960#bib.bib3)\), goal\-conditioned inverse dynamics model \(GC\-IDM\)\([Nguyen et al\., 2026](https://arxiv.org/html/2610.08960#bib.bib2)\), goal\-conditioned behavioral cloning \(GCBC\)\([Ghosh et al\., 2021](https://arxiv.org/html/2610.08960#bib.bib17)\), goal\-conditioned IQL \(GC\-IQL\)\([Kostrikov et al\., 2022](https://arxiv.org/html/2610.08960#bib.bib19);[Park et al\., 2025a](https://arxiv.org/html/2610.08960#bib.bib12)\), and a goal\-conditioned adaptation of Hilbert foundation policies \(HILP\)\([Park et al\., 2024](https://arxiv.org/html/2610.08960#bib.bib1)\)\. Baseline and implementation details are provided in Appendices[A](https://arxiv.org/html/2610.08960#A1)–[D](https://arxiv.org/html/2610.08960#A4)\.

### 4\.2Goal\-Conditioned Control Performance

We first evaluate end\-to\-end goal\-conditioned control performance\. The four LeWM tasks provide a direct comparison with predictive\-model planning and goal\-conditioned policy baselines\. We then evaluate on six additional visual OGBench tasks to assess performance across a broader set of navigation and manipulation environments\.

Figure 2:Goal\-conditioned control on the four LeWM tasks\.Bars report mean success rate over five evaluation seeds; error bars denote sample standard deviation\.Figure[2](https://arxiv.org/html/2610.08960#S4.F2)compares DTRC with predictive\-model planning and direct\-policy baselines on PushT, TwoRoom, Reacher, and OGBench\-Cube\. DTRC achieves the highest or tied\-highest success on PushT, TwoRoom, and Reacher, and the second\-highest success on Cube\. Relative to LeWM, DTRC matches its performance on PushT and improves success by14\.714\.7,24\.024\.0, and30\.030\.0percentage points on TwoRoom, Reacher, and Cube, respectively\. The results show that temporal representation learning can support direct goal\-conditioned control without test\-time trajectory search\.

Table 1:Goal\-conditioned control on six additional visual OGBench tasks\.Results report mean success rate \(%\)±\\pmsample standard deviation over five evaluation seeds\. Best results are shown in bold and second\-best results are underlined\.Table[1](https://arxiv.org/html/2610.08960#S4.T1)extends the comparison to six additional visual OGBench tasks\. DTRC achieves the best or tied\-best result on Cube\-Double and Puzzle\-3x3, and the second\-best result on Scene, AntMaze\-Medium, and HumanoidMaze\-Medium\. Performance is less favorable on PointMaze, where DTRC reaches11\.3%11\.3\\%success compared with28\.7%28\.7\\%for GCBC\. Overall, DTRC ranks among the top two methods on five of six tasks; larger goal offsets are evaluated in Appendix[E\.2](https://arxiv.org/html/2610.08960#A5.SS2)\. We next examine the learned temporal structure and its contribution to policy learning\.

### 4\.3Temporal Structure of the Learned Representation

We first evaluate the learned temporal representation independently of policy performance\. We use the EMA target network at 200,000 updates and sample state–future\-goal pairs from held\-out episodes\. For each task, we draw 200 pairs at every temporal offset from 1 to 100 transitions, yielding 20,000 pairs\. All distances are computed in the full learned representation\. Recorded temporal offset is used as a reference for trajectory duration, rather than as ground\-truth optimal reaching cost\. Figure[3](https://arxiv.org/html/2610.08960#S4.F3)illustrates three properties on PushT: temporal structure across state–goal offsets, directional state–goal cost, and transition\-level temporal progress\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/pusht_diagnostics.png)Figure 3:Temporal structure learned by DTRC on PushT\.We evaluate 20,000 held\-out state–future\-goal pairs, with 200 pairs at each temporal offset from 1 to 100 transitions\.\(a\)Forward temporal distance as a function of recorded temporal offset\. The solid curve and shaded region show the median and interquartile range in 5\-step bins; the dashed line denotes equality with the recorded offset \(ρ=0\.845\\rho=0\.845\)\.\(b\)Forward distanced⁡\(et,eg\)d\(e\_\{t\},e\_\{g\}\)and reverse distanced⁡\(eg,et\)d\(e\_\{g\},e\_\{t\}\)for the same state–goal pairs\. The gray curve shows the symmetric contribution of the same DTRC model\.\(c\)Distribution of one\-step temporal progressPt,g\(1\)=d⁡\(et,eg\)−d⁡\(et\+1,eg\)P\_\{t,g\}^\{\(1\)\}=d\(e\_\{t\},e\_\{g\}\)\-d\(e\_\{t\+1\},e\_\{g\}\)\. Positive progress occurs for81\.41%81\.41\\%of the evaluated pairs\.Figure[3](https://arxiv.org/html/2610.08960#S4.F3)a shows that the learned forward distance retains substantial temporal structure beyond the directly supervised local range, with Spearman correlationρ=0\.845\\rho=0\.845over offsets from 1 to 100 transitions\. The median distance increases with recorded offset while deviating from the equality reference at longer horizons\. This behavior is consistent with the learning objective: short\-range offsets calibrate the local distance scale, while bootstrapped targets represent longer\-horizon state–goal reaching cost rather than recorded trajectory duration\. Figure[3](https://arxiv.org/html/2610.08960#S4.F3)b further shows pronounced directional structure on PushT\. The reverse cost exceeds the forward cost for92\.76%92\.76\\%of the sampled pairs, while the symmetric contribution is unchanged under reversal\. The asymmetry therefore arises from the order\-sensitive component \(see Appendices[F\.3](https://arxiv.org/html/2610.08960#A6.SS3)and[F\.4](https://arxiv.org/html/2610.08960#A6.SS4)\)\. Figure[3](https://arxiv.org/html/2610.08960#S4.F3)c evaluates the one\-step instance of temporal progress from Eq\.[12](https://arxiv.org/html/2610.08960#S3.E12)\. The learned distance decreases after the recorded transition for81\.41%81\.41\\%of the sampled state–goal pairs\. This result shows that the learned geometry resolves local goal\-relative progress over the balanced offset range\. Temporal progress is subsequently converted into the temporal advantage used for policy supervision\.

We next summarize the same diagnostics across the four LeWM tasks\. Table[2](https://arxiv.org/html/2610.08960#S4.T2)separates short\-range temporal calibration from longer\-range, directional, and transition\-level properties of the learned geometry\.

Table 2:Temporal representation diagnostics across the four LeWM tasks\.MAE1:8measures the mean absolute difference between forward temporal distance and recorded offset over temporal offsets 1–8\.ρ1:100\\rho\_\{1:100\}denotes Spearman’s rank correlation between forward temporal distance and recorded offset over offsets 1–100\. Reverse\>\>Forward reports the fraction of state–goal pairs satisfyingd⁡\(eg,et\)\>d⁡\(et,eg\)d\(e\_\{g\},e\_\{t\}\)\>d\(e\_\{t\},e\_\{g\}\)\. Positive progress reports the fraction satisfyingPt,g\(1\)\>0P\_\{t,g\}^\{\(1\)\}\>0\.Short\-range MAE remains below two transitions on all four tasks, while long\-range association and directional structure vary by task \(Table[2](https://arxiv.org/html/2610.08960#S4.T2)\)\. Positive one\-step progress remains above50%50\\%on these tasks and above60%60\\%on all six additional OGBench tasks \(Appendix[F\.2](https://arxiv.org/html/2610.08960#A6.SS2)\)\. Thus, DTRC resolves goal\-relative transition progress across all ten tasks without requiring a common long\-range or directional geometry\.

### 4\.4Effect of Temporal Supervision on Policy Learning

The representation diagnostics above establish temporal structure in the learned geometry\. We next evaluate whether this structure provides useful supervision for policy learning\. We compare unweighted flow behavioral cloning with the same flow\-policy parameterization augmented by the learned DTRC control representation and temporal\-advantage weighting\. Model\-assisted targets are disabled in this comparison\.

Table 3:Effect of temporal supervision on policy learning\.Success rates \(%\) are reported as mean±\\pmsample standard deviation over five evaluation seeds\. Both variants use the same flow\-policy parameterization\. Temporal supervision adds the learned DTRC control representation and temporal\-advantage weighting without model\-assisted targets\. All variants use the same 50 start–goal pairs per task and seed, a goal offset of 25 recorded transitions, and a 50\-step interaction budget\.Temporal supervision improves the same flow\-policy parameterization on all four tasks \(Table[3](https://arxiv.org/html/2610.08960#S4.T3)\)\. Success increases by9\.39\.3percentage points on PushT,5\.35\.3on TwoRoom,32\.032\.0on Reacher, and6\.06\.0on Cube\. The largest change occurs on Reacher, where success increases from68\.0%68\.0\\%to100\.0%100\.0\\%\. Since model\-assisted targets are disabled, this comparison isolates the effect of the learned temporal representation and temporal\-advantage supervision from model\-assisted policy refinement\. With the representation diagnostics, these results connect the learned temporal geometry to policy learning\. Full component ablations, including behavior\-support and directional\-residual variants, are reported in Appendix[E\.1](https://arxiv.org/html/2610.08960#A5.SS1)\.

### 4\.5Qualitative Structure of the Learned Representation

![Refer to caption](https://arxiv.org/html/2610.08960v1/four_spaces.png)Figure 4:Qualitative organization of the learned representation on PushT\.We project frozen LeWM featureszz, learned control representationsee, symmetric coordinatesS⁡\(e\)S\(e\), and directed potentialsA⁡\(e\)A\(e\)onto their first two principal components\. Points are colored by normalized time within the offline trajectory\. The projections are used only for visualization; all temporal distances and quantitative diagnostics are computed in the full learned representation\.Figure[4](https://arxiv.org/html/2610.08960#S4.F4)provides a qualitative view of the transformation from frozen predictive features to the learned temporal representation\. We visualize frozen LeWM features, the learned control representation, the symmetric coordinates, and the directed potentials using their leading principal components\. Temporal ordering is less apparent in the leading principal components of the frozen LeWM features\. After DTRC training, the control representation exhibits a more structured progression with offline episode time\. The symmetric coordinates and directed potentials further exhibit distinct organizations, consistent with their separate roles in the directed quasimetric\. These projections are illustrative; the temporal scale, directional structure, and transition\-level progress are evaluated quantitatively in Section[4\.3](https://arxiv.org/html/2610.08960#S4.SS3)\.

#### Task\-dependent geometry\.

We next visualize recorded trajectories in the learned control representation across tasks\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/tworoom_reacher_pointmaze_environment_control.png)Figure 5:Task\-dependent geometry of the learned control representation\.Recorded trajectories are shown in the environment \(top\) and in the first two principal components of the DTRC control representation \(bottom\) for TwoRoom, Reacher, and PointMaze\. Gray points show the offline representation distribution, and colored curves show recorded trajectories\. Stars denote trajectory goals\. The PCA projections are used only for qualitative visualization\.Figure[5](https://arxiv.org/html/2610.08960#S4.F5)shows distinct geometric organizations across control problems\. TwoRoom and PointMaze exhibit curved trajectory structure under constrained navigation, while Reacher forms an approximately cyclic organization in the PCA projection\. The learned trajectories therefore remain structured within task\-specific geometries rather than collapsing onto a common one\-dimensional temporal coordinate\. This variation is consistent with the task\-dependent long\-range and directional statistics in Table[2](https://arxiv.org/html/2610.08960#S4.T2), and illustrates why recorded trajectory duration need not induce the same global ordering across tasks\. Additional representation\-space visualizations and their evolution during training across all ten tasks are provided in Appendices[F\.1](https://arxiv.org/html/2610.08960#A6.SS1)and[G](https://arxiv.org/html/2610.08960#A7)\.

## 5Conclusion

We introduced Directed Temporal Representations for Control \(DTRC\), which learns a directed temporal geometry on top of frozen predictive visual features\. The learned quasimetric represents goal\-relative temporal reaching cost, while distance reduction provides transition\-level progress for direct policy learning\. Across ten visual goal\-conditioned control tasks, DTRC achieves strong control performance and exhibits task\-dependent long\-range and directional structure together with consistent local progress\. Controlled comparisons further show that temporal supervision improves the same flow\-policy parameterization across the four LeWM tasks\. These results support directed temporal geometry as an effective interface between predictive visual representations and goal\-conditioned control\.

### AI use statement

In this work, we used ChatGPT to improve the readability and clarity of the manuscript and to assist with organizing the structure of the paper\. We did not use generative AI tools for any tasks requiring mandatory disclosure under the ICLR 2027 AI Policy for Authors\. All AI\-assisted edits were reviewed and, where necessary, revised by the authors\. We take responsibility for the final content of this work, including text produced or modified with the aid of generative AI\.

## References

- M\. Andrychowicz, F\. Wolski, A\. Ray, J\. Schneider, R\. Fong, P\. Welinder, B\. McGrew, J\. Tobin, P\. Abbeel, and W\. ZarembaHindsight experience replay\.InAdvances in Neural Information Processing Systems,Vol\.30\.Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p2.1),[§3\.2](https://arxiv.org/html/2610.08960#S3.SS2.p1.1)\.
- Baet al\.\(2016\)J\. L\. Ba, J\. R\. Kiros, and G\. E\. HintonLayer normalization\.arXiv preprint arXiv:1607\.06450\.Cited by:[Appendix D](https://arxiv.org/html/2610.08960#A4.SS0.SSS0.Px1.p1.1)\.
- Eysenbachet al\.\(2022\)B\. Eysenbach, T\. Zhang, S\. Levine, and R\. SalakhutdinovContrastive learning as goal\-conditioned reinforcement learning\.InAdvances in Neural Information Processing Systems,Vol\.35\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/e7663e974c4ee7a2b475a4775201ce1f-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p2.1)\.
- Ghoshet al\.\(2021\)D\. Ghosh, A\. Gupta, A\. Reddy, J\. Fu, C\. Devin, B\. Eysenbach, and S\. LevineLearning to reach goals via iterated supervised learning\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=rALA0Xo6yNJ)Cited by:[§A\.3](https://arxiv.org/html/2610.08960#A1.SS3.p1.1),[§1](https://arxiv.org/html/2610.08960#S1.p2.1),[§3\.2](https://arxiv.org/html/2610.08960#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2610.08960#S4.SS1.SSS0.Px2.p1.1)\.
- Hafneret al\.\(2020\)D\. Hafner, T\. Lillicrap, J\. Ba, and M\. NorouziDream to control: learning behaviors by latent imagination\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p1.1)\.
- Hafneret al\.\(2019\)D\. Hafner, T\. Lillicrap, I\. Fischer, R\. Villegas, D\. Ha, H\. Lee, and J\. DavidsonLearning latent dynamics for planning from pixels\.InProceedings of the 36th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.97,pp\. 2555–2565\.External Links:[Link](https://proceedings.mlr.press/v97/hafner19a.html)Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p1.1)\.
- Hansenet al\.\(2024\)N\. Hansen, H\. Su, and X\. WangTD\-MPC2: scalable, robust world models for continuous control\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p1.1)\.
- Hansenet al\.\(2022\)N\. Hansen, X\. Wang, and H\. SuTemporal difference learning for model predictive control\.InInternational Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p1.1)\.
- Hendrycks and Gimpel \(2016\)D\. Hendrycks and K\. GimpelGaussian error linear units \(gelus\)\.arXiv preprint arXiv:1606\.08415\.Cited by:[Appendix D](https://arxiv.org/html/2610.08960#A4.SS0.SSS0.Px1.p1.1)\.
- Kostrikovet al\.\(2022\)I\. Kostrikov, A\. Nair, and S\. LevineOffline reinforcement learning with implicit q\-learning\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=68n2s9ZJWF8)Cited by:[§A\.4](https://arxiv.org/html/2610.08960#A1.SS4.p1.1),[§1](https://arxiv.org/html/2610.08960#S1.p4.1),[§3\.2](https://arxiv.org/html/2610.08960#S3.SS2.SSS0.Px2.p2.1),[§4\.1](https://arxiv.org/html/2610.08960#S4.SS1.SSS0.Px2.p1.1)\.
- Lipmanet al\.\(2023\)Y\. Lipman, R\. T\. Q\. Chen, H\. Ben\-Hamu, M\. Nickel, and M\. LeFlow matching for generative modeling\.InInternational Conference on Learning Representations,Cited by:[Appendix B](https://arxiv.org/html/2610.08960#A2.SS0.SSS0.Px6.p2.1),[§3\.3](https://arxiv.org/html/2610.08960#S3.SS3.SSS0.Px2.p1.1)\.
- Liuet al\.\(2023\)B\. Liu, Y\. Feng, Q\. Liu, and P\. StoneMetric residual networks for sample efficient goal\-conditioned reinforcement learning\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.37,pp\. 8799–8806\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v37i7.26058)Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p3.1),[§3\.1](https://arxiv.org/html/2610.08960#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2610.08960#S3.SS1.p2.2)\.
- Maeset al\.\(2026\)L\. Maes, Q\. L\. Lidec, D\. Scieur, Y\. LeCun, and R\. BalestrieroLeWorldModel: stable end\-to\-end joint\-embedding predictive architecture from pixels\.External Links:2603\.19312,[Link](https://arxiv.org/abs/2603.19312)Cited by:[§A\.1](https://arxiv.org/html/2610.08960#A1.SS1.p1.1),[§1](https://arxiv.org/html/2610.08960#S1.p1.1),[§1](https://arxiv.org/html/2610.08960#S1.p3.1),[§2](https://arxiv.org/html/2610.08960#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2610.08960#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.08960#S4.SS1.SSS0.Px2.p1.1)\.
- Newey and Powell \(1987\)W\. K\. Newey and J\. L\. PowellAsymmetric least squares estimation and testing\.Econometrica55\(4\),pp\. 819–847\.External Links:[Document](https://dx.doi.org/10.2307/1911031)Cited by:[§3\.2](https://arxiv.org/html/2610.08960#S3.SS2.SSS0.Px2.p2.1)\.
- Nguyenet al\.\(2026\)H\. Nguyen, X\. Xu, and X\. HuangLatent geometry beyond search: amortizing planning in world models\.External Links:2605\.08732,[Link](https://arxiv.org/abs/2605.08732)Cited by:[§A\.2](https://arxiv.org/html/2610.08960#A1.SS2.p1.1),[§4\.1](https://arxiv.org/html/2610.08960#S4.SS1.SSS0.Px2.p1.1)\.
- Parket al\.\(2025a\)S\. Park, K\. Frans, B\. Eysenbach, and S\. LevineOGBench: benchmarking offline goal\-conditioned rl\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.4](https://arxiv.org/html/2610.08960#A1.SS4.p1.1),[§4\.1](https://arxiv.org/html/2610.08960#S4.SS1.SSS0.Px2.p1.1)\.
- Parket al\.\(2023\)S\. Park, D\. Ghosh, B\. Eysenbach, and S\. LevineHIQL: offline goal\-conditioned rl with latent states as actions\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p2.1)\.
- Parket al\.\(2024\)S\. Park, T\. Kreiman, and S\. LevineFoundation policies with Hilbert representations\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 39737–39761\.External Links:[Link](https://proceedings.mlr.press/v235/park24g.html)Cited by:[§A\.5](https://arxiv.org/html/2610.08960#A1.SS5.p1.1),[§1](https://arxiv.org/html/2610.08960#S1.p2.1),[§3\.1](https://arxiv.org/html/2610.08960#S3.SS1.p2.2),[§4\.1](https://arxiv.org/html/2610.08960#S4.SS1.SSS0.Px2.p1.1)\.
- Parket al\.\(2025b\)S\. Park, Q\. Li, and S\. LevineFlow q\-learning\.InInternational Conference on Machine Learning,Cited by:[§3\.3](https://arxiv.org/html/2610.08960#S3.SS3.SSS0.Px2.p1.1)\.
- Penget al\.\(2019\)X\. B\. Peng, A\. Kumar, G\. Zhang, and S\. LevineAdvantage\-weighted regression: simple and scalable off\-policy reinforcement learning\.arXiv preprint arXiv:1910\.00177\.Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p4.1)\.
- Polyak and Juditsky \(1992\)B\. T\. Polyak and A\. B\. JuditskyAcceleration of stochastic approximation by averaging\.SIAM Journal on Control and Optimization30\(4\),pp\. 838–855\.External Links:[Document](https://dx.doi.org/10.1137/0330046)Cited by:[Appendix B](https://arxiv.org/html/2610.08960#A2.SS0.SSS0.Px3.p1.2)\.
- Sikchiet al\.\(2022\)H\. Sikchi, W\. Zhou, and D\. HeldLearning off\-policy with online planning\.InConference on Robot Learning,Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p4.1)\.
- Wanget al\.\(2023\)T\. Wang, A\. Torralba, P\. Isola, and A\. ZhangOptimal goal\-reaching reinforcement learning via quasimetric learning\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 36411–36430\.External Links:[Link](https://proceedings.mlr.press/v202/wang23al.html)Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p1.1),[§1](https://arxiv.org/html/2610.08960#S1.p2.1),[§3\.1](https://arxiv.org/html/2610.08960#S3.SS1.p1.1)\.
- Wanget al\.\(2026\)Y\. Wang, O\. Bounou, G\. Zhou, R\. Balestriero, T\. G\. J\. Rudner, Y\. LeCun, and M\. RenTemporal straightening for latent planning\.InInternational Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p1.1)\.
- Yuet al\.\(2021\)T\. Yu, A\. Kumar, R\. Rafailov, A\. Rajeswaran, S\. Levine, and C\. FinnCOMBO: conservative offline model\-based policy optimization\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p4.1)\.
- Yuet al\.\(2020\)T\. Yu, G\. Thomas, L\. Yu, S\. Ermon, J\. Zou, S\. Levine, C\. Finn, and T\. MaMOPO: model\-based offline policy optimization\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p4.1)\.
- Zhenget al\.\(2026\)B\. Zheng, V\. Myers, B\. Eysenbach, and S\. LevineScaling goal\-conditioned reinforcement learning with multistep quasimetric distances\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p2.1)\.
- Zhouet al\.\(2025\)G\. Zhou, H\. Pan, Y\. LeCun, and L\. PintoDINO\-WM: world models on pre\-trained visual features enable zero\-shot planning\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 79115–79135\.External Links:[Link](https://proceedings.mlr.press/v267/zhou25t.html)Cited by:[§1](https://arxiv.org/html/2610.08960#S1.p1.1)\.

## Appendix ABaselines

All baselines operate on LeWM features extracted from current and goal images\. The visual encoders are kept fixed during downstream training and evaluation\. Below, we describe the control models and the adaptations used in our comparison\.

### A\.1LeWM

We use the pretrained LeWM encoder and action\-conditioned predictor with CEM\-based model\-predictive control\([Maes et al\., 2026](https://arxiv.org/html/2610.08960#bib.bib3)\)\. Candidate action sequences are evaluated by rolling out the predictor in latent space and comparing the predicted outcomes with the encoded goal\. Control proceeds in a receding\-horizon manner using updated visual observations\. We retain the original latent\-planning formulation without adding the temporal distance or policy\-learning components of DTRC\.

### A\.2GC\-IDM

We use the goal\-conditioned inverse\-dynamics policy of GC\-IDM\([Nguyen et al\., 2026](https://arxiv.org/html/2610.08960#bib.bib2)\)\. The policy takes the current LeWM feature, goal feature, and remaining horizon as inputs and predicts the recorded action through a supervised regression objective\. We retain its MLP architecture and horizon\-conditioning mechanism\. For the additional OGBench tasks, the action output dimension is adjusted to the corresponding environment\. At evaluation time, the policy predicts actions directly from the updated visual input without latent rollout or trajectory search\.

### A\.3GCBC

GCBC learns a direct goal\-conditioned policy by supervised imitation\([Ghosh et al\., 2021](https://arxiv.org/html/2610.08960#bib.bib17)\)\. Future observations from the same trajectory are sampled as goals, and an MLP predicts the recorded action from the current feature, goal feature, and remaining horizon\. Training minimizes the mean squared action\-prediction error\. This baseline uses neither a value function nor advantage weighting\. It is distinct from the flow\-matching behavioral\-cloning variant used in our component comparisons\.

### A\.4GC\-IQL

GC\-IQL learns a goal\-conditioned action\-value functionQ⁡\(zt,at,zg\)Q\(z\_\{t\},a\_\{t\},z\_\{g\}\)and value functionV⁡\(zt,zg\)V\(z\_\{t\},z\_\{g\}\)\([Kostrikov et al\., 2022](https://arxiv.org/html/2610.08960#bib.bib19);[Park et al\., 2025a](https://arxiv.org/html/2610.08960#bib.bib12)\)\. The action\-value function is trained by Bellman regression using a target value network\. The value function is trained by upper\-expectile regression toward the target action\-value estimates for recorded actions\. Goal relabeling supplies a reward of zero at the relabeled goal and−1\-1otherwise, with bootstrapping terminated at the goal\. We first train the value networks and then extract a direct goal\-conditioned policy using the weighted regression procedure described below\.

### A\.5HILP

We adapt the Hilbert value parameterization of HILP\([Park et al\., 2024](https://arxiv.org/html/2610.08960#bib.bib1)\)to frozen LeWM features:

V⁡\(zt,zg\)=−‖ϕH​\(zt\)−ϕH​\(zg\)‖2,V\(z\_\{t\},z\_\{g\}\)=\-\\left\\\|\\phi\_\{\\mathrm\{H\}\}\(z\_\{t\}\)\-\\phi\_\{\\mathrm\{H\}\}\(z\_\{g\}\)\\right\\\|\_\{2\},\(18\)whereϕH\\phi\_\{\\mathrm\{H\}\}is a learned embedding network\. The value function is trained with goal\-conditioned expectile temporal\-difference learning\. We then extract a direct goal\-conditioned policy using the same weighted regression procedure as GC\-IQL\. This adaptation retains symmetric Hilbert value structure but replaces its directional skill learning and goal\-directed skill selection with direct policy extraction\. We denote this adaptation by HILP throughout the control results\.

#### Policy extraction for GC\-IQL and HILP\.

For both baselines, the policy is conditioned on the current feature, goal feature, and an embedding of the normalized remaining horizon\. Recorded actions are fitted with a weighted Gaussian negative log\-likelihood\. The weights are computed from the target\-value difference

Δ​Vt,g=Vtarg​\(zt\+1,zg\)−Vtarg​\(zt,zg\),wt,g=min⁡\{100,exp⁡\(αbase​Δ​Vt,g\)\},\\Delta V\_\{t,g\}=V\_\{\\mathrm\{targ\}\}\(z\_\{t\+1\},z\_\{g\}\)\-V\_\{\\mathrm\{targ\}\}\(z\_\{t\},z\_\{g\}\),\\qquad w\_\{t,g\}=\\min\\left\\\{100,\\exp\(\\alpha\_\{\\mathrm\{base\}\}\\Delta V\_\{t,g\}\)\\right\\\},\(19\)withαbase=10\\alpha\_\{\\mathrm\{base\}\}=10for GC\-IQL andαbase=3\\alpha\_\{\\mathrm\{base\}\}=3for HILP\. The weights are treated as fixed during policy optimization\. This implementation uses the value difference rather than theQ−VQ\-Vadvantage of the original IQL policy\-extraction rule\. At evaluation time, both policies execute their predicted mean action\.

#### Network parameterization\.

GCBC, GC\-IQL, and HILP use MLPs with three hidden layers of width 512\. Their policies receive a 64\-dimensional sinusoidal embedding of the normalized remaining horizon\. The HILP embeddingϕH\\phi\_\{\\mathrm\{H\}\}has 64 dimensions\. These policies predict a single action at each decision step and are queried again after receiving the next observation\.

## Appendix BImplementation Details

#### Goal sampling\.

For the temporal objective in Section[3\.2](https://arxiv.org/html/2610.08960#S3.SS2), each current state is paired with a goal sampled from three sources\. We use the next state as the goal with probability0\.10\.1, sample a future state from the same trajectory with probability0\.60\.6, and sample an observation uniformly from the offline dataset with probability0\.30\.3\. Future states are sampled geometrically withPr⁡\(Δ=m\)=0\.02​\(0\.98\)m−1\\Pr\(\\Delta=m\)=0\.02\(0\.98\)^\{m\-1\}form≥1m\\geq 1, with offsets beyond the remaining trajectory clipped to its final state\.

#### Temporal\-critic hyperparameters\.

We use a bootstrap horizon ofn=8n=8and a local supervision range ofJ=8J=8\. The expectile parameterτe=0\.3\\tau\_\{e\}=0\.3is used for bothℒgoal\\mathcal\{L\}\_\{\\mathrm\{goal\}\}andℒlocal\\mathcal\{L\}\_\{\\mathrm\{local\}\}\. The loss coefficients areλlocal=1\\lambda\_\{\\mathrm\{local\}\}=1andλcons=0\.1\\lambda\_\{\\mathrm\{cons\}\}=0\.1\.

#### Target networks\.

We maintain slowly updated target networks for the control representation and directed temporal distance\. After each critic update, each target parameterw¯\\bar\{w\}is updated from its online counterpartwwby exponential moving average \(EMA\)

w¯←0\.995​w¯\+0\.005​w,\\bar\{w\}\\leftarrow 0\.995\\,\\bar\{w\}\+0\.005\\,w,\(20\)following standard parameter averaging\([Polyak and Juditsky, 1992](https://arxiv.org/html/2610.08960#bib.bib22)\)\. Bootstrap targets are computed entirely with the target representation and target temporal distance, with gradients stopped through all target quantities\. Target\-distance estimates and the resulting bootstrapped targets are capped atdmax=200d\_\{\\max\}=200for numerical stability, while online critic predictions are left unclipped\.

#### Numerical distance evaluation\.

For numerical stability, the Euclidean component of Eq\. equation[4](https://arxiv.org/html/2610.08960#S3.E4)is evaluated using the smoothed normr2\+ϵ2−ϵ\\sqrt\{r^\{2\}\+\\epsilon^\{2\}\}\-\\epsilon, wherer=‖S⁡\(et\)−S⁡\(eg\)‖2r=\\\|S\(e\_\{t\}\)\-S\(e\_\{g\}\)\\\|\_\{2\}andϵ=10−4\\epsilon=10^\{\-4\}, which preserves zero self\-distance while smoothing the norm near zero\.

#### Latent dynamics\.

To obtain the predicted successorz^t\+k​\(𝐚\)\\hat\{z\}\_\{t\+k\}\(\\mathbf\{a\}\)in Eq\. equation[27](https://arxiv.org/html/2610.08960#A2.E27), we learn an ensemble of residual dynamics predictors in the frozen LeWM latent space\. For themm\-th ensemble member, the next latent state is defined as

z^t\+1\(m\)=zt\+fm​\(zt,at\),m∈\{1,…,M\},\\hat\{z\}\_\{t\+1\}^\{\(m\)\}=z\_\{t\}\+f\_\{m\}\(z\_\{t\},a\_\{t\}\),\\qquad m\\in\\\{1,\\ldots,M\\\},\(21\)wherefmf\_\{m\}predicts the action\-conditioned change in the LeWM latent andMMdenotes the number of ensemble members\. These predictors are trained separately from the pretrained LeWM predictor\. They operate in the visual latent spacezz, whereas the transition headTξT\_\{\\xi\}used in the representation\-consistency objective operates in the control representation space\.

For multi\-step training, each predictor is initialized atz^t\(m\)=zt\\hat\{z\}\_\{t\}^\{\(m\)\}=z\_\{t\}and recursively applied under the recorded actions:

z^t\+ℓ\(m\)=z^t\+ℓ−1\(m\)\+fm\(z^t\+ℓ−1\(m\),at\+ℓ−1\),ℓ=1,…,HF\.\\hat\{z\}\_\{t\+\\ell\}^\{\(m\)\}=\\hat\{z\}\_\{t\+\\ell\-1\}^\{\(m\)\}\+f\_\{m\}\\left\(\\hat\{z\}\_\{t\+\\ell\-1\}^\{\(m\)\},a\_\{t\+\\ell\-1\}\\right\),\\qquad\\ell=1,\\ldots,H\_\{F\}\.\(22\)The predicted rollout is matched to the corresponding recorded LeWM latents with

ℒdyn=𝔼𝒟​\[1M​HF​dz​∑m=1M∑ℓ=1HF‖z^t\+ℓ\(m\)−zt\+ℓ‖22\],\\mathcal\{L\}\_\{\\mathrm\{dyn\}\}=\\mathbb\{E\}\_\{\\mathcal\{D\}\}\\left\[\\frac\{1\}\{MH\_\{F\}d\_\{z\}\}\\sum\_\{m=1\}^\{M\}\\sum\_\{\\ell=1\}^\{H\_\{F\}\}\\left\\\|\\hat\{z\}\_\{t\+\\ell\}^\{\(m\)\}\-z\_\{t\+\\ell\}\\right\\\|\_\{2\}^\{2\}\\right\],\(23\)whereHFH\_\{F\}is the dynamics training horizon anddzd\_\{z\}is the LeWM latent dimension\. For candidate evaluation, each ensemble member is rolled out forkktransitions under the proposed action sequence𝐚\\mathbf\{a\}\. The successor used in Eq\. equation[27](https://arxiv.org/html/2610.08960#A2.E27)is the ensemble\-mean prediction

z^t\+k​\(𝐚\)=1M​∑m=1Mz^t\+k\(m\)​\(𝐚\),\\hat\{z\}\_\{t\+k\}\(\\mathbf\{a\}\)=\\frac\{1\}\{M\}\\sum\_\{m=1\}^\{M\}\\hat\{z\}\_\{t\+k\}^\{\(m\)\}\(\\mathbf\{a\}\),\(24\)which is mapped through the target representationψ¯\\bar\{\\psi\}and then evaluated by the target temporal critic\.

#### Model\-assisted policy augmentation\.

As a training\-time augmentation to the advantage\-weighted policy objective, we use the policy to generate alternative action targets that remain supported by the offline behavior and can be predicted consistently by the learned dynamics\. Specifically, under the same policy conditionctc\_\{t\}, we sampleNNcandidate action sequences\{𝐚t\(i\)\}i=1N\\\{\\mathbf\{a\}\_\{t\}^\{\(i\)\}\\\}\_\{i=1\}^\{N\}from the conditional flow policy\. Before ranking candidates with the temporal critic, we filter them using a behavior\-support error and a dynamics\-disagreement score, since a low predicted temporal cost is informative only for candidates whose actions remain compatible with the offline data and whose predicted outcomes are consistent across the learned dynamics ensemble\.

Behavior support\.We train an auxiliary state\-conditioned flow modelvνv\_\{\\nu\}on recorded actions by minimizing the standard conditional flow\-matching loss\([Lipman et al\., 2023](https://arxiv.org/html/2610.08960#bib.bib10)\), with the current LeWM latentztz\_\{t\}as its condition\. For theii\-th candidate𝐚t\(i\)\\mathbf\{a\}\_\{t\}^\{\(i\)\}, we define the behavior\-support error as

qbeh\(i\)=1R​k​da​∑r=1R‖vν​\(xsr\(i,r\),sr,zt\)−\(𝐚t\(i\)−x0\(r\)\)‖22,q\_\{\\mathrm\{beh\}\}^\{\(i\)\}=\\frac\{1\}\{Rkd\_\{a\}\}\\sum\_\{r=1\}^\{R\}\\left\\\|v\_\{\\nu\}\\left\(x\_\{s\_\{r\}\}^\{\(i,r\)\},s\_\{r\},z\_\{t\}\\right\)\-\\left\(\\mathbf\{a\}\_\{t\}^\{\(i\)\}\-x\_\{0\}^\{\(r\)\}\\right\)\\right\\\|\_\{2\}^\{2\},\(25\)wherex0\(r\)∼𝒩⁡\(0,Ik​da\)x\_\{0\}^\{\(r\)\}\\sim\\mathcal\{N\}\(0,I\_\{kd\_\{a\}\}\),sr∼𝒰⁡\(0\.1,0\.9\)s\_\{r\}\\sim\\mathcal\{U\}\(0\.1,0\.9\), andxsr\(i,r\)=\(1−sr\)​x0\(r\)\+sr​𝐚t\(i\)x\_\{s\_\{r\}\}^\{\(i,r\)\}=\(1\-s\_\{r\}\)x\_\{0\}^\{\(r\)\}\+s\_\{r\}\\mathbf\{a\}\_\{t\}^\{\(i\)\}\. The parameterRRdenotes the number of flow residuals used to estimate the error\. A candidate satisfyingqbeh\(i\)≤δbehq\_\{\\mathrm\{beh\}\}^\{\(i\)\}\\leq\\delta\_\{\\mathrm\{beh\}\}is considered sufficiently supported by the recorded behavior\.

Dynamics agreement\.For the same candidate𝐚t\(i\)\\mathbf\{a\}\_\{t\}^\{\(i\)\}, each dynamics model predicts a latent trajectory\{z^t\+ℓ\(m,i\)\}ℓ=1k\\\{\\hat\{z\}\_\{t\+\\ell\}^\{\(m,i\)\}\\\}\_\{\\ell=1\}^\{k\}\. We define the dynamics\-disagreement score as

qdyn\(i\)=1k​dz​∑ℓ=1k∑j=1dzStdm=1,…,M⁡\(\[z^t\+ℓ\(m,i\)\]j\),q\_\{\\mathrm\{dyn\}\}^\{\(i\)\}=\\frac\{1\}\{kd\_\{z\}\}\\sum\_\{\\ell=1\}^\{k\}\\sum\_\{j=1\}^\{d\_\{z\}\}\\operatorname\{Std\}\_\{m=1,\\ldots,M\}\\left\(\[\\hat\{z\}\_\{t\+\\ell\}^\{\(m,i\)\}\]\_\{j\}\\right\),\(26\)wheredzd\_\{z\}denotes the LeWM latent dimension\. A candidate satisfyingqdyn\(i\)≤δdynq\_\{\\mathrm\{dyn\}\}^\{\(i\)\}\\leq\\delta\_\{\\mathrm\{dyn\}\}is considered sufficiently consistent across the dynamics ensemble\.

The thresholdsδbeh\\delta\_\{\\mathrm\{beh\}\}andδdyn\\delta\_\{\\mathrm\{dyn\}\}are calibrated from the corresponding scores of recorded actions\. Every5050training updates, we compute the9090th percentile of each score in the current data batch and update the corresponding threshold by exponential moving average with retention coefficient0\.990\.99\.

We denote by𝒱t\\mathcal\{V\}\_\{t\}the candidate action sequences satisfying bothqbeh\(i\)≤δbehq\_\{\\mathrm\{beh\}\}^\{\(i\)\}\\leq\\delta\_\{\\mathrm\{beh\}\}andqdyn\(i\)≤δdynq\_\{\\mathrm\{dyn\}\}^\{\(i\)\}\\leq\\delta\_\{\\mathrm\{dyn\}\}\. Among these candidates, we select the sequence with the lowest predicted temporal cost:

𝐚t⋆=arg⁡min𝐚∈𝒱t​\[k\+dθ¯​\(ψ¯​\(z^t\+k​\(𝐚\)\),e¯g\)\]\.\\mathbf\{a\}\_\{t\}^\{\\star\}=\\arg\\min\_\{\\mathbf\{a\}\\in\\mathcal\{V\}\_\{t\}\}\\left\[k\+d\_\{\\bar\{\\theta\}\}\\left\(\\bar\{\\psi\}\\left\(\\hat\{z\}\_\{t\+k\}\(\\mathbf\{a\}\)\\right\),\\bar\{e\}\_\{g\}\\right\)\\right\]\.\(27\)If𝒱t\\mathcal\{V\}\_\{t\}is empty, no model\-assisted update is applied to that training sample\.

We compare the selected candidate with the recorded continuation through the predicted improvement

𝒜t,gim=C𝒟−C⋆,\\mathcal\{A\}\_\{t,g\}^\{\\mathrm\{im\}\}=C\_\{\\mathcal\{D\}\}\-C\_\{\\star\},\(28\)whereC𝒟=k\+dθ¯​\(e¯t\+k,e¯g\)C\_\{\\mathcal\{D\}\}=k\+d\_\{\\bar\{\\theta\}\}\(\\bar\{e\}\_\{t\+k\},\\bar\{e\}\_\{g\}\)is the predicted temporal cost of the recorded continuation, andC⋆=k\+dθ¯​\(ψ¯​\(z^t\+k​\(𝐚t⋆\)\),e¯g\)C\_\{\\star\}=k\+d\_\{\\bar\{\\theta\}\}\\left\(\\bar\{\\psi\}\(\\hat\{z\}\_\{t\+k\}\(\\mathbf\{a\}\_\{t\}^\{\\star\}\)\),\\bar\{e\}\_\{g\}\\right\)is the predicted temporal cost of the selected candidate\. A positive𝒜t,gim\\mathcal\{A\}\_\{t,g\}^\{\\mathrm\{im\}\}indicates that the candidate is predicted to improve upon the recorded continuation\.

Accepted candidates provide additional flow\-matching targets with weight

wt,gim=min⁡\{wmax,exp⁡\(𝒜t,gimβ\)\},w\_\{t,g\}^\{\\mathrm\{im\}\}=\\min\\left\\\{w\_\{\\max\},\\exp\\left\(\\frac\{\\mathcal\{A\}\_\{t,g\}^\{\\mathrm\{im\}\}\}\{\\beta\}\\right\)\\right\\\},\(29\)and the corresponding model\-assisted policy objective is

ℒpolicyim=𝔼⁡\[𝟏​\(𝒜t,gim\>0\)​wt,gim​ℓFM​\(ω,𝐚t⋆,ct\)\],\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\\mathrm\{im\}\}=\\mathbb\{E\}\\left\[\\mathbf\{1\}\\left\(\\mathcal\{A\}\_\{t,g\}^\{\\mathrm\{im\}\}\>0\\right\)w\_\{t,g\}^\{\\mathrm\{im\}\}\\,\\ell\_\{\\mathrm\{FM\}\}\\left\(\\omega;\\mathbf\{a\}\_\{t\}^\{\\star\},c\_\{t\}\\right\)\\right\],\(30\)whereλim\\lambda\_\{\\mathrm\{im\}\}controls the contribution of the model\-assisted targets\. The candidate contributes to the policy update only when𝒜t,gim\>0\\mathcal\{A\}\_\{t,g\}^\{\\mathrm\{im\}\}\>0\.

## Appendix CAlgorithm and Training Procedure

Algorithm 1Training Directed Temporal Representations for Control1:Input:offline trajectories

𝒟\\mathcal\{D\}, frozen visual encoder

EE
2:Initialize

ψ,dθ,Tξ,πω,\{fm\}m=1M,vν\\psi,d\_\{\\theta\},T\_\{\\xi\},\\pi\_\{\\omega\},\\\{f\_\{m\}\\\}\_\{m=1\}^\{M\},v\_\{\\nu\}and target networks

ψ¯,dθ¯\\bar\{\\psi\},d\_\{\\bar\{\\theta\}\}
3:for

u=1,…,Uu=1,\\ldots,Udo

4:Sample

\(ot,𝐚t,ot\+k\)∼𝒟\(o\_\{t\},\\mathbf\{a\}\_\{t\},o\_\{t\+k\}\)\\sim\\mathcal\{D\}and relabel goal

ogo\_\{g\}
5:

zt←E⁡\(ot\)z\_\{t\}\\leftarrow E\(o\_\{t\}\),

zg←E⁡\(og\)z\_\{g\}\\leftarrow E\(o\_\{g\}\);

et←ψ⁡\(zt\)e\_\{t\}\\leftarrow\\psi\(z\_\{t\}\),

eg←ψ⁡\(zg\)e\_\{g\}\\leftarrow\\psi\(z\_\{g\}\)
6:Set

ct←\[zt,zg,et,eg,Emb⁡\(ht/H\)\]c\_\{t\}\\leftarrow\[z\_\{t\},z\_\{g\},e\_\{t\},e\_\{g\},\\operatorname\{Emb\}\(h\_\{t\}/H\)\]
7:Update

\{fm\}m=1M\\\{f\_\{m\}\\\}\_\{m=1\}^\{M\}with

ℒdyn\\mathcal\{L\}\_\{\\mathrm\{dyn\}\}and update

vνv\_\{\\nu\}on recorded actions

8:Compute

yt,gy\_\{t,g\}with Eq\.[7](https://arxiv.org/html/2610.08960#S3.E7)

9:Compute

𝒜t,gdata\\mathcal\{A\}\_\{t,g\}^\{\\mathrm\{data\}\},

wt,gdataw\_\{t,g\}^\{\\mathrm\{data\}\}, and

ℒpolicydata\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\\mathrm\{data\}\}
10:

ℒpolicyim←0\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\\mathrm\{im\}\}\\leftarrow 0
11:if

u≥Uimu\\geq U\_\{\\mathrm\{im\}\}then

12:Sample

\{𝐚t\(i\)\}i=1N∼πω\(⋅∣ct\)\\\{\\mathbf\{a\}\_\{t\}^\{\(i\)\}\\\}\_\{i=1\}^\{N\}\\sim\\pi\_\{\\omega\}\(\\cdot\\mid c\_\{t\}\)
13:Compute

\{qbeh\(i\),qdyn\(i\)\}i=1N\\\{q\_\{\\mathrm\{beh\}\}^\{\(i\)\},q\_\{\\mathrm\{dyn\}\}^\{\(i\)\}\\\}\_\{i=1\}^\{N\}and obtain

𝒱t\\mathcal\{V\}\_\{t\}
14:if

𝒱t≠∅\\mathcal\{V\}\_\{t\}\\neq\\varnothingthen

15:Select

𝐚t⋆\\mathbf\{a\}\_\{t\}^\{\\star\}with Eq\.[27](https://arxiv.org/html/2610.08960#A2.E27)

16:Compute

𝒜t,gim\\mathcal\{A\}\_\{t,g\}^\{\\mathrm\{im\}\}with Eq\.[28](https://arxiv.org/html/2610.08960#A2.E28)

17:if

𝒜t,gim\>0\\mathcal\{A\}\_\{t,g\}^\{\\mathrm\{im\}\}\>0then

18:Compute

ℒpolicyim\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\\mathrm\{im\}\}with Eq\.[30](https://arxiv.org/html/2610.08960#A2.E30)

19:Refine eligible bootstrapped temporal targets

20:endif

21:endif

22:endif

23:Update

ψ,dθ,Tξ\\psi,d\_\{\\theta\},T\_\{\\xi\}by minimizing

ℒtemp\\mathcal\{L\}\_\{\\mathrm\{temp\}\}
24:Update

ψ¯,dθ¯\\bar\{\\psi\},d\_\{\\bar\{\\theta\}\}with Eq\.[20](https://arxiv.org/html/2610.08960#A2.E20)

25:Update

πω\\pi\_\{\\omega\}by minimizing

ℒpolicy=ℒpolicydata\+λim​ℒpolicyim\\mathcal\{L\}\_\{\\mathrm\{policy\}\}=\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\\mathrm\{data\}\}\+\\lambda\_\{\\mathrm\{im\}\}\\mathcal\{L\}\_\{\\mathrm\{policy\}\}^\{\\mathrm\{im\}\}
26:endfor

27:Return:

πω\\pi\_\{\\omega\}

## Appendix DArchitectures and Hyperparameters

#### Network architectures\.

We implement all trainable modules as multilayer perceptrons\. Hidden layers use GELU activations\([Hendrycks and Gimpel, 2016](https://arxiv.org/html/2610.08960#bib.bib23)\), with LayerNorm applied in the representation, policy, behavior, and dynamics networks\([Ba et al\., 2016](https://arxiv.org/html/2610.08960#bib.bib24)\)\. The symmetric headSS, order\-sensitive headAθA\_\{\\theta\}, and transition headTξT\_\{\\xi\}use GELU without LayerNorm\. The representation head additionally applies LayerNorm to its output\. We use linear output layers throughout and do not use dropout\. The directed\-distance scale is initialized toα=1\\alpha=1\. Table[4](https://arxiv.org/html/2610.08960#A4.T4)reports the corresponding input, hidden, and output dimensions\.

Table 4:Trainable network architectures\. Hidden widths exclude output layers\. The policy condition has dimension2​dz\+2​de\+64=7042d\_\{z\}\+2d\_\{e\}\+64=704\.Emb⁡\(s\)\\operatorname\{Emb\}\(s\)denotes the6464\-dimensional flow\-time embedding\.TξT\_\{\\xi\}predicts a residual update toee, and eachfmf\_\{m\}predicts a residual update tozz\. Dynamics ensemble members have independent parameters and are trained on the same minibatches\.

#### Hyperparameters\.

Table[5](https://arxiv.org/html/2610.08960#A4.T5)summarizes the default configuration\.

Table 5:Default training and inference hyperparameters for DTRC\.HyperparameterValueOptimizationTraining iterationsUU200,000200\{,\}000Minibatch size10241024OptimizerAdamLearning rate3×10−43\\times 10^\{\-4\}\(constant\)Adam moment coefficients\(0\.9,0\.999\)\(0\.9,0\.999\)Adam numerical epsilon10−810^\{\-8\}Weight decay00Gradient clippingNoneTemporal critic and goal samplingNext / future / dataset\-wide goal probabilities0\.1/0\.6/0\.30\.1/0\.6/0\.3Future\-goal continuationγg\\gamma\_\{g\}0\.980\.98Bootstrap horizonnn88Local supervision rangeJJ88Expectileτe\\tau\_\{e\}0\.30\.3Local coefficientλlocal\\lambda\_\{\\mathrm\{local\}\}11Consistency coefficientλcons\\lambda\_\{\\mathrm\{cons\}\}0\.10\.1Target\-network update rate0\.0050\.005Target distance / bootstrap capdmaxd\_\{\\max\}200200Norm\-smoothing constantϵ\\epsilon10−410^\{\-4\}Flow policyAction horizonkk11Horizon\-conditioning capHH5050Flow sampling solverForward EulerFlow sampling steps88Advantage temperatureβ\\beta11Maximum weightwmaxw\_\{\\max\}2020Action clip \(standardized space\)\[−3,3\]\[\-3,3\]Test\-time policy samples per decision11Model\-assisted augmentationDynamics ensemble sizeMM44Dynamics training horizonHFH\_\{F\}44Model\-assisted start iterationUimU\_\{\\mathrm\{im\}\}10,00010\{,\}000Policy candidates per conditionNN1616Behavior\-score samplesRR22Model\-assisted coefficientλim\\lambda\_\{\\mathrm\{im\}\}11

## Appendix EAdditional Results

### E\.1Component Ablations

Table 6:Ablation study on the four LeWM tasks\.Success rates \(%\) are reported as mean±\\pmsample standard deviation over five evaluation seeds\. All variants use the same 50 start–goal pairs per task and seed, a goal offset of 25 recorded transitions, and a 50\-step interaction budget\.#### Configuration details\.

Flow BC uses unweighted conditional flow matching\. Flow BC \+ temporal supervision adds the learned DTRC control representation and temporal\-advantage weighting, without model\-assisted augmentation\. The behavior\-support ablation retains the dynamics\-agreement filter\. The directional\-residual ablation removes the order\-sensitive component and retains only the symmetric metric component\.

### E\.2Performance across Goal Offsets

Table 7:Goal\-conditioned control across goal offsets\.Task\-averaged success rates \(%\) over six OGBench tasks\.#### Evaluation details\.

The six tasks are PointMaze, Cube\-Double, Scene, AntMaze\-Medium, HumanoidMaze\-Medium, and Puzzle\-3x3, with equal weight in the task average\. Within each goal offset, all methods use the same 50 start–goal pairs per task and evaluation seed\. We first average success rates across tasks for each seed, then report the mean and sample standard deviation over 5 evaluation seeds\. Goal offsets count recorded trajectory transitions rather than minimum reaching times\. HILP denotes the Hilbert\-value adaptation used in our visual\-control comparison\. Checkpoints are fixed across goal offsets and evaluation seeds, so the reported deviations do not measure variability across independent training runs\.

## Appendix FAdditional Representation Analyses

### F\.1Representation Spaces across Tasks

The main text visualizes the transformation from frozen LeWM features to the learned DTRC representation on PushT\. We provide the corresponding visualizations for the remaining three LeWM tasks in Figures[6](https://arxiv.org/html/2610.08960#A6.F6)and[7](https://arxiv.org/html/2610.08960#A6.F7)\. For each task, we visualize frozen predictive featureszz, the learned control representationee, symmetric coordinatesS⁡\(e\)S\(e\), and directed potentialsA⁡\(e\)A\(e\)\. Each space is projected independently onto its first two principal components\. Colors indicate normalized time within the offline episode\. The projections are used only for qualitative visualization; all temporal distances and quantitative diagnostics are computed in the full learned representation\. PCA orientations and scales are therefore not directly comparable across spaces or tasks\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/tworoom_four_spaces.png)\(a\)TwoRoom
![Refer to caption](https://arxiv.org/html/2610.08960v1/reacher_four_spaces.png)\(b\)Reacher

Figure 6:Learned representation spaces on TwoRoom and Reacher\.From left to right, each row shows frozen LeWM featureszz, the learned control representationee, symmetric coordinatesS⁡\(e\)S\(e\), and directed potentialsA⁡\(e\)A\(e\)\. Colors denote normalized offline episode time\. PCA is fitted independently in each space\.![Refer to caption](https://arxiv.org/html/2610.08960v1/cube_four_spaces.png)Figure 7:Learned representation spaces on OGBench\-Cube\.From left to right, the panels show frozen LeWM featureszz, the learned control representationee, symmetric coordinatesS⁡\(e\)S\(e\), and directed potentialsA⁡\(e\)A\(e\)\. Colors denote normalized offline episode time\. PCA is fitted independently in each space\.The learned spaces exhibit distinct organizations across the three tasks\. TwoRoom develops a curved control representation with separate structures in the symmetric and directed heads\. Reacher exhibits an approximately cyclic organization in both the control representation and directed potentials\. Cube produces a different non\-cyclic geometry, with the symmetric and directed heads again organizing the representation differently\. Together with the PushT visualization in the main text, these examples show that DTRC does not impose a single low\-dimensional geometric form across tasks\. The relevant common structure is instead defined by the induced state–goal temporal distance and its control semantics\.

#### Additional OGBench tasks\.

We further visualize the same four representation spaces on the six additional OGBench tasks\. These tasks exhibit substantially different projected geometries, providing a broader qualitative view of the task\-dependent representations learned by DTRC\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/pointmaze_four_spaces.png)\(a\)PointMaze
![Refer to caption](https://arxiv.org/html/2610.08960v1/antmaze_four_spaces.png)\(b\)AntMaze\-Medium

Figure 8:Learned representation spaces on PointMaze and AntMaze\-Medium\.For each task, the panels show frozen LeWM featureszz, the learned control representationee, symmetric coordinatesS⁡\(e\)S\(e\), and directed potentialsA⁡\(e\)A\(e\)\. Colors indicate normalized offline episode time\. PCA is fitted independently in each space, so orientations and scales are not directly comparable across panels or tasks\.![Refer to caption](https://arxiv.org/html/2610.08960v1/cube_double_four_spaces.png)\(a\)Cube\-Double
![Refer to caption](https://arxiv.org/html/2610.08960v1/scene_four_spaces.png)\(b\)Scene

Figure 9:Learned representation spaces on Cube\-Double and Scene\.The visualization follows the same construction as Figure[8](https://arxiv.org/html/2610.08960#A6.F8)\. Each space is projected independently onto its first two principal components and colored by normalized offline episode time\.![Refer to caption](https://arxiv.org/html/2610.08960v1/humanoidmaze_four_spaces.png)\(a\)HumanoidMaze\-Medium
![Refer to caption](https://arxiv.org/html/2610.08960v1/puzzle_four_spaces.png)\(b\)Puzzle\-3x3

Figure 10:Learned representation spaces on HumanoidMaze\-Medium and Puzzle\-3x3\.From left to right, each task shows frozen LeWM featureszz, the learned control representationee, symmetric coordinatesS⁡\(e\)S\(e\), and directed potentialsA⁡\(e\)A\(e\)\. PCA is fitted independently in each space\.Across the ten tasks, the learned representation does not converge to a single common projected geometry\. Navigation, reaching, manipulation, and puzzle environments produce different organizations inee,S⁡\(e\)S\(e\), andA⁡\(e\)A\(e\)\. In several tasks, the leading principal components of the frozen LeWM features explain only a small fraction of the variance, while the learned control and distance\-head representations exhibit substantially more structured low\-dimensional organization\. The symmetric and directed heads also differ markedly across tasks, consistent with their distinct roles in the quasimetric and with the task\-dependent directional statistics reported in the main text\. These visualizations therefore support interpreting DTRC through the induced state–goal temporal geometry rather than through any single low\-dimensional representation shape\.

### F\.2Additional Temporal Diagnostics

The main text reports detailed temporal\-distance diagnostics on PushT\. Here we provide the corresponding results for TwoRoom, Reacher, and OGBench\-Cube under the same held\-out, offset\-balanced evaluation protocol\. For each task, we evaluate 20,000 state–future\-goal pairs, with 200 pairs at every integer temporal offset from 1 to 100 transitions\. All distances are computed with the EMA target network at 200,000 training updates in the full learned representation\. Recorded temporal offset is used as a reference for trajectory duration and is not treated as ground\-truth optimal reaching cost\.

Each figure reports three properties\. The left panel compares forward temporal distance with recorded temporal offset\. The middle panel compares forward and reverse temporal costs together with the shared symmetric contribution of the same DTRC model\. The right panel reports one\-step temporal progress,

Pt,g\(1\)=d⁡\(et,eg\)−d⁡\(et\+1,eg\)\.P\_\{t,g\}^\{\(1\)\}=d\(e\_\{t\},e\_\{g\}\)\-d\(e\_\{t\+1\},e\_\{g\}\)\.These diagnostics use the same definitions as Section[4\.3](https://arxiv.org/html/2610.08960#S4.SS3)\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/tworoom_diagnostics.png)

![Refer to caption](https://arxiv.org/html/2610.08960v1/reacher_diagnostics.png)

Figure 11:Additional temporal diagnostics on TwoRoom and Reacher\.Each task is evaluated on 20,000 held\-out state–future\-goal pairs with balanced temporal offsets from 1 to 100 transitions\. The panels show temporal structure across recorded offsets, directional state–goal cost, and one\-step temporal progress\. The shared symmetric term is computed from the same trained DTRC model and is not a separately trained baseline\.![Refer to caption](https://arxiv.org/html/2610.08960v1/cube_diagnostics.png)Figure 12:Additional temporal diagnostics on OGBench\-Cube\.The evaluation uses the same held\-out, offset\-balanced protocol as the main text\. The panels show temporal structure across recorded offsets, directional state–goal cost, and one\-step temporal progress\.The diagnostics reveal distinct long\-range structure across tasks\. TwoRoom retains a strong rank association between temporal distance and recorded offset, withρ=0\.80\\rho=0\.80, while Reacher shows a weaker association ofρ=0\.51\\rho=0\.51and Cube shows a substantially weaker association ofρ=0\.22\\rho=0\.22\. The forward and reverse curves are also much closer on these tasks than on PushT, consistent with the task\-dependent directional statistics reported in the main text\. Despite these differences, positive one\-step progress remains above50%50\\%on all three tasks:65\.8%65\.8\\%on TwoRoom,59\.4%59\.4\\%on Reacher, and72\.3%72\.3\\%on Cube\. These results reinforce the distinction between local actionable temporal structure and global agreement with recorded trajectory duration\.

#### Additional OGBench tasks\.

We further apply the same held\-out, offset\-balanced diagnostic protocol to the six additional visual OGBench tasks\. For each task, we evaluate 20,000 state–future\-goal pairs, with 200 pairs at every integer temporal offset from 1 to 100 transitions\. All distances are computed with the EMA target network at 200,000 training updates in the full learned representation\. The figures report temporal structure across recorded offsets, directional state–goal cost, and one\-step temporal progress using the same definitions as in the main text\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/pointmaze_diagnostics.png)\(a\)PointMaze
![Refer to caption](https://arxiv.org/html/2610.08960v1/antmaze-medium_diagnostics.png)\(b\)AntMaze\-Medium

Figure 13:Temporal diagnostics on PointMaze and AntMaze\-Medium\.The panels show temporal structure across recorded offsets, directional state–goal cost, and one\-step temporal progress\. Forward, reverse, and shared symmetric curves are computed from the same trained DTRC model\.![Refer to caption](https://arxiv.org/html/2610.08960v1/cube-double_diagnostics.png)\(a\)Cube\-Double
![Refer to caption](https://arxiv.org/html/2610.08960v1/scene_diagnostics.png)\(b\)Scene

Figure 14:Temporal diagnostics on Cube\-Double and Scene\.The evaluation follows the same held\-out, offset\-balanced protocol used for the main\-text representation diagnostics\. Recorded offset is used as a reference for trajectory duration rather than as ground\-truth optimal reaching cost\.![Refer to caption](https://arxiv.org/html/2610.08960v1/humanoidmaze-medium_diagnostics.png)\(a\)HumanoidMaze\-Medium
![Refer to caption](https://arxiv.org/html/2610.08960v1/puzzle-3x3_diagnostics.png)\(b\)Puzzle\-3x3

Figure 15:Temporal diagnostics on HumanoidMaze\-Medium and Puzzle\-3x3\.Each task is evaluated on the same offset\-balanced cohort used throughout the additional temporal diagnostics\.Across the six additional tasks, the relation between temporal distance and recorded trajectory duration remains task\-dependent\. Spearman correlation over offsets 1–100 ranges from0\.530\.53on Cube\-Double to0\.740\.74on PointMaze\. The directional curves also exhibit different degrees of forward–reverse separation across tasks, further supporting the task\-dependent use of the order\-sensitive component\. In contrast, positive one\-step progress remains above60%60\\%on every task, ranging from60\.4%60\.4\\%on HumanoidMaze\-Medium to84\.9%84\.9\\%on PointMaze\. Together with the four LeWM tasks, these results show that transition\-level temporal progress is observed across all ten evaluated control problems without requiring a common long\-range or directional geometry\.

### F\.3Quasimetric Decomposition

We further examine the two components of the directed temporal quasimetric along recorded trajectories\. For each task, we evaluate pairwise distances between states from a recorded episode and index source and goal states by normalized episode time\. We separately visualize the symmetric metric contribution, the order\-sensitive directed contribution, and their sum\. These matrices provide a qualitative view of the structure induced by the two components of Eq\.[4](https://arxiv.org/html/2610.08960#S3.E4)\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/pusht_trajectory_distance_matrix.png)\(a\)PushT
![Refer to caption](https://arxiv.org/html/2610.08960v1/tworoom_trajectory_distance_matrix.png)\(b\)TwoRoom

Figure 16:Pairwise decomposition of the learned temporal distance on PushT and TwoRoom\.Rows index source states and columns index goal states by normalized episode time\. From left to right, each row shows the symmetric metric contribution, the order\-sensitive directed contribution, and their total temporal distance\. Each task uses its own distance scale\.![Refer to caption](https://arxiv.org/html/2610.08960v1/reacher_trajectory_distance_matrix.png)\(a\)Reacher
![Refer to caption](https://arxiv.org/html/2610.08960v1/cube_trajectory_distance_matrix.png)\(b\)OGBench\-Cube

Figure 17:Pairwise decomposition of the learned temporal distance on Reacher and OGBench\-Cube\.Rows index source states and columns index goal states by normalized episode time\. From left to right, each row shows the symmetric metric contribution, the order\-sensitive directed contribution, and their total temporal distance\. Each task uses its own distance scale\.The decomposition varies substantially across tasks\. On PushT, the pairwise structure of the total temporal distance is dominated by the order\-sensitive component, consistent with the strong forward–reverse asymmetry observed in the main\-text diagnostics\. TwoRoom, Reacher, and Cube exhibit different relative contributions and pairwise structures\. The directed component therefore provides task\-dependent order\-sensitive structure rather than imposing a common asymmetric pattern across environments\. The symmetric component remains invariant to source–goal reversal, while asymmetry in the total distance arises from the directed component by construction\.

#### Additional OGBench tasks\.

We provide the same quasimetric decomposition for the six additional visual OGBench tasks\. As in the four LeWM tasks, each matrix evaluates pairwise temporal distance along a recorded trajectory and separates the symmetric metric contribution from the order\-sensitive directed contribution\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/pointmaze_trajectory_distance_matrix.png)\(a\)PointMaze
![Refer to caption](https://arxiv.org/html/2610.08960v1/antmaze_trajectory_distance_matrix.png)\(b\)AntMaze\-Medium

Figure 18:Pairwise decomposition of the learned temporal distance on PointMaze and AntMaze\-Medium\.Rows index source states and columns index goal states by normalized episode time\. From left to right, each row shows the symmetric metric contribution, the order\-sensitive directed contribution, and their total temporal distance\. Each task uses its own distance scale\.![Refer to caption](https://arxiv.org/html/2610.08960v1/cube_double_trajectory_distance_matrix.png)\(a\)Cube\-Double
![Refer to caption](https://arxiv.org/html/2610.08960v1/scene_trajectory_distance_matrix.png)\(b\)Scene

Figure 19:Pairwise decomposition of the learned temporal distance on Cube\-Double and Scene\.The visualization follows the same construction as Figure[18](https://arxiv.org/html/2610.08960#A6.F18)\. Symmetric and directed contributions are evaluated separately before forming the total temporal distance\.![Refer to caption](https://arxiv.org/html/2610.08960v1/humanoidmaze_trajectory_distance_matrix.png)\(a\)HumanoidMaze\-Medium
![Refer to caption](https://arxiv.org/html/2610.08960v1/puzzle_trajectory_distance_matrix.png)\(b\)Puzzle\-3x3

Figure 20:Pairwise decomposition of the learned temporal distance on HumanoidMaze\-Medium and Puzzle\-3x3\.Rows and columns index source and goal states by normalized episode time\. Each task uses its own distance scale\.Across the additional tasks, the relative contributions of the symmetric and directed components vary substantially\. PointMaze, Cube\-Double, and Scene exhibit visible structure in both components, while AntMaze\-Medium, HumanoidMaze\-Medium, and Puzzle\-3x3 place comparatively greater structure in the order\-sensitive component\. The total distance inherits these task\-specific decompositions rather than following a common pairwise pattern across environments\. Together with the four LeWM tasks, these visualizations show that the metric–residual construction provides a common structural form while allowing the learned balance between symmetric and directed geometry to adapt to the control problem\.

### F\.4Goal\-Relative Directional Landscapes

We further visualize the directional temporal cost around a fixed recorded goal\. For each task, we evaluate the temporal distance from every sampled state to the reference goal and in the reverse direction from the goal to each state\. We also visualize the directional differenced⁡\(eg,e\)−d⁡\(e,eg\)d\(e\_\{g\},e\)\-d\(e,e\_\{g\}\)\. Distances are computed in the full learned representation\. PCA is used only to position states for visualization\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/pusht_reference_goal_landscape.png)\(a\)PushT
![Refer to caption](https://arxiv.org/html/2610.08960v1/tworoom_reference_goal_landscape.png)\(b\)TwoRoom

Figure 21:Goal\-relative directional landscapes on PushT and TwoRoom\.For a fixed recorded goal, the panels show the goal observation, forward temporal distance from each state to the goal, reverse temporal distance from the goal to each state, and the reverse–forward difference\. PCA is used only for visualization; temporal distances are evaluated in the full learned representation\.![Refer to caption](https://arxiv.org/html/2610.08960v1/reacher_reference_goal_landscape.png)\(a\)Reacher
![Refer to caption](https://arxiv.org/html/2610.08960v1/cube_reference_goal_landscape.png)\(b\)OGBench\-Cube

Figure 22:Goal\-relative directional landscapes on Reacher and OGBench\-Cube\.The visualization follows the same construction as Figure[21](https://arxiv.org/html/2610.08960#A6.F21)\. The forward and reverse landscapes are evaluated for the same fixed goal, and the final panel shows their difference\.The directional landscapes vary substantially across tasks\. PushT exhibits large spatial differences between forward and reverse costs, consistent with its pronounced forward–reverse asymmetry in the main\-text diagnostics\. TwoRoom, Reacher, and Cube show substantially weaker or more localized differences\. These visualizations therefore provide a spatial view of the task\-dependent directional structure captured by the quasimetric\. They complement the aggregate forward–reverse statistics reported in Section[4\.3](https://arxiv.org/html/2610.08960#S4.SS3)\.

#### Additional OGBench tasks\.

We provide the same fixed\-goal directional visualizations for the six additional OGBench tasks\. For each task, we evaluate forward temporal distance from sampled states to a fixed recorded goal, reverse temporal distance from the goal to the same states, and their difference\. All distances are computed in the full learned representation\. PCA is used only to position states for visualization\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/pointmaze_reference_goal_landscape.png)\(a\)PointMaze
![Refer to caption](https://arxiv.org/html/2610.08960v1/antmaze_reference_goal_landscape.png)\(b\)AntMaze\-Medium

Figure 23:Goal\-relative directional landscapes on PointMaze and AntMaze\-Medium\.For a fixed recorded goal, the panels show the goal observation, forward temporal distance from each state to the goal, reverse temporal distance from the goal to each state, and the reverse–forward difference\. PCA is used only for visualization\.![Refer to caption](https://arxiv.org/html/2610.08960v1/cube_double_reference_goal_landscape.png)\(a\)Cube\-Double
![Refer to caption](https://arxiv.org/html/2610.08960v1/scene_reference_goal_landscape.png)\(b\)Scene

Figure 24:Goal\-relative directional landscapes on Cube\-Double and Scene\.Forward and reverse temporal distances are evaluated for the same fixed goal, and the final panel shows their difference\. Distances are computed in the full learned representation\.![Refer to caption](https://arxiv.org/html/2610.08960v1/humanoidmaze_reference_goal_landscape.png)\(a\)HumanoidMaze\-Medium
![Refer to caption](https://arxiv.org/html/2610.08960v1/puzzle_reference_goal_landscape.png)\(b\)Puzzle\-3x3

Figure 25:Goal\-relative directional landscapes on HumanoidMaze\-Medium and Puzzle\-3x3\.The visualization follows the same construction as Figure[23](https://arxiv.org/html/2610.08960#A6.F23)\.The additional tasks further illustrate that directional structure is not uniform across control problems\. Some environments exhibit pronounced spatial differences between forward and reverse temporal costs, while others remain closer to symmetric around the selected reference goal\. The spatial pattern of the reverse–forward difference also varies substantially across tasks\. Together with the aggregate diagnostics, these visualizations show that the directed quasimetric provides capacity for order\-sensitive state–goal evaluation without imposing a fixed directional pattern across environments\.

## Appendix GTraining Dynamics

We visualize the evolution of the learned DTRC representation during training on the four LeWM tasks\. For each task, we show the control representationee, symmetric coordinatesS⁡\(e\)S\(e\), and directed potentialsA⁡\(e\)A\(e\)at 0, 1,000, 10,000, 20,000, 60,000, and 200,000 training updates\. Points are colored by normalized offline episode time\. These visualizations are intended to illustrate the development of representation structure during optimization rather than to provide quantitative convergence criteria\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/pusht_evolution.png)\(a\)PushT
![Refer to caption](https://arxiv.org/html/2610.08960v1/tworoom_evolution.png)\(b\)TwoRoom

Figure 26:Evolution of the learned representation on PushT and TwoRoom\.Columns correspond to training checkpoints and rows show the control representationee, symmetric coordinatesS⁡\(e\)S\(e\), and directed potentialsA⁡\(e\)A\(e\)\. Colors indicate normalized offline episode time\.![Refer to caption](https://arxiv.org/html/2610.08960v1/reacher_evolution.png)\(a\)Reacher
![Refer to caption](https://arxiv.org/html/2610.08960v1/cube_evolution.png)\(b\)OGBench\-Cube

Figure 27:Evolution of the learned representation on Reacher and OGBench\-Cube\.Columns correspond to training checkpoints and rows show the control representationee, symmetric coordinatesS⁡\(e\)S\(e\), and directed potentialsA⁡\(e\)A\(e\)\. Colors indicate normalized offline episode time\.The learned structures emerge progressively during optimization and differ substantially across tasks\. On TwoRoom, the control representation develops a curved geometry while the directed potentials expand into a distinct order\-sensitive structure\. Reacher develops an approximately cyclic control representation and directed\-potential geometry\. PushT and Cube exhibit different non\-cyclic organizations, consistent with the task\-dependent final representations reported in Section[F\.1](https://arxiv.org/html/2610.08960#A6.SS1)\. These visualizations indicate that DTRC does not converge toward a single common projected geometry across environments\.

#### Additional OGBench tasks\.

We further visualize representation evolution on the six additional OGBench tasks using the same checkpoints and PCA protocol\. These results show that task\-specific geometric structure also emerges progressively beyond the four LeWM tasks\.

![Refer to caption](https://arxiv.org/html/2610.08960v1/pointmaze_evolution.png)\(a\)PointMaze
![Refer to caption](https://arxiv.org/html/2610.08960v1/antmaze_evolution.png)\(b\)AntMaze\-Medium

Figure 28:Evolution of the learned representation on PointMaze and AntMaze\-Medium\.Columns correspond to training checkpoints and rows show the control representationee, symmetric coordinatesS⁡\(e\)S\(e\), and directed potentialsA⁡\(e\)A\(e\)\. Colors indicate normalized offline episode time\. PCA is fitted independently at each checkpoint, so orientation and scale are not directly comparable across columns\.![Refer to caption](https://arxiv.org/html/2610.08960v1/cube_double_evolution.png)\(a\)Cube\-Double
![Refer to caption](https://arxiv.org/html/2610.08960v1/scene_evolution.png)\(b\)Scene

Figure 29:Evolution of the learned representation on Cube\-Double and Scene\.The visualization follows the same checkpoint and PCA protocol as Figure[28](https://arxiv.org/html/2610.08960#A7.F28)\.![Refer to caption](https://arxiv.org/html/2610.08960v1/humanoidmaze_evolution.png)\(a\)HumanoidMaze\-Medium
![Refer to caption](https://arxiv.org/html/2610.08960v1/puzzle_evolution.png)\(b\)Puzzle\-3x3

Figure 30:Evolution of the learned representation on HumanoidMaze\-Medium and Puzzle\-3x3\.Columns correspond to training checkpoints and rows showee,S⁡\(e\)S\(e\), andA⁡\(e\)A\(e\)\. PCA is fitted independently at each checkpoint\.Across the additional tasks, representation structure develops progressively during optimization while remaining strongly task\-dependent\. PointMaze and AntMaze\-Medium develop distinct nonlinear organizations in the control representation and directed potentials\. Cube\-Double and Puzzle\-3x3 form different structured manifolds, while Scene and HumanoidMaze\-Medium exhibit more diffuse projected organizations\. The symmetric and directed heads also evolve differently across tasks\. These visualizations are therefore consistent with the final task\-dependent geometries reported in the representation analyses, rather than with convergence toward a single common projected form\.