SPT-Flow：一次积分，连续时间世界模型中的多种未来

# One Integration, Many Futuresin Continuous-Time World Models
Source: [https://arxiv.org/html/2610.41273](https://arxiv.org/html/2610.41273)
## SPT\-Flow: One Integration, Many Futures in Continuous\-Time World Models

Mingyang Lyu Yinqian Sun Gongce Cao Yuhao Zhang Lu Jia Qian Zhang Bo Wang Pin LyuInstitute of Automation, Chinese Academy of SciencesUniversity of the Chinese Academy of SciencesXi’an Jiaotong UniversityUniversity of Science and Technology Beijing\*Equal contribution\.†Corresponding authors\.

###### Abstract

Continuous\-time world models generate a latent trajectory through a single ODE integration, but adding stochastic conditioning does not ensure that the model uses it\. In our teacher\-forced baseline, posterior means vary by only about1%1\\%of the prior scale, and matched and swapped latents yield indistinguishable reconstruction PSNR at the reported precision\. We study a supervision\-shortcut hypothesis: ground\-truth intermediate states can reduce the need for trajectory\-associated latent information\. We introduceSPT\-Flow, which samples one latent per trajectory and matches a free\-running ODE rollout to demonstration states at multiple readout times\. Re\-anchoring experiments support the proposed mechanism\. On LIBERO, with both models trained for3030k optimizer steps,SPT\-Flowimproves best\-of\-10 PSNR over our retrained ODEWorld baseline by2\.02\.0and2\.42\.4dB at the mid\-trajectory and final frames and reduces mean LPIPS by23%23\\%, with comparable per\-sample inference cost\. The rollout control recovers most of the accuracy gain; learned latent conditioning adds diversity and best\-of\-KKgains while reducing mean sample accuracy relative to this control\. The diagnostics support trajectory\-associated latent utilization; semantic mode coverage under matched conditions in the robotic setting remains unverified\.

## 1Introduction

World models support robotic manipulation by predicting future observations conditioned on an initial observation and a goal\([Du et al\., 2023](https://arxiv.org/html/2610.41273#bib.bib18);[Black et al\., 2024](https://arxiv.org/html/2610.41273#bib.bib4);[Hu et al\., 2025](https://arxiv.org/html/2610.41273#bib.bib31)\)\. Many approaches use discrete stochastic\([Ha & Schmidhuber, 2018](https://arxiv.org/html/2610.41273#bib.bib26);[Hafner et al\., 2019](https://arxiv.org/html/2610.41273#bib.bib27);[Hafner et al\., 2025](https://arxiv.org/html/2610.41273#bib.bib28)\)or deterministic\([Hansen et al\., 2024](https://arxiv.org/html/2610.41273#bib.bib29);[Zhou et al\., 2025](https://arxiv.org/html/2610.41273#bib.bib67)\)latent transitions\. ODEWorld\([Liu et al\., 2026](https://arxiv.org/html/2610.41273#bib.bib41)\)instead models latent dynamics as an ordinary differential equation in physical time, generating a trajectory through a single numerical integration with arbitrary\-time readout\.

![Refer to caption](https://arxiv.org/html/2610.41273v1/fig_teaser.png)Figure 1:Predictions conditioned on the same initial frame and instruction\. ODEWorld produces one trajectory;SPT\-Flowsamples three, each through one integration\. Rows showt/T=0\.6t/T=0\.6and1\.01\.0\. The example is selected for diversity; model settings follow Figure[5](https://arxiv.org/html/2610.41273#A7.F5)\.This formulation nevertheless produces only one trajectory for a fixed initial state and goal\. Manipulation tasks can admit multiple valid continuations:*put both pots on the stove*, for example, permits either pot to be moved first\. When the velocity\-field inputs cannot distinguish these alternatives, squared\-error supervision favors their conditional mean, whose integration may match neither continuation\. Appendix[E](https://arxiv.org/html/2610.41273#A5)illustrates this failure in a controlled setting and presents distinct continuations recorded on a physical robot arm\.

Conditional latent variables offer a standard approach to multimodal prediction\([Sohn et al\., 2015](https://arxiv.org/html/2610.41273#bib.bib53)\), but improved accuracy does not guarantee informative latent representations\. In our experiments, adding a trajectory\-level latentϵ\\epsilonimproves best\-of\-KKaccuracy while posterior means vary little across trajectories and swapping inferred latents barely affects reconstruction accuracy \(Section[3\.4](https://arxiv.org/html/2610.41273#S3.SS4)\)\. We term this behavior*decorative latent conditioning*\.

A potential explanation is a supervision shortcut: ground\-truth intermediate states can reveal the demonstrated continuation, allowing the velocity field to predict its target without usingϵ\\epsilon\. At inference, these states are unavailable, yet training has not explicitly required the latent to resolve the remaining ambiguity\.

We address this mismatch withSPT\-Flow, which replaces teacher\-forced velocity supervision with rollout matching\. During training, a latent sampled from the approximate posterior conditions a free rollout fromz0z\_\{0\}, whose predicted states are matched to the demonstration\. Removing intermediate ground\-truth inputs encourages the model to encode and use trajectory\-associated information throughϵ\\epsilon\. At inference, different prior samples generate different trajectories, each through a single integration \(Figure[1](https://arxiv.org/html/2610.41273#S1.F1)\)\. Here, multiple futures denotes sampled trajectories; the LIBERO diagnostics assess latent utilization, while recovery of known continuation modes is tested in the synthetic setting of Appendix[E](https://arxiv.org/html/2610.41273#A5)\.

#### Contributions\.

- •We introduceSPT\-Flow, combining trajectory\-level latent conditioning with rollout matching for stochastic physical\-time prediction\. It preserves the deterministic baseline’s number of velocity\-field evaluations per trajectory and achieves comparable inference latency\.
- •We characterize decorative latent conditioning using posterior specificity and latent swapping\. Under rollout matching, specificity exceeds the predefined threshold in all three seeds, and matched latents outperform swapped latents by1\.051\.05dB\. Controlled re\-anchoring experiments provide evidence for the proposed supervision shortcut\.
- •On LIBERO, at3030k optimizer steps,SPT\-Flowimproves best\-of\-10 PSNR over our retrained ODEWorld baseline by2\.02\.0and2\.42\.4dB, and mean sample PSNR by1\.01\.0and1\.31\.3dB, at the mid\-trajectory and final frames, respectively, while reducing mean LPIPS by23%23\\%\. The rollout control recovers most of the accuracy gain; learned latent conditioning improves best\-of\-KKperformance and diversity at the expense of mean sample accuracy relative to that control\. The prediction advantage persists when both methods receive four times the standard training steps\.

![Refer to caption](https://arxiv.org/html/2610.41273v1/fig_method.png)Figure 2:SPT\-Flowand its training objective\.\(a\)Teacher\-forced velocity supervision \(Equation[2](https://arxiv.org/html/2610.41273#S3.E2)\) evaluates the field at a ground\-truth latentzτ⋆z^\{\\star\}\_\{\\tau\}, which can reveal the demonstrated continuation\.\(b\)Rollout matching \(Equation[4](https://arxiv.org/html/2610.41273#S3.E4)\) samples oneϵ\\epsilon, integrates fromz0z\_\{0\}, and supervises predictions atGGreadout times without intermediate ground\-truth inputs\.\(c\)Architecture: snowflakes indicate frozen modules and dashed paths are used only during training\. At inference,ϵ\\epsilonis sampled frompψ​\(ϵ∣z0,zg\)p\_\{\\psi\}\(\\epsilon\\mid z\_\{0\},z\_\{g\}\)\.

## 2Related work

#### World models\.

World models support manipulation through goal\-conditioned visual prediction\([Du et al\., 2023](https://arxiv.org/html/2610.41273#bib.bib18);[Black et al\., 2024](https://arxiv.org/html/2610.41273#bib.bib4);[Hu et al\., 2025](https://arxiv.org/html/2610.41273#bib.bib31);[Huang et al\., 2025b](https://arxiv.org/html/2610.41273#bib.bib33)\)\. Many approaches use discrete stochastic\([Ha & Schmidhuber, 2018](https://arxiv.org/html/2610.41273#bib.bib26);[Hafner et al\., 2019](https://arxiv.org/html/2610.41273#bib.bib27);[Hafner et al\., 2025](https://arxiv.org/html/2610.41273#bib.bib28)\)or deterministic\([Hansen et al\., 2024](https://arxiv.org/html/2610.41273#bib.bib29);[Zhou et al\., 2025](https://arxiv.org/html/2610.41273#bib.bib67)\)latent transitions\. Continuous\-time alternatives include neural ODEs\([Chen et al\., 2018](https://arxiv.org/html/2610.41273#bib.bib9)\), latent ODEs\([Rubanova et al\., 2019](https://arxiv.org/html/2610.41273#bib.bib49)\), neural SDEs\([Li et al\., 2020](https://arxiv.org/html/2610.41273#bib.bib38)\), and continuous\-time video models\([Park et al\., 2021](https://arxiv.org/html/2610.41273#bib.bib45);[Ye & Bilodeau, 2024](https://arxiv.org/html/2610.41273#bib.bib62)\)\. ODEWorld\([Liu et al\., 2026](https://arxiv.org/html/2610.41273#bib.bib41)\)generates latent trajectories by integration in physical time\.111We distinguish posterior collapse of a variational latent from the representation collapse discussed in ODEWorld\.Recent stochastic world models include object\-centric latent dynamics\([Daniel et al\., 2026](https://arxiv.org/html/2610.41273#bib.bib15)\)and flow matching in pretrained feature spaces\([Porcher et al\., 2026](https://arxiv.org/html/2610.41273#bib.bib46)\)\.SPT\-Flowbuilds on ODEWorld’s representation and single\-integration formulation, introducing trajectory\-level latent conditioning trained through rollout matching\.

#### Latent conditioning and utilization\.

Under standard uniqueness conditions, deterministic ODE dynamics produce one trajectory for a fixed initial state and conditioning\([Dupont et al\., 2019](https://arxiv.org/html/2610.41273#bib.bib19)\)\. Latent conditioning provides one way to represent multiple futures, as explored in neural ODEs\([Norcliffe et al\., 2021](https://arxiv.org/html/2610.41273#bib.bib44);[Cheng et al\., 2024](https://arxiv.org/html/2610.41273#bib.bib11)\)and stochastic video prediction\([Babaeizadeh et al\., 2018](https://arxiv.org/html/2610.41273#bib.bib1);[Lee et al\., 2018](https://arxiv.org/html/2610.41273#bib.bib37);[Denton & Fergus, 2018](https://arxiv.org/html/2610.41273#bib.bib17);[Franceschi et al\., 2020](https://arxiv.org/html/2610.41273#bib.bib21);[Ye et al\., 2019](https://arxiv.org/html/2610.41273#bib.bib63)\)\. The same argument underlies generative action heads, where a sample from a diffusion or flow\-matching policy selects an action chunk instead of regressing to the mean of the demonstrated ones\([Chi et al\., 2023](https://arxiv.org/html/2610.41273#bib.bib12);[Black et al\., 2025](https://arxiv.org/html/2610.41273#bib.bib5);[Hu et al\., 2024](https://arxiv.org/html/2610.41273#bib.bib30);[Zhai et al\., 2025](https://arxiv.org/html/2610.41273#bib.bib64)\), and[Guo & Schwing \(2025\)](https://arxiv.org/html/2610.41273#bib.bib24)attach a latent to a flow\-matching field for the same reason in generative flow time\. However, variational latents may remain uninformative because of optimization dynamics, decoder conditioning, or nonidentifiability\([Bowman et al\., 2016](https://arxiv.org/html/2610.41273#bib.bib6);[Chen et al\., 2017](https://arxiv.org/html/2610.41273#bib.bib10);[Wang et al\., 2021](https://arxiv.org/html/2610.41273#bib.bib60);[Dang et al\., 2024](https://arxiv.org/html/2610.41273#bib.bib14)\); a further reading attributes the neglect to a decoder that sees the ground truth throughout training\([De Fauw et al\., 2019](https://arxiv.org/html/2610.41273#bib.bib16)\), or to data that pair each input with a single recorded future\([Zhou et al\., 2021](https://arxiv.org/html/2610.41273#bib.bib68)\)\. Proposed mitigations include KL scheduling, free bits, auxiliary objectives, and restrictions on decoder inputs\([Sønderby et al\., 2016](https://arxiv.org/html/2610.41273#bib.bib54);[Kingma et al\., 2016](https://arxiv.org/html/2610.41273#bib.bib35);[Goyal et al\., 2017](https://arxiv.org/html/2610.41273#bib.bib23);[Roberts et al\., 2018](https://arxiv.org/html/2610.41273#bib.bib48);[Miladinović et al\., 2022](https://arxiv.org/html/2610.41273#bib.bib43)\)\. Closely related,[Sip et al\. \(2025\)](https://arxiv.org/html/2610.41273#bib.bib52)examine how teacher\-forcing intervals affect the use of stochastic inputs in dynamical VAEs\. We study this interaction in physical\-time world models, distinguishing sampling\-related accuracy gains from trajectory\-associated latent utilization through posterior specificity and latent swapping\.

#### Rollout supervision\.

The mismatch between training on observed states and predicting from generated states is known as exposure bias\([Ranzato et al\., 2016](https://arxiv.org/html/2610.41273#bib.bib47)\)and motivates scheduled sampling\([Bengio et al\., 2015](https://arxiv.org/html/2610.41273#bib.bib2);[Huszár, 2015](https://arxiv.org/html/2610.41273#bib.bib34)\), data aggregation\([Venkatraman et al\., 2015](https://arxiv.org/html/2610.41273#bib.bib58);[Talvitie, 2017](https://arxiv.org/html/2610.41273#bib.bib55)\), adversarial matching\([Lamb et al\., 2016](https://arxiv.org/html/2610.41273#bib.bib36)\), prediction\-dependent transitions in recurrent world models\([Chiappa et al\., 2017](https://arxiv.org/html/2610.41273#bib.bib13)\), and training with self\-generated context\([Chen et al\., 2024](https://arxiv.org/html/2610.41273#bib.bib8);[Huang et al\., 2025a](https://arxiv.org/html/2610.41273#bib.bib32)\)\. Latent overshooting extends variational training to multi\-step predictions\([Hafner et al\., 2019](https://arxiv.org/html/2610.41273#bib.bib27)\), while RDR supervises decoded free\-running trajectories in a deterministic latent world model\([Shah & Shrestha, 2026](https://arxiv.org/html/2610.41273#bib.bib51)\)\. Our focus is the effect of rollout supervision on a trajectory\-level variational latent, beyond improvements in prediction accuracy\. Re\-anchoring experiments vary access to ground\-truth states while holding the rollout posterior fixed\. Additional implementation differences between teacher\-forced and rollout training are documented in Appendix[A](https://arxiv.org/html/2610.41273#A1)\.

## 3SPT\-Flow: a stochastic physical\-time flow

### 3\.1Physical\-time flow under teacher\-forced supervision

Given an initial observationo0o\_\{0\}and instructionℓ\\ell, a frozen image encoder produces tokensst=enc⁡\(ot\)s\_\{t\}=\\mathrm\{enc\}\(o\_\{t\}\), and a decoupling module maps\(s0,st\)\(s\_\{0\},s\_\{t\}\)tozt=dec⁡\(s0,st\)z\_\{t\}=\\mathrm\{dec\}\(s\_\{0\},s\_\{t\}\)\. A frozen instruction encoder provideszgz\_\{g\}, and a frozen decoder reconstructs frames\. ODEWorld’s physical\-time flow \(PT\-Flow\) evolves these latents through

d​zd​τ=vθ​\(z0,z,zg,τ\),z⁡\(0\)=z0,\\frac\{\\mathrm\{d\}z\}\{\\mathrm\{d\}\\tau\}=v\_\{\\theta\}\(z\_\{0\},z,z\_\{g\},\\tau\),\\qquad z\(0\)=z\_\{0\},\(1\)A single numerical integration generates the trajectory, with readouts at arbitrary query times\. We use fixed\-step Runge–Kutta integration at inference \(Appendix[A](https://arxiv.org/html/2610.41273#A1)\)\. Teacher\-forced training evaluates the field at demonstration stateszτ⋆z^\{\\star\}\_\{\\tau\}and matches a target velocityz˙τ⋆\\dot\{z\}^\{\\star\}\_\{\\tau\}:

ℒtf=‖vθ​\(z0,zτ⋆⏟ground truth,zg,τ\)−z˙τ⋆‖2\.\\mathcal\{L\}\_\{\\mathrm\{tf\}\}\\;=\\;\\big\\\|v\_\{\\theta\}\\big\(z\_\{0\},\\underbrace\{z\_\{\\tau\}^\{\\star\}\}\_\{\\text\{ground truth\}\},z\_\{g\},\\tau\\big\)\-\\dot\{z\}\_\{\\tau\}^\{\\star\}\\big\\\|^\{2\}\.\(2\)Appendix[A](https://arxiv.org/html/2610.41273#A1)details target construction and differences from the released model\. Crucially, the field receives ground\-truth states during training but its own predictions during inference\.

For fixed initial conditions and goal, the deterministic ODE produces one trajectory\. Where its inputs do not distinguish alternative continuations, squared\-error supervision favors their conditional mean velocity\([Bishop, 1994](https://arxiv.org/html/2610.41273#bib.bib3);[Mathieu et al\., 2016](https://arxiv.org/html/2610.41273#bib.bib42)\), which can yield a rollout inconsistent with either alternative\.

### 3\.2The trajectory\-level latent and the supervision shortcut

To represent multiple futures, we condition the field onϵ\\epsilon, held fixed throughout each rollout:vθ​\(z0,z,zg,ϵ,τ\)v\_\{\\theta\}\(z\_\{0\},z,z\_\{g\},\\epsilon,\\tau\)\. Under teacher forcing, the posteriorq⁡\(ϵ∣z0,zτ⋆,zg,τ\)q\(\\epsilon\\mid z\_\{0\},z^\{\\star\}\_\{\\tau\},z\_\{g\},\\tau\)infers the latent from a demonstration state; a learned priorp⁡\(ϵ∣z0,zg\)p\(\\epsilon\\mid z\_\{0\},z\_\{g\}\)supplies samples at inference\. A KL penalty regularizes the posterior toward the prior\.

However, the field is evaluated atzτ⋆z^\{\\star\}\_\{\\tau\}\(Figure[2](https://arxiv.org/html/2610.41273#S1.F2)a\), and that state can already identify the demonstrated continuation, allowing the field to predict its velocity without informative latent conditioning\. This possible shortcut reduces the incentive to encode continuation\-specific information inϵ\\epsilon\. It does not guarantee latent neglect, as our low\-dimensional example shows \(Appendix[E](https://arxiv.org/html/2610.41273#A5)\)\. In our main experiments, however, posterior means vary little across trajectories, and swapping inferred latents provides no measurable accuracy change \(Section[4\.3](https://arxiv.org/html/2610.41273#S4.SS3)\)\. We call this*decorative latent conditioning*: sampling improves best\-of\-KKaccuracy despite limited evidence of trajectory\-specific encoding and use\.

### 3\.3Rollout matching

SPT\-Flowinstead supervises freely integrated trajectories\. For readout timesτ1<⋯<τG\\tau\_\{1\}<\\cdots<\\tau\_\{G\}and demonstration statesz⋆1:Gz^\{\\star\}\_\{1:G\}, a permutation\-invariant posterior pools the state–time pairs and produces one sample:

q⁡\(ϵ\|z0,zg,\{\(zi⋆,τi\)\}i=1G\),q\\big\(\\epsilon\\,\\big\|\\,z\_\{0\},z\_\{g\},\\\{\(z^\{\\star\}\_\{i\},\\tau\_\{i\}\)\\\}\_\{i=1\}^\{G\}\\big\),\(3\)Conditioned on this sample, the field integrates fromz0z\_\{0\}without receiving intermediate ground\-truth states\. We match its predictionsz^​\(τi,ϵ\)\\hat\{z\}\(\\tau\_\{i\};\\epsilon\)to the demonstration:

ℒroll=1G∑i=1G∥z^\(τi;ϵ\)−zi⋆∥2\+βKL\[q∥p\],\\mathcal\{L\}\_\{\\mathrm\{roll\}\}\\;=\\;\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\big\\\|\\hat\{z\}\(\\tau\_\{i\};\\epsilon\)\-z^\{\\star\}\_\{i\}\\big\\\|^\{2\}\\;\+\\;\\beta\\,\\mathrm\{KL\}\\\!\\left\[q\\,\\\|\\,p\\right\],\(4\)We useG=4G=4readout times per trajectory, and the KL regularizer uses a per\-dimension free\-bits floor\. Solver settings, hyperparameters, and differences in posterior inputs and KL aggregation between training variants are documented in Appendix[A](https://arxiv.org/html/2610.41273#A1)\.

Conditional onz0z\_\{0\}andzgz\_\{g\},ϵ\\epsilonis the only input through which the demonstrated future enters the rollout \(Figure[2](https://arxiv.org/html/2610.41273#S1.F2)b\)\. Removing intermediate ground\-truth conditioning therefore encourages the posterior to encode information useful for reconstructing that future\. At inference, we sampleϵ\\epsilonfrom the prior, integrate once, and decode\. Training and inference share the frozen modules and velocity field in Figure[2](https://arxiv.org/html/2610.41273#S1.F2)c; the posterior supplies the training latent and the prior supplies the inference latent\. Solver settings differ as detailed in Appendix[A](https://arxiv.org/html/2610.41273#A1)\. Each sampled trajectory uses the same number of velocity\-field evaluations as PT\-Flow, with comparable measured latency \(Section[4\.2](https://arxiv.org/html/2610.41273#S4.SS2)\); generatingKKsamples requiresKKintegrations\.

### 3\.4Posterior specificity and the latent swap

Because best\-of\-KKgains alone do not establish informative latent conditioning, we apply two complementary diagnostics to trained models\. Inspired by the active\-units statistic of[Burda et al\. \(2016\)](https://arxiv.org/html/2610.41273#bib.bib7), posterior specificity measures the across\-trajectory standard deviation of each posterior mean, normalized by the corresponding average prior standard deviation and averaged over dimensions\. This normalization accounts for our learned conditional prior\. Low specificity indicates limited variation in posterior means, but does not exclude information in other posterior statistics\.

Inspired by latent control swapping\([Seitzer et al\., 2024](https://arxiv.org/html/2610.41273#bib.bib50)\), the latent swap compares rollouts conditioned on latents inferred from the target trajectory and an unrelated trajectory, holding the initial state and goal fixed\. Both are scored against the same demonstration\. An advantage for the matched latent supports the use of trajectory\-associated information; equal scores indicate no measurable benefit under this intervention, rather than independence fromϵ\\epsilon\. Such information may include task, scene, or timing differences; this intervention does not isolate continuation choice under matched initial conditions and instructions\. Both diagnostics require one evaluation pass over held\-out trajectories and no retraining\. Section[4\.3](https://arxiv.org/html/2610.41273#S4.SS3)reports thresholds, sample sizes, and results\.

## 4Experiments

### 4\.1Setup

We evaluate on LIBERO\([Liu et al\., 2023](https://arxiv.org/html/2610.41273#bib.bib40)\), using5,0005\{,\}000trajectories across100100tasks, re\-rendered at256×256256\\times 256by replaying stored simulator states\. We allocate4,5004\{,\}500trajectories to training and250250each to fixed development and test sets; the test set is excluded from model development\. Unless otherwise stated, models train for3030k steps, evaluation uses the full test set withK=10K=10conditional\-prior samples per stochastic model, and results report the mean and standard deviation across three seeds\. Seed spreads are normalized bynnrather thann−1n\-1, since they describe the dispersion of three runs rather than estimate a population, and the one development\-split result is labeled where it appears\. We retrain the released ODEWorld architecture using Equation[2](https://arxiv.org/html/2610.41273#S3.E2), since the release provides inference code only; all models share the same frozen encoders and decoder\. Training comparisons match optimizer steps; training NFE and GPU\-hours are not reported, so these comparisons do not establish equal\-compute performance\.

PSNR is evaluated against recorded observations at mid\-trajectory and final frames, reporting both best\-of\-KKand mean sample performance\. Best\-of\-KKfollows the selection principle of variety loss\([Gupta et al\., 2018](https://arxiv.org/html/2610.41273#bib.bib25)\)and may favor dispersed predictions\([Thiede & Brahma, 2019](https://arxiv.org/html/2610.41273#bib.bib56)\), motivating complementary mean reporting\. These frame\-level metrics do not establish whole\-trajectory consistency\. Diversity is the mean pairwise distance between terminal latent states and measures representation\-space variation\. All frames are compared after decoding with the shared frozen decoder, so differences reflect the predicted latent trajectory rather than the decoder\. LPIPS and FVD are reported in Section[4\.4](https://arxiv.org/html/2610.41273#S4.SS4); implementation details and baseline deviations are documented in Appendix[A](https://arxiv.org/html/2610.41273#A1)\.

### 4\.2Accuracy and diversity at comparable inference cost

Table 1:Results on the LIBERO test set \(250250trajectories\), withK=10K=10samples\. PSNR is reported in dB; diversity denotes mean pairwise terminal latent distance\. Both training budgets use three seeds; standard deviations are shown for3030k steps, and means only for6060k steps\. The rollout control is trained withϵ=0\\epsilon=0but retains random conditioning at evaluation, as discussed in the text\. Bold indicates the highest PSNR within each training budget\.At3030k steps,SPT\-Flowimproves best\-of\-10 PSNR over ODEWorld by2\.012\.01and2\.442\.44dB at the mid\-trajectory and final frames, respectively \(Table[1](https://arxiv.org/html/2610.41273#S4.T1)\)\. Rollout matching without learned latent conditioning recovers1\.521\.52and2\.202\.20dB, whileSPT\-Flowexceeds this control by0\.490\.49and0\.240\.24dB\. Mean sample PSNR also exceeds ODEWorld by1\.041\.04and1\.311\.31dB, although it remains0\.170\.17and0\.480\.48dB below the rollout control\. At6060k steps, the best\-of\-10 advantage over ODEWorld persists at1\.131\.13and2\.392\.39dB\. Learned latent conditioning increases terminal diversity to2\.032\.03, compared with1\.461\.46under teacher forcing and0\.510\.51for the rollout control; ODEWorld has none, since a deterministic field integrates to one trajectory per initial state and goal\. The control’s residual diversity arises from evaluation with an untrained prior despite training atϵ=0\\epsilon=0; it is therefore not strictly deterministic\. The reported increments are relative to this control; they do not isolate the latent’s contribution against rollout evaluation atϵ=0\\epsilon=0\.

Figure 3:Best\-of\-KKPSNR as a function of sample count at3030k training steps\. Lines show means across three seeds, and shaded bands indicate the minimum\-to\-maximum seed range\. Rollout matching improves performance atK=1K=1, whileSPT\-Flowexhibits larger gains asKKincreases\. ODEWorld is independent ofKK\.The sampling curves further distinguish the two contributions \(Figure[3](https://arxiv.org/html/2610.41273#S4.F3)\): rollout matching improves accuracy atK=1K=1, by1\.21\.2and1\.81\.8dB at the two frames, while increasingKKfrom11to3030yields gains of1\.301\.30and1\.471\.47dB forSPT\-Flow, versus0\.430\.43and0\.550\.55dB for the control\. At the final frame,SPT\-Flowovertakes the control atK=3K=3and achieves a0\.410\.41dB advantage atK=30K=30\.

Per\-sample inference costs remain comparable: a5050\-frame latent rollout takes151\.1151\.1ms forSPT\-Flowversus149\.9149\.9ms for ODEWorld on the same GPU, an approximately0\.8%0\.8\\%increase, with200200velocity\-field evaluations for both\. The latent is sampled once and held fixed throughout integration\. Best\-of\-KKevaluation nevertheless requiresKKrollouts, with computational cost scaling approximately linearly under sequential execution\.

### 4\.3Latent specificity and utilization

Table 2:Both diagnostics of Section[3\.4](https://arxiv.org/html/2610.41273#S3.SS4)are applied to trained models on held\-out trajectories\. Specificity measures posterior\-mean variation normalized by the prior scale; the threshold0\.250\.25was fixed before evaluation\.*Own*and*swapped*use latents inferred from the target trajectory and another trajectory, respectively\. PSNR is measured at the true temporal midpoint and is comparable within this evaluation protocol\. At3030k steps, PSNR uses250250trajectories across three seeds; specificity ranges use4040trajectories\. The6060k and low\-floor settings use seed00and4040trajectories\.Teacher\-forced training yields posterior specificity of0\.0070\.007–0\.0120\.012, corresponding to posterior\-mean variation of approximately1%1\\%of the prior scale \(Table[2](https://arxiv.org/html/2610.41273#S4.T2)\)\. Latents inferred from the target and unrelated trajectories produce indistinguishable PSNR at the reported precision across all three seeds\. Neither doubling the training budget nor lowering the KL floor changes this pattern in the tested configurations\. In contrast, rollout matching exceeds the predefined specificity threshold of0\.250\.25in every seed, with all3232dimensions above0\.290\.29, whereas no dimension of the teacher\-forced posterior exceeds0\.0220\.022in any seed \(Appendix[G](https://arxiv.org/html/2610.41273#A7)\)\. Using the target trajectory’s inferred latent improves PSNR by1\.051\.05dB over the swapped latent, with a per\-seed range of0\.780\.78to1\.191\.19dB \(Table[4](https://arxiv.org/html/2610.41273#S4.T4)\), providing evidence that trajectory\-associated information contributes to prediction\.

These results support the distinction between decorative and informative latent conditioning introduced in Section[3\.2](https://arxiv.org/html/2610.41273#S3.SS2): best\-of\-KKgains can coexist with weak posterior specificity and no measurable advantage from the trajectory\-matched latent\. However, the diagnostics do not establish that the latent selects among distinct continuations within a task\. The association between task\-level branching and the latent’s accuracy gain is weak, with Spearman correlations of0\.0050\.005and0\.1150\.115at the mid\-trajectory and final frames \(Appendix[G](https://arxiv.org/html/2610.41273#A7)\)\. We therefore interpret the results as evidence of trajectory\-associated latent utilization, without attributing that information specifically to within\-task continuation selection\.

### 4\.4Perceptual and distributional quality

Table 3:Perceptual and distributional evaluation across three seeds \(mean±\\pmstandard deviation; lower is better\)\. LPIPS\([Zhang et al\., 2018](https://arxiv.org/html/2610.41273#bib.bib66)\)is evaluated at the mid\-trajectory frame on5050held\-out trajectories withK=10K=10\. FVD\([Unterthiner et al\., 2018](https://arxiv.org/html/2610.41273#bib.bib57)\)uses200200recorded reference clips and5050generated clips for ODEWorld or100100for each other model\. The rollout control has no learned trajectory latent but retains random conditioning during evaluation\.Perceptual evaluation shows a pattern consistent with the PSNR results \(Table[3](https://arxiv.org/html/2610.41273#S4.T3)\)\. The rollout control reduces mean LPIPS from0\.1810\.181to0\.1290\.129, a29%29\\%improvement over ODEWorld\. Adding learned latent conditioning further reduces best\-of\-KKLPIPS from0\.1220\.122to0\.1160\.116, a36%36\\%reduction against ODEWorld, while increasing mean LPIPS by0\.0100\.010\. With both rollout\-based models evaluated using100100generated clips,SPT\-Flowhas a lower estimated FVD,187187versus210210, approximately11%11\\%\. These results indicate that rollout matching improves average perceptual accuracy, while learned latent conditioning improves best\-of\-KKperformance\. The FVD estimates are exploratory distributional measurements and do not establish semantic mode coverage\. Comparisons with ODEWorld’s FVD remain qualified by the unequal generated sample counts,5050clips against100100for each rollout\-trained model, against the same200200recorded references\.

### 4\.5Training budget and supervision ablations

Table 4:Ground\-truth states are withdrawn from the training rollout in three stages\. Rollout variants share the same posterior; teacher forcing follows Equation[2](https://arxiv.org/html/2610.41273#S3.E2)\. Specificity is reported as the mean \[minimum, maximum\] across three seeds\. PSNR and diversity are seed means\. The swap gap is own\-minus\-swapped mid\-trajectory PSNR in dB, measured on seed00except forSPT\-Flow, which reports the three\-seed mean and range\.#### Training budget\.

At6060k steps, teacher\-forced latent conditioning improves mid\-trajectory best\-of\-10 PSNR over ODEWorld by only0\.070\.07dB, whereasSPT\-Flowretains a1\.131\.13dB advantage \(Table[1](https://arxiv.org/html/2610.41273#S4.T1)\)\. Its diversity decreases from2\.032\.03to1\.181\.18, and the rollout control accounts for a larger share of the improvement:0\.940\.94of1\.131\.13dB, compared with1\.521\.52of2\.012\.01dB at3030k steps\. At120120k steps, the mid\-trajectory advantage over ODEWorld remains0\.970\.97dB in one seed and1\.531\.53dB in the other, with final\-frame advantages of2\.072\.07and2\.372\.37dB \(Appendix[D](https://arxiv.org/html/2610.41273#A4)\)\. A similar decline occurs on the ten\-task subset, with gains of1\.361\.36,1\.151\.15and0\.840\.84dB at1010k,2020k and3030k steps \(Appendix[B](https://arxiv.org/html/2610.41273#A2)\)\. This subset entails greater data reuse but also changes task coverage, so it does not isolate the effect of repeated exposure\.

#### Ground\-truth re\-anchoring\.

Reducing access to intermediate ground\-truth states increases mean posterior specificity from0\.2360\.236to0\.2980\.298and0\.3320\.332across the rollout variants \(Table[4](https://arxiv.org/html/2610.41273#S4.T4)\)\. Only fully free\-running training exceeds the0\.250\.25threshold in all three seeds\. Nevertheless, re\-anchoring at every readout already yields a1\.071\.07dB swap gap, indicating latent utilization even below this threshold\. The swap gap does not increase monotonically with specificity, and the most frequently anchored rollout achieves the highest PSNR\. These results distinguish posterior variation, latent utilization, and prediction accuracy\. Because the rollout variants share their posterior configuration, their specificity trend supports the proposed supervision\-shortcut interpretation; comparisons with teacher forcing additionally involve the implementation differences documented in Appendix[A](https://arxiv.org/html/2610.41273#A1)\.

Table 5:Each variant changes one setting relative to the reference:β=3×10−3\\beta=3\\times 10^\{\-3\}, latent dimension3232, and88Euler steps during training\. Variants use one seed; the reference reports three\-seed means and the specificity range\. PSNR is in dB\.Table 6:TemperatureTTscales the prior standard deviation at inference, without retraining;T=1T=1is the default andSPT\-Flowis trained for3030k steps\. PSNR is reported as mean±\\pmstandard deviation across three seeds, and diversity as the seed mean\.
#### Hyperparameter sensitivity\.

Mid\-trajectory best\-of\-10 PSNR ranges from19\.5319\.53to19\.8519\.85dB across the tested configurations, while posterior specificity varies more substantially \(Table[5](https://arxiv.org/html/2610.41273#S4.T5)\)\. Increasingβ\\betafrom10−310^\{\-3\}to10−210^\{\-2\}reduces specificity from0\.440\.44to0\.200\.20, with the swap gap decreasing to0\.380\.38dB at the higher weight\. A1616\-dimensional latent achieves the highest final\-frame best\-of\-10 PSNR, whereas1616Euler steps yield the highest mid\-trajectory mean PSNR and lowest diversity\. These comparisons are exploratory because each non\-reference setting uses one seed\. Reducing the effective batch size from1616to88also produces near\-zero specificity \(0\.0040\.004–0\.0050\.005\) and zero diversity in both tested seeds, indicating sensitivity to the optimization configuration\.

#### Sampling temperature\.

Scaling the prior standard deviation trades mean accuracy against best\-of\-10 accuracy and diversity without retraining \(Table[6](https://arxiv.org/html/2610.41273#S4.T6)\)\. Lower temperatures improve mean PSNR but reduce both best\-of\-10 PSNR and diversity, which falls from2\.522\.52atT=1\.3T=1\.3to1\.081\.08atT=0\.5T=0\.5\. AtT=0\.5T=0\.5, final\-frame best\-of\-10 PSNR falls slightly below the rollout control \(18\.1218\.12versus18\.1818\.18\), whereasT=0\.7T=0\.7already exceeds it \(18\.2518\.25\)\. None of the tested temperatures matches the control’s final\-frame mean PSNR of17\.7717\.77\. Relative toT=1T=1, increasing the temperature to1\.31\.3improves best\-of\-10 PSNR by0\.090\.09dB at both frames while reducing mean PSNR by0\.080\.08dB\. The default therefore represents an intermediate setting within the evaluated range\.

#### KL\-based mitigations\.

The tested KL adjustments do not recover trajectory\-associated latent utilization under teacher forcing \(Appendix[F](https://arxiv.org/html/2610.41273#A6)\)\. KL annealing, a threefold reduction in KL weight, and a fourfold increase in the free\-bits floor yield specificity values of0\.0100\.010–0\.0270\.027and swap gaps no larger than0\.0010\.001dB\. These results indicate that the failure is not resolved by the tested regularization settings and are consistent with the supervision\-shortcut hypothesis\.

## 5Conclusion

We introducedSPT\-Flow, a stochastic extension of ODEWorld that combines a trajectory\-level latent variable with rollout matching while retaining a single physical\-time integration per sampled trajectory\. On LIBERO, at3030k steps, it improves best\-of\-10 PSNR over our retrained ODEWorld baseline by2\.02\.0and2\.42\.4dB at the mid\-trajectory and final frames, respectively, and reduces mean LPIPS by23%23\\%, with comparable per\-sample inference latency\. The prediction advantage persists when both models receive a fourfold increase in training steps\. The rollout control recovers most of the accuracy improvement; learned latent conditioning increases diversity and best\-of\-KKperformance, with lower mean sample accuracy relative to that control\.

Our results also distinguish predictive accuracy from latent utilization\. Under teacher\-forced supervision, best\-of\-KKaccuracy can improve despite low posterior specificity and no measurable PSNR benefit from trajectory\-matched latents\. Rollout matching exceeds the predefined specificity threshold across all three seeds, and the corresponding inferred latent outperforms a swapped latent by1\.051\.05dB\. Re\-anchoring experiments further associate reduced access to intermediate ground\-truth states with higher posterior specificity\.

#### Scope and limitations\.

These findings motivate reporting latent diagnostics alongside prediction metrics\. They support trajectory\-associated information use, while leaving same\-condition semantic mode coverage unresolved\. Evaluation covers held\-out trajectories from tasks represented in training; generalization to unseen tasks and downstream control benefits remain untested\. Latent utilization also depends on the optimization setting, including effective batch size \(Section[4\.5](https://arxiv.org/html/2610.41273#S4.SS5)\)\. The training comparisons match steps, and the rollout control retains random evaluation inputs, limiting claims about compute efficiency and the isolated contribution of learned conditioning\.

## Reproducibility Statement

Section[4\.1](https://arxiv.org/html/2610.41273#S4.SS1)describes the data splits, evaluation protocols, and reporting across random seeds\. Appendix[A](https://arxiv.org/html/2610.41273#A1)documents data preparation, model architectures, hyperparameters, training procedures, numerical solvers, and implementation differences between comparison models\. Diagnostic definitions and thresholds appear in Sections[3\.4](https://arxiv.org/html/2610.41273#S3.SS4)and[4\.3](https://arxiv.org/html/2610.41273#S4.SS3), with additional analyses in Appendix[G](https://arxiv.org/html/2610.41273#A7)\. Hyperparameter and sampling\-temperature sensitivity analyses are reported in Section[4\.5](https://arxiv.org/html/2610.41273#S4.SS5)and Appendix[C](https://arxiv.org/html/2610.41273#A3); extended\-budget results appear in Appendix[D](https://arxiv.org/html/2610.41273#A4)\. Development\-set results are explicitly identified in Appendix[B](https://arxiv.org/html/2610.41273#A2)\. Training and evaluation code is included in the supplementary material and will be publicly released\.

## AI Use Statement

Generative AI tools assisted with language editing and with reviewing the clarity and scope of manuscript claims\. The authors reviewed all AI\-assisted revisions and take full responsibility for the final manuscript, including its claims, citations, and experimental reporting\.

## References

- Babaeizadeh et al\. \(2018\)Mohammad Babaeizadeh, Chelsea Finn, Dumitru Erhan, Roy H\. Campbell, and Sergey Levine\.Stochastic variational video prediction\.In*International Conference on Learning Representations \(ICLR\)*, 2018\.
- Bengio et al\. \(2015\)Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer\.Scheduled sampling for sequence prediction with recurrent neural networks\.In*Advances in Neural Information Processing Systems \(NIPS\)*, 2015\.
- Bishop \(1994\)Christopher M\. Bishop\.Mixture density networks\.Technical Report NCRG/94/004, Aston University, 1994\.
- Black et al\. \(2024\)Kevin Black, Mitsuhiko Nakamoto, Pranav Atreya, Homer Walke, Chelsea Finn, Aviral Kumar, and Sergey Levine\.Zero\-shot robotic manipulation with pre\-trained image\-editing diffusion models\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.
- Black et al\. \(2025\)Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, et al\.π0\\pi\_\{0\}: A vision\-language\-action flow model for general robot control\.In*Robotics: Science and Systems \(RSS\)*, 2025\.
- Bowman et al\. \(2016\)Samuel R\. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M\. Dai, Rafal Jozefowicz, and Samy Bengio\.Generating sentences from a continuous space\.In*Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning \(CoNLL\)*, 2016\.
- Burda et al\. \(2016\)Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov\.Importance weighted autoencoders\.In*International Conference on Learning Representations \(ICLR\)*, 2016\.Active\-units statistic: Sec\. 5\.2, threshold justified in App\. C\.
- Chen et al\. \(2024\)Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann\.Diffusion forcing: Next\-token prediction meets full\-sequence diffusion\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2024\.
- Chen et al\. \(2018\)Ricky T\. Q\. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud\.Neural ordinary differential equations\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2018\.
- Chen et al\. \(2017\)Xi Chen, Diederik P\. Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, and Pieter Abbeel\.Variational lossy autoencoder\.In*International Conference on Learning Representations \(ICLR\)*, 2017\.
- Cheng et al\. \(2024\)Sheng Cheng, Deqian Kong, Jianwen Xie, Kookjin Lee, Ying Nian Wu, and Yezhou Yang\.Latent space energy\-based neural ODEs\.*arXiv preprint arXiv:2409\.03845*, 2024\.
- Chi et al\. \(2023\)Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song\.Diffusion Policy: Visuomotor policy learning via action diffusion\.In*Robotics: Science and Systems \(RSS\)*, 2023\.
- Chiappa et al\. \(2017\)Silvia Chiappa, Sébastien Racanière, Daan Wierstra, and Shakir Mohamed\.Recurrent environment simulators\.In*International Conference on Learning Representations \(ICLR\)*, 2017\.
- Dang et al\. \(2024\)Hien Dang, Tho Tran, Tan Nguyen, and Nhat Ho\.Beyond vanilla variational autoencoders: Detecting posterior collapse in conditional and hierarchical variational autoencoders\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.
- Daniel et al\. \(2026\)Tal Daniel, Carl Qi, Dan Haramati, Amir Zadeh, Chuan Li, Aviv Tamar, Deepak Pathak, and David Held\.Latent particle world models: Self\-supervised object\-centric stochastic dynamics modeling\.In*International Conference on Learning Representations \(ICLR\)*, 2026\.
- De Fauw et al\. \(2019\)Jeffrey De Fauw, Sander Dieleman, and Karen Simonyan\.Hierarchical autoregressive image models with auxiliary decoders\.*arXiv preprint arXiv:1903\.04933*, 2019\.
- Denton & Fergus \(2018\)Emily Denton and Rob Fergus\.Stochastic video generation with a learned prior\.In*Proceedings of the 35th International Conference on Machine Learning \(ICML\)*, volume 80 of*PMLR*, pp\. 1174–1183, 2018\.
- Du et al\. \(2023\)Yilun Du, Mengjiao Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Joshua B\. Tenenbaum, Dale Schuurmans, and Pieter Abbeel\.Learning universal policies via text\-guided video generation\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2023\.
- Dupont et al\. \(2019\)Emilien Dupont, Arnaud Doucet, and Yee Whye Teh\.Augmented neural ODEs\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2019\.
- Efron & Tibshirani \(1993\)Bradley Efron and Robert J\. Tibshirani\.*An Introduction to the Bootstrap*\.Chapman & Hall/CRC, 1993\.
- Franceschi et al\. \(2020\)Jean\-Yves Franceschi, Edouard Delasalles, Mickaël Chen, Sylvain Lamprier, and Patrick Gallinari\.Stochastic latent residual video prediction\.In*Proceedings of the 37th International Conference on Machine Learning \(ICML\)*, volume 119 of*PMLR*, pp\. 89–102, 2020\.
- Ge et al\. \(2024\)Songwei Ge, Aniruddha Mahapatra, Gaurav Parmar, Jun\-Yan Zhu, and Jia\-Bin Huang\.On the content bias in Fréchet video distance\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, 2024\.
- Goyal et al\. \(2017\)Anirudh Goyal, Alessandro Sordoni, Marc\-Alexandre Côté, Nan Rosemary Ke, and Yoshua Bengio\.Z\-forcing: Training stochastic recurrent networks\.In*Advances in Neural Information Processing Systems \(NIPS\)*, 2017\.
- Guo & Schwing \(2025\)Pengsheng Guo and Alexander G\. Schwing\.Variational rectified flow matching\.In*International Conference on Machine Learning \(ICML\)*, 2025\.
- Gupta et al\. \(2018\)Agrim Gupta, Justin Johnson, Li Fei\-Fei, Silvio Savarese, and Alexandre Alahi\.Social GAN: Socially acceptable trajectories with generative adversarial networks\.In*Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\)*, 2018\.
- Ha & Schmidhuber \(2018\)David Ha and Jürgen Schmidhuber\.World models\.*arXiv preprint arXiv:1803\.10122*, 2018\.
- Hafner et al\. \(2019\)Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson\.Learning latent dynamics for planning from pixels\.In*Proceedings of the 36th International Conference on Machine Learning \(ICML\)*, volume 97 of*PMLR*, pp\. 2555–2565, 2019\.
- Hafner et al\. \(2025\)Danijar Hafner, Jurgis Pašukonis, Jimmy Ba, and Timothy Lillicrap\.Mastering diverse control tasks through world models\.*Nature*, 640:647–653, 2025\.doi:10\.1038/s41586\-025\-08744\-2\.
- Hansen et al\. \(2024\)Nicklas Hansen, Hao Su, and Xiaolong Wang\.TD\-MPC2: Scalable, robust world models for continuous control\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.
- Hu et al\. \(2024\)Xixi Hu, Bo Liu, Xingchao Liu, and Qiang Liu\.AdaFlow: Imitation learning with variance\-adaptive flow\-based policies\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2024\.
- Hu et al\. \(2025\)Yucheng Hu, Yanjiang Guo, Pengchao Wang, Xiaoyu Chen, Yen\-Jen Wang, Jianke Zhang, Koushil Sreenath, Chaochao Lu, and Jianyu Chen\.Video prediction policy: A generalist robot policy with predictive visual representations\.In*International Conference on Machine Learning \(ICML\)*, 2025\.
- Huang et al\. \(2025a\)Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman\.Self forcing: Bridging the train\-test gap in autoregressive video diffusion\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2025a\.
- Huang et al\. \(2025b\)Yuhang Huang, Jiazhao Zhang, Shilong Zou, Xinwang Liu, Ruizhen Hu, and Kai Xu\.LaDi\-WM: A latent diffusion\-based world model for predictive manipulation\.In*Conference on Robot Learning \(CoRL\)*, 2025b\.
- Huszár \(2015\)Ferenc Huszár\.How \(not\) to train your generative model: Scheduled sampling, likelihood, adversary?*arXiv preprint arXiv:1511\.05101*, 2015\.
- Kingma et al\. \(2016\)Diederik P\. Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling\.Improving variational inference with inverse autoregressive flow\.In*Advances in Neural Information Processing Systems \(NIPS\)*, 2016\.Free bits: Appendix C\.8, Eq\. \(15\)\.
- Lamb et al\. \(2016\)Alex Lamb, Anirudh Goyal, Ying Zhang, Saizheng Zhang, Aaron Courville, and Yoshua Bengio\.Professor forcing: A new algorithm for training recurrent networks\.In*Advances in Neural Information Processing Systems \(NIPS\)*, 2016\.
- Lee et al\. \(2018\)Alex X\. Lee, Richard Zhang, Frederik Ebert, Pieter Abbeel, Chelsea Finn, and Sergey Levine\.Stochastic adversarial video prediction\.*arXiv preprint arXiv:1804\.01523*, 2018\.
- Li et al\. \(2020\)Xuechen Li, Ting\-Kam Leonard Wong, Ricky T\. Q\. Chen, and David Duvenaud\.Scalable gradients for stochastic differential equations\.In*International Conference on Artificial Intelligence and Statistics \(AISTATS\)*, 2020\.
- Lipman et al\. \(2023\)Yaron Lipman, Ricky T\. Q\. Chen, Heli Ben\-Hamu, Maximilian Nickel, and Matt Le\.Flow matching for generative modeling\.In*International Conference on Learning Representations \(ICLR\)*, 2023\.
- Liu et al\. \(2023\)Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone\.LIBERO: Benchmarking knowledge transfer in lifelong robot learning\.In*Advances in Neural Information Processing Systems \(NeurIPS\), Datasets and Benchmarks Track*, 2023\.
- Liu et al\. \(2026\)Dongxiu Liu, Haoyi Niu, Peng Cheng, Yuan Gao, Xirui Kang, Sangli Teng, Koushil Sreenath, and Xianyuan Zhan\.ODEWorld: A continuous predictive architecture via physical\-time flow\.*arXiv preprint arXiv:2607\.27924*, 2026\.
- Mathieu et al\. \(2016\)Michael Mathieu, Camille Couprie, and Yann LeCun\.Deep multi\-scale video prediction beyond mean square error\.In*International Conference on Learning Representations \(ICLR\)*, 2016\.
- Miladinović et al\. \(2022\)Đorđe Miladinović, Kumar Shridhar, Kushal Jain, Max B\. Paulus, Joachim M\. Buhmann, Mrinmaya Sachan, and Carl Allen\.Learning to drop out: An adversarial approach to training sequence VAEs\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2022\.
- Norcliffe et al\. \(2021\)Alexander Norcliffe, Cristian Bodnar, Ben Day, Jacob Moss, and Pietro Liò\.Neural ODE processes\.In*International Conference on Learning Representations \(ICLR\)*, 2021\.
- Park et al\. \(2021\)Sunghyun Park, Kangyeol Kim, Junsoo Lee, Jaegul Choo, Joonseok Lee, Sookyung Kim, and Edward Choi\.Vid\-ODE: Continuous\-time video generation with neural ordinary differential equation\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, volume 35, pp\. 2412–2422, 2021\.
- Porcher et al\. \(2026\)Francois Porcher, Nicolas Carion, Karteek Alahari, and Shizhe Chen\.Flow matching in feature space for stochastic world modeling\.*arXiv preprint arXiv:2606\.29059*, 2026\.
- Ranzato et al\. \(2016\)Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba\.Sequence level training with recurrent neural networks\.In*International Conference on Learning Representations \(ICLR\)*, 2016\.
- Roberts et al\. \(2018\)Adam Roberts, Jesse Engel, Colin Raffel, Curtis Hawthorne, and Douglas Eck\.A hierarchical latent vector model for learning long\-term structure in music\.In*Proceedings of the 35th International Conference on Machine Learning \(ICML\)*, volume 80 of*PMLR*, 2018\.
- Rubanova et al\. \(2019\)Yulia Rubanova, Ricky T\. Q\. Chen, and David Duvenaud\.Latent ordinary differential equations for irregularly\-sampled time series\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2019\.
- Seitzer et al\. \(2024\)Maximilian Seitzer, Sjoerd van Steenkiste, Thomas Kipf, Klaus Greff, and Mehdi S\. M\. Sajjadi\.DyST: Towards dynamic neural scene representations on real\-world videos\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.
- Shah & Shrestha \(2026\)Rishi Shah and Rishav Shrestha\.Rollout\-decoded reconstruction for long\-horizon prediction in latent world models\.*arXiv preprint arXiv:2608\.25017*, 2026\.Concurrent work\.
- Sip et al\. \(2025\)Viktor Sip, Martin Breyton, Spase Petkoski, and Viktor Jirsa\.Double projection for reconstructing dynamical systems: between stochastic and deterministic regimes\.*arXiv preprint arXiv:2510\.01089*, 2025\.
- Sohn et al\. \(2015\)Kihyuk Sohn, Honglak Lee, and Xinchen Yan\.Learning structured output representation using deep conditional generative models\.In*Advances in Neural Information Processing Systems \(NIPS\)*, pp\. 3483–3491, 2015\.
- Sønderby et al\. \(2016\)Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, and Ole Winther\.Ladder variational autoencoders\.In*Advances in Neural Information Processing Systems \(NIPS\)*, 2016\.
- Talvitie \(2017\)Erik Talvitie\.Self\-correcting models for model\-based reinforcement learning\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, 2017\.
- Thiede & Brahma \(2019\)Luca Anthony Thiede and Pratik Prabhanjan Brahma\.Analyzing the variety loss in the context of probabilistic trajectory prediction\.In*Proceedings of the IEEE/CVF International Conference on Computer Vision \(ICCV\)*, 2019\.
- Unterthiner et al\. \(2018\)Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly\.Towards accurate generative models of video: A new metric and challenges\.*arXiv preprint arXiv:1812\.01717*, 2018\.
- Venkatraman et al\. \(2015\)Arun Venkatraman, Martial Hebert, and J\. Andrew Bagnell\.Improving multi\-step prediction of learned time series models\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, 2015\.
- Villani \(2009\)Cédric Villani\.*Optimal Transport: Old and New*, volume 338 of*Grundlehren der mathematischen Wissenschaften*\.Springer, 2009\.
- Wang et al\. \(2021\)Yixin Wang, David M\. Blei, and John P\. Cunningham\.Posterior collapse and latent variable non\-identifiability\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2021\.
- Wang et al\. \(2004\)Zhou Wang, Alan C\. Bovik, Hamid R\. Sheikh, and Eero P\. Simoncelli\.Image quality assessment: From error visibility to structural similarity\.*IEEE Transactions on Image Processing*, 13\(4\):600–612, 2004\.
- Ye & Bilodeau \(2024\)Xi Ye and Guillaume\-Alexandre Bilodeau\.STDiff: Spatio\-temporal diffusion for continuous stochastic video prediction\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, 2024\.
- Ye et al\. \(2019\)Yufei Ye, Maneesh Singh, Abhinav Gupta, and Shubham Tulsiani\.Compositional video prediction\.In*Proceedings of the IEEE/CVF International Conference on Computer Vision \(ICCV\)*, 2019\.
- Zhai et al\. \(2025\)Xuanran Zhai, Qianyou Zhao, Qiaojun Yu, and Ce Hao\.VFP: Variational flow\-matching policy for multi\-modal robot manipulation\.*arXiv preprint arXiv:2508\.01622*, 2025\.
- Zhang et al\. \(2026\)Jianing Zhang, Kefan Jiang, Bohan Dai, Yiwen Lu, Alan Yuille, Yilun Du, and Kaiming Chen\.World\-in\-world: World models in a closed\-loop world\.In*International Conference on Learning Representations \(ICLR\)*, 2026\.Oral\.
- Zhang et al\. \(2018\)Richard Zhang, Phillip Isola, Alexei A\. Efros, Eli Shechtman, and Oliver Wang\.The unreasonable effectiveness of deep features as a perceptual metric\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, 2018\.
- Zhou et al\. \(2025\)Gaoyue Zhou, Hengkai Pan, Yann LeCun, and Lerrel Pinto\.DINO\-WM: World models on pre\-trained visual features enable zero\-shot planning\.In*International Conference on Machine Learning \(ICML\)*, 2025\.
- Zhou et al\. \(2021\)Yizhou Zhou, Chong Luo, Xiaoyan Sun, Zheng\-Jun Zha, and Wenjun Zeng\.VAE2: Preventing posterior collapse of variational video predictions in the wild\.*arXiv preprint arXiv:2101\.12050*, 2021\.

## Appendix AImplementation details

#### Encoders and latent\.

The image encoder is a frozen DINOv2\-with\-registers backbone at224×224224\\times 224; the language encoder is frozen\. The decoupling module is a four\-layer cross\-attention block producing a single768768\-dimensional latent token\. The velocity field is a three\-layer MLP of width40964096with FiLM conditioning on a learned sinusoidal embedding ofτ\\tau\. The posterior and prior are two\-layer MLPs of width10241024and512512producing a3232\-dimensional Gaussian\.

#### Conditioning\.

We condition on the instruction rather than on a goal image\. Supplying the final frame determines most of the continuation from the conditioning alone, which leaves little for a trajectory\-level latent to represent; the instruction leaves the choice of continuation open, which is the setting the diagnostics of Section[3\.4](https://arxiv.org/html/2610.41273#S3.SS4)are about\.

#### Optimization\.

AdamW, learning rate10−410^\{\-4\}with500500warm\-up steps and cosine decay to zero over the declared step budget, weight decay10−410^\{\-4\}, gradient clipping at1\.01\.0, batch of four trajectories with four sampled times each, distributed over four A100 or A800 GPUs \(effective batch1616; Section[4\.5](https://arxiv.org/html/2610.41273#S4.SS5)shows why this matters\)\. The first10001000steps train the reconstruction term only\. Rollout matching integrates with eight Euler steps between consecutive sampled times; inference uses a fixed\-step fourth\-order Runge–Kutta solver with200200evaluations for a5050\-frame trajectory\.

Because the learning rate anneals over the declared budget, a model taken partway through a6060k\-step run is not comparable to the final model of a3030k\-step run\. Every comparison in the main text is between runs at their own final step; the sweep of Appendix[B](https://arxiv.org/html/2610.41273#A2)compares the same runs at matched intermediate steps, which is a comparison across arms at each step and not across steps\.

#### Differences between the teacher\-forced and rollout arms\.

The two arms share the encoder, decoder, latent dimension, KL weight and free\-bits floor, but they are not identical in every other respect, and we list the differences here so that the comparison can be read for what it is\. In the teacher\-forced arm the posterior reads\(z0,zτ⋆,zg,τ\)\(z\_\{0\},z^\{\\star\}\_\{\\tau\},z\_\{g\},\\tau\)at the single sampled time and a freshϵ\\epsilonis drawn for each sampled time; in the rollout arm it reads allG=4G=4readout latents through a permutation\-invariant pooling and oneϵ\\epsilonis shared by the four readouts\. The free\-bits floor is applied per dimension in both arms, to the batch\-averaged KL in the teacher\-forced arm and per trajectory before averaging in the rollout arm\. We did not run a teacher\-forced arm with the pooled posterior, so the main\-text attribution to the supervision rests on the re\-anchoring series of Table[4](https://arxiv.org/html/2610.41273#S4.T4), which keeps the rollout posterior fixed\. Our retraining of the base objective also lets gradients flow through the Jacobian\-vector\-product target, whereas[Liu et al\. \(2026\)](https://arxiv.org/html/2610.41273#bib.bib41)describe a stop\-gradient; the released code ships inference only, so we could not run the original training objective as is\.

#### KL term\.

β=3×10−3\\beta=3\\times 10^\{\-3\}with a per\-dimension floor of0\.050\.05applied before summation\. The floor is required \(Section[4\.5](https://arxiv.org/html/2610.41273#S4.SS5)\); the value was selected on the development half\.

#### Dataset\.

LIBERO demonstrations are re\-rendered at256×256256\\times 256by restoring each stored simulator state and rendering offscreen, rather than by replaying actions, so the rendered trajectory matches the recorded one exactly\. This yields5,0005\{,\}000trajectories over100100tasks\. The held\-out set contains every tenth demonstration of each task\.

#### Evaluation\.

PSNR, LPIPS\([Zhang et al\., 2018](https://arxiv.org/html/2610.41273#bib.bib66)\)and SSIM\([Wang et al\., 2004](https://arxiv.org/html/2610.41273#bib.bib61)\)are computed on decoded frames at a mid\-trajectory index and at the final index\. Best\-of\-KKand the mean overKKsamples follow the protocol of stochastic video prediction\([Babaeizadeh et al\., 2018](https://arxiv.org/html/2610.41273#bib.bib1);[Denton & Fergus, 2018](https://arxiv.org/html/2610.41273#bib.bib17)\), and diversity is the mean pairwise distance between samples as in[Lee et al\. \(2018\)](https://arxiv.org/html/2610.41273#bib.bib37), taken here between terminal latents\. FVD\([Unterthiner et al\., 2018](https://arxiv.org/html/2610.41273#bib.bib57)\)uses a standard I3D feature extractor on1616\-frame clips and is biased toward per\-frame quality over temporal realism\([Ge et al\., 2024](https://arxiv.org/html/2610.41273#bib.bib22)\), so we read it as one number of three rather than a summary\. SSIM moves with LPIPS and is omitted from Table[3](https://arxiv.org/html/2610.41273#S4.T3)for space: over the same three seed pairings, ODEWorld0\.7430\.743, rollout training without the latent0\.7890\.789andSPT\-Flow0\.7920\.792\(best ofKK\) at the mid\-trajectory frame, and0\.7140\.714,0\.7740\.774and0\.7810\.781at the final frame\. Perceptual and distributional metrics use at least5050trajectories: on ten trajectories we measured differences that disappeared at fifty, in both directions, so smaller samples are not reported\.

#### Per\-step baseline\.

An untuned per\-step flow\-matching world model\([Lipman et al\., 2023](https://arxiv.org/html/2610.41273#bib.bib39);[Porcher et al\., 2026](https://arxiv.org/html/2610.41273#bib.bib46)\)trained on the same encoder and data reaches13\.513\.5dB at the mid\-trajectory frame with500500network evaluations, and doubling its solver steps changes this by only0\.130\.13dB; further tuning might narrow the gap, and we do not report it in the main tables\.

## Appendix BCross\-scale check on ten tasks

Table 7:A1010\-task subset \(450450training trajectories\): the same runs at matched intermediate steps, one seed, mid\-trajectory best\-of\-10 on that subset’s4040\-trajectory development split \(not the test split used elsewhere\)\.On a1010\-task subset \(Table[7](https://arxiv.org/html/2610.41273#A2.T7)\)SPT\-Flowleads at every step, by1\.361\.36,1\.151\.15and0\.840\.84dB\. The margin shrinks as training continues, in the same direction as the budget result\. These two settings differ by an order of magnitude in passes over the data at equal step counts, which is consistent with the erosion being governed by data repetition rather than by step count, though we have not isolated the two\.

## Appendix CHyperparameter sensitivity of the fixed method

Table[5](https://arxiv.org/html/2610.41273#S4.T5)in the main text changes one knob at a time from the reported configuration, and Section[4\.5](https://arxiv.org/html/2610.41273#S4.SS5)discusses it\. The swap gap of theβ=10−2\\beta=10^\{\-2\}model is0\.380\.38dB against1\.051\.05for the reference\.

The sampling\-temperature sweep is Table[6](https://arxiv.org/html/2610.41273#S4.T6)in Section[4\.5](https://arxiv.org/html/2610.41273#S4.SS5)\.

## Appendix DFour times the training steps

Table 8:Seed 0 of each arm trained for120120k steps on a different accelerator \(RTX 5090; other tables use A100s, and the reference model reproduces its Table[1](https://arxiv.org/html/2610.41273#S4.T1)numbers on this node to four decimals\),250250\-trajectory test split,K=10K=10\. A second seed is reported in the text\. Models taken partway through share the120120k cosine schedule and are not comparable with the endpoints of Table[1](https://arxiv.org/html/2610.41273#S4.T1)\.At the120120k endpoint \(Table[8](https://arxiv.org/html/2610.41273#A4.T8)\) the gap is0\.970\.97dB at the mid\-trajectory frame and2\.072\.07dB at the final frame, and the deterministic model’s own curve has flattened \(19\.0019\.00at9090k,19\.0719\.07at120120k\)\. The latent stays trajectory\-specific at this budget \(specificity0\.310\.31, all3232dimensions above the threshold; swap gap0\.740\.74dB against1\.051\.05at3030k\), and the best\-of\-KKslope fromK=1K=1to3030is0\.750\.75dB against1\.301\.30at3030k, so the longer run retains latent utilization while showing a smaller gain from sampling\. Its diversity falls from1\.561\.56to0\.680\.68over the run, continuing the2\.032\.03to1\.181\.18drop between the3030k and6060k rows of Table[1](https://arxiv.org/html/2610.41273#S4.T1)\. This comparison does not include a120120k\-step rollout control and therefore does not isolate the learned latent’s incremental benefit\.

A second seed reproduces the endpoint on Ampere hardware: trained on an A100 \(SPT\-Flow\) and an A800 \(ODEWorld\), with the reference model reproducing to four decimals on both nodes, the two models reach20\.4720\.47and18\.9418\.94dB at the mid\-trajectory frame and19\.3719\.37and17\.0017\.00dB at the final frame, a gap of1\.531\.53and2\.372\.37dB against0\.970\.97and2\.072\.07for seed 0\. The latent remains trajectory\-specific at this budget in both seeds \(specificity0\.3650\.365and0\.3070\.307, all3232dimensions above the threshold\)\.

## Appendix EMode averaging in a controlled setting

A field trained with first\-order supervision on two\-mode data integrates to a path between the modes\. Trajectories in this task start at a common point and reach a common endpoint along an upper or a lower arc with equal probability\. The deterministic field produces neither arc\. Adding unstructured Gaussian noise to the field broadens the paths without recovering the two modes in this experiment\. A latent\-conditioned field recovers both arcs\. Wasserstein\-1 distance\([Villani, 2009](https://arxiv.org/html/2610.41273#bib.bib59)\)between predicted and demonstrated mid\-trajectory positions, over three seeds, is0\.3500\.350for the deterministic field — exactly the arc amplitude, meaning it predicts the midline —0\.1430\.143with additive Gaussian noise, and0\.0310\.031and0\.0400\.040for a latent\-conditioned field trained with teacher forcing and with rollout matching respectively\. Both latent variants recover the modes here: in this low\-dimensional setting, where the posterior reads the whole trajectory, even the teacher\-forced latent gets used\. Latent neglect is therefore not inevitable under teacher forcing\. This positive control verifies mode recovery where modes are known, but does not isolate the effects of scale, posterior inputs, and optimization that differ from the LIBERO setting\.

![Refer to caption](https://arxiv.org/html/2610.41273v1/fig_real_strip.png)Figure 4:Four teleoperated demonstrations of one instruction \(*open a drawer and put a snack in the bowl*\) on a real arm, one per row; fixed camera except the wrist\-camera grasp column\. All start with both drawers closed and diverge in which drawer is opened and which snack is taken\.t/Tt/Tis normalized time\. Recorded in separate sessions; an illustration of the premise, not an evaluation\.The diagnostics of Section[4\.3](https://arxiv.org/html/2610.41273#S4.SS3)read as expected where the latent is known to be used\. On400400fresh trajectories per seed, one of the four latent dimensions carries the mode in every model, with a per\-dimension specificity of0\.890\.89–0\.980\.98against the learned conditional prior \(mean over dimensions0\.31±0\.020\.31\\pm 0\.02teacher\-forced and0\.34±0\.040\.34\\pm 0\.04rollout\-trained; the mean is diluted by unused dimensions, so the per\-dimension value is the one to compare\)\. The latent swap separates cleanly: the mid\-trajectory error of a free rollout is0\.0160\.016and0\.0100\.010with the trajectory’s own latent,0\.380\.38and0\.370\.37with the latent of a random other trajectory, which lies in the opposite mode about half the time \(0\.700\.70, the full arc separation, when it does\), and0\.370\.37and0\.330\.33with a prior draw\. Both diagnostics detect mode\-associated information in this positive control\. Their low readings for the teacher\-forced LIBERO model remain specific to that model and evaluation setting\. At scale the per\-dimension reading agrees with the mean: no dimension of the teacher\-forced posterior exceeds0\.0220\.022in any seed, and under rollout training all3232dimensions exceed0\.250\.25in every seed\.

## Appendix FClassical mitigations in detail

Annealing the KL weight from zero over the first55k steps leaves the teacher\-forced posterior at specificity0\.0140\.014and0\.0120\.012across two seeds, against0\.0070\.007–0\.0120\.012without annealing and0\.310\.31–0\.380\.38for rollout training\. The free\-bits floor is load\-bearing in the other direction: lowering it from0\.050\.05to0\.0020\.002or to zero collapses the KL to0\.0140\.014and10−410^\{\-4\}, and the low\-floor row of Table[2](https://arxiv.org/html/2610.41273#S4.T2)shows the resulting specificity\. The tested KL schedules and floors do not recover trajectory\-associated latent utilization in these teacher\-forced configurations\.

The additional KL settings in Table[9](https://arxiv.org/html/2610.41273#A6.T9)yield the same qualitative pattern\. Lowering the KL weight by a factor of three or raising the free\-bits floor fourfold moves mid\-trajectory accuracy by at most0\.200\.20dB from the reference configuration and leaves specificity between0\.0100\.010and0\.0270\.027, with the latent of a trajectory’s own future scoring within0\.0010\.001dB of a shuffled one in all three configurations\. These results are consistent with the supervision\-shortcut hypothesis, but do not isolate it from the other implementation differences in Appendix[A](https://arxiv.org/html/2610.41273#A1)\.

Table 9:Teacher\-forced training with one KL knob changed; two GPUs with twice the per\-GPU batch \(effective batch1616\) on the node of Appendix[D](https://arxiv.org/html/2610.41273#A4),3030k steps, seed 0,250250\-trajectory test split,K=10K=10\. The reference row is retrained on the same node, so rows are comparable with each other but not with Table[1](https://arxiv.org/html/2610.41273#S4.T1)\.
## Appendix GDiagnostics

#### The rollout control’s residual noise\.

The control is trained atϵ=0\\epsilon=0but evaluated by the same code as the full model, which drawsϵ\\epsilonfrom a prior the control never trained\. Residual sensitivity to this input produces its diversity of0\.510\.51in Table[1](https://arxiv.org/html/2610.41273#S4.T1)and allows best\-of\-KKgains\. This control does not measure strictly deterministic rollout performance, and its sample mean cannot be substituted for an evaluation atϵ=0\\epsilon=0\. The0\.490\.49dB mid\-trajectory increment in Section[4\.2](https://arxiv.org/html/2610.41273#S4.SS2)compares best\-of\-1010against best\-of\-1010under the reported protocol\. Per seed, a paired bootstrap\([Efron & Tibshirani, 1993](https://arxiv.org/html/2610.41273#bib.bib20)\)over the250250test trajectories gives increments of\+0\.25\+0\.25,\+0\.57\+0\.57and\+0\.63\+0\.63dB, each with a95%95\\%interval excluding zero; at the final frame one seed’s interval includes zero \(\+0\.05\+0\.05,\[−0\.10,\+0\.21\]\[\-0\.10,\+0\.21\]\)\. These intervals describe trajectory resampling within each fitted seed; they do not establish robustness across unseen tasks or additional training seeds\.

#### Detail behind Table[2](https://arxiv.org/html/2610.41273#S4.T2)\.

The per\-dimension results agree with the aggregate diagnostic: teacher\-forced posterior means vary little, whereas under rollout training every one of the3232dimensions exceeds0\.290\.29in every seed\. Thus, the latter result does not rest on a few active dimensions\. On the ten\-task subset of Appendix[B](https://arxiv.org/html/2610.41273#A2), the rollout posterior reaches specificity0\.470\.47; this result alone does not identify the information represented by the latent\.

#### Exploratory association between task dispersion and prediction gain\.

We measured per\-task branching as the mean pairwise distance between demonstrations’ proprioceptive states at mid\-trajectory, resampled to a common time grid, over1212demonstrations for each of the100100tasks, and correlated it with the full model’s gain over the reported rollout control per test trajectory\. Over the250250\-trajectory test split the Spearman correlation is0\.0050\.005\(p=0\.94p=0\.94\) at the mid\-trajectory frame and0\.1150\.115\(p=0\.07p=0\.07\) at the final frame\. On a6060\-trajectory subset the same statistic read0\.220\.22, which is why we report the larger sample\. These pooled estimates are exploratory: trajectories share task\-level dispersion values, and the reported analysis does not establish task\-level inferential uncertainty\. The small correlations neither establish zero association nor distinguish semantic continuation selection from other trajectory\-associated variation\. More generally, prediction quality and downstream utility require separate evaluation\([Zhang et al\., 2026](https://arxiv.org/html/2610.41273#bib.bib65)\)\.

Posterior specificity isstdn⁡\[μq​\(xn\)\]/mean⁡\[σp\]\\operatorname\{std\}\_\{n\}\[\\mu\_\{q\}\(x\_\{n\}\)\]/\\operatorname\{mean\}\[\\sigma\_\{p\}\], whereμq\\mu\_\{q\}is the posterior mean for held\-out trajectorynnandσp\\sigma\_\{p\}the prior standard deviation, averaged over latent dimensions\. Both diagnostics also record whether two distinct latents change the rollout at all, which distinguishes an uninformative posterior from a latent that is not wired into the field\.

#### Qualitative rollouts\.

Figure[5](https://arxiv.org/html/2610.41273#A7.F5)shows one test trajectory decoded from every arm of Table[1](https://arxiv.org/html/2610.41273#S4.T1)\. The three teacher\-forced samples are visually identical, which is the decorative latent of Section[4\.3](https://arxiv.org/html/2610.41273#S4.SS3)seen directly; the threeSPT\-Flowsamples differ from one another in the arm’s path and follow the recorded continuation more closely than the deterministic model\. The trajectory was chosen as one of the highest\-diversity test trajectories forSPT\-Flow, so the spread shown is toward the upper end of what the model produces\. Animated versions of four such trajectories are in the supplementary material\.

![Refer to caption](https://arxiv.org/html/2610.41273v1/fig_qual.png)Figure 5:One test trajectory \(*put both the cream cheese box and the butter in the basket*\), six of5050frames, decoded through the shared frozen decoder\. Rows: recorded continuation, ODEWorld, three samples from the teacher\-forced latent model, three samples fromSPT\-Flow; all seed\-00models at3030k steps from Table[1](https://arxiv.org/html/2610.41273#S4.T1)\.
#### The recorded continuations separate in the latent once the branch is taken\.

We asked whether the frozen encoder’s latent separates the four continuations of Figure[4](https://arxiv.org/html/2610.41273#A5.F4)over time, encoding6060time\-normalized frames per episode from the fixed camera and subtracting each episode’s first\-frame latent\. Within one recording session, a leave\-one\-episode\-out nearest\-centroid classifier cannot tell the two drawers apart reliably from the first frame \(0\.540\.54, near chance\) and then identifies the drawer with accuracy0\.920\.92fromt/T=0\.05t/T=0\.05and1\.001\.00fromt/T=0\.15t/T=0\.15\. This classifier therefore distinguishes the recorded drawer choices more accurately after branching; its first\-frame result does not establish that the representation contains no predictive information\. Each continuation was recorded as a block: the same classifier also identifies the snack within a drawer \(0\.710\.71–1\.001\.00\) and the session within a continuation \(0\.610\.61–1\.001\.00\), and the centroid distance between two sessions of the same continuation \(2\.22\.2–3\.73\.7\) matches the distance between drawers within one session \(3\.23\.2\), so session and operator style travel with the choice\. Interleaving the orders within one session would separate them\.

#### Time resolution\.

In the earlier128128\-pixel configuration we read the trajectory out at0\.5×0\.5\\times,1×1\\times,2×2\\timesand4×4\\timesthe training resolution; PSNR was18\.118\.1,18\.0518\.05,18\.0518\.05and18\.0518\.05dB at the mid\-trajectory frame and16\.7716\.77at the final frame in all four cases\. This checks prediction consistency at the evaluated frames across readout resolutions; it does not measure the cost of decoding additional frames\.