JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts

Hugging Face Daily Papers Papers
test-time-training world-models jepa reinforcement-learning planning self-supervised-learning

Summary

JEPA-TTT adapts the latent dynamics predictor of a pretrained JEPA world model during test time using self-supervised dense replay, improving planning under dynamics shifts with 83% lower latent prediction error and 153% better planning performance over the frozen model across four continuous-control environments.

World models enable agents to plan by predicting future states of the environment, but their predictions can become unreliable when test-time dynamics differ from those seen during training. We present JEPA-TTT, which adapts the latent dynamics predictor of a pretrained action-conditioned Joint-Embedding Predictive Architecture world model throughout test time. Self-supervised updates accumulate across episodes, while the visual encoder and reward head remain fixed, preserving the pretrained representation and task objective. Planning requires neither a goal image nor online environment reward. JEPA-TTT uses dense replay, which forms prediction windows at every temporal offset, retains them in a growing buffer, and samples minibatches from that buffer for predictor updates. Across eight dynamics shifts in four continuous-control environments, JEPA-TTT improves planning on every shift. After 500 test-time episodes, it reduces autoregressive latent prediction error by 83% on average and improves planning performance by 153% over the frozen JEPA world model. These results show that persistent self-supervised test-time training can adapt a pretrained latent world model under changed dynamics.
Original Article
View Cached Full Text

Cached at: 10/06/26, 12:51 AM

Paper page - JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts

Source: https://huggingface.co/papers/2610.00722

Abstract

Worldmodelsenableagentstoplanbypredictingfuturestatesoftheenvironment,buttheirpredictionscanbecomeunreliablewhentest-timedynamicsdifferfromthoseseenduringtraining.WepresentJEPA-TTT,whichadaptsthelatentdynamicspredictorofapretrainedaction-conditionedJoint-EmbeddingPredictiveArchitectureworldmodelthroughouttesttime.Self-supervisedupdatesaccumulateacrossepisodes,whilethevisualencoderandrewardheadremainfixed,preservingthepretrainedrepresentationandtaskobjective.Planningrequiresneitheragoalimagenoronlineenvironmentreward.JEPA-TTTusesdensereplay,whichformspredictionwindowsateverytemporaloffset,retainstheminagrowingbuffer,andsamplesminibatchesfromthatbufferforpredictorupdates.Acrosseightdynamicsshiftsinfourcontinuous-controlenvironments,JEPA-TTTimprovesplanningoneveryshift.After500test-timeepisodes,itreducesautoregressivelatentpredictionerrorby83%onaverageandimprovesplanningperformanceby153%overthefrozenJEPAworldmodel.Theseresultsshowthatpersistentself-supervisedtest-timetrainingcanadaptapretrainedlatentworldmodelunderchangeddynamics.

View arXiv pageView PDFProject pageAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.00722 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.00722 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.00722 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

D-JEPA: A Decision-Aligned Latent World Model

Hugging Face Daily Papers

D-JEPA is a decision-aligned latent world model that learns decision-relevant relations among candidate futures to improve action selection in tasks like manipulation and autonomous driving, demonstrating significant performance gains.