JEPA-Anything: Learning Predictive Models across Different Worlds

Hugging Face Daily Papers Papers
world-modeling predictive-ai joint-embedding multi-domain academic-research machine-learning ai-framework

Summary

JEPA-Anything presents a domain-agnostic framework based on orthogonal predictive factorization for learning predictive models across diverse systems like vision, biology, and control, with demonstrated improvements and experimental validation.

World modeling enables intelligence to anticipate consequences, guide interventions, and learn from interaction. Yet predictive models remain domain-specific: can a common learning principle support world modeling across radically different systems? We introduce JEPA-Anything, a domain-agnostic framework based on orthogonal predictive factorization (OPF). Extending joint-embedding predictive architectures, OPF decomposes latent targets into complementary factors, learns them through dedicated pathways, and recombines them within a shared predictive design. We evaluate JEPA-Anything across seven domains: vision, biology, clinical trajectories, control, molecular dynamics, physical fields, and weather. Experiments span representation learning, intervention prediction, out-of-distribution generalization, and long-horizon dynamics, including 10 matched dynamics tasks, forecasting of over 1,000 clinical events, and 100-step molecular rollouts across four systems. Against matched JEPA baselines, JEPA-Anything improves reported metrics on all 10 dynamics tasks and reduces single-intervention prediction error on Interventional Pong by 34.8%. It achieves the lowest one-step and 100-step molecular errors among compared methods in all four systems. Beyond prediction, a factor-nominated biological intervention receives experimental support in cell co-cultures, patient-derived organoids, tumor fragments, and mice; latent orbital modes recover the Keplerian scaling exponent with a fitted slope of -1.4991. These results support a common factorized predictive principle across heterogeneous worlds, connecting world modeling with intervention and experimentally grounded scientific discovery. Code: https://github.com/Gen-Verse/JEPA-Anything
Original Article
View Cached Full Text

Cached at: 09/18/26, 03:00 AM

Paper page - JEPA-Anything: Learning Predictive Models across Different Worlds

Source: https://huggingface.co/papers/2609.20800 Published on Sep 17

#3 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

Abstract

Worldmodelingenablesintelligencetoanticipateconsequences,guideinterventions,andlearnfrominteraction.Yetpredictivemodelsremaindomain-specific:canacommonlearningprinciplesupportworldmodelingacrossradicallydifferentsystems?WeintroduceJEPA-Anything,adomain-agnosticframeworkbasedonorthogonalpredictivefactorization(OPF).Extendingjoint-embeddingpredictivearchitectures,OPFdecomposeslatenttargetsintocomplementaryfactors,learnsthemthroughdedicatedpathways,andrecombinesthemwithinasharedpredictivedesign.WeevaluateJEPA-Anythingacrosssevendomains:vision,biology,clinicaltrajectories,control,moleculardynamics,physicalfields,andweather.Experimentsspanrepresentationlearning,interventionprediction,out-of-distributiongeneralization,andlong-horizondynamics,including10matcheddynamicstasks,forecastingofover1,000clinicalevents,and100-stepmolecularrolloutsacrossfoursystems.AgainstmatchedJEPAbaselines,JEPA-Anythingimprovesreportedmetricsonall10dynamicstasksandreducessingle-interventionpredictionerroronInterventionalPongby34.8%.Itachievesthelowestone-stepand100-stepmolecularerrorsamongcomparedmethodsinallfoursystems.Beyondprediction,afactor-nominatedbiologicalinterventionreceivesexperimentalsupportincellco-cultures,patient-derivedorganoids,tumorfragments,andmice;latentorbitalmodesrecovertheKeplerianscalingexponentwithafittedslopeof-1.4991.Theseresultssupportacommonfactorizedpredictiveprincipleacrossheterogeneousworlds,connectingworldmodelingwithinterventionandexperimentallygroundedscientificdiscovery.Code:https://github.com/Gen-Verse/JEPA-Anything

View arXiv pageView PDFGitHubAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.20800 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.20800 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.20800 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

DVD-JEPA: an open-source, fully-reproducible JEPA world model [P]

Reddit r/MachineLearning

DVD-JEPA is an open-source, minimal JEPA world model that learns representations from video by predicting future embeddings rather than pixels. It uses a bouncing DVD logo to demonstrate position recovery, dreaming, and anomaly detection, all running in a browser.

Diagnosing JEPA World Models with Action-Conditioned Predictive Consistency

arXiv cs.LG

Introduces Action-Conditioned Predictive Consistency (ACPC), a diagnostic for JEPA world models that measures how clean and perturbed observations diverge under action-conditioned rollouts, with theoretical bounds on prediction error and planner cost. Experiments on visual control tasks validate the diagnostic across models like LeWM and PLDM.