jepa

Yann LeCun and Randall Balestriero mathematically prove the Platonic Representation Hypothesis and show that JEPA training with scene-related positive views achieves image-text alignment with 42× less data, with a 109M-parameter text JEPA outperforming LLMs up to 72B on cross-modal benchmarks.

The paper turns the Platonic Representation Hypothesis into a theorem, proving that JEPAs trained separately on multiple modalities converge to shared representations up to a single rotation, and shows that next-token prediction is essentially a JEPA with 'bad views' — making it far less efficient for cross-modal alignment (good views need 42x fewer samples).

论文提出 AC-JEPA（Anchored Cross-Modal JEPA），一种可与 SFT 兼容的即插即用模块，通过在双向跨模态预测掩码中间表示来提升视觉到符号序列的翻译能力。在光学化学结构识别（OCSR）任务上，AC-JEPA 在 WildMol Simple 上达到 76.07% 准确率（基线 SFT 为 67.40%），并在使用含噪声伪标签数据自训练时提升至 80.23%，而直接 SFT 反而性能下降。

This NeurReps workshop extended abstract evaluates whether pretrained LeWorldModel preserves local action geometry, measuring action sensitivity via Gram matrices across perturbation scales on Reacher and OGBench Cube, finding significant directional discrepancies at small scales and recommending scale-specific evaluation.

The paper proposes 'predictive closure,' a set-level characterization of how JEPA selects features to encode, showing that feature retention depends on relationships within feature sets rather than individual predictability, with experiments demonstrating that unpredictable features can be preserved when they aid prediction of others.

This paper proposes a post-hoc latent projection method for latent world model planning that prunes noisy latent directions by fitting the state dynamics matrix and restricting the ℓ2 planning cost to the least-uncertain subspace, reporting up to +54.7 points success-rate gains on 2D navigation tasks like TwoRoom and PointMaze.

Slot-JEPA 将 JEPA 的稠密表示分解为可复用的语义槽与空间掩码，提供面向对象的预测接口，并结合 Transition Memory Predictor 与非平衡最优传输提升视觉理解迁移和目标条件规划，在七项视觉基准与 REALM-JEPA 机器人规划任务上取得改进。

提出JEPA-Prune框架，通过联合嵌入引导的后验神经元剪枝，在仅剪枝1.6%神经元的情况下将PII推断准确率降低最多7.4倍，同时基本保持模型效用。

A perspective paper proposing a four-stage research agenda (Ground, Structure, Verify, Act) for interpreting JEPA-based latent world models, arguing interpretability should shift from decoding representations to understanding how latent variables interact, evolve, and causally drive prediction.

KITE 提出一种知识注入的时间序列编码器，将工程师的领域先验（语义先验）通过可切换的协变量与注意力偏置机制注入自监督事件预测器，在稀有事件预测上显著优于无先验基线，并支持跨域迁移。

SPERA 是一种采用 JEPA 潜空间预测架构的 EEG 基础模型，引入 Legendre 多项式空间先验与分解式时空注意力，在约 8 万小时、29,048 名受试者的 EEG 数据上预训练，并在九项下游任务上取得最高平均平衡准确率。

RSI-Anything 提出一个基于 JEPA-Anything 的领域通用递归自我改进框架，利用预测性世界模型指导行动与学习数据选择，在代码修复、GUI 任务、实验室规划和反应优化等多个领域均取得显著性能提升。

LeWAM 提出一种端到端的 JEPA 控制框架，通过联合训练下一隐状态预测与动作流匹配目标，并对隐状态施加各向异性高斯约束，使单一模型同时支持直接策略执行与高效规划，在接触丰富任务和长程任务上将成功率从 28.6% 提升至 89.7% 和从 10.9% 提升至 46.2%，规划速度提升 32.3 倍。

RoboJEPA 基于 JEPA 架构，在涵盖 12 种机器人具身的大型真实机器人数据集上训练，首次为多具身机器人世界模型建立了 scaling law：其 latent rollout 的“想象力误差”随计算量呈二阶幂律变化，并可作为真实机器人规划性能的可靠代理。该模型参数量达 8B，是迄今最大的 JEPA 预测模型，支持零样本部署为长时程规划的机器人智能体，并开源全部检查点与训练/部署代码。

DSReg提出通过依赖稀疏性正则化（Structural Diversity条件），在无需重建、解码器或标签的情况下可证明地恢复单个世界隐变量，建立了首个完全可识别的JEPA表示学习方法。

JEPA-Anything introduces a domain-agnostic world modeling framework based on orthogonal predictive factorization (OPF) that works across vision, biology, clinical trajectories, control, molecular dynamics, physical fields, and weather, outperforming matched JEPA baselines on all 10 dynamics tasks and supporting experimentally validated biological intervention nomination.

The paper develops an early-training stability theory for Joint-Embedding Predictive Architectures that unifies empirical collapse-avoidance heuristics via a drive/decay stability ratio, predicts a phase boundary confirmed across 800+ Tabular-JEPA configurations, and introduces a ResidualPred transformer predictor that improves effective rank and accuracy on tabular and I-JEPA image benchmarks.

H-JEPА introduces an action-conditioned joint-embedding predictive architecture that separates a wide perceptual code from an inherited control state governed by dissipative port-Hamiltonian dynamics, using port-inverse consistency as a parameter-free reweighting of prediction error, and matches or beats baselines like Delta-JEPA on pixel-based control benchmarks.

JEPA 有一个未被讨论的严重盲点：它严重依赖稳定、缓慢变化的特征。只要在每个 episode 中加入一个固定不变的无关 patch，规划就会崩溃；若让该 patch 不可预测，性能反而几乎恢复正常。

JEPA-TTT adapts the latent dynamics predictor of a pretrained JEPA world model during test time using self-supervised dense replay, improving planning under dynamics shifts with 83% lower latent prediction error and 153% better planning performance over the frozen model across four continuous-control environments.