Proving and Improving the Platonic Representation Hypothesis with JEPAs
Summary
The paper turns the Platonic Representation Hypothesis into a theorem, proving that JEPAs trained separately on multiple modalities converge to shared representations up to a single rotation, and shows that next-token prediction is essentially a JEPA with 'bad views' — making it far less efficient for cross-modal alignment (good views need 42x fewer samples).
View Cached Full Text
Cached at: 10/30/26, 03:13 PM
Paper page - Proving and Improving the Platonic Representation Hypothesis with JEPAs
Source: https://huggingface.co/papers/2610.39302
Abstract
Joint-embedding predictive architectures trained on aligned multi-modal observations converge to shared representations with far greater efficiency than next-token prediction, demonstrating that positive view design—not scale alone—drives cross-modal alignment.
Whether models trained on different modalities converge to the same representation of the world is central to AI. ThePlatonic Representation Hypothesis(PRH) conjectures that they do as they scale, but without a proof it offers neither a guarantee nor a recipe. We turn the hypothesis into a theorem. First, we prove a Platonic Representation Theorem:Joint-Embedding Predictive Architectures(JEPAs) trained separately on any number of modalities learn the same representation up to one rotation whenever theirpositive pairsare two observations of one scene--good views--and their misalignment is bounded otherwise. Second, we show thatnext-token predictionis a JEPA with bad views, parts of a single document, which explains why language models align with vision only through scale. Third, we validate the theory from 145k captions to the texts of 940M web images, and uncover ascaling law of alignmentthat predicts unseen runs within seed noise.Good viewsneed 42 times fewer training samples thannext-token predictionon the same corpus to reach the same alignment, and a 109M-parametertext JEPAaligns withDINOv2andCLIP’s image encoders better than every language model we evaluated, up to 72B parameters, and thanCLIP’s text encoder:next-token predictionis a remarkably inefficient route to platonic representations. As a showcase, one rotation turns separately trained encoders into an image captioner (CIDEr .838 on COCO). Even that rotation needs no pair: on the latest pair-free benchmark, ourtext JEPAbeats every zero-shot accuracy reported with language models up to 8B parameters, e.g., 64\% against at most 56\% top-5 on ImageNet-100. Positive views, and not scale alone, are thus the lever to build platonic representations.
View arXiv pageView PDFProject pageGitHubAdd to collection
Get this paper in your agent:
hf papers read 2610\.39302
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.39302 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.39302 in a dataset README.md to link it from this page.
Spaces citing this paper1
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
new research: A tiny 109M-parameter model outperforms models up to 72B parameters at image-text alignment, using 42× less training data
Yann LeCun and Randall Balestriero mathematically prove the Platonic Representation Hypothesis and show that JEPA training with scene-related positive views achieves image-text alignment with 42× less data, with a 109M-parameter text JEPA outperforming LLMs up to 72B on cross-modal benchmarks.
Representation Without Reward: A JEPA Audit for LLM Fine-Tuning
This paper audits Joint-embedding predictive architectures (JEPA) for LLM fine-tuning on a natural-language-to-regex task, testing twenty-two auxiliary objectives. The results show that hidden-state representation improvements are only weakly coupled to decoded-task accuracy, with no auxiliary surviving family-wise correction.
The JEPA Paradox in Language: The Geometry of Linguistic Alternatives
This paper analyzes why deterministic JEPA-style latent prediction works for images but not for text, attributing the failure to high conditional variance in language where masked contexts admit multiple valid completions whose representations lack a coherent center.
Your Probabilistic JEPA Is Secretly a Hidden Markov Model: A State-Space Interpretation of Joint-Embedding Predictive Learning
The paper establishes a theoretical connection between probabilistic Joint-Embedding Predictive Learning (JEPA) and Hidden Markov Models (HMMs), providing a state-space interpretation and introducing Markov-Chain JEPA for enhanced consistency.
Slot-JEPA: Object-Structured Prediction Interfaces for Latent World Models
Slot-JEPA 将 JEPA 的稠密表示分解为可复用的语义槽与空间掩码，提供面向对象的预测接口，并结合 Transition Memory Predictor 与非平衡最优传输提升视觉理解迁移和目标条件规划，在七项视觉基准与 REALM-JEPA 机器人规划任务上取得改进。