@LeoKharon: NEW WORLD MODEL: @ylecun's team is back with an efficient model! This project involves @ylecun, @lukaskuhn77, @lucasmae…

X AI KOLs Timeline Models
self-supervised-learning video-representation efficient-ai model-release representation-collapse open-source

Summary

LeVJEPA introduces a more efficient self-supervised video pretraining approach by eliminating the need for target networks and predictors, using a single shared encoder with SIGReg regularizer, and achieving competitive performance with lower compute costs.

NEW WORLD MODEL: @ylecun's team is back with an efficient model! This project involves @ylecun, @lukaskuhn77, @lucasmaes_, @quentinlldc, and @randall_balestr. A couple definitions first: - DINO: self-DIstillation with NO labels. A self-supervised image model (Meta, 2021) where a student network learns to match a teacher (an EMA copy of itself) across two crops of the same image, with no labels and no negatives. - SIGReg: a regularizer that prevents embedding collapse by forcing the embeddings to match an isotropic Gaussian, tested with a normality test (Epps–Pulley) on many random 1-D projections instead of in full dimension. LeVJEPA is a self-supervised video pretraining method, released with open code, weights, and checkpoints. It learns a video representation by pushing the embeddings of global and local crops of the same clip together (an invariance loss), while a regularizer called SIGReg forces the embeddings toward an isotropic Gaussian to provably prevent representation collapse. Unlike V-JEPA and V-JEPA 2 it uses a single shared encoder with a projector and no target network, no predictor and no stop-gradient. It drops 95% of tokens per view, uses block-causal attention (each frame attends only to past frames), and has a single loss weight. It is evaluated purely as a representation learner via frozen probing on ImageNet-1K, Something-Something-v2 and Kinetics-400, not on any robot. What I find interesting, is that V-JEPA and V-JEPA 2 need an EMA target encoder, stop-gradients and a capacity-limited predictor to avoid collapse; LeVJEPA drops all of it for one shared encoder plus projector, preventing collapse instead with the SIGReg regularizer under a provable guarantee and a single hyperparameter. The "P" (predictor) in JEPA is effectively gone. LeVJEPA is also less compute intensive: - 5.6x to 20.8x lower total pretraining compute than V-JEPA 2 - 7.6 points higher on ImageNet-1K at matched FLOPs - trains at batch size 128 within 8GB where V-JEPA 2 saturates at batch size 28 Also worth mentioning: ImageNet-1K accuracy rises monotonically with the token-drop rate, from 33.9% at rho = 0 to 47.6% at rho = 0.95. The aggressive dropping is actually doing regularization work. On the JEPA-versus-DINO debate: - it loses to DINOv2 by 3.1 points on ImageNet-1K (appearance, static) - but wins on Something-Something-v2 by nearly 2x (motion, temporal) - and beats V-JEPA 2 by 1.9 points on ViT-L at 5.6x lower cost. -> optimized for temporal and motion understanding per compute dollar.
Original Article
View Cached Full Text

Cached at: 09/01/26, 07:47 PM

NEW WORLD MODEL: @ylecun’s team is back with an efficient model!

This project involves @ylecun, @lukaskuhn77, @lucasmaes_, @quentinlldc, and @randall_balestr.

A couple definitions first:

  • DINO: self-DIstillation with NO labels. A self-supervised image model (Meta, 2021) where a student network learns to match a teacher (an EMA copy of itself) across two crops of the same image, with no labels and no negatives.
  • SIGReg: a regularizer that prevents embedding collapse by forcing the embeddings to match an isotropic Gaussian, tested with a normality test (Epps–Pulley) on many random 1-D projections instead of in full dimension.

LeVJEPA is a self-supervised video pretraining method, released with open code, weights, and checkpoints. It learns a video representation by pushing the embeddings of global and local crops of the same clip together (an invariance loss), while a regularizer called SIGReg forces the embeddings toward an isotropic Gaussian to provably prevent representation collapse.

Unlike V-JEPA and V-JEPA 2 it uses a single shared encoder with a projector and no target network, no predictor and no stop-gradient. It drops 95% of tokens per view, uses block-causal attention (each frame attends only to past frames), and has a single loss weight.

It is evaluated purely as a representation learner via frozen probing on ImageNet-1K, Something-Something-v2 and Kinetics-400, not on any robot.

What I find interesting, is that V-JEPA and V-JEPA 2 need an EMA target encoder, stop-gradients and a capacity-limited predictor to avoid collapse; LeVJEPA drops all of it for one shared encoder plus projector, preventing collapse instead with the SIGReg regularizer under a provable guarantee and a single hyperparameter. The “P” (predictor) in JEPA is effectively gone.

LeVJEPA is also less compute intensive:

  • 5.6x to 20.8x lower total pretraining compute than V-JEPA 2
  • 7.6 points higher on ImageNet-1K at matched FLOPs
  • trains at batch size 128 within 8GB where V-JEPA 2 saturates at batch size 28

Also worth mentioning: ImageNet-1K accuracy rises monotonically with the token-drop rate, from 33.9% at rho = 0 to 47.6% at rho = 0.95. The aggressive dropping is actually doing regularization work.

On the JEPA-versus-DINO debate:

  • it loses to DINOv2 by 3.1 points on ImageNet-1K (appearance, static)
  • but wins on Something-Something-v2 by nearly 2x (motion, temporal)
  • and beats V-JEPA 2 by 1.9 points on ViT-L at 5.6x lower cost. -> optimized for temporal and motion understanding per compute dollar.

thanks gpt

Let’s do it, let me know!

It sure is!

cool

tldr; what’s your point?

This is not a video model, it is not generating a video/image as output

they did

Similar Articles

LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels

Papers with Code Trending

LeWorldModel introduces a stable, end-to-end Joint-Embedding Predictive Architecture that trains directly from pixels with minimal hyperparameters and provable anti-collapse guarantees. It achieves significant speedups in planning compared to foundation models while maintaining competitive performance on robotic manipulation tasks.

@lukaskuhn77: We introduce LeVLJEPA: the first fully non-contrastive end-to-end vision-language pretraining method competitive with C…

X AI KOLs Following

LeVLJEPA is the first fully non-contrastive end-to-end vision-language pretraining method, achieving competitive performance with CLIP and SigLIP without negatives, temperature, or momentum encoder. It learns via cross-modal prediction with stop-gradient targets and per-modality distributional regularization, providing stronger dense semantic features for downstream tasks like VLM backbones and semantic segmentation.