@iScienceLuvr: Music-JEPA: Learning a World Model of Sound from Action "we propose to learn a world model of piano sound using JEPA by…
Summary
This paper proposes Music-JEPA, a world model that learns piano sound representations by framing audio as a state and piano roll as an action. It captures action-sound relationships and enables downstream tasks like beat tracking and piano transcription via planning.
View Cached Full Text
Cached at: 07/28/26, 04:36 PM
Music-JEPA: Learning a World Model of Sound from Action
“we propose to learn a world model of piano sound using JEPA by framing music as an action-conditioned system: the audio is treated as the state, and the pianoroll as the instrument action.”
“Experiments show that the learned model captures the relationships between musical actions and their resulting sound. The resulting representations support downstream tasks, including beat tracking, composer identification, and key estimation, and enable piano transcription via planning, by searching for actions that best explain a target sound.”
Similar Articles
DVD-JEPA: an open-source, fully-reproducible JEPA world model [P]
DVD-JEPA is an open-source, minimal JEPA world model that learns representations from video by predicting future embeddings rather than pixels. It uses a bouncing DVD logo to demonstrate position recovery, dreaming, and anomaly detection, all running in a browser.
Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decoding
Delta-JEPA introduces a reconstruction-free world model that augments latent forward prediction with a Latent Difference Action Decoder to prevent collapse and improve action-sensitivity, achieving better planning performance on visual continuous-control tasks.
@AbdelStark: It’s time to JEPA pill the world! awesome-jepa: A curated list of papers, models, code, datasets, and learning resource…
A curated list of papers, models, code, datasets, and learning resources for Joint Embedding Predictive Architectures (JEPA), the self-supervised approach to world models proposed by Yann LeCun.
@iScienceLuvr: Learning Sparse Latent Predictive Foundation Model for Multimodal Neuroimaging This paper introduces Neuro-JEPA, a foun…
This paper introduces Neuro-JEPA, a foundation model that uses a latent predictive objective and Mixture-of-Experts architecture to encode brain MRI scans across T1w, T2w, and FLAIR sequences, pretrained on a large dataset of 1.55 million scans.
AV-JEPA: Extending LeJEPA to Audio-Visual Self-Supervised Learning
AV-JEPA extends LeJEPA to audio-visual self-supervised learning, achieving cross-modal alignment in latent space without decoders, contrastive negatives, or complex losses, and obtains competitive classification on VGGSound and AudioSet.