self-supervised-pretraining

DISTAL is a dual-prior framework that combines self-supervised compositional pretraining and structure-aware knowledge distillation to achieve robust structure-agnostic materials property prediction in low-data settings, improving performance across multiple benchmark tasks.

Introduces CGM-JEPA, a self-supervised pretraining framework for continuous glucose monitor data that improves cross-modal and cross-cohort performance through masked latent prediction and distributional objectives.