Tag
This paper systematically explores the physics of multimodal pretraining, uncovering how knowledge flows between modalities, conditions for synergy vs. competition, the benefits of early unification, and efficient pretraining recipes validated on 13.5B MoE models.