Tag
A Meta/Oxford study finds that multimodal models need surprisingly little image-generation data if language and visual understanding are trained together, suggesting an optimal 70/25/5 split of language, image understanding, and image generation tokens. It also warns that delaying vision training causes 'vision laziness' where models ignore images.
A systematic study from Meta FAIR, Reality Labs, and Oxford on multimodal pretraining, revealing asymmetric knowledge flow between modalities, synergy vs. competition dynamics, the benefits of early unification, and efficient training recipes validated with 13.5B MoE models.
This paper systematically explores the physics of multimodal pretraining, uncovering how knowledge flows between modalities, conditions for synergy vs. competition, the benefits of early unification, and efficient pretraining recipes validated on 13.5B MoE models.