Tag
This paper investigates the scaling properties of native multimodal pre-training, deriving compute-optimal model sizes and token counts under a fixed budget, and revealing distinct scaling behaviors for language and multimodal objectives.