Paper page - Abra: Scaling Diffusion Image Training

Source: https://huggingface.co/papers/2608.17286

Abstract

Scaling laws for text-to-image diffusion models reveal predictable compute-optimal training requiring far more data per parameter than language models, with robust overtraining behavior and universal curve shapes.

Compute-optimalscaling lawsguide the training of frontier language models yet remain largely unexplored for visual generation. We present a systematic scaling law study fortext-to-image diffusion modelsusing Abra, a controlled family offlow-matching transformerstrained across three orders of magnitude worth of compute (10^{19} to 10^{22} FLOPs), reaching significantly larger compute budgets than previous works. We demonstrate that diffusion models scale just as predictably as language models but require far more data to train optimally: compute optimality occurs at approximately 200 image tokens per parameter, ten times theChinchillacompute-optimalprescription for LLMs. We show that unlike language models, diffusion models are robust to overtraining and that practitioners should err on the side of more data rather than a larger model. Finally, we show that this predictability extends beyond training loss togenerative quality metrics, optimalCFGsettings,representation quality, and even the shape of the training curves, which collapse onto a universal form.

View arXiv pageView PDFAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.17286 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.17286 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.17286 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.