An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models

Hugging Face Daily Papers Papers
diffusion-models text-to-image pixel-space training-strategy generative-ai empirical-study

Summary

This paper proposes a latent-to-pixel training strategy for pixel-space text-to-image diffusion models, accelerating convergence and improving inference speed while matching or surpassing latent-space counterparts.

This paper investigates an increasingly important topic in generative modeling: pixel-space diffusion models. Although numerous studies have explored this topic, most focus on small-scale or class-conditional settings. Consequently, a practical recipe for training pixel-space models that rival or exceed well-established latent-space counterparts remains elusive. Through a comprehensive empirical study, we first observe that direct large-scale pre-training in pixel space converges substantially more slowly than in latent space. This observation motivates a latent-to-pixel strategy that acquires generative priors efficiently in latent space and transitions to pixel space during post-training. We then systematically investigate the key design choices governing this transition, including weight initialization, data composition, prediction target, decoder architecture, and noise schedule, and identify a practical recipe that makes the resulting pixel-space models match or outperform their latent-space counterparts while delivering 3.18 to 4.75 times end-to-end inference speedups. We hope that our findings provide useful empirical insights and practical guidelines for future research on pixel-space generation.
Original Article
View Cached Full Text

Cached at: 08/18/26, 03:49 AM

Paper page - An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models

Source: https://huggingface.co/papers/2608.16887 Authors:

,

,

,

,

,

,

,

,

,

,

,

Abstract

Researchers propose a latent-to-pixel training strategy that accelerates convergence and improves inference speed for large-scale pixel-space diffusion models.

This paper investigates an increasingly important topic in generative modeling:pixel-space diffusion models. Although numerous studies have explored this topic, most focus on small-scale or class-conditional settings. Consequently, a practical recipe for training pixel-space models that rival or exceed well-established latent-space counterparts remains elusive. Through a comprehensive empirical study, we first observe that direct large-scale pre-training in pixel space converges substantially more slowly than in latent space. This observation motivates alatent-to-pixel strategythat acquiresgenerative priorsefficiently in latent space and transitions to pixel space during post-training. We then systematically investigate the key design choices governing this transition, includingweight initialization, data composition,prediction target,decoder architecture, andnoise schedule, and identify a practical recipe that makes the resulting pixel-space models match or outperform their latent-space counterparts while delivering 3.18 to 4.75 times end-to-end inference speedups. We hope that our findings provide useful empirical insights and practical guidelines for future research on pixel-space generation.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2608\.16887

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.16887 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.16887 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.16887 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Abra: Scaling Diffusion Image Training

Hugging Face Daily Papers

This paper presents a systematic scaling law study for text-to-image diffusion models, showing they scale predictably but require significantly more data per parameter than language models for optimal training.

L2P: Unlocking Latent Potential for Pixel Generation

Hugging Face Daily Papers

The L2P paper introduces a Latent-to-Pixel transfer paradigm that leverages pre-trained latent diffusion models to create efficient pixel-space models capable of 4K generation with minimal training overhead.