A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples

Hugging Face Daily Papers Papers
diffusion-models self-guidance pixel-space image-generation synthetic-data

Summary

This paper introduces Synthetic Self-Guidance (SSG), a method that attaches a lightweight prediction head to a frozen pretrained pixel-space diffusion model, using the discrepancy between intermediate and final predictions as self-guidance during sampling. It shows that model-generated samples suffice for training the head, improving FID by over 50% on several variants without classifier-free guidance and enhancing strong baselines with CFG.

Pixel-space diffusion models aim to learn an end-to-end generator directly over raw pixels. This is challenging because a single model must capture both global structure and local texture in the same high-dimensional space. While recent work improves pixel diffusion through alternative prediction targets, training objectives, and architectures, these advances typically require training a new model from scratch. We show there is a cheaper, complementary strategy: a frozen, pretrained pixel diffusion model can guide itself. Our key observation is that intermediate layers of a pretrained pixel diffusion transformer can be decoded into coarse predictions that capture the main low-frequency structure, while the final layers progressively refine local, high-frequency details. We therefore attach a lightweight prediction head to an intermediate layer, keep the backbone frozen, and use the discrepancy between the intermediate and final predictions as a self-guidance direction during sampling. To train this head, we further find that real images are not necessary. Instead, model-generated samples suffice and even outperform real images for training the head, especially in enhancing the high-frequency components that pixel diffusion tends to underfit. Across multiple pixel diffusion models on ImageNet, our Synthetic Self-Guidance (SSG) consistently improves generation while adapter training requires less than 1% of full-model training compute: it reduces FID by over 50% across the evaluated JiT variants without classifier-free guidance (CFG) and further improves strong baselines with CFG, e.g., JiT-H/16 from 1.86 to 1.67 and PixelREPA-H/16 from 1.81 to 1.59. Our code is available at https://github.com/zfu006/SSG.
Original Article
View Cached Full Text

Cached at: 08/04/26, 01:39 PM

Paper page - A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples

Source: https://huggingface.co/papers/2607.29122

Abstract

Pixel-spacediffusionmodelsaimtolearnanend-to-endgeneratordirectlyoverrawpixels.Thisischallengingbecauseasinglemodelmustcapturebothglobalstructureandlocaltextureinthesamehigh-dimensionalspace.Whilerecentworkimprovespixeldiffusionthroughalternativepredictiontargets,trainingobjectives,andarchitectures,theseadvancestypicallyrequiretraininganewmodelfromscratch.Weshowthereisacheaper,complementarystrategy:afrozen,pretrainedpixeldiffusionmodelcanguideitself.Ourkeyobservationisthatintermediatelayersofapretrainedpixeldiffusiontransformercanbedecodedintocoarsepredictionsthatcapturethemainlow-frequencystructure,whilethefinallayersprogressivelyrefinelocal,high-frequencydetails.Wethereforeattachalightweightpredictionheadtoanintermediatelayer,keepthebackbonefrozen,andusethediscrepancybetweentheintermediateandfinalpredictionsasaself-guidancedirectionduringsampling.Totrainthishead,wefurtherfindthatrealimagesarenotnecessary.Instead,model-generatedsamplessufficeandevenoutperformrealimagesfortrainingthehead,especiallyinenhancingthehigh-frequencycomponentsthatpixeldiffusiontendstounderfit.AcrossmultiplepixeldiffusionmodelsonImageNet,ourSyntheticSelf-Guidance(SSG)consistentlyimprovesgenerationwhileadaptertrainingrequireslessthan1%offull-modeltrainingcompute:itreducesFIDbyover50%acrosstheevaluatedJiTvariantswithoutclassifier-freeguidance(CFG)andfurtherimprovesstrongbaselineswithCFG,e.g.,JiT-H/16from1.86to1.67andPixelREPA-H/16from1.81to1.59.Ourcodeisavailableathttps://github.com/zfu006/SSG.

View arXiv pageView PDFGitHub2Add to collection

Get this paper in your agent:

hf papers read 2607\.29122

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.29122 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.29122 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.29122 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Self Gradient Forcing: Native Long Video Extrapolation

Papers with Code Trending

Proposes Self Gradient Forcing (SGF), a two-pass training strategy for autoregressive video diffusion models that provides missing supervision for writing useful context memory, enabling strong long-video extrapolation even from short training windows.