Registers Matter for Pixel-Space Diffusion Transformers

Hugging Face Daily Papers Papers
diffusion-transformers pixel-space registers vision-transformers feature-maps register-guidance

Summary

This paper explores the use of register tokens in pixel-space Diffusion Transformers (DiTs), finding they improve feature map quality despite DiTs lacking patch-token outliers. The authors propose Register Guidance, a technique to amplify register contributions for better visual structure.

Vision Transformers (ViTs) are known to exhibit high-norm patch-token outliers that degrade feature map quality, a problem effectively mitigated by register tokens. As diffusion models increasingly adopt transformer architectures and move toward pixel-space training, they become closer in form to ViTs, raising the question of whether register tokens are also useful for Diffusion Transformers (DiTs). In this work, we show that DiTs differ from ViTs in a key respect: they do not exhibit patch-token outliers but still benefit from registers. Interestingly, registers are more effective in pixel-space DiTs than in latent-space DiTs. By analyzing intermediate representations, we find that register tokens produce cleaner feature maps at high noise levels, which may contribute to their effectiveness in pixel-space generation. We further observe that recent pixel-space DiT architectures implicitly incorporate register-like mechanisms, which may partially account for their strong empirical performance. Motivated by these observations, we propose Register Guidance, a technique that amplifies the contribution of register tokens responsible for improving visual structure and coherence.
Original Article
View Cached Full Text

Cached at: 07/16/26, 09:42 AM

Paper page - Registers Matter for Pixel-Space Diffusion Transformers

Source: https://huggingface.co/papers/2605.16147 Published on Jul 6

·

Submitted byhttps://huggingface.co/quickjkee

nikitaon Jul 16

Abstract

VisionTransformers(ViTs)areknowntoexhibithigh-normpatch-tokenoutliersthatdegradefeaturemapquality,aproblemeffectivelymitigatedbyregistertokens.Asdiffusionmodelsincreasinglyadopttransformerarchitecturesandmovetowardpixel-spacetraining,theybecomecloserinformtoViTs,raisingthequestionofwhetherregistertokensarealsousefulforDiffusionTransformers(DiTs).Inthiswork,weshowthatDiTsdifferfromViTsinakeyrespect:theydonotexhibitpatch-tokenoutliersbutstillbenefitfromregisters.Interestingly,registersaremoreeffectiveinpixel-spaceDiTsthaninlatent-spaceDiTs.Byanalyzingintermediaterepresentations,wefindthatregistertokensproducecleanerfeaturemapsathighnoiselevels,whichmaycontributetotheireffectivenessinpixel-spacegeneration.Wefurtherobservethatrecentpixel-spaceDiTarchitecturesimplicitlyincorporateregister-likemechanisms,whichmaypartiallyaccountfortheirstrongempiricalperformance.Motivatedbytheseobservations,weproposeRegisterGuidance,atechniquethatamplifiesthecontributionofregistertokensresponsibleforimprovingvisualstructureandcoherence.

View arXiv pageView PDFProject pageGitHub7Add to collection

Get this paper in your agent:

hf papers read 2605\.16147

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.16147 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.16147 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.16147 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation

Hugging Face Daily Papers

PointDiT presents a minimalist pixel-space diffusion transformer using a plain ViT architecture for monocular geometry estimation, outperforming complex latent-based models while maintaining simplicity and robustness in ambiguous regions.