PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion

Hugging Face Daily Papers Papers
diffusion-models dense-prediction representation-alignment pixel-diffusion training-acceleration computer-vision

Summary

PixelDense reframes dense prediction tasks as representation alignment targets for pixel-space diffusion, routing semantic (DINOv2, SAM2) and geometric (DepthAnythingv2, Metric3Dv2) teachers through separate projection streams with an orthogonality penalty. The method improves GenEval, DPG-Bench, and HPSv2.1 scores on PixelGen and DeCo, achieves up to 53.1% PQ gain in partial-noise reconstruction, and trains 1.23x faster than baseline.

Representation alignment (REPA) accelerates diffusion transformer training, but its alignment targets are almost exclusively semantic encoders such as DINOv2 and CLIP. Recent analysis points to spatial structure, not global semantics, as the carrier of the alignment effect, yet dense-prediction foundation models trained to predict that structure remain overlooked as REPA targets. In pixel-space diffusion, SAM2, Depth Anything v2, and Metric3D v2 each outperform the DINOv2-only GenEval baseline, with the two geometric teachers leading the segmentation teacher. A flat sum of all four teachers, however, lands below the best single geometric teacher, as semantic and geometric gradients compete for one denoiser projection. We introduce PixelDense, which routes DINOv2 and SAM2 through a semantic projection stream, routes Depth Anything v2 and Metric3D v2 through a geometric projection stream, and adds a weight-space orthogonality penalty that keeps the two streams in disjoint subspaces. All four teachers are frozen during training and dropped at inference. Applied to PixelGen and DeCo with a single recipe, PixelDense improves GenEval, DPG-Bench, and HPS v2.1, raises PixelGen-XXL's GenEval Overall from 0.7927 to 0.8093, and beats every single-teacher and unfactored multi-teacher variant. In partial-noise reconstruction, independent panoptic, depth, and surface-normal probes show up to 53.1% PQ gain and 36.0% depth AbsRel reduction at τ=0.5 across COCO and Flickr30K. From random initialization, PixelDense also reaches the baseline's peak GenEval 1.23x faster. In SDEdit editing on PIE-Bench, PixelDense keeps more of the source background and layout at every edit strength, raising background PSNR by up to 2.2 dB.
Original Article
View Cached Full Text

Cached at: 10/02/26, 04:27 AM

Paper page - PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion

Source: https://huggingface.co/papers/2610.00483 Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

Representationalignment(REPA)acceleratesdiffusiontransformertraining,butitsalignmenttargetsarealmostexclusivelysemanticencoderssuchasDINOv2andCLIP.Recentanalysispointstospatialstructure,notglobalsemantics,asthecarrierofthealignmenteffect,yetdense-predictionfoundationmodelstrainedtopredictthatstructureremainoverlookedasREPAtargets.Inpixel-spacediffusion,SAM2,DepthAnythingv2,andMetric3Dv2eachoutperformtheDINOv2-onlyGenEvalbaseline,withthetwogeometricteachersleadingthesegmentationteacher.Aflatsumofallfourteachers,however,landsbelowthebestsinglegeometricteacher,assemanticandgeometricgradientscompeteforonedenoiserprojection.WeintroducePixelDense,whichroutesDINOv2andSAM2throughasemanticprojectionstream,routesDepthAnythingv2andMetric3Dv2throughageometricprojectionstream,andaddsaweight-spaceorthogonalitypenaltythatkeepsthetwostreamsindisjointsubspaces.Allfourteachersarefrozenduringtraininganddroppedatinference.AppliedtoPixelGenandDeCowithasinglerecipe,PixelDenseimprovesGenEval,DPG-Bench,andHPSv2.1,raisesPixelGen-XXL’sGenEvalOverallfrom0.7927to0.8093,andbeatseverysingle-teacherandunfactoredmulti-teachervariant.Inpartial-noisereconstruction,independentpanoptic,depth,andsurface-normalprobesshowupto53.1%PQgainand36.0%depthAbsRelreductionatτ=0.5acrossCOCOandFlickr30K.Fromrandominitialization,PixelDensealsoreachesthebaseline’speakGenEval1.23xfaster.InSDEditeditingonPIE-Bench,PixelDensekeepsmoreofthesourcebackgroundandlayoutateveryeditstrength,raisingbackgroundPSNRbyupto2.2dB.

View arXiv pageView PDFProject pageGitHub3Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.00483 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.00483 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.00483 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation

Hugging Face Daily Papers

PointDiT presents a minimalist pixel-space diffusion transformer using a plain ViT architecture for monocular geometry estimation, outperforming complex latent-based models while maintaining simplicity and robustness in ambiguous regions.

Learning Representations to Guide Pixel Diffusion Transformers

Hugging Face Daily Papers

论文提出 Representation Guidance（RG），在保持纯 Transformer 架构的前提下，通过低维瓶颈形成并主动利用内部表示来指导像素扩散模型的生成。RG-XL/16 在 ImageNet 256x256 上取得 FID 1.55，并可扩展至文本到图像生成（GenEval 0.89，DPG-Bench 87.1）。

Adversarial Training for Pixel Diffusion

Hugging Face Daily Papers

This paper presents the first systematic study of adversarial post-training for pixel diffusion models, showing it effectively corrects high-frequency content deficiencies to improve distribution fidelity, coverage, and perceptual quality without altering model architecture.