Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning

Hugging Face Daily Papers Papers
pixel-space-diffusion text-to-image image-generation diffusion-models depth-estimation image-restoration

Summary

Iris-3B 是一个 3B 参数的像素空间文本到图像扩散模型，通过课程式预训练从零训练并与潜空间模型 FLUX.2 Klein 进行对比。结果显示像素空间生成先验在深度估计和图像修复任务上并无显著优势，但证明了像素空间预训练可扩展到与潜空间模型相当的画质。

Pixel-space diffusion models avoid the lossy VAE of latent models, which suggests an advantage on downstream tasks where fine-grained detail matters. We test this claim along both routes to a pixel-space backbone. We pretrain Iris-3B, a 3B-parameter pixel-space text-to-image transformer, from scratch through a 256to512to1024 curriculum, after first ablating the prediction target and representation alignment at 256^2 to decide what to scale. We also convert a pretrained latent model, FLUX.2 Klein base 4B, to pixel space. We fine-tune both families for monocular depth estimation and for image restoration/super-resolution. We find no significant improvement from using a pixel-space generative prior. Fine-tuned for depth with one matched direct-regression recipe, Iris-3B is level with the latent FLUX.2 Klein and the converted pixel FLUX.2 Klein falls behind it, and on 4times DIV2K restoration neither pixel model beats a latent FLUX.2 Klein fine-tune, the converted one trailing it slightly. We document the recipes, the failure modes and the remaining confounds behind this negative result. Nevertheless, Iris-3B shows that pixel-space pretraining with the pixel-transformer (PiT) head of PixelDiT scales to 3B parameters and to text-to-image quality competitive with latent models, matching Qwen-Image on OneIG under the official evaluators at 1024^2. We release its weights and training code in the hope that they help pave the way for further work on pixel-space generation.
Original Article
View Cached Full Text

Cached at: 10/08/26, 07:59 PM

Paper page - Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning

Source: https://huggingface.co/papers/2610.09450

Abstract

Pixel-spacediffusionmodelsavoidthelossyVAEoflatentmodels,whichsuggestsanadvantageondownstreamtaskswherefine-graineddetailmatters.Wetestthisclaimalongbothroutestoapixel-spacebackbone.WepretrainIris-3B,a3B-parameterpixel-spacetext-to-imagetransformer,fromscratchthrougha256to512to1024curriculum,afterfirstablatingthepredictiontargetandrepresentationalignmentat256^2todecidewhattoscale.Wealsoconvertapretrainedlatentmodel,FLUX.2Kleinbase4B,topixelspace.Wefine-tunebothfamiliesformonoculardepthestimationandforimagerestoration/super-resolution.Wefindnosignificantimprovementfromusingapixel-spacegenerativeprior.Fine-tunedfordepthwithonematcheddirect-regressionrecipe,Iris-3BislevelwiththelatentFLUX.2KleinandtheconvertedpixelFLUX.2Kleinfallsbehindit,andon4timesDIV2KrestorationneitherpixelmodelbeatsalatentFLUX.2Kleinfine-tune,theconvertedonetrailingitslightly.Wedocumenttherecipes,thefailuremodesandtheremainingconfoundsbehindthisnegativeresult.Nevertheless,Iris-3Bshowsthatpixel-spacepretrainingwiththepixel-transformer(PiT)headofPixelDiTscalesto3Bparametersandtotext-to-imagequalitycompetitivewithlatentmodels,matchingQwen-ImageonOneIGundertheofficialevaluatorsat1024^2.Wereleaseitsweightsandtrainingcodeinthehopethattheyhelppavethewayforfurtherworkonpixel-spacegeneration.

View arXiv pageView PDFProject pageGitHub3Add to collection

Get this paper in your agent:

hf papers read 2610\.09450

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### speridlabs/iris-3b Text-to-Image• 3B• Updatedabout 3 hours ago • 18 • 3

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.09450 in a dataset README.md to link it from this page.

Spaces citing this paper1

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

speridlabs/iris-3b

Hugging Face Models Trending

Sperid Labs 发布 Iris-3B，一个 30 亿参数的像素空间生成模型，直接输出每个像素、无需 VAE 或潜空间，并可通过微调完成单目深度估计、图像修复与超分辨率等密集视觉任务。

FLUX 3 Image

Hacker News Top

Black Forest Labs has released FLUX 3 Image, a controllable image generation model that supports synthesizing images from scratch, allowing precise layout of elements in the scene via bounding boxes and maximum control over every pixel.

@FeitengLi: NVIDIA Spatial Intelligence Lab proposes PiD, redesigning the decoding stage in latent diffusion models. Current mainstream text-to-image generation happens in latent space, then uses a VAE decoder to map back to pixels. This decoder's…

X AI KOLs Timeline

NVIDIA Spatial Intelligence Lab proposes PiD, which redesigns the decoding stage of latent diffusion models as a conditional pixel diffusion process, unifying decoding and upsampling to achieve low-latency, high-resolution decoding.

black-forest-labs/flux-3-image-klein

Hugging Face Models Trending

Black Forest Labs 发布 FLUX 3 Image Klein，一个 5.6B 开放权重文本生图与图像编辑模型，支持最多 10 张参考图像、提示词中的边界框定位，并提供 BF16、FP8 和 NVFP4 多种检查点格式。