NVIDIA Spatial Intelligence Lab 提出 PiD，重新设计了 latent 扩散模型里的解码环节。

目前主流文生图都在 latent 空间生成，再用 VAE decoder 映射回像素。这个 decoder 的目标是逆转 encoder、重建图像，本身不负责合成新细节，且在百万像素级别开销明显上升。

PiD 把「latent→像素」的解码重新表述为一个条件像素扩散过程，将解码与上采样统一成单个模块。

512×512 latent 解码到 2048×2048，在 RTX 5090 上耗时低于 1 秒（峰值显存 13GB），在 GB200 上约 210ms；相比级联超分基线 SeedVR2 约快 5.9 倍（211ms vs 1237ms）。

通过噪声 latent 训练和 sigma-aware gating，对未完全去噪的 latent 保持鲁棒，因此 base LDM 可以提前退出，剩下交由 PiD 在高分辨率完成。方法同时支持 VAE latent 与语义 latent（DINOv2 / SigLIP / RAE）。

实际效果与适用边界还需要更多场景去验证，不过把 decoder 当成一个能合成细节的扩散模块来重做，是个值得关注的方向。

https://research.nvidia.com/labs/sil/projects/pid/…

Fast and High-Resolution Latent Decoding with Pixel Diffusion

Source: https://research.nvidia.com/labs/sil/projects/pid/

VAE Decoder

PiD

RAE Decoder

PiD

VAE Decoder

PiD

VAE Decoder

PiD

Abstract

Most practical high-resolution text-to-image systems rely on latent diffusion models, where generation is performed in a compact latent space and a decoder maps latents back to pixels. Yet the latent-to-pixel decoder is reconstruction-oriented, optimized to invert the encoder rather than synthesize more details, and becomes increasingly costly at megapixel scale. This drawback calls for a more expressive and efficient decoding paradigm. Motivated by recent progress in scalable pixel-space diffusion, we introducePiD, aPixel diffusionDecoder that reformulates latent decoding as conditional pixel diffusion, unifying decoding and upsampling into one generative module. By denoising directly in high-resolution pixel space,PiDsynthesizes 4× and even 8× upscaled images with low latency. For latent conditioning, a lightweight sigma-aware adapter injects noise-corrupted latents into the pixel diffusion backbone, enablingPiDto decode partially denoised latents and terminate the latent diffusion process early. To further improve efficiency, we distill the model using DMD2, reducing inference to just 4 steps.PiDapplies to both conventional VAE latents and semantic latents (e.g., SigLIP, DINOv2) used in recent RAE-based models.PiDdecodes latents of 512×512 images into 2048×2048 pixels in under 1 second with 13 GB peak memory on a consumer RTX 5090, and as fast as 210 ms on a GB200 GPU, about 6× faster than cascaded diffusion-based super-resolution pipelines with better visual fidelity.

Results

From Latent to Pixels

Select a latent space and move the step slider to compare PiD decoding quality at different early-termination points. Drag thewhite divideron each image to reveal the VAE/RAE decode vs. PiD decode.

4K Decode

Direct latent→4K decoding with PiD. Click any image to launch a side-by-side comparison against the VAE decoder.

Baseline Comparison

Hover over any image to activate the synchronized zoom lens across all six views.

Quantitative Results(Decoding + Upsampling, 512² → 2048²)

End-to-End Decoding Latency (ms) ↓

PiD is up to5.9× fasterthan SeedVR2 (211.2 ms vs 1237.5 ms)

Gemini-3-Flash Judge Rating (%) ↑

% of evaluations where judges preferPiDover each baseline

Method

**Overview of PiD.**PiD unifies latent decoding and upsampling as a single latent-conditioned pixel diffusion model that predicts the target-resolution pixel-space velocity field. Noise-corrupted latent training and sigma-aware gating make the decoder robust to partially denoised latents, enabling early exit from the base LDM while preserving high-resolution output quality.