Efficient Image Synthesis with Sphere Latent Encoder

Hugging Face Daily Papers Papers
image-synthesis spherical-latent-space denoising efficient-sampling latent-encoder

Summary

This paper proposes Sphere Latent Encoder, an efficient few-step image generation framework that performs denoising entirely in a spherical latent space, achieving high-quality 256×256 images with significantly reduced computational cost and improved FID scores on ImageNet-1K.

Few-step image generation has seen rapid progress, with consistency and meanflow-based methods significantly reducing the number of sampling steps. Despite their low inference cost, these approaches often suffer from training instability and limited scalability. Sphere Encoder is a recent alternative that produces high-quality images in only a few steps; however, it requires repeated transitions between the pixel space and latent space during inference while jointly optimizing reconstruction and generation within a single architecture. This design leads to computational inefficiency and objective conflict between reconstruction and generation. To address these limitations, we decouple the framework into a fixed pretrained image encoder and a separate latent denoising model trained entirely in a spherical latent space. Our approach eliminates repeated pixel-space operations during training and inference, improving efficiency and allowing reconstruction and generation to specialize independently. On Animal-Faces, Oxford-Flowers and ImageNet-1K datasets, our method significantly outperforms Sphere Encoder in both generation quality and inference speed, while achieving competitive results against strong few-step and multi-step baselines.
Original Article
View Cached Full Text

Cached at: 05/18/26, 02:26 PM

Paper page - Efficient Image Synthesis with Sphere Latent Encoder

Source: https://huggingface.co/papers/2605.15592 🚀Sphere Latent Encoder: Efficient Image Synthesis with Spherical Latent Denoising

This paper proposesSphere Latent Encoder, an efficient few-step image generation framework that performs denoising entirely in aspherical latent space. Instead of repeatedly moving between pixel space and latent space as in the original Sphere Encoder, the method uses a fixed pretrained representation autoencoder and trains a separate latent denoising model. This decouples reconstruction from generation and makes sampling much more efficient.

Key idea: Use a pretrained RAE/DINOv2-based encoder as a strong image tokenizer, project noisy latents onto a hypersphere, and train a transformer denoiser directly in that latent space. During inference, the model refines latents over only a few steps and calls the decoder once at the end.

Why it matters: The approach keeps the simplicity of Sphere Encoder while removing its main bottleneck: repeated encoder-decoder transitions. This leads to substantially lower computational cost and better sample quality in the low-step regime.

Highlights:

  • Generates high-quality 256×256 images in only a few sampling steps.
  • Reduces inference cost by avoiding repeated pixel-latent conversions.
  • Improves over Sphere Encoder on Animal-Faces, Oxford-Flowers, and ImageNet-1K.
  • Achieves strong ImageNet-1K results, improving FID from4.02to2.25at the same 4-step CFG setting, and to2.11with 6 steps.
  • Ablations show that spherical projection, consistency loss, noise distribution, and the choice of representation autoencoder are all important for performance.

A particularly interesting takeaway is that strong semantic latent representations plus spherical latent modeling can provide a practical alternative to standard diffusion/flow sampling, especially when low-NFE generation is the priority.

Limitations are also clear: the current experiments focus on class-conditional generation, rely on a strong pretrained representation autoencoder, and still find high-quality one-step generation challenging. Overall, this is a promising direction for efficient latent-space generative modeling.

Similar Articles

L2P: Unlocking Latent Potential for Pixel Generation

Hugging Face Daily Papers

The L2P paper introduces a Latent-to-Pixel transfer paradigm that leverages pre-trained latent diffusion models to create efficient pixel-space models capable of 4K generation with minimal training overhead.

SP^3: Spherical Priors for Plug-and-Play Restoration

Hugging Face Daily Papers

This paper introduces SP³, a method using Spherical Encoder priors for Plug-and-Play image restoration, achieving perceptual quality comparable to zero-shot diffusion priors while being 3–630× faster across tasks.

@FeitengLi: NVIDIA Spatial Intelligence Lab proposes PiD, redesigning the decoding stage in latent diffusion models. Current mainstream text-to-image generation happens in latent space, then uses a VAE decoder to map back to pixels. This decoder's…

X AI KOLs Timeline

NVIDIA Spatial Intelligence Lab proposes PiD, which redesigns the decoding stage of latent diffusion models as a conditional pixel diffusion process, unifying decoding and upsampling to achieve low-latency, high-resolution decoding.