Cross-scale Aligned Supervision for Training GANs

Hugging Face Daily Papers Papers
cross-scale-aligned transformer gans adversarial-supervision image-generation consistency-regularization coarse-to-fine

Summary

This paper proposes CAT, a cross-scale aligned transformer that enforces consistency between intermediate and final GAN outputs to resolve trajectory misalignment, achieving state-of-the-art FID of 1.56 on ImageNet-256.

Modern GANs often introduce adversarial supervision on intermediate generator outputs and interpret the resulting multi-stage synthesis as coarse-to-fine hierarchical generation. In this work, we challenge this interpretation. We argue that standard scale-wise adversarial supervision does not construct a proper coarse-to-fine hierarchy: each intermediate image is independently pushed toward the real distribution at its own resolution, but this scale-wise realism does not ensure that outputs across stages represent the identical generated sample. Moreover, the scale-specific image produced at each stage is not used as an explicit refinement target for the subsequent stage. Therefore, its adversarial loss can improve a scale-specific output without constraining later stages to preserve the same sample trajectory, allowing them to move toward a different sample rather than refine the previous output. We refer to this problem as a cross-scale trajectory misalignment problem. To resolve it, we propose CAT, a Cross-scale Aligned Transformer for multi-scale adversarial generation. CAT keeps the discriminator scale-wise, so each intermediate output is evaluated at its own resolution, while adding a simple generator-side consistency regularization that aligns intermediate outputs with the final output. On class-conditional ImageNet-256, CAT-H/2 achieves an FID-50K of 1.56 with one-step inference after only 60 training epochs, outperforming strong one-step GAN and diffusion/flow baselines.
Original Article
View Cached Full Text

Cached at: 05/27/26, 02:48 AM

Paper page - Cross-scale Aligned Supervision for Training GANs

Source: https://huggingface.co/papers/2605.26449

Abstract

Standard GANs with adversarial supervision on intermediate outputs fail to maintain consistent sample trajectories across scales, leading to misalignment; a new transformer-based approach called CAT addresses this by enforcing consistency between intermediate and final outputs.

ModernGANsoften introduceadversarial supervisionon intermediate generator outputs and interpret the resultingmulti-stage synthesisas coarse-to-fine hierarchical generation. In this work, we challenge this interpretation. We argue that standard scale-wiseadversarial supervisiondoes not construct a propercoarse-to-fine hierarchy: each intermediate image is independently pushed toward the real distribution at its own resolution, but thisscale-wise realismdoes not ensure that outputs across stages represent the identical generated sample. Moreover, the scale-specific image produced at each stage is not used as an explicit refinement target for the subsequent stage. Therefore, its adversarial loss can improve a scale-specific output without constraining later stages to preserve the same sample trajectory, allowing them to move toward a different sample rather than refine the previous output. We refer to this problem as across-scale trajectory misalignmentproblem. To resolve it, we propose CAT, aCross-scale Aligned Transformerfor multi-scale adversarial generation. CAT keeps the discriminator scale-wise, so each intermediate output is evaluated at its own resolution, while adding a simplegenerator-side consistency regularizationthat aligns intermediate outputs with the final output. On class-conditionalImageNet-256, CAT-H/2 achieves anFID-50Kof 1.56 with one-step inference after only 60 training epochs, outperforming strong one-step GAN and diffusion/flow baselines.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2605\.26449

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.26449 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.26449 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.26449 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Improving GANs using optimal transport

OpenAI Blog

OT-GAN introduces a novel GAN variant using optimal transport combined with energy distance in an adversarially learned feature space to improve training stability and image generation quality. The method demonstrates state-of-the-art results on benchmark problems with stable training using large mini-batches.

GEAR: Guided End-to-End AutoRegression for Image Synthesis

Hugging Face Daily Papers

GEAR proposes a method to jointly train a vector-quantized tokenizer and autoregressive generator end-to-end via representation alignment, achieving up to 10x faster convergence on ImageNet gFID compared to strong baselines.

Improved Techniques for Training Consistency Models

OpenAI Blog

OpenAI presents improved techniques for training consistency models that enable high-quality single-step image generation without distillation, achieving significant FID improvements on CIFAR-10 and ImageNet 64×64 through novel loss functions and training strategies.