From SRA to Self-Flow: Data Augmentation or Self-Supervision?
Summary
This paper investigates the mechanisms behind self-alignment methods in diffusion transformers, revealing that performance improvements from methods like Self-Flow primarily come from data augmentation along the noise dimension rather than token interactions between noise levels. The authors introduce Attention Separation to demonstrate this and propose an effective design combining self-representation alignment with dual-timestep augmentation.
View Cached Full Text
Cached at: 07/03/26, 07:53 AM
Paper page - From SRA to Self-Flow: Data Augmentation or Self-Supervision?
Source: https://huggingface.co/papers/2607.02508
Abstract
Research investigates the mechanisms behind self-alignment methods in diffusion transformers, finding that performance improvements stem primarily from data augmentation along the noise dimension rather than token interactions between noise levels.
Representation alignmenthas become an effective way to acceleratediffusion transformertraining and improve generation quality. Recentself-alignmentmethods, such asSRAandSelf-Flow, further remove the dependency on external pretrained encoders by constructing alignment within the diffusion model itself. However, the mechanism behind the improvement fromSRAtoSelf-Flow,dual-time scheduling, remains under-examined:Self-Flowattributes its gain to interactions between tokens at different noise levels, where cleaner tokens help infer noisier ones. In this work, we revisit this explanation and ask whether the gain instead comes fromdata augmentationalong thenoise dimension. To disentangle these factors, we introduceAttention Separation, which preserves the same dual-timestep input asSelf-Flowwhile blocking attention between tokens assigned to different noise levels. Surprisingly, removing such interaction does not degrade performance and can even improve it, suggesting that the improvement fromSRAtoSelf-Flowmainly comes fromdata augmentation. Furthermore,We show thatAttention Separationitself provides an augmentation effect by splitting a single image into multiple effective training parts to expand the training data. Based on these observations, we combine self-representation alignmentwith dual-timestep and attention-separation augmentation, and demonstrate the effectiveness of this design on ImageNet.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2607\.02508
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.02508 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.02508 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.02508 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
The Reward Was in Your Data All Along: Correcting Flow Matching with Discriminator-Guided RL
This paper proposes Discriminator-Guided Reinforcement Learning (DRL) to correct alignment issues in score- and flow-matching models by using a pretrained representation space discriminator as an optimal reward signal, significantly improving visual fidelity and semantic quality without human preferences.
@svlevine: Diffusion (or flow) makes for excellent policies, but training them with RL is notoriously hard: BPTT is unstable, RL o…
New paper shows how to optimize flow matching actors for reinforcement learning by approximating the Jacobian of the flow denoising process with the identity matrix, making training feasible.
LeapAlign: Post-Training Flow Matching Models at Any Generation Step by Building Two-Step Trajectories
LeapAlign is a post-training method that improves flow matching model alignment with human preferences by reducing computational costs through two-step trajectory shortcuts while enabling stable gradient propagation to early generation steps. The method outperforms state-of-the-art approaches when fine-tuning Flux models across various image quality and text-alignment metrics.
@svlevine: A new way to do off-policy RL with diffusion: if we have off-policy data, we need to figure out what the diffusion late…
A new method for off-policy reinforcement learning with diffusion models, using flow reversal to handle off-policy data by reversing the diffusion process on it.
Multi-Resolution Flow Matching: Training-Free Diffusion Acceleration via Staged Sampling
MrFlow is a training-free multi-resolution acceleration strategy for flow-matching text-to-image models that combines low-resolution generation with pixel-space super-resolution and noise injection, achieving up to 25x end-to-end speedup without training or runtime modifications.