Tag
The paper shows that diffusion training acceleration methods like REPA and dispersive regularization primarily reorganize the AdaLN-Zero residual gate α, and proposes a teacher-free radial objective that broadens gate participation to improve FID on ImageNet 256×256 with SiT-B/2.
A speculative discussion about the next architectural innovations likely to appear in AI models, including looped transformers, Gated Delta Net 2 variants, attention residuals, and a potential revival of diffusion transformers without quality loss.
该论文提出扩散Transformer的训练加速部分源于AdaLN-Zero残差门控α的重组，并引入归一化参与度比(nPR)和一种无教师的径向目标，在ImageNet 256x256上以SiT-B/2改进了FID。
提出在由2D基础模型导出的高维隐式3D潜在空间中直接进行扩散建模，实现3D-grounded的场景级3D生成，相比2D多视角方法在空间一致性和效率上更具优势，并验证了其在表示与模型容量上的可扩展性。
提出AReA：一种训练自由的注意力机制，其感受野随去噪过程从粗到细自适应演化，用于实现高效高分辨率Diffusion Transformer图像生成，在多个分辨率和模型上超越现有无训练基线且更快速。
提出 MC-Sparse，一个免训练的元缓存稀疏注意力框架，通过分析稠密-稀疏注意力差距的三大来源，在视频与3D生成中实现最高2.32倍去噪加速且几乎无质量损失。
LiFT introduces a family of looped generative models that scale computation by repeatedly applying a shared Diffusion Transformer (DiT) core, training each loop step on a continuous-depth flow-matching regression target that allows inference far beyond training depth. On ImageNet 256x256, LiFT-L/2 beats a dense DiT-XL/2 baseline by 3.34 FID points while using ~60% fewer parameters and 52% fewer inference FLOPs.
The paper proposes Tex-Zero, a native 3D texture generation framework trained entirely on constructed 2D images rather than real 3D assets, by converting high-quality 2D images into 3D training samples via plane representations and patch-wise random rotations, showing high-fidelity texture synthesis is possible without 3D asset data.
The paper introduces Persistence Forcing (PerF), which assigns heterogeneous refinement budgets to feature groups in pixel-space Diffusion Transformers, producing persistent global-structure features that guide active detail features. PerF-H reaches FID 1.63 on ImageNet 256x256, outperforming JiT-H with competitive efficiency.
The paper introduces a structured residual connectivity design for diffusion transformers that enables adaptive cross-layer connections, leading to faster convergence and improved image synthesis quality with minimal parameter increase.
The paper introduces NowWAM, a future-target-free co-training formulation that denoises the current observation and predicts robot actions from the same visual stream, showing that a separate future visual target is not essential for adapting pretrained generative DiTs to robot control while achieving 87.7% on LIBERO-Plus with FLUX2-Klein and a 1.8x speedup.
The paper proposes CoMA-DiT, a bidirectional cross-modal Diffusion Transformer for latent augmentation in multimodal brain state decoding, which enhances performance in tasks like auditory attention decoding and emotion recognition by using paired modalities as mutual supervisory signals.
Marigold V2 repurposes diffusion transformers for monocular depth estimation via single-step inference and a novel fine-tuning protocol, achieving sharper depth maps and significant improvements on benchmarks like KITTI and ETH3D.
This paper scales the Muon optimizer for Diffusion Transformers from 1.3B to 15B parameters, introducing Periodic Row-wise Muon to reduce computational overhead while preserving generative quality improvements over AdamW.
HelloWorld is a video world model that enables socially interactive characters, allowing users to prompt on-screen characters to respond via a single button press. It uses self-distillation and training-free cross-attention masking to naturalize interactions, and introduces HelloWorldBench for evaluation.
MiniWorld is a reproducible framework for training video world models from scratch using a block-causal Video Diffusion Transformer with Flow Matching, enabling efficient streaming generation and trainable in days on a single 8-GPU server.
Sol-Attn introduces a training-free method to sparsify attention for video generation inference, achieving over 2x speedup by dynamically selecting key-value blocks during online softmax with minimal quality loss.
This paper investigates how text template tokens in diffusion transformers (DiTs) act as implicit semantic registers that causally maintain object identity during denoising, and proposes a training-free pruning rule that removes 20% of attention FLOPs with minimal performance drop.
Introduces appearance pointers, compact tokens that guide Diffusion Transformers to apply correct appearance cues at specified spatial locations, enabling modality-agnostic localized multimodal control without retraining the base model.
ReChannel adapts pretrained diffusion transformers (e.g., FLUX-Klein) for dense prediction tasks by mapping tokens directly to pixel-space patches, achieving state-of-the-art results on trimap-free matting, KITTI depth, and referring segmentation with minimal additional parameters.