AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers

Hugging Face Daily Papers Papers
diffusion-transformers attention high-resolution image-generation inference-efficiency training-free

Summary

提出AReA：一种训练自由的注意力机制，其感受野随去噪过程从粗到细自适应演化，用于实现高效高分辨率Diffusion Transformer图像生成，在多个分辨率和模型上超越现有无训练基线且更快速。

Generating images beyond the training resolution of Diffusion Transformers (DiTs) remains challenging, as the number of tokens grows rapidly with resolution, full attention becomes expensive, and outputs often suffer from structural distortions and degraded detail. We identify uniform full attention as the common source of both problems. Attending to all tokens at every step causes quadratic cost and exposes the model to more tokens and relative distances beyond the training range, pushing the model out of distribution. We observe that attention's effective receptive field narrows during denoising, from global in early steps that form structure to local in later steps that refine details. We propose AReA, a training-free attention mechanism whose receptive field evolves from coarse to fine, following the denoising progression. In early steps, AReA applies cross-scale coarse attention over downsampled keys and values, capturing global context, while in later steps it applies local attention within progressively shrinking windows. Without additional parameters or training, experiments show that AReA consistently outperforms state-of-the-art training-free baselines across target resolutions and different models, while also being faster and more efficient.
Original Article
View Cached Full Text

Cached at: 10/14/26, 06:52 AM

Paper page - AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers

Source: https://huggingface.co/papers/2610.17227

Abstract

GeneratingimagesbeyondthetrainingresolutionofDiffusionTransformers(DiTs)remainschallenging,asthenumberoftokensgrowsrapidlywithresolution,fullattentionbecomesexpensive,andoutputsoftensufferfromstructuraldistortionsanddegradeddetail.Weidentifyuniformfullattentionasthecommonsourceofbothproblems.Attendingtoalltokensateverystepcausesquadraticcostandexposesthemodeltomoretokensandrelativedistancesbeyondthetrainingrange,pushingthemodeloutofdistribution.Weobservethatattention’seffectivereceptivefieldnarrowsduringdenoising,fromglobalinearlystepsthatformstructuretolocalinlaterstepsthatrefinedetails.WeproposeAReA,atraining-freeattentionmechanismwhosereceptivefieldevolvesfromcoarsetofine,followingthedenoisingprogression.Inearlysteps,AReAappliescross-scalecoarseattentionoverdownsampledkeysandvalues,capturingglobalcontext,whileinlaterstepsitapplieslocalattentionwithinprogressivelyshrinkingwindows.Withoutadditionalparametersortraining,experimentsshowthatAReAconsistentlyoutperformsstate-of-the-arttraining-freebaselinesacrosstargetresolutionsanddifferentmodels,whilealsobeingfasterandmoreefficient.

View arXiv pageView PDFProject pageGitHub2Add to collection

Get this paper in your agent:

hf papers read 2610\.17227

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.17227 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.17227 in a dataset README.md to link it from this page.

Spaces citing this paper1

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles