Appearance Pointers -- Multimodal Region Control of Diffusion Transformers
Summary
Introduces appearance pointers, compact tokens that guide Diffusion Transformers to apply correct appearance cues at specified spatial locations, enabling modality-agnostic localized multimodal control without retraining the base model.
View Cached Full Text
Cached at: 07/22/26, 02:40 AM
Paper page - Appearance Pointers – Multimodal Region Control of Diffusion Transformers
Source: https://huggingface.co/papers/2607.19344
Abstract
Controllableimagegenerationremainschallengingforcreativeprofessionals,whooftenrequirepreciseregionalcontrolovermaterials,objectidentities,andspatialarrangementsthatcannotbereliablyachievedthroughtextpromptingalone.DiffusionTransformers(DiTs)cannativelyingestheterogeneoustokensstemmingfromtextsandimages,buttheylackmechanismsfordeterminingwhereandhowthesetokensshouldinfluencetheoutput.Weintroduceappearancepointers,compacttokensthatguideDiTstowardthecorrectappearancecuesatthecorrectspatiallocationsbyaligningtextorimageinputswithuser-specifiedmasks.Appearancepointersareproducedbyaregioncorrespondencenetworkandrefinedthroughaspatialaggregationmechanism,enablingthemodeltohandlemultipleregionaldescriptionswithoutsignificantlyincreasingtokenload.Ourapproachintroducesthefirstmodality-agnosticinterfaceforlocalizedmultimodalcontrolinaDiTwithoutretrainingthebasemodelfromscratch.Acrossarangeofmetrics,oursinglemodelreachesorsurpassestheperformanceofmodality-specificstateoftheartmethods,offeringasimpleandextensiblepathtowardprecise,region-aware,multimodalguidanceingenerativeimagesynthesis.
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2607\.19344
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.19344 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.19344 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.19344 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
MMDiff: Extending Diffusion Transformers for Multi-Modal Generation
MMDiff extends frozen diffusion transformers into multi-modal generative systems using lightweight decoders, achieving significant improvements in semantic segmentation and other perceptual tasks through multi-timestep feature fusion.
PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models
PerceptionDLM introduces a multimodal diffusion language model that enables parallel region perception via structured attention masking and efficient prompting, achieving faster inference without sacrificing caption quality. Experiments show competitive performance with substantial speed improvements for multi-region perception tasks.
Registers Matter for Pixel-Space Diffusion Transformers
This paper explores the use of register tokens in pixel-space Diffusion Transformers (DiTs), finding they improve feature map quality despite DiTs lacking patch-token outliers. The authors propose Register Guidance, a technique to amplify register contributions for better visual structure.
Exploring Diffusion Transformers for Cross-Modal Augmentation in Multimodal Brain State Decoding
The paper proposes CoMA-DiT, a bidirectional cross-modal Diffusion Transformer for latent augmentation in multimodal brain state decoding, which enhances performance in tasks like auditory attention decoding and emotion recognition by using paired modalities as mutual supervisory signals.
Attention-DP3: Spatially Object-aware 3D Diffusion Policy via Geometry-aligned Attentional Conditioning
Attention-DP3 enhances 3D diffusion policies by incorporating object-level geometric cues through attention to improve stability in cluttered environments, achieving state-of-the-art performance across benchmarks.