PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models

Hugging Face Daily Papers Papers
multimodal diffusion-language-models parallel-decoding region-captioning visual-perception efficient-inference structured-attention

Summary

PerceptionDLM introduces a multimodal diffusion language model that enables parallel region perception via structured attention masking and efficient prompting, achieving faster inference without sacrificing caption quality. Experiments show competitive performance with substantial speed improvements for multi-region perception tasks.

Multimodal large language models (MLLMs) have achieved remarkable progress in visual understanding tasks. However, most existing MLLMs rely on autoregressive generation, which limits their efficiency for perception tasks that require captioning multiple regions. In this work, we propose PerceptionDLM, a multimodal diffusion language model optimized for efficient parallel region perception. Built upon PerceptionDLM-Base, a strong foundational baseline that achieves state-of-the-art performance among open-source diffusion MLLMs, our architecture fully leverages the parallel decoding nature of DLMs. Specifically, we introduce efficient prompting and structured attention masking to enable simultaneous perception of multiple masked regions, allowing the model to generate region descriptions in parallel at both the sequence and token levels. This design significantly improves inference efficiency compared with existing approaches that process regions sequentially. To systematically evaluate the parallelism property of visual perception capability for DLMs, we construct a new Parallel Detailed Localized Captioning Benchmark (ParaDLC-Bench) by scaling the DLC-Bench to include multiple region masks per image, enabling joint evaluation of both caption quality and inference efficiency. Experiments demonstrate that PerceptionDLM maintains competitive performance in region captioning while achieving substantial speed improvements for multi-region perception tasks. Our results highlight the potential of multimodal diffusion language models for efficient, parallel visual perception. To the best of our knowledge, we are the first to achieve parallel region caption and perception by leveraging the advantages of diffusion language models. Code, models, and datasets are released.
Original Article
View Cached Full Text

Cached at: 06/22/26, 05:29 AM

Paper page - PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models

Source: https://huggingface.co/papers/2606.19534

Abstract

PerceptionDLM enables efficient parallel region perception in multimodal diffusion language models through structured attention masking and efficient prompting, achieving faster inference without sacrificing caption quality.

Multimodal large language models(MLLMs) have achieved remarkable progress in visual understanding tasks. However, most existing MLLMs rely on autoregressive generation, which limits their efficiency for perception tasks that require captioning multiple regions. In this work, we proposePerceptionDLM, a multimodal diffusion language model optimized for efficient parallel region perception. Built uponPerceptionDLM-Base, a strong foundational baseline that achieves state-of-the-art performance among open-source diffusion MLLMs, our architecture fully leverages theparallel decodingnature of DLMs. Specifically, we introduce efficient prompting andstructured attention maskingto enable simultaneous perception of multiple masked regions, allowing the model to generate region descriptions in parallel at both the sequence and token levels. This design significantly improvesinference efficiencycompared with existing approaches that process regions sequentially. To systematically evaluate the parallelism property ofvisual perceptioncapability for DLMs, we construct a new Parallel Detailed Localized Captioning Benchmark (ParaDLC-Bench) by scaling the DLC-Bench to include multiple region masks per image, enabling joint evaluation of both caption quality andinference efficiency. Experiments demonstrate thatPerceptionDLMmaintains competitive performance inregion captioningwhile achieving substantial speed improvements for multi-region perception tasks. Our results highlight the potential of multimodaldiffusion language modelsfor efficient, parallelvisual perception. To the best of our knowledge, we are the first to achieve parallel region caption and perception by leveraging the advantages ofdiffusion language models. Code, models, and datasets are released.

View arXiv pageView PDFProject pageGitHub11Add to collection

Get this paper in your agent:

hf papers read 2606\.19534

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper3

#### MSALab/PerceptionDLM Image-Text-to-Text• 9B• Updated3 days ago • 14 • 6 #### MSALab/PerceptionDLM-Base Image-Text-to-Text• 9B• Updated3 days ago • 15 • 4 #### MSALab/LLaDA-8B-Instruct-HF Text Generation• 8B• Updated3 days ago • 13

Datasets citing this paper1

#### MSALab/ParaDLC-Bench Updated3 days ago • 253 • 1

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.19534 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention

arXiv cs.AI

PreDiff-LM proposes a hybrid attention mechanism that preserves causal attention for prompt tokens and bidirectional attention for masked target tokens, enabling adaptation of pretrained autoregressive models for discrete masked diffusion language modeling, achieving improvements in perplexity and downstream tasks over prior diffusion baselines.

Multi-Block Diffusion Language Models

Hugging Face Daily Papers

This paper proposes Multi-Block Diffusion Language Models (MBD-LMs), extending single-block diffusion to concurrent multi-block decoding with improved training strategies like Multi-block Teacher Forcing and an optimized Block Buffer decoding algorithm. Experiments show increased tokens per forward pass and improved accuracy on benchmarks.

Uni-LaDiR: Latent Diffusion Unifies Multimodal Reasoning

arXiv cs.LG

Uni-LaDiR introduces a unified latent diffusion framework for multimodal reasoning, mapping modality-specific thoughts into a shared latent space and using diffusion to generate reasoning steps, achieving improved performance on vision-language benchmarks.