DiffImaginE: Imagine to Verify Entity Types with Diffusio

# DiffImaginE: Imagine to Verify Entity Types with Diffusion
Source: [https://arxiv.org/html/2608.03025](https://arxiv.org/html/2608.03025)
Feng Zhang1,\*, Feiyu Han2,\*, Rongxin Yang3,\*, Yang Liu2, Yancheng Chen2Rui Wang4, Yingguang Yang5, Tian Xueyun2, Chongyang Zhang6,7, Hao Zheng6Xu Kefu3, Congjing Ran8, Fuhai Chen1, Bin Chong3,†1Fuzhou University;2Chinese Academy of Sciences;3Peking University;4Alibaba Group 5University of Science and Technology of China;6Fullive Innovation \(Beijing\) AI Technology Co\., Ltd\.;7Baidu;8Wuhan University \*Equal contribution\.†Corresponding author:[chongbin@pku\.edu\.cn](https://arxiv.org/html/2608.03025v1/mailto:[email protected])

###### Abstract

Multimodal named entity recognition \(MNER\) determines, for each candidate span, whether an entity\-type hypothesis is supported by the joint textual and visual evidence\. Existing imagine\-and\-compare verifiers typically map each\(span,type\)\(\\text\{span\},\\text\{type\}\)pair to a single predicted visual feature and compare it with the observed image representation\. Such deterministic imagination compresses the diverse visual realisations of an entity type into one prototype, making the verifier brittle when the same type appears through substantially different visual cues\. Moreover, the resulting compatibility score lacks a probabilistic interpretation and provides only indirect supervision for rejecting confusable type hypotheses\.

We proposeDiffImaginE, which formulates MNER type verification as conditional latent diffusion inference\. Given span\-localised visual evidence, a type\-conditioned denoiser predicts the noise injected into its standardised latent representation\. The resulting denoising error provides an ELBO\-consistent surrogate for the type\-conditional negative log\-likelihood, enabling different type hypotheses to be compared according to how well they explain the observed evidence\. DiffImaginE preserves a standard multimodal encoder stack and replaces only the deterministic verifier with a classifier\-free\-guided diffusion scorer trained using Min\-SNR weighting\. To bridge generative likelihood estimation and discriminative prediction, we directly supervise per\-type diffusion scores as classification logits, learn to aggregate evidence across noise levels, and use antithetic sampling to reduce Monte\-Carlo comparison variance\. Our theoretical analysis shows that classifier\-free guidance sharpens the induced type posterior and characterises when antithetic pairing yields lower variance at equal denoiser cost\. Experiments on Twitter\-2015 and Twitter\-2017 show that DiffImaginE consistently improves over a matched deterministic ImaginE control under the same encoder, auxiliary objectives, and evaluation protocol, with further support from ablations and paired significance tests\.

## 1Introduction

Named entity recognition \(NER\) on social\-media text is challenging because posts are often short, noisy, and lexically ambiguous\. Multimodal NER \(MNER\) uses accompanying images to resolve such ambiguity\. For example, “Jordan dropped 40” may refer to a person, a brand, or another entity, while the associated image can reveal whether “Jordan” denotes an athlete, an organisation, or a miscellaneous entity\(Moonet al\.,[2018](https://arxiv.org/html/2608.03025#bib.bib17); Zhanget al\.,[2018](https://arxiv.org/html/2608.03025#bib.bib18); Luet al\.,[2018](https://arxiv.org/html/2608.03025#bib.bib19)\)\. A widely adopted formulation therefore enumerates candidate spans and verifies whether each proposed entity type is compatible with the joint text–image context\(Yuet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib20); Chenet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib23); Wanget al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib24)\)\. Under this formulation, verification is central: the encoders provide contextualised multimodal representations, but the final decision depends on the compatibility assigned to each\(span,type\)\(\\text\{span\},\\text\{type\}\)hypothesis\.

Prior work commonly performs verification through*imagination*\. Given a span and candidate type, a predictor synthesises the visual feature expected under that hypothesis, and a comparator measures its agreement with the observed evidence\(Chenet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib23); Xuet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib25)\)\. Although intuitive, this deterministic formulation compresses the diverse visual realisations of a type into a single imagined point\. APERmention may appear as a frontal face, profile, distant figure, or action scene, while anORGmention may be supported by a logo, storefront, uniform, or advertisement\. A single prototype cannot adequately capture such intra\-type variation, particularly for heterogeneous and visually overlapping categories such asMISC\.

Moreover, deterministic compatibility scores provide no explicit likelihood interpretation\. They measure resemblance to one imagined feature, but not how plausibly the observed evidence can be explained under each competing type\. Negative hypotheses therefore receive mainly indirect contrastive pressure, allowing related types to obtain similarly high scores\. A more suitable verifier should instead ask:*under which type hypothesis is the observed span\-localised visual evidence most plausible?*

Diffusion models offer a natural mechanism for this comparison\. A conditional denoiser trained across multiple corruption levels models a distribution of possible observations rather than a single representative point\. Its expected class\-conditional denoising error can be interpreted as a negative variational bound on the conditional log\-likelihood, enabling classification by comparing reconstruction errors across hypotheses\(Liet al\.,[2023](https://arxiv.org/html/2608.03025#bib.bib12); Clark and Jaini,[2023](https://arxiv.org/html/2608.03025#bib.bib13)\)\. However, existing diffusion classifiers mainly operate on full images\. Applying them to MNER is non\-trivial because the scored object is a low\-dimensional, span\-conditioned cross\-modal latent whose scale changes with the jointly trained encoder, while the generative score must ultimately support discriminative span classification and rejection of the non\-entity hypothesis\.

We proposeDiffImaginE, a conditional latent diffusion verifier for MNER\. It preserves the standard text encoder, vision encoder, and span–visual interaction modules, while replacing deterministic imagination with type\-conditioned diffusion scoring\. For each candidate span, cross\-attention extracts span\-localised visual evidence, which is standardised and corrupted with Gaussian noise\. A shared denoiser predicts the injected noise under each candidate type, and the resulting Min\-SNR\-weighted errors define type\-specific verification scores\. A NULL\-conditioned branch further enables classifier\-free guidance\(Ho and Salimans,[2022](https://arxiv.org/html/2608.03025#bib.bib8); Hanget al\.,[2023](https://arxiv.org/html/2608.03025#bib.bib9)\)\. DiffImaginE thus evaluates each hypothesis according to how well its conditional denoising process explains the same observation\.

Because a generative likelihood surrogate is not necessarily an optimal discriminative classifier, we further adapt the scorer in three ways\. We directly supervise per\-type scores as classification logits, learn timestep aggregation weights to emphasise discriminative noise levels, and use antithetic noise pairs to reduce Monte\-Carlo comparison variance when the odd component dominates\. The resulting scores are fused with the original multimodal representations by the final entity classifier\.

DiffImaginE changes only the verification mechanism\. Our matchedImaginEcontrol uses the same encoders, span\-localisation modules, auxiliary objectives, classifier head, optimisation procedure, and decoding protocol, but retains deterministic imagine\-and\-compare verification\. This controlled comparison isolates the effect of replacing single\-point imagination with distributional diffusion scoring \([Section˜4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px1)\)\.

Our contributions are summarised as follows:

- •Diffusion\-based type verification\.We formulate MNER verification as conditional latent diffusion inference, replacing single\-point visual imagination with an ELBO\-based distributional scorer\.
- •Adaptation to multimodal span latents\.We combine latent standardisation, span\- and type\-conditioned denoising, classifier\-free guidance, and Min\-SNR weighting for low\-dimensional span\-localised evidence\.
- •Discriminative and variance\-reduced scoring\.We supervise diffusion scores as type logits, learn their aggregation across timesteps, and derive the condition under which antithetic estimation reduces comparison variance \([Sections˜3\.7](https://arxiv.org/html/2608.03025#S3.SS7)and[3\.9](https://arxiv.org/html/2608.03025#S3.SS9)\)\.
- •Controlled evaluation\.Experiments on Twitter\-2015 and Twitter\-2017 show consistent improvements over a matched deterministic verifier, supported by ablations and paired tests\.

![Refer to caption](https://arxiv.org/html/2608.03025v1/x1.png)Figure 1:DiffImaginE replaces single\-point imagine\-and\-compare verification with type\-conditioned denoising on span\-localised visual evidence: each type is scored by its denoising error, which better separates confusable hypotheses when visual evidence for a type is diverse\.
## 2Preliminaries

We introduce the span\-level MNER formulation, the visual evidence scored by DiffImaginE, and the diffusion\-classifier estimator underlying our verifier\.

### 2\.1Multimodal NER and Span\-Level Type Verification

An MNER instance consists of a token sequence𝒙=\(x1,…,xL\)\\bm\{x\}=\(x\_\{1\},\\dots,x\_\{L\}\)and an image𝒎\\bm\{m\}\. The task is to identify all entity spans and assign each a type from𝒴=\{PER,LOC,ORG,MISC\}\\mathcal\{Y\}=\\\{\\texttt\{PER\},\\texttt\{LOC\},\\texttt\{ORG\},\\texttt\{MISC\}\\\}\. Following the enumerate\-and\-verify formulation, we enumerate candidate spanss=\(i,j\)s=\(i,j\)satisfying1≤i≤j≤L1\\leq i\\leq j\\leq Landj−i\+1≤Wj\-i\+1\\leq W, and classify each into one ofK=\|𝒴\|\+1=5K=\|\\mathcal\{Y\}\|\+1=5labels, including the non\-entity classO\. Thus, the verifier jointly determines whether a candidate is an entity and, if so, its type\.

### 2\.2Span\-Localised Visual Evidence

A text encoder \(RoBERTa\(Liuet al\.,[2019](https://arxiv.org/html/2608.03025#bib.bib28)\)\) produces contextual token states, while a vision encoder \(CLIP\-ViT\(Radfordet al\.,[2021](https://arxiv.org/html/2608.03025#bib.bib29); Dosovitskiyet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib30)\)\) produces global and patch\-level image features\. Both are projected into a shareddd\-dimensional space\. Boundary\-aware pooling yields a span representation𝒔∈ℝd\\bm\{s\}\\in\\mathbb\{R\}^\{d\}, which attends over the image patches to produce span\-localised visual evidence𝒛sv∈ℝd\\bm\{z\}^\{\\mathrm\{v\}\}\_\{s\}\\in\\mathbb\{R\}^\{d\}\(Vaswaniet al\.,[2017](https://arxiv.org/html/2608.03025#bib.bib31); Chenet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib23)\)\. DiffImaginE scores type hypotheses against𝒛sv\\bm\{z\}^\{\\mathrm\{v\}\}\_\{s\}; all upstream encoders and interaction modules are shared with the deterministic baseline\.

### 2\.3Diffusion Models and the Diffusion Classifier

We use a variance\-preserving DDPM\(Hoet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib2); Nichol and Dhariwal,[2021](https://arxiv.org/html/2608.03025#bib.bib3)\)withTTsteps and cosine schedule\{α¯t\}t=0T−1\\\{\\bar\{\\alpha\}\_\{t\}\\\}\_\{t=0\}^\{T\-1\}\. Given a clean latent𝒗∈ℝd\\bm\{v\}\\in\\mathbb\{R\}^\{d\}, the forward process is

𝒗t=α¯t​𝒗\+1−α¯t​ϵ,ϵ∼𝒩​\(0,I\),\\bm\{v\}\_\{t\}=\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\,\\bm\{v\}\+\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}\\,\{\\epsilon\},\\qquad\{\\epsilon\}\\sim\\mathcal\{N\}\(0,I\),\(1\)where a conditional denoiserϵθ​\(𝒗t,𝒄,t\)\{\\epsilon\}\_\{\\theta\}\(\\bm\{v\}\_\{t\},\\bm\{c\},t\)predicts the injected noise under condition𝒄\\bm\{c\}\. The corresponding signal\-to\-noise ratio isSNR​\(t\)=α¯t/\(1−α¯t\)\\mathrm\{SNR\}\(t\)=\\bar\{\\alpha\}\_\{t\}/\(1\-\\bar\{\\alpha\}\_\{t\}\)\.

A diffusion classifier\(Liet al\.,[2023](https://arxiv.org/html/2608.03025#bib.bib12); Clark and Jaini,[2023](https://arxiv.org/html/2608.03025#bib.bib13)\)evaluates class hypothesiskkthrough its expected denoising error:

errk=𝔼t,ϵ​\[w​\(t\)​‖ϵ−ϵθ​\(α¯t​𝒗\+1−α¯t​ϵ,𝒄k,t\)‖22\]\.\\mathrm\{err\}\_\{k\}=\\mathbb\{E\}\_\{t,\{\\epsilon\}\}\\\!\\left\[w\(t\)\\left\\\|\{\\epsilon\}\-\{\\epsilon\}\_\{\\theta\}\\\!\\left\(\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\bm\{v\}\+\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}\{\\epsilon\},\\bm\{c\}\_\{k\},t\\right\)\\right\\\|\_\{2\}^\{2\}\\right\]\.\(2\)With an appropriatew​\(t\)w\(t\),errk\\mathrm\{err\}\_\{k\}corresponds to the negative ELBO oflog⁡pθ​\(𝒗∣𝒄k\)\\log p\_\{\\theta\}\(\\bm\{v\}\\mid\\bm\{c\}\_\{k\}\)up to a class\-independent constant, so lower error indicates a more likely class\(Kingmaet al\.,[2021](https://arxiv.org/html/2608.03025#bib.bib6); Hoet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib2)\)\.

DiffImaginE sets𝒗=𝒛sv\\bm\{v\}=\\bm\{z\}^\{\\mathrm\{v\}\}\_\{s\}and uses the composite condition𝒄k=\(𝒔,𝒆k\)\\bm\{c\}\_\{k\}=\(\\bm\{s\},\\bm\{e\}\_\{k\}\), yielding the denoiserϵθ​\(𝒗t,𝒔,𝒆k,t\)\{\\epsilon\}\_\{\\theta\}\(\\bm\{v\}\_\{t\},\\bm\{s\},\\bm\{e\}\_\{k\},t\)\. The resulting error measures how well typekkexplains the visual evidence associated with spanss\. We estimate the expectation by Monte\-Carlo sampling over\(t,ϵ\)\(t,\{\\epsilon\}\), writingerrk,n\\mathrm\{err\}\_\{k,n\}for an individual estimate anderrk\\mathrm\{err\}\_\{k\}for the aggregate\.

## 3The DiffImaginE Model

### 3\.1Overview

DiffImaginE replaces the deterministic imagine\-and\-compare verifier with the conditional diffusion scorer of[Equation˜2](https://arxiv.org/html/2608.03025#S2.E2)\. For each valid span, span–visual cross\-attention produces𝒛sv\\bm\{z\}^\{\\mathrm\{v\}\}\_\{s\}, which is standardised \([Section˜3\.2](https://arxiv.org/html/2608.03025#S3.SS2)\), scored under allKKtype hypotheses \([Section˜3\.4](https://arxiv.org/html/2608.03025#S3.SS4)\), and fused with textual and visual representations for final classification \([Figure˜2](https://arxiv.org/html/2608.03025#S3.F2)\)\. Reverse imagination and the visual\-relevance gate are inherited unchanged from ImaginE and detailed in the technical appendix\. Padding and invalid spans are masked from all diffusion forwards\.

![Refer to caption](https://arxiv.org/html/2608.03025v1/x2.png)Figure 2:DiffImaginE pipeline: cross\-attention yields span representationssand span\-localised visual evidencezz; the diffusion verifier scores each type by type\-conditioned denoising \(with a NULL branch for classifier\-free guidance\), and the classifier fuses these scores with multimodal features for the final prediction\.
### 3\.2Latent Standardisation

The signal scale of𝒛sv\\bm\{z\}^\{\\mathrm\{v\}\}\_\{s\}is set by the encoder, not by the diffusion schedule, so a cosine schedule calibrated for unit\-variance data is mismatched on a raw, low\-dimensional cross\-modal latent\. Before diffusion we estimate a per\-dimension mean𝝁\\bm\{\\mu\}and standard deviation𝝈\\bm\{\\sigma\}of𝒛sv\\bm\{z\}^\{\\mathrm\{v\}\}\_\{s\}over a calibration set of training batches and operate the diffusion process on the standardised latent𝒗=\(𝒛sv−𝝁\)⊘𝝈\\bm\{v\}=\(\\bm\{z\}^\{\\mathrm\{v\}\}\_\{s\}\-\\bm\{\\mu\}\)\\oslash\\bm\{\\sigma\}, mirroring the latent scaling of latent diffusion models\(Rombachet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib10)\)\. Because the encoder drifts during joint training,\(𝝁,𝝈\)\(\\bm\{\\mu\},\\bm\{\\sigma\}\)are re\-estimated after the denoiser warmup and periodically thereafter, so the SNR curve tracks the live encoder\. Unlike image diffusion, the scored object here is add\-dimensional span summary rather than a spatial field, so miscalibrated scale directly distorts the relative noise levels at which types are compared\. We treat standardisation as a prerequisite for stable scoring and isolate its effect in theno\_latent\_normablation\.

### 3\.3Type\-Conditioned Denoiser

The denoiserϵθ​\(𝒗t,𝒔,𝒆k,t\)\{\\epsilon\}\_\{\\theta\}\(\\bm\{v\}\_\{t\},\\bm\{s\},\\bm\{e\}\_\{k\},t\)is an adaptive\-LayerNorm \(AdaLN\) Transformer/MLP block\(Peebles and Xie,[2023](https://arxiv.org/html/2608.03025#bib.bib11)\): the timestep embedding and the type embedding𝒆k\\bm\{e\}\_\{k\}are mapped to per\-layer scale/shift parameters that modulate the normalised activations, while the span representation𝒔\\bm\{s\}enters as an input feature\. The type\-embedding table hasK\+1K\+1rows:KKrows for the supervised classes \(includingO\) and one extra NULL row𝒆∅\\bm\{e\}\_\{\\varnothing\}, the unconditional embedding used for classifier\-free guidance \([Section˜3\.4](https://arxiv.org/html/2608.03025#S3.SS4)\)\. The NULL row is a guidance device, distinct from theOclass, which is a genuine type hypothesis the scorer must rank\. The AdaLN projection is half\-zero\-initialised so the type condition has a small but non\-zero effect from the first step, avoiding a long regime in which the conditioning signal is silenced\.

### 3\.4Classifier\-Free\-Guided ELBO Scoring

For spansswith standardised evidence𝒗\\bm\{v\}, the per\-type error of[Equation˜2](https://arxiv.org/html/2608.03025#S2.E2)is estimated by Monte\-Carlo over timesteps and noise\. Following classifier\-free guidance\(Ho and Salimans,[2022](https://arxiv.org/html/2608.03025#bib.bib8); Dhariwal and Nichol,[2021](https://arxiv.org/html/2608.03025#bib.bib7)\), we also evaluate the denoiser under the NULL condition𝒆∅\\bm\{e\}\_\{\\varnothing\}and form a guided score with guidance scaleg≥0g\\geq 0:

err∅=𝔼t,ϵ​\[w​\(t\)​‖ϵ−ϵθ​\(𝒗t,𝒔,𝒆∅,t\)‖22\],\\mathrm\{err\}\_\{\\varnothing\}=\\mathbb\{E\}\_\{t,\{\\epsilon\}\}\\\!\\bigl\[w\(t\)\\,\\\|\{\\epsilon\}\-\{\\epsilon\}\_\{\\theta\}\(\\bm\{v\}\_\{t\},\\bm\{s\},\\bm\{e\}\_\{\\varnothing\},t\)\\\|\_\{2\}^\{2\}\\bigr\],\(3\)scorek\(g\)\\displaystyle\\mathrm\{score\}\_\{k\}^\{\(g\)\}=−\(errk−g​\(err∅−errk\)\)\\displaystyle=\-\\bigl\(\\mathrm\{err\}\_\{k\}\-g\\,\(\\mathrm\{err\}\_\{\\varnothing\}\-\\mathrm\{err\}\_\{k\}\)\\bigr\)\(4\)=−\(1\+g\)​errk\+g​err∅\.\\displaystyle=\-\(1\+g\)\\,\\mathrm\{err\}\_\{k\}\+g\\,\\mathrm\{err\}\_\{\\varnothing\}\.The same noise sampleϵ\{\\epsilon\}is shared across allK\+1K\{\+\}1hypotheses at each\(t,s\)\(t,s\), so comparisons differ only through the denoiser response\. Training draws timesteps by stratified sampling overMMstrata of\[0,T\)\[0,T\); evaluation stacks a fixed set ofNNtimesteps in\[tlo,thi\]\[t\_\{\\mathrm\{lo\}\},t\_\{\\mathrm\{hi\}\}\]into one batched forward\. The resulting errorserrk,n\(g\)=\(1\+g\)​errk,n−g​err∅,n\\mathrm\{err\}^\{\(g\)\}\_\{k,n\}=\(1\+g\)\\mathrm\{err\}\_\{k,n\}\-g\\mathrm\{err\}\_\{\\varnothing,n\}are aggregated asscorek=−∑nw​\(tn/T\)​errk,n\(g\)\\mathrm\{score\}\_\{k\}=\-\\sum\_\{n\}w\(t\_\{n\}/T\)\\mathrm\{err\}^\{\(g\)\}\_\{k,n\}\. Becauseerr∅\\mathrm\{err\}\_\{\\varnothing\}is type\-independent,ggacts as a posterior\-sharpening factor in the idealised ELBO regime \([Proposition˜1](https://arxiv.org/html/2608.03025#Thmproposition1)\)\.

### 3\.5Min\-SNR Weighting and the Denoising Objective

The denoiser is trained with a denoising score\-matching loss weighted per timestep by the Min\-SNR\-γ\\gammarule\(Hanget al\.,[2023](https://arxiv.org/html/2608.03025#bib.bib9)\), which equalises the gradient contribution across noise levels by clipping the SNR weight atγ\\gamma:

ℒdiff\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{diff\}\}=𝔼t,ϵ​\[wγ​\(t\)​‖ϵ−ϵθ​\(𝒗t,𝒔,𝒆y,t\)‖22\],\\displaystyle=\\mathbb\{E\}\_\{t,\{\\epsilon\}\}\\\!\\bigl\[\\,w\_\{\\gamma\}\(t\)\\,\\\|\{\\epsilon\}\-\{\\epsilon\}\_\{\\theta\}\(\\bm\{v\}\_\{t\},\\bm\{s\},\\bm\{e\}\_\{y\},t\)\\\|\_\{2\}^\{2\}\\,\\bigr\],\(5\)wγ​\(t\)\\displaystyle w\_\{\\gamma\}\(t\)=min⁡\(SNR​\(t\),γ\)SNR​\(t\),\\displaystyle=\\frac\{\\min\(\\mathrm\{SNR\}\(t\),\\gamma\)\}\{\\mathrm\{SNR\}\(t\)\},whereyyis the gold type of the span \(Ois a valid supervised type, so the score onOis well\-defined at test time\)\. With probabilitypcfp\_\{\\mathrm\{cf\}\}the gold condition𝒆y\\bm\{e\}\_\{y\}is replaced by𝒆∅\\bm\{e\}\_\{\\varnothing\}during training, which fits the unconditional denoiser branch that classifier\-free guidance contrasts against in[Equation˜4](https://arxiv.org/html/2608.03025#S3.E4)\. The classifier scores andℒdiff\\mathcal\{L\}\_\{\\mathrm\{diff\}\}are produced by a single fused denoiser forward per Monte\-Carlo step\.

### 3\.6Training Objectives

The verifier and encoder are trained jointly\. The primary term is class\-weighted cross\-entropyℒtask\\mathcal\{L\}\_\{\\mathrm\{task\}\}, with a lower weight forOto mitigate span\-enumeration imbalance\. The diffusion path addsℒdiff\\mathcal\{L\}\_\{\\mathrm\{diff\}\}\([Equation˜5](https://arxiv.org/html/2608.03025#S3.E5)\) and a score\-level contrastive lossℒico\\mathcal\{L\}\_\{\\mathrm\{ico\}\}over entity spans within the evaluation timestep window\. Reverse\-imagination alignment, VICReg regularisation\(Bardeset al\.,[2021](https://arxiv.org/html/2608.03025#bib.bib33)\), BIO supervision, cross\-modal InfoNCE\(Oordet al\.,[2018](https://arxiv.org/html/2608.03025#bib.bib32); Radfordet al\.,[2021](https://arxiv.org/html/2608.03025#bib.bib29)\), grounding, type ranking, and hard\-example objectives are inherited by both DiffImaginE and the deterministic control\. Their definitions and weights are deferred to the technical appendix, ensuring that the main comparison isolates the verifier\.

### 3\.7From a Generative Score to a Discriminative Scorer

A bare ELBO score is a generative quantity used as a discriminative feature, which is known to be suboptimal as a classifier\. DiffImaginE closes this gap with three independent components\.

#### Likelihood\-based classification objective\.

We supervise the per\-type scores*directly*: treatingscorek\\mathrm\{score\}\_\{k\}as a logit,ℒclf\\mathcal\{L\}\_\{\\mathrm\{clf\}\}is a cross\-entropy over allKKtypes \(includingO\) with a learnable temperatureτclf\\tau\_\{\\mathrm\{clf\}\},

ℒclf=CE​\(softmaxk​\(scorek/τclf\),y\),\\mathcal\{L\}\_\{\\mathrm\{clf\}\}=\\mathrm\{CE\}\\\!\\bigl\(\\mathrm\{softmax\}\_\{k\}\(\\mathrm\{score\}\_\{k\}/\\tau\_\{\\mathrm\{clf\}\}\),\\,y\\bigr\),\(6\)which aligns the ordering of the scores with the gold type during training, so the scores are trained as classification logits rather than read off a frozen generative model\(Xianet al\.,[2024](https://arxiv.org/html/2608.03025#bib.bib14)\)\. We parameteriseτclf=exp⁡\(θτ\)\\tau\_\{\\mathrm\{clf\}\}=\\exp\(\\theta\_\{\\tau\}\)to keep it positive, initialised so thatτclf=0\.1\\tau\_\{\\mathrm\{clf\}\}=0\.1\.

#### Learnable timestep aggregation\.

The score\-side weightingw​\(t\)w\(t\)of[Equation˜2](https://arxiv.org/html/2608.03025#S2.E2)is left unspecified by the ELBO up to the variational choice; we instantiate it as a learned aggregation\. Instead of a uniform mean over the active timesteps, a tiny zero\-initialised network maps the normalised timestept/Tt/Tto a logit, and the per\-timestep errors are aggregated with the resulting softmax weights; zero initialisation makes the aggregation identical to a uniform mean at start\-up, after which the model up\-weights the most discriminative noise levels\(Jeonget al\.,[2025](https://arxiv.org/html/2608.03025#bib.bib15)\)\. This weight is distinct from the Min\-SNR loss weightwγ​\(t\)w\_\{\\gamma\}\(t\)of[Equation˜5](https://arxiv.org/html/2608.03025#S3.E5): the former shapes the evaluation\-time score, whilewγ​\(t\)w\_\{\\gamma\}\(t\)acts only onℒdiff\\mathcal\{L\}\_\{\\mathrm\{diff\}\}\.

#### Antithetic variance reduction\.

The Monte\-Carlo noise is drawn in same\-timestep pairs\(ϵ,−ϵ\)\(\{\\epsilon\},\-\{\\epsilon\}\)\. Because the type\-discriminative part of the score is, to leading order, odd inϵ\{\\epsilon\}and shared across types, antithetic pairing lowers the variance of the per\-timestep type comparison relative to an i\.i\.d\. estimator of equal cost when the odd error component dominates\(Jiaet al\.,[2026](https://arxiv.org/html/2608.03025#bib.bib16)\);[Proposition˜2](https://arxiv.org/html/2608.03025#Thmproposition2)makes the condition precise\. Finally, a score LayerNorm with a learnable temperature rescales theKKscore channels so they are not dwarfed by thedd\-dimensional representations when fused in the classifier\.

### 3\.8Decoding

At test time, weighted\-interval\-scheduling dynamic programming selects the maximum\-confidence non\-overlapping span set inO​\(S​log⁡S\)O\(S\\log S\)time, whereS=O​\(L​W\)S=O\(LW\)\. A span is accepted only when its best entity logit exceeds theOlogit and its diffusion marginmaxk≠O⁡scorek−scoreO\\max\_\{k\\neq\\texttt\{O\}\}\\mathrm\{score\}\_\{k\}\-\\mathrm\{score\}\_\{\\texttt\{O\}\}exceeds an abstention threshold\. The guidance scale and both margins are tuned on the development set and reused unchanged for strict testF1F\_\{1\}; greedy decoding is retained only as an ablation\.

### 3\.9Theoretical Analysis

We state two results about the scorer; both are proved in the technical appendix\. The first relates guided scoring to a tempered posterior under an idealised uniform type prior; in training, the empirical dominance ofOis handled by class weighting inℒtask\\mathcal\{L\}\_\{\\mathrm\{task\}\}and the classifier head rather than inside the score layer\. The second compares antithetic and i\.i\.d\. estimators of the type\-difference score\.

###### Assumption 1\(ELBO\-consistent weighting\)\.

We adopt the standard diffusion\-classifier modelling assumption\(Hoet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib2); Kingmaet al\.,[2021](https://arxiv.org/html/2608.03025#bib.bib6); Liet al\.,[2023](https://arxiv.org/html/2608.03025#bib.bib12)\): the timestep weightingw​\(t\)w\(t\)in[Equation˜2](https://arxiv.org/html/2608.03025#S2.E2)is the variational weighting under whicherrk\\mathrm\{err\}\_\{k\}equals the negative variational bound onlog⁡pθ​\(𝒗∣𝒔,𝒆k\)\\log p\_\{\\theta\}\(\\bm\{v\}\\mid\\bm\{s\},\\bm\{e\}\_\{k\}\), and the gap between this bound and the exact log\-likelihood, together with the prior and reconstruction terms and the entropy of the forward process, is folded into a constantCCindependent of the typekk\. We thus treaterrk=−log⁡pθ​\(𝒗∣𝒔,𝒆k\)\+C\\mathrm\{err\}\_\{k\}=\-\\log p\_\{\\theta\}\(\\bm\{v\}\\mid\\bm\{s\},\\bm\{e\}\_\{k\}\)\+Cas the operative model rather than an exact identity\.

###### Proposition 1\(Guided score is a tempered posterior\)\.

Under[˜1](https://arxiv.org/html/2608.03025#Thmassumption1)and a uniform prior over theKKtypes, the unguided scorescorek=−errk\\mathrm\{score\}\_\{k\}=\-\\mathrm\{err\}\_\{k\}satisfiessoftmaxk​\(scorek\)=pθ​\(𝐞k∣𝐯,𝐬\)\\mathrm\{softmax\}\_\{k\}\(\\mathrm\{score\}\_\{k\}\)=p\_\{\\theta\}\(\\bm\{e\}\_\{k\}\\mid\\bm\{v\},\\bm\{s\}\), soarg​maxk⁡scorek\\operatorname\*\{arg\\,max\}\_\{k\}\\mathrm\{score\}\_\{k\}is the Bayes\-optimal type andsoftmaxk​\(scorek/τ\)\\mathrm\{softmax\}\_\{k\}\(\\mathrm\{score\}\_\{k\}/\\tau\)is a temperature\-τ\\tauposterior estimate\. For guided scoring, with any guidance scaleg≥0g\\geq 0and temperatureτ\>0\\tau\>0, the classifier\-free\-guided score of[Equation˜4](https://arxiv.org/html/2608.03025#S3.E4)satisfies

softmaxk​\(scorek\(g\)/τ\)\\displaystyle\\mathrm\{softmax\}\_\{k\}\\\!\\bigl\(\\mathrm\{score\}\_\{k\}^\{\(g\)\}/\\tau\\bigr\)=softmaxk​\(scorek/τ1\+g\)\\displaystyle=\\mathrm\{softmax\}\_\{k\}\\\!\\bigl\(\\mathrm\{score\}\_\{k\}/\\tfrac\{\\tau\}\{1\+g\}\\bigr\)\(7\)∝pθ​\(𝒆k∣𝒗,𝒔\)\(1\+g\)/τ\.\\displaystyle\\propto p\_\{\\theta\}\(\\bm\{e\}\_\{k\}\\mid\\bm\{v\},\\bm\{s\}\)^\{\(1\+g\)/\\tau\}\.That is, guidance and temperature combine into a single effective exponent\(1\+g\)/τ\(1\+g\)/\\tauon the posterior: increasinggg\(or loweringτ\\tau\) sharpens the type distribution, while decreasinggg\(or raisingτ\\tau\) flattens it, with no effect on thearg​max\\operatorname\*\{arg\\,max\}\. Atτ=1\\tau=1the guided score is exactly the posterior raised to the power1\+g1\+g\.

The proof is the diffusion\-classifier ELBO identity followed by the observation thaterr∅\\mathrm\{err\}\_\{\\varnothing\}is type\-independent and hence cancels inside the softmax, leaving an exact rescaling of the logits by1\+g1\+g\. This reading motivates the dev\-tuning grid forggand the choice of−errk\-\\mathrm\{err\}\_\{k\}as the initial logit parameterisation in[Equation˜6](https://arxiv.org/html/2608.03025#S3.E6)\. We next evaluate whether the resulting scorer improves MNER in practice\.

###### Assumption 2\(Square\-integrable, symmetric noise\)\.

The noiseϵ∼𝒩​\(0,I\)\{\\epsilon\}\\sim\\mathcal\{N\}\(0,I\)is symmetric \(ϵ=d−ϵ\{\\epsilon\}\\stackrel\{\{\\scriptstyle d\}\}\{\{=\}\}\-\{\\epsilon\}\), and for a fixed timestepttand spanssthe per\-sample squared errorsϕk​\(ϵ\)=‖ϵθ​\(α¯t​𝒗\+1−α¯t​ϵ,𝒔,𝒆k,t\)−ϵ‖22\\phi\_\{k\}\(\{\\epsilon\}\)=\\\|\{\\epsilon\}\_\{\\theta\}\(\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\bm\{v\}\+\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}\\,\{\\epsilon\},\\bm\{s\},\\bm\{e\}\_\{k\},t\)\-\{\\epsilon\}\\\|\_\{2\}^\{2\}are square\-integrable for everykk\.

###### Proposition 2\(Antithetic variance reduction\)\.

Fixttand a type pair\(k,j\)\(k,j\), and letf​\(ϵ\)=ϕk​\(ϵ\)−ϕj​\(ϵ\)f\(\{\\epsilon\}\)=\\phi\_\{k\}\(\{\\epsilon\}\)\-\\phi\_\{j\}\(\{\\epsilon\}\)be the single\-sample type\-difference, with even/odd partsfe​\(ϵ\)=12​\(f​\(ϵ\)\+f​\(−ϵ\)\)f\_\{e\}\(\{\\epsilon\}\)=\\tfrac\{1\}\{2\}\(f\(\{\\epsilon\}\)\+f\(\-\{\\epsilon\}\)\)andfo​\(ϵ\)=12​\(f​\(ϵ\)−f​\(−ϵ\)\)f\_\{o\}\(\{\\epsilon\}\)=\\tfrac\{1\}\{2\}\(f\(\{\\epsilon\}\)\-f\(\-\{\\epsilon\}\)\)\. Under[˜2](https://arxiv.org/html/2608.03025#Thmassumption2), the antithetic estimatorD^anti=12​\(f​\(ϵ\)\+f​\(−ϵ\)\)\\widehat\{D\}^\{\\,\\mathrm\{anti\}\}=\\tfrac\{1\}\{2\}\(f\(\{\\epsilon\}\)\+f\(\-\{\\epsilon\}\)\)and the two\-sample i\.i\.d\. estimatorD^iid=12​\(f​\(ϵ\(1\)\)\+f​\(ϵ\(2\)\)\)\\widehat\{D\}^\{\\,\\mathrm\{iid\}\}=\\tfrac\{1\}\{2\}\(f\(\{\\epsilon\}^\{\(1\)\}\)\+f\(\{\\epsilon\}^\{\(2\)\}\)\)use the same number of denoiser evaluations and are both unbiased for𝔼​\[f\]=errk−errj\\mathbb\{E\}\[f\]=\\mathrm\{err\}\_\{k\}\-\\mathrm\{err\}\_\{j\}, with

Var​\(D^anti\)\\displaystyle\\mathrm\{Var\}\\bigl\(\\widehat\{D\}^\{\\,\\mathrm\{anti\}\}\\bigr\)=Var​\(fe\),\\displaystyle=\\mathrm\{Var\}\(f\_\{e\}\),\(8\)Var​\(D^iid\)\\displaystyle\\mathrm\{Var\}\\bigl\(\\widehat\{D\}^\{\\,\\mathrm\{iid\}\}\\bigr\)=12​\(Var​\(fe\)\+Var​\(fo\)\)\.\\displaystyle=\\tfrac\{1\}\{2\}\\bigl\(\\mathrm\{Var\}\(f\_\{e\}\)\+\\mathrm\{Var\}\(f\_\{o\}\)\\bigr\)\.HenceVar​\(D^anti\)≤Var​\(D^iid\)\\mathrm\{Var\}\(\\widehat\{D\}^\{\\,\\mathrm\{anti\}\}\)\\leq\\mathrm\{Var\}\(\\widehat\{D\}^\{\\,\\mathrm\{iid\}\}\)if and only ifVar​\(fe\)≤Var​\(fo\)\\mathrm\{Var\}\(f\_\{e\}\)\\leq\\mathrm\{Var\}\(f\_\{o\}\): the antithetic estimator achieves lower variance precisely when the odd component carries at least as much variance as the even one\. A first\-order expansion of the denoiser aboutϵ=0\{\\epsilon\}=0shows the odd part offfis dominated by−2​⟨ϵ,𝐚k−𝐚j⟩\-2\\langle\{\\epsilon\},\\,\\bm\{a\}\_\{k\}\-\\bm\{a\}\_\{j\}\\rangle, with𝐚k=ϵθ​\(α¯t​𝐯,𝐬,𝐞k,t\)\\bm\{a\}\_\{k\}=\{\\epsilon\}\_\{\\theta\}\(\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\bm\{v\},\\bm\{s\},\\bm\{e\}\_\{k\},t\)\. The criterion therefore holds when the inter\-type gap𝐚k−𝐚j\\bm\{a\}\_\{k\}\-\\bm\{a\}\_\{j\}is large relative to the denoiser’s local sensitivity toϵ\{\\epsilon\}\. Proofs and the termwise aggregation argument are in the technical appendix\.

## 4Experiments

Having specified the scorer \([Section˜3](https://arxiv.org/html/2608.03025#S3)\) and stated its theoretical properties \([Section˜3\.9](https://arxiv.org/html/2608.03025#S3.SS9)\), we ask whether diffusion\-based type verification improves MNER over deterministic imagination and whether the ablations match those properties\. We evaluate on Twitter\-2015 and Twitter\-2017; the protocol is summarised in[Section˜4\.5](https://arxiv.org/html/2608.03025#S4.SS5)\.

### 4\.1Datasets

We evaluate on the two standard Twitter\-MNER benchmarks \([Table˜1](https://arxiv.org/html/2608.03025#S4.T1)\)\. The text splits follow the canonical UMT release\(Yuet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib20)\); the accompanying images follow the HVPNeT distribution\(Chenet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib23)\)\. The data\-preparation pipeline pins the upstream mirror to a fixed commit and verifies a golden SHA\-256 of every split file, so any silent drift raises an error rather than changing the numbers\.

Table 1:Datasets used in the protocol\. Entity types arePER/LOC/ORG/MISC\.
### 4\.2Main Results

Table 2:Main results: span\-level strict precision \(P\), recall \(R\), andF1F\_\{1\}\(%\) on the two Twitter\-MNER benchmarks\. Published baselines are cited from their original papers; a dash marks a value not reported in the source\. ImaginE and DiffImaginE \(below the rule\) are, respectively, our matched deterministic control and our full model, which differ only in the verifier\. DiffImaginE \(last row\) is shown in bold; the best value in each column is likewise in bold\.[Table˜2](https://arxiv.org/html/2608.03025#S4.T2)reports strict precision/recall/F1F\_\{1\}on both benchmarks for published baselines, the matched ImaginE control, and DiffImaginE\. The primary contrast is DiffImaginE vs\. ImaginE \([Section˜4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px1)\); per\-typeF1F\_\{1\}and a paired test appear in[Table˜3](https://arxiv.org/html/2608.03025#S4.T3)\.

On Twitter\-2015, DiffImaginE improves over ImaginE by\+1\.73\+1\.73strictF1F\_\{1\}\(77\.1777\.17vs\.75\.4475\.44\), driven mainly by higher precision \(77\.4177\.41vs\.76\.3276\.32\) and recall \(76\.9376\.93vs\.74\.5874\.58\)\. On Twitter\-2017, swapping the ImaginE verifier for the diffusion scorer raises strictF1F\_\{1\}from87\.7287\.72to88\.4488\.44\(\+0\.72\+0\.72\), with precision rising from86\.6086\.60to87\.7287\.72and recall from88\.8688\.86to89\.1689\.16\. The paired per\-type test in[Table˜3](https://arxiv.org/html/2608.03025#S4.T3)rejects the null atp=0\.032p=0\.032, so the gain is statistically significant at the0\.050\.05level\. Because DiffImaginE and ImaginE share the same encoder and training recipe \([Section˜4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px1)\), we attribute the improvement to the verifier\. DiffImaginE also reaches the best listedF1F\_\{1\}on both datasets \(77\.1777\.17on Twitter\-2015 and88\.4488\.44on Twitter\-2017\); published baselines use different encoders and fusion designs and are shown for context only\. The larger Twitter\-2015 margin suggests that diffusion scoring helps most when visual evidence is noisier and single\-point imagination is brittle\.

### 4\.3Per\-Type Results

[Table˜3](https://arxiv.org/html/2608.03025#S4.T3)breaks down the Twitter\-2017 contrast from[Section˜4\.2](https://arxiv.org/html/2608.03025#S4.SS2)\. The paired test compares DiffImaginE and ImaginE on the same training seeds; it rejects the null atp=0\.032p=0\.032\. Gains concentrate onPER\(92\.7692\.76to93\.9193\.91\), where faces provide strong visual cues, and onORG\(85\.4485\.44to86\.2486\.24\), where logos and brand imagery help disambiguation\.LOCis unchanged \(87\.1287\.12vs\.87\.0887\.08\), andMISCimproves only slightly \(75\.0675\.06to75\.8475\.84\), remaining the hardest type because of high visual and lexical diversity\. The per\-type pattern is consistent with the diffusion scorer exploiting diverse visual evidence rather than a single imagined feature vector\.

Table 3:Per\-type strictF1F\_\{1\}\(↑\\uparrow\) on Twitter\-2017 and the paired DiffImaginE\-vs\-ImaginE test\. DiffImaginE \(last row\) is shown in bold; the best value per type is likewise in bold\.
### 4\.4Qualitative Case Study

[Figure˜3](https://arxiv.org/html/2608.03025#S4.F3)visualises how DiffImaginE maps localised visual evidence to a type decision\. The input establishes disambiguating comic context for the person\-like mention “Donald Duck\.” The span–visual attention \(SVA\) panel answers*where*the model looks: the shared encoder grounds the span in character and comic regions, but attention does not assign a label\. The prediction rows hold this evidence constant and isolate the verifier: ImaginE’s single imagined\-feature match returnsPER, whereas DiffImaginE recoversMISC\. The five\-cell bar answers*which type*the evidence supports\. Each cell schematically gives the normalised contribution from one fixed noise level to the aggregatedMISC\-over\-PERmargin; darker blue means stronger positive support\. Learned weighting combines these probes into aMISC\-favouring signal that helps the fused classifier overcome the person\-name prior\. Together, the panels expose two advantages: multi\-noise validation across complementary signal\-to\-noise regimes and likelihood\-based comparison of how well competing type\-conditioned denoisers explain the observed latent\.

![Refer to caption](https://arxiv.org/html/2608.03025v1/x3.png)Figure 3:From evidence localisation to type verification\. The input supplies disambiguating context, SVA shows where mention\-relevant evidence is localised, and the prediction rows contrast single\-point ImaginE with DiffImaginE\. The five cells schematically show fixed\-noise contributions to theMISC\-over\-PERmargin; learned aggregation produces the final verification signal, with darker blue denoting stronger positive support\.Table 4:Ablation results on Twitter\-2017 \(strictF1F\_\{1\},↑\\uparrow\)\. Each variant is the mean over three seeds \(42/43/44\)\. Bold marks the reference configuration \(main\) and the highestF1F\_\{1\}among variants\.Unless noted, ablation and budget\-sweep rows are means over three seeds, whereas[Table˜2](https://arxiv.org/html/2608.03025#S4.T2)reports the single selected operating point \(EMA weights at the best\-dev threshold\); the two are therefore not directly comparable digit\-for\-digit, which also explains whyno\_diffusion\(87\.7187\.71\) and the ImaginE control in[Table˜2](https://arxiv.org/html/2608.03025#S4.T2)\(87\.7287\.72\) differ by0\.010\.01\. Removing the diffusion verifier entirely \(no\_diffusion\) causes the largest drop, from88\.7888\.78to87\.7187\.71\(−1\.07\-1\.07F1F\_\{1\}\), confirming that the gain comes from the scorer rather than the shared stack\. Score normalisation \(88\.0988\.09\), Min\-SNR weighting \(88\.1088\.10\), and the antithetic estimator \(88\.1488\.14\) each cost about0\.60\.6to0\.70\.7F1F\_\{1\}when removed, matching the roles identified in[Sections˜3\.7](https://arxiv.org/html/2608.03025#S3.SS7)and[3\.9](https://arxiv.org/html/2608.03025#S3.SS9); classifier\-free guidance \(88\.3888\.38\), learnable timestep aggregation \(88\.4288\.42\), and the contrastive score termℒico\\mathcal\{L\}\_\{\\mathrm\{ico\}\}\(88\.2188\.21\) matter less on Twitter\-2017\. we treat this as seed noise on one dataset and keep both terms for their benefit on Twitter\-2015 and in per\-type behaviour\. The technical appendix reports the same ablation variants on Twitter\-2015 over the same seeds, whereno\_diffusionagain yields the largest drop\.

Table 5:Evaluation\-budget sweep on Twitter\-2017: strictF1F\_\{1\}as a function of the number of evaluation timestepsNNand of the diffusion warmup length\. The best value in each sweep row is in bold\.The evaluation\-budget sweep \([Table˜5](https://arxiv.org/html/2608.03025#S4.T5)\) shows that strictF1F\_\{1\}is essentially flat in the number of evaluation timesteps \(between88\.4788\.47and88\.7188\.71forNNfrom11to2020\), so a small Monte\-Carlo budget already captures most of the score signal and the verifier can be run cheaply at test time\. The warmup\-length sweep favours a moderate schedule, with ten warmup epochs \(88\.5888\.58\) ahead of five \(88\.1988\.19\); together with the ablation grid, these results trace the empirical gains back to the diffusion scorer and to the training choices analysed in[Sections˜3](https://arxiv.org/html/2608.03025#S3),[3\.7](https://arxiv.org/html/2608.03025#S3.SS7)and[3\.9](https://arxiv.org/html/2608.03025#S3.SS9)\. At inference, verifier cost scales linearly with the number of candidate spans, types scored, and evaluation timesteps; antithetic evaluation doubles the denoiser forwards relative to a single\-noise draw but stabilises type ranking, which is why[Table˜5](https://arxiv.org/html/2608.03025#S4.T5)reports similarF1F\_\{1\}for smallNN\. Seed\-controlled mechanism plots in the technical appendix show that gold\-type score channels separate entity from non\-entity spans, that mid\-range noise yields the lowest reconstruction error for the correct type, and that antithetic pairing reduces score variance at fixed forward budget\.

### 4\.5Experimental Setup

All experiments follow a fixed and reproducible protocol\.

#### Baselines\.

We report two baseline families\.*Published baselines*include representative MNER methods: UMT\(Yuet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib20)\), co\-attention\(Zhanget al\.,[2018](https://arxiv.org/html/2608.03025#bib.bib18)\), OCSGA\(Wuet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib21)\), RpBERT\(Sunet al\.,[2021](https://arxiv.org/html/2608.03025#bib.bib22)\), HVPNeT\(Chenet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib23)\), ITA\(Wanget al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib24)\), and MAF\(Xuet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib25)\)\. Our control baseline,*ImaginE*, is a deterministic reimplementation of imagine\-and\-compare verification\. It shares the encoder stack, auxiliary objectives, classifier head, and regularisation with DiffImaginE, including R\-Drop\(Wuet al\.,[2021](https://arxiv.org/html/2608.03025#bib.bib34)\)and augmentation\-aware consistency, and differs only in the verifier\. This controlled comparison isolates the effect of diffusion\-based scoring\.

#### Metrics\.

We report span\-level strict\-match precision, recall, andF1F\_\{1\}, where a prediction is correct only when both its boundaries and type match the gold annotation\. Per\-typeF1F\_\{1\}is also reported, with strictF1F\_\{1\}as the primary metric\. Evaluation uses a fixed Monte\-Carlo seed, making repeated evaluation of the same checkpoint deterministic\. Ablations and matched comparisons use training seeds42/43/4442/43/44and paired tests with multiple\-comparison correction\. The main table reports the EMA checkpoint and threshold selected on the development set, while ablations and budget analyses report averages across seeds\.

#### Implementation\.

The model uses RoBERTa\-base and CLIP\-ViT\-B/32 projected to a shared widthdd\. The diffusion verifier adopts a cosine schedule withT=1000T=1000,MMstratified training samples,NNevaluation timesteps within a selected window, classifier\-free dropoutpcfp\_\{\\mathrm\{cf\}\}, guidance scalegg, and Min\-SNR clipγ\\gamma\. Training uses AdamW\(Loshchilov and Hutter,[2017](https://arxiv.org/html/2608.03025#bib.bib37)\), separate learning rates for pretrained and newly introduced modules, linear warmup, AMP, EMA, and distributed data parallelism\. Antithetic sampling uses paired noise\(ϵ,−ϵ\)\(\{\\epsilon\},\-\{\\epsilon\}\): it preserves the training budget but increases evaluation fromNNto2​N2Ndenoiser forwards, and can be disabled for anNN\-forward budget\. Diffusion hyperparameters and decoding margins are selected on the development set and fixed for test evaluation; the full search space is provided in the technical appendix\.

## 5Related Work

Prior MNER systems improve encoding and fusion through co\-attention\(Moonet al\.,[2018](https://arxiv.org/html/2608.03025#bib.bib17); Zhanget al\.,[2018](https://arxiv.org/html/2608.03025#bib.bib18); Luet al\.,[2018](https://arxiv.org/html/2608.03025#bib.bib19)\), unified multimodal transformers with span\-detection auxiliaries\(Yuet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib20)\), object\-level and relation\-propagation attention\(Wuet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib21); Sunet al\.,[2021](https://arxiv.org/html/2608.03025#bib.bib22)\), and hierarchical visual prefixes with alignment objectives\(Chenet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib23); Wanget al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib24); Xuet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib25)\)\. DiffImaginE is complementary: it leaves the encoder stack unchanged and modifies only the per\-\(span, type\) verification rule\.

A parallel line within MNER replaces the verifier with deterministic imagination: given a span and a type, the model predicts a single visual feature vector and scores agreement with the observation\(Chenet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib23); Xuet al\.,[2022](https://arxiv.org/html/2608.03025#bib.bib25)\)\. This design is fast but cannot represent multimodal visual realisations of a type and offers no likelihood interpretation of the score\. Our ImaginE control reimplements this paradigm with the same encoders and losses as DiffImaginE, which makes the diffusion\-versus\-deterministic comparison in[Sections˜4\.2](https://arxiv.org/html/2608.03025#S4.SS2)and[4](https://arxiv.org/html/2608.03025#S4.T4)a direct test of the verification rule rather than of the surrounding MNER stack\.

Diffusion models generate data by iterative denoising\(Sohl\-Dicksteinet al\.,[2015](https://arxiv.org/html/2608.03025#bib.bib1); Hoet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib2); Song and Ermon,[2019](https://arxiv.org/html/2608.03025#bib.bib4); Songet al\.,[2020](https://arxiv.org/html/2608.03025#bib.bib5); Nichol and Dhariwal,[2021](https://arxiv.org/html/2608.03025#bib.bib3); Kingmaet al\.,[2021](https://arxiv.org/html/2608.03025#bib.bib6)\)and support classifier\-free guidance\(Dhariwal and Nichol,[2021](https://arxiv.org/html/2608.03025#bib.bib7); Ho and Salimans,[2022](https://arxiv.org/html/2608.03025#bib.bib8)\)\. A related line treats class\-conditional denoising error as a zero\-shot classifier\(Liet al\.,[2023](https://arxiv.org/html/2608.03025#bib.bib12); Clark and Jaini,[2023](https://arxiv.org/html/2608.03025#bib.bib13)\); likelihood\-based discriminative objectives and learned timestep weightings recover much of the gap over bare ELBO scores\(Xianet al\.,[2024](https://arxiv.org/html/2608.03025#bib.bib14); Jeonget al\.,[2025](https://arxiv.org/html/2608.03025#bib.bib15)\), and antithetic noise pairing reduces estimator variance\(Jiaet al\.,[2026](https://arxiv.org/html/2608.03025#bib.bib16)\), an effect we formalise in[Proposition˜2](https://arxiv.org/html/2608.03025#Thmproposition2)\. DiffImaginE applies these ideas to span\-conditioned cross\-modal latents in structured MNER\.

## 6Conclusion

DiffImaginE recasts multimodal NER type verification as conditional latent diffusion, scoring each type hypothesis by how well a type\-conditioned denoiser reconstructs noise injected into span\-localised visual evidence\. We supervise the scores as classification logits, learn how to aggregate errors across timesteps, and estimate expectations antithetically; two propositions in[Section˜3\.9](https://arxiv.org/html/2608.03025#S3.SS9)justify guidance as posterior sharpening and antithetic pairing as variance reduction at fixed cost under an even/odd criterion\. A matched ImaginE control attributes the observed gains to the diffusion verifier rather than to encoder or fusion changes\. On Twitter\-2015 and Twitter\-2017 the diffusion scorer improves strictF1F\_\{1\}over this control, with ablations and budget sweeps that trace the benefit to the verifier and to the design choices in[Section˜3\.7](https://arxiv.org/html/2608.03025#S3.SS7)\.

Results are limited to short\-text, single\-image Twitter posts, where the number of evaluation timesteps trades compute for score fidelity and where image quality varies widely\. Natural next steps are multi\-image or video evidence, open\-vocabulary types, and applying the same span\-conditioned diffusion verifier to other structured labelling tasks with visual context\.

## Supplementary Material

This supplementary material contains the proofs of the two propositions together with the full loss, hyperparameter, and ablation\-configuration specifications\. Equation, proposition, and assumption numbers refer to the corresponding statements in the main paper\.

## Appendix AProof of Proposition 1 \(Guided score is a tempered posterior\)

#### Unguided score and Bayes posterior\.

Under the ELBO\-consistent weighting assumption, for every typekkand spanssthe expected weighted denoising error equals the negative conditional log\-likelihood up to a type\-independent constant,

errk=−log⁡pθ​\(𝒗∣𝒔,𝒆k\)\+C,\\mathrm\{err\}\_\{k\}=\-\\log p\_\{\\theta\}\(\\bm\{v\}\\mid\\bm\{s\},\\bm\{e\}\_\{k\}\)\+C,\(9\)which is the diffusion\-classifier identity\(Liet al\.,[2023](https://arxiv.org/html/2608.03025#bib.bib12); Kingmaet al\.,[2021](https://arxiv.org/html/2608.03025#bib.bib6)\)\. With a uniform priorp​\(𝒆k∣𝒔\)=1/Kp\(\\bm\{e\}\_\{k\}\\mid\\bm\{s\}\)=1/K, Bayes’ rule gives

pθ​\(𝒆k∣𝒗,𝒔\)\\displaystyle p\_\{\\theta\}\(\\bm\{e\}\_\{k\}\\mid\\bm\{v\},\\bm\{s\}\)=pθ​\(𝒗∣𝒔,𝒆k\)∑jpθ​\(𝒗∣𝒔,𝒆j\)\\displaystyle=\\frac\{p\_\{\\theta\}\(\\bm\{v\}\\mid\\bm\{s\},\\bm\{e\}\_\{k\}\)\}\{\\sum\_\{j\}p\_\{\\theta\}\(\\bm\{v\}\\mid\\bm\{s\},\\bm\{e\}\_\{j\}\)\}\(10\)=exp⁡\(−errk\+C\)∑jexp⁡\(−errj\+C\)\\displaystyle=\\frac\{\\exp\(\-\\mathrm\{err\}\_\{k\}\+C\)\}\{\\sum\_\{j\}\\exp\(\-\\mathrm\{err\}\_\{j\}\+C\)\}=softmaxk​\(scorek\),\\displaystyle=\\mathrm\{softmax\}\_\{k\}\(\\mathrm\{score\}\_\{k\}\),sincescorek=−errk\\mathrm\{score\}\_\{k\}=\-\\mathrm\{err\}\_\{k\}and the common factoreCe^\{C\}cancels\. Thereforearg​maxk⁡scorek=arg​maxk⁡pθ​\(𝒆k∣𝒗,𝒔\)\\operatorname\*\{arg\\,max\}\_\{k\}\\mathrm\{score\}\_\{k\}=\\operatorname\*\{arg\\,max\}\_\{k\}p\_\{\\theta\}\(\\bm\{e\}\_\{k\}\\mid\\bm\{v\},\\bm\{s\}\)is the Bayes\-optimal type, andsoftmaxk​\(scorek/τ\)\\mathrm\{softmax\}\_\{k\}\(\\mathrm\{score\}\_\{k\}/\\tau\)is the temperature\-τ\\tauposterior\.

#### Guidance as a temperature\.

The guided score satisfiesscorek\(g\)=−\(1\+g\)​errk\+g​err∅\\mathrm\{score\}\_\{k\}^\{\(g\)\}=\-\(1\+g\)\\,\\mathrm\{err\}\_\{k\}\+g\\,\\mathrm\{err\}\_\{\\varnothing\}\. The termg​err∅g\\,\\mathrm\{err\}\_\{\\varnothing\}is independent ofkk, so it is an additive constant inside the softmax and cancels:

softmaxk​\(scorek\(g\)/τ\)\\displaystyle\\mathrm\{softmax\}\_\{k\}\\\!\\bigl\(\\mathrm\{score\}\_\{k\}^\{\(g\)\}/\\tau\\bigr\)=softmaxk​\(−\(1\+g\)​errk\+g​err∅τ\)\\displaystyle=\\mathrm\{softmax\}\_\{k\}\\\!\\Bigl\(\\tfrac\{\-\(1\+g\)\\mathrm\{err\}\_\{k\}\+g\\,\\mathrm\{err\}\_\{\\varnothing\}\}\{\\tau\}\\Bigr\)\(11\)=softmaxk​\(−\(1\+g\)​errkτ\)\\displaystyle=\\mathrm\{softmax\}\_\{k\}\\\!\\Bigl\(\\tfrac\{\-\(1\+g\)\\mathrm\{err\}\_\{k\}\}\{\\tau\}\\Bigr\)=softmaxk​\(scorekτ/\(1\+g\)\)\.\\displaystyle=\\mathrm\{softmax\}\_\{k\}\\\!\\Bigl\(\\tfrac\{\\mathrm\{score\}\_\{k\}\}\{\\tau/\(1\+g\)\}\\Bigr\)\.Substitutingscorek=log⁡pθ​\(𝒗∣𝒔,𝒆k\)\+const\\mathrm\{score\}\_\{k\}=\\log p\_\{\\theta\}\(\\bm\{v\}\\mid\\bm\{s\},\\bm\{e\}\_\{k\}\)\+\\text\{const\}and using the uniform prior,

softmaxk​\(scorek\(g\)/τ\)\\displaystyle\\mathrm\{softmax\}\_\{k\}\\\!\\bigl\(\\mathrm\{score\}\_\{k\}^\{\(g\)\}/\\tau\\bigr\)∝exp⁡\(1\+gτ​log⁡pθ​\(𝒗∣𝒔,𝒆k\)\)\\displaystyle\\propto\\exp\\\!\\Bigl\(\\tfrac\{1\+g\}\{\\tau\}\\log p\_\{\\theta\}\(\\bm\{v\}\\mid\\bm\{s\},\\bm\{e\}\_\{k\}\)\\Bigr\)\(12\)∝pθ​\(𝒆k∣𝒗,𝒔\)\(1\+g\)/τ\.\\displaystyle\\propto p\_\{\\theta\}\(\\bm\{e\}\_\{k\}\\mid\\bm\{v\},\\bm\{s\}\)^\{\(1\+g\)/\\tau\}\.Forτ=1\\tau=1this ispθ​\(𝒆k∣𝒗,𝒔\)1\+gp\_\{\\theta\}\(\\bm\{e\}\_\{k\}\\mid\\bm\{v\},\\bm\{s\}\)^\{1\+g\}, a posterior raised to the power1\+g1\+g\. The map is a monotone \(rank\-preserving\) sharpening: it does not changearg​maxk\\operatorname\*\{arg\\,max\}\_\{k\}, and it makes the distribution strictly more peaked forg\>0g\>0\.□\\square

## Appendix BProof of Proposition 2 \(Antithetic variance reduction\)

We first record the even/odd decomposition, then compute the two variances, then justify the leading\-order claim\.

###### Lemma 1\(Orthogonal even/odd decomposition\)\.

Letϵ=d−ϵ\{\\epsilon\}\\stackrel\{\{\\scriptstyle d\}\}\{\{=\}\}\-\{\\epsilon\}and letffbe square\-integrable\. Withfe​\(ϵ\)=12​\(f​\(ϵ\)\+f​\(−ϵ\)\)f\_\{e\}\(\{\\epsilon\}\)=\\tfrac\{1\}\{2\}\(f\(\{\\epsilon\}\)\+f\(\-\{\\epsilon\}\)\)andfo​\(ϵ\)=12​\(f​\(ϵ\)−f​\(−ϵ\)\)f\_\{o\}\(\{\\epsilon\}\)=\\tfrac\{1\}\{2\}\(f\(\{\\epsilon\}\)\-f\(\-\{\\epsilon\}\)\)we havef=fe\+fof=f\_\{e\}\+f\_\{o\},𝔼​\[f\]=𝔼​\[fe\]\\mathbb\{E\}\[f\]=\\mathbb\{E\}\[f\_\{e\}\],𝔼​\[fo\]=0\\mathbb\{E\}\[f\_\{o\}\]=0,Cov​\(fe,fo\)=0\\mathrm\{Cov\}\(f\_\{e\},f\_\{o\}\)=0, andVar​\(f\)=Var​\(fe\)\+Var​\(fo\)\\mathrm\{Var\}\(f\)=\\mathrm\{Var\}\(f\_\{e\}\)\+\\mathrm\{Var\}\(f\_\{o\}\)\.

###### Proof\.

f=fe\+fof=f\_\{e\}\+f\_\{o\}is immediate\. Sinceϵ=d−ϵ\{\\epsilon\}\\stackrel\{\{\\scriptstyle d\}\}\{\{=\}\}\-\{\\epsilon\},𝔼​\[fo\]=12​\(𝔼​\[f​\(ϵ\)\]−𝔼​\[f​\(−ϵ\)\]\)=0\\mathbb\{E\}\[f\_\{o\}\]=\\tfrac\{1\}\{2\}\(\\mathbb\{E\}\[f\(\{\\epsilon\}\)\]\-\\mathbb\{E\}\[f\(\-\{\\epsilon\}\)\]\)=0, hence𝔼​\[f\]=𝔼​\[fe\]\\mathbb\{E\}\[f\]=\\mathbb\{E\}\[f\_\{e\}\]\. For the covariance,fe​fof\_\{e\}f\_\{o\}is an odd function ofϵ\{\\epsilon\}\(a product of an even and an odd function\), so𝔼​\[fe​fo\]=0\\mathbb\{E\}\[f\_\{e\}f\_\{o\}\]=0by symmetry, andCov​\(fe,fo\)=𝔼​\[fe​fo\]−𝔼​\[fe\]​𝔼​\[fo\]=0\\mathrm\{Cov\}\(f\_\{e\},f\_\{o\}\)=\\mathbb\{E\}\[f\_\{e\}f\_\{o\}\]\-\\mathbb\{E\}\[f\_\{e\}\]\\mathbb\{E\}\[f\_\{o\}\]=0\. Orthogonality then givesVar​\(f\)=Var​\(fe\)\+Var​\(fo\)\\mathrm\{Var\}\(f\)=\\mathrm\{Var\}\(f\_\{e\}\)\+\\mathrm\{Var\}\(f\_\{o\}\)\. ∎

#### Variances of the two estimators\.

The per\-sample type differencef​\(ϵ\)=ϕk​\(ϵ\)−ϕj​\(ϵ\)f\(\{\\epsilon\}\)=\\phi\_\{k\}\(\{\\epsilon\}\)\-\\phi\_\{j\}\(\{\\epsilon\}\)is square\-integrable, and both estimators use two denoiser evaluations \(the antithetic estimator evaluatesϕk,ϕj\\phi\_\{k\},\\phi\_\{j\}atϵ\{\\epsilon\}and at−ϵ\-\{\\epsilon\}; the i\.i\.d\. estimator at two independent draws\)\. Unbiasedness of both for𝔼​\[f\]=errk−errj\\mathbb\{E\}\[f\]=\\mathrm\{err\}\_\{k\}\-\\mathrm\{err\}\_\{j\}follows from[Lemma˜1](https://arxiv.org/html/2608.03025#Thmlemma1)and linearity\. For the antithetic estimator,

D^anti\\displaystyle\\widehat\{D\}^\{\\,\\mathrm\{anti\}\}=12​\(f​\(ϵ\)\+f​\(−ϵ\)\)=fe​\(ϵ\),\\displaystyle=\\tfrac\{1\}\{2\}\\bigl\(f\(\{\\epsilon\}\)\+f\(\-\{\\epsilon\}\)\\bigr\)=f\_\{e\}\(\{\\epsilon\}\),\(13\)Var​\(D^anti\)\\displaystyle\\mathrm\{Var\}\(\\widehat\{D\}^\{\\,\\mathrm\{anti\}\}\)=Var​\(fe\)\.\\displaystyle=\\mathrm\{Var\}\(f\_\{e\}\)\.For two independent samplesϵ\(1\),ϵ\(2\)\{\\epsilon\}^\{\(1\)\},\{\\epsilon\}^\{\(2\)\},

Var​\(D^iid\)\\displaystyle\\mathrm\{Var\}\(\\widehat\{D\}^\{\\,\\mathrm\{iid\}\}\)=Var​\(12​\(f​\(ϵ\(1\)\)\+f​\(ϵ\(2\)\)\)\)\\displaystyle=\\mathrm\{Var\}\\\!\\Bigl\(\\tfrac\{1\}\{2\}\\bigl\(f\(\{\\epsilon\}^\{\(1\)\}\)\+f\(\{\\epsilon\}^\{\(2\)\}\)\\bigr\)\\Bigr\)\(14\)=12​Var​\(f\)\\displaystyle=\\tfrac\{1\}\{2\}\\mathrm\{Var\}\(f\)=12​\(Var​\(fe\)\+Var​\(fo\)\),\\displaystyle=\\tfrac\{1\}\{2\}\\bigl\(\\mathrm\{Var\}\(f\_\{e\}\)\+\\mathrm\{Var\}\(f\_\{o\}\)\\bigr\),using independence and[Lemma˜1](https://arxiv.org/html/2608.03025#Thmlemma1)\. Therefore,

Var​\(D^anti\)≤Var​\(D^iid\)\\displaystyle\\mathrm\{Var\}\(\\widehat\{D\}^\{\\,\\mathrm\{anti\}\}\)\\leq\\mathrm\{Var\}\(\\widehat\{D\}^\{\\,\\mathrm\{iid\}\}\)⇔Var​\(fe\)≤12​\(Var​\(fe\)\+Var​\(fo\)\)\\displaystyle\\quad\\iff\\mathrm\{Var\}\(f\_\{e\}\)\\leq\\tfrac\{1\}\{2\}\\bigl\(\\mathrm\{Var\}\(f\_\{e\}\)\+\\mathrm\{Var\}\(f\_\{o\}\)\\bigr\)⇔Var​\(fe\)≤Var​\(fo\)\.\\displaystyle\\quad\\iff\\mathrm\{Var\}\(f\_\{e\}\)\\leq\\mathrm\{Var\}\(f\_\{o\}\)\.

#### Leading\-order dominance of the odd component\.

Writeϕk​\(ϵ\)=‖ϵθ​\(⋅,𝒆k\)‖22−2​⟨ϵ,ϵθ​\(⋅,𝒆k\)⟩\+‖ϵ‖22\\phi\_\{k\}\(\{\\epsilon\}\)=\\\|\{\\epsilon\}\_\{\\theta\}\(\\cdot,\\bm\{e\}\_\{k\}\)\\\|\_\{2\}^\{2\}\-2\\langle\{\\epsilon\},\{\\epsilon\}\_\{\\theta\}\(\\cdot,\\bm\{e\}\_\{k\}\)\\rangle\+\\\|\{\\epsilon\}\\\|\_\{2\}^\{2\}\. The data\-independent term‖ϵ‖22\\\|\{\\epsilon\}\\\|\_\{2\}^\{2\}is identical across types and cancels inf=ϕk−ϕjf=\\phi\_\{k\}\-\\phi\_\{j\}:

f​\(ϵ\)=\\displaystyle f\(\{\\epsilon\}\)=\{\}‖ϵθ​\(⋅,𝒆k\)‖22−‖ϵθ​\(⋅,𝒆j\)‖22\\displaystyle\\\|\{\\epsilon\}\_\{\\theta\}\(\\cdot,\\bm\{e\}\_\{k\}\)\\\|\_\{2\}^\{2\}\-\\\|\{\\epsilon\}\_\{\\theta\}\(\\cdot,\\bm\{e\}\_\{j\}\)\\\|\_\{2\}^\{2\}\(15\)−2​⟨ϵ,ϵθ​\(⋅,𝒆k\)−ϵθ​\(⋅,𝒆j\)⟩,\\displaystyle\{\}\-2\\langle\{\\epsilon\},\\,\{\\epsilon\}\_\{\\theta\}\(\\cdot,\\bm\{e\}\_\{k\}\)\-\{\\epsilon\}\_\{\\theta\}\(\\cdot,\\bm\{e\}\_\{j\}\)\\rangle,where eachϵθ​\(⋅,𝒆k\)=ϵθ​\(α¯t​𝒗\+1−α¯t​ϵ,𝒔,𝒆k,t\)\{\\epsilon\}\_\{\\theta\}\(\\cdot,\\bm\{e\}\_\{k\}\)=\{\\epsilon\}\_\{\\theta\}\(\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\bm\{v\}\+\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}\\,\{\\epsilon\},\\bm\{s\},\\bm\{e\}\_\{k\},t\)depends onϵ\{\\epsilon\}through its input\. Expand the denoiser to first order aroundϵ=0\{\\epsilon\}=0,ϵθ​\(⋅,𝒆k\)=𝒂k\+1−α¯t​Jk​ϵ\+o​\(‖ϵ‖\)\{\\epsilon\}\_\{\\theta\}\(\\cdot,\\bm\{e\}\_\{k\}\)=\\bm\{a\}\_\{k\}\+\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}\\,J\_\{k\}\\,\{\\epsilon\}\+o\(\\\|\{\\epsilon\}\\\|\), with𝒂k=ϵθ​\(α¯t​𝒗,𝒔,𝒆k,t\)\\bm\{a\}\_\{k\}=\{\\epsilon\}\_\{\\theta\}\(\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}\\bm\{v\},\\bm\{s\},\\bm\{e\}\_\{k\},t\)and JacobianJkJ\_\{k\}\. The explicit cross term contributes the odd component−2​⟨ϵ,𝒂k−𝒂j⟩\+O​\(‖ϵ‖2\)\-2\\langle\{\\epsilon\},\\bm\{a\}\_\{k\}\-\\bm\{a\}\_\{j\}\\rangle\+O\(\\\|\{\\epsilon\}\\\|^\{2\}\)\. The squared\-norm term is not purely even:‖ϵθ​\(⋅,𝒆k\)‖22=‖𝒂k‖22\+2​1−α¯t​⟨Jk⊤​𝒂k,ϵ⟩\+O​\(‖ϵ‖2\)\\\|\{\\epsilon\}\_\{\\theta\}\(\\cdot,\\bm\{e\}\_\{k\}\)\\\|\_\{2\}^\{2\}=\\\|\\bm\{a\}\_\{k\}\\\|\_\{2\}^\{2\}\+2\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}\\,\\langle J\_\{k\}^\{\\\!\\top\}\\bm\{a\}\_\{k\},\{\\epsilon\}\\rangle\+O\(\\\|\{\\epsilon\}\\\|^\{2\}\), whose first\-order part is also odd inϵ\{\\epsilon\}\. Collecting both contributions, the odd part offfis, to first order,

fo​\(ϵ\)≈−2​⟨ϵ,\(𝒂k−𝒂j\)−1−α¯t​\(Jk⊤​𝒂k−Jj⊤​𝒂j\)⟩,f\_\{o\}\(\{\\epsilon\}\)\\approx\-2\\bigl\\langle\{\\epsilon\},\\;\(\\bm\{a\}\_\{k\}\-\\bm\{a\}\_\{j\}\)\-\\sqrt\{1\-\\bar\{\\alpha\}\_\{t\}\}\\,\(J\_\{k\}^\{\\\!\\top\}\\bm\{a\}\_\{k\}\-J\_\{j\}^\{\\\!\\top\}\\bm\{a\}\_\{j\}\)\\bigr\\rangle,\(16\)while the quadratic remainders are even to leading order\. When the denoiser is well\-conditioned, i\.e\. the Jacobian terms are small relative to the inter\-type gap‖𝒂k−𝒂j‖\\\|\\bm\{a\}\_\{k\}\-\\bm\{a\}\_\{j\}\\\|, the odd part is dominated by−2​⟨ϵ,𝒂k−𝒂j⟩\-2\\langle\{\\epsilon\},\\bm\{a\}\_\{k\}\-\\bm\{a\}\_\{j\}\\rangleandVar​\(fo\)≈4​‖𝒂k−𝒂j‖22\\mathrm\{Var\}\(f\_\{o\}\)\\approx 4\\,\\\|\\bm\{a\}\_\{k\}\-\\bm\{a\}\_\{j\}\\\|\_\{2\}^\{2\}\. In this regimeVar​\(fo\)\>Var​\(fe\)\\mathrm\{Var\}\(f\_\{o\}\)\>\\mathrm\{Var\}\(f\_\{e\}\), so by the exact criterion above the antithetic estimator reduces the variance of the type\-discriminative score\. We stress that the criterionVar​\(fe\)≤Var​\(fo\)\\mathrm\{Var\}\(f\_\{e\}\)\\leq\\mathrm\{Var\}\(f\_\{o\}\)is exact; the first\-order analysis only supplies a sufficient condition under which it holds\.□\\square

## Appendix CComposite Loss Definitions

The total training objective is the weighted sum

ℒ=\\displaystyle\\mathcal\{L\}=\{\}ℒtask\+αdiff​ℒdiff\+β​ℒico\+λclf​ℒclf\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{task\}\}\+\\alpha\_\{\\mathrm\{diff\}\}\\mathcal\{L\}\_\{\\mathrm\{diff\}\}\+\\beta\\,\\mathcal\{L\}\_\{\\mathrm\{ico\}\}\+\\lambda\_\{\\mathrm\{clf\}\}\\mathcal\{L\}\_\{\\mathrm\{clf\}\}\+αrev​ℒirarev\+βrev​ℒicorev\\displaystyle\+\\alpha\_\{\\mathrm\{rev\}\}\\mathcal\{L\}\_\{\\mathrm\{ira\}\}^\{\\mathrm\{rev\}\}\+\\beta\_\{\\mathrm\{rev\}\}\\mathcal\{L\}\_\{\\mathrm\{ico\}\}^\{\\mathrm\{rev\}\}\+γrev​ℒsigrev\+λbio​ℒbio\\displaystyle\+\\gamma\_\{\\mathrm\{rev\}\}\\mathcal\{L\}\_\{\\mathrm\{sig\}\}^\{\\mathrm\{rev\}\}\+\\lambda\_\{\\mathrm\{bio\}\}\\mathcal\{L\}\_\{\\mathrm\{bio\}\}\+λxm​ℒxmodal\+λgr​ℒground\\displaystyle\+\\lambda\_\{\\mathrm\{xm\}\}\\mathcal\{L\}\_\{\\mathrm\{xmodal\}\}\+\\lambda\_\{\\mathrm\{gr\}\}\\mathcal\{L\}\_\{\\mathrm\{ground\}\}\+λhard​ℒhard\+λfp​ℒfp\.\\displaystyle\+\\lambda\_\{\\mathrm\{hard\}\}\\mathcal\{L\}\_\{\\mathrm\{hard\}\}\+\\lambda\_\{\\mathrm\{fp\}\}\\mathcal\{L\}\_\{\\mathrm\{fp\}\}\.\(17\)[Table˜6](https://arxiv.org/html/2608.03025#A3.T6)lists each term, its role, and its default weight\. Every term returns a graph\-connected zero when it has no contributing spans, which keeps distributed gradient synchronisation intact\.[Equation˜17](https://arxiv.org/html/2608.03025#A3.E17)omits the R\-Drop regulariser\(Wuet al\.,[2021](https://arxiv.org/html/2608.03025#bib.bib34)\)used by both DiffImaginE and the deterministic baseline, which is applied as a symmetric\-KL consistency between two dropout forward passes \(weightαrdrop=0\.5\\alpha\_\{\\mathrm\{rdrop\}\}=0\.5by default\)\.

Table 6:Composite loss terms\.Odenotes the non\-entity class; “entity spans” are spans whose gold label is notO\.## Appendix DHyperparameters and Search Ranges

[Table˜7](https://arxiv.org/html/2608.03025#A4.T7)lists the default hyperparameters and the ranges searched on the development set\. The deterministic baseline shares every shared\-stack and regularisation hyperparameter; only the verifier\-specific diffusion hyperparameters are unique to DiffImaginE\.

Table 7:Default hyperparameters and development\-set search ranges\.GroupParameterDefaultRangeEncoderstext encoderRoBERTa\-base\{base, large\}vision encoderCLIP\-ViT\-B/32\{B/32, L/14\}shared widthdd256\{256, 384, 512\}DiffusiontimestepsTT1000–eval timestepsNN5\{1, 2, 5, 10, 20\}train MC stepsMM4\{2, 4\}Min\-SNRγ\\gamma5\.0\{1, 5,∞\\infty\}GuidanceCFG dropoutpcfp\_\{\\mathrm\{cf\}\}0\.15\[0\.0, 0\.3\]guidance scaleggdev\-tuned\{1, 2, 3, 4\}FβF\_\{\\beta\}for selection1\.0\{0\.5, 1\.0\}Warmup / losswarmup epochs10\{5, 10, 20\}αdiff\\alpha\_\{\\mathrm\{diff\}\}1\.0\{0\.1, 1\.0, 2\.0\}β\\beta\(ℒico\\mathcal\{L\}\_\{\\mathrm\{ico\}\}\)0\.5\[0\.0, 1\.0\]λclf\\lambda\_\{\\mathrm\{clf\}\}0\.5\{0\.1, 0\.5, 1\.0\}Optimisationencoder LR2e\-5–new\-module LR1e\-4–EMA decay0\.999–

## Appendix EAblation Configurations

[Table˜8](https://arxiv.org/html/2608.03025#A5.T8)describes the core ablation variants referenced in the main paper and the hypothesis each one tests\. The full grid contains 54 variants: the 15 core variants listed below plus extended variants drawn from the evaluation\-budget, loss\-weight, auxiliary\-objective, precision\-calibration, and architecture sweeps; the complete variant registry, with each variant’s command\-line configuration, is released with the code, and every variant is run over multiple seeds on both datasets\.

Table 8:Core ablation variants and the hypothesis each tests\.

## References

- Vicreg: variance\-invariance\-covariance regularization for self\-supervised learning\.arXiv preprint arXiv:2105\.04906\.Cited by:[§3\.6](https://arxiv.org/html/2608.03025#S3.SS6.p1.3)\.
- X\. Chen, N\. Zhang, L\. Li, Y\. Yao, S\. Deng, C\. Tan, F\. Huang, L\. Si, and H\. Chen \(2022\)Good visual guidance make a better extractor: hierarchical visual prefix for multimodal entity and relation extraction\.InFindings of the association for computational linguistics: NAACL 2022,pp\. 1607–1618\.Cited by:[§1](https://arxiv.org/html/2608.03025#S1.p1.1),[§1](https://arxiv.org/html/2608.03025#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.03025#S2.SS2.p1.4),[§4\.1](https://arxiv.org/html/2608.03025#S4.SS1.p1.1),[§4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2608.03025#S4.T2.3.5.5.1),[§5](https://arxiv.org/html/2608.03025#S5.p1.1),[§5](https://arxiv.org/html/2608.03025#S5.p2.1)\.
- K\. Clark and P\. Jaini \(2023\)Text\-to\-image diffusion models are zero shot classifiers\.Advances in Neural Information Processing Systems36,pp\. 58921–58937\.Cited by:[§1](https://arxiv.org/html/2608.03025#S1.p4.1),[§2\.3](https://arxiv.org/html/2608.03025#S2.SS3.p2.1),[§5](https://arxiv.org/html/2608.03025#S5.p3.1)\.
- P\. Dhariwal and A\. Nichol \(2021\)Diffusion models beat gans on image synthesis\.Advances in neural information processing systems34,pp\. 8780–8794\.Cited by:[§3\.4](https://arxiv.org/html/2608.03025#S3.SS4.p1.4),[§5](https://arxiv.org/html/2608.03025#S5.p3.1)\.
- G\. Ding, Y\. Kong, and X\. Li \(2025\)SEPA: an semantic projection alignment framework for multimodal named entity recognition\.InProceedings of the 2025 International Conference on Multimedia Retrieval,pp\. 202–210\.Cited by:[Table 2](https://arxiv.org/html/2608.03025#S4.T2.3.10.10.1)\.
- A\. Dosovitskiy, L\. Beyer, A\. Kolesnikov, D\. Weissenborn, X\. Zhai, T\. Unterthiner, M\. Dehghani, M\. Minderer, G\. Heigold, S\. Gelly,et al\.\(2020\)An image is worth 16x16 words: transformers for image recognition at scale\.arXiv preprint arXiv:2010\.11929\.Cited by:[§2\.2](https://arxiv.org/html/2608.03025#S2.SS2.p1.4)\.
- T\. Hang, S\. Gu, C\. Li, J\. Bao, D\. Chen, H\. Hu, X\. Geng, and B\. Guo \(2023\)Efficient diffusion training via min\-snr weighting strategy\.InProceedings of the IEEE/CVF international conference on computer vision,pp\. 7441–7451\.Cited by:[§1](https://arxiv.org/html/2608.03025#S1.p5.1),[§3\.5](https://arxiv.org/html/2608.03025#S3.SS5.p1.2)\.
- J\. Ho, A\. Jain, and P\. Abbeel \(2020\)Denoising diffusion probabilistic models\.Advances in neural information processing systems33,pp\. 6840–6851\.Cited by:[§2\.3](https://arxiv.org/html/2608.03025#S2.SS3.p1.3),[§2\.3](https://arxiv.org/html/2608.03025#S2.SS3.p2.4),[§5](https://arxiv.org/html/2608.03025#S5.p3.1),[Assumption 1](https://arxiv.org/html/2608.03025#Thmassumption1.p1.6)\.
- J\. Ho and T\. Salimans \(2022\)Classifier\-free diffusion guidance\.arXiv preprint arXiv:2207\.12598\.Cited by:[§1](https://arxiv.org/html/2608.03025#S1.p5.1),[§3\.4](https://arxiv.org/html/2608.03025#S3.SS4.p1.4),[§5](https://arxiv.org/html/2608.03025#S5.p3.1)\.
- Y\. Jeong, A\. Uselis, S\. J\. Oh, and A\. Rohrbach \(2025\)Diffusion classifiers understand compositionality, but conditions apply\.arXiv preprint arXiv:2505\.179552,pp\. 9\.Cited by:[§3\.7](https://arxiv.org/html/2608.03025#S3.SS7.SSS0.Px2.p1.5),[§5](https://arxiv.org/html/2608.03025#S5.p3.1)\.
- J\. Jia, S\. Liu, B\. Song, W\. Yuan, L\. Shen, and G\. Wang \(2026\)Antithetic noise in diffusion models\.InInternational Conference on Learning Representations,Cited by:[§3\.7](https://arxiv.org/html/2608.03025#S3.SS7.SSS0.Px3.p1.4),[§5](https://arxiv.org/html/2608.03025#S5.p3.1)\.
- D\. Kingma, T\. Salimans, B\. Poole, and J\. Ho \(2021\)Variational diffusion models\.Advances in neural information processing systems34,pp\. 21696–21707\.Cited by:[Appendix A](https://arxiv.org/html/2608.03025#A1.SS0.SSS0.Px1.p1.3),[§2\.3](https://arxiv.org/html/2608.03025#S2.SS3.p2.4),[§5](https://arxiv.org/html/2608.03025#S5.p3.1),[Assumption 1](https://arxiv.org/html/2608.03025#Thmassumption1.p1.6)\.
- A\. C\. Li, M\. Prabhudesai, S\. Duggal, E\. Brown, and D\. Pathak \(2023\)Your diffusion model is secretly a zero\-shot classifier\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 2206–2217\.Cited by:[Appendix A](https://arxiv.org/html/2608.03025#A1.SS0.SSS0.Px1.p1.3),[§1](https://arxiv.org/html/2608.03025#S1.p4.1),[§2\.3](https://arxiv.org/html/2608.03025#S2.SS3.p2.1),[§5](https://arxiv.org/html/2608.03025#S5.p3.1),[Assumption 1](https://arxiv.org/html/2608.03025#Thmassumption1.p1.6)\.
- Y\. Liu, M\. Ott, N\. Goyal, J\. Du, M\. Joshi, D\. Chen, O\. Levy, M\. Lewis, L\. Zettlemoyer, and V\. Stoyanov \(2019\)Roberta: a robustly optimized bert pretraining approach\.arXiv preprint arXiv:1907\.11692\.Cited by:[§2\.2](https://arxiv.org/html/2608.03025#S2.SS2.p1.4)\.
- I\. Loshchilov and F\. Hutter \(2017\)Decoupled weight decay regularization\.arXiv preprint arXiv:1711\.05101\.Cited by:[§4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px3.p1.11)\.
- D\. Lu, L\. Neves, V\. Carvalho, N\. Zhang, and H\. Ji \(2018\)Visual attention model for name tagging in multimodal social media\.InProceedings of the 56th annual meeting of the association for computational linguistics \(volume 1: Long papers\),pp\. 1990–1999\.Cited by:[§1](https://arxiv.org/html/2608.03025#S1.p1.1),[§5](https://arxiv.org/html/2608.03025#S5.p1.1)\.
- S\. Moon, L\. Neves, and V\. Carvalho \(2018\)Multimodal named entity recognition for short social media posts\.InProceedings of the 2018 conference of the North American chapter of the association for computational linguistics: Human language technologies, volume 1 \(long papers\),pp\. 852–860\.Cited by:[§1](https://arxiv.org/html/2608.03025#S1.p1.1),[§5](https://arxiv.org/html/2608.03025#S5.p1.1)\.
- A\. Q\. Nichol and P\. Dhariwal \(2021\)Improved denoising diffusion probabilistic models\.InInternational conference on machine learning,pp\. 8162–8171\.Cited by:[§2\.3](https://arxiv.org/html/2608.03025#S2.SS3.p1.3),[§5](https://arxiv.org/html/2608.03025#S5.p3.1)\.
- A\. v\. d\. Oord, Y\. Li, and O\. Vinyals \(2018\)Representation learning with contrastive predictive coding\.arXiv preprint arXiv:1807\.03748\.Cited by:[§3\.6](https://arxiv.org/html/2608.03025#S3.SS6.p1.3)\.
- W\. Peebles and S\. Xie \(2023\)Scalable diffusion models with transformers\.InProceedings of the IEEE/CVF international conference on computer vision,pp\. 4195–4205\.Cited by:[§3\.3](https://arxiv.org/html/2608.03025#S3.SS3.p1.6)\.
- A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark,et al\.\(2021\)Learning transferable visual models from natural language supervision\.InInternational conference on machine learning,pp\. 8748–8763\.Cited by:[§2\.2](https://arxiv.org/html/2608.03025#S2.SS2.p1.4),[§3\.6](https://arxiv.org/html/2608.03025#S3.SS6.p1.3)\.
- R\. Rombach, A\. Blattmann, D\. Lorenz, P\. Esser, and B\. Ommer \(2022\)High\-resolution image synthesis with latent diffusion models\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 10684–10695\.Cited by:[§3\.2](https://arxiv.org/html/2608.03025#S3.SS2.p1.7)\.
- J\. Sohl\-Dickstein, E\. Weiss, N\. Maheswaranathan, and S\. Ganguli \(2015\)Deep unsupervised learning using nonequilibrium thermodynamics\.InInternational conference on machine learning,pp\. 2256–2265\.Cited by:[§5](https://arxiv.org/html/2608.03025#S5.p3.1)\.
- Y\. Song and S\. Ermon \(2019\)Generative modeling by estimating gradients of the data distribution\.Advances in neural information processing systems32\.Cited by:[§5](https://arxiv.org/html/2608.03025#S5.p3.1)\.
- Y\. Song, J\. Sohl\-Dickstein, D\. P\. Kingma, A\. Kumar, S\. Ermon, and B\. Poole \(2020\)Score\-based generative modeling through stochastic differential equations\.arXiv preprint arXiv:2011\.13456\.Cited by:[§5](https://arxiv.org/html/2608.03025#S5.p3.1)\.
- L\. Sun, J\. Wang, K\. Zhang, Y\. Su, and F\. Weng \(2021\)RpBERT: a text\-image relation propagation\-based bert model for multimodal ner\.InProceedings of the AAAI conference on artificial intelligence,Vol\.35,pp\. 13860–13868\.Cited by:[§4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2608.03025#S5.p1.1)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin \(2017\)Attention is all you need\.Advances in neural information processing systems30\.Cited by:[§2\.2](https://arxiv.org/html/2608.03025#S2.SS2.p1.4)\.
- X\. Wang, M\. Gui, Y\. Jiang, Z\. Jia, N\. Bach, T\. Wang, Z\. Huang, and K\. Tu \(2022\)ITA: image\-text alignments for multi\-modal named entity recognition\.InProceedings of the 2022 conference of the North American chapter of the association for computational linguistics: human language technologies,pp\. 3176–3189\.Cited by:[§1](https://arxiv.org/html/2608.03025#S1.p1.1),[§4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2608.03025#S5.p1.1)\.
- P\. Wei, H\. Ouyang, Q\. Hu, B\. Zeng, G\. Feng, and Q\. Wen \(2024\)Vec\-mner: hybrid transformer with visual\-enhanced cross\-modal multi\-level interaction for multimodal ner\.InProceedings of the 2024 International Conference on Multimedia Retrieval,pp\. 469–477\.Cited by:[Table 2](https://arxiv.org/html/2608.03025#S4.T2.3.9.9.1)\.
- L\. Wu, J\. Li, Y\. Wang, Q\. Meng, T\. Qin, W\. Chen, M\. Zhang, T\. Liu,et al\.\(2021\)R\-drop: regularized dropout for neural networks\.Advances in neural information processing systems34,pp\. 10890–10905\.Cited by:[Appendix C](https://arxiv.org/html/2608.03025#A3.p1.1),[§4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px1.p1.1)\.
- Z\. Wu, C\. Zheng, Y\. Cai, J\. Chen, H\. Leung, and Q\. Li \(2020\)Multimodal representation with embedded visual guiding objects for named entity recognition in social media posts\.InProceedings of the 28th ACM International conference on multimedia,pp\. 1038–1046\.Cited by:[§4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2608.03025#S5.p1.1)\.
- J\. J\. C\. Xian, S\. Mahdavi, R\. Liao, and O\. Schulte \(2024\)From graph diffusion to graph classification\.arXiv preprint arXiv:2411\.17236\.Cited by:[§3\.7](https://arxiv.org/html/2608.03025#S3.SS7.SSS0.Px1.p1.6),[§5](https://arxiv.org/html/2608.03025#S5.p3.1)\.
- B\. Xu, S\. Huang, C\. Sha, and H\. Wang \(2022\)MAF: a general matching and alignment framework for multimodal named entity recognition\.InProceedings of the fifteenth ACM international conference on web search and data mining,pp\. 1215–1223\.Cited by:[§1](https://arxiv.org/html/2608.03025#S1.p2.1),[§4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2608.03025#S4.T2.3.6.6.1),[§5](https://arxiv.org/html/2608.03025#S5.p1.1),[§5](https://arxiv.org/html/2608.03025#S5.p2.1)\.
- J\. Yu, J\. Jiang, L\. Yang, and R\. Xia \(2020\)Improving multimodal named entity recognition via entity span detection with unified multimodal transformer\.InProceedings of the 58th annual meeting of the association for computational linguistics,pp\. 3342–3352\.Cited by:[§1](https://arxiv.org/html/2608.03025#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.03025#S4.SS1.p1.1),[§4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2608.03025#S4.T2.3.3.3.1),[§5](https://arxiv.org/html/2608.03025#S5.p1.1)\.
- Q\. Zhang, J\. Fu, X\. Liu, and X\. Huang \(2018\)Adaptive co\-attention network for named entity recognition in tweets\.InProceedings of the AAAI conference on artificial intelligence,Vol\.32\.Cited by:[§1](https://arxiv.org/html/2608.03025#S1.p1.1),[§4\.5](https://arxiv.org/html/2608.03025#S4.SS5.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2608.03025#S4.T2.3.4.4.1),[§5](https://arxiv.org/html/2608.03025#S5.p1.1)\.
- X\. Zhang, J\. Yuan, L\. Li, and J\. Liu \(2023\)Reducing the bias of visual objects in multimodal named entity recognition\.InProceedings of the Sixteenth ACM international conference on web search and data mining,pp\. 958–966\.Cited by:[Table 2](https://arxiv.org/html/2608.03025#S4.T2.3.7.7.1)\.
- B\. Zhou, Y\. Zhang, K\. Song, W\. Guo, G\. Zhao, H\. Wang, and X\. Yuan \(2022\)A span\-based multimodal variational autoencoder for semi\-supervised multimodal named entity recognition\.InProceedings of the 2022 conference on empirical methods in natural language processing,pp\. 6293–6302\.Cited by:[Table 2](https://arxiv.org/html/2608.03025#S4.T2.3.8.8.1)\.