Multi-Perspective Evidence Synthesis and Reasoning for Unsupervised Multimodal Entity Linking

# Multi-Perspective Evidence Synthesis and Reasoning for Unsupervised Multimodal Entity Linking
Source: [https://arxiv.org/html/2604.20283](https://arxiv.org/html/2604.20283)
\(2027\)

###### Abstract\.

Multimodal Entity Linking \(MEL\) is a fundamental task in data management that maps ambiguous mentions with diverse modalities to the multimodal entities in a knowledge base\. However, most existing MEL approaches primarily focus on optimizing instance\-centric features and evidence, leaving broader forms of evidence and their intricate interdependencies insufficiently explored\. Motivated by the observation that human expert decision\-making process relies on multi\-perspective judgment, in this work, we proposeMSR\-MEL, aMulti\-perspective EvidenceSynthesis andReasoning framework with Large Language Models \(LLMs\) for unsupervisedMEL\. Specifically, we adopt a two\-stage framework:\(1\) Offline Multi\-Perspective Evidence Synthesisconstructs a comprehensive set of evidence\. This includes instance\-centric evidence capturing the instance\-centric multimodal information of mentions and entities, group\-level evidence that aggregates neighborhood information, lexical evidence based on string overlap ratio, and statistical evidence based on simple summary statistics\. A core contribution of our framework is the synthesis of group\-level evidence, which effectively aggregates vital neighborhood information by graph\. We first construct LLM\-enhanced contextualized graphs\. Subsequently, different modalities are jointly aligned through an asymmetric teacher\-student graph neural network\.\(2\) Online Multi\-Perspective Evidence Reasoningleverages the power of LLM as a reasoning module to analyze the correlation and semantics of the multi\-perspective evidence to induce an effective ranking strategy for accurate entity linking without supervision\. Extensive experiments on widely used MEL benchmarks demonstrate that MSR\-MEL consistently outperforms state\-of\-the\-art unsupervised methods\. The source code of this paper was available at:[https://anonymous\.4open\.science/r/MSR\-MEL\-C21E/](https://anonymous.4open.science/r/MSR-MEL-C21E/)\.

Multimodal Entity Linking, Multi\-Perspective Reasoning, Graph Neural Networks, Large Language Models

††copyright:none††journalyear:2027††conference:ACM SIGMOD/PODS International Conference on Management of Data; June 13–19, 2027; Huntington Beach, CA, USA††isbn:978\-1\-4503\-XXXX\-X/2018/06††ccs:Do Not Use This Code Generate the Correct Terms for Your Paper††ccs:Do Not Use This Code Generate the Correct Terms for Your Paper††ccs:Do Not Use This Code Generate the Correct Terms for Your Paper††ccs:Do Not Use This Code Generate the Correct Terms for Your Paper## 1\.Introduction

Entity Linking \(EL\) is a core task in intelligent information processing that maps ambiguous mentions to specific entries in a Knowledge Base \(KB\)\(Cucerzan,[2007](https://arxiv.org/html/2604.20283#bib.bib35); Delpeuch,[2019](https://arxiv.org/html/2604.20283#bib.bib36); Dubeyet al\.,[2018](https://arxiv.org/html/2604.20283#bib.bib37); Fanget al\.,[2019](https://arxiv.org/html/2604.20283#bib.bib38); Chenet al\.,[2024](https://arxiv.org/html/2604.20283#bib.bib7)\)\. With the rapid growth of multimedia data on the web, this task has extended to Multimodal Entity Linking \(MEL\)\. Unlike traditional EL which relies solely on text, MEL commonly utilizes textual context and visual information to identify entities\(Moonet al\.,[2018](https://arxiv.org/html/2604.20283#bib.bib16)\)\. As a pivotal upstream task, MEL plays a vital role in various real\-world applications, such as multimodal information retrieval, question answering, and social media recommendation\(De Caoet al\.,[2019](https://arxiv.org/html/2604.20283#bib.bib11); Longpreet al\.,[2021](https://arxiv.org/html/2604.20283#bib.bib12); Xionget al\.,[2019](https://arxiv.org/html/2604.20283#bib.bib13); Genget al\.,[2022](https://arxiv.org/html/2604.20283#bib.bib14)\)\.

###### Example0\.

Figure[1](https://arxiv.org/html/2604.20283#S1.F1)shows a representative multimodal entity linking example\. The mention combines a short text \(“Oxford published a new study on vaccine efficacy”\) with an associated image of a university campus\. Given multiple candidate entities sharing the same word ‘Oxford’, several candidates exhibit partial textual or visual relevance\. However, only the University of Oxford correctly matches the real\-world reference, highlighting the ambiguity that arises from relying on isolated modality\.

![Refer to caption](https://arxiv.org/html/2604.20283v1/x1.png)Figure 1\.An example of multimodal entity linking\.Given its significance, MEL has attracted increasing research attention\. Existing approaches are typically categorized into supervised and unsupervised paradigms, with representative methods compared in Table[1](https://arxiv.org/html/2604.20283#S1.T1)\. Supervised methods model the MEL task as a classification problem on annotated mention–entity pairs, learning a score that indicates the confidence in assigning multimodal mentions to entities\(Eshelet al\.,[2017](https://arxiv.org/html/2604.20283#bib.bib17); Wanget al\.,[2022a](https://arxiv.org/html/2604.20283#bib.bib1); Songet al\.,[2024](https://arxiv.org/html/2604.20283#bib.bib5); Huet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib19); Liuet al\.,[2024b](https://arxiv.org/html/2604.20283#bib.bib9); Moonet al\.,[2018](https://arxiv.org/html/2604.20283#bib.bib16); Luoet al\.,[2023](https://arxiv.org/html/2604.20283#bib.bib23)\)\. For example, DWE enhances multimodal matching through query and entity refinement\(Songet al\.,[2024](https://arxiv.org/html/2604.20283#bib.bib5)\), while M3EL models finer\-grained intra\- and cross\-modal interactions\(Huet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib19)\)\. UniMEL and KGMEL further incorporate Large Language Models \(LLMs\) for contextual enhancement or candidate selection\(Liuet al\.,[2024b](https://arxiv.org/html/2604.20283#bib.bib9); Kimet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib10)\)\.

Supervised methods rely heavily on large amounts of labeled data in MEL, and collecting such multimodal mention–entity annotations is expensive\. To alleviate the dependence on manual annotation, recent unsupervised methods have emerged\. These approaches generally leverage multimodal models to generate representations and perform MEL via ranking on the similarity score of the representations\(Luoet al\.,[2023](https://arxiv.org/html/2604.20283#bib.bib23); Shiet al\.,[2024](https://arxiv.org/html/2604.20283#bib.bib24); Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21)\)\. Among these unsupervised approaches, BERT\-based methods serve as simple baselines\(Devlinet al\.,[2019](https://arxiv.org/html/2604.20283#bib.bib20)\), while OpenMEL and DeepMEL further introduce LLMs\-based enhancement or agent\-style interaction\(Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21); Wanget al\.,[2026](https://arxiv.org/html/2604.20283#bib.bib26)\)\.

Table 1\.Comparison of MEL methods\. Evidence type: Einst\{\}\_\{\\text\{inst\}\}= instance\-centric evidence, Egroup\{\}\_\{\\text\{group\}\}= group\-level evidence, Elex\{\}\_\{\\text\{lex\}\}= lexical evidence, Estat\{\}\_\{\\text\{stat\}\}= statistical evidence\.ApproachLabel\-free?Evidence TypeLLM RoleEinst\{\}\_\{\\text\{inst\}\}Egroup\{\}\_\{\\text\{group\}\}Elex\{\}\_\{\\text\{lex\}\}Estat\{\}\_\{\\text\{stat\}\}DWE\(Songet al\.,[2024](https://arxiv.org/html/2604.20283#bib.bib5)\)✘✔✘✘✘✘M3EL\(Huet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib19)\)✘✔✘✘✘✘UniMEL\(Liuet al\.,[2024b](https://arxiv.org/html/2604.20283#bib.bib9)\)✘✔✘✘✘Context EnhancerKGMEL\(Kimet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib10)\)✘✔✘✘✘Context Enhancer \+ SelectorI2CR\(Liuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib6)\)✘✔✘✔✘ControllerOpenMEL\(Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21)\)✔✔✘✔✘Context EnhancerBERT\(Devlinet al\.,[2019](https://arxiv.org/html/2604.20283#bib.bib20)\)✔✔✘✘✘✘DeepMEL\(Wanget al\.,[2026](https://arxiv.org/html/2604.20283#bib.bib26)\)✔✔✘✘✘AgentsOurs \(MSR\-MEL\)✔✔✔✔✔Context Enhancer \+ Decision inducer

Motivations\.Despite this progress, current supervised and unsupervised MEL methods primarily focus on optimizing instance\-centric multimodal features through various representation learning techniques\(Wanget al\.,[2024a](https://arxiv.org/html/2604.20283#bib.bib58); Xinget al\.,[2023](https://arxiv.org/html/2604.20283#bib.bib40)\)\. Although some methods employ contrastive learning with context information, they do not directly fuse the neighborhood feature\(Luoet al\.,[2024](https://arxiv.org/html/2604.20283#bib.bib39)\)\. Therefore, they fail to exploit broader forms of evidence and capture the intricate relationships and nuanced interactions within multimodal evidence, making it insufficient for complex disambiguation tasks in MEL settings, where multimodal data is often partially missing and inconsistent\(Wuet al\.,[2024](https://arxiv.org/html/2604.20283#bib.bib41),[2024](https://arxiv.org/html/2604.20283#bib.bib41); Zhanget al\.,[2024](https://arxiv.org/html/2604.20283#bib.bib43)\)\. As detailed in our experimental analysis \(Table[3](https://arxiv.org/html/2604.20283#S7.T3)\), even widely\-used MEL benchmarks still suffer from significant data incompleteness, and the entity image ratios for WikiMEL, RichpediaMEL, and WikiDiverse are only 67\.26%, 57\.41%, and 50\.16%, respectively\.

These observations suggest that relying solely on instance\-centric evidence is insufficient for robust MEL, especially when multimodal information is incomplete or inconsistent\. In contrast, human expert decision\-making rarely relies on a single cue or evidence\. Instead, humans naturally aggregate evidence from multiple perspectives\(de Langeet al\.,[2010](https://arxiv.org/html/2604.20283#bib.bib44)\), such as semantic relevance, contextual coherence, and relational consistency, and reason about their consistency to reach a decision\. This observation suggests that effective MEL may benefit from multi\-perspective evidence with explicit reasoning over their interactions\.

Challenges\.Multi\-perspective MEL is inherently non\-trivial and faces two fundamental challenges:

Challenge I: How to extract and synthesize stable, complementary multi\-perspective evidence from sparse and low\-quality multimodal data?In real\-world scenarios, multimodal data are harvested from heterogeneous sources, which inherently leads to missing modalities and inconsistent data quality\(Xuet al\.,[2021](https://arxiv.org/html/2604.20283#bib.bib60)\)\. Moreover, different modalities often provide complementary cues at different levels of granularity, but these cues are not always well aligned semantically\(Li and Tang,[2024](https://arxiv.org/html/2604.20283#bib.bib53)\)\. Therefore, synthesizing robust and comprehensive evidence from such sparse, noisy, and weakly aligned data remains a fundamental challenge\.

Challenge II: How to effectively reason over multi\-perspective evidence in a fully unsupervised setting?Different evidence perspectives carry distinct semantic meanings and may provide complementary or even conflicting signals\. Furthermore, the lack of labels makes it difficult to decide how to weigh and utilize different types of evidence effectively\. Effectively reasoning over such multi\-perspective evidence to support accurate MEL without supervision thus poses a significant challenge\.

Ours approaches\.To address the aforementioned challenges, we proposeMSR\-MEL, aMulti\-perspective EvidenceSynthesis andReasoning framework with LLMs for unsupervisedMEL\. Specifically, MSR\-MEL follows a two\-stage design:

\(1\) Offline Stage: Multi\-Perspective Evidence Synthesis\.To addressChallenge I, this stage performs offline evidence synthesis to construct a comprehensive set of complementary evidence from multiple perspectives\. This includes: \(i\) instance\-centric evidence derived from the multimodal representations of the instance; \(ii\) group\-level evidence that captures vital neighborhood information\. Since instance\-level features are often sparse or low\-quality in real\-world scenarios, aggregating these broader relational contexts is essential for robust disambiguation and MEL; \(iii\) lexical evidence based on string overlap; \(iv\) statistical evidence based on a simple summary of the previous evidence\.

A core component of this stage is the synthesis of group\-level evidence, which effectively aggregates rich neighborhood information\. To achieve this, we first construct LLM\-enhanced contextualized graphs to encode high\-fidelity semantic relationships between mentions and entities\. Then, different modalities are jointly aligned through an asymmetric teacher\-student graph neural network, enabling coherent and noise\-resistant evidence representation across perspectives\.

\(2\) Online Stage: Multi\-Perspective Evidence Reasoning\.To addressChallenge II, we leverage LLMs to integrate and reason over multi\-perspective evidence for candidate re\-ranking\. Specifically, we first perform candidate selection to obtain a high\-recall candidate set\. Then, the LLM explicitly analyzes the synthesized evidence and re\-ranks candidate entities by inducing a tree\-like reasoning strategy\. The resulting decision tree defines an interpretable reasoning path for each mention–entity pair\. Candidate entities are scored by aggregating evidence contributions along the reasoning path, yielding a final ranking score that reflects evidence strength and cross\-evidence consistency\.

Contributions\.The main contributions can be summarized:

- •We proposeMSR\-MEL, a novel multi\-perspective evidence synthesis and reasoning framework with LLMs for unsupervised MEL, which formulates the task as a two\-stage process: an offline stage for multi\-perspective evidence synthesis and an online stage for multi\-perspective evidence reasoning\.
- •We introduce a comprehensive multi\-perspective evidence set that integrates instance\-centric, group\-level, lexical, and statistical evidence, enabling robust evidence synthesis from sparse and low\-quality multimodal data\.
- •We develop an LLM\-driven evidence reasoning mechanism that induces tree\-like ranking strategies at inference time, allowing effective entity selection without supervision\.
- •Extensive experiments on widely used MEL benchmarks demonstrate that MSR\-MEL consistently outperforms prior unsupervised methods\. In particular, compared with OpenMEL\(Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21)\), the previous state\-of\-the\-art unsupervised approach, MSR\-MEL improves Hit@1 by an average of 13\.04% across benchmarks, while also exhibiting stronger robustness and efficiency in challenging multimodal scenarios\.

## 2\.Preliminary

In this section, we formally define the MEL problem\.

Notations\.The multimodal corpus comprises a set of mentionsℳ=\{mi\}i=1\|ℳ\|\\mathcal\{M\}=\\\{m\_\{i\}\\\}\_\{i=1\}^\{\|\\mathcal\{M\}\|\}\. Each mentionmim\_\{i\}is defined by a textual description \(including the mention and its context\) and an associated visual image\. Formally, each mentionmim\_\{i\}is defined as a tripletmi=\(ni,ti,vi\)m\_\{i\}=\(n\_\{i\},t\_\{i\},v\_\{i\}\), wherenin\_\{i\}denotes the mention name,tit\_\{i\}represents the surrounding textual context, andviv\_\{i\}is the associated visual image\. It is important to note thatviv\_\{i\}may be null \(∅\\emptyset\) or noisy due to data incompleteness\. Correspondingly, the target KB comprises a collection of entitiesℰ=\{ej\}j=1\|ℰ\|\\mathcal\{E\}=\\\{e\_\{j\}\\\}\_\{j=1\}^\{\|\\mathcal\{E\}\|\}\. Each entityeje\_\{j\}is characterized by a similar triplet structureej=\(nj,tj,vj\)e\_\{j\}=\(n\_\{j\},t\_\{j\},v\_\{j\}\), wherenjn\_\{j\}is the canonical name,tjt\_\{j\}is the textual description derived from the KB, andvjv\_\{j\}is the corresponding image\. In practice, entity modalities may also be incomplete or noisy, which further increases the ambiguity of the linking process\. Moreover, multiple candidate entities may share similar names or partially overlapping textual and visual signals, making reliable disambiguation non\-trivial\. Based on these notations, the task is defined as follows:

###### Definition0 \(Multi\-modal Entity Linking\)\.

Given a query mentionmi∈ℳm\_\{i\}\\in\\mathcal\{M\}and a KBℰ\\mathcal\{E\}, the goal is to retrieve the ground\-truth entitye∗e^\{\*\}\. Formally, for each mentionmim\_\{i\}, we aim to identify the predicted entitye^i\\hat\{e\}\_\{i\}from a candidate set𝒞i⊆ℰ\\mathcal\{C\}\_\{i\}\\subseteq\\mathcal\{E\}by maximizing a multi\-modal matching scoreΦ​\(mi,ej\)\\Phi\(m\_\{i\},e\_\{j\}\):

\(1\)e^i=arg⁡maxej∈𝒞i⁡Φ​\(mi,ej\)\.\\hat\{e\}\_\{i\}=\\arg\\max\_\{e\_\{j\}\\in\\mathcal\{C\}\_\{i\}\}\\Phi\(m\_\{i\},e\_\{j\}\)\.

## 3\.Framework Overview

![Refer to caption](https://arxiv.org/html/2604.20283v1/x2.png)Figure 2\.Overview of the proposed MSR\-MEL framework\. MSR\-MEL adopts a two\-stage design\. Offline multi\-perspective evidence synthesis \(Top\): given a multimodal corpus, we construct four evidence types for each mention–entity pair: \(1\) instance\-centric evidence; \(2\) group\-level evidence; \(3\) lexical evidence; and \(4\) statistical evidence\. Group\-level evidence synthesis \(Top\-Right\): candidate edges are generated through CLIP\-based gating and LLM\-enhanced semantic expansion, followed by graph construction and modality alignment to obtain neighborhood\-aware representations\. Online multi\-perspective evidence reasoning \(Bottom\): A coarse multi\-view retrieval stage first assembles a high\-recall candidate set and outputs an initial fused score\. The synthesized evidence is then fed into an LLM\-driven reasoning module, which integrates heterogeneous evidence and refines an initial fused score to produce the final entity ranking\.Table 2\.Symbols and descriptions\.\\rowcolor\[HTML\]C0C0C0CategorySymbolDescriptionEvidence Type𝐟inst​\(m,e\)\\mathbf\{f\}\_\{\\text\{inst\}\}\(m,e\)Instance\-centric evidence𝐟group​\(m,e\)\\mathbf\{f\}\_\{\\text\{group\}\}\(m,e\)Group\-level evidence𝐟lex​\(m,e\)\\mathbf\{f\}\_\{\\text\{lex\}\}\(m,e\)Lexical evidence𝐟stat​\(m,e\)\\mathbf\{f\}\_\{\\text\{stat\}\}\(m,e\)Statistical evidenceSets / Graphℳ,ℰ\\mathcal\{M\},\\mathcal\{E\}Mention set and entity set𝒱=ℳ∪ℰ\\mathcal\{V\}=\\mathcal\{M\}\\cup\\mathcal\{E\}Node set \(mentions \+ entities\)E\(0\)E^\{\(0\)\}CLIP\-gated candidate edge setE\(1\)E^\{\(1\)\}LLM\-enhanced candidate edge setEgraphE\_\{\\text\{graph\}\}Final edge set \(E\(0\)∪E\(1\)E^\{\(0\)\}\\cup E^\{\(1\)\}\)𝒢=\(𝒱,Egraph\)\\mathcal\{G\}=\(\\mathcal\{V\},E\_\{\\text\{graph\}\}\)Contextualized multimodal graphSubgraph𝒢vi\\mathcal\{G\}\_\{v\_\{i\}\}PPR subgraph centered at nodeviv\_\{i\}KKNumber of neighbors in PPR subgraph extraction𝒢vitext\\mathcal\{G\}^\{\\text\{text\}\}\_\{v\_\{i\}\}Text\-view subgraph with features𝐭v\\mathbf\{t\}\_\{v\}𝒢viimg\\mathcal\{G\}^\{\\text\{img\}\}\_\{v\_\{i\}\}Image\-view subgraph with features𝐯v\\mathbf\{v\}\_\{v\}GNN / Repsfθ​\(⋅\)f\_\{\\theta\}\(\\cdot\)GNN encoder \(shared backbone\)𝐡vi\\mathbf\{h\}\_\{v\_\{i\}\}Contextualized node representation𝐬vi\\mathbf\{s\}\_\{v\_\{i\}\}Subgraph summary \(mean pooling\)𝐡vitxt,𝐡viimg\\mathbf\{h\}^\{\\text\{txt\}\}\_\{v\_\{i\}\},\\mathbf\{h\}^\{\\text\{img\}\}\_\{v\_\{i\}\}Text / image GNN representationsEvidence Reasoning𝒞​\(m\)\\mathcal\{C\}\(m\)Candidate entity set for mentionmmΔn\\Delta\_\{n\}Adjustment score of pathnnsprior​\(m,e\)s\_\{\\text\{prior\}\}\(m,e\)Initial score generated by the LLMsfinal​\(m,e\)s\_\{\\text\{final\}\}\(m,e\)Final score after re\-rankingπLLM​\(⋅\)\\pi\_\{\\text\{LLM\}\}\(\\cdot\)LLM\-generated ranking functionLoss / Paramsℒstruct\\mathcal\{L\}\_\{\\text\{struct\}\}Structural contrastive lossℒxmodal\\mathcal\{L\}\_\{\\text\{xmodal\}\}Cross\-modal InfoNCE lossλ1,λ2\\lambda\_\{1\},\\lambda\_\{2\}Loss balancing coefficientsτ,η\\tau,\\etaTemperature and margin hyperparameters

The overall architecture of MSR\-MEL is illustrated in Figure[2](https://arxiv.org/html/2604.20283#S3.F2)\. MSR\-MEL adopts a two\-stage design consisting of an offline multi\-perspective evidence synthesis stage and an online multi\-perspective evidence reasoning stage for unsupervised MEL\. For clarity, the key symbols and notations used throughout this framework are summarized in Table[2](https://arxiv.org/html/2604.20283#S3.T2)\.

In the Offline Stage, for each mention\-entity pair, we construct an evidence set composed of instance\-centric, group\-level, lexical, and statistical evidence\. A core component of this stage is the synthesis of group\-level evidence, which aggregates vital neighborhood context beyond the individual mention\-entity pair\. Specifically, we construct LLM\-enhanced contextualized graphs to encode high\-fidelity semantic relationships and aggregate broader neighborhood information that compensates for isolated local information\. Subsequently, an asymmetric teacher\-student alignment strategy is employed to learn robust cross\-modal representations\. By guiding the image student with the stable text teacher, this strategy ensures consistent alignment and protects the model from being skewed by noisy signals\.

In the Online Stage, MSR\-MEL executes a structured inference process to balance efficiency and accuracy\. It first performs candidate selection to ensure high recall\. Then, the synthesized evidence vector is processed by an LLM\-driven re\-ranking module\. Instead of acting as a traditional black\-box scorer, the LLM induces an interpretable reasoning path to evaluate the consistency of multi\-perspective evidence and generates dynamic score adjustments to produce the final refined entity ranking\.

To theoretically support this design, we first present the following theorem on the benefit of using multiple evidence\.

###### Theorem1\.

Consider an idealized multi\-perspective estimatorθm​p\\theta^\{mp\}that integratesKKcomplementary evidence channels, and a single\-perspective estimatorθks​p\\theta^\{sp\}\_\{k\}that relies only on thekk\-th channel\. Letℛ​\(⋅\)\\mathcal\{R\}\(\\cdot\)denote the estimation risk, defined as the expected squared error in predicting the latent compatibility score between a mention and an entity\. Assume that each evidence channel provides an unbiased noisy estimate of the same latent compatibility, and that the channel noises admit a positive\-definite covariance matrix\. Then, the minimum\-risk multi\-perspective estimator satisfies

ℛ​\(θm​p\)≤ℛ​\(θks​p\),∀k∈\{1,…,K\}\.\\mathcal\{R\}\(\\theta^\{mp\}\)\\leq\\mathcal\{R\}\(\\theta^\{sp\}\_\{k\}\),\\quad\\forall k\\in\\\{1,\\dots,K\\\}\.Moreover, if the optimal fusion uses a non\-degenerate combination of at least two complementary channels, the inequality is strict\.

## 4\.Multi\-Perspective Evidence Synthesis

### 4\.1\.Multi\-Perspective Evidence Formulation

Relying only on instance\-centric similarity is often insufficient in MEL, this motivates the need for a more comprehensive representation that captures complementary signals from multiple perspectives\. Accordingly, we design a structured set of multi\-perspective evidence that jointly characterizes the compatibility between a mention–entity pair\. Each mention–entity pair\(m,e\)\(m,e\)is represented by a multi\-perspective evidence vector𝐟​\(m,e\)∈ℝ14\\mathbf\{f\}\(m,e\)\\in\\mathbb\{R\}^\{14\}, which serves as the unified input to the LLM\-driven reasoning module\. Formally, the evidence vector is composed of four evidence types:

\(2\)𝐟​\(m,e\)\\displaystyle\\mathbf\{f\}\(m,e\)=\[𝐟inst\(m,e\),𝐟group\(m,e\),\\displaystyle=\\big\[\\mathbf\{f\}\_\{\\text\{inst\}\}\(m,e\),\\ \\mathbf\{f\}\_\{\\text\{group\}\}\(m,e\),𝐟lex\(m,e\),𝐟stat\(m,e\)\]∈ℝ14\.\\displaystyle\\qquad\\mathbf\{f\}\_\{\\text\{lex\}\}\(m,e\),\\ \\mathbf\{f\}\_\{\\text\{stat\}\}\(m,e\)\\big\]\\in\\mathbb\{R\}^\{14\}\.
\(a\) Instance\-centric Evidence \(𝐟inst​\(m,e\)\\mathbf\{f\}\_\{\\text\{inst\}\}\(m,e\)\)\.This evidence type captures instance\-centric textual and visual features between the mention and the candidate entity based on CLIP embeddings\(Radfordet al\.,[2021](https://arxiv.org/html/2604.20283#bib.bib4)\)\. We compute cosine similarity scores across four modality combinations, where the first and second letters denote mention and the entity, respectively: text–text \(TT\), text–image \(TV\), image–text \(VT\), and image–image \(VV\)\. Accordingly, the instance\-centric evidence is defined as:

\(3\)𝐟inst​\(m,e\)=\[sinstTT,sinstTV,sinstVT,sinstVV\]\.\\mathbf\{f\}\_\{\\text\{inst\}\}\(m,e\)=\\big\[s^\{\\text\{TT\}\}\_\{\\text\{inst\}\},s^\{\\text\{TV\}\}\_\{\\text\{inst\}\},s^\{\\text\{VT\}\}\_\{\\text\{inst\}\},s^\{\\text\{VV\}\}\_\{\\text\{inst\}\}\\big\]\.
\(b\) Group\-level Evidence \(𝐟group​\(m,e\)\\mathbf\{f\}\_\{\\text\{group\}\}\(m,e\)\)\.Given that instance\-centric features are often insufficient due to sparse and low\-quality multimodal data, we introduce group\-level evidence to aggregate vital neighborhood information\. This evidence type comprises cosine similarity scores computed from graph\-enhanced embeddings learned via a contextualized multimodal graph, details are in Section[4\.2](https://arxiv.org/html/2604.20283#S4.SS2)and Section[4\.3](https://arxiv.org/html/2604.20283#S4.SS3)\. Formally, we define the group\-level evidence as:

\(4\)𝐟group​\(m,e\)=\[sgroupTT,sgroupTV,sgroupVT,sgroupVV\]\.\\mathbf\{f\}\_\{\\text\{group\}\}\(m,e\)=\\big\[s^\{\\text\{TT\}\}\_\{\\text\{group\}\},s^\{\\text\{TV\}\}\_\{\\text\{group\}\},s^\{\\text\{VT\}\}\_\{\\text\{group\}\},s^\{\\text\{VV\}\}\_\{\\text\{group\}\}\\big\]\.
\(c\) Lexical Evidence \(𝐟lex​\(m,e\)\\mathbf\{f\}\_\{\\text\{lex\}\}\(m,e\)\)\.We also incorporate a normalized lexical matching score based on the string overlap between the mention and the entity name\. This evidence serves as a symbolic anchor that is particularly effective for handling aliases, name variants, and near\-exact matches\. Specifically, given a mention stringmmand a candidate entity nameee, we compute their lexical similarity using the Gestalt Pattern Matching principle, which recursively identifies matched subsequences between the two strings\. LetM​\(m,e\)M\(m,e\)denote the total length of all matched subsequences, and let\|s\|\|s\|denote the length of a stringss\. The lexical similarity score is defined as:

\(5\)slex​\(m,e\)=2⋅M​\(m,e\)\|m\|\+\|e\|,s\_\{\\text\{lex\}\}\(m,e\)=\\frac\{2\\cdot M\(m,e\)\}\{\|m\|\+\|e\|\},whereslex​\(m,e\)∈\[0,1\]s\_\{\\text\{lex\}\}\(m,e\)\\in\[0,1\]\. This case\-insensitive metric is robust to minor character\-level variations, such as typos or spelling differences, making it particularly reliable for near\-exact lexical matching\. The lexical evidence is then represented as:

\(6\)𝐟lex​\(m,e\)=\[slex​\(m,e\)\]\.\\mathbf\{f\}\_\{\\text\{lex\}\}\(m,e\)=\\big\[s\_\{\\text\{lex\}\}\(m,e\)\\big\]\.
\(d\) Statistical Evidence \(𝐟stat​\(m,e\)\\mathbf\{f\}\_\{\\text\{stat\}\}\(m,e\)\)\.To enable reliability\-aware reasoning, we summarize the distributional characteristics of neural similarity scores, as such descriptors are crucial for capturing inherent modal uncertainty\(Zhouet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib52)\)\. Specifically, we summarize the similarity evidence by three statistical descriptors: the mean similarityμ​\(m,e\)\\mu\(m,e\), the maximum similaritysmax​\(m,e\)s\_\{\\max\}\(m,e\), and the dominance gapΔ​\(m,e\)\\Delta\(m,e\)\. The mean similarity is defined as the average of the preceding similarity scores:

\(7\)μ​\(m,e\)=∑s,s∈\{𝐟inst​\(m,e\),𝐟group​\(m,e\),𝐟lex​\(m,e\)\}\|𝐟inst​\(m,e\)\|\+\|𝐟group​\(m,e\)\|\+\|𝐟lex​\(m,e\)\|\.\\mu\(m,e\)=\\frac\{\\sum s,\\quad s\\in\\left\\\{\\mathbf\{f\}\_\{\\text\{inst\}\}\(m,e\),\\mathbf\{f\}\_\{\\text\{group\}\}\(m,e\),\\mathbf\{f\}\_\{\\text\{lex\}\}\(m,e\)\\right\\\}\}\{\\left\|\\mathbf\{f\}\_\{\\text\{inst\}\}\(m,e\)\\right\|\+\\left\|\\mathbf\{f\}\_\{\\text\{group\}\}\(m,e\)\\right\|\+\\left\|\\mathbf\{f\}\_\{\\text\{lex\}\}\(m,e\)\\right\|\}\.whereμ​\(m,e\)\\mu\(m,e\)denotes the mean of all preceding similarity values contained in𝐟inst​\(m,e\)\\mathbf\{f\}\_\{\\text\{inst\}\}\(m,e\),𝐟group​\(m,e\)\\mathbf\{f\}\_\{\\text\{group\}\}\(m,e\), and𝐟lex​\(m,e\)\\mathbf\{f\}\_\{\\text\{lex\}\}\(m,e\)\. The dominance gap is further defined as:

\(8\)Δ​\(m,e\)=smax​\(m,e\)−μ​\(m,e\)\.\\Delta\(m,e\)=s\_\{\\max\}\(m,e\)\-\\mu\(m,e\)\.
A large gap accompanied by a low mean typically indicates over\-reliance on a single weak cue\.

We further encode modality availability using binary indicators that denote whether visual information is present for the mention and the entity, allowing the reasoning module to down\-weight unreliable visual evidence when necessary\. Accordingly, the statistical evidence is defined as:

𝐟stat​\(m,e\)=\[μ​\(m,e\),smax​\(m,e\),Δ​\(m,e\),𝕀img​\(m\),𝕀img​\(e\)\]\.\\mathbf\{f\}\_\{\\text\{stat\}\}\(m,e\)=\\big\[\\mu\(m,e\),\\ s\_\{\\max\}\(m,e\),\\ \\Delta\(m,e\),\\ \\mathbb\{I\}\_\{\\text\{img\}\}\(m\),\\ \\mathbb\{I\}\_\{\\text\{img\}\}\(e\)\\big\]\.

### 4\.2\.Contextualized Graph Construction

To obtain group\-level evidence, we construct an undirected multimodal graph𝒢=\(𝒱,Eg​r​a​p​h\)\\mathcal\{G\}=\(\\mathcal\{V\},E\_\{graph\}\)where𝒱=ℳ∪ℰ\\mathcal\{V\}=\\mathcal\{M\}\\cup\\mathcal\{E\}\. Nodes are initially encoded via CLIP into a shared embedding space\. To refine connectivity, we combine modality\-aware gating with LLM\-driven semantic expansion\.

Step 1: CLIP\-based Candidate Selection\.We first construct coarse candidate edges using CLIP\-based representations\. Each nodevi∈𝒱v\_\{i\}\\in\\mathcal\{V\}is associated with a textual embedding𝐭vi\\mathbf\{t\}\_\{v\_\{i\}\}and a visual embedding𝐯vi\\mathbf\{v\}\_\{v\_\{i\}\}\(when available\)\. For any node pair\(vi,vj\)\(v\_\{i\},v\_\{j\}\), wherevi,vj∈𝒱v\_\{i\},v\_\{j\}\\in\\mathcal\{V\}may correspond to a mention–entity \(M–E\), mention–mention \(M–M\), or entity–entity \(E–E\) pair, we define a modality\-aware gated similarity:

\(9\)sgate​\(vi,vj\)=cos⁡\(𝐭vi,𝐭vj\)\+𝕀i​j⋅cos⁡\(𝐯vi,𝐯vj\),s\_\{\\text\{gate\}\}\(v\_\{i\},v\_\{j\}\)=\\cos\(\\mathbf\{t\}\_\{v\_\{i\}\},\\mathbf\{t\}\_\{v\_\{j\}\}\)\+\\mathbb\{I\}\_\{ij\}\\cdot\\cos\(\\mathbf\{v\}\_\{v\_\{i\}\},\\mathbf\{v\}\_\{v\_\{j\}\}\),where𝕀i​j=1\\mathbb\{I\}\_\{ij\}=1if bothviv\_\{i\}andvjv\_\{j\}have visual embeddings, and𝕀i​j=0\\mathbb\{I\}\_\{ij\}=0otherwise\. This gating mechanism explicitly conditions similarity computation on the availability of visual modalities\. When visual information is missing for either node, the similarity score is computed solely based on textual embeddings, thereby avoiding the introduction of noisy or undefined visual similarity terms\. Usingsgates\_\{\\text\{gate\}\}, we construct a set of candidate edges by thresholding the pairwise gating score:

\(10\)E\(0\)=\{\(vi,vj\)∈𝒱×𝒱\|sgate\(vi,vj\)≥δgate,\}\.E^\{\(0\)\}=\\left\\\{\(v\_\{i\},v\_\{j\}\)\\in\\mathcal\{V\}\\times\\mathcal\{V\}\\;\\middle\|\\;s\_\{\\text\{gate\}\}\(v\_\{i\},v\_\{j\}\)\\geq\\delta\_\{\\text\{gate\}\},\\right\\\}\.whereδgate\\delta\_\{\\text\{gate\}\}is a predefined similarity threshold\. The resulting candidate edge setE\(0\)E^\{\(0\)\}includes mention–entity \(M–E\), mention–mention \(M–M\), and entity–entity \(E–E\) edges\.

Prompt Design A: Entity & Mention EnrichmentA: Entity Enhancement \(1\)Task:Entity Description Generation for MEL task\(2\)Inputs:∙\\bulletTarget Entity:\{name\}∙\\bulletEntity Type:\{instance\_type\}∙\\bulletKnown Attributes:"\{attr\_str\}"\(use"No specific attributes provided\."if missing\)\(3\)Instructions:\(a\)You are an expert encyclopedia editor\. Identify the specific entity using name and attributes\.\(b\)Write a concise definition \(30–50 words\)\.\(c\)\[CRITICAL\]Do not list attributes\. Integrate them with specific facts \(e\.g\., works, significance\)\.\(d\)If ambiguous, use attributes to pick the correct entity\.\(4\)Output format:\[Canonical Name\]: \[Definition\]B: Mention Augmentation \(1\)Task:Entity Linking Positive Pair Generation\(2\)Inputs:∙\\bulletContext:"\{context\}"∙\\bulletTarget Mention:"\{mention\}"\(3\)Instructions:\(a\)Identify the unique canonical entity referred to by the mention in the given context\.\(b\)Provide theCanonical Name\.\(c\)Provide a concise definition \(30–50 words\) that uniquely identifies the entityglobally, independent of the context\.\(4\)Output format:\[Canonical Name\]: \[Definition\]

Step 2: LLM\-enhanced Candidate Selection\.To resolve ambiguity, we use LLMs to generate contextualized semantic descriptions\. Specifically, given the mention setℳ\\mathcal\{M\}and the entity setℰ\\mathcal\{E\}, we use an LLM to generate enhanced textual descriptions for all mentionsm∈ℳm\\in\\mathcal\{M\}and for a selected subset of entitiesℰk⊂ℰ\\mathcal\{E\}\_\{k\}\\subset\\mathcal\{E\}\. The generated textual descriptions are encoded by the CLIP text encoder, yielding contextualized text embeddings:

\(11\)𝐭~x=CLIPtext​\(LLMθ​\(Prompt​\(x\)\)\),x∈ℳ∪ℰk\.\\tilde\{\\mathbf\{t\}\}\_\{x\}=\\mathrm\{CLIP\}\_\{\\text\{text\}\}\\\!\\left\(\\mathrm\{LLM\}\_\{\\theta\}\\big\(\\mathrm\{Prompt\}\(x\)\\big\)\\right\),\\quad x\\in\\mathcal\{M\}\\cup\\mathcal\{E\}\_\{k\}\.Detailed prompt designs are provided in[Prompt Box A](https://arxiv.org/html/2604.20283#S4.SS2)\. We then compute an expanded similaritysl​l​ms\_\{llm\}by fusing original and enhanced signals between each node pair\(vi,vj\)\(v\_\{i\},v\_\{j\}\):

\(12\)sllm​\(vi,vj\)=w1​cos⁡\(𝐭vi,𝐭~vj\)\+w2​cos⁡\(𝐭~vi,𝐭vj\)\+w3​cos⁡\(𝐭~vi,𝐭~vj\)\.\\small s\_\{\\text\{llm\}\}\(v\_\{i\},v\_\{j\}\)=w\_\{1\}\\cos\(\\mathbf\{t\}\_\{v\_\{i\}\},\\tilde\{\\mathbf\{t\}\}\_\{v\_\{j\}\}\)\+w\_\{2\}\\cos\(\\tilde\{\\mathbf\{t\}\}\_\{v\_\{i\}\},\\mathbf\{t\}\_\{v\_\{j\}\}\)\+w\_\{3\}\\cos\(\\tilde\{\\mathbf\{t\}\}\_\{v\_\{i\}\},\\tilde\{\\mathbf\{t\}\}\_\{v\_\{j\}\}\)\.A semantic edge setE\(1\)E^\{\(1\)\}is constructed by retrieving the Top\-Kl​l​mK\_\{llm\}neighbors for each node based onsl​l​ms\_\{llm\}\.

Step 3: Edge Union for Final Graph\.Finally, we combine the CLIP\-based and LLM\-enhanced candidate edges:

\(13\)Egraph=E\(0\)∪E\(1\)\.E\_\{\\text\{graph\}\}=E^\{\(0\)\}\\cup E^\{\(1\)\}\.yielding the final undirected multimodal graph𝒢=\(𝒱,Egraph\)\\mathcal\{G\}=\(\\mathcal\{V\},E\_\{\\text\{graph\}\}\)\.

Through this multi\-stage construction process, MSR\-MEL builds a high\-fidelity contextualized graph that captures reliable neighborhood structures by combining coarse multimodal similarity cues with LLM\-enhanced semantic signals\(Xuet al\.,[2026](https://arxiv.org/html/2604.20283#bib.bib61)\)\. The constructed graph enables effective propagation of contextual information in the subsequent learning stage\.

### 4\.3\.Asymmetric Teacher–Student Alignment

Directly learning representations across modalities is often unstable in MEL, because different modalities exhibit substantially different data quality: textual signals are generally more complete and semantically stable, whereas visual signals are more noisy, ambiguous, or missing\. To address this challenge and learn robust group\-level representations, we propose an asymmetric teacher\-student alignment framework over two modality\-specific graph views\. Specifically, based on the same graph topology, we define a text view and an image view using textual and visual node features, respectively\. The text\-based GNN is trained with a structural contrastive loss to learn stable neighborhood structure, while the image\-based GNN is optimized with both structural and cross\-modal distillation loss from the text teacher\. This design is motivated by the fact that textual evidence is generally more complete and stable, whereas visual evidence is more likely to be noisy or missing in real\-world MEL data\.

4\.3\.1 Shared Subgraph Encoding\.For scalability, we utilize a subgraph\-based encoding strategy\. For each nodeviv\_\{i\}, a local subgraph𝒢vi\\mathcal\{G\}\_\{v\_\{i\}\}is extracted using Top\-KKPersonalized PageRank \(PPR\) scores\(Gasteigeret al\.,[2018](https://arxiv.org/html/2604.20283#bib.bib48)\)\. We define two modal\-specific views,𝒢vitext\\mathcal\{G\}^\{\\text\{text\}\}\_\{v\_\{i\}\}and𝒢viimg\\mathcal\{G\}^\{\\text\{img\}\}\_\{v\_\{i\}\}, which share the same topology but use textual𝐭v\\mathbf\{t\}\_\{v\}and visual𝐯v\\mathbf\{v\}\_\{v\}features respectively\.

While the proposed framework is agnostic to the choice of GNN architecture, we adopt a Graph Convolutional Network \(GCN\) as the base graph encoder, denoted byfθ​\(⋅\)f\_\{\\theta\}\(\\cdot\), for its computational efficiency and robust ability to capture local structural dependencies\(Kipf and Welling,[2016](https://arxiv.org/html/2604.20283#bib.bib54)\)\. Given a target nodeviv\_\{i\}and its subgraph𝒢vi\\mathcal\{G\}\_\{v\_\{i\}\}with node features𝐱v∈\{𝐭v,𝐯v\}\\mathbf\{x\}\_\{v\}\\in\\\{\\mathbf\{t\}\_\{v\},\\mathbf\{v\}\_\{v\}\\\}, the encoder produces contextualized representations\{𝐡v\}v∈𝒱vi\\\{\\mathbf\{h\}\_\{v\}\\\}\_\{v\\in\\mathcal\{V\}\_\{v\_\{i\}\}\}for all nodes in𝒢vi\\mathcal\{G\}\_\{v\_\{i\}\}\. In particular, the representation of the target nodeviv\_\{i\}is:

\(14\)𝐡vi=fθ​\(𝒢vi,\{𝐱v\}v∈𝒱vi\)\.\\mathbf\{h\}\_\{v\_\{i\}\}=f\_\{\\theta\}\\\!\\left\(\\mathcal\{G\}\_\{v\_\{i\}\},\\\{\\mathbf\{x\}\_\{v\}\\\}\_\{v\\in\\mathcal\{V\}\_\{v\_\{i\}\}\}\\right\)\.To capture subgraph\-level contextual information, we further aggregate the contextualized node representations in𝒢vi\\mathcal\{G\}\_\{v\_\{i\}\}through mean pooling\(Jiaoet al\.,[2020](https://arxiv.org/html/2604.20283#bib.bib25)\), yielding a neighborhood summary vector:

\(15\)𝐬vi=1\|𝒱vi\|​∑v∈𝒱vi𝐡v\.\\mathbf\{s\}\_\{v\_\{i\}\}=\\frac\{1\}\{\|\\mathcal\{V\}\_\{v\_\{i\}\}\|\}\\sum\_\{v\\in\\mathcal\{V\}\_\{v\_\{i\}\}\}\\mathbf\{h\}\_\{v\}\.
The text view and image view share the same subgraph structure, but use different node features, namely textual features and visual features, respectively\.

4\.3\.2 Text\-based Teacher GNN Training\.In the first stage, we train a text\-based GNN on the constructed contextualized graph using CLIP text embeddings as node features, i\.e\.,𝐱v=𝐭v\\mathbf\{x\}\_\{v\}=\\mathbf\{t\}\_\{v\}\. The goal of this stage is to learn stable and semantically coherent contextualized representations from the graph structure\.

Although no explicit LLM outputs are used as supervision in this training stage, the teacher GNN is trained on a graph whose topology has already been enriched during graph construction \(Section[4\.2](https://arxiv.org/html/2604.20283#S4.SS2)\)\. In particular, the LLM contributes by introducing semantically enhanced edges in the graph construction step, so its effect is incorporated implicitly through the resulting neighborhood structure rather than through direct supervision signals\. Here we adopt astructural contrastive objectiveto encourage consistency between a target node representation and its local neighborhood context\. For a target nodeviv\_\{i\}, we obtain its contextualized representation𝐡vi\\mathbf\{h\}\_\{v\_\{i\}\}from the text GNN and a subgraph\-level summary𝐬vi\\mathbf\{s\}\_\{v\_\{i\}\}via mean pooling over the corresponding subgraph\. We treat\(𝐡vi,𝐬vi\)\(\\mathbf\{h\}\_\{v\_\{i\}\},\\mathbf\{s\}\_\{v\_\{i\}\}\)as a positive pair\. To construct a negative pair, we randomly sample another nodevj≠viv\_\{j\}\\neq v\_\{i\}from the same mini\-batch and pair𝐡vi\\mathbf\{h\}\_\{v\_\{i\}\}with the summary vector𝐬vj\\mathbf\{s\}\_\{v\_\{j\}\}of its subgraph\. In other words, the negative pair is formed by shuffling the correspondence between target nodes and subgraph summaries within the batch\(Jiaoet al\.,[2020](https://arxiv.org/html/2604.20283#bib.bib25)\)\. Here, the structural contrastive loss is defined as:

\(16\)ℒstructtxt=1\|ℬ\|​∑vi∈ℬmax⁡\(0,η−σ​\(𝐡vi⊤​𝐬vi\)\+σ​\(𝐡vi⊤​𝐬vj\)\),\\mathcal\{L\}\_\{\\text\{struct\}\}^\{\\text\{txt\}\}=\\frac\{1\}\{\|\\mathcal\{B\}\|\}\\sum\_\{v\_\{i\}\\in\\mathcal\{B\}\}\\max\\\!\\left\(0,\\;\\eta\-\\sigma\(\\mathbf\{h\}\_\{v\_\{i\}\}^\{\\top\}\\mathbf\{s\}\_\{v\_\{i\}\}\)\+\\sigma\(\\mathbf\{h\}\_\{v\_\{i\}\}^\{\\top\}\\mathbf\{s\}\_\{v\_\{j\}\}\)\\right\),whereℬ\\mathcal\{B\}denotes a mini\-batch of target nodes,𝐬vj\\mathbf\{s\}\_\{v\_\{j\}\}is sampled from a shuffled subgraph,η\\etais a margin hyperparameter, andσ​\(⋅\)\\sigma\(\\cdot\)denotes the sigmoid function\.

The resulting text\-based GNN serves as the teacher network, providing semantically grounded contextualized representations for subsequent cross\-modal distillation\. Accordingly, the training objective of this stage isℒtext=ℒstructtxt\\mathcal\{L\}\_\{\\text\{text\}\}=\\mathcal\{L\}\_\{\\text\{struct\}\}^\{\\text\{txt\}\}\.

1

Input:Mention set

ℳ\\mathcal\{M\}, entity set

ℰ\\mathcal\{E\},

textual features

\{𝐭v\}\\\{\\mathbf\{t\}\_\{v\}\\\}, visual features

\{𝐯v\}\\\{\\mathbf\{v\}\_\{v\}\\\}
Output:Contextualized graph

𝒢=\(𝒱,Egraph\)\\mathcal\{G\}=\(\\mathcal\{V\},E\_\{\\mathrm\{graph\}\}\), teacher encoder

fθt​x​tf\_\{\\theta\}^\{txt\}, student encoder

fθi​m​gf\_\{\\theta\}^\{img\}
2

//Part I: Contextualized Graph Construction

3Initialize node set

𝒱←ℳ∪ℰ\\mathcal\{V\}\\leftarrow\\mathcal\{M\}\\cup\\mathcal\{E\}and

E\(0\)←∅E^\{\(0\)\}\\leftarrow\\emptyset
4

//\(1\) CLIP\-based candidate selection

5foreach*\(vi,vj\)∈𝒱×𝒱\(v\_\{i\},v\_\{j\}\)\\in\\mathcal\{V\}\\times\\mathcal\{V\}*do

6Compute

sgate​\(vi,vj\)s\_\{\\mathrm\{gate\}\}\(v\_\{i\},v\_\{j\}\)by Eq\. \([9](https://arxiv.org/html/2604.20283#S4.E9)\)

7if*sgate​\(vi,vj\)≥δgates\_\{\\mathrm\{gate\}\}\(v\_\{i\},v\_\{j\}\)\\geq\\delta\_\{\\mathrm\{gate\}\}*then

8

E\(0\)←E\(0\)∪\{\(vi,vj\)\}E^\{\(0\)\}\\leftarrow E^\{\(0\)\}\\cup\\\{\(v\_\{i\},v\_\{j\}\)\\\}
9

10

11

//\(2\) LLM\-enhanced candidate selection

12foreach*x∈ℳ∪ℰkx\\in\\mathcal\{M\}\\cup\\mathcal\{E\}\_\{k\}*do

13Generate LLM\-enhanced embedding

𝐭~x\\tilde\{\\mathbf\{t\}\}\_\{x\}
14

15

//\(3\) LLM\-enhanced edge construction

16Initialize

E\(1\)←∅E^\{\(1\)\}\\leftarrow\\emptyset
17foreach*vi∈𝒱v\_\{i\}\\in\\mathcal\{V\}*do

18Compute fused similarity

sllm​\(vi,vj\)s\_\{\\mathrm\{llm\}\}\(v\_\{i\},v\_\{j\}\)
19Retrieve Top\-

KllmK\_\{\\mathrm\{llm\}\}neighbors to update

E\(1\)E^\{\(1\)\}
20

21Set

Egraph←E\(0\)∪E\(1\)E\_\{\\mathrm\{graph\}\}\\leftarrow E^\{\(0\)\}\\cup E^\{\(1\)\}
22Set

𝒢←\(𝒱,Egraph\)\\mathcal\{G\}\\leftarrow\(\\mathcal\{V\},E\_\{\\mathrm\{graph\}\}\)
23

//Part II: Teacher\-\-Student Alignment

//Preparation: subgraph extraction

24foreach*vi∈𝒱v\_\{i\}\\in\\mathcal\{V\}*do

25Extract local subgraph

𝒢vi\\mathcal\{G\}\_\{v\_\{i\}\}via PPR

26

27

//Stage I: Text\-based Teacher Training

28Initialize teacher encoder

fθt​x​tf\_\{\\theta\}^\{txt\}
29while*not converged*do

30Compute

𝐡vi\\mathbf\{h\}\_\{v\_\{i\}\}and

𝐬vi\\mathbf\{s\}\_\{v\_\{i\}\}by Eq\. \([14](https://arxiv.org/html/2604.20283#S4.E14)\) and Eq\. \([15](https://arxiv.org/html/2604.20283#S4.E15)\)

31Minimize

ℒtext=ℒstructtxt\\mathcal\{L\}\_\{\\mathrm\{text\}\}=\\mathcal\{L\}^\{\\mathrm\{txt\}\}\_\{\\mathrm\{struct\}\}by Eq\. \([16](https://arxiv.org/html/2604.20283#S4.E16)\)

32

33Freeze

fθt​x​tf\_\{\\theta\}^\{txt\}
34

//Stage II: Image\-based Student Distillation

35Initialize student encoder

fθi​m​gf\_\{\\theta\}^\{img\}
36while*not converged*do

37Obtain teacher representation

𝐡vit​x​t\\mathbf\{h\}\_\{v\_\{i\}\}^\{txt\}from

fθt​x​tf\_\{\\theta\}^\{txt\}
38Compute student representation

𝐡vii​m​g\\mathbf\{h\}\_\{v\_\{i\}\}^\{img\}
39Minimize

ℒimg=ℒstructi​m​g\+λ1​ℒxmodal\\mathcal\{L\}\_\{\\mathrm\{img\}\}=\\mathcal\{L\}\_\{\\mathrm\{struct\}\}^\{img\}\+\\lambda\_\{1\}\\mathcal\{L\}\_\{\\mathrm\{xmodal\}\}by Eq\. \([18](https://arxiv.org/html/2604.20283#S4.E18)\)

40

41

42return*𝒢,fθt​x​t,fθi​m​g\\mathcal\{G\},f\_\{\\theta\}^\{txt\},f\_\{\\theta\}^\{img\}*

Algorithm 1Offline Group\-Level Evidence Synthesis4\.3\.3 Image\-based Student GNN Distillation\.While textual representations learned in Stage I are stable and semantically informative, visual features are often noisy, ambiguous, or entirely missing\. Directly learning image representations from the graph may therefore propagate noise and lead to unreliable embeddings\. To address this issue, we treat the text\-based GNN as a reliable semantic reference and distill its knowledge to guide visual representation learning\.

Specifically, we freeze the text\-based GNN trained in Stage I as a teacher network\. Given the same subgraph𝒢vi\\mathcal\{G\}\_\{v\_\{i\}\}, the teacher produces a fixed textual representation𝐡vitxt\\mathbf\{h\}^\{\\text\{txt\}\}\_\{v\_\{i\}\}, while an image\-based GNN \(the*student*\) takes CLIP image features𝐱v=𝐯v\\mathbf\{x\}\_\{v\}=\\mathbf\{v\}\_\{v\}as input and outputs a learnable visual representation𝐡viimg\\mathbf\{h\}^\{\\text\{img\}\}\_\{v\_\{i\}\}\. To align the student with the teacher in a shared semantic space, we introduce a cross\-modal contrastive objective\. For each target nodeviv\_\{i\}in a mini\-batchℬ\\mathcal\{B\}, we treat\(𝐡viimg,𝐡vitxt\)\(\\mathbf\{h\}^\{\\text\{img\}\}\_\{v\_\{i\}\},\\mathbf\{h\}^\{\\text\{txt\}\}\_\{v\_\{i\}\}\)as a positive pair, and use the teacher representations of all other nodes in the same mini\-batch, i\.e\.,\{𝐡vjtxt\}vj∈ℬ,vj≠vi\\\{\\mathbf\{h\}^\{\\text\{txt\}\}\_\{v\_\{j\}\}\\\}\_\{v\_\{j\}\\in\\mathcal\{B\},\\,v\_\{j\}\\neq v\_\{i\}\}, as negatives\. In this way, the image representation of each node is encouraged to align with its matched text representation, while being separated from the text representations of non\-matching nodes in the same mini\-batch\. The cross\-modal InfoNCE loss is defined as\(Oordet al\.,[2018](https://arxiv.org/html/2604.20283#bib.bib57)\):

\(17\)ℒxmodal=−1\|ℬ\|​∑vi∈ℬlog⁡exp⁡\(sim​\(𝐡viimg,𝐡vitxt\)/τ\)∑vj∈ℬexp⁡\(sim​\(𝐡viimg,𝐡vjtxt\)/τ\),\\mathcal\{L\}\_\{\\text\{xmodal\}\}=\-\\frac\{1\}\{\|\\mathcal\{B\}\|\}\\sum\_\{v\_\{i\}\\in\\mathcal\{B\}\}\\log\\frac\{\\exp\\\!\\left\(\\mathrm\{sim\}\(\\mathbf\{h\}^\{\\text\{img\}\}\_\{v\_\{i\}\},\\mathbf\{h\}^\{\\text\{txt\}\}\_\{v\_\{i\}\}\)/\\tau\\right\)\}\{\\sum\_\{v\_\{j\}\\in\\mathcal\{B\}\}\\exp\\\!\\left\(\\mathrm\{sim\}\(\\mathbf\{h\}^\{\\text\{img\}\}\_\{v\_\{i\}\},\\mathbf\{h\}^\{\\text\{txt\}\}\_\{v\_\{j\}\}\)/\\tau\\right\)\},wheresim​\(⋅,⋅\)\\mathrm\{sim\}\(\\cdot,\\cdot\)denotes cosine similarity andτ\\tauis a temperature hyperparameter controlling the sharpness\. Here, all non\-matching teacher representations within the same mini\-batch are treated as negatives\. In addition to cross\-modal alignment, we apply the same structural contrastive objective to the image view\. Specifically, we use the student representations𝐡img\\mathbf\{h\}^\{\\text\{img\}\}and the corresponding subgraph summaries𝐬img\\mathbf\{s\}^\{\\text\{img\}\}to define the image\-side structural lossℒstructimg\\mathcal\{L\}\_\{\\text\{struct\}\}^\{\\text\{img\}\}, which has the same definition as the structural contrastive loss in Eq\.[16](https://arxiv.org/html/2604.20283#S4.E16)and differs only in that it is computed from image\-based representations\. The overall training objective for the image\-based student GNN is:

\(18\)ℒimg=ℒstructimg\+λ1​ℒxmodal,\\mathcal\{L\}\_\{\\text\{img\}\}=\\mathcal\{L\}\_\{\\text\{struct\}\}^\{\\text\{img\}\}\+\\lambda\_\{1\}\\mathcal\{L\}\_\{\\text\{xmodal\}\},whereλ1\\lambda\_\{1\}is a weighting hyperparameter that balances the structural contrastive loss with the cross\-modal InfoNCE loss\.

After training, the text\-based and image\-based GNNs yield complementary contextualized representations, which are jointly used in the subsequent online evidence reasoning stage\.

The overall offline procedure of MSR\-MEL is summarized in Algorithm[1](https://arxiv.org/html/2604.20283#algorithm1), which consists of two phases: contextualized graph construction and teacher–student alignment\. In Part I, we first initialize the node set𝒱=ℳ∪ℰ\\mathcal\{V\}=\\mathcal\{M\}\\cup\\mathcal\{E\}and construct the initial edge setE\(0\)E^\{\(0\)\}by computing modality\-aware gated similarities for node pairs \(Lines 1–5\)\. We then generate LLM\-enhanced embeddings for mentions and selected entities \(Lines 6–7\)\. Based on these enhanced embeddings, we compute fused similarities and retrieve Top\-Kl​l​mK\_\{llm\}neighbors to construct the semantic edge setE\(1\)E^\{\(1\)\}\(Lines 8–11\)\. The final contextualized graph𝒢\\mathcal\{G\}is obtained by combiningE\(0\)E^\{\(0\)\}andE\(1\)E^\{\(1\)\}\(Lines 12–13\)\. In Part II, we perform teacher–student alignment to learn robust group\-level representations\. We first extract local subgraphs for all nodes via Personalized PageRank \(Lines 14–15\)\. In Stage I, a text\-based teacher encoder is trained with the structural contrastive loss to capture stable neighborhood semantics \(Lines 16–19\), after which the teacher is frozen \(Line 20\)\. In Stage II, an image\-based student encoder is optimized under the guidance of the frozen teacher \(Lines 21–25\)\. By minimizing a joint objective that combines image\-side structural learning with cross\-modal alignment, the student is encouraged to align noisy visual representations with the more stable semantic space provided by the teacher\.

To theoretically support the robustness of this teacher\-student alignment strategy, we present the following theorem\.

###### Theorem1\.

Let𝐡v∈ℝd\\mathbf\{h\}\_\{v\}\\in\\mathbb\{R\}^\{d\}denote the latent ideal semantic representation of nodevv, and let𝐡vtxt\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}and𝐡vimg\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}denote the teacher and student representations, respectively\. Assuming the teacher is a stable approximation to the ideal target such that‖𝐡vtxt−𝐡v∗‖2≤δt\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}\-\\mathbf\{h\}\_\{v\}^\{\*\}\\\|\_\{2\}\\leq\\delta\_\{t\}for some bounded errorδt≥0\\delta\_\{t\}\\geq 0, then the student error satisfies: \*

‖𝐡vimg−𝐡v∗‖22≤2​‖𝐡vimg−𝐡vtxt‖22\+2​δt2,\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}\-\\mathbf\{h\}\_\{v\}^\{\*\}\\\|\_\{2\}^\{2\}\\leq 2\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}\-\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}\\\|\_\{2\}^\{2\}\+2\\delta\_\{t\}^\{2\},indicating that minimizing the teacher–student discrepancy‖𝐡vimg−𝐡vtxt‖22\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}\-\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}\\\|\_\{2\}^\{2\}effectively tightens the upper bound on the student’s error relative to the ideal semantic target\.

## 5\.Multi\-Perspective Evidence Reasoning

Following the offline generation of multi\-perspective evidence, this section presents the online inference stage of MSR\-MEL\. While LLMs offer strong reasoning ability, directly applying them to end\-to\-end entity linking over a large candidate space is computationally expensive and time\-consuming\. To improve efficiency while maintaining accuracy, we adopt an online reasoning process that leverages the synthesized multi\-perspective evidence for effective disambiguation\. Specifically, the online stage consists of: \(1\) Candidate selection \(Section[5\.1](https://arxiv.org/html/2604.20283#S5.SS1)\): a retrieval\-based strategy for constructing a high\-recall candidate set; and \(2\) LLM\-Driven evidence re\-ranking \(Section[5\.2](https://arxiv.org/html/2604.20283#S5.SS2)\): which induces an interpretable decision tree from the synthesized evidence, defining a transparent reasoning path for refined ranking\.

### 5\.1\.Candidate Selection

To support effective contextual evidence reasoning in the online stage, the first step is to construct a high\-quality candidate set that maximizes recall while maintaining practical efficiency\. Rather than relying on a single similarity signal, we adopt a multi\-perspective fast retrieval strategy that aggregates complementary instance\-centric, group\-level, and lexical evidence\. Importantly, we do not include statistical evidence in this retrieval stage, which is deliberately restricted to relevance\-oriented signals; statistical evidence is instead deferred to the reasoning stage, as it primarily reflects evidence reliability and conflict patterns rather than directly rankable similarity\.

Specifically, we perform candidate generation using a hybrid 9\-way retrieval scheme\. This scheme combines similarity scores from 8 channels, comprising text\-text \(TT\), text\-image \(TV\), image\-text \(VT\), and image\-image \(VV\) combinations for both original CLIP and GNN\-enhanced embeddings, alongside one lexical retrieval channel based on the string overlap ratio\. For each of the 8 retrieval channels \(4 views×\\times2 embedding types\), we retrieve the top\-KchK\_\{\\text\{ch\}\}nearest entities using FAISS, and merge them via union to form the final candidate set\. Formally, let𝒮=\{T​T,T​V,V​T,V​V\}\\mathcal\{S\}=\\\{TT,TV,VT,VV\\\}denote the set of modality\-combination views and𝒯=\{i​n​s​t,g​r​o​u​p\}\\mathcal\{T\}=\\\{inst,group\\\}represent the embedding types\. We construct the candidate set𝒞​\(m\)\\mathcal\{C\}\(m\)by aggregating the topKc​hK\_\{ch\}nearest entities from eight channels and one lexical channel:

\(19\)𝒞​\(m\)=\(⋃t∈𝒯⋃s∈𝒮Top\-​Kch​\(sts​\(m,e\)\)\)∪Top\-​Kch​\(sl​e​x​\(m,e\)\)\\small\\mathcal\{C\}\(m\)=\\left\(\\bigcup\_\{t\\in\\mathcal\{T\}\}\\bigcup\_\{s\\in\\mathcal\{S\}\}\\text\{Top\-\}K\_\{\\text\{ch\}\}\(s\_\{t\}^\{s\}\(m,e\)\)\\right\)\\cup\\text\{Top\-\}K\_\{\\text\{ch\}\}\(s\_\{lex\}\(m,e\)\)wherests​\(m,e\)s\_\{t\}^\{s\}\(m,e\)denotes the cosine similarity for viewssand embedding typett, andsl​e​x​\(m,e\)s\_\{lex\}\(m,e\)represents the string overlap ratio\. In our implementation, we evenly allocate the retrieval budget by settingKc​h=250K\_\{ch\}=250for each of the nine channels, yielding a maximum candidate pool size of\|𝒞​\(m\)\|≤2250\|\\mathcal\{C\}\(m\)\|\\leq 2250\.

For each retrieved candidatee∈𝒞​\(m\)e\\in\\mathcal\{C\}\(m\), we compute an initial fused scoresp​r​i​o​r​\(m,e\)s\_\{prior\}\(m,e\)by aggregating the normalized similarities across all retrieval views\. This score serves as a global prior that reflects the baseline relevance of the candidate, providing a stable starting point for subsequent fine\-grained reasoning\.

### 5\.2\.LLM\-Driven Re\-ranking

While the candidate selection stage ensures a high\-recall candidate set𝒞​\(m\)\\mathcal\{C\}\(m\)by aggregating both instance\-centric, group\-level and lexical evidence, the resulting prior scoresp​r​i​o​r​\(m,e\)s\_\{prior\}\(m,e\)remains a coarse numerical aggregation and is insufficient for resolving ambiguities\. On the other hand, directly applying LLMs to reason over all candidates would be time\-consuming and computationally expensive\. To balance effectiveness and efficiency, we therefore leverage the LLM to induce a structured decision\-tree re\-ranking strategy based on the multi\-perspective evidence vector𝐟​\(m,e\)∈ℝ14\\mathbf\{f\}\(m,e\)\\in\\mathbb\{R\}^\{14\}, enabling effective re\-ranking of candidate entities through interpretable evidence\-based reasoning\.

LLM\-Guided Evidence Reasoning\.Instead of treating the LLM as a black\-box scorer, we formulate the ranking process as a conditional decision strategy induction task, where the model explicitly and systematically reasons over heterogeneous evidence to refine candidate rankings\. As specified in[Prompt Box B](https://arxiv.org/html/2604.20283#S5.SS2), we provide the LLM with:

- •Inputs: The initial prior scoresp​r​i​o​rs\_\{prior\}and the structured 14\-dimensional evidence features encompassing instance\-centric similarity, graph\-enhanced context, and modality availability\.
- •Guidelines: A set of reasoning constraints that direct the LLM to prioritize cross\-view consistency, handle missing modalities toward single\-modality dominance\.

The LLM generates an executable decision tree for candidate re\-ranking, in which each node evaluates a specific evidence type \(e\.g\.,fl​e​xf\_\{lex\},fg​r​o​u​pf\_\{group\}\), and each branch is associated with a score adjustmentΔn\\Delta\_\{n\}generated by the LLM as part of the reasoning strategy, rather than being manually predefined\.

This reasoning\-centric approach defines an interpretable reasoning path for each mention\-entity pair\. The final ranking function is formally defined as:

\(20\)πLLM​\(m,e\)=sp​r​i​o​r​\(m,e\)\+∑n∈pathΔn​\(𝐟​\(m,e\)\),\\pi\_\{\\text\{LLM\}\}\(m,e\)=s\_\{prior\}\(m,e\)\+\\sum\_\{n\\in\\text\{path\}\}\\Delta\_\{n\}\(\\mathbf\{f\}\(m,e\)\),whereΔn\\Delta\_\{n\}is the score adjustment generated by the LLM for decision nodennalong the reasoning path\.

Prompt Design B: LLM\-Guided Evidence Reason\(1\)Instruction:You are tasked with designing a ranking strategy for MEL by reasoning over structured evidence\.\(2\)Input:∙\\bulletInterpretable evidence features capturing multi\-modal instance\-centric similarity, group\-level similarity, lexical evidence, statistical consistency \(mean, max, gap\), and modality availability;∙\\bulletA prior score aggregate from the above evidence\.\(3\)Guidelines:∙\\bulletTreat the prior score as a reference, not a final decision\.∙\\bulletPrefer candidates supported by consistent evidence across multiple perspectives\.∙\\bulletBe cautious when high similarity is primarily dominated by a single modality, as such evidence may be unreliable\.∙\\bulletBe cautious when visual evidence is missing\.\(4\)Output:Generate an executable ranking function that combines the base score and evidence features to produce a final ranking score\. Output only the ranking logic\.

![Refer to caption](https://arxiv.org/html/2604.20283v1/x3.png)Figure 3\.An example of the LLM\-driven online reasoning process\. The prior scorer is misled by noisy visual similarity for candidatee1e\_\{1\}\. The LLM module corrects this through a conditional decision tree\. Nodes represent distinct evidence types \(e\.g\.,𝐟l​e​x\\mathbf\{f\}\_\{lex\},𝐟g​r​o​u​p\\mathbf\{f\}\_\{group\}\), and Q1–Q5 denote specific reasoning questions evaluating signal reliability\. This logic promotes ground\-truthe2e\_\{2\}\.As illustrated in the representative case in Figure[3](https://arxiv.org/html/2604.20283#S5.F3), the initial retrieval stage misranks the incorrect entitye1e\_\{1\}at the top due to deceptive visual instance similarity \(sp​r​i​o​r=0\.498s\_\{prior\}=0\.498\), while the ground\-truthe2e\_\{2\}is ranked 5th \(sp​r​i​o​r=0\.438s\_\{prior\}=0\.438\)\. Our module corrects this by executing a structured path from Q1 to Q5:

- •Logical Verification \(Q1\-Q2\): The module first identifies a lack of strong lexical evidence \(fl​e​x<0\.8f\_\{lex\}<0\.8\) and confirms the presence of visual data fore1e\_\{1\}\(fs​t​a​t=1f\_\{stat\}=1\), initializing a skeptical evaluation of the visual signal\.
- •Conflict Resolution \(Q3\-Q4\): It detects that whilee1e\_\{1\}exhibits high instance\-level visual similarity \(fi​n​s​tv​v\>0\.45f\_\{inst\}^\{vv\}\>0\.45\), it lacks supporting group\-level contextual consistency \(fg​r​o​u​pt​t<0\.65f\_\{group\}^\{tt\}<0\.65\)\.
- •Core Refinement: By sequentially penalizing this noisy visual dominance \(Δ<0\\Delta<0\) and rewardinge2e\_\{2\}’s consistent contextual alignment across evidence cues, the system progressively demotese1e\_\{1\}\(Final Score: 0\.38\) and promotes the ground\-truthe2e\_\{2\}to the top rank \(Final Score: 0\.588\)\.

This mechanism ensures that MSR\-MEL selectively attends to informative signals while down\-weighting unreliable ones, yielding a final ranking that robustly reflects both evidence strength and cross\-evidence consistency\.

The online inference process of MSR\-MEL is summarized in Algorithm[2](https://arxiv.org/html/2604.20283#algorithm2), which follows candidate selection, prior scoring, and LLM\-driven re\-ranking\. In Step 1, the framework constructs a high\-recall candidate set𝒞​\(m\)\\mathcal\{C\}\(m\)through a 9\-way candidate selection scheme\. Specifically, it aggregates the Top\-Kc​hK\_\{ch\}entities from eight neural similarity channels, corresponding to four modality\-combination views over both instance\-centric and group\-level embeddings, together with one lexical channel based onsl​e​xs\_\{lex\}\(Lines 1–6\)\. In Step 2, an initial prior scoresp​r​i​o​r​\(m,e\)s\_\{prior\}\(m,e\)is computed for each candidate from the normalized retrieval scores, providing a stable baseline for global relevance \(Lines 7–8\)\. In Step 3, the framework performs LLM\-driven re\-ranking over the retrieved candidates\. For each candidate, the LLM identifies a decision path in the induced decision tree and refines the prior score by accumulating score adjustmentsΔn\\Delta\_\{n\}based on the multi\-perspective evidence vector𝐟​\(m,e\)\\mathbf\{f\}\(m,e\)\(Lines 9–11\)\. This yields the final scoresf​i​n​a​l​\(m,e\)s\_\{final\}\(m,e\), which enables the model to account for evidence reliability and cross\-evidence consistency in an interpretable manner\. The candidates are then ranked bysf​i​n​a​l​\(m,e\)s\_\{final\}\(m,e\)and the entity with the highest score is returned as the final prediction \(Lines 12–13\)\.

1

Input:Query mention

mm, entity set

ℰ\\mathcal\{E\}, instance\-centric and group\-level embeddings, lexical score

sl​e​xs\_\{lex\}, LLM\-induced decision tree

πLLM\\pi\_\{\\mathrm\{LLM\}\}
Output:Predicted entity

e^\\hat\{e\}
2

//Step 1: Candidate Selection

3Initialize candidate set

𝒞​\(m\)←∅\\mathcal\{C\}\(m\)\\leftarrow\\emptyset
4foreach*t∈\{i​n​s​t,g​r​o​u​p\}t\\in\\\{inst,group\\\}*do

5foreach*s∈\{T​T,T​V,V​T,V​V\}s\\in\\\{TT,TV,VT,VV\\\}*do

6Retrieve Top\-

KchK\_\{\\mathrm\{ch\}\}entities using

sts​\(m,e\)s\_\{t\}^\{s\}\(m,e\)
7

𝒞​\(m\)←𝒞​\(m\)∪Top​\-​Kch​\(sts\)\\mathcal\{C\}\(m\)\\leftarrow\\mathcal\{C\}\(m\)\\cup\\mathrm\{Top\}\\text\{\-\}K\_\{\\mathrm\{ch\}\}\(s\_\{t\}^\{s\}\)
8

9

10

𝒞​\(m\)←𝒞​\(m\)∪Top​\-​Kch​\(sl​e​x\)\\mathcal\{C\}\(m\)\\leftarrow\\mathcal\{C\}\(m\)\\cup\\mathrm\{Top\}\\text\{\-\}K\_\{\\mathrm\{ch\}\}\(s\_\{lex\}\)
11

//Step 2: Prior Scoring

12foreach*e∈𝒞​\(m\)e\\in\\mathcal\{C\}\(m\)*do

13Compute

sprior​\(m,e\)s\_\{\\mathrm\{prior\}\}\(m,e\)from normalized retrieval scores

14

15

//Step 3: LLM\-Driven Re\-ranking

16foreach*e∈𝒞​\(m\)e\\in\\mathcal\{C\}\(m\)*do

17Identify the decision path in

πLLM\\pi\_\{\\mathrm\{LLM\}\}
18Compute

sfinal​\(m,e\)=sprior​\(m,e\)\+∑n∈pathΔn​\(𝐟​\(m,e\)\)s\_\{\\mathrm\{final\}\}\(m,e\)=s\_\{\\mathrm\{prior\}\}\(m,e\)\+\\sum\_\{n\\in\\mathrm\{path\}\}\\Delta\_\{n\}\(\\mathbf\{f\}\(m,e\)\)
19

20

21Rank candidates in

𝒞​\(m\)\\mathcal\{C\}\(m\)by

sfinal​\(m,e\)s\_\{\\mathrm\{final\}\}\(m,e\)
22return*e^=arg⁡maxe∈𝒞​\(m\)⁡sfinal​\(m,e\)\\hat\{e\}=\\arg\\max\_\{e\\in\\mathcal\{C\}\(m\)\}s\_\{\\mathrm\{final\}\}\(m,e\)*

Algorithm 2Online Evidence Reasoning

## 6\.Complexity Analysis and Proofs

### 6\.1\.Complexity Analysis

The computational complexity of MSR\-MEL consists of an offline stage and an online stage\. In the offline stage, letN=\|V\|N=\|V\|be the number of nodes,ddthe hidden dimension,LLthe number of GNN layers, and\|Egraph\|\|E\_\{\\mathrm\{graph\}\}\|the number of graph edges\. The cost of multimodal node encoding isO​\(N​Tenc\)O\(NT\_\{\\mathrm\{enc\}\}\)\. The cost of contextualized graph construction can be written asO​\(N​Tenc\+Tgraph\)O\(NT\_\{\\mathrm\{enc\}\}\+T\_\{\\mathrm\{graph\}\}\), whereTgraphT\_\{\\mathrm\{graph\}\}denotes the cost of candidate edge retrieval and graph assembly, which is subquadratic in practice under efficient nearest\-neighbor indexing\. Given the constructed graph, one epoch of teacher/student GNN training costsO​\(L​\|Egraph\|​d\)O\(L\|E\_\{\\mathrm\{graph\}\}\|d\), and thus the total offline complexity isO​\(N​Tenc\+Tgraph\+E​Tgnn\)=O​\(N​Tenc\+Tgraph\+E​L​\|Egraph\|​d\),O\(NT\_\{\\mathrm\{enc\}\}\+T\_\{\\mathrm\{graph\}\}\+ET\_\{\\mathrm\{gnn\}\}\)=O\(NT\_\{\\mathrm\{enc\}\}\+T\_\{\\mathrm\{graph\}\}\+EL\|E\_\{\\mathrm\{graph\}\}\|d\),whereEEis the number of training epochs\. In the online stage, for each query mentionmm, multi\-perspective candidate selection performs 9 retrieval channels, givingO​\(9​Tret\)\.O\(9T\_\{\\mathrm\{ret\}\}\)\.Let\|𝒞​\(m\)\|\|\\mathcal\{C\}\(m\)\|denote the retrieved candidate size\. The subsequent LLM\-based re\-ranking is applied only to𝒞​\(m\)\\mathcal\{C\}\(m\), with complexityO​\(\|𝒞​\(m\)\|​Tllm\)\.O\(\|\\mathcal\{C\}\(m\)\|T\_\{\\mathrm\{llm\}\}\)\.Therefore, the overall online complexity per query isO​\(9​Tret\+\|𝒞​\(m\)\|​Tllm\)\.O\(9T\_\{\\mathrm\{ret\}\}\+\|\\mathcal\{C\}\(m\)\|T\_\{\\mathrm\{llm\}\}\)\.Overall, the offline cost is incurred once and can be amortized over all test queries, while the online cost scales with the retrieval cost and the size of the candidate set rather than the full entity space\.

### 6\.2\.Theoretical Proofs

In this section, we provide the proofs of Theorem[1](https://arxiv.org/html/2604.20283#S3.Thmtheorem1)and Theorem[1](https://arxiv.org/html/2604.20283#S4.Thmtheorem1), which justify the effectiveness of multi\-perspective evidence synthesis and asymmetric teacher–student alignment in MSR\-MEL and provide theoretical support for our core design\.

Proof of Theorem[1](https://arxiv.org/html/2604.20283#S3.Thmtheorem1)\.Lety​\(m,e\)∈ℝy\(m,e\)\\in\\mathbb\{R\}denote the latent compatibility score between a mentionmmand a candidate entityee\. We assume each evidence channelk∈\{1,…,K\}k\\in\\\{1,\\dots,K\\\}provides a noisy estimatorzk​\(m,e\)=y​\(m,e\)\+εk​\(m,e\)z\_\{k\}\(m,e\)=y\(m,e\)\+\\varepsilon\_\{k\}\(m,e\), whereεk\\varepsilon\_\{k\}represents zero\-mean observation noise \(𝔼​\[εk\]=0\\mathbb\{E\}\[\\varepsilon\_\{k\}\]=0\) with a covariance matrixΣ∈ℝK×K\\Sigma\\in\\mathbb\{R\}^\{K\\times K\}for the error vector𝜺=\[ε1,…,εK\]⊤\\bm\{\\varepsilon\}=\[\\varepsilon\_\{1\},\\dots,\\varepsilon\_\{K\}\]^\{\\top\}\. The multi\-perspective modelθm​p\\theta^\{mp\}constructs a fused estimatory^=∑k=1Kwk​zk\\hat\{y\}=\\sum\_\{k=1\}^\{K\}w\_\{k\}z\_\{k\}with weightswk≥0w\_\{k\}\\geq 0and∑wk=1\\sum w\_\{k\}=1, yielding an estimation riskℛ​\(θm​p\)=𝔼​\[\(y^−y\)2\]=𝔼​\[\(∑wk​εk\)2\]=𝐰⊤​Σ​𝐰\\mathcal\{R\}\(\\theta^\{mp\}\)=\\mathbb\{E\}\[\(\\hat\{y\}\-y\)^\{2\}\]=\\mathbb\{E\}\[\(\\sum w\_\{k\}\\varepsilon\_\{k\}\)^\{2\}\]=\\mathbf\{w\}^\{\\top\}\\Sigma\\mathbf\{w\}\. To find the minimum\-risk fusion, we solve the constrained optimization problemmin𝐰⁡𝐰⊤​Σ​𝐰\\min\_\{\\mathbf\{w\}\}\\mathbf\{w\}^\{\\top\}\\Sigma\\mathbf\{w\}subject to𝟏⊤​𝐰=1\\mathbf\{1\}^\{\\top\}\\mathbf\{w\}=1\. Using the Lagrange multiplier method, the optimal weights are found to be𝐰∗=Σ−1​𝟏/\(𝟏⊤​Σ−1​𝟏\)\\mathbf\{w\}^\{\*\}=\\Sigma^\{\-1\}\\mathbf\{1\}/\(\\mathbf\{1\}^\{\\top\}\\Sigma^\{\-1\}\\mathbf\{1\}\), which results in the minimum risk:

ℛ​\(θm​p\)=1𝟏⊤​Σ−1​𝟏\.\\mathcal\{R\}\(\\theta^\{mp\}\)=\\frac\{1\}\{\\mathbf\{1\}^\{\\top\}\\Sigma^\{\-1\}\\mathbf\{1\}\}\.Conversely, a single\-perspective modelθs​p\\theta^\{sp\}relying on an isolated channelkkcorresponds to a weight vector𝐰=𝐞k\\mathbf\{w\}=\\mathbf\{e\}\_\{k\}, resulting in a risk ofℛ​\(θs​p\)=𝐞k⊤​Σ​𝐞k=Σk​k\\mathcal\{R\}\(\\theta^\{sp\}\)=\\mathbf\{e\}\_\{k\}^\{\\top\}\\Sigma\\mathbf\{e\}\_\{k\}=\\Sigma\_\{kk\}\. Since𝐰∗\\mathbf\{w\}^\{\*\}is the global minimizer for the risk function over the probability simplex, it follows thatℛ​\(θm​p\)≤ℛ​\(θs​p\)\\mathcal\{R\}\(\\theta^\{mp\}\)\\leq\\mathcal\{R\}\(\\theta^\{sp\}\)for allk∈\{1,…,K\}k\\in\\\{1,\\dots,K\\\}\. Furthermore, if at least two evidence channels are non\-redundant, meaning that the noise terms are not perfectly correlated, the inequality becomes strict, thereby proving the theorem\.

Proof of Theorem[1](https://arxiv.org/html/2604.20283#S4.Thmtheorem1)\.To analyze the robustness of the student representation𝐡vimg\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}, we decompose its total error relative to the ideal semantic target𝐡v∗\\mathbf\{h\}\_\{v\}^\{\*\}into the alignment error and the teacher’s intrinsic approximation error\. By the triangle inequality in the representation space, we have‖𝐡vimg−𝐡v∗‖2≤‖𝐡vimg−𝐡vtxt‖2\+‖𝐡vtxt−𝐡v∗‖2\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}\-\\mathbf\{h\}\_\{v\}^\{\*\}\\\|\_\{2\}\\leq\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}\-\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}\\\|\_\{2\}\+\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}\-\\mathbf\{h\}\_\{v\}^\{\*\}\\\|\_\{2\}\. To relate this to our optimization objective, we square the bound and apply the inequality\(a\+b\)2≤2​a2\+2​b2\(a\+b\)^\{2\}\\leq 2a^\{2\}\+2b^\{2\}:

‖𝐡vimg−𝐡v∗‖22\\displaystyle\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}\-\\mathbf\{h\}\_\{v\}^\{\*\}\\\|\_\{2\}^\{2\}≤\(‖𝐡vimg−𝐡vtxt‖2\+‖𝐡vtxt−𝐡v∗‖2\)2\\displaystyle\\leq\\left\(\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}\-\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}\\\|\_\{2\}\+\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}\-\\mathbf\{h\}\_\{v\}^\{\*\}\\\|\_\{2\}\\right\)^\{2\}≤2​‖𝐡vimg−𝐡vtxt‖22\+2​‖𝐡vtxt−𝐡v∗‖22\.\\displaystyle\\leq 2\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}\-\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}\\\|\_\{2\}^\{2\}\+2\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}\-\\mathbf\{h\}\_\{v\}^\{\*\}\\\|\_\{2\}^\{2\}\.By substituting the stability assumption‖𝐡vtxt−𝐡v∗‖2≤δt\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}\-\\mathbf\{h\}\_\{v\}^\{\*\}\\\|\_\{2\}\\leq\\delta\_\{t\}into the second term of the expansion, we obtain the final error bound:

‖𝐡vimg−𝐡v∗‖22≤2​‖𝐡vimg−𝐡vtxt‖22\+2​δt2\.\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}\-\\mathbf\{h\}\_\{v\}^\{\*\}\\\|\_\{2\}^\{2\}\\leq 2\\\|\\mathbf\{h\}\_\{v\}^\{\\mathrm\{img\}\}\-\\mathbf\{h\}\_\{v\}^\{\\mathrm\{txt\}\}\\\|\_\{2\}^\{2\}\+2\\delta\_\{t\}^\{2\}\.This bound clearly shows that the error of the image\-based student is jointly controlled by two factors: the discrepancy between the student and the teacher, and the teacher’s own approximation error relative to the ideal semantic target\. Therefore, if the textual teacher provides a stable semantic reference, encouraging alignment between the image student and the teacher helps effectively constrain the student’s error and thereby improves the robustness of visual representation learning\.

## 7\.Experiments

### 7\.1\.Experimental Setup

Table 3\.The Statistics of MEL DatasetsStatisticsWikiMEL\(Wanget al\.,[2022a](https://arxiv.org/html/2604.20283#bib.bib1)\)RichpediaMEL\(Wanget al\.,[2022a](https://arxiv.org/html/2604.20283#bib.bib1)\)WikiDiverse\(Wanget al\.,[2022b](https://arxiv.org/html/2604.20283#bib.bib3)\)\# mentions25,84617,80515,093\# images of mentions22,136 \(85\.6%\)15,852 \(89\.0%\)6,697 \(44\.4%\)\# entities109,976160,935132,460\# images of entities67,195 \(61\.1%\)86,769 \(53\.9%\)67,309 \(50\.8%\)image ratio67\.26%57\.41%50\.16%Table 4\.Main results on multimodal entity linking benchmarks\. We report Hit@1/5/10 on WikiMEL\(Wanget al\.,[2022a](https://arxiv.org/html/2604.20283#bib.bib1)\), RichpediaMEL,\(Wanget al\.,[2022a](https://arxiv.org/html/2604.20283#bib.bib1)\), and WikiDiverse\(Wanget al\.,[2022b](https://arxiv.org/html/2604.20283#bib.bib3)\), comparing our unsupervised method with representative supervised and unsupervised baselines\. Bold numbers indicate the best performance among unsupervised methods, while underlined numbers denote the second\-best unsupervised results\. Supervised results are reported as reference points\. The last three columns report the average absolute improvement of our method over each baseline on Hit@1, Hit@5, and Hit@10 across the three benchmarks, computed asOurs−\-Baseline\.CategoryMMethodsWikiMEL\(Wanget al\.,[2022a](https://arxiv.org/html/2604.20283#bib.bib1)\)RichpediaMEL\(Wanget al\.,[2022a](https://arxiv.org/html/2604.20283#bib.bib1)\)WikiDiverse\(Wanget al\.,[2022b](https://arxiv.org/html/2604.20283#bib.bib3)\)\\columncolorblue\!8ImprovementHIT@1HIT@5HIT@10HIT@1HIT@5HIT@10HIT@1HIT@5HIT@10HIT@1HIT@5HIT@10SupervisedTARNN\(Eshelet al\.,[2017](https://arxiv.org/html/2604.20283#bib.bib17)\)32\.045\.856\.631\.239\.345\.922\.450\.568\.447\.8542\.7833\.46T\+VMEL\-HI\(Zhanget al\.,[2021](https://arxiv.org/html/2604.20283#bib.bib18)\)38\.655\.165\.234\.943\.150\.645\.776\.588\.636\.6529\.7522\.29T\+VGHMFC\(Wanget al\.,[2022a](https://arxiv.org/html/2604.20283#bib.bib1)\)43\.664\.074\.438\.750\.958\.546\.077\.588\.933\.6123\.8516\.49T\+VDWE\(Songet al\.,[2024](https://arxiv.org/html/2604.20283#bib.bib5)\)44\.765\.980\.867\.697\.198\.647\.581\.392\.023\.116\.55\-0\.04T\+VM3​E​LM^\{3\}EL\(10%\)\(Huet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib19)\)76\.7290\.1993\.764\.5187\.2891\.666\.3684\.3189\.717\.180\.72\-1\.24UnsupervisedTBERT\(Devlinet al\.,[2019](https://arxiv.org/html/2604.20283#bib.bib20)\)31\.748\.857\.831\.642\.047\.622\.253\.859\.847\.8839\.7835\.36T\+VCLIP\(Radfordet al\.,[2021](https://arxiv.org/html/2604.20283#bib.bib4)\)40\.756\.064\.638\.154\.562\.434\.459\.762\.238\.6531\.2527\.36T\+VBM\(Ganet al\.,[2021](https://arxiv.org/html/2604.20283#bib.bib15)\)33\.250\.757\.545\.162\.369\.928\.848\.858\.140\.6834\.0528\.59T\+VOpenMEL\(Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21)\)69\.881\.083\.465\.677\.480\.467\.182\.285\.28\.887\.787\.43UnsupervisedT\+VMSR\-MEL \(Ours\)84\.9991\.7692\.8674\.487\.4889\.5669\.7584\.788\.86\-\-\-

Datasets\.We evaluate MSR\-MEL on three widely used multimodal entity linking benchmarks: WikiMEL, RichpediaMEL\(Wanget al\.,[2022a](https://arxiv.org/html/2604.20283#bib.bib1)\), and WikiDiverse\(Wanget al\.,[2022b](https://arxiv.org/html/2604.20283#bib.bib3)\)\. These datasets differ in domain coverage, entity types, and modality complexity, as summarized in Table[3](https://arxiv.org/html/2604.20283#S7.T3)\. WikiMEL and RichpediaMEL mainly focus on person entities collected from Wikipedia\-based sources, while WikiDiverse covers a broader range of entity types \(e\.g\., persons, organizations and locations\), posing greater challenges for generalization\.

Baselines\.We compare MSR\-MEL with representative unsupervised and supervised MEL methods\.

- •Unsupervised methodsinclude BM25\(Ganet al\.,[2021](https://arxiv.org/html/2604.20283#bib.bib15)\), BERT \(Zero\-shot\)\(Devlinet al\.,[2019](https://arxiv.org/html/2604.20283#bib.bib20)\), and CLIP \(Zero\-shot\)\(Radfordet al\.,[2021](https://arxiv.org/html/2604.20283#bib.bib4)\), which perform entity linking via lexical or representation\-based similarity ranking without task\-specific supervision, as well as the recent state\-of\-the\-art framework OpenMEL\(Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21)\)\.
- •Supervised methodsinclude ARNN\(Eshelet al\.,[2017](https://arxiv.org/html/2604.20283#bib.bib17)\), MEL\-HI\(Zhanget al\.,[2021](https://arxiv.org/html/2604.20283#bib.bib18)\), GHMFC\(Wanget al\.,[2022a](https://arxiv.org/html/2604.20283#bib.bib1)\), DWE\(Songet al\.,[2024](https://arxiv.org/html/2604.20283#bib.bib5)\), and M3EL\(Huet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib19)\), which model MEL as a discriminative ranking or classification problem using annotated mention–entity pairs during training\.

Metrics\.We evaluate performance using Hit@kk\(Recall@kk\), which measures whether the ground\-truth entity appears within the top\-kkranked candidates\. We report Hit@\{1,5,10\}\\\{1,5,10\\\}, where Hit@1 reflects top\-ranked accuracy and Hit@5/10 evaluate recall under a limited candidate pool\.

Implementation details\.Experiments are conducted on a workstation with an Intel Xeon Silver 4314 CPU, an NVIDIA RTX A5000 GPU, and 512 GB RAM\. We use ChatGPT\-4o as the default backbone for both offline and online phases\. For offline synthesis, we set the retrieval budgetKc​h=250K\_\{ch\}=250, expansion budgetKl​l​m=30K\_\{llm\}=30, subgraph sizeKp​p​r=20K\_\{ppr\}=20, and distillation weightλ2=0\.75\\lambda\_\{2\}=0\.75to balance coverage and efficiency\. Online, the LLM induces decision trees with a maximum depth of 5 to enable controlled and stable evidence reasoning\.

### 7\.2\.Experimental Results

Exp\-1: Overall Performance\.Table[4](https://arxiv.org/html/2604.20283#S7.T4)reports the overall results on WikiMEL, RichpediaMEL, and WikiDiverse\. MSR\-MEL consistently achieves the best performance among all unsupervised methods across all datasets and evaluation metrics\. This strong and consistent advantage shows that our framework generalizes well across benchmarks with different data characteristics and ambiguity levels\.

We first compare MSR\-MEL with representative unsupervised baselines, including text\-only semantic matching \(BERT\), vision–language alignment \(CLIP\), multimodal bi\-encoder matching \(BM\), and the recent strong framework OpenMEL\. While CLIP and BM improve over BERT by incorporating visual information, their performance remains limited by instance\-level similarity matching\. OpenMEL further enhances unsupervised MEL by leveraging LLM\-based contextual expansion and global coherence modeling\. Nevertheless, its graph construction and decision process are still driven by top\-kksimilarity scores at the instance level\. Notably, we surpass the previous SOTA, OpenMEL, with a relative HIT@1 improvement of 21\.7% on WikiMEL \(84\.99 vs\. 69\.8\) and 13\.4% on RichpediaMEL \(74\.4 vs\. 65\.6\)\. More broadly, as shown in the last three columns of Table[4](https://arxiv.org/html/2604.20283#S7.T4), MSR\-MEL improves over OpenMEL by an average of 8\.88, 7\.78, and 7\.43 points on Hit@1, Hit@5, and Hit@10 across the three benchmarks, respectively\. This observation demonstrates the effectiveness of group\-level representation learning and multi\-perspective evidence reasoning beyond instance\-centric similarity optimization\.

Since our primary focus is unsupervised MEL, supervised results are reported as reference points\. Despite using no labeled data, MSR\-MEL achieves competitive performance compared with several supervised baselines \(ARNN, MEL\-HI, and GHMFC\), and substantially outperforms DWE across all datasets\. Under the low\-resource setting, MSR\-MEL also consistently surpassesM3​E​LM^\{3\}ELtrained with only 10% labeled data\. These results are particularly encouraging, as they suggest that robust multi\-perspective evidence synthesis and reasoning can effectively reduce the dependence on manual annotations\. Overall, the results validate that the proposed framework provides an effective and scalable solution for unsupervised MEL\.

![Refer to caption](https://arxiv.org/html/2604.20283v1/x4.png)Figure 4\.Efficiency comparison on MEL benchmarks\. \(a\) Average training cost per epoch \(log scale\)\. \(b\) End\-to\-end inference time \(log scale\)\.Exp\-2: Efficiency Evaluation\.Figure[4](https://arxiv.org/html/2604.20283#S7.F4)compares the training and inference efficiency of MSR\-MEL on three MEL benchmarks\. For training, we report the average time per epoch in log scale\. MSR\-MEL incurs consistently low per\-epoch training cost across all datasets, and we further decompose the cost into the text and image branches to reflect its computational structure\. Both branches remain lightweight, with per\-epoch cost staying around the order of10110^\{1\}seconds, whereasM3​E​LM^\{3\}ELis higher by more than one order of magnitude on all three benchmarks\. This result indicates that the proposed offline evidence synthesis framework is computationally efficient in practice, despite involving contextualized graph construction and teacher–student alignment\. For inference, we report the end\-to\-end runtime in log scale\. WhileM3​E​LM^\{3\}ELachieves the fastest inference as a fully end\-to\-end model, its strong efficiency comes with the benefit of supervised training\. By contrast, OpenMEL exhibits substantially higher inference cost across all datasets, which is consistent with its more involved structural construction and optimization procedure\. MSR\-MEL consistently outperforms OpenMEL across all benchmarks, reducing end\-to\-end inference time by approximately one order of magnitude \(around 11–13×\\times\) while maintaining strong accuracy\. This shows that our online reasoning framework avoids the heavy inference overhead of explicit structural optimization, while still retaining the advantages of LLM\-guided evidence reasoning\.

Exp\-3: Parameter Sensitivity Analysis\.

![Refer to caption](https://arxiv.org/html/2604.20283v1/x5.png)Figure 5\.Parameter sensitivity analysis of MSR\-MEL on WikiMEL, RichpediaMEL, and WikiDiverse\. Each row corresponds to a dataset, and each column varies one hyper\-parameter while keeping the others fixed:Kl​l​mK\_\{llm\}\(left\),Kp​p​rK\_\{ppr\}\(middle\), andλ1\\lambda\_\{1\}\(right\)\.We evaluate the sensitivity of MSR\-MEL to three key offline hyper\-parameters: the LLM\-based expansion budgetKl​l​mK\_\{llm\}, the PPR subgraph sizeKp​p​rK\_\{ppr\}, and the cross\-modal distillation weightλ1\\lambda\_\{1\}\. As shown in Figure[5](https://arxiv.org/html/2604.20283#S7.F5), performance exhibits consistent unimodal trends, improving with increasing values and degrading under extreme settings\. Among them,λ1\\lambda\_\{1\}has the largest impact, underscoring the importance of cross\-modal balancing\. WikiMEL is relatively robust to parameter variations, RichpediaMEL shows moderate sensitivity, while WikiDiverse is the most sensitive\. Overall, MSR\-MEL remains robust across a broad parameter range, with more challenging datasets imposing stricter tuning requirements\.

Exp\-4: Effect of LLM\.

Table 5\.Impact of LLM variants on semantic augmentation performance \(Hit@1\)\.LLM VariantWikiMELRichpediaMELWikiDiverseMSR\-MEL \+ ChatGPT\-4o\(Achiamet al\.,[2023](https://arxiv.org/html/2604.20283#bib.bib46)\)84\.9974\.4069\.75A\. Alternative LLMs for Semantic ExpansionDeepSeek\-Chat \(Ours\)\(Liuet al\.,[2024a](https://arxiv.org/html/2604.20283#bib.bib45)\)82\.7371\.2965\.10DeepSeek\-Chat \(OpenMEL\(Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21)\)\)67\.2069\.3063\.80LLaMA3\-8B \(Ours\)\(Touvronet al\.,[2023](https://arxiv.org/html/2604.20283#bib.bib47)\)81\.7670\.3167\.30LLaMA3\-8B \(OpenMEL\(Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21)\)\)69\.8065\.6067\.10

This experiment investigates the influence of different LLMs employed for semantic augmentation during the offline stage of MSR\-MEL\. As illustrated in Table[5](https://arxiv.org/html/2604.20283#S7.T5), LLMs with more extensive world knowledge and superior contextual discernment consistently enhance the quality of generated descriptions, thereby improving retrieval and linking performance\. ChatGPT\-4o attains the optimal results, due to its ability to produce highly precise and informative semantic signals even with sparse mention context\. The performance gap is most significant on WikiDiverse, as its diverse entity types and complex contexts require much more precise descriptions to distinguish between candidates\. This suggests that high\-fidelity semantic expansion is vital for resolving ambiguities in noisy, cross\-domain multimodal environments\. Consequently, we adopt ChatGPT\-4o as the core backbone for the offline semantic enhancement module\.

Table 6\.Performance under different modality conditions \(Hit@1\)\. “All modality” denotes samples in which both nodes in the mention–entity pair have complete modalities, while “Missing modality” denotes samples for which at least one image modality is missing\.SettingMethodWikiMELRichpediaWikiDiverseFull dataset\\cellcolorgray\!15Ours\\cellcolorgray\!1584\.99\\cellcolorgray\!1574\.40\\cellcolorgray\!1569\.75OpenMEL\(Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21)\)69\.8065\.6067\.10All modality\\cellcolorgray\!15Ours\\cellcolorgray\!1586\.60\\cellcolorgray\!1576\.20\\cellcolorgray\!1572\.80OpenMEL\(Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21)\)71\.4067\.3468\.17Missing modality\\cellcolorgray\!15Ours\\cellcolorgray\!1583\.03\\cellcolorgray\!1571\.97\\cellcolorgray\!1569\.70OpenMEL\(Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21)\)68\.7065\.4366\.09

Exp\-5: Modality Robustness\.Table[6](https://arxiv.org/html/2604.20283#S7.T6)reports Hit@1 results under different modality conditions on WikiMEL, Richpedia, and WikiDiverse\. MSR\-MEL consistently outperforms OpenMEL on the full dataset, the all\-modality subset, and the missing\-modality subset across all three benchmarks\. As expected, both methods achieve better performance when complete modalities are available, since richer multimodal evidence provides stronger support for disambiguation\. Under the missing\-modality setting, the performance of both methods decreases, but MSR\-MEL still maintains a clear advantage over OpenMEL on all datasets\. These results suggest that MSR\-MEL remains robust even when modality information is incomplete, benefiting from its joint use of complementary evidence beyond direct visual matching\.

![Refer to caption](https://arxiv.org/html/2604.20283v1/x6.png)Figure 6\.Case study of MSR\-MEL on a hard example with missing visual modality\.Exp\-6: Case Study\.Figure[6](https://arxiv.org/html/2604.20283#S7.F6)shows a representative hard case where the mention contains the ambiguous surface form Oxford, while the ground\-truth entity, University of Oxford \(E1\), has a missing visual modality\. In this example, relying only on local matching is insufficient: the distractor Oxford University Press \(E2\) receives the strongest instance\-centric textual similarity and lexical overlap\. Nevertheless, MSR\-MEL still ranks E1 first\. As shown in Figure[6](https://arxiv.org/html/2604.20283#S7.F6)\(2\), the model preserves shared graph topology across the text and image views and masks missing visual modalities instead of removing the corresponding nodes, allowing E1 to remain connected to semantically related neighbors through the text view\. Compared with E2, E1 is supported by stronger group\-level evidence and receives the best overall interpretation, while E2 is identified as a misleading local match due to its strong but locally biased visual cue, as shown in Figure[6](https://arxiv.org/html/2604.20283#S7.F6)\(3\)\. This indicates that E1 is supported by more globally consistent multi\-perspective evidence, whereas E2 depends more on a few locally strong cues\. During LLM\-driven reasoning, the unreliable visual signal is down\-weighted, and the model finally chose E1 because its instance\-centric, group\-level, lexical, and statistical evidence are more coherent\. This case illustrates the advantage of MSR\-MEL for resolving ambiguous mentions under incomplete multimodal conditions\.

### 7\.3\.Ablation study

Table 7\.Ablation study of offline evidence synthesis and online evidence reasoning in MSR\-MEL \(Hit@1\)\. All variants are evaluated under the full inference pipeline unless otherwise specified\.VariantWikiMELRichpediaWikiDiverseMSR\-MEL \(Full Model\)84\.9974\.4069\.75A\. Offline Evidence Synthesisw/o LLM\-enhanced Edges81\.1971\.5566\.71w/o Intra\-type Edges \(M–M, E–E\)82\.3672\.9167\.30w/o Teacher–Student Alignment80\.0870\.7165\.24Text\-only Group Evidence \(w/o Student\)83\.2973\.8868\.23Image\-only Group Evidence \(w/o Teacher\)76\.9866\.2562\.30B\. Online Evidence Reasoningw/o CLIP Features \(𝐟inst\\mathbf\{f\}\_\{\\text\{inst\}\}\)53\.8749\.3046\.12w/o GNN Features \(𝐟group\\mathbf\{f\}\_\{\\text\{group\}\}\)74\.5065\.3263\.44w/o String Features \(𝐟lex\\mathbf\{f\}\_\{\\text\{lex\}\}\)78\.9672\.8362\.90w/o Distributional Statistics \(𝐟stat\\mathbf\{f\}\_\{\\text\{stat\}\}:μ,smax,Δ\\mu,s\_\{\\max\},\\Delta\)83\.2972\.9068\.71w/o Modality Availability Evidence \(𝐟stat\\mathbf\{f\}\_\{\\text\{stat\}\}:𝕀\\mathbb\{I\}\)82\.7873\.1266\.80w/o Prior Scoring \(spriors\_\{\\text\{prior\}\}\)76\.6670\.3463\.90w/o LLM\-driven Evidence Reasoning \(πLLM\\pi\_\{\\text\{LLM\}\}\)81\.4372\.3466\.37

Exp\-7: The Effect of Offline Evidence Synthesis\.We analyze the contributions of key design choices in the offline stage of MSR\-MEL, including contextualized graph construction and representation learning objectives\. As shown in Table[7](https://arxiv.org/html/2604.20283#S7.T7), removing LLM\-enhanced edges or intra\-type edges \(M–M, E–E\) consistently degrades performance across all datasets, indicating that both LLM\-based semantic expansion and higher\-order neighborhood connectivity are crucial for building a high\-fidelity contextualized graph\. On the learning side, disabling teacher–student alignment leads to a notable performance drop, highlighting the importance of cross\-modal knowledge distillation for stabilizing visual representations\. While a text\-only GNN achieves relatively strong results, it still underperforms the full model, and an image\-only GNN performs the worst, reflecting the noise and incompleteness of visual signals\. Overall, these results demonstrate that effective offline MEL requires both robust contextualized graph construction and joint optimization of structural and cross\-modal learning objectives\.

Exp\-8: The Effect of Online Evidence Reasoning\.We evaluate the online inference design in Algorithm[2](https://arxiv.org/html/2604.20283#algorithm2)by ablating evidence sources and reasoning modules\. As shown in Table[7](https://arxiv.org/html/2604.20283#S7.T7), removing CLIP feature, GNN features, or string features causes large performance drops, indicating that both graph\-based contextual signals and symbolic anchors are important for constructing reliable multi\-view evidence\. Consistency features further improve performance by capturing cross\-view agreement, which helps filter candidates that are only supported by a single modality\. On the inference side, removing the base scorer or the LLM\-guided ranking policy also leads to clear degradation, demonstrating that lightweight coarse scoring and LLM\-based evidence reasoning play complementary roles in the fine\-grained re\-ranking stage\. Overall, these results validate the effectiveness of our coarse\-to\-fine inference framework that combines multi\-view candidate generation with contextual evidence reasoning\.

## 8\.Related Work

### 8\.1\.Entity Linking

Entity linking \(EL\) aims to map entity mentions in text to corresponding entries in a KB\(Shenet al\.,[2014](https://arxiv.org/html/2604.20283#bib.bib8); Raoet al\.,[2012](https://arxiv.org/html/2604.20283#bib.bib27)\), where a key challenge is resolving ambiguous mentions whose interpretation depends on context\(Hasibiet al\.,[2016](https://arxiv.org/html/2604.20283#bib.bib28)\)\. Early EL methods relied on lexical and statistical features, such as tf–idf\(Aizawa,[2003](https://arxiv.org/html/2604.20283#bib.bib29)\)and word embeddings\(Mikolovet al\.,[2013](https://arxiv.org/html/2604.20283#bib.bib30)\), and performed linking by ranking candidate entities according to similarity scores\(Ganea and Hofmann,[2017](https://arxiv.org/html/2604.20283#bib.bib31); Gillicket al\.,[2019](https://arxiv.org/html/2604.20283#bib.bib32)\)\. Subsequent approaches replaced handcrafted features with neural encoders, while more recent work leverages large pre\-trained language models, including dense retrieval methods such as BLINK\(Wuet al\.,[2020](https://arxiv.org/html/2604.20283#bib.bib33)\)and generative formulations such as GENRE\(De Caoet al\.,[2020](https://arxiv.org/html/2604.20283#bib.bib34)\)\. Despite architectural differences, these methods largely follow a common paradigm: they first learn contextualized representations for mentions and entities, and then make linking decisions by directly scoring individual mention–entity pairs, remaining instance\-centric and relying on a single aggregated matching signal\.

### 8\.2\.Multimodal Entity Linking

As multimedia content becomes increasingly common on the web, entity linking has been extended to incorporate both textual and visual information for disambiguation\. Existing approaches are typically categorized into supervised and unsupervised methods\. Supervised methods model the task as a classification problem on annotated mention–entity pairs, learning confidence scores for entity assignment\. While early efforts introduced social media benchmarks like SnapCaptionsKB\(Moonet al\.,[2018](https://arxiv.org/html/2604.20283#bib.bib16)\)and quasi\-automatically constructed datasets from Twitter posts\(Adjaliet al\.,[2020](https://arxiv.org/html/2604.20283#bib.bib22)\), recent advances focus on aligning multimodal representations via contrastive learning \(e\.g\., M3EL\(Huet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib19)\)\) or LLM\-driven semantic enrichment, such as UniMEL\(Liuet al\.,[2024b](https://arxiv.org/html/2604.20283#bib.bib9)\), KGMEL\(Kimet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib10)\), I2CR\(Liuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib6)\), and the multi\-agent DeepMEL\(Wanget al\.,[2026](https://arxiv.org/html/2604.20283#bib.bib26)\)\. Conversely, unsupervised methods alleviate the dependence on manual annotation by leveraging pre\-trained models \(e\.g\., CLIP\(Radfordet al\.,[2021](https://arxiv.org/html/2604.20283#bib.bib4)\)or BERT\(Devlinet al\.,[2019](https://arxiv.org/html/2604.20283#bib.bib20)\)\) to generate representations and performing linking via similarity\-based ranking\. Recent frameworks like OpenMEL\(Zhuet al\.,[2025](https://arxiv.org/html/2604.20283#bib.bib21)\)incorporate structural context via tree formulations but remain fundamentally driven by similarity\-based, instance\-level matching\.

### 8\.3\.Graph Neural Network

Graph neural networks \(GNNs\) have become a standard paradigm for learning contextualized node representations by propagating and aggregating information over local graph neighborhoods\(Wanget al\.,[2024b](https://arxiv.org/html/2604.20283#bib.bib59)\)\. Compared with independent node encoders, GNNs explicitly model relational structure and are therefore particularly suitable for scenarios where neighborhood context provides important complementary signals\. Representative architectures include Graph Convolutional Networks \(GCN\)\(Kipf and Welling,[2016](https://arxiv.org/html/2604.20283#bib.bib54)\), GraphSAGE\(Hamiltonet al\.,[2017](https://arxiv.org/html/2604.20283#bib.bib55)\), and Graph Attention Networks \(GAT\)\(Veličkovićet al\.,[2017](https://arxiv.org/html/2604.20283#bib.bib56)\)\. GCN performs neighborhood aggregation through normalized graph convolution and is widely adopted for its simplicity and efficiency\. GraphSAGE extends this idea with inductive neighborhood sampling and aggregation, making it more scalable to large graphs\. GAT further introduces attention\-based message passing, enabling the model to assign different importance weights to different neighbors\. Beyond these representative models, GNNs have been widely used in graph\-based reasoning, where relational dependencies are important\. In MEL, GNNs are particularly suitable because they enable contextual representation learning over mention–entity graphs\. This makes them effective for handling sparse, noisy, or incomplete multimodal evidence\. Among existing variants, GCN remains a widely adopted backbone due to its simplicity and computational efficiency\.

## 9\.Conclusion

In this paper, we propose MSR\-MEL, a multi\-perspective evidence synthesis and reasoning framework with LLM for unsupervised multimodal entity linking\. MSR\-MEL explicitly constructs instance\-centric, group\-level, lexical, and statistical evidence through an offline synthesis stage\. At inference time, it performs reasoning over heterogeneous evidence via an LLM\-driven, tree\-structured re\-ranking strategy\. In particular, group\-level evidence is synthesized through LLM\-enhanced contextualized graphs and asymmetric teacher–student graph alignment\. This design enables robust aggregation of neighborhood information under sparse and incomplete multimodal data\. Extensive experiments on widely used MEL benchmarks show that MSR\-MEL consistently outperforms state\-of\-the\-art unsupervised methods, demonstrating the effectiveness of explicit multi\-perspective evidence modeling and reasoning for MEL\.

## References

- J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.\(2023\)Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[Table 5](https://arxiv.org/html/2604.20283#S7.T5.4.1.2.1.1)\.
- O\. Adjali, R\. Besançon, O\. Ferret, H\. Le Borgne, and B\. Grau \(2020\)Building a multimodal entity linking dataset from tweets\.InProceedings of the Twelfth Language Resources and Evaluation Conference,pp\. 4285–4292\.Cited by:[§8\.2](https://arxiv.org/html/2604.20283#S8.SS2.p1.1)\.
- A\. Aizawa \(2003\)An information\-theoretic perspective of tf–idf measures\.Information Processing & Management39\(1\),pp\. 45–65\.Cited by:[§8\.1](https://arxiv.org/html/2604.20283#S8.SS1.p1.1)\.
- Z\. Chen, Y\. Zhang, Y\. Fang, Y\. Geng, L\. Guo, X\. Chen, Q\. Li, W\. Zhang, J\. Chen, Y\. Zhu,et al\.\(2024\)Knowledge graphs meet multi\-modal learning: a comprehensive survey\.arXiv preprint arXiv:2402\.05391\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p1.1)\.
- S\. Cucerzan \(2007\)Large\-scale named entity disambiguation based on wikipedia data\.InProceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning \(EMNLP\-CoNLL\),pp\. 708–716\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p1.1)\.
- N\. De Cao, W\. Aziz, and I\. Titov \(2019\)Question answering by reasoning across documents with graph convolutional networks\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human language technologies, Volume 1 \(long and short papers\),pp\. 2306–2317\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p1.1)\.
- N\. De Cao, G\. Izacard, S\. Riedel, and F\. Petroni \(2020\)Autoregressive entity retrieval\.arXiv preprint arXiv:2010\.00904\.Cited by:[§8\.1](https://arxiv.org/html/2604.20283#S8.SS1.p1.1)\.
- F\. P\. de Lange, O\. Jensen, and S\. Dehaene \(2010\)Accumulation of evidence during sequential decision making: the importance of top–down factors\.Journal of Neuroscience30\(2\),pp\. 731–738\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p5.1)\.
- A\. Delpeuch \(2019\)Opentapioca: lightweight entity linking for wikidata\.arXiv preprint arXiv:1904\.09131\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p1.1)\.
- J\. Devlin, M\. Chang, K\. Lee, and K\. Toutanova \(2019\)Bert: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 \(long and short papers\),pp\. 4171–4186\.Cited by:[Table 1](https://arxiv.org/html/2604.20283#S1.T1.13.5.12.1),[§1](https://arxiv.org/html/2604.20283#S1.p3.1),[1st item](https://arxiv.org/html/2604.20283#S7.I1.i1.p1.1),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.8.3),[§8\.2](https://arxiv.org/html/2604.20283#S8.SS2.p1.1)\.
- M\. Dubey, D\. Banerjee, D\. Chaudhuri, and J\. Lehmann \(2018\)EARL: joint entity and relation linking for question answering over knowledge graphs\.InInternational Semantic Web Conference,pp\. 108–126\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p1.1)\.
- Y\. Eshel, N\. Cohen, K\. Radinsky, S\. Markovitch, I\. Yamada, and O\. Levy \(2017\)Named entity disambiguation for noisy text\.arXiv preprint arXiv:1706\.09147\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p2.1),[2nd item](https://arxiv.org/html/2604.20283#S7.I1.i2.p1.1),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.4.3)\.
- Z\. Fang, Y\. Cao, Q\. Li, D\. Zhang, Z\. Zhang, and Y\. Liu \(2019\)Joint entity linking with deep reinforcement learning\.InThe world wide web conference,pp\. 438–447\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p1.1)\.
- J\. Gan, J\. Luo, H\. Wang, S\. Wang, W\. He, and Q\. Huang \(2021\)Multimodal entity linking: a new dataset and a baseline\.InProceedings of the 29th ACM international conference on multimedia,pp\. 993–1001\.Cited by:[1st item](https://arxiv.org/html/2604.20283#S7.I1.i1.p1.1),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.10.2)\.
- O\. Ganea and T\. Hofmann \(2017\)Deep joint entity disambiguation with local neural attention\.arXiv preprint arXiv:1704\.04920\.Cited by:[§8\.1](https://arxiv.org/html/2604.20283#S8.SS1.p1.1)\.
- J\. Gasteiger, A\. Bojchevski, and S\. Günnemann \(2018\)Predict then propagate: graph neural networks meet personalized pagerank\.arXiv preprint arXiv:1810\.05997\.Cited by:[§4\.3](https://arxiv.org/html/2604.20283#S4.SS3.p2.7)\.
- S\. Geng, S\. Liu, Z\. Fu, Y\. Ge, and Y\. Zhang \(2022\)Recommendation as language processing \(rlp\): a unified pretrain, personalized prompt & predict paradigm \(p5\)\.InProceedings of the 16th ACM conference on recommender systems,pp\. 299–315\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p1.1)\.
- D\. Gillick, S\. Kulkarni, L\. Lansing, A\. Presta, J\. Baldridge, E\. Ie, and D\. Garcia\-Olano \(2019\)Learning dense representations for entity retrieval\.InProceedings of the 23rd conference on computational natural language learning \(CoNLL\),pp\. 528–537\.Cited by:[§8\.1](https://arxiv.org/html/2604.20283#S8.SS1.p1.1)\.
- W\. Hamilton, Z\. Ying, and J\. Leskovec \(2017\)Inductive representation learning on large graphs\.Advances in neural information processing systems30\.Cited by:[§8\.3](https://arxiv.org/html/2604.20283#S8.SS3.p1.1)\.
- F\. Hasibi, K\. Balog, and S\. E\. Bratsberg \(2016\)Exploiting entity linking in queries for entity retrieval\.InProceedings of the 2016 acm international conference on the theory of information retrieval,pp\. 209–218\.Cited by:[§8\.1](https://arxiv.org/html/2604.20283#S8.SS1.p1.1)\.
- Z\. Hu, V\. Gutiérrez\-Basulto, R\. Li, and J\. Z\. Pan \(2025\)Multi\-level matching network for multimodal entity linking\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 1,pp\. 508–519\.Cited by:[Table 1](https://arxiv.org/html/2604.20283#S1.T1.13.5.5.1),[§1](https://arxiv.org/html/2604.20283#S1.p2.1),[2nd item](https://arxiv.org/html/2604.20283#S7.I1.i2.p1.1),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.1.1),[§8\.2](https://arxiv.org/html/2604.20283#S8.SS2.p1.1)\.
- Y\. Jiao, Y\. Xiong, J\. Zhang, Y\. Zhang, T\. Zhang, and Y\. Zhu \(2020\)Sub\-graph contrast for scalable self\-supervised graph representation learning\.In2020 IEEE international conference on data mining \(ICDM\),pp\. 222–231\.Cited by:[§4\.3](https://arxiv.org/html/2604.20283#S4.SS3.p3.8),[§4\.3](https://arxiv.org/html/2604.20283#S4.SS3.p6.7)\.
- J\. Kim, G\. Lee, T\. Kim, and K\. Shin \(2025\)KGMEL: knowledge graph\-enhanced multimodal entity linking\.InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 3015–3019\.Cited by:[Table 1](https://arxiv.org/html/2604.20283#S1.T1.13.5.9.1),[§1](https://arxiv.org/html/2604.20283#S1.p2.1),[§8\.2](https://arxiv.org/html/2604.20283#S8.SS2.p1.1)\.
- T\. N\. Kipf and M\. Welling \(2016\)Semi\-supervised classification with graph convolutional networks\.arXiv preprint arXiv:1609\.02907\.Cited by:[§4\.3](https://arxiv.org/html/2604.20283#S4.SS3.p3.7),[§8\.3](https://arxiv.org/html/2604.20283#S8.SS3.p1.1)\.
- S\. Li and H\. Tang \(2024\)Multimodal alignment and fusion: a survey\.arXiv preprint arXiv:2411\.17040\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p7.1)\.
- A\. Liu, B\. Feng, B\. Xue, B\. Wang, B\. Wu, C\. Lu, C\. Zhao, C\. Deng, C\. Zhang, C\. Ruan,et al\.\(2024a\)Deepseek\-v3 technical report\.arXiv preprint arXiv:2412\.19437\.Cited by:[Table 5](https://arxiv.org/html/2604.20283#S7.T5.4.1.4.1)\.
- Q\. Liu, Y\. He, T\. Xu, D\. Lian, C\. Liu, Z\. Zheng, and E\. Chen \(2024b\)Unimel: a unified framework for multimodal entity linking with large language models\.InProceedings of the 33rd ACM International Conference on Information and Knowledge Management,pp\. 1909–1919\.Cited by:[Table 1](https://arxiv.org/html/2604.20283#S1.T1.13.5.8.1),[§1](https://arxiv.org/html/2604.20283#S1.p2.1),[§8\.2](https://arxiv.org/html/2604.20283#S8.SS2.p1.1)\.
- Z\. Liu, J\. Li, K\. Li, T\. Ruan, C\. Wang, X\. He, Z\. Wang, X\. Cao, and J\. Liu \(2025\)I2CR: intra\-and inter\-modal collaborative reflections for multimodal entity linking\.InProceedings of the 33rd ACM International Conference on Multimedia,pp\. 4942–4951\.Cited by:[Table 1](https://arxiv.org/html/2604.20283#S1.T1.13.5.10.1),[§8\.2](https://arxiv.org/html/2604.20283#S8.SS2.p1.1)\.
- S\. Longpre, K\. Perisetla, A\. Chen, N\. Ramesh, C\. DuBois, and S\. Singh \(2021\)Entity\-based knowledge conflicts in question answering\.arXiv preprint arXiv:2109\.05052\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p1.1)\.
- P\. Luo, T\. Xu, C\. Liu, S\. Zhang, L\. Xu, M\. Li, and E\. Chen \(2024\)Bridging gaps in content and knowledge for multimodal entity linking\.InProceedings of the 32nd ACM International Conference on Multimedia,pp\. 9311–9320\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p4.1)\.
- P\. Luo, T\. Xu, S\. Wu, C\. Zhu, L\. Xu, and E\. Chen \(2023\)Multi\-grained multimodal interaction network for entity linking\.InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 1583–1594\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p2.1),[§1](https://arxiv.org/html/2604.20283#S1.p3.1)\.
- T\. Mikolov, K\. Chen, G\. Corrado, and J\. Dean \(2013\)Efficient estimation of word representations in vector space\.arXiv preprint arXiv:1301\.3781\.Cited by:[§8\.1](https://arxiv.org/html/2604.20283#S8.SS1.p1.1)\.
- S\. Moon, L\. Neves, and V\. Carvalho \(2018\)Multimodal named entity disambiguation for noisy social media posts\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 2000–2008\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p1.1),[§1](https://arxiv.org/html/2604.20283#S1.p2.1),[§8\.2](https://arxiv.org/html/2604.20283#S8.SS2.p1.1)\.
- A\. v\. d\. Oord, Y\. Li, and O\. Vinyals \(2018\)Representation learning with contrastive predictive coding\.arXiv preprint arXiv:1807\.03748\.Cited by:[§4\.3](https://arxiv.org/html/2604.20283#S4.SS3.p9.8)\.
- A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark,et al\.\(2021\)Learning transferable visual models from natural language supervision\.InInternational conference on machine learning,pp\. 8748–8763\.Cited by:[§4\.1](https://arxiv.org/html/2604.20283#S4.SS1.p2.1),[1st item](https://arxiv.org/html/2604.20283#S7.I1.i1.p1.1),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.9.2),[§8\.2](https://arxiv.org/html/2604.20283#S8.SS2.p1.1)\.
- D\. Rao, P\. McNamee, and M\. Dredze \(2012\)Entity linking: finding extracted entities in a knowledge base\.InMulti\-source, multilingual information extraction and summarization,pp\. 93–115\.Cited by:[§8\.1](https://arxiv.org/html/2604.20283#S8.SS1.p1.1)\.
- W\. Shen, J\. Wang, and J\. Han \(2014\)Entity linking with a knowledge base: issues, techniques, and solutions\.IEEE Transactions on Knowledge and Data Engineering27\(2\),pp\. 443–460\.Cited by:[§8\.1](https://arxiv.org/html/2604.20283#S8.SS1.p1.1)\.
- S\. Shi, Z\. Xu, B\. Hu, and M\. Zhang \(2024\)Generative multimodal entity linking\.InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\),pp\. 7654–7665\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p3.1)\.
- S\. Song, S\. Zhao, C\. Wang, T\. Yan, S\. Li, X\. Mao, and M\. Wang \(2024\)A dual\-way enhanced framework from text matching point of view for multimodal entity linking\.InProceedings of the AAAI conference on artificial intelligence,Vol\.38,pp\. 19008–19016\.Cited by:[Table 1](https://arxiv.org/html/2604.20283#S1.T1.13.5.7.1),[§1](https://arxiv.org/html/2604.20283#S1.p2.1),[2nd item](https://arxiv.org/html/2604.20283#S7.I1.i2.p1.1),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.7.2)\.
- H\. Touvron, T\. Lavril, G\. Izacard, X\. Martinet, M\. Lachaux, T\. Lacroix, B\. Rozière, N\. Goyal, E\. Hambro, F\. Azhar,et al\.\(2023\)Llama: open and efficient foundation language models\.arXiv preprint arXiv:2302\.13971\.Cited by:[Table 5](https://arxiv.org/html/2604.20283#S7.T5.4.1.6.1)\.
- P\. Veličković, G\. Cucurull, A\. Casanova, A\. Romero, P\. Lio, and Y\. Bengio \(2017\)Graph attention networks\.arXiv preprint arXiv:1710\.10903\.Cited by:[§8\.3](https://arxiv.org/html/2604.20283#S8.SS3.p1.1)\.
- F\. Wang, T\. Yan, Z\. Yang, M\. Hu, J\. Zhang, Z\. Luo, and X\. Bai \(2026\)DeepMEL: a multi\-agent collaboration framework for multimodal entity linking\.Information Processing & Management63\(3\),pp\. 104507\.Cited by:[Table 1](https://arxiv.org/html/2604.20283#S1.T1.13.5.13.1),[§1](https://arxiv.org/html/2604.20283#S1.p3.1),[§8\.2](https://arxiv.org/html/2604.20283#S8.SS2.p1.1)\.
- J\. Wang, K\. Wang, X\. Lin, W\. Zhang, and Y\. Zhang \(2024a\)Efficient unsupervised community search with pre\-trained graph transformer\.arXiv preprint arXiv:2403\.18869\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p4.1)\.
- J\. Wang, K\. Wang, X\. Lin, W\. Zhang, and Y\. Zhang \(2024b\)Neural attributed community search at billion scale\.Proceedings of the ACM on Management of Data1\(4\),pp\. 1–25\.Cited by:[§8\.3](https://arxiv.org/html/2604.20283#S8.SS3.p1.1)\.
- P\. Wang, J\. Wu, and X\. Chen \(2022a\)Multimodal entity linking with gated hierarchical fusion and contrastive training\.InProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 938–948\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p2.1),[2nd item](https://arxiv.org/html/2604.20283#S7.I1.i2.p1.1),[§7\.1](https://arxiv.org/html/2604.20283#S7.SS1.p1.1),[Table 3](https://arxiv.org/html/2604.20283#S7.T3.4.1.2),[Table 3](https://arxiv.org/html/2604.20283#S7.T3.4.1.3),[Table 4](https://arxiv.org/html/2604.20283#S7.T4),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.2.1),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.2.4),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.2.5),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.6.2)\.
- X\. Wang, J\. Tian, M\. Gui, Z\. Li, R\. Wang, M\. Yan, L\. Chen, and Y\. Xiao \(2022b\)WikiDiverse: a multimodal entity linking dataset with diversified contextual topics and entity types\.arXiv preprint arXiv:2204\.06347\.Cited by:[§7\.1](https://arxiv.org/html/2604.20283#S7.SS1.p1.1),[Table 3](https://arxiv.org/html/2604.20283#S7.T3.4.1.4),[Table 4](https://arxiv.org/html/2604.20283#S7.T4),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.2.1),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.2.6)\.
- L\. Wu, F\. Petroni, M\. Josifoski, S\. Riedel, and L\. Zettlemoyer \(2020\)Scalable zero\-shot entity linking with dense entity retrieval\.InProceedings of the 2020 conference on empirical methods in natural language processing \(EMNLP\),pp\. 6397–6407\.Cited by:[§8\.1](https://arxiv.org/html/2604.20283#S8.SS1.p1.1)\.
- R\. Wu, H\. Wang, H\. Chen, and G\. Carneiro \(2024\)Deep multimodal learning with missing modality: a survey\.arXiv preprint arXiv:2409\.07825\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p4.1)\.
- S\. Xing, F\. Zhao, Z\. Wu, C\. Li, J\. Zhang, and X\. Dai \(2023\)Drin: dynamic relation interactive network for multimodal entity linking\.InProceedings of the 31st ACM International Conference on Multimedia,pp\. 3599–3608\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p4.1)\.
- W\. Xiong, M\. Yu, S\. Chang, X\. Guo, and W\. Y\. Wang \(2019\)Improving question answering over incomplete kbs with knowledge\-aware reader\.arXiv preprint arXiv:1905\.07098\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p1.1)\.
- Y\. Xu, Y\. Yin, J\. Wang, J\. Wei, J\. Liu, L\. Yao, and W\. Zhang \(2021\)Unsupervised cross\-view feature selection on incomplete data\.Knowledge\-Based Systems234,pp\. 107595\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p7.1)\.
- Y\. Xu, W\. Zhang, Y\. Zhang, X\. Lin, and X\. Xu \(2026\)Unlocking multi\-modal potentials for link prediction on dynamic text\-attributed graphs\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 27386–27394\.Cited by:[§4\.2](https://arxiv.org/html/2604.20283#S4.SS2.p6.1)\.
- L\. Zhang, Z\. Li, and Q\. Yang \(2021\)Attention\-based multimodal entity linking with high\-quality images\.InInternational conference on database systems for advanced applications,pp\. 533–548\.Cited by:[2nd item](https://arxiv.org/html/2604.20283#S7.I1.i2.p1.1),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.5.2)\.
- Q\. Zhang, Y\. Wei, Z\. Han, H\. Fu, X\. Peng, C\. Deng, Q\. Hu, C\. Xu, J\. Wen, D\. Hu,et al\.\(2024\)Multimodal fusion on low\-quality data: a comprehensive survey\.arXiv preprint arXiv:2404\.18947\.Cited by:[§1](https://arxiv.org/html/2604.20283#S1.p4.1)\.
- M\. Zhou, J\. Wang, X\. Zhang, D\. Campbell, K\. Wang, L\. Yuan, W\. Zhang, and X\. Lin \(2025\)ProbDiffFlow: an efficient learning\-free framework for probabilistic single\-image optical flow estimation\.arXiv preprint arXiv:2503\.12348\.Cited by:[§4\.1](https://arxiv.org/html/2604.20283#S4.SS1.p5.4)\.
- X\. Zhu, Y\. Zhang, and L\. Chen \(2025\)OpenMEL: unsupervised multimodal entity linking using noise\-free expanded queries and global coherence\.Proceedings of the VLDB Endowment18\(8\),pp\. 2454–2467\.Cited by:[4th item](https://arxiv.org/html/2604.20283#S1.I1.i4.p1.1),[Table 1](https://arxiv.org/html/2604.20283#S1.T1.13.5.11.1),[§1](https://arxiv.org/html/2604.20283#S1.p3.1),[1st item](https://arxiv.org/html/2604.20283#S7.I1.i1.p1.1),[Table 4](https://arxiv.org/html/2604.20283#S7.T4.3.1.11.2),[Table 5](https://arxiv.org/html/2604.20283#S7.T5.4.1.5.1),[Table 5](https://arxiv.org/html/2604.20283#S7.T5.4.1.7.1),[Table 6](https://arxiv.org/html/2604.20283#S7.T6.4.1.3.1),[Table 6](https://arxiv.org/html/2604.20283#S7.T6.4.1.5.1),[Table 6](https://arxiv.org/html/2604.20283#S7.T6.4.1.7.1),[§8\.2](https://arxiv.org/html/2604.20283#S8.SS2.p1.1)\.