Collaboration of Fusion and Independence: Hypercomplex-driven Robust Multi-Modal Knowledge Graph Completion

# Collaboration of Fusion and Independence: Hypercomplex-driven Robust Multi-Modal Knowledge Graph Completion Source: https://arxiv.org/html/2509.23714 Zhiqiang Liu1,3, Yichi Zhang2,3, Mengshu Sun4, Lei Liang4, Wen Zhang1,3 1School of Software Technology, Zhejiang University 2College of Computer Science and Technology, Zhejiang University 3ZJU\-Ant Group Joint Lab of Knowledge Graph 4Ant Group \{zhiqiangliu,zhang\.wen\}@zju\.edu\.cn ###### Abstract Multi\-modal knowledge graph completion \(MMKGC\) aims to discover missing facts in multi\-modal knowledge graphs \(MMKGs\) by leveraging both structural relationships and diverse modality information of entities\. Existing MMKGC methods follow two multi\-modal paradigms: fusion\-based and ensemble\-based\. Fusion\-based methods employ fixed fusion strategies, which inevitably leads to the loss of modality\-specific information and a lack of flexibility to adapt to varying modality relevance across contexts\. In contrast, ensemble\-based methods retain modality independence through dedicated sub\-models but struggle to capture the nuanced, context\-dependent semantic interplay between modalities\. To overcome these dual limitations, we propose a novel MMKGC methodM\-Hyper, which achieves the coexistence and collaboration of fused and independent modality representations\. Our method integrates the strengths of both paradigms, enabling effective cross\-modal interactions while maintaining modality\-specific information\. Inspired by “quaternion” algebra, we utilize its four orthogonal bases to represent multiple independent modalities and employ the Hamilton product to efficiently model pair\-wise interactions among them\. Specifically, we introduce a Fine\-grained Entity Representation Factorization \(FERF\) module and a Robust Relation\-aware Modality Fusion \(R2MF\) module to obtain robust representations for three independent modalities and one fused modality\. The resulting four modality representations are then mapped to the four orthogonal bases of a biquaternion for comprehensive modality interaction\. Extensive experiments indicate its state\-of\-the\-art performance with better robustness\. Our dataset and code are available athttps://github.com/zjukg/M-Hyper\. Collaboration of Fusion and Independence: Hypercomplex\-driven Robust Multi\-Modal Knowledge Graph Completion Zhiqiang Liu1,3, Yichi Zhang2,3, Mengshu Sun4, Lei Liang4, Wen Zhang1,3††thanks:Corresponding Author\.1School of Software Technology, Zhejiang University2College of Computer Science and Technology, Zhejiang University3ZJU\-Ant Group Joint Lab of Knowledge Graph4Ant Group\{zhiqiangliu,zhang\.wen\}@zju\.edu\.cn ## 1Introduction Multi\-modal Knowledge Graphs \(MMKGs\)Liuet al\.\(2019 (https://arxiv.org/html/2509.23714#bib.bib40)\)expand traditional knowledge graphs by incorporating additional multi\-modal information, making them more powerful toolsChenet al\.\(2024 (https://arxiv.org/html/2509.23714#bib.bib13)\)for knowledge representation\. This makes MMKGs valuable for various applications, including recommendation systemsWanget al\.\(2019a (https://arxiv.org/html/2509.23714#bib.bib35)\)and natural language processingChenet al\.\(2023b (https://arxiv.org/html/2509.23714#bib.bib37)\); Liuet al\.\(2025 (https://arxiv.org/html/2509.23714#bib.bib31)\)\. However, like traditional uni\-modal knowledge graphsLiuet al\.\(2024 (https://arxiv.org/html/2509.23714#bib.bib30)\), MMKGs also suffer from incomplete informationXieet al\.\(2017 (https://arxiv.org/html/2509.23714#bib.bib21)\); this limitation has been ameliorated through Multi\-Modal Knowledge Graph Completion \(MMKGC\) methods\. Refer to captionFigure 1:A simple example illustrates the difference between M\-Hyper and existing paradigms\.As shown in Figure1 (https://arxiv.org/html/2509.23714#S1.F1), existing MMKGC approaches fall into two paradigms: fusion\-based and ensemble\-based\. Fusion\-based methodsZhanget al\.\(2025a (https://arxiv.org/html/2509.23714#bib.bib39)\)achieve cross\-modality interaction via explicit fusion modules or dedicated cross\-modality loss functions\. Yet, their reliance on fixed fusion strategies often leads to suboptimal representation: crucial unique modality cues can be lost during fusion, and the model struggles to flexibly adapt to varying modality salience and synergies required in distinct reasoning contexts\. Conversely, ensemble\-based methodsLiet al\.\(2023 (https://arxiv.org/html/2509.23714#bib.bib25)\)preserve modality\-specific characteristics by employing independent sub\-models, but inevitably fail to capture subtle inter\-modal dependencies and interactions that are critical for complex reasoning scenarios\. This highlights a fundamental challenge: the modality requirements in MMKGs exhibit dynamic, context\-dependent, and task\-specific contributions, making rigid adherence to either independent or fully fused paradigms a significant limitation to the expressive power and adaptability of MMKGC models\. Hence, we propose the following research question: is it possible to develop a method thatcombines the strengths of both paradigms, adapting to both fused and independent modality requirements while dynamically enabling comprehensive cross\-modal interactions? To address these limitations, we introduceM\-Hyper, the first method to model MMKGs in ahypercomplex space\. Inspired by quaternion algebra, where the four orthogonal basis elements preserve linear independence, M\-Hyper explicitly separates distinct modality representations to retain original modal information and leverages the Hamilton product to facilitate comprehensive pairwise interactions among modalities\. To enhance the robustness of modality representations, we design two novel modules: Fine\-grained Entity Representation Factorization \(FERF\), which yields robust representations for three independent modalities, and Robust Relation\-aware Modality Fusion \(R2MF\), which produces one robust fused modality representation\. These four representations are mapped to the four orthogonal bases of a biquaternion, and a biquaternion\-based scoring function is used to fully capture cross\-modal semantic information\. Experimental results show that our M\-Hyper achieves state\-of\-the\-art performance on three MMKGC datasets and exhibits high robustness and computational efficiency\. Our contributions can be summarized as follows: - •We highlight the limitations of existing MMKGC paradigms and propose a novel biquaternion\-based representation approach that simultaneously preserves both individual and fused modalities\. - •We propose M\-Hyper, the first MMKGC method operating in a hypercomplex \(biquaternion\) space, enabling robust coexistence and collaboration of fused and independent modality representations\. - •Extensive empirical evaluation on three MMKGC benchmarks demonstrates that M\-Hyper outperforms 18 existing baseline methods, exhibiting superior robustness and computational efficiency\. ## 2Related Works ### 2\.1Hypercomplex\-based KG Embedding Knowledge graph embedding \(KGE\) aims to project entities and relations into continuous vector spaces to capture complex relational patterns\. Classic KGE methods include translational models \(e\.g, TransEBordeset al\.\(2013 (https://arxiv.org/html/2509.23714#bib.bib14)\)\) and semantic\-matching models \(e\.g\., ComplExTrouillonet al\.\(2016 (https://arxiv.org/html/2509.23714#bib.bib19)\)\)\. To enhance representation capabilityLianget al\.\(2024 (https://arxiv.org/html/2509.23714#bib.bib38)\), hypercomplex spaces have been introduced: QuatEZhanget al\.\(2019 (https://arxiv.org/html/2509.23714#bib.bib10)\)first extends embeddings to quaternion space, improving the modeling of symmetry and hierarchy\. Subsequently, DualECaoet al\.\(2021 (https://arxiv.org/html/2509.23714#bib.bib11)\)and BiQUEGuo and Kok \(2021 (https://arxiv.org/html/2509.23714#bib.bib12)\)further generalize to dual quaternions and biquaternion spaces, supporting richer relational composition via translation and rotation\. Hypercomplex representations exhibit strong expressiveness for hierarchical, symmetric, and complex relational structures, and have recently been applied to more advanced KGC scenariosChung and Whang \(2023 (https://arxiv.org/html/2509.23714#bib.bib9)\)\. However, prior hypercomplex\-based methods focus only on uni\-modal knowledge graphs, and their potential for handling rich multi\-modal semantics remains underexplored\. In contrast, our approach is the first to leverage biquaternion space for MMKGs, supporting both multi\-modality and complex relational transformations\. ### 2\.2Multi\-modal Knowledge Graph Completion Existing Multi\-modal Knowledge Graph Completion \(MMKGC\) methods extend traditional KGC models by integrating various modalities \(e\.g\., structural information in MMKG, as well as image and textual information of entities\)\. From the perspective of multi\-modality modeling, current MMKGC methods can be categorized into multi\-modal fusion\-based methods and multi\-modal ensemble\-based methods\. Multi\-modal fusion\-based methods aim to design sophisticated multi\-modal fusion modules to achieve modality alignment\. Earlier modality fusion methods like IKRLXieet al\.\(2017 (https://arxiv.org/html/2509.23714#bib.bib21)\)and TransAEWanget al\.\(2019b (https://arxiv.org/html/2509.23714#bib.bib22)\)achieve efficient modality fusion by introducing cross\-modal loss functions, demonstrating the effectiveness of cross\-modal interactions\. Furthermore, research community continues to propose more complex modality fusion designs with advanced techniques, such as OTKGECaoet al\.\(2022 (https://arxiv.org/html/2509.23714#bib.bib24)\)with optimal transfer, AdaMFZhanget al\.\(2024 (https://arxiv.org/html/2509.23714#bib.bib36)\)with adversarial training and MyGOZhanget al\.\(2025a (https://arxiv.org/html/2509.23714#bib.bib39)\)with fine\-grained multi\-modal tokenization\. However, these modal fusion methods rarely preserve independent modalities and excessively rely on fixed fusion strategies\. Therefore, this paradigm inevitably introduces information loss during the modality fusion stage and makes it difficult to adapt to the flexible modality requirements during the reasoning stage\. In contrast, classic modality ensemble methods like MoSEZhaoet al\.\(2022 (https://arxiv.org/html/2509.23714#bib.bib20)\)usually design individual sub\-models for different modalities, and the individual representations obtained by these sub\-models are integrated for joint decision\-making\. Subsequently, IMFLiet al\.\(2023 (https://arxiv.org/html/2509.23714#bib.bib25)\)utilizes tensor decomposition to fuse multi\-modality information and introduces a sub\-model of joint modalities into the modality ensemble method\. We consider this a promising beginning for achieving joint decision\-making that incorporates both fused and independent modalities\. After that, MoMoKZhanget al\.\(2025b (https://arxiv.org/html/2509.23714#bib.bib3)\)follows this idea and decouples the modal representations through the MoE network with minimizing their mutual information\. However, under the multi\-modality ensemble paradigm, the sub\-models lack explicit mechanisms for comprehensive cross\-modal interaction, thereby limiting their overall modeling capability\. ## 3Preliminaries Quaternionnumber system was first proposed byHamilton \(1844 (https://arxiv.org/html/2509.23714#bib.bib8)\)to extend the complex numbers\. The algebraic representation of a quaternion is typically expressed as: Q=a1+bi+cj+dk,Q=a\\mathbf\{1\}+b\\mathbf\{i\}+c\\mathbf\{j\}+d\\mathbf\{k\},\(1\)where the coefficientaais a real number representing real part, the coefficientsb,c,db,c,dare real numbers representing imaginary part, and1,i,j,k\\mathbf\{1\},\\mathbf\{i\},\\mathbf\{j\},\\mathbf\{k\}are the orthogonal basis vectors or basis elements, which satisfy the following multiplication properties:i1=1i=i\\mathbf\{i\}1=1\\mathbf\{i\}=\\mathbf\{i\},j1=1j=j\\mathbf\{j\}1=1\\mathbf\{j\}=\\mathbf\{j\},k1=1k=k\\mathbf\{k\}1=1\\mathbf\{k\}=\\mathbf\{k\},i2=j2=k2=−1\\mathbf\{i\}^\{2\}=\\mathbf\{j\}^\{2\}=\\mathbf\{k\}^\{2\}=\-1,ij=−ji=k\\mathbf\{ij\}=\-\\mathbf\{ji\}=\\mathbf\{k\},jk=−kj=i\\mathbf\{jk\}=\-\\mathbf\{kj\}=\\mathbf\{i\},ki=−ik=j\\mathbf\{ki\}=\-\\mathbf\{ik\}=\\mathbf\{j\}, andijk=−1\\mathbf\{ijk\}=\-1\. Hamilton Productcan be regarded as “Quaternion Multiplication”, which is composed of all standard multiplications of factors in quaternions, defined as: Q1⊗Q2\\displaystyle Q\_\{1\}\\otimes Q\_\{2\}=\(a1a2−b1b2−c1c2−d1d2\)\\displaystyle=\(a\_\{1\}a\_\{2\}\-b\_\{1\}b\_\{2\}\-c\_\{1\}c\_\{2\}\-d\_\{1\}d\_\{2\}\)\(2\)\+\(a1b2\+b1a2\+c1d2−d1c2\)i\\displaystyle\\,\+\(a\_\{1\}b\_\{2\}\+b\_\{1\}a\_\{2\}\+c\_\{1\}d\_\{2\}\-d\_\{1\}c\_\{2\}\)\\mathbf\{i\}\+\(a1c2−b1d2\+c1a2\+d1b2\)j\\displaystyle\\,\+\(a\_\{1\}c\_\{2\}\-b\_\{1\}d\_\{2\}\+c\_\{1\}a\_\{2\}\+d\_\{1\}b\_\{2\}\)\\mathbf\{j\}\+\(a1d2\+b1c2−c1b2\+d1a2\)k\.\\displaystyle\\,\+\(a\_\{1\}d\_\{2\}\+b\_\{1\}c\_\{2\}\-c\_\{1\}b\_\{2\}\+d\_\{1\}a\_\{2\}\)\\mathbf\{k\}\. Biquaternionsfurther extend quaternions, and their algebra can be considered as a tensor productC⊗RH\\mathbb\{C\}\\otimes\_\{\\mathbb\{R\}\}\\mathbb\{H\}, whereC\\mathbb\{C\}is the field of complex numbers andH\\mathbb\{H\}is the division algebra of \(real\) quaternions\. Biquaternions extend the coefficients of quaternions to complex numbers, denoted as: Q=\(ar\+aiI\)\+\(br\+biI\)i\+\(cr\+ciI\)j\+\(dr\+diI\)k,Q=\(a\_\{r\}\+a\_\{i\}\\mathbf\{I\}\)\+\(b\_\{r\}\+b\_\{i\}\\mathbf\{I\}\)\\mathbf\{i\}\+\(c\_\{r\}\+c\_\{i\}\\mathbf\{I\}\)\\mathbf\{j\}\+\(d\_\{r\}\+d\_\{i\}\\mathbf\{I\}\)\\mathbf\{k\},\(3\)whereI\\mathbf\{I\}is the imaginary unit of the complex number fieldC\\mathbb\{C\}, satisfyingI2=−1\\textbf\{I\}^\{2\}=\-1\. The algebraC⊗RH\\mathbb\{C\}\\otimes\_\{\\mathbb\{R\}\}\\mathbb\{H\}satisfies the commutation relationsIi=iI\\textbf\{Ii\}=\\textbf\{iI\},Ij=jI\\textbf\{Ij\}=\\textbf\{jI\},Ik=kI\\textbf\{Ik\}=\\textbf\{kI\}\. Hamilton Product of Biquaternionscan be seen as an extension of the Hamilton product of quaternions\. Similarly, for two biquaternionsQ1=a1\+b1i\+c1j\+d1k=\(ar,1\+ai,1I\)\+\(br,1\+bi,1I\)i\+\(cr,1\+ci,1I\)j\+\(dr,1\+di,1I\)kQ\_\{1\}=a\_\{1\}\+b\_\{1\}\\mathbf\{i\}\+c\_\{1\}\\mathbf\{j\}\+d\_\{1\}\\mathbf\{k\}=\(a\_\{\\text\{r\},1\}\+a\_\{\\text\{i\},1\}\\mathbf\{I\}\)\+\(b\_\{\\text\{r\},1\}\+b\_\{\\text\{i\},1\}\\mathbf\{I\}\)\\mathbf\{i\}\+\(c\_\{\\text\{r\},1\}\+c\_\{\\text\{i\},1\}\\mathbf\{I\}\)\\mathbf\{j\}\+\(d\_\{\\text\{r\},1\}\+d\_\{\\text\{i\},1\}\\mathbf\{I\}\)\\mathbf\{k\}andQ2=a2\+b2i\+c2j\+d2k=\(ar,2\+ai,2I\)\+\(br,2\+bi,2I\)i\+\(cr,2\+ci,2I\)j\+\(dr,2\+di,2I\)kQ\_\{2\}=a\_\{2\}\+b\_\{2\}\\mathbf\{i\}\+c\_\{2\}\\mathbf\{j\}\+d\_\{2\}\\mathbf\{k\}=\(a\_\{\\text\{r\},2\}\+a\_\{\\text\{i\},2\}\\mathbf\{I\}\)\+\(b\_\{\\text\{r\},2\}\+b\_\{\\text\{i\},2\}\\mathbf\{I\}\)\\mathbf\{i\}\+\(c\_\{\\text\{r\},2\}\+c\_\{\\text\{i\},2\}\\mathbf\{I\}\)\\mathbf\{j\}\+\(d\_\{\\text\{r\},2\}\+d\_\{\\text\{i\},2\}\\mathbf\{I\}\)\\mathbf\{k\}, the multiplication is performed exactly as in Equation2 (https://arxiv.org/html/2509.23714#S3.E2)for quaternions, but with all coefficients treated as complex numbers \(withI2=−1\\mathbf\{I\}^\{2\}=\-1\)\. That is, the Hamilton product is defined in the same way, with addition and multiplication of coefficients carried out in the field of complex numbersC\\mathbb\{C\}\. Refer to captionFigure 2:The overview of our M\-Hyper, which integrates the Fine\-grained Entity Representation Factorization \(FERF\) module and the Robust Relation\-aware Modality Fusion \(R2MF\) module to learn robust representations for three modalities and their fusion, enabling unified multi\-modal knowledge graph modeling in hypercomplex spaces\. ## 4Methodology In this section, we introduceM\-Hyper, which modelsMulti\-modal knowledge graphs \(MMKG\) inHypercomplex spaces\. As shown in Figure2