Boosting Knowledge Graph Foundation Models via Enhanced Negative Sampling

# Boosting Knowledge Graph Foundation Models via Enhanced Negative Sampling
Source: [https://arxiv.org/html/2605.27023](https://arxiv.org/html/2605.27023)
\(2026\)

###### Abstract\.

Knowledge graphs \(KGs\) have become the core backbone of numerous downstream tasks such as question answering and recommender systems\. However, despite all this, KGs are often very incomplete\. To perform zero\-shot knowledge graph completion in unseen KGs, which have different relational vocabularies from those used for pre\-training, KG foundation models \(KGFMs\) receive a wide range of attention\. Existing KGFMs often perform training using random negative triples, which are constructed by replacing the head or tail entity of a positive triple with a random entity\. However, these negative triples are often constructed with limited quality, providing weak supervision for KGFM training\. In this paper, we propose a simple yet effective adaptive negative sampling approach, KMAS, to enhance existing KGFMs\. KMAS constructs hard negative triples through the updated relation embeddings generated from the existing KGFM’s relation encoder\. To further adaptively align with the evolving capability of the KGFM during the training process, KMAS adjusts the ratio of hard negative triples dynamically throughout the whole training process: after a warmup phrase, it increases the ratio linearly and then decreases linearly\. Extensive experiments are conducted over 44 data sets\. Experimental results demonstrate that our proposed negative sampling method can enhance many SOTA KGFMs without requiring excessive additional time or memory consumption\.

Knowledge Graph Foundation Model, Negative Sampling

††journalyear:2026††copyright:acmlicensed††conference:Proceedings of the 32th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 9–13, 2026; Jeju, Korea††booktitle:Proceedings of the 32th ACM SIGKDD Conference on Knowledge Discovery and Data Mining \(KDD ’26\), August 9–13, 2026, Jeju, Korea††doi:XXXXXXX\.XXXXXXX††isbn:978\-1\-4503\-XXXX\-X/2018/06## 1\.Introduction

Knowledge graphs \(KGs\), which consist of massive amounts of knowledge in the form of triples \(head entity, relation, tail entity\), have become the core backbone of numerous downstream tasks such as question answering\(Zhaoet al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib1)\)and recommender systems\(Wanget al\.,[2019](https://arxiv.org/html/2605.27023#bib.bib2)\)\. However, despite all this, KGs are often very incomplete\(Panet al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib50)\), making the task of link prediction\(Rossiet al\.,[2021](https://arxiv.org/html/2605.27023#bib.bib52),[2022](https://arxiv.org/html/2605.27023#bib.bib53)\)increasingly important\. Traditional transductive link prediction methods \(e\.g\., TransE\(Bordeset al\.,[2013](https://arxiv.org/html/2605.27023#bib.bib35)\)and RotatE\(Sunet al\.,[2019](https://arxiv.org/html/2605.27023#bib.bib7)\)\) learn entity/relation embeddings constrained by specific entity and relation vocabularies, lacking the ability to generalize to unseen entities/relations on KGs\. To address the limitation, partially inductive link prediction approaches\(Liuet al\.,[2021](https://arxiv.org/html/2605.27023#bib.bib10)\)relax the constraint on entities and generalize to unseen entities while keeping the relation vocabulary fixed\. Fully inductive link prediction approaches\(Leeet al\.,[2023](https://arxiv.org/html/2605.27023#bib.bib12)\)further relax the constraint on relations and generalize to both unseen entities and unseen relations, which motivates the development of knowledge graph foundation models \(KGFMs\)\. KGFMs learn invariance of the relation structure\(Galkinet al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib3)\)by pre\-training on multiple KGs of different entity/relation vocabularies, thereby enabling zero\-shot reasoning on unseen KGs\.

Existing KGFMs \(e\.g\., ULTRA\(Galkinet al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib3)\), TRIX\(Zhanget al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib8)\), MOTIF\(Huanget al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib6)\), and SEMMA\(Arunet al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib4)\)\) usually adopt a common procedure: \(1\) construct a relation graph based on the KG; \(2\) apply a relation encoder on this relation graph to obtain relation embeddings; \(3\) use the obtained relation embeddings and employ an entity encoder on the KG to obtain final link prediction results\. It is worth noting that the training process of these KGFMs’ relation and entity encoders is based on random negative triples, which are constructed by replacing the head or tail entity of a positive triple with a random entity\. Although straightforward and efficient, these negative triples are often constructed with limited quality \(i\.e\., “easy negative triples”\), providing weak supervision\. Since these easy negative triples do not be dynamically adjusted throughout the whole training process, they fail to align with the KGFM’s evolving capabilities\. For instance, given a positive triple \(Obama,born\_in,Hawaii\), a randomly constructed negative triple like \(Obama,born\_in,Basketball\) is trivial for a well\-trained KGFM\. Conversely, constructing “hard negative triples” such as \(Obama,born\_in,Chicago\)—which is factually incorrect yet semantically plausible—can provide more challenging training samples to enhance the learned embeddings, improving the generalization ability of the KGFM\.

In this paper, to enhance existingKGFMs, we propose a simple yet effective adaptive negAtiveSampling method KMAS\. To construct hard negative triples for improving the training process of the KGFM, KMAS leverages the relation embeddings generated by the relation encoder of an existing KGFM to construct the tail \(head\) entity distribution, which enables hybrid negative sampling\. In order to adaptively align with the evolving capability of the KGFM during the training process, KMAS iteratively updates relation embeddings using the encoder from the last training iteration\. This update refines the tail \(head\) entity distribution and thereby generates progressively more effective negative samples\. Notably, KMAS dynamically adjusts the ratio of hard negative triples throughout the training process: after a warmup phase that employs pure random negative sampling \(as in existing KGFMs\), it linearly increases the ratio of hard negative triples to a peak, then linearly decreases it in subsequent iterations\.

The main contributions of this paper are summarized as follows:

- •To the best of our knowledge, we are the first to enhance the negative sampling method in KGFMs, boosting the existing KGFMs’ performance\.
- •We propose a simple yet effective negative sampling method by adaptively constructing hard negative samples with a dynamic hard negative ratio adjustment strategy, which is flexible to adapt to many KGFMs\.
- •We conduct extensive experiments on4444data sets\. Experimental results demonstrate that our proposed negative sampling method can improve the performance of many SOTA KGFMs without requiring excessive additional time or memory consumption\.

## 2\.Preliminaries

A knowledge graph \(KG\) is denoted by𝒢=\(𝒱,ℛ,𝒯\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{R\},\\mathcal\{T\}\), where𝒱\\mathcal\{V\}denotes a set of entities,ℛ\\mathcal\{R\}denotes a set of relations, and𝒯⊆𝒱×ℛ×𝒱\\mathcal\{T\}\\subseteq\\mathcal\{V\}\\times\\mathcal\{R\}\\times\\mathcal\{V\}denotes a set of triples\. A triple\(h,r,t\)∈𝒯\(h,r,t\)\\in\\mathcal\{T\}connects the head entityh∈𝒱h\\in\\mathcal\{V\}with the tail entityt∈𝒱t\\in\\mathcal\{V\}through the relationr∈ℛr\\in\\mathcal\{R\}\. Given a KG𝒢=\(𝒱,ℛ,𝒯\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{R\},\\mathcal\{T\}\), the corresponding relation graph𝒢ℛ=\(𝒱ℛ,ℰℛ\)\\mathcal\{G\}\_\{\\mathcal\{R\}\}=\(\\mathcal\{V\}\_\{\\mathcal\{R\}\},\\mathcal\{E\}\_\{\\mathcal\{R\}\}\)is a directed graph where nodes𝒱ℛ\\mathcal\{V\}\_\{\\mathcal\{R\}\}represent the relationsℛ\\mathcal\{R\}\. The edgesℰℛ\\mathcal\{E\}\_\{\\mathcal\{R\}\}capture interactions between relations based on their connectivity in𝒯\\mathcal\{T\}\.

Given a knowledge graph𝒢=\(𝒱,ℛ,𝒯\)\\mathcal\{G\}=\(\\mathcal\{V\},\\mathcal\{R\},\\mathcal\{T\}\), the task of link prediction aims to infer missing facts based on observed ones\. Formally, given a query\(h,r,?\)\(h,r,?\)\(resp\.\(?,r,t\)\(?,r,t\)\), the goal is to predict the missing tail entityt∈𝒱t\\in\\mathcal\{V\}\(resp\. head entityh∈𝒱h\\in\\mathcal\{V\}\)\. During inference, for a query\(h,r,?\)\(h,r,?\), we calculate scores for all candidate entitiest′∈𝒱t^\{\\prime\}\\in\\mathcal\{V\}and rank them in descending order\. The objective is to rank the ground\-truth entity higher than other candidates\. We denote𝒢t​r​a​i​n=\(𝒱t​r​a​i​n,ℛt​r​a​i​n,𝒯t​r​a​i​n\)\\mathcal\{G\}\_\{train\}=\(\\mathcal\{V\}\_\{train\},\\mathcal\{R\}\_\{train\},\\mathcal\{T\}\_\{train\}\)as the training graph and𝒢t​e​s​t=\(𝒱t​e​s​t,ℛt​e​s​t,𝒯t​e​s​t\)\\mathcal\{G\}\_\{test\}=\(\\mathcal\{V\}\_\{test\},\\mathcal\{R\}\_\{test\},\\mathcal\{T\}\_\{test\}\)as the inference graph\. We consider three generalization settings based on the overlap between𝒢t​r​a​i​n\\mathcal\{G\}\_\{train\}and𝒢t​e​s​t\\mathcal\{G\}\_\{test\}:

- •Transductive setting: The entity and relation sets are shared by the training and inference processes, i\.e\.,𝒱t​r​a​i​n=𝒱t​e​s​t\\mathcal\{V\}\_\{train\}=\\mathcal\{V\}\_\{test\},ℛt​r​a​i​n=ℛt​e​s​t\\mathcal\{R\}\_\{train\}=\\mathcal\{R\}\_\{test\}, and𝒯t​r​a​i​n=𝒯t​e​s​t\\mathcal\{T\}\_\{train\}=\\mathcal\{T\}\_\{test\}\.
- •Partially inductive setting \(unseen entities\): Entities at inference time are unseen during training, while the relation set is shared by the training and inference processes, i\.e\.,𝒱t​r​a​i​n∩𝒱t​e​s​t=∅\\mathcal\{V\}\_\{train\}\\cap\\mathcal\{V\}\_\{test\}=\\emptyset,ℛt​r​a​i​n=ℛt​e​s​t\\mathcal\{R\}\_\{train\}=\\mathcal\{R\}\_\{test\}, and𝒯t​r​a​i​n≠𝒯t​e​s​t\\mathcal\{T\}\_\{train\}\\neq\\mathcal\{T\}\_\{test\}\.
- •Fully inductive setting \(unseen entities & relations\): The most challenging foundation model setting, where both entities and relations at inference time are unseen during training, i\.e\.,𝒱t​r​a​i​n∩𝒱t​e​s​t=∅\\mathcal\{V\}\_\{train\}\\cap\\mathcal\{V\}\_\{test\}=\\emptyset,ℛt​r​a​i​n∩ℛt​e​s​t=∅\\mathcal\{R\}\_\{train\}\\cap\\mathcal\{R\}\_\{test\}=\\emptyset, and𝒯t​r​a​i​n≠𝒯t​e​s​t\\mathcal\{T\}\_\{train\}\\neq\\mathcal\{T\}\_\{test\}\.

For existing KGFMs, we useEncoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}to denote the relation encoder that generates relation embeddings\.Encoderθe\\text\{Encoder\}\_\{\\theta\_\{e\}\}denote the entity encoder that generates entity embeddings\.fωf\_\{\\omega\}denotes an MLP that maps these entity embeddings to final scores\.θr\\theta\_\{r\}denotes the parameters of the GNN architecture ofEncoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}\.θe\\theta\_\{e\}denotes the parameters of the GNN architecture ofEncoderθe\\text\{Encoder\}\_\{\\theta\_\{e\}\}\.ω\\omegadenotes the parameters of the MLP that generates the final scores\. For KGFMs, the GNN architecture often adopts NBFNet\(Zhuet al\.,[2021](https://arxiv.org/html/2605.27023#bib.bib13)\)\. In KFGMs, a labeling trick is used to initialize relations to generate conditional relation embeddings based on the relation graph𝒢ℛ\\mathcal\{G\}\_\{\\mathcal\{R\}\}, which means thatrrof the given query \(i\.e\.,\(h,r,?\)\(h,r,?\)or\(?,r,t\)\(?,r,t\)\) is initialized as an all\-one vector, while other relations are initialized as all\-zero vectors\.Encoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}uses initialized relation embeddings to generate conditional relation embeddings forEncoderθe\\text\{Encoder\}\_\{\\theta\_\{e\}\}\. InEncoderθe\\text\{Encoder\}\_\{\\theta\_\{e\}\}, the entity in query is initialized as the conditional relation embedding ofrr, and other entities are initialized as all\-zero vectors\. Then,Encoderθe\\text\{Encoder\}\_\{\\theta\_\{e\}\}uses the initialized entity embeddings and conditional relation embeddings to generate entity embeddings forfωf\_\{\\omega\}to obtain the scores, which are used to predict the missing entity in the given query\.

![Refer to caption](https://arxiv.org/html/2605.27023v1/x1.png)Figure 1

Figure 1\.The overall training process of our proposed negative sampling method KMAS\.
## 3\.The Method KMAS

### 3\.1\.Overview

In this section, we briefly introduce our proposed method KMAS\. Specifically, during each iteration of an existing KGFM’s training process, KMAS first calculates relation similarities to identify the relations that are semantically similar to the target relation of each positive triple based on the relation embeddings generated throughEncoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}based on the relation graph \(Section[3\.2](https://arxiv.org/html/2605.27023#S3.SS2)\)\. Next, using these similarities, KMAS assigns weights to entities to construct head \(tail\) entity distributions for hard negative sample generation \(Section[3\.3](https://arxiv.org/html/2605.27023#S3.SS3)\)\. Then, a hybrid sampling strategy is employed to construct the final negative triple set for each triple by combining random and hard negative samples according to different distributions \(Section[3\.4](https://arxiv.org/html/2605.27023#S3.SS4)\)\. Furthermore, KMAS dynamically adjusts the ratio of hard negative samples versus random negative samples throughout training iterations, which follows a trade\-off policy by combining both “easier first” as curriculum learning and “harder first” as hard example mining: beginning with a warmup phase without hard negative samples, increasing linearly to a peak, and then decreasing linearly \(Section[3\.5](https://arxiv.org/html/2605.27023#S3.SS5)\)\. Thus, the KGFM’sEncoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}can be trained based on the training set consisting of positive triples and adaptively constructed negative triples \(Section[3\.6](https://arxiv.org/html/2605.27023#S3.SS6)\)\. After multiple iterations, the relation embeddings generated fromEncoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}, together with the KGFM’s entity encoderEncodere\\text\{Encoder\}\_\{e\}andfωf\_\{\\omega\}, are utilized to score all candidates for each test triple \(Section[3\.6](https://arxiv.org/html/2605.27023#S3.SS6)\)\.

### 3\.2\.Relation Similarity Acquisition

To construct high\-quality hard negative samples for positive triples, we first utilize the relation embeddings generated byEncoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}to measure the similarities between the relations\. For the relationrrof a given positive triple\(h,r,t\)\(h,r,t\), we input it intoEncoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}to obtain relation embeddings for all relations in𝒢\{\\mathcal\{G\}\}as follows:

\(1\)H←Encoderθr​\(𝒢ℛ,r,t\),H\\leftarrow\\text\{Encoder\}\_\{\\theta\_\{r\}\}\(\\mathcal\{G\}\_\{\\mathcal\{R\}\},r,t\),wheredddenotes the dimension of embeddings inEncoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\},H∈ℝ\|ℛ\|×dH\\in\\mathbb\{R\}^\{\|\\mathcal\{R\}\|\\times d\}denotes the set of relation embeddings\. If SEMMA’s relation encoder is selected asEncoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\},ttis set to the text of relations\. For other KGFMs \(i\.e, ULTRA, TRIX, and MOTIF\),ttis set to none\. Then, based onHH, we can calculate the similarity vector𝐒∈ℝ\|ℛ\|\\mathbf\{S\}\\in\\mathbb\{R\}^\{\|\\mathcal\{R\}\|\}betweenrrand the relationror\_\{o\}inℛ\\mathcal\{R\}as follows:

\(2\)𝐒​\(r,ro\)=Sim​\(𝐡r,𝐡ro\)\\mathbf\{S\}\(\{r,r\_\{o\}\}\)=\\text\{Sim\}\(\\mathbf\{h\}\_\{r\},\\mathbf\{h\}\_\{r\_\{o\}\}\)where the function Sim\(⋅\\cdot\) is implemented by the cosine similarity,𝐡r\\mathbf\{h\}\_\{r\}denotes the relation embedding ofrrand𝐡ro\\mathbf\{h\}\_\{r\_\{o\}\}denotes the relation embedding ofror\_\{o\}\. Specifically, we set𝐒​\(r,r\)\\mathbf\{S\}\(\{r,r\}\)to−∞\-\\infty, which will prevent the relation itself from sampling in the following process\. Based on this, we construct the vectorWr∈ℝ\|ℛ\|W\_\{r\}\\in\\mathbb\{R\}^\{\|\\mathcal\{R\}\|\}forrrto store relation similarities using the Softmax function with a temperature coefficientτ\\tauas follows:

\(3\)Wr,j=exp⁡\(𝐒r,j/τ\)∑j′=1\|ℛ\|exp⁡\(𝐒r,j′/τ\),j=1,2,…,\|ℛ\|\.W\_\{r,j\}=\\frac\{\\exp\(\\mathbf\{S\}\_\{r,j\}/\\tau\)\}\{\\sum^\{\|\\mathcal\{R\}\|\}\_\{j^\{\\prime\}=1\}\\exp\(\\mathbf\{S\}\_\{r,j^\{\\prime\}\}/\\tau\)\},j=1,2,\\ldots,\\mathcal\{\|R\|\}\.

### 3\.3\.Relation\-Aware Entity Distribution Generation

First, we sort the weights of relations inWrW\_\{r\}in descending order to obtainWr∗W^\{\*\}\_\{r\}\. Next, forrr, we select the topKKrelationsRt​o​p​K​\(r\)R\_\{topK\}\{\(r\)\}whose sum of weights exceeds a thresholdqqas follows:

\(4\)K=arg⁡min𝑧​z,s\.t\.∑y=1zWr,y∗≥q,z∈\{1,…,\|ℛ\|\}\.K=\\underset\{z\}\{\\arg\\min\}\\;z,\\quad\\text\{s\.t\.\}\\quad\\sum\_\{y=1\}^\{z\}W^\{\*\}\_\{r,y\}\\geq q,\\quad z\\in\\\{1,\\ldots,\|\\mathcal\{R\}\|\\\}\.For the relations not inRt​o​p​K​\(r\)R\_\{topK\}\{\(r\)\}, we set their weights to zero\. Subsequently, we reweight the selected relations to construct a weight matrixW′W^\{\\prime\}by calculating the weightWr,b′W^\{\\prime\}\_\{r,b\}for every relationrrand its corresponding selected relationbbas follows:

\(5\)Wr,b′=Wr,b∗∑b′=1KWr,b′∗,b=1,2,…,K\.W^\{\\prime\}\_\{r,b\}=\\frac\{W^\{\*\}\_\{r,b\}\}\{\\sum\_\{b^\{\\prime\}=1\}^\{K\}W^\{\*\}\_\{r,b^\{\\prime\}\}\},b=1,2,\\ldots,K\.To generate hard negative samples for the given positive triple\(h,r,t\)\(h,r,t\), we construct the sets of candidate entitieswrt​a​i​lw^\{tail\}\_\{r\}\(resp\.wrh​e​a​dw^\{head\}\_\{r\}\) by searching entities in𝒱\\mathcal\{V\}that can act as tail \(resp\. head\) entities of the selected relations fromRt​o​p​K​\(r\)R\_\{topK\}\(r\)in𝒢\\mathcal\{G\}\. Thus, we can obtain the weightxr,et​a​i​lx^\{tail\}\_\{r,e\}of an entityeeas the tail entity ofrrby aggregating the relation weights as follows:

\(6\)xr,et​a​i​l=∑r′∈Rt​o​p​K​\(r\)Wr,r′′⋅It​a​i​l​\(r′,e\)\.x^\{tail\}\_\{r,e\}=\\sum\_\{r^\{\\prime\}\\in R\_\{topK\}\(r\)\}W^\{\\prime\}\_\{r,r^\{\\prime\}\}\\cdot I\_\{tail\}\(r^\{\\prime\},e\)\.\(7\)It​a​i​l​\(r′,e\)=\{1if​e∈wrt​a​i​l​and​r′∈Rt​o​p​K​\(r\),0otherwise\.I\_\{tail\}\(r^\{\\prime\},e\)=\\begin\{cases\}1&\\text\{if \}e\\in w^\{tail\}\_\{r\}\\text\{ and \}r^\{\\prime\}\\in R\_\{topK\}\(r\),\\\\ 0&\\text\{otherwise\}\.\\end\{cases\}It can be seen that if an entity frequently appears as the tail entity of the relation that is similar torr, this entity will be assigned a high probability score\. Then, we normalize these weights of entities to obtain a tail entity probability distribution𝒫rt​a​i​l\\mathcal\{P\}^\{tail\}\_\{r\}forrras follows:

\(8\)𝒫rt​a​i​l​\(e\)=xr,et​a​i​l∑e′∈𝒱xr,e′t​a​i​l\.\\mathcal\{P\}^\{tail\}\_\{r\}\(e\)=\\frac\{x^\{tail\}\_\{r,e\}\}\{\\sum\_\{e^\{\\prime\}\\in\\mathcal\{V\}\}x^\{tail\}\_\{r,e^\{\\prime\}\}\}\.Note that, we can obtain the weightxr,eh​e​a​dx^\{head\}\_\{r,e\}of an entityeeas the head entity ofrrand the head entity probability distribution𝒫rh​e​a​d\\mathcal\{P\}^\{head\}\_\{r\}forrrin a similar way, which is omitted due to the length of the paper\.

### 3\.4\.Hybrid Negative Sampling

To improve the model’s performance, we propose a hybrid sampling strategy to provide more challenging negative samples while still retaining the simpler samples, preserving the fundamental ability to distinguish between positive and negative samples\. We denoteℬ\+\\mathcal\{B\}^\{\+\}as a batch of positive triples\. For each positive triple\(h,r,t\)\(h,r,t\)inℬ\+\\mathcal\{B\}^\{\+\}, we construct its corresponding negative sample setℬ\(h,r,t\)−\\mathcal\{B\}^\{\-\}\_\{\(h,r,t\)\}\. Specifically, for\(h,r,t\)\(h,r,t\), we drawNr​a​n​dN\_\{rand\}entities randomly from𝒱\\mathcal\{V\}to obtain𝒯\(h,r,t\)r​a​n​d\\mathcal\{T\}^\{rand\}\_\{\(h,r,t\)\}\(resp\.ℋ\(h,r,t\)r​a​n​d\\mathcal\{H\}^\{rand\}\_\{\(h,r,t\)\}\), and draw theNh​a​r​dN\_\{hard\}entities according to the distribution𝒫rt​a​i​l\\mathcal\{P\}^\{tail\}\_\{r\}\(resp\.𝒫rh​e​a​d\\mathcal\{P\}^\{head\}\_\{r\}\) to obtain𝒯\(h,r,t\)h​a​r​d\\mathcal\{T\}^\{hard\}\_\{\(h,r,t\)\}\(resp\.ℋ\(h,r,t\)h​a​r​d\\mathcal\{H\}^\{hard\}\_\{\(h,r,t\)\}\)\. The number of random negative samplesNr​a​n​dN\_\{rand\}and hard negative samplesNh​a​r​dN\_\{hard\}are calculated as follows:

\(9\)Nr​a​n​d=⌊N⋅\(1−αm\)⌋,N\_\{rand\}=\\lfloor N\\cdot\(1\-\\alpha\_\{m\}\)\\rfloor,\(10\)Nh​a​r​d=N−Nr​a​n​d\.N\_\{hard\}=N\-N\_\{rand\}\.Note thatαm\\alpha\_\{m\}denotes the current hard negative ratio, which will be calculated in Section[3\.5](https://arxiv.org/html/2605.27023#S3.SS5)\. The negative sample sets𝒞\(h,r,t\)t​a​i​l\\mathcal\{C\}^\{tail\}\_\{\(h,r,t\)\}and𝒞\(h,r,t\)h​e​a​d\\mathcal\{C\}^\{head\}\_\{\(h,r,t\)\}are constructed as follows:

\(11\)𝒞\(h,r,t\)t​a​i​l=\{\(h,r,t′\)\|t′∈𝒯\(h,r,t\)r​a​n​d∪𝒯\(h,r,t\)h​a​r​d\},\\mathcal\{C\}^\{tail\}\_\{\(h,r,t\)\}=\\\{\(h,r,t^\{\\prime\}\)\|t^\{\\prime\}\\in\\mathcal\{T\}^\{rand\}\_\{\(h,r,t\)\}\\cup\\mathcal\{T\}^\{hard\}\_\{\(h,r,t\)\}\\,\\\},
\(12\)𝒞\(h,r,t\)h​e​a​d=\{\(h′,r,t\)\|h′∈ℋ\(h,r,t\)r​a​n​d∪ℋ\(h,r,t\)h​a​r​d\}\.\\mathcal\{C\}^\{head\}\_\{\(h,r,t\)\}=\\\{\(h^\{\\prime\},r,t\)\|h^\{\\prime\}\\in\\mathcal\{H\}^\{rand\}\_\{\(h,r,t\)\}\\cup\\mathcal\{H\}^\{hard\}\_\{\(h,r,t\)\}\\,\\\}\.Based on these, we can construct the final negative sample setℬ\(h,r,t\)−\\mathcal\{B\}^\{\-\}\_\{\(h,r,t\)\}for\(h,r,t\)\(h,r,t\)as follows:

\(13\)ℬ\(h,r,t\)−=\{𝒞\(h,r,t\)t​a​i​lif​\(h,r,t\)​is in the first half of​ℬ\+,𝒞\(h,r,t\)h​e​a​dotherwise\.\\mathcal\{B\}^\{\-\}\_\{\(h,r,t\)\}=\\begin\{cases\}\\mathcal\{C\}^\{tail\}\_\{\(h,r,t\)\}&\\text\{if \}\(h,r,t\)\\text\{is in the first half of \}\\mathcal\{B\}^\{\+\},\\\\ \\mathcal\{C\}^\{head\}\_\{\(h,r,t\)\}&\\text\{otherwise\}\.\\par\\end\{cases\}Thus, by constructing negative samples for each positive triple inℬ\+\\mathcal\{B\}^\{\+\}according to the above procedure, we can obtain the corresponding negative sample setℬ−\\mathcal\{B\}^\{\-\}ofℬ\+\\mathcal\{B\}^\{\+\}\.

### 3\.5\.Hard Negative Ratio Adjustment Strategy

Inspired by the previous study\(Wanget al\.,[2021](https://arxiv.org/html/2605.27023#bib.bib34)\), which combines both “easier first” as curriculum learning and “harder first” as hard example mining, we propose a strategy for hard negative sampling to dynamically adjust the ratio between random negative samples and our constructed hard negative samples\. We define the hard negative ratioαm\\alpha\_\{m\}as a piecewise linear function, which is controlled by the warmup ratioβw​a​r​m\\beta\_\{warm\}ranging from 0 to 1, the peak positionγp​e​a​k\\gamma\_\{peak\}, and the maximum ratio of the hard negative samplesαm​a​x\\alpha\_\{max\}\. Crucially,γp​e​a​k\\gamma\_\{peak\}determines the point whereαm\\alpha\_\{m\}reaches its maximum valueαm​a​x\\alpha\_\{max\}within the remaining training process after the warmup phase, which meansγp​e​a​k\\gamma\_\{peak\}is greater thanβw​a​r​m\\beta\_\{warm\}and less than11\. Letmmdenote themm\-th iteration of the whole training iterations andTt​o​t​a​lT\_\{total\}denote the total number of training iterations\. We calculate the ratio of completed iterationspmp\_\{m\}as follows:

\(14\)pm=mTt​o​t​a​l\.p\_\{m\}=\\frac\{m\}\{T\_\{total\}\}\.We can calculateαm\\alpha\_\{m\}for the currentmm\-th iteration based on the following rules\.

- •Ifpmp\_\{m\}is less thanβw​a​r​m\\beta\_\{warm\}, we setαm\\alpha\_\{m\}to0\.
- •Ifpmp\_\{m\}is greater than or equal toβw​a​r​m\\beta\_\{warm\}and less thanPp​e​a​kP\_\{peak\},αm\\alpha\_\{m\}is calculated as follows: \(15\)αm=αm​a​x⋅pm−βw​a​r​mPp​e​a​k−βw​a​r​m\.\\alpha\_\{m\}=\\alpha\_\{max\}\\cdot\\frac\{p\_\{m\}\-\\beta\_\{warm\}\}\{P\_\{peak\}\-\\beta\_\{warm\}\}\.
- •Ifpmp\_\{m\}is greater than or equal toPp​e​a​kP\_\{peak\},αm\\alpha\_\{m\}is calculated as follows: \(16\)αm=αm​a​x⋅1−pm1−Pp​e​a​k,\\alpha\_\{m\}=\\alpha\_\{max\}\\cdot\\frac\{1\-p\_\{m\}\}\{1\-P\_\{peak\}\},wherePp​e​a​kP\_\{peak\}equalsβw​a​r​m\+γp​e​a​k​\(1−βw​a​r​m\)\\beta\_\{warm\}\+\\gamma\_\{peak\}\(1\-\\beta\_\{warm\}\)\.

For the hard negative ratio during the whole training process, this heuristic adjustment strategy is simple yet effective, which is verified by our experiments \(Section[4\.5](https://arxiv.org/html/2605.27023#S4.SS5)\)\.

### 3\.6\.Training

The KGFM is often trained by minimizing the binary cross\-entropy loss function\. We obtain the prediction scores​\(h,r,t\)s\(h,r,t\)of a triple\(h,r,t\)\(h,r,t\)viafωf\_\{\\omega\}\. The optimization objective for a given positive triple\(h,r,t\)\(h,r,t\)and its corresponding negative triple setℬ\(h,r,t\)−\\mathcal\{B\}^\{\-\}\_\{\(h,r,t\)\}is defined as follows:

\(17\)ℒ​\(h,r,t\)=−log⁡\(σ​\(s​\(h,r,t\)\)\)−∑\(h′,r,t′\)∈ℬ\(h,r,t\)−w​\(h′,r,t′\)​log⁡\(1−σ​\(s​\(h′,r,t′\)\)\),\\begin\{split\}\\mathcal\{L\}\(h,r,t\)=&\-\\log\(\\sigma\(s\(h,r,t\)\)\)\-\\\\ &\\sum\_\{\(h^\{\\prime\},r,t^\{\\prime\}\)\\in\\mathcal\{B\}^\{\-\}\_\{\(h,r,t\)\}\}w\(h^\{\\prime\},r,t^\{\\prime\}\)\\log\(1\-\\sigma\(s\(h^\{\\prime\},r,t^\{\\prime\}\)\)\),\\end\{split\}whereσ\\sigmais the Sigmoid activation function\. Additionally, considering that different negative samples have different importance to the training process of the KGFM, we assign the normalized weight to each negative sample using the Softmax function with a temperature coefficientτa​d​v\{\\tau\_\{adv\}\}as follows:

\(18\)w​\(h′,r,t′\)=exp⁡\(s​\(h′,r,t′\)/τa​d​v\)∑\(hk,r,tk\)∈ℬ\(h,r,t\)−exp⁡\(s​\(hk,r,tk\)/τa​d​v\)\.w\(h^\{\\prime\},r,t^\{\\prime\}\)=\\frac\{\\exp\(\{s\(h^\{\\prime\},r,t^\{\\prime\}\)\}/\\tau\_\{adv\}\)\}\{\\sum\_\{\(h\_\{k\},r,t\_\{k\}\)\\in\\mathcal\{B\}^\{\-\}\_\{\(h,r,t\)\}\}\\exp\(\{s\(h\_\{k\},r,t\_\{k\}\)\}/\\tau\_\{adv\}\)\}\.Finally, the total optimization objective for the given batchℬ\+\\mathcal\{B\}^\{\+\}is calculated by averaging the losses over all positive triples inℬ\+\\mathcal\{B\}^\{\+\}as follows:

\(19\)ℒt​o​t​a​l=1\|ℬ\+\|​∑\(h,r,t\)∈ℬ\+ℒ​\(h,r,t\)\.\\mathcal\{L\}\_\{total\}=\\frac\{1\}\{\|\\mathcal\{B\}^\{\+\}\|\}\{\\sum\_\{\(h,r,t\)\\in\\mathcal\{B\}^\{\+\}\}\{\\mathcal\{L\}\(h,r,t\)\}\}\.
It is worth mentioning that the processes of the relation similarity acquisition \(Section[3\.2](https://arxiv.org/html/2605.27023#S3.SS2)\) and the relation\-aware entity distribution generation \(Section[3\.3](https://arxiv.org/html/2605.27023#S3.SS3)\) are fully adaptive\. Since the parameters of the relation encoderθr\\theta\_\{r\}are updated continuously through the backpropagation mechanism during the training process of the KGFM, the embeddings of the relations also evolve throughout the whole training process\. Consequently, even for the same KG and the same relation sampled in different iterations, the values of weightsWrW\_\{r\}and the tail \(resp\. head\) entity distribution𝒫rt​a​i​l\\mathcal\{P\}^\{tail\}\_\{r\}\(resp\.𝒫rh​e​a​d\\mathcal\{P\}^\{head\}\_\{r\}\) also change dynamically for each iteration\. This mechanism ensures that the constructed negative samples are time\-variant\. Although these negative samples may not be the most difficult, they represent the most challenging negative samples for the current state of the KGFM\. The details of the training process are shown in Algorithm[1](https://arxiv.org/html/2605.27023#algorithm1)\. Additionally, we adopt the same inference procedure as previous KGFM studies\(Galkinet al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib3); Zhanget al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib8); Huanget al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib6); Arunet al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib4)\)\.

1

2Input:KGs

𝒢1,𝒢2​…,𝒢n\\mathcal\{G\}\_\{1\},\\mathcal\{G\}\_\{2\}\\ldots,\\mathcal\{G\}\_\{n\}, relation graphs

𝒢R1,𝒢R2,…,𝒢Rn\\mathcal\{G\}\_\{R\_\{1\}\},\\mathcal\{G\}\_\{R\_\{2\}\},\\ldots,\\mathcal\{G\}\_\{R\_\{n\}\}, the entity encoder

Encoderθe\\text\{Encoder\}\_\{\\theta\_\{e\}\}, the relation encoder

Encoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}, the MLP

fωf\_\{\\omega\}, the number of negatives

NN, threshold

qq, temperatures

τ\\tauand

τa​d​v\\tau\_\{adv\}, total number of iterations

TtotalT\_\{\\text\{total\}\}, warmup ratio

βwarm\\beta\_\{\\text\{warm\}\}, peak position

γpeak\\gamma\_\{\\text\{peak\}\}, maximum ratio of the hard negetive samples

αmax\\alpha\_\{\\max\}
3

4Output:Optimized parameters

θr\\theta\_\{r\},

θe\\theta\_\{e\}and

ω\\omega
5

6Initialize

θr\\theta\_\{r\},

θe\\theta\_\{e\}, and

ω\\omegarandomly

7for*m=1m=1toTt​o​t​a​lT\_\{total\}*do

8Randomly sample a KG

𝒢i\\mathcal\{G\}\_\{i\}and a batch of positive triples

ℬ\+\\mathcal\{B\}^\{\+\}from

𝒢i\\mathcal\{G\}\_\{i\}
9

10Calculate the ratio of completed iterations

pmp\_\{m\}via Formula \([14](https://arxiv.org/html/2605.27023#S3.E14)\)

11for*each positive triple\(h,r,t\)∈ℬ\+\(h,r,t\)\\in\\mathcal\{B\}^\{\+\}*do

12if*pm<βw​a​r​mp\_\{m\}<\\beta\_\{warm\}*then

13

αm←0\\alpha\_\{m\}\\leftarrow 0
14Draw

Nr​a​n​dN\_\{rand\}entities from

𝒱\\mathcal\{V\}randomly

15Construct the negative sample set

ℬ\(h,r,t\)−\\mathcal\{B\}^\{\-\}\_\{\(h,r,t\)\}for

\(h,r,t\)\(h,r,t\)by replacing the head/tail entity with a random entity in

𝒢i\\mathcal\{G\}\_\{i\}
16else

17

H←Encoderθr​\(𝒢Ri,r,t\)H\\leftarrow\\text\{Encoder\}\_\{\\theta\_\{r\}\}\(\\mathcal\{G\}\_\{R\_\{i\}\},r,t\)
18Calculate the weight vector

WrW\_\{r\}via Formula \([3](https://arxiv.org/html/2605.27023#S3.E3)\)

19Generate the relation set

Rt​o​p​K​\(r\)R\_\{topK\}\(r\)based on

WrW\_\{r\}
20Calculate

W′W^\{\\prime\}via Formula \([5](https://arxiv.org/html/2605.27023#S3.E5)\)

21Calculate the weights of entities

xr,et​a​i​l\{x\}^\{tail\}\_\{r,e\}and

xr,eh​e​a​dx^\{head\}\_\{r,e\}via Formula \([6](https://arxiv.org/html/2605.27023#S3.E6)\) based on

W′W^\{\\prime\}and

Rt​o​p​K​\(r\)R\_\{topK\}\(r\)
22Generate entity probability distributions

𝒫rt​a​i​l\\mathcal\{P\}^\{tail\}\_\{r\}and

𝒫rh​e​a​d\\mathcal\{P\}^\{head\}\_\{r\}via Formula \([8](https://arxiv.org/html/2605.27023#S3.E8)\)

23if*pmp\_\{m\}¡Pp​e​a​kP\_\{peak\}*then

24Update

αm\\alpha\_\{m\}via Formula \([15](https://arxiv.org/html/2605.27023#S3.E15)\)

25else

26Update

αm\\alpha\_\{m\}via Formula \([16](https://arxiv.org/html/2605.27023#S3.E16)\)

27Draw

Nr​a​n​dN\_\{rand\}entities from

𝒱\\mathcal\{V\}randomly

28Draw

N−Nr​a​n​dN\-N\_\{rand\}entities according to

𝒫rt​a​i​l\\mathcal\{P\}^\{tail\}\_\{r\}and

𝒫rh​e​a​d\\mathcal\{P\}^\{head\}\_\{r\}
29Construct the negative sample set

ℬ\(h,r,t\)−\\mathcal\{B\}^\{\-\}\_\{\(h,r,t\)\}for

\(h,r,t\)\(h,r,t\)via Formula \([13](https://arxiv.org/html/2605.27023#S3.E13)\)

30

31

32

33Calculate the loss via Formula \([19](https://arxiv.org/html/2605.27023#S3.E19)\) based on

ℬ\+\\mathcal\{B\}^\{\+\}and its corresponding negative triple set

ℬ−=⋃\(h,r,t\)∈ℬ\+ℬ\(h,r,t\)−\\mathcal\{B\}^\{\-\}=\\bigcup\\limits\_\{\(h,r,t\)\\in\\mathcal\{B\}^\{\+\}\}\\mathcal\{B\}^\{\-\}\_\{\(h,r,t\)\}
34Update

θr\\theta\_\{r\},

θe\\theta\_\{e\}, and

ω\\omega

Algorithm 1KMAS
### 3\.7\.Complexity Analysis

Here, we analyze the time and memory complexity of KMAS\.\|𝒱\|\|\\mathcal\{V\}\|and\|ℛ\|\|\\mathcal\{R\}\|denote the numbers of entities and relations, respectively\.\|𝒯\|\|\\mathcal\{T\}\|denotes the number of triples in the KG\.LLdenotes the number of GNN layers\.\|ℬ\+\|\\mathcal\{\|B^\{\+\}\|\}denotes the batch size\.d¯r​e​l\\bar\{d\}\_\{rel\}denotes the average relation degree\.

The time complexity of KMAS is analyzed as follows\. The time complexity of the relation similarity acquisition \(Section[3\.2](https://arxiv.org/html/2605.27023#S3.SS2)\) is𝒪​\(\|ℬ\+\|⋅\|ℛ\|⋅d\)\\mathcal\{O\}\(\|\\mathcal\{B\}^\{\+\}\|\\cdot\|\\mathcal\{R\}\|\\cdot d\)\(line 13 \- 14 in Algorithm[1](https://arxiv.org/html/2605.27023#algorithm1)\)\. The time complexity of the relation\-aware entity distribution generation \(Section[3\.3](https://arxiv.org/html/2605.27023#S3.SS3)\) is𝒪​\(\|ℬ\+\|⋅\(\|ℛ\|​\(log⁡\|ℛ\|\+d¯r​e​l\)\+\|𝒱\|\)\)\\mathcal\{O\}\(\|\\mathcal\{B\}^\{\+\}\|\\cdot\(\|\\mathcal\{R\}\|\(\\log\|\\mathcal\{R\}\|\+\\bar\{d\}\_\{rel\}\)\+\\mathcal\{\|V\|\}\)\)\(line 15 \- 18 in Algorithm[1](https://arxiv.org/html/2605.27023#algorithm1)\)\. The time complexity of the hybrid sampling strategy \(Section[3\.4](https://arxiv.org/html/2605.27023#S3.SS4)\) is𝒪​\(\|ℬ\+\|⋅N\)\\mathcal\{O\}\(\\mathcal\{\|B^\{\+\}\|\}\\cdot N\)\(line 19 \- 25 in Algorithm[1](https://arxiv.org/html/2605.27023#algorithm1)\)\. Consequently, the total time complexity for training is𝒪​\(Tt​o​t​a​l⋅\|ℬ\+\|⋅\(\|ℛ\|​\(d\+log⁡\|ℛ\|\+d¯r​e​l\)\+\|𝒱\|\+N\)\)\\mathcal\{O\}\(T\_\{total\}\\cdot\\mathcal\{\|B^\{\+\}\|\}\\cdot\(\|\\mathcal\{R\}\|\(d\+\\log\|\\mathcal\{R\}\|\+\\bar\{d\}\_\{rel\}\)\+\|\\mathcal\{V\}\|\+N\)\)\. Considering\|ℛ\|\|\\mathcal\{R\}\|is usually small and\|ℬ\+\|\\mathcal\{\|B^\{\+\}\|\}/d¯r​e​l\\bar\{d\}\_\{rel\}/Tt​o​t​a​lT\_\{total\}/NNis a constant, therefore the time complexity of KMAS is linear to\|𝒱\|\|\\mathcal\{V\}\|\. The time complexity of the existing base KGFMs is dominated by\|𝒯\|\\mathcal\{\|T\|\}\. Compared with\|𝒯\|\\mathcal\{\|T\|\},\|𝒱\|\|\\mathcal\{V\}\|is small, indicating a marginal overhead generated by KMAS, which is verified in Section[4\.3](https://arxiv.org/html/2605.27023#S4.SS3)\.

The memory complexity of KMAS is analyzed as follows\. The memory complexity of the relation similarity acquisition \(Section[3\.2](https://arxiv.org/html/2605.27023#S3.SS2)\) is𝒪​\(\|ℬ\+\|⋅\|ℛ\|\)\\mathcal\{O\}\(\|\\mathcal\{B^\{\+\}\}\|\\cdot\|\\mathcal\{R\}\|\)\(line 13 \- 14 in Algorithm[1](https://arxiv.org/html/2605.27023#algorithm1)\)\. The memory complexity of the relation\-aware entity distribution generation \(Section[3\.3](https://arxiv.org/html/2605.27023#S3.SS3)\) is𝒪​\(\|ℬ\+\|⋅\|𝒱\|\)\\mathcal\{O\}\(\|\\mathcal\{B^\{\+\}\}\|\\cdot\|\\mathcal\{V\}\|\)\(line 15 \- 18 in Algorithm[1](https://arxiv.org/html/2605.27023#algorithm1)\)\. The memory complexity of the hybrid negative sampling \(Section[3\.4](https://arxiv.org/html/2605.27023#S3.SS4)\) is𝒪​\(\|ℬ\+\|⋅N\)\\mathcal\{O\}\(\\mathcal\{\|B^\{\+\}\|\}\\cdot N\)\(line 19 \- 25 in Algorithm[1](https://arxiv.org/html/2605.27023#algorithm1)\)\. Consequently, the total memory complexity of our method is𝒪​\(\|ℬ\+\|​\(\|ℛ\|\+\|𝒱\|\+N\)\)\.\\mathcal\{O\}\(\|\\mathcal\{B^\{\+\}\}\|\(\|\\mathcal\{R\}\|\+\|\\mathcal\{V\}\|\+N\)\)\.Considering\|ℛ\|\\mathcal\{\|R\|\}is usually small andNNis a constant, therefore the memory complexity of KMAS is linear to\|ℬ\+\|⋅\|𝒱\|\\mathcal\{\|B^\{\+\}\|\}\\cdot\\mathcal\{\|V\|\}\. The base KGFMs need to store intermediate node embeddings across allLLlayers to compute gradients during backpropagation, with a memory complexity of𝒪​\(\|ℬ\+\|⋅\|𝒱\|⋅d⋅L\)\\mathcal\{O\}\(\\mathcal\{\|B^\{\+\}\|\}\\cdot\|\\mathcal\{V\}\|\\cdot d\\cdot L\)\. Compared with\|ℬ\+\|⋅\|𝒱\|⋅d⋅L\\mathcal\{\|B^\{\+\}\|\}\\cdot\|\\mathcal\{V\}\|\\cdot d\\cdot L,\|ℬ\+\|⋅\|𝒱\|\\mathcal\{\|B^\{\+\}\|\}\\cdot\\mathcal\{\|V\|\}is small, indicating a marginal overhead generated by KMAS, which is verified in Section[4\.4](https://arxiv.org/html/2605.27023#S4.SS4)\.

## 4\.Experiments

### 4\.1\.Experimental Setting

#### 4\.1\.1\.Data Sets

We utilize a unified pre\-training corpus containing33common KGs to train KGFMs: FB15k\-237\(Toutanova and Chen,[2015](https://arxiv.org/html/2605.27023#bib.bib29)\), WN18RR\(Dettmerset al\.,[2018](https://arxiv.org/html/2605.27023#bib.bib31)\), and CoDEx\-Medium\(Safavi and Koutra,[2020](https://arxiv.org/html/2605.27023#bib.bib32)\)\. This combination covers general\-world knowledge and diverse Wikipedia domains, enabling the KGFM to learn general structural patterns\. We perform extensive experiments on4444data sets that are categorized into transductive, partially inductive, and fully inductive settings to evaluate the generalizability of our proposed method\. The definitions of these settings have been introduced in Section[2](https://arxiv.org/html/2605.27023#S2)\. More detailed information of the data sets used in our experiments can be found in Appendix[A](https://arxiv.org/html/2605.27023#A1)\.

Table 1\.Performance on the task of link prediction under different settings \(i\.e\., inductive \(e, r\), inductive \(e\), transductive\) over 44 data sets\. Since SCR’s source codes are not publicly available, we obtain the results of SCR from\(Wanget al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib36)\)\. The performance of other KGFMs \(i\.e\., ULTRA, TRIX, MOTIF, and SEMMA\) is reproduced via their open\-source solutions\.ModelInductive\(e, r\)Inductive\(e\)TransductiveTotal Avg\(19 graphs\)\(16 graphs\)\(9 graphs\)\(44 graphs\)MRRHits@10MRRHits@10MRRHits@10MRRHits@10SCR \(NIPS’25\)\(Wanget al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib36)\)0\.3200\.4980\.4350\.5920\.2980\.4470\.3580\.523ULTRA \(ICLR’24\)\(Galkinet al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib3)\)0\.3170\.4990\.4290\.5780\.3310\.4830\.3600\.524\\rowcolorblue\!10 ULTRA \+ KMAS0\.3260\.5080\.4340\.5840\.3370\.4900\.3670\.532TRIX \(LoG’24\)\(Zhanget al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib8)\)0\.3370\.5230\.4450\.6010\.3040\.4670\.3700\.540\\rowcolorblue\!10 TRIX \+ KMAS0\.3400\.5230\.4540\.6010\.3070\.4710\.3750\.541MOTIF \(ICML’25\)\(Huanget al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib6)\)0\.3290\.4980\.4330\.5820\.3240\.4770\.3600\.516\\rowcolorblue\!10 MOTIF \+ KMAS0\.3330\.5080\.4370\.5880\.3330\.4860\.3650\.524SEMMA \(EMNLP’25\)\(Arunet al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib4)\)0\.3310\.5020\.4470\.5900\.3270\.4790\.3720\.529\\rowcolorblue\!10 SEMMA \+ KMAS0\.3410\.5170\.4510\.5930\.3360\.4870\.3800\.539

#### 4\.1\.2\.Evaluation Metrics

We adopt the same evaluation metrics, mean reciprocal rank \(MRR\) and Hits@10, as previous KGFM studies\(Galkinet al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib3); Zhanget al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib8); Huanget al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib6); Arunet al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib4)\)to evaluate the performance of KGFMs and KGFMs using KMAS as the negative sampling method\. Both metrics are reported under the filtered ranking protocol\(Bordeset al\.,[2013](https://arxiv.org/html/2605.27023#bib.bib35)\)\.

#### 4\.1\.3\.Base KGFMs

To demonstrate the effectiveness of KMAS, we perform KMAS on many SOTA KGFMs \(i\.e\., ULTRA, TRIX, MOTIF, and SEMMA\), which are introduced in detail as follows\.

ULTRA\(Galkinet al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib3)\)constructs the relation graph, where nodes denote relations in the KG and edges denote four fundamental interactions between relations in the KG:*h2h*,*t2t*,*h2t*, and*t2h*\. An edge between relationrir\_\{i\}andrjr\_\{j\}is established in𝒢ℛ\\mathcal\{G\}\_\{\\mathcal\{R\}\}if there exist triples\(hi,ri,ti\)\(h\_\{i\},r\_\{i\},t\_\{i\}\)and\(hj,rj,tj\)∈𝒯\(h\_\{j\},r\_\{j\},t\_\{j\}\)\\in\\mathcal\{T\}satisfying one of the following conditions:\(1\)*h2h*:hi=hjh\_\{i\}=h\_\{j\}; \(2\)*t2t*:ti=tjt\_\{i\}=t\_\{j\}; \(3\)*h2t*:hi=tjh\_\{i\}=t\_\{j\}; \(4\)*t2h*:ti=hjt\_\{i\}=h\_\{j\}\. Specifically, the four fundamental interactions are learnable embeddings, which form part ofθr\\theta\_\{r\}\.Encoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}works on the relation graph to generate relation embeddings\.Encoderθe\\text\{Encoder\}\_\{\\theta\_\{e\}\}uses these relation embeddings to generate entity embeddings forfωf\_\{\\omega\}to predict entities\.

TRIX\(Zhanget al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib8)\)constructs four relation graphs, where nodes denote relations in the KG and edges denote specific entities that bridge two relations through one of four fundamental interactions:*h2h*,*t2t*,*h2t*, and*t2h*\.Encoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}of TRIX captures relation interactions through this entity\-aware edge information\.Encoderθe\\text\{Encoder\}\_\{\\theta\_\{e\}\}andfωf\_\{\\omega\}of TRIX adopt the same mechanism as ULTRA\. Specially, TRIX adopts an iterative architecture, stacking modules in the sequence:Encoderθr1→Encoderθe1→Encoderθr2→Encoderθe2→…→Encoderθrn→Encoderθen\\text\{Encoder\}\_\{\\theta\_\{r\_\{1\}\}\}\\rightarrow\\text\{Encoder\}\_\{\\theta\_\{e\_\{1\}\}\}\\rightarrow\\text\{Encoder\}\_\{\\theta\_\{r\_\{2\}\}\}\\rightarrow\\text\{Encoder\}\_\{\\theta\_\{e\_\{2\}\}\}\\rightarrow\\ldots\\rightarrow\\text\{Encoder\}\_\{\\theta\_\{r\_\{n\}\}\}\\rightarrow\\text\{Encoder\}\_\{\\theta\_\{e\_\{n\}\}\}\. Note thatEncoderθr1,Encoderθr2,…,Encoderθrn\\text\{Encoder\}\_\{\\theta\_\{r\_\{1\}\}\},\\text\{Encoder\}\_\{\\theta\_\{r\_\{2\}\}\},\\ldots,\\\\ \\text\{Encoder\}\_\{\\theta\_\{r\_\{n\}\}\}work on the same four relation graphs andEncoderθe1,Encoderθe2,…,Encoderθen\\text\{Encoder\}\_\{\\theta\_\{e\_\{1\}\}\},\\\\ \\text\{Encoder\}\_\{\\theta\_\{e\_\{2\}\}\},\\ldots,\\text\{Encoder\}\_\{\\theta\_\{e\_\{n\}\}\}work on the same KG\.

MOTIF\(Huanget al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib6)\)constructs the relation hypergraph, where hyperedges are matches of arbitrary graph motifs \(i\.e\., 2\-path and 3\-path motifs\) from the KG, and nodes are relations in the KG\.Encoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}of MOTIF works on the hypergraph, and the arbitrary graph motifs are the learnable embeddings that are part ofθr\\theta\_\{r\}forEncoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}\.Encoderθe\\text\{Encoder\}\_\{\\theta\_\{e\}\}andfωf\_\{\\omega\}of MOTIF adopt the same mechanism as ULTRA\.

SEMMA\(Arunet al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib4)\)constructs a structural relation graph and a textual relation graph\. For the textual relation graph, nodes denote relations, and weighted edges between them denote the cosine similarity of their textual embeddings\.Encoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}of SEMMA works on the two relation graphs in parallel and uses an MLP to integrate their information and obtain the final relation embeddings\.Encoderθe\\text\{Encoder\}\_\{\\theta\_\{e\}\}andfωf\_\{\\omega\}of SEMMA adopt the same mechanism as ULTRA\.

![Refer to caption](https://arxiv.org/html/2605.27023v1/x2.png)Figure 2\.Training time cost comparison of KGFMs \(i\.e, ULTRA, TRIX, MOTIF, and SEMMA\) and these KGFMs using KMAS as the negative sampling method\.
#### 4\.1\.4\.Implementation Details

We adopt the official implementation of44KGFMs \(i\.e, ULTRA, TRIX, MOTIF, and SEMMA\) as base KGFMs\. HyperparametersNN,qq,τ\\tau,τa​d​v\\tau\_\{adv\},βw​a​r​m\\beta\_\{warm\},γp​e​a​k\\gamma\_\{peak\}, andαm​a​x\\alpha\_\{max\}are set to512512,0\.40\.4,0\.80\.8,11,0\.250\.25,0\.250\.25and0\.40\.4for all KGFMs with KMAS as the negative sampling method, respectively\.Tt​o​t​a​lT\_\{total\}is set to800000800000for ULTRA and SEMMA and1000010000for TRIX and MOTIF\. All experiments are conducted on NVIDIA 4080 GPUs with the same version of PyTorch and PyG\. We make the data sets and source code available for future research111https://anonymous\.4open\.science/r/KGFMs\-8C8B\.

### 4\.2\.Effectiveness Study

We utilizeEncoderθr\\text\{Encoder\}\_\{\\theta\_\{r\}\}to generate relation embeddings for the construction of hard negative samples in ULTRA, TRIX, MOTIF, and SEMMA\. Specifically, for TRIX, we useEncoderθr1\\text\{Encoder\}\_\{\\theta\_\{r\_\{1\}\}\}to generate relation embeddings to construct hard negative samples\. Additionally, for SCR, the relation graph is enhanced by deriving both global and query\-aware relation embeddings and injecting them to guide semantic\-conditioned message passing on the KG\. Note that SCR does not open\-source their code, so we show the results from the original paper\(Wanget al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib36)\)\.

![Refer to caption](https://arxiv.org/html/2605.27023v1/x3.png)Figure 3\.Memory cost comparison of KGFMs \(i\.e, ULTRA, TRIX, MOTIF, and SEMMA\) and these KGFMs using KMAS as the negative sampling method\.Table 2\.Performance of different variants of KMAS with Ultra/SEMMA as the base KGFM for analysis of the hard negative ratio adjustment strategy under different settings \(i\.e\., inductive \(e, r\), inductive \(e\), transductive\) over 44 data sets\.From the experimental results shown in Table[1](https://arxiv.org/html/2605.27023#S4.T1), it can be seen that our proposed method KMAS can enhance various SOTA KGFMs \(i\.e\., ULTRA, TRIX, MOTIF, and SEMMA\) under different settings \(i\.e\., inductive \(e, r\), inductive \(e\), and transductive\) in terms of MRR and Hits@10, demonstrating the effectiveness of KMAS\. We report the detailed results in Appendix[B](https://arxiv.org/html/2605.27023#A2)\.

### 4\.3\.Efficiency Study

To evaluate the efficiency of KMAS, we test the training time for KMAS combined with different KGFMs on the same training data set \(i\.e\., FB15k\-237, WN18RR, and CoDEx\-Medium\)\. Since KMAS only performs in the training process, we do not show the inference time\. The experimental results plotted in Figure[2](https://arxiv.org/html/2605.27023#S4.F2)show that KMAS brings a very slight overhead to training time cost across different KGFMs \(i\.e\., ULTRA, TRIX, MOTIF, and SEMMA\), demonstrating the efficiency of KMAS\. These results are consistent with the time complexity analysis in Section[3\.7](https://arxiv.org/html/2605.27023#S3.SS7)\.

### 4\.4\.Memory Cost Study

To evaluate the training memory cost of KMAS, we test the memory cost for KMAS combined with different KGFMs on the same training data set \(i\.e\., FB15k\-237, WN18RR, and CoDEx\-Medium\) by tracking the peak memory usage via thenvidia\-smicommand\. As plotted in Figure[3](https://arxiv.org/html/2605.27023#S4.F3), the experimental results show that KMAS does not incur significant additional memory overhead across different KGFMs \(i\.e\., ULTRA, TRIX, MOTIF, and SEMMA\), demonstrating that KMAS is a memory\-efficient method\. These results are consistent with the memory complexity analysis in Section[3\.7](https://arxiv.org/html/2605.27023#S3.SS7)\.

### 4\.5\.Effect Analysis of Hard Negative Ratio Adjustment

To evaluate the effect of hard negative ratio adjustment strategy, we define33different variants of KMAS: \(1\)KMASh​o​l​d\\text\{KMAS\}\_\{hold\}\(i\.e\., KMAS using the hard negative ratioαm​a​x\\alpha\_\{max\}asαm\\alpha\_\{m\}for the whole training process\); \(2\)KMASi​n​c​r​e​a​s​e\\text\{KMAS\}\_\{increase\}\(i\.e\., KMAS increasing the hard negative ratioαm\\alpha\_\{m\}from0toαm​a​x\\alpha\_\{max\}without decreasing\); \(3\)KMASd​e​c​r​e​a​s​e\\text\{KMAS\}\_\{decrease\}\(i\.e\., KMAS decreasingαm\\alpha\_\{m\}fromαm​a​x\\alpha\_\{max\}to0without increasing\)\. From the experimental results shown in Table[2](https://arxiv.org/html/2605.27023#S4.T2), we can see that KMAS outperforms these three variants \(i\.e\.,KMASh​o​l​d\\text\{KMAS\}\_\{hold\},KMASi​n​c​r​e​a​s​e\\text\{KMAS\}\_\{increase\}, andKMASd​e​c​r​e​a​s​e\\text\{KMAS\}\_\{decrease\}\) in terms of MRR and Hits@10, whether ULTRA or SEMMA is used as the base KGFM, demonstrating the superiority of our hard negative ratio adjustment strategy\.

### 4\.6\.Parameter Study

To investigate the robustness of KMAS, we conduct a parameter analysis to understand the impact ofqq,γp​e​a​k\\gamma\_\{peak\}, andαm​a​x\\alpha\_\{max\}on the performance of SEMMA and KMAS using SEMMA as the base KGFM\. Experimental results shown in Figure[4](https://arxiv.org/html/2605.27023#S4.F4)show that the performance of KMAS using SEMMA as the KGFM outperforms SEMMA itself under different settings \(i\.e\., inductive \(e\) and transductive\) for the different ranges of the three parameters in terms of MRR and Hits@10, demonstrating the robustness of KMAS\.

![Refer to caption](https://arxiv.org/html/2605.27023v1/x4.png)Parameter sensitivity analysis for top\-p, gamma peak, and alpha max

Figure 4\.Parameter study results ofqq,γpeak\\gamma\_\{\\text\{peak\}\}, andαmax\\alpha\_\{\\text\{max\}\}\.

## 5\.Related Work

Three aspects of studies are related to our work: \(1\) knowledge graph foundation model; \(2\) negative sampling; \(3\) curriculum learning\. We will introduce them in detail as follows\.

Knowledge graph foundation models \(KGFMs\) have emerged as a paradigm for enabling zero\-shot reasoning over unseen KGs by learning invariance of the relational structure across diverse KGs\. ULTRA\(Galkinet al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib3)\)pioneered KGFM through the construction of a relation graph𝒢R\\mathcal\{G\}\_\{R\}that edges are four fundamental interaction\(i\.e\.,*h2h*,*t2h*,*h2t*,*t2t*\), enabling zero\-shot inference on unseen entities and relations\. TRIX\(Zhanget al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib8)\)enhances ULTRA by using entities as edges bridging relation pairs in its relation graphs and introduces an iterative update schema for joint representation refinement\. MOTIF\(Huanget al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib6)\)provides a theoretical framework formalizing KGFMs based on graph motifs through three steps:Lift,Relation encoder, andEntityEncoder\. It reveals ULTRA’s equivalence to binary 2\-path motifs and finds that new motifs increase expressiveness only when not covered by core\-onto homomorphisms\. SEMMA\(Arunet al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib4)\)extends KGFMs by combining structural and textual signals\. SEMMA obtains textual information via a text relation graph\. The outputs from the parallel structural and textual modules are fused via an MLP to generate the final relation embeddings, empowering robust zero\-shot link prediction across diverse KGs\. SCR\(Wanget al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib36)\)improves KGFMs by bridging the gap between structural patterns and node semantic features, while unifying diverse graph tasks into a single reasoning framework\. KG\-ICL\(Zhaoet al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib1)\)enables universal transfer across diverse KGs by employing a unified tokenizer that maps different entities/relations to the shared tokens based on their relative structural roles and query identity\. This mechanism does not rely on the relation graph and processes query\-specific prompt graphs, which comprise example facts and their local neighbor and path contexts\. Despite significant progress that has been made in KGFMs, the negative sample quality issue is ignored\. All KGFMs above rely on corrupting either the head or tail entity of the positive sample to obtain negative samples where negative triples are independent of the invariance of the relation structure\. Our work focuses on boosting KGFMs, which are based on relation graphs via an effective negative sampling method\.

The negative sampling method serves as an effective strategy applied in many tasks, such as KG\-based recommendation\(Yanget al\.,[2022](https://arxiv.org/html/2605.27023#bib.bib38)\), KG completion\(Wanget al\.,[2022](https://arxiv.org/html/2605.27023#bib.bib46)\), and KG embedding\(Linet al\.,[2023](https://arxiv.org/html/2605.27023#bib.bib40); Kanojiaet al\.,[2017](https://arxiv.org/html/2605.27023#bib.bib41)\)\. In this paper, we dynamically construct hard negative samples for KGFMs through a hybrid negative sampling strategy, enhancing existing KGFMs\.

Curriculum learning\(Bengioet al\.,[2009](https://arxiv.org/html/2605.27023#bib.bib42)\)functions as a sophisticated training paradigm that organizes training examples from easy to hard\. Its core idea is applied across various tasks, e\.g\., complex reasoning over KG\(Xiaet al\.,[2025](https://arxiv.org/html/2605.27023#bib.bib43); Zhanget al\.,[2019](https://arxiv.org/html/2605.27023#bib.bib45)\), KG embedding\(Sunet al\.,[2018](https://arxiv.org/html/2605.27023#bib.bib44); Guoet al\.,[2018](https://arxiv.org/html/2605.27023#bib.bib47)\), and joint open knowledge graph canonicalization and linking\(Shenet al\.,[2024](https://arxiv.org/html/2605.27023#bib.bib48)\)\. Specially, we propose a hard negative ratio adjustment strategy inspired by the trade\-off policy, which combines the strategy of “easier first” as curriculum learning and “harder first” as hard example mining\(Wanget al\.,[2021](https://arxiv.org/html/2605.27023#bib.bib34)\)\.

## 6\.Conclusion and Future Work

To enhance existing KGFMs, we propose a simple yet effective adaptive negative sampling method to generate hard negative samples whose ratio is dynamically adjusted to train KGFMs\. KMAS is flexible to adapt to many KGFMs\. Extensive experiments are conducted on 44 data sets\. Experimental results have demonstrated that KMAS can improve the performance of many SOTA KGFMs without requiring excessive additional time or memory consumption\. In the future, we will evaluate the performance of KMAS on more KGFMs and data sets\.

## References

- A\. Arun, S\. Kumar, M\. Nayyeri, B\. Xiong, P\. Kumaraguru, A\. Vergari, and S\. Staab \(2025\)SEMMA: a semantic aware knowledge graph foundation model\.InEMNLP,pp\. 31825–31848\.Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p2.1),[§3\.6](https://arxiv.org/html/2605.27023#S3.SS6.p5.4),[§4\.1\.2](https://arxiv.org/html/2605.27023#S4.SS1.SSS2.p1.1),[§4\.1\.3](https://arxiv.org/html/2605.27023#S4.SS1.SSS3.p5.3),[Table 1](https://arxiv.org/html/2605.27023#S4.T1.7.1.1.1.11.11.1),[§5](https://arxiv.org/html/2605.27023#S5.p2.1)\.
- Y\. Bengio, J\. Louradour, R\. Collobert, and J\. Weston \(2009\)Curriculum learning\.InICML,pp\. 41–48\.Cited by:[§5](https://arxiv.org/html/2605.27023#S5.p4.1)\.
- A\. Bordes, N\. Usunier, A\. Garcia\-Duran, J\. Weston, and O\. Yakhnenko \(2013\)Translating embeddings for modeling multi\-relational data\.InNIPS,pp\. 2787–2795\.Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p1.1),[§4\.1\.2](https://arxiv.org/html/2605.27023#S4.SS1.SSS2.p1.1)\.
- T\. Dettmers, P\. Minervini, P\. Stenetorp, and S\. Riedel \(2018\)Convolutional 2d knowledge graph embeddings\.InAAAI,pp\. 1811–1818\.Cited by:[§4\.1\.1](https://arxiv.org/html/2605.27023#S4.SS1.SSS1.p1.2)\.
- M\. Galkin, X\. Yuan, H\. Mostafa, J\. Tang, and Z\. Zhu \(2024\)Towards foundation models for knowledge graph reasoning\.InICLR,Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p1.1),[§1](https://arxiv.org/html/2605.27023#S1.p2.1),[§3\.6](https://arxiv.org/html/2605.27023#S3.SS6.p5.4),[§4\.1\.2](https://arxiv.org/html/2605.27023#S4.SS1.SSS2.p1.1),[§4\.1\.3](https://arxiv.org/html/2605.27023#S4.SS1.SSS3.p2.13),[Table 1](https://arxiv.org/html/2605.27023#S4.T1.7.1.1.1.5.5.1),[§5](https://arxiv.org/html/2605.27023#S5.p2.1)\.
- S\. Guo, Q\. Wang, L\. Wang, B\. Wang, and L\. Guo \(2018\)Knowledge graph embedding with iterative guidance from soft rules\.InAAAI,pp\. 4816–4823\.Cited by:[§5](https://arxiv.org/html/2605.27023#S5.p4.1)\.
- X\. Huang, P\. Barceló, M\. M\. Bronstein, İ\. İ\. Ceylan, M\. Galkin, J\. L\. Reutter, and M\. R\. Orth \(2025\)How expressive are knowledge graph foundation models?\.InICML,Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p2.1),[§3\.6](https://arxiv.org/html/2605.27023#S3.SS6.p5.4),[§4\.1\.2](https://arxiv.org/html/2605.27023#S4.SS1.SSS2.p1.1),[§4\.1\.3](https://arxiv.org/html/2605.27023#S4.SS1.SSS3.p4.5),[Table 1](https://arxiv.org/html/2605.27023#S4.T1.7.1.1.1.9.9.1),[§5](https://arxiv.org/html/2605.27023#S5.p2.1)\.
- V\. Kanojia, H\. Maeda, R\. Togashi, and S\. Fujita \(2017\)Enhancing knowledge graph embedding with probabilistic negative sampling\.InWWW,pp\. 801–802\.Cited by:[§5](https://arxiv.org/html/2605.27023#S5.p3.1)\.
- J\. Lee, C\. Chung, and J\. J\. Whang \(2023\)INGRAM: inductive knowledge graph embedding via relation graphs\.InICML,Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p1.1)\.
- Z\. Lin, Z\. Zhao, J\. Xie, and Y\. Shen \(2023\)Hierarchical type enhanced negative sampling for knowledge graph embedding\.InSIGIR,pp\. 2047–2051\.Cited by:[§5](https://arxiv.org/html/2605.27023#S5.p3.1)\.
- S\. Liu, B\. C\. Grau, I\. Horrocks, and E\. V\. Kostylev \(2021\)INDIGO: gnn\-based inductive knowledge graph completion using pair\-wise encoding\.InNIPS,pp\. 2034–2045\.Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p1.1)\.
- S\. Pan, L\. Luo, Y\. Wang, C\. Chen, J\. Wang, and X\. Wu \(2024\)Unifying large language models and knowledge graphs: a roadmap\.IEEE Transactions on Knowledge and Data Engineering36\(7\),pp\. 3580–3599\.Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p1.1)\.
- A\. Rossi, D\. Barbosa, D\. Firmani, A\. Matinata, and P\. Merialdo \(2021\)Knowledge graph embedding for link prediction: a comparative analysis\.ACM Transactions on Knowledge Discovery from Data15\(2\)\.Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p1.1)\.
- A\. Rossi, D\. Firmani, P\. Merialdo, and T\. Teofili \(2022\)Explaining link prediction systems based on knowledge graph embeddings\.InSIGMOD,pp\. 2062–2075\.Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p1.1)\.
- T\. Safavi and D\. Koutra \(2020\)CoDEx: a Comprehensive Knowledge Graph Completion Benchmark\.InEMNLP,pp\. 8328–8350\.Cited by:[§4\.1\.1](https://arxiv.org/html/2605.27023#S4.SS1.SSS1.p1.2)\.
- W\. Shen, B\. Yang, and Y\. Liu \(2024\)Jointly canonicalizing and linking open knowledge base via unified embedding learning\.InWWW,pp\. 2304–2314\.Cited by:[§5](https://arxiv.org/html/2605.27023#S5.p4.1)\.
- Z\. Sun, W\. Hu, Q\. Zhang, and Y\. Qu \(2018\)Bootstrapping entity alignment with knowledge graph embedding\.InIJCAI,pp\. 4396–4402\.Cited by:[§5](https://arxiv.org/html/2605.27023#S5.p4.1)\.
- Z\. Sun, Z\. Deng, J\. Nie, and J\. Tang \(2019\)RotatE: knowledge graph embedding by relational rotation in complex space\.InICLR,Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p1.1)\.
- K\. Toutanova and D\. Chen \(2015\)Observed versus latent features for knowledge base and text inference\.InProceedings of the 3rd workshop on continuous vector space models and their compositionality,pp\. 57–66\.Cited by:[§4\.1\.1](https://arxiv.org/html/2605.27023#S4.SS1.SSS1.p1.2)\.
- K\. Wang, S\. Luo, C\. Shan, and Y\. Shen \(2025\)Towards graph foundation models: training on knowledge graphs enables transferability to general graphs\.InNIPS,Cited by:[§4\.2](https://arxiv.org/html/2605.27023#S4.SS2.p1.2),[Table 1](https://arxiv.org/html/2605.27023#S4.T1),[Table 1](https://arxiv.org/html/2605.27023#S4.T1.7.1.1.1.4.4.1),[§5](https://arxiv.org/html/2605.27023#S5.p2.1)\.
- L\. Wang, W\. Zhao, Z\. Wei, and J\. Liu \(2022\)SimKGC: simple contrastive knowledge graph completion with pre\-trained language models\.InACL,pp\. 4281–4294\.Cited by:[§5](https://arxiv.org/html/2605.27023#S5.p3.1)\.
- X\. Wang, X\. He, Y\. Cao, M\. Liu, and T\. Chua \(2019\)KGAT: knowledge graph attention network for recommendation\.InSIGKDD,pp\. 950–958\.Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p1.1)\.
- X\. Wang, Y\. Chen, and W\. Zhu \(2021\)A survey on curriculum learning\.IEEE Transactions on Pattern Analysis and Machine Intelligence14\(8\),pp\. 1–20\.Cited by:[§3\.5](https://arxiv.org/html/2605.27023#S3.SS5.p1.14),[§5](https://arxiv.org/html/2605.27023#S5.p4.1)\.
- T\. Xia, L\. Ding, G\. Wan, Y\. Zhan, B\. Du, and D\. Tao \(2025\)Improving complex reasoning over knowledge graph with logic\-aware curriculum tuning\.InAAAI,pp\. 12881–12889\.Cited by:[§5](https://arxiv.org/html/2605.27023#S5.p4.1)\.
- Y\. Yang, C\. Huang, L\. Xia, and C\. Li \(2022\)Knowledge graph contrastive learning for recommendation\.InSIGIR,pp\. 1434–1443\.Cited by:[§5](https://arxiv.org/html/2605.27023#S5.p3.1)\.
- W\. Zhang, B\. Paudel, L\. Wang, J\. Chen, H\. Zhu, W\. Zhang, A\. Bernstein, and H\. Chen \(2019\)Iteratively learning embeddings and rules for knowledge graph reasoning\.InWWW,pp\. 2366–2377\.Cited by:[§5](https://arxiv.org/html/2605.27023#S5.p4.1)\.
- Y\. Zhang, B\. Bevilacqua, M\. Galkin, and B\. Ribeiro \(2024\)TRIX: a more expressive model for zero\-shot domain transfer in knowledge graphs\.InLoG,Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p2.1),[§3\.6](https://arxiv.org/html/2605.27023#S3.SS6.p5.4),[§4\.1\.2](https://arxiv.org/html/2605.27023#S4.SS1.SSS2.p1.1),[§4\.1\.3](https://arxiv.org/html/2605.27023#S4.SS1.SSS3.p3.6),[Table 1](https://arxiv.org/html/2605.27023#S4.T1.7.1.1.1.7.7.1),[§5](https://arxiv.org/html/2605.27023#S5.p2.1)\.
- R\. Zhao, F\. Zhao, L\. Wang, X\. Wang, and G\. Xu \(2024\)KG\-CoT: chain\-of\-thought prompting of large language models over knowledge graphs for knowledge\-aware question answering\.InIJCAI,pp\. 6642–6650\.Cited by:[§1](https://arxiv.org/html/2605.27023#S1.p1.1),[§5](https://arxiv.org/html/2605.27023#S5.p2.1)\.
- Z\. Zhu, Z\. Zhang, L\. Xhonneux, and J\. Tang \(2021\)Neural bellman\-ford networks: a general graph neural network framework for link prediction\.InNIPS,pp\. 29476–29490\.Cited by:[§2](https://arxiv.org/html/2605.27023#S2.p3.18)\.

## Appendix

## Appendix AData Sets

We train KGFMs on 3 KGs and inference on 44 data sets\. The statistics of 3 KGs for training in Table[3](https://arxiv.org/html/2605.27023#A2.T3)and 9 data sets of the transductive setting for inference are shown in Table[4](https://arxiv.org/html/2605.27023#A2.T4)\. The statistics of 16 data sets of the partially inductive setting \(i\.e\., inductive \(e\)\) for inference are shown in Table[5](https://arxiv.org/html/2605.27023#A2.T5)\. The statistics of 19 data sets of the fully inductive setting \(i\.e\., inductive \(e, r\)\) for inference are shown in Table[6](https://arxiv.org/html/2605.27023#A2.T6)\.

## Appendix BDetailed Results

We show detailed results of four KGFMs \(i\.e\., ULTRA, TRIX, MOTIF, and SEMMA\) and these KGFMs enhanced by KMAS over 44 data sets in Table[7](https://arxiv.org/html/2605.27023#A2.T7), Table[8](https://arxiv.org/html/2605.27023#A2.T8), Table[9](https://arxiv.org/html/2605.27023#A2.T9), and Table[10](https://arxiv.org/html/2605.27023#A2.T10)corresponding to the results in Table[1](https://arxiv.org/html/2605.27023#S4.T1)\.

Table 3\.3 KGs for training\.Edenotes the number of entities\.Rdenotes the number of relations\. Train, Valid, and Test denote the number of triples for training, validation, and test in the corresponding data set\. Task denotes the prediction task\. h/t means the task of predicting both head entities and tail entities\.Table 4\.9 data sets of the transductive setting for inference\. Task denotes the prediction task\. h/t means the task of predicting both head entities and tail entities, tails means the task of predicting the tail entity\.Table 5\.16 data sets of the partially inductive setting\. T denotes the number of triples in their corresponding graph\. Valid and Test denote the triples that need to be predicted in the validation and test data sets in their corresponding graph\. We predict both head entities and tail entities for these data sets\.Table 6\.19 data sets of the fully inductive setting\. We predict both head entities and tail entities for these data sets\.Table 7\.Detail results of the KGFM ULTRA and ULTRA using KMAS as the negative sampling method\.Data setULTRAULTRA \+ KMASMRRHits@10MRRHits@10TransductiveCoDExSmall0\.4730\.6700\.4760\.670CoDExLarge0\.3400\.4740\.3390\.471NELL9950\.4720\.5940\.4760\.605WDsinger0\.3510\.4800\.3870\.504NELL23k0\.2350\.4020\.2350\.408FB15k237\_100\.2350\.3800\.2400\.391FB15k237\_200\.2650\.4230\.2680\.431FB15k237\_500\.3230\.5190\.3240\.523Hetionet0\.2840\.4060\.2840\.406Inductive \(e\)WN18RRInductive:v10\.6580\.7760\.6520\.794WN18RRInductive:v20\.6580\.7720\.6570\.771WN18RRInductive:v30\.3730\.4960\.3830\.510WN18RRInductive:v40\.6050\.7140\.6050\.716FB15k237Inductive:v10\.4840\.6450\.4950\.647FB15k237Inductive:v20\.4860\.6840\.4910\.693FB15k237Inductive:v30\.4830\.6500\.4850\.646FB15k237Inductive:v40\.4830\.6720\.4820\.674NELLInductive:v20\.4990\.6960\.5360\.720NELLInductive:v30\.5210\.7060\.5300\.713NELLInductive:v40\.4770\.7110\.4950\.729ILPC2022:small0\.2920\.4440\.2930\.452ILPC2022:large0\.2830\.4200\.2950\.424HM:3k0\.0610\.1070\.0540\.109HM:5k0\.0540\.1020\.0500\.099HM:indigo0\.4400\.6490\.4360\.646Inductive \(e, r\)FBIngram:1000\.4390\.6270\.4390\.632FBIngram:750\.3940\.5950\.4000\.599FBIngram:500\.3290\.5320\.3330\.541FBIngram:250\.3870\.6340\.3920\.639WKIngram:1000\.1780\.2910\.1840\.302WKIngram:750\.3730\.5200\.3790\.536WKIngram:500\.1430\.2950\.1510\.308WKIngram:250\.2860\.4980\.2970\.512NLIngram:750\.3470\.5130\.3250\.502NLIngram:500\.3820\.5590\.3760\.532NLIngram:250\.3810\.5560\.3710\.558WikiTopicsMT1:tax0\.2230\.3080\.2330\.320WikiTopicsMT2:org0\.0880\.1530\.0900\.153WikiTopicsMT2:sci0\.2360\.3900\.2960\.428WikiTopicsMT3:art0\.2440\.4090\.2610\.422WikiTopicsMT3:infra0\.6400\.7790\.6360\.780WikiTopicsMT4:sci0\.2980\.4550\.2990\.460Metafam0\.1770\.7260\.2670\.788FBNELL0\.4790\.6460\.4820\.646Table 8\.Detail results of the KGFM TRIX and TRIX using KMAS as the negative sampling method\.Data setTRIXTRIX \+ KMASMRRHits@10MRRHits@10TransductiveCoDExSmall0\.3910\.6380\.3900\.637CoDExLarge0\.3170\.4460\.3100\.446NELL9950\.4320\.5880\.4720\.612WDsinger0\.3700\.4950\.3790\.501NELL23k0\.2280\.3880\.2240\.380FB15k237\_100\.2330\.3790\.2230\.369FB15k237\_200\.2590\.4150\.2580\.416FB15k237\_500\.3140\.5060\.3120\.509Hetionet0\.1910\.3520\.1950\.366Inductive \(e\)WN18RRInductive:v10\.6440\.7980\.6870\.804WN18RRInductive:v20\.6460\.7870\.6840\.785WN18RRInductive:v30\.4450\.5690\.4460\.569WN18RRInductive:v40\.5880\.7100\.6470\.727FB15k237Inductive:v10\.4840\.6460\.4890\.651FB15k237Inductive:v20\.4990\.7110\.5160\.709FB15k237Inductive:v30\.4860\.6580\.4920\.656FB15k237Inductive:v40\.4760\.6830\.4850\.676NELLInductive:v20\.5630\.7690\.5580\.761NELLInductive:v30\.5670\.7570\.5630\.749NELLInductive:v40\.5380\.7640\.5330\.765ILPC2022:small0\.3030\.4550\.3030\.453ILPC2022:large0\.3050\.4300\.3000\.427HM:3k0\.0720\.1260\.0660\.122HM:5k0\.0660\.1090\.0630\.110HM:indigo0\.4320\.6400\.4330\.644Inductive \(e, r\)FBIngram:1000\.4410\.6400\.4450\.642FBIngram:750\.3970\.6060\.3920\.595FBIngram:500\.3380\.5450\.3270\.540FBIngram:250\.3910\.6470\.3910\.644WKIngram:1000\.1840\.2940\.1840\.294WKIngram:750\.3840\.5240\.3760\.519WKIngram:500\.1700\.3050\.1600\.291WKIngram:250\.2990\.4850\.2960\.466NLIngram:750\.3480\.5210\.3470\.526NLIngram:500\.3910\.5820\.3820\.551NLIngram:250\.3890\.5890\.3680\.562WikiTopicsMT1:tax0\.2700\.4330\.2760\.445WikiTopicsMT2:org0\.0960\.1530\.0940\.151WikiTopicsMT2:sci0\.2790\.4330\.3280\.466WikiTopicsMT3:art0\.2960\.4500\.2960\.454WikiTopicsMT3:infra0\.6380\.7810\.6610\.793WikiTopicsMT4:sci0\.2940\.4510\.2920\.448Metafam0\.3160\.8230\.3770\.880FBNELL0\.4910\.6810\.4750\.670Table 9\.Detail results of the KGFM MOTIF and MOTIF using KMAS as the negative sampling method\.Data setMOTIFMOTIF \+ KMASMRRHits@10MRRHits@10TransductiveCoDExSmall0\.4720\.6680\.4700\.667CoDExLarge0\.3310\.4620\.3400\.471NELL9950\.4400\.5960\.4680\.616WDsinger0\.3720\.4850\.3790\.498NELL23k0\.2200\.3740\.2230\.389FB15k237\_100\.2440\.3900\.2440\.390FB15k237\_200\.2630\.4210\.2660\.424FB15k237\_500\.3130\.5100\.3180\.514Hetionet0\.2640\.3860\.2840\.406Inductive \(e\)WN18RRInductive:v10\.6630\.7770\.6860\.787WN18RRInductive:v20\.6630\.7690\.6580\.773WN18RRInductive:v30\.4300\.5650\.4250\.547WN18RRInductive:v40\.6290\.7050\.6300\.709FB15k237Inductive:v10\.4850\.6690\.4840\.681FB15k237Inductive:v20\.4940\.7190\.4940\.715FB15k237Inductive:v30\.4860\.6640\.4710\.663FB15k237Inductive:v40\.4800\.6730\.4780\.668NELLInductive:v20\.5270\.7240\.5320\.742NELLInductive:v30\.5040\.6780\.5210\.696NELLInductive:v40\.4750\.6850\.4710\.731ILPC2022:small0\.2860\.4390\.2920\.445ILPC2022:large0\.2740\.4160\.2690\.413HM:3k0\.0590\.1000\.1000\.107HM:5k0\.0500\.0870\.0560\.099HM:indigo0\.4280\.6380\.4310\.640Inductive \(e, r\)FBIngram:1000\.4370\.6290\.4280\.625FBIngram:750\.3940\.6010\.3890\.601FBIngram:500\.3320\.5280\.3360\.542FBIngram:250\.3800\.6260\.3880\.637WKIngram:1000\.1740\.2940\.1640\.288WKIngram:750\.3660\.5310\.3680\.518WKIngram:500\.1650\.3180\.1650\.311WKIngram:250\.3210\.5140\.3090\.506NLIngram:750\.3240\.4880\.3270\.498NLIngram:500\.3800\.5270\.3740\.538NLIngram:250\.3260\.4540\.3360\.519WikiTopicsMT1:tax0\.2440\.3420\.2580\.408WikiTopicsMT2:org0\.0940\.1530\.0900\.153WikiTopicsMT2:sci0\.2750\.4220\.2660\.428WikiTopicsMT3:art0\.2590\.4090\.2630\.412WikiTopicsMT3:infra0\.6200\.7640\.6380\.772WikiTopicsMT4:sci0\.2870\.4620\.2760\.429Metafam0\.4170\.7640\.4880\.807FBNELL0\.4630\.6330\.4680\.651Table 10\.Detail results of the KGFM SEMMA and SEMMA using KMAS as the negative sampling method\.Data setSEMMASEMMA \+ KMASMRRHits@10MRRHits@10TransductiveCoDExSmall0\.4890\.6760\.4840\.679CoDExLarge0\.3460\.4790\.3370\.473NELL9950\.4160\.5460\.4680\.604WDsinger0\.3900\.5030\.3950\.502NELL23k0\.2410\.4130\.2380\.412FB15k237\_100\.2330\.3850\.2410\.385FB15k237\_200\.2620\.4300\.2660\.427FB15k237\_500\.3260\.5240\.3290\.525Hetionet0\.2370\.3560\.2640\.375Inductive \(e\)WN18RRInductive:v10\.7090\.8060\.7020\.808WN18RRInductive:v20\.6950\.7950\.7020\.800WN18RRInductive:v30\.4450\.5910\.4450\.592WN18RRInductive:v40\.6580\.7390\.6540\.730FB15k237Inductive:v10\.4950\.6470\.5040\.656FB15k237Inductive:v20\.5130\.6960\.5080\.695FB15k237Inductive:v30\.4960\.6540\.5040\.658FB15k237Inductive:v40\.4980\.6740\.4960\.681NELLInductive:v20\.5260\.7110\.5310\.728NELLInductive:v30\.4970\.6910\.5250\.715NELLInductive:v40\.4800\.7260\.4920\.727ILPC2022:small0\.3030\.4550\.3010\.452ILPC2022:large0\.3030\.4260\.3040\.426HM:3k0\.0530\.0840\.0570\.094HM:5k0\.0510\.0960\.0510\.086HM:indigo0\.4300\.6430\.4330\.644Inductive \(e, r\)FBIngram:1000\.4540\.6430\.4520\.639FBIngram:750\.4030\.5980\.4090\.607FBIngram:500\.3420\.5480\.3410\.546FBIngram:250\.3990\.6440\.4000\.649WKIngram:1000\.1850\.2950\.1870\.303WKIngram:750\.3820\.5260\.3810\.515WKIngram:500\.1730\.3190\.1750\.322WKIngram:250\.3120\.4810\.3080\.489NLIngram:750\.3490\.5230\.3860\.528NLIngram:500\.3940\.5730\.3860\.550NLIngram:250\.3820\.5320\.3880\.573WikiTopicsMT1:tax0\.2330\.3100\.2520\.398WikiTopicsMT2:org0\.0950\.1580\.0950\.159WikiTopicsMT2:sci0\.2370\.3400\.2580\.398WikiTopicsMT3:art0\.2700\.4160\.2770\.423WikiTopicsMT3:infra0\.6340\.7690\.6430\.778WikiTopicsMT4:sci0\.2830\.4480\.2780\.447Metafam0\.3000\.7690\.4080\.856FBNELL0\.4640\.6440\.4850\.647