Pattern Over-Generalization of Knowledge Graph Embedding

# Pattern Over-Generalization of Knowledge Graph Embedding
Source: [https://arxiv.org/html/2609.03487](https://arxiv.org/html/2609.03487)
Kangil Kim††thanks:Corresponding author\.Affiliation:AI Graduate SchoolAffiliation:Gwangju Institute of Science and TechnologyAffiliation:junsikkim@gm\.gist\.ac\.kr,kangil\.kim\.01@gmail\.com

###### Abstract

Knowledge graph embedding \(KGE\) demonstrates its effectiveness for predicting missing links in knowledge graphs \(KGs\) by projecting entities and relations into a low\-dimensional vector space\. It is crucial for KGE models to effectively capture inference patterns \(patterns\) inherent in KGs, such as symmetry/antisymmetry, inversion and composition\. Although recent KGE models exhibit strong capabilities in modeling such diverse patterns, they suffer from inherent limitations stemming frompattern over\-generalization, where embeddings learned from only a single pattern instance inevitably generalize that pattern to all related instances, i\.e\., generalize the pattern universally\. To address this issue, we propose PogRE \(Pattern Over\-Generalization Robust Embedding\), a simple but effective method that utilizes dense linear transformations and compound operations for relation representation\. Our theoretical analysis demonstrates that a dense linear transformation allows a pattern to become progressively universal as more triples are observed in the pattern\. Furthermore, after observingd\+1d\+1linearly independent entities \(d\+1d\+1denotes the dimension of entity\), the linear transformation guarantees universal generalization of the pattern across all related instances\. Experimental results on three standard benchmark datasets show that PogRE outperforms existing state\-of\-the\-art KGE models in link prediction\. Moreover, our empirical results indicate that PogRE effectively addresses the negative impact of over\-generalization\.

## 1Introduction

Knowledge graphs \(KGs\) store vast amounts of human knowledge in the form of triples\(h,r,t\)\(h,r,t\), wherehhandttrepresent the head and tail entities andrrdenotes the relationship between entities\. KGs have demonstrated their effectiveness in various downstream taskssui2025fidelis;ma\-etal\-2025\-large\-language\-models\-meet\. However, real\-world KGs such as Freebasebollacker2008freebaseand WordNetmiller1995wordnet, even on a large scale, still suffer from incompletenessbordes2013translating\. To address this issue, Knowledge graph embedding \(KGE\), which represents entities and relations in a low\-dimensional vector space, has been widely studied as an effective method for predicting missing links\.

A fundamental challenge in KGE lies in effectively capturing the inference patterns \(patterns\) inherent in KGs, such as symmetry/antisymmetry, inversion, and composition\. To address this, existing works focus on designing specific score functions to capture these patterns\. For instance, TransEbordes2013translatingrepresents relations as translations to model inversion and composition, while RotatEsun2019rotateemploys rotations to capture symmetry/antisymmetry, inversion and composition\. PairREchao2021pairreand CompoundEge2023compoundingleverage scaling and compound operators to effectively model more patterns, including subrelation as well as complex relations\.

Despite their strong ability to capture various patterns, existing KGE models tend to over\-generalize the patterns they observe\. In particular, once a model observes a pattern, it generalizes the pattern universally across the entire graph, even when the pattern is supported by only a small number of observed triples\. Consequently, patterns that are valid locally in the graph are treated as universally valid\. We refer to this phenomenon as over\-generalization, which leads to erroneous predictions\.

To address this issue, we propose a simple but effective method, PogRE, that prevents locally valid patterns from being generalized universally\. PogRE uses dense linear transformations and compound operations for relation representation, where the linear transformation is decomposed into a relation\-specific orthogonal matrix and a shared upper\-triangular matrix\. This framework theoretically guarantees that patterns supported by only a small number of observed triples are generalized locally, while ensuring that any pattern supported by sufficient observed triples is generalized universally across the entire graph, when patterns are represented as connected relational paths\.

Our contributions are as follows:

- •We introduce pattern over\-generalization, the phenomenon in which patterns supported by only a small number of observed triples are generalized universally across the entire graph in existing KGE models\.
- •We propose a novel KGE method, Pattern Over\-Generalization Robust Embedding \(PogRE\), and theoretically guarantee that it effectively addresses over\-generalization\. In particular, PogRE allows any pattern represented as a connected relational path to become progressively universal as more triples are observed in the pattern\. Moreover, sufficient triples are observed, PogRE guarantees universal generalization of the pattern across all related instances\.
- •Experimental results on three benchmark datasets demonstrate that PogRE consistently outperforms baseline KGE models in link prediction and effectively addresses the negative impact of over\-generalization\.

## 2Background

#### Knowledge Graph Embedding

Given sets of entities and relationsEEandRR, a KG can be defined as a collection of factual triplesG=\{\(h,r,t\)\|h,t∈E,r∈R\}G=\\\{\(h,r,t\)\|h,t\\in E,r\\in R\\\}, wherehhandttare the head and tail entities, andrris the relation\. KGE mapsEEandRRto low\-dimensional vector space and defines a score function to measure triple plausibility\.

Distance\-based models \(DBMs\) are trained to minimize the distance of the factual triple\(h,r,t\)\(h,r,t\), while maximizing the distance of corrupted negative triples\(h′,r,t\)\(h^\{\\prime\},r,t\)or\(h,r,t′\)\(h,r,t^\{\\prime\}\), which are generated by randomly replacing the headhhor tailttwith other entities inEE\. PairREchao2021pairre, a representative model of DBMs, defines the score function as follows:

fr​\(h,t\)=‖h∘rH−t∘rT‖,\\displaystyle\\textstyle f\_\{r\}\(h,t\)=\\\|h\\circ r^\{H\}\-t\\circ r^\{T\}\\\|,\(1\)
whereh,t,rH,rT∈ℝdh,t,r^\{H\},r^\{T\}\\in\\mathbb\{R\}^\{d\},∘\\circdenotes a Hadamard product and∥⋅∥\\\|\\cdot\\\|is a vector norm\.

Tensor decomposition models \(TDMs\) are trained to maximize the score \(or semantic similarity\) of the factual triple calculated via the multi\-linear product of the head entityhh, the relationrrand the tail entitytt, while minimizing the score of negative triples\. DistMultyang2015embedding, a representative model of TDMs, defines the score function as follows:

fr​\(h,t\)=⟨h,r,t⟩=∑i=1dhi​ri​ti,\\displaystyle\\textstyle f\_\{r\}\(h,t\)=\\langle h,r,t\\rangle=\\sum\_\{i=1\}^\{d\}h\_\{i\}r\_\{i\}t\_\{i\},\(2\)whereh,r,t∈ℝdh,r,t\\in\\mathbb\{R\}^\{d\}, and⟨⋅,⋅,⋅⟩\\langle\\cdot,\\cdot,\\cdot\\rangledenotes the sum of element\-wise products\.

#### Inference Pattern

Inference patterns \(patterns\) are widely used to analyze the generalization capabilities of KGEs\. A pattern, notated asψ⇒ϕ\\psi\\Rightarrow\\phi, has the bodyψ\\psiand the headϕ\\phi, which are sets of triples composed of observed entities and relations in the data\. For example, a composition pattern for relationsr1,r2,r3∈Rr\_\{1\},r\_\{2\},r\_\{3\}\\in Ris defined asr1​\(X,Y\)∧r2​\(Y,Z\)⇒r3​\(X,Z\)r\_\{1\}\(X,Y\)\\wedge r\_\{2\}\(Y,Z\)\\Rightarrow r\_\{3\}\(X,Z\)\. A pattern implies thatif the body is in the graph, the head is also in the graphpavlovicexpressive\.

## 3Problem

### 3\.1Problem Formulation

#### Pattern Instance

We further define a pattern instance as an instantiation of patternψ⇒ϕ\\psi\\Rightarrow\\phi\. For the composition pattern of relationsr1,r2,r3∈Rr\_\{1\},r\_\{2\},r\_\{3\}\\in R, a pattern instanceψ1⇒ϕ1\\psi\_\{1\}\\Rightarrow\\phi\_\{1\}is expressed asr1​\(ex,ey\)∧r2​\(ey,ez\)⇒r3​\(ex,ez\)r\_\{1\}\(e\_\{x\},e\_\{y\}\)\\wedge r\_\{2\}\(e\_\{y\},e\_\{z\}\)\\Rightarrow r\_\{3\}\(e\_\{x\},e\_\{z\}\), wherer1​\(ex,ey\)∧r2​\(ey,ez\)r\_\{1\}\(e\_\{x\},e\_\{y\}\)\\wedge r\_\{2\}\(e\_\{y\},e\_\{z\}\)corresponds to a body instanceψ1\\psi\_\{1\},r3​\(ex,ez\)r\_\{3\}\(e\_\{x\},e\_\{z\}\)corresponds to the head instanceϕ1\\phi\_\{1\}, andex,ey,ez∈Ee\_\{x\},e\_\{y\},e\_\{z\}\\in E\.

#### Pattern Over\-Generalization

Although well\-known KGE models such as TransEbordes2013translating, RotatEsun2019rotate, PairREchao2021pairre, and CompoundEge2023compoundingdemonstrate strong generalization capabilities by modeling various patterns, they suffer from inherent limitations stemming frompattern over\-generalization\. Pattern over\-generalization is the phenomenon where a model, after observing a single instance of a pattern \(ψ1⇒ϕ1\\psi\_\{1\}\\Rightarrow\\phi\_\{1\}\) in the graphGG, generalizes the pattern to every body instance that appears in the graph; for example, if the bodyψ2\\psi\_\{2\}appears in graphGG, the model infers that the corresponding headϕ2\\phi\_\{2\}must also exist \(i\.e\.,ψ2⇒ϕ2\\psi\_\{2\}\\Rightarrow\\phi\_\{2\}\)\. This issue arises because existing models are trained to generalize a pattern universally\. We formally define the phenomenon ofpattern over\-generalizationas well aslocal pattern, anduniversal patternas follows:

Pattern Over\-GeneralizationThe model generalizes alocal patternto all unseen triples without sufficient evidence, i\.e\., the model treats alocal patternas auniversal pattern\.• Local Pattern:ψi∈Go⇒ϕi∈Gp\\psi\_\{i\}\\in G\_\{o\}\\Rightarrow\\phi\_\{i\}\\in G\_\{p\}\(s\.t\.Gp⊂GuG\_\{p\}\\subset G\_\{u\}andGp≠GuG\_\{p\}\\neq G\_\{u\}\) • Universal Pattern:ψi∈Go⇒ϕi∈Gp\\psi\_\{i\}\\in G\_\{o\}\\Rightarrow\\phi\_\{i\}\\in G\_\{p\}\(s\.t\.Gp=GuG\_\{p\}=G\_\{u\}\)for a given set of instantiated relations\{ri\}i=1n\\\{r\_\{i\}\\\}\_\{i=1\}^\{n\}\.

where

Go:\\displaystyle G\_\{o\}:\{f\|fis a set of triples observed in the given\\displaystyle\\\{f\|f\\text\{ is a set of triples observed in the given\}\(3\)KG\}\\displaystyle\\text\{KG\}\\\}Gp:\\displaystyle G\_\{p\}:\{f\|fisψi∈Goorϕithat corresponds toψi,\\displaystyle\\\{f\|f\\text\{ is \}\\psi\_\{i\}\\in G\_\{o\}\\text\{ or \}\\phi\_\{i\}\\text\{ that corresponds to \}\\psi\_\{i\},s\.t\.ϕiis semantically correct\.\}\\displaystyle\\text\{ s\.t\. \}\\phi\_\{i\}\\text\{ is semantically correct\.\}\\\}Gu:\\displaystyle G\_\{u\}:\{f\|fisψi∈Goorϕithat corresponds toψi\)\}\\displaystyle\\\{f\|f\\text\{ is \}\\psi\_\{i\}\\in G\_\{o\}\\text\{ or \}\\phi\_\{i\}\\text\{ that corresponds to \}\\psi\_\{i\}\)\\\}
While this phenomenon can serve as a crucial inductive bias in KGE, universally generalizing a local pattern without sufficient evidence can lead to erroneous predictions by injecting incorrect information into the embeddings\.

### 3\.2Cause and Evidence

#### Why Does The Problem Appear?

The cause is that the pattern condition depends only on relation embeddings in KGE methods\. This entity\-independent pattern condition allows the model to generalize local patterns to unseen triples\.

For example, in PairRE that is fully trained to satisfy Equation[1](https://arxiv.org/html/2609.03487#S2.E1)for all triples inGG, if body instance\(ex1,r1,ey1\),\(ey1,r2,ez1\)∈G\(e\_\{x\_\{1\}\},r\_\{1\},e\_\{y\_\{1\}\}\),\(e\_\{y\_\{1\}\},r\_\{2\},e\_\{z\_\{1\}\}\)\\in Gand head instance\(ex1,r3,ez1\)∈G\(e\_\{x\_\{1\}\},r\_\{3\},e\_\{z\_\{1\}\}\)\\in G, we have

ex1∘r1H=ey1∘r1T∧ey1∘r2H=ez1∘r2T\\displaystyle\\textstyle e\_\{x\_\{1\}\}\\circ r\_\{1\}^\{H\}=e\_\{y\_\{1\}\}\\circ r\_\{1\}^\{T\}\\;\\wedge\\;e\_\{y\_\{1\}\}\\circ r\_\{2\}^\{H\}=e\_\{z\_\{1\}\}\\circ r\_\{2\}^\{T\}\(4\)∧ex1∘r3H=ez1∘r3T\\displaystyle\\textstyle\\;\\wedge\\;e\_\{x\_\{1\}\}\\circ r\_\{3\}^\{H\}=e\_\{z\_\{1\}\}\\circ r\_\{3\}^\{T\}⇒r1T∘r2T∘r3H=r1H∘r2H∘r3T\\displaystyle\\textstyle\\Rightarrow\\quad r\_\{1\}^\{T\}\\circ r\_\{2\}^\{T\}\\circ r\_\{3\}^\{H\}=r\_\{1\}^\{H\}\\circ r\_\{2\}^\{H\}\\circ r\_\{3\}^\{T\}
Under this pattern condition, if a new body instance\(ex2,r1,ey2\),\(ey2,r2,ez2\)\(e\_\{x\_\{2\}\},r\_\{1\},e\_\{y\_\{2\}\}\),\(e\_\{y\_\{2\}\},r\_\{2\},e\_\{z\_\{2\}\}\)is observed as

ex2∘r1H=ey2∘r1T∧ey2∘r2H=ez2∘r2T\\displaystyle\\textstyle e\_\{x\_\{2\}\}\\circ r\_\{1\}^\{H\}=e\_\{y\_\{2\}\}\\circ r\_\{1\}^\{T\}\\;\\wedge\\;e\_\{y\_\{2\}\}\\circ r\_\{2\}^\{H\}=e\_\{z\_\{2\}\}\\circ r\_\{2\}^\{T\}\(5\)
then the model guarantees that

ex2∘r3H=ez2∘r3T,\\displaystyle\\textstyle e\_\{x\_\{2\}\}\\circ r\_\{3\}^\{H\}=e\_\{z\_\{2\}\}\\circ r\_\{3\}^\{T\},\(6\)thereby leading the model to treat the corresponding head as valid for every new body instance of the same pattern\. This phenomenon is further illustrated in Figure[1](https://arxiv.org/html/2609.03487#S3.F1), where the model generalizes the pattern universally to every body instance\. However, not all patterns in KGs are universally valid, especially those with a few pattern instances\.111In Appendix[I](https://arxiv.org/html/2609.03487#A9), we present examples of semantically local patterns that have low frequency, along with failure cases where pattern frequency does not match the pattern semantics\.

Figure 1:Illustration of local and universal patterns, and the process and examples of over\-generalization\. Existing models suffer from over\-generalization by treating local patterns as universal patterns\. The example shows that a local pattern is generalized universally, which leads to erroneous predictions\.\(a\)Histograms of local patterns that are supported by scarce pattern instances\. The relation sets for the left and right figures are\(film/written\_by, actor/film, film/prequel\)and\(film/director, film/prequel, actor/film\), respectively
\(b\)Histograms of universal patterns that are supported by many pattern instances\. The relation sets for the left and right figures are\(actor/film, film/country, people/nationality\)and\(people/place\_of\_birth, location/country, people/nationality\), respectively

Figure 2:Histograms of embedding differenceΔ=r1T∘r2T∘r3H−r1H∘r2H∘r3T\\Delta=r\_\{1\}^\{T\}\\circ r\_\{2\}^\{T\}\\circ r\_\{3\}^\{H\}\-r\_\{1\}^\{H\}\\circ r\_\{2\}^\{H\}\\circ r\_\{3\}^\{T\}for different relation set\(r1,r2,r3\)\(r\_\{1\},r\_\{2\},r\_\{3\}\)\. \# of PI denotes the number of pattern instances\.\(r1,r2,r3\)\(r\_\{1\},r\_\{2\},r\_\{3\}\)are retrieved from FB15k\-237\.Figure 3:The number of pattern instances and body instances for the local patterns introduced in Figure[2\(a\)](https://arxiv.org/html/2609.03487#S3.F2.sf1)\.
#### Empirical Evidence

Nevertheless, existing KGE models overlook the difference between local and universal patterns, treating all observed patterns as universally valid regardless of their frequency\. Figure[2](https://arxiv.org/html/2609.03487#S3.F2)shows the histograms of the embedding differenceΔ=r1T∘r2T∘r3H−r1H∘r2H∘r3T\\Delta=r\_\{1\}^\{T\}\\circ r\_\{2\}^\{T\}\\circ r\_\{3\}^\{H\}\-r\_\{1\}^\{H\}\\circ r\_\{2\}^\{H\}\\circ r\_\{3\}^\{T\}, that is presented in Equation[4](https://arxiv.org/html/2609.03487#S3.E4)\. Elements ofΔ\\Deltaclose to zero indicate that the model recognizes the given relation set\(r1,r2,r3\)\(r\_\{1\},r\_\{2\},r\_\{3\}\)as a valid pattern, therefore, the model generalizes the pattern universally to every body instance\. Figures[2\(a\)](https://arxiv.org/html/2609.03487#S3.F2.sf1)and[2\(b\)](https://arxiv.org/html/2609.03487#S3.F2.sf2)show that the elements ofΔ\\Deltaare concentrated near zero for both local and universal patterns, indicating that the model recognizes both as valid composition patterns regardless of instance frequency\. This empirically demonstrates that PairRE is trained to generalize local patterns as if they were universal, even when the supporting instances are scarce\.

While local patterns are supported by only a scarce number of pattern instances, they often have a vast number of body instances\. Figure[3](https://arxiv.org/html/2609.03487#S3.F3)presents the number of pattern instances and body instances of the local patterns in Figure[2\(a\)](https://arxiv.org/html/2609.03487#S3.F2.sf1)\. This indicates that a large number of body instances are affected by only a few pattern instances, leading the model to predict the corresponding head instances as valid for all body instances\. The distribution of pattern instances and body instances of the universal patterns in Figure[2\(b\)](https://arxiv.org/html/2609.03487#S3.F2.sf2)and empirical evidence for another pattern type are presented in Appendix[J](https://arxiv.org/html/2609.03487#A10)\.

To address this issue, we propose a novel KGE framework PogRE that explicitly models the distinction between pattern universality and locality\. Our core idea is to generalize patterns differentially based on their observation frequency inGG, rather than generalizing all patterns equally\.

#### Which Methods Are Affected?

Table[1](https://arxiv.org/html/2609.03487#S3.T1)presents representative examples of KGE methods that suffer from over\-generalization\. To verify whether these models actually suffer from over\-generalization, we propose the Over\-Generalization \(OG\) ratio\. Specifically, for the local patterns presented in Figure[3](https://arxiv.org/html/2609.03487#S3.F3), we extract the head instances corresponding to the body instances and categorize them into True triples \(if triples are inGG\) and False triples \(others\)\. The OG ratio is defined as the average score produced by a model for the True triples divided by the average score for the False triples\. An OG ratio close to 1 indicates that a model suffers from over\-generalization, as it assigns similar scores to both True and False triples\. Conversely, an OG ratio closer to 0 implies that the model effectively avoids this issue by assigning higher scores to False triples than to True triples\. PogRE exhibits a lower OG ratio than other models\. This indicates that PogRE effectively addresses over\-generalization\. For more details of the OG ratio, please refer to Appendix[H](https://arxiv.org/html/2609.03487#A8)\.

Table 1:Comparison between PogRE and KGE models\.hhandttdenote head and tail embeddings andhrh\_\{r\}andtrt\_\{r\}indicate head and tail embeddings in the relation\-specific space, as presented in Equation[10](https://arxiv.org/html/2609.03487#S4.E10)\.

## 4Method

In this section, we present the formulation of PogRE and provide a theoretical analysis demonstrating how PogRE addresses over\-generalization\.

### 4\.1Pattern Over\-Generalization Robust Embedding \(PogRE\)

#### Final Form

We define the score function as the distance between the head entityhrh\_\{r\}and tail entitytrt\_\{r\}in relation\-specific space, after the linear transformationLr∈ℝd×dL\_\{r\}\\in\\mathbb\{R\}^\{d\\times d\}:

fr​\(h,t\)=‖Lr​hr−tr‖\\textstyle f\_\{r\}\(h,t\)=\\\|L\_\{r\}h\_\{r\}\-t\_\{r\}\\\|\(7\)wherehr,tr∈ℝdh\_\{r\},t\_\{r\}\\in\\mathbb\{R\}^\{d\}denote the head and tail embeddings in relation\-specific space, respectively\.

#### Comparison Between Existing Linear Transformation Models

Although existing models such as RESCALnickel2011threeand TransRlin2015learningemploy dense linear transformations, they suffer from overfitting and representing relations asℝn×n\\mathbb\{R\}^\{n\\times n\}dense linear matrix incurs significant computational costs\. As a result, recent KGE models rarely adopt such dense linear transformations\. CompoundEge2023compoundingutilizes sparse affine operators; consequently, it suffers from over\-generalization, as presented in Table[1](https://arxiv.org/html/2609.03487#S3.T1)\. In contrast, PogRE addresses over\-generalization by employing dense linear transformations through a QR decomposition\-inspired method, which reduces computational costs\. Detailed differences are presented in Appendix[L](https://arxiv.org/html/2609.03487#A12)\.

#### QR Decomposition and Partial Sharing for Efficient Parameterization

Linear transformationLrL\_\{r\}of PogRE is decomposed into a relation\-specific orthogonal matrixQrQ\_\{r\}and an upper\-triangular matrixRRthat is shared across all relations:

Lr\\displaystyle\\textstyle L\_\{r\}=Qr​R\\displaystyle=Q\_\{r\}R\(8\)Qr\\displaystyle\\textstyle Q\_\{r\}=H1​H2​…​Hk\\displaystyle=H\_\{1\}H\_\{2\}\.\.\.H\_\{k\}In addition,QrQ\_\{r\}is approximated using a product ofkkHouseholder reflections \(wherek≪dk\\ll d\)\. This approximation significantly reduces the number of parameters fromnr​d2n\_\{r\}d^\{2\}tod⁡\(d\+1\)/2\+nr​k​dd\(d\+1\)/2\+n\_\{r\}kd, wherenrn\_\{r\}denotes the number of relations, thereby effectively reducing the model complexity\. Details about computational complexity with respect tokkare presented in Appendix[C](https://arxiv.org/html/2609.03487#A3)\.

Additionally, letR¯\\bar\{R\}be the learnable upper\-triangular parameter matrix\. The final shared matrixRRis formulated as:

R=R¯‖R¯‖2R=\\frac\{\\bar\{R\}\}\{\\\|\\bar\{R\}\\\|\_\{2\}\}\(9\)where∥⋅∥2\\\|\\cdot\\\|\_\{2\}denotes the spectral norm\. We argue that even for local patterns, the pattern should be generalized to entities that are not observed in the patterns but are semantically similar to entities that are observed in the patterns\. Spectral Normalization \(SN\) enables this generalization by bounding the Lipschitz constant of the transformations to onemiyato2018spectral\. Detailed derivations are provided in the Appendix[A](https://arxiv.org/html/2609.03487#A1)\.

#### Relation\-Specific Affine Mapping for Expressive Power

Sharing an upper\-triangular matrixRRreduces the expressive power of relation\-specific transformations\. To address this limitation and enhance the model capacity, followingge2023compounding, each entity is mapped into an relation\-specific space via three affine operators before applyingLrL\_\{r\}\. By employing homogeneous coordinates, these operators can be unified into a single matrix multiplication:

hr\\displaystyle\\textstyle h\_\{r\}=Mr​h,\\displaystyle=M\_\{r\}h,tr\\displaystyle t\_\{r\}=M^r⋅t\\displaystyle=\\hat\{M\}\_\{r\}\\cdot t\(10\)Mr\\displaystyle\\textstyle M\_\{r\}=Sr⋅Rr⋅Tr,\\displaystyle=S\_\{r\}\\cdot R\_\{r\}\\cdot T\_\{r\},M^r\\displaystyle\\qquad\\hat\{M\}\_\{r\}=S^r⋅R^r⋅T^r\\displaystyle=\\hat\{S\}\_\{r\}\\cdot\\hat\{R\}\_\{r\}\\cdot\\hat\{T\}\_\{r\}whereh,th,tare head and tail embeddings,Sr,RrS\_\{r\},R\_\{r\}, andTrT\_\{r\}denote the scaling, rotation, and translation operators, andS^r,R^r\\hat\{S\}\_\{r\},\\hat\{R\}\_\{r\}, andT^r\\hat\{T\}\_\{r\}denote the scaling, rotation, and translation operators for tail entity embedding, respectively\. This mapping strategy ensures that each relation has sufficient expressive power despite the shared components inLrL\_\{r\}\.

#### Optimization

Followingsun2019rotate, we adopt self\-adversarial negative sampling for training\. The loss function can be written as:

L\\displaystyle\\textstyle L=−log⁡σ⁡\(γ−fr​\(h,t\)\)\\displaystyle=\-\\log\\sigma\(\\gamma\-f\_\{r\}\(h,t\)\)\(11\)−∑ni=1p\(hi′,r,ti′\)logσ\(fr\(hi′,ti′\)−γ\)\\displaystyle\\textstyle\\displaystyle\-\\sum^\{n\}\_\{i=1\}p\(h\_\{i\}^\{\\prime\},r,t\_\{i\}^\{\\prime\}\)\\log\\sigma\(f\_\{r\}\(h\_\{i\}^\{\\prime\},t\_\{i\}^\{\\prime\}\)\-\\gamma\)whereσ\\sigmais the sigmoid function,γ\\gammais a fixed margin,\(hi′,r,ti′\)\(h\_\{i\}^\{\\prime\},r,t\_\{i\}^\{\\prime\}\)is theii\-th negative triple andp⁡\(hi′,r,ti′\)p\(h\_\{i\}^\{\\prime\},r,t\_\{i\}^\{\\prime\}\)is the weight of the negative triple, defined as:

p⁡\(hj′,r,tj′\|\{\(hi,ri,ti\)\}\)=exp​α​fr​\(hj′,tj′\)∑iexp​α​fr​\(hi′,ti′\)\\textstyle p\(h\_\{j\}^\{\\prime\},r,t\_\{j\}^\{\\prime\}\|\\\{\(h\_\{i\},r\_\{i\},t\_\{i\}\)\\\}\)=\\frac\{\\text\{exp\}\\alpha f\_\{r\}\(h\_\{j\}^\{\\prime\},t\_\{j\}^\{\\prime\}\)\}\{\\sum\_\{i\}\\text\{exp\}\\alpha f\_\{r\}\(h\_\{i\}^\{\\prime\},t\_\{i\}^\{\\prime\}\)\}\(12\)
whereα\\alphais the temperature of sampling\.

### 4\.2How Is Pattern Over\-Generalization Addressed?

To analyze how PogRE addresses over generalization, we first consider using only the linear transformationLrL\_\{r\}, and then extend this analysis to our framework, which incorporates the affine operators\.

#### Theoretical Analysis: Linear Transformation

To the best of our knowledge, all patterns studied in existing research are based on connected paths formed by relations\. This implies that a body \(ψ\\psi\) and head \(ϕ\\phi\) can be represented as a relational path between the start entityeue\_\{u\}and the end entityeve\_\{v\}, where each path is formulated as a product of linear matrix multiplications\. Consequently, the body \(ψ\\psi\) and head \(ϕ\\phi\) of a pattern can be expressed as:

Body​\(ψ\):\\displaystyle\\textstyle\\text\{Body \}\(\\psi\):Lψ​eu=Lrn​…​Lr2​Lr1​eu=ev,\\displaystyle L\_\{\\psi\}e\_\{u\}=L\_\{r\_\{n\}\}\\dots L\_\{r\_\{2\}\}L\_\{r\_\{1\}\}e\_\{u\}=e\_\{v\},\(13\)Head​\(ϕ\):\\displaystyle\\textstyle\\text\{Head \}\(\\phi\):Lϕ​eu=Lrm′​…​Lr2′​Lr1′​eu=ev\.\\displaystyle L\_\{\\phi\}e\_\{u\}=L\_\{r\_\{m\}^\{\\prime\}\}\\dots L\_\{r\_\{2\}^\{\\prime\}\}L\_\{r\_\{1\}^\{\\prime\}\}e\_\{u\}=e\_\{v\}\.whereL∈ℝd×dL\\in\\mathbb\{R\}^\{d\\times d\}ande∈ℝde\\in\\mathbb\{R\}^\{d\}denote the transformation matrix and entity vector, respectively\.

From Equation[13](https://arxiv.org/html/2609.03487#S4.E13), since both paths mapeue\_\{u\}to the same entityeve\_\{v\}, we explicitly haveLψ​eu=Lϕ​euL\_\{\\psi\}e\_\{u\}=L\_\{\\phi\}e\_\{u\}, which is equivalent to:

\(Lψ−Lϕ\)eu=0⇔Eeu=0\\textstyle\(L\_\{\\psi\}\-L\_\{\\phi\}\)e\_\{u\}=0\\quad\\iff\\quad Ee\_\{u\}=0\(14\)whereE=Lψ−LϕE=L\_\{\\psi\}\-L\_\{\\phi\}denotes the constraint matrix\. Next, consider a set ofddlinearly independent entities\{eu1,eu2,…,eud\}\\\{e\_\{u\_\{1\}\},e\_\{u\_\{2\}\},\\dots,e\_\{u\_\{d\}\}\\\}that satisfy the pattern, such that222Appendix[A](https://arxiv.org/html/2609.03487#A1)details constraint matrix for various patterns\.:

Eeui=0,for alli=1,2,…,d\\textstyle Ee\_\{u\_\{i\}\}=0,\\quad\\text\{for all \}i=1,2,\\dots,d\(15\)For any arbitrary entitya∈ℝda\\in\\mathbb\{R\}^\{d\}, since\{eui\}\\\{e\_\{u\_\{i\}\}\\\}forms a basis inℝd\\mathbb\{R\}^\{d\},aacan be expressed as a linear combinationa=c1​eu1\+c2​eu2\+⋯\+cd​euda=c\_\{1\}e\_\{u\_\{1\}\}\+c\_\{2\}e\_\{u\_\{2\}\}\+\\dots\+c\_\{d\}e\_\{u\_\{d\}\}\. By the linearity of the transformationEE, it follows that:

E​a=c1​E​eu1\+c2​E​eu2\+⋯\+cd​E​eud=0\\textstyle Ea=c\_\{1\}Ee\_\{u\_\{1\}\}\+c\_\{2\}Ee\_\{u\_\{2\}\}\+\\dots\+c\_\{d\}Ee\_\{u\_\{d\}\}=0\(16\)
These results indicate that as PogRE observes more linearly independent entitieseue\_\{u\}, the dimension of the space spanned by these entities increases\. Consequently, when the number of observed entities reachesdd, PogRE guarantees the universal generalization of the pattern across all related instances\. In other words, it can be expected that a pattern becomes progressively universal as the number of observed entities increases\.

#### Extension to Relation\-Specific Affine Mapping

This analysis can be extended to our proposed framework by employing homogeneous coordinates\. By representing entities in an augmented\(d\+1\)\(d\+1\)\-dimensional space, the integration of affine operators and linear transformations for a relationrrcan be unified into a single linear matrixAr∈ℝ\(d\+1\)×\(d\+1\)A\_\{r\}\\in\\mathbb\{R\}^\{\(d\+1\)\\times\(d\+1\)\}whenM^r\\hat\{M\}\_\{r\}is non\-singular:

Ar=M^r−1​Lr​Mr\\textstyle A\_\{r\}=\\hat\{M\}\_\{r\}^\{\-1\}L\_\{r\}M\_\{r\}\(17\)Therefore, the relational path can be expressed as a product of linear transformationArA\_\{r\}\. Consequently, the same proof used in the linear case can be applied, demonstrating that the pattern becomes universal only whend\+1d\+1linearly independent entities are observed in the augmented space\. The linear independence of entity embeddings is discussed in Section[6\.3](https://arxiv.org/html/2609.03487#S6.SS3.SSS0.Px1)\. Our theoretical guarantees rely on the ideal assumption that‖E​ei‖=0\\\|Ee\_\{i\}\\\|=0\. Since satisfying this exact constraint is challenging in practice, we provide further analysis in Appendix[K](https://arxiv.org/html/2609.03487#A11), proving that an approximate constraint \(‖E​ei‖<ϵ\\\|Ee\_\{i\}\\\|<\\epsilon\) still bounds the pattern constraint of unseen entities, along with a discussion on the practical strength of the approximate constraint assumption\.

## 5Related Work

### 5\.1Distance\-based Models

Distance\-based models capture patterns through various relational operations\. TransEbordes2013translating, RotatEsun2019rotate, Rotate3Dgao2020rotate3d, DualEcao2021dual, ReflectEzhang2022knowledgeand, RotatQxie2025rotatqmodel relations through translation, rotation, 3D rotation, a combination of translation and rotation, reflection transformation, and quaternion\-based transformation, respectively\. Other models enrich these operations: HAKEzhang2020learninguses polar coordinates for semantic hierarchies, PairREchao2021pairreand CompoundEge2023compoundingapply scaling and compound operators, and DensElu2022densedecomposes relations into rotation and scaling in 3D Euclidean space\. Recent models further diversify relation modeling: ExpressivEpavlovicexpressiveand OctagonEcharpenay2024capturingrepresent relations as hyper\-parallelograms and axis\-aligned octagons, respectively\. SpeedEpavlovic2024speedeimproves efficiency in low\-dimensional Euclidean settings, OrthogonalEzhu2024blockadopts block\-diagonal orthogonal matrices with Riemannian optimization, andcharpenay2025lesstheoretically analyze the ability of MuRE to capture inference patterns\. Although these models effectively capture patterns, their pattern conditions are determined by relation embeddings, which can generalize patterns even when the pattern is supported by only a few instances\.

Table 2:Link prediction results on WN18RR, FB15k\-237 and YAGO3\-10\. Bold indicates the best result and underline indicates the second best result\.±\\pmindicates standard deviation\.
### 5\.2Tensor Decomposition Models

Tensor decomposition models capture patterns through interactions among entity and relation embeddings\. DistMultyang2015embeddingand ComplExtrouillon2016complexuse bilinear scoring functions, whereas HolEnickel2016holographicemploys circular correlation\. ANALOGYliu2017analogical, SimplEkazemi2018simple, and TuckERbalavzevic2019tuckeruse normal linear operators, enhanced CP decomposition, and Tucker decomposition, respectively\. QuatEzhang2019quaternionextends interactions with quaternion representations, while CustomizEguan2025shouldintroduces customized embeddings to address the long\-tail problem\. Although these models provide strong representation capacity, they are not explicitly designed to distinguish between local patterns and universal patterns\. As a result, they may capture observed patterns, but they do not directly control the scope of pattern generalization based on supporting evidence\.

## 6Experiments

### 6\.1Experimental Setting

#### Dataset

We evaluate PogRE on three widely used KG datasets: WN18RRdettmers2018convolutional, FB15k\-237toutanova2015observedand YAGO3\-10mahdisoltani2013yago3\. The statistics of these datasets are presented in Appendix[E](https://arxiv.org/html/2609.03487#A5)

#### Evaluation Protocol

We evaluate link prediction performance in the filtered settingbordes2013translating\. In this setting, test triples are ranked against all other candidate triples that are generated by corrupting subjects or objects:\(h′,r,t\)\{\(h^\{\\prime\},r,t\)\}or\(h,r,t′\)\{\(h,r,t^\{\\prime\}\)\}, and all the triples that appear either in the training, validation or test set are removed from the candidate triples, except the test triple of interest\. We adopt MRR, Hits@1 \(H@1\), and Hits@10 \(H@10\) to compare the performance of different KGE models\. MRR denotes the mean reciprocal rank of the correct entities, and H@N represents the proportion of correct entities ranked within the topNN\. For performance comparison, we evaluate PogRE against all KGE models discussed in Section[5](https://arxiv.org/html/2609.03487#S5)\.

Table 3:Ablation study of PogRE on WN18RR, FB15k\-237 and YAGO3\-10\. MRR is used for performance comparison\.RR,QrQ\_\{r\},S​NSN,LrL\_\{r\}, and OOM denote the shared upper triangular parameter matrix, relation specific Householder reflection, Spectral Normalization, linear transformation of PogRE, and Out of Memory, respectively\. In w/o QR Decomposition,n×nn\\times ndense linear transformations are used forLrL\_\{r\}\.

### 6\.2Main Results

#### Link Prediction Performance

As shown in Table[2](https://arxiv.org/html/2609.03487#S5.T2), PogRE exhibits superior or competitive performance compared with the baselines\. For instance, PogRE achieves MRR improvements of 0\.009, 0\.004, and 0\.011 over the second\-best models, DualE and HAKE, on WN18RR, FB15k\-237 and YAGO3\-10, respectively\. These results indicate the effectiveness and robustness of PogRE across diverse datasets\. In addition to the standard benchmarks presented above, Appendix[F](https://arxiv.org/html/2609.03487#A6)provides link prediction results on large\-scale KG datasets\.

#### Ablation Study

Table[3](https://arxiv.org/html/2609.03487#S6.T3)summarizes the results of an ablation study conducted to verify the effectiveness of each proposed component\. As shown in the results, PogRE consistently outperforms the ablated models across all datasets\. Specifically, w/oLrL\_\{r\}\(equivalent to CompoundE\) exhibits significant performance degradation\. w/oLrL\_\{r\}does not employ a dense matrix and thus suffers from over\-generalization, which suggests that overlooking this problem results in significant performance loss\. Furthermore, employing dense linear transformations without QR decomposition was infeasible across all datasets; this demonstrates that models such as TransRlin2015learning, which rely on dense linear transformations, lack scalability due to their high computational costs\.

### 6\.3Analysis

#### Entity Independence

As discussed in Section[4\.2](https://arxiv.org/html/2609.03487#S4.SS2), PogRE ensures that any pattern becomes progressively universal as more linearly independent entities are observed in the pattern\. This implies that if the entity embeddings trained by PogRE are linearly independent, PogRE can achieve such progressive universality in practice\. To investigate entity independence, we randomly sample entity embeddings trained by PogRE and compute the rank of the space spanned by the sampled entities\. Table[4](https://arxiv.org/html/2609.03487#S6.T4)shows the mean rank of the subspace spanned by sampled entities over 100 random trials\. We empirically observe that the rank of the space spanned by the randomly sampled entities is approximately equal to the number of sampled entities, demonstrating that the sampled entities are linearly independent\. These results indicate that, since entities in practice are shown to be linearly independent, the dimension of the space spanned by the entities increases as the number of observed entities increases, and that universal generalization is achieved when aroundd\+1d\+1entities are observed\. We also present empirical results on the independence of entities observed in specific patterns in Appendix[G](https://arxiv.org/html/2609.03487#A7)\.

Table 4:Mean rank of the subspace spanned by randomly sampled entities over 100 random trials across three benchmarks\. The entity dimensions of PogRE are 1,000 on WN18RR and YAGO3\-10, and 1,500 on FB15k\-237\.Table 5:Distribution of composition patterns according to the number of pattern instances \(nn\) across three benchmark datasets\. \# of Pattern Instances and \# of Patterns indicate the number of pattern instances and the number of patterns, respectively\. Prop\. \(%\) is calculated as the number of patterns within each range ofnndivided by the total number of patterns, within each dataset\.
#### Distribution of Patterns by Number of Pattern Instances

Table[5](https://arxiv.org/html/2609.03487#S6.T5)presents the distribution of composition patterns according to the number of their pattern instancesnnacross three KG benchmarks\. We compute the number of composition patterns as the number of relation sets\(rx,ry,rz\)∈R\(r\_\{x\},r\_\{y\},r\_\{z\}\)\\in Rthat have at least one observed composition pattern instance\(ψ1⇒ϕ1\)\(\\psi\_\{1\}\\Rightarrow\\phi\_\{1\}\)in the KG\. In WN18RR, FB15k\-237 and YAGO3\-10, 88\.6%, 65\.3% and 52\.8% of composition patterns have 10 or fewer pattern instances, respectively\. This result indicates that a substantial proportion of patterns in KGs are observed in only a few instances\. These patterns can be generalized universally when a model suffers from over\-generalization\. The distributions of other patterns are presented in Appendix[D](https://arxiv.org/html/2609.03487#A4)\.

Figure 4:MRR comparison between PogRE and baseline models for various sparsity conditions of pattern instances on FB15k\-237\. The black line indicates the number of test triples ofGo​v​e​rG\_\{over\}\.
#### Quantified Impact of Over\-generalization

To investigate the impact of over\-generalization, we extractGo​v​e​rG\_\{over\}, a set of triples\(h,r,t\)\(h,r,t\), where candidates\(h,r,t′\)\(h,r,t^\{\\prime\}\)or\(h′,r,t\)\(h^\{\\prime\},r,t\)\(witht′≠tt^\{\\prime\}\\neq tandh′≠hh^\{\\prime\}\\neq h\) are heads of pattern instances whose bodies are in the training set, i\.e\., candidates\(h,r,t′\)\(h,r,t^\{\\prime\}\)or\(h′,r,t\)\(h^\{\\prime\},r,t\)areϕi∈Gu∖Go\\phi\_\{i\}\\in G\_\{u\}\\setminus G\_\{o\}for which there is a correspondingψi∈Go\\psi\_\{i\}\\in G\_\{o\}\. Intuitively, if the model suffers from over\-generalization, the rank of\(h,r,t\)\(h,r,t\)is lower than the rank of the candidate triples\. To extractGo​v​e​rG\_\{over\}, we consider symmetry, inversion, composition, hierarchy, intersection, transitive, g\.intersection, b\. transitive and b\. composition where the number of pattern instances isn=1n=1,n≤10n\\leq 10,n≤102n\\leq 10^\{2\}, andn≤103n\\leq 10^\{3\}\. We compare the MRR of PogRE with other baselines: TransE, RotatE, PairRE, and CompoundE onGo​v​e​rG\_\{over\}\. Figure[4](https://arxiv.org/html/2609.03487#S6.F4)presents the results on FB15k\-237\. We observe that PogRE consistently outperforms the baselines, regardless of the number of pattern instances\. These results show that PogRE effectively addresses the negative impact of over\-generalization\. The detailed definitions ofGo​v​e​rG\_\{over\}and the comparison results for WN18RR and YAGO3\-10 are presented in Appendix[D](https://arxiv.org/html/2609.03487#A4)\.

## 7Conclusion

In this paper, we propose PogRE, a novel KGE method that utilizes linear transformations and compound operations\. PogRE addresses over\-generalization, a phenomenon in which a model generalizes a pattern to every body instance in the graph after observing only a single instance\. Our theoretical analysis shows that PogRE allows a pattern to become progressively universal as more linearly independent entities are observed\. Experimental results on three benchmark datasets demonstrate the effectiveness of PogRE\.

## Limitations

To universally generalize patterns, PogRE does not utilize the semantics of patterns, which can be a useful inductive bias for pattern generalization\. Therefore, for universal but low\-frequency patterns, PogRE may fail to generalize them universally, and for local but high\-frequency patterns, PogRE may generalize them universally, resulting in inappropriate generalization\. This limitation arises when pattern frequency does not align with semantic universality\. Accordingly, PogRE should be understood as alleviating, rather than fully resolving, pattern over\-generalization\. To address this limitation, in future work, we will leverage the semantics of patterns for pattern generalization\.

Furthermore, PogRE is limited to the transductive setting, where the goal is to learn and improve embedding structures for a fixed set of known entities and relations\. Since PogRE explicitly learns entity and relation embeddings for entities and relations observed during training, it cannot directly represent entities or relations not observed during training\. While extending PogRE to the inductive setting is an important problem for handling unknown entities and relations, addressing it requires substantially different assumptions and architectural designs\. For this reason, we leave extending PogRE to the inductive setting as future work\.

## Acknowledgements

This work was supported by the National Research Foundation of Korea\(NRF\) grant funded by the Korea government\(MSIT\) \(RS\-2026\-25520248\) \(Contribution Rate: 50%\); the National Research Foundation of Korea \(NRF\) grant funded by the Korea government \(MSIT\) \(No\.2022R1A2C2012054, Development of AI for Canonicalized Expression of Trained Hypotheses by Resolving Ambiguity in Various Relation Levels of Representation Learning\) \(Contribution Rate: 40%\); and Institute of Information & communications Technology Planning & Evaluation \(IITP\) grant funded by the Korea government \(MSIT\) \(No\.2019\-0\-01842, Artificial Intelligence Graduate School Program \(GIST\)\) \(Contribution Rate: 10%\)\.

## References

Table 6:Constraint matrices corresponding to various inference patterns\. The patterns are presented inkrishnan2024methodTable 7:The best hyperparameter settings of PogRE for link prediction\.BB,NN,DD,γ\\gamma,α\\alpha,l​rlr, andkkdenote batch size, negative sampling size, dimension, gamma \(presented in Equation[11](https://arxiv.org/html/2609.03487#S4.E11)\), alpha \(presented in Equation[12](https://arxiv.org/html/2609.03487#S4.E12)\), learning rate, and number of Householder reflections, respectively\.Table 8:Space complexity comparison of PogRE and baseline models\.nrn\_\{r\},dd,kk, andbbdenote the number of relations, embedding dimension, number of Householder reflections, and batch size, respectively\. Peak GPU memory consumption is measured during training on WN18RR and FB15k\-237 using a single NVIDIA GeForce RTX 3090 under the same experimental settings\.## Appendix AConstraint Matrices for Patterns and Local Pattern Generalization

#### Constraint Matrices for Various Patterns

Table[6](https://arxiv.org/html/2609.03487#A0.T6)summarizes the derived constraint matrices for various patterns widely used in KG\. Note that for patterns having multiple paths \(e\.g\., Intersection\),EErepresents a set of matrices\{E1,E2,…\}\\\{E\_\{1\},E\_\{2\},\\dots\\\}to be satisfied simultaneously\.

#### Bounding Lipschitz Constants for Local Pattern Generalization to Unobserved Entities

By applying Spectral Normalization to the shared matrixRR, we ensure that the spectral norm of each relation\-specific linear transformation is bounded:‖Lr‖2≤1\\\|L\_\{r\}\\\|\_\{2\}\\leq 1\. Since the constraint matrixEEis defined asLψ−LϕL\_\{\\psi\}\-L\_\{\\phi\}, the spectral norm ofEEis also bounded by the triangle inequality:

‖E‖2=‖Lψ−Lϕ‖2≤‖Lψ‖2\+‖Lϕ‖2≤2\\\|E\\\|\_\{2\}=\\\|L\_\{\\psi\}\-L\_\{\\phi\}\\\|\_\{2\}\\leq\\\|L\_\{\\psi\}\\\|\_\{2\}\+\\\|L\_\{\\phi\}\\\|\_\{2\}\\leq 2This bound ensures that the transformation defined by the constraint matrix is Lipschitz continuous\. For an entityeo​b​se\_\{obs\}that is known to satisfy the pattern \(i\.e\.,‖E​eo​b​s‖≈0\\\|Ee\_\{obs\}\\\|\\approx 0\) and a semantically similar but unobserved entityeu​n​o​b​se\_\{unobs\}, the pattern error‖E​eu​n​o​b​s‖\\\|Ee\_\{unobs\}\\\|foreu​n​o​b​se\_\{unobs\}is bounded as follows:

‖E​eu​n​o​b​s‖≤‖E‖2​‖eu​n​o​b​s−eo​b​s‖\+‖E​eo​b​s‖\\\|Ee\_\{unobs\}\\\|\\leq\\\|E\\\|\_\{2\}\\\|e\_\{unobs\}\-e\_\{obs\}\\\|\+\\\|Ee\_\{obs\}\\\|As shown in the inequality, if the distance‖eu​n​o​b​s−eo​b​s‖\\\|e\_\{unobs\}\-e\_\{obs\}\\\|is small, the error‖E​eu​n​o​b​s‖\\\|Ee\_\{unobs\}\\\|remains small\. This mathematically guarantees that the model generalizes the learned pattern from observed entities to semantically similar entities with similar embeddings\.

Table 9:Time complexity, training time, and link prediction performance of PogRE and baseline models\.bbanddddenote the batch size and embedding dimension, respectively\. Training times are measured using a single NVIDIA GeForce RTX 3090 under the same experimental settings\.\(a\)WN18RR\(b\)FB15k\-237\(c\)YAGO3\-10
Figure 5:MRR, training time, inference time, and number of parameters of PogRE on three benchmark datasets\.

## Appendix BImplementation Details

For the experiments, we adopt the hyperparameter settings from RotatEsun2019rotatefor WN18RR and YAGO3\-10, and from PairREchao2021pairrefor FB15k\-237\. Additionally, for PogRE, the number of Householder reflectionskkis selected from\{2,4,8,12,20\}\\\{2,4,8,12,20\\\}\. More specifically, we utilized the official implementations of RotatEsun2019rotateand PairREchao2021pairreas our codebase\. For the datasets, we used WN18RR, FB15k\-237, and YAGO3\-10 as provided in the official RotatE repository, and the biokg and wikikg2 datasets as provided in the PairRE repository\. Table[7](https://arxiv.org/html/2609.03487#A0.T7)presents the exact batch size, negative sampling size, embedding dimension,γ\\gamma, learning rate, andkkused for each dataset\. Our presented results represent the mean of three independent runs for each dataset\. Furthermore, ScalingSrS\_\{r\}and RotationRrR\_\{r\}are used for FB15k\-237 and YAGO3\-10, whereas TranslationTrT\_\{r\}and RotationRrR\_\{r\}are used for WN18RR\. Finally, following Rotate3Dgao2020rotate3d, anL2L\_\{2\}regularizer is applied to entity embeddings for WN18RR\. Experiments for the PogRE were conducted on an NVIDIA 3090 with 24GB of memory\.

## Appendix CComputational Complexity

#### Space Complexity

Table[8](https://arxiv.org/html/2609.03487#A0.T8)compares the number of relation parameters, the relation tensors required during batch scoring, and the peak GPU memory consumption during training\.nrn\_\{r\},dd,kk, andbbdenote the number of relations, embedding dimension, number of Householder reflections, and batch size, respectively\.

In PogRE, the relation\-specific orthogonal transformation is represented usingkkHouseholder vectors, while the upper\-triangular matrix is shared across all relations\. Therefore, PogRE requiresO⁡\(nr​k​d\+d2\)O\(n\_\{r\}kd\+d^\{2\}\)parameters for its dense linear transformations\. In contrast, PogRE w/o QR assigns an independentd×dd\\times ddense matrix to each relation, resulting inO⁡\(nr​d2\)O\(n\_\{r\}d^\{2\}\)relation parameters\. The difference becomes more pronounced during batch scoring\. PogRE w/o QR requires ab×d×db\\times d\\times dtensor containing relation\-specific dense matrices, whereas PogRE requiresb×k×db\\times k\\times drelation\-specific Householder vectors and a single sharedd×\(d\+1\)/2d\\times\(d\+1\)/2matrix\. This quadratic memory requirement at the batch level makes PogRE w/o QR infeasible under the same experimental setting and results in OOM\.

#### Time Complexity

Table[9](https://arxiv.org/html/2609.03487#A1.T9)compares the theoretical scoring complexity and the actual training time\. PogRE w/o QR has the same theoretical time complexity as PogRE but is infeasible under the same experimental setting due to its substantially higher space complexity\. PogRE has a higher theoretical time complexity than RotatE and PairRE because of the shared matrix multiplication\. Nevertheless, its practical training time withk=2k=2remains comparable to that of the baselines, while achieving higher performance in link prediction\. Increasingkkto 20 requires additional training time but further improves the MRR on both datasets\. These results demonstrate that PogRE provides a practical trade\-off between computational cost and performance\.

Figure[5](https://arxiv.org/html/2609.03487#A1.F5)presents computational complexity and performance with respect to the Householder reflectionkk\. In WN18RR and FB15k\-237, performance improves askkincreases but shows no significant improvement afterk=20k=20\. This performance gain is accompanied by an increase in computational cost askkgrows\. In YAGO3\-10, the MRR is highest atk=2k=2and decreases askkincreases\. These results suggest that while a largerkkcan improve performance by increasing the expressive power, excessive complexity may lead to a decrease in performance due to overfitting\. Furthermore, there is almost no variation in inference time across differentkkvalues, implying thatkkcan be selected during training without concerns regarding inference time\.

## Appendix DImpact of Over\-generalization

#### Detailed Definition of GroupGo​v​e​rG\_\{over\}

GroupGo​v​e​rG\_\{over\}consists of test triples\(h,r,t\)\(h,r,t\)where at least one candidate triple\(h,r,t′\)\(h,r,t^\{\\prime\}\)\(wheret′≠tt^\{\\prime\}\\neq t\) isϕi∈Gu∖Go\\phi\_\{i\}\\in G\_\{u\}\\setminus G\_\{o\}for which there is a correspondingψi∈Go\\psi\_\{i\}\\in G\_\{o\}\. In this case, the body instances corresponding to the candidate appear in the training set\. For instance, consider a local composition patternψ⇒ϕ\\psi\\Rightarrow\\phiconsisting of the relation triplet\(r1,r2,r\)\(r\_\{1\},r\_\{2\},r\)\. If the training set contains the body instances\(h,r1,x\)\(h,r\_\{1\},x\)and\(x,r2,t′\)\(x,r\_\{2\},t^\{\\prime\}\)for at least one candidatet′t^\{\\prime\}and some entityx∈Ex\\in E, then the test triple\(h,r,t\)\(h,r,t\)is assigned toGo​v​e​rG\_\{over\}\. If KGE models suffer from over\-generalization, they are likely to assign a high score to such a candidate\(h,r,t′\)\(h,r,t^\{\\prime\}\), treating it as a valid triple\. For simplicity, we only describe the case of tail prediction, but the same procedure applies to head prediction\. For pattern, we consider symmetry, inversion, composition, hierarchy, intersection, transitive, g\.intersection, b\. transitive and b\. composition patterns, as they are the most representative inference patterns extensively investigated across a wide range of KGE models\. Antisymmetry is not considered because it ensures the absence of a head, rather than ensuring the presence of head\.

\(a\)WN18RR\(b\)YAGO3\-10
Figure 6:MRR comparison between PogRE and baseline models for various sparsity conditions of pattern instances on two benchmarks: WN18RR \(a\), and YAGO3\-10 \(b\)\. The black line indicates the number of test triples for each group\.Table 10:Link prediction results on large\-scale KGs, including ogbl\-biokg and ogbl\-wikikg2\. Bold indicates the best result, and underline indicates the second best result\.±\\pmindicates the standard deviation\.
#### Quantified Impact of Over\-generalization

Figure[6](https://arxiv.org/html/2609.03487#A4.F6)presents the performance comparison on theGo​v​e​rG\_\{over\}across WN18RR and YAGO3\-10\. Also in WN18RR and YAGO3\-10, PogRE consistently outperforms the baselines regardless of the local pattern criteria and the dataset\. These results demonstrate that PogRE effectively addresses the negative impact of over\-generalization, and shows robustness across datasets\.

#### Distribution of Patterns by Number of Pattern Instances

Table[20](https://arxiv.org/html/2609.03487#A13.T20)presents the distribution of each pattern according to the number of its pattern instances \(nn\) across three KG benchmark datasets\.In WN18RR, FB15k\-237, and YAGO3\-10, respectively, 20\.0%, 22\.7%, and 41\.7% of symmetry patterns, 27\.3%, 24\.6%, and 20\.0% of antisymmetry patterns, and 50\.0%, 49\.3%, and 58\.8% of inversion patterns have 10 or fewer pattern instances\. This result indicates that for symmetry, antisymmetry, and inversion, a substantial proportion of patterns in real\-world KGs are observed in only a few instances\.

Table 11:Statistics of three benchmark datasets

## Appendix EDatasets

WN18RR, FB15k\-237 and YAGO3\-10 are used to evaluate PogRE\. WN18RR and FB15k\-237 are subsets of WN18bordes2013translatingand FB15kbordes2013translatingwith inverse relations removed, and YAGO3\-10 is a subset of YAGO3mahdisoltani2013yago3containing only entities with a minimum of 10 relations each\. ogbl\-biokghu2020openis a large\-scale biomedical KG, and ogbl\-wikikg2hu2020openis a Wikidata knowledge graph that contains a large number of triples\. The statistics are summarized in Table[11](https://arxiv.org/html/2609.03487#A4.T11)\.

## Appendix FLink Prediction Performance on Large\-scale Knowledge Graph

To verify the effectiveness of PogRE in large\-scale KGs, we conduct additional experiments on biokg and wikikg2\. As shown in the table[11](https://arxiv.org/html/2609.03487#A4.T11), biokg and wikikg2 contain 4\.7 million and 16\.1 million triples, respectively, making them significantly larger than the standard benchmarks\. The comparative results on biokg and wikikg2 are presented in Table[10](https://arxiv.org/html/2609.03487#A4.T10)\. PogRE achieves the highest performance in both valid and test MRR compared to the baselines\. For instance, PogRE achieves MRR improvements of 0\.0027 and 0\.0133 over the second\-best models, PairRE and CompoundE, on biokg and wikikg2 in Test MRR, respectively\. Furthermore, PogRE exhibits a low standard deviation \(±0\.001\\pm 0\.001\) across all datasets and settings, indicating that it consistently maintains stable performance regardless of initialization\. This comparison demonstrates the robustness of our method across different datasets, especially in large\-scale KGs\.

Table 12:Mean rank of the subspace spanned by randomly sampled entities in the specific pattern, over 100 random trials across three benchmarks\. The entity dimensions of PogRE are 1,000 on WN18RR and YAGO3\-10, and 1,500 on FB15k\-237\.
## Appendix GLinear Independence of Entities Observed Within a Pattern

In Section[6\.3](https://arxiv.org/html/2609.03487#S6.SS3.SSS0.Px1), we empirically observed that the dimension of the space spanned by entities randomly sampled from the entire KG is approximately equal to the number of sampled entities\. In this section, we further investigate whether entities observed within specific patterns—rather than across the entire KG—are also linearly independent of each other\. Our experimental procedure is as follows: First, we randomly select a pattern containing at least 2,000 instances\. We then randomly sampleNNentities from this pattern \(varyingNNfrom 100 to 2,000\) and measure the rank of the space spanned by these sampled entities\. Finally, we repeat this process for 100 independent trials and report the average rank in Table[12](https://arxiv.org/html/2609.03487#A6.T12)\. Our results demonstrate that the rank of the space spanned by these entities is approximately equal to the number of sampled entities, indicating that linear independence is indeed preserved even within specific patterns\.

## Appendix HDetailed Definition of the OG Ratio and Its Comparison on Various Patterns

In this section, we provide a detailed definition of the OG Ratio and additional experiments on the OG Ratio\.

Table 13:OG ratio comparison between KGE baselines and PogRE on FB15k\-237\.nnindicates the number of pattern instances\.#### Detailed Definition of OG Ratio

Since KGE is a relative distance\-based method, if the score of a triple\(h,r,t\)\(h,r,t\)is closer to 0 than that of another triple\(h,r,t′\)\(h,r,t^\{\\prime\}\), it implies that the model considers\(h,r,t\)\(h,r,t\)to be more plausible than\(h,r,t′\)\(h,r,t^\{\\prime\}\)\. Based on this relative property, we define the OG ratio as follows\. First, we extract head instances corresponding to the body instances of patterns in the KG\. We then divide them into two groups—True triples and False triples—and measure the average score of each group \(we exclude head instances that are in the training data, as they are used to train the models\)\. Next, we define the OG ratio as the average score of True triples divided by that of False triples\. An OG ratio≈\\approx1 indicates that the model suffers from over\-generalization, as it assigns similar scores to both True and False triples\. An OG ratio≈\\approx0 implies that the model effectively avoids over\-generalization by assigning higher scores to False triples compared to True triples\.

#### OG Ratio Comparison on Various Patterns

We use the OG ratio to verify whether our model effectively addresses over\-generalization in the entire KG\. Specifically, we extract True and False triples for patterns whose number of pattern instancesnnsatisfiesn≤10n\\leq 10,n≤102n\\leq 10^\{2\}, andn≤103n\\leq 10^\{3\}in FB15k\-237, and compare the OG ratio of PogRE against KGE baselines, including TransE, RotatE, PairRE, and CompoundE\. Table[13](https://arxiv.org/html/2609.03487#A8.T13)presents a comparison of the OG ratios between PogRE and the KGE baselines\. Our model exhibits a lower OG ratio than all other models across all settings\. This demonstrates that our model effectively addresses pattern over\-generalization in the entire KG\. This analysis was not conducted for WN18RR and YAGO3\-10; due to their small validation/test sets and data sparsity, these datasets contain very few True triples \(fewer than 10\)\.

## Appendix IEmpirical Analysis of Local and Universal Patterns

In Table[21](https://arxiv.org/html/2609.03487#A13.T21), we present examples of local and universal patterns in real\-world KGs based on human verification\. Because symmetry and inversion patterns can be easily classified as either universal or local, we extract these patterns from three benchmark datasets: WN18RR, FB15k\-237, and YAGO3\-10\. For FB15k\-237, due to the large number of patterns, we randomly extract only 40 patterns\. We then manually classify each pattern into two groups: semantically universal and semantically local\. We further categorize these patterns based on the number of pattern instances \(nn\)\. Empirically, we observe that, in general, patterns that have low frequencies \(n≤10n\\leq 10\) tend to be semantically local, which can cause over\-generalization in existing KGE models, leading to erroneous predictions\.

Table 14:Representative examples of patterns categorized into the two failure case sets: universal but low frequency, and local but high frequency\. The numbers in parentheses indicate the frequency of pattern instances, while WN, FB, and YAGO denote the WN18RR, FB15k\-237, and YAGO3\-10, respectively\.However, we also note that, due to the semantic complexity of real\-world KGs, this general tendency may not always hold\. We identify two representative failure cases for our proposed method\.

- •First,universal but low\-frequency patternsmay appear when semantically universal patterns are observed in only a few instances due to dataset sparsity\. In this case, PogRE may fail to generalize them universally\.
- •Second,local but high\-frequency patternsmay appear when semantically local patterns have many observed instances\. In this case, PogRE may generalize them universally, resulting in inappropriate generalization\.

Table[14](https://arxiv.org/html/2609.03487#A9.T14), a subset of Table[21](https://arxiv.org/html/2609.03487#A13.T21), shows examples of patterns in these two case sets\. Each pattern is reported with its frequency and dataset\. The universal but low\-frequency set indicates semantically universal patterns whose frequency is lower than the entity embedding dimension\. The local but high\-frequency set indicates semantically local patterns whose frequency is higher than the entity embedding dimension\.

We compare PogRE with KGE baselines on these cases\. Specifically, for each dataset, we extract test triples that contain relations included in each failure case set and measure MRR\. Table[15](https://arxiv.org/html/2609.03487#A9.T15)reports the MRR results for the universal but low frequency set\. PogRE underperforms compared to CompoundE on FB15k\-237 and YAGO3\-10\. This demonstrates the negative impact of the universal but low frequency failure case on performance of PogRE\.

Table 15:MRR comparison between PogRE and baseline KGE models for the universal but low frequency case set across WN18RR, FB15k\-237, and YAGO3\-10\. Bold indicates the best performance\.Table 16:MRR comparison between PogRE and baseline KGE models for the local but high frequency case set\. Note that the evaluated patterns for this case are observed exclusively within YAGO3\-10\. Bold indicates the best performance\.Table 17:Comparison of Pattern Instances \(PI\) and Body Instances \(BI\) between the local and universal patterns introduced in Figure[2](https://arxiv.org/html/2609.03487#S3.F2)\. ’\# of PI’ and ’\# of BI’ represent the number of pattern instances and body instances, respectively\.Interestingly, PogRE still outperforms CompoundE on WN18RR\. We analyze the reasons for this as follows\. For WN18RR, we measured the MRR for two patterns:\_similar\_toand\_also\_see\. First, despite the\_similar\_topattern having only 74 instances, all models except TransE achieved an MRR of 1\.0\. This pattern is a potential failure case, but not empirically harmful because of dataset\-specific or relatively easy test structure\. Furthermore, the\_also\_seepattern has 828 instances\. Although these instances may not span the entire embedding space, they still provide observed instances that can support generalization\. As theoretically shown in Appendix[A](https://arxiv.org/html/2609.03487#A1), our spectral normalization can help keep the pattern error bounded for unobserved entities if they are semantically similar to the observed entities\. Therefore, PogRE can achieve generalization to semantically similar entities for\_also\_see, which can explain its superior performance on WN18RR\.

Table[16](https://arxiv.org/html/2609.03487#A9.T16)reports the MRR results for the local but high\-frequency set \(only evaluated on YAGO3\-10\)\. In this case, PogRE also does not outperform CompoundE\. This result indicates that PogRE may underperform existing models when semantically local patterns have high frequencies\.

Through this analysis, we clarify the applicability boundary of PogRE\.

- •Our method is designed to prevent the over\-generalization of local patterns\.
- •As shown in Table[21](https://arxiv.org/html/2609.03487#A13.T21), most local patterns have low frequency, and therefore PogRE can generally improve performance on the overall datasets\.
- •However, PogRE may underperform existing models in failure cases where pattern frequency does not match pattern semantics, such as universal but low\-frequency patterns or local but high\-frequency patterns\.
- •Therefore, PogRE is most suitable for KGs where low\-frequency patterns are likely to be local and high\-frequency patterns are likely to be universal\.

Figure 7:The number of pattern instances and body instances for the universal patterns introduced in Figure[2\(b\)](https://arxiv.org/html/2609.03487#S3.F2.sf2)\.
## Appendix JAdditional Empirical Evidence for Over\-Generalization

#### Comparison of the Number of Pattern and Body Instances Between Local and Universal Patterns

In Figure[7](https://arxiv.org/html/2609.03487#A9.F7), we present the number of pattern and body instances for the universal patterns introduced in Figure[2\(b\)](https://arxiv.org/html/2609.03487#S3.F2.sf2)\. Additionally, Table[17](https://arxiv.org/html/2609.03487#A9.T17)compares the PI and BI of the local and universal patterns from Figure[2](https://arxiv.org/html/2609.03487#S3.F2)\. As shown in Figure[7](https://arxiv.org/html/2609.03487#A9.F7), the universal patterns are supported by 5,033 and 1,082 pattern instances, respectively\. This significantly exceeds the number of pattern instances of local patterns \(13 and 22\), demonstrating that universal patterns are supported by a much larger number of observations\. Furthermore, as shown in Table[17](https://arxiv.org/html/2609.03487#A9.T17), universal patterns exhibit higher PI/BI ratios \(0\.432 and 0\.761\) compared to local patterns \(0\.176 and 0\.012\)\. This demonstrates that the patterns in Figure[2\(b\)](https://arxiv.org/html/2609.03487#S3.F2.sf2)exhibit universal characteristics\.

\(a\)Histograms of local patterns that are supported by scarce pattern instances\. The relations for the left and right figures are\(…/gardening\_hint/split\_to\)and\(…/us\_county/county\_seat\), respectively\.
\(b\)Histograms of universal patterns that are supported by many pattern instances\. The relations for the left and right figures are\(…/location/adjoining\_relationship…\)and\(…/award/award\_nomination…\), respectively\.

Figure 8:Histograms of embedding differenceΔ=\(r1H\)2−\(r1T\)2\\Delta=\(r\_\{1\}^\{H\}\)^\{2\}\-\(r\_\{1\}^\{T\}\)^\{2\}for different symmetric relationsr1r\_\{1\}\. \# of PI denotes the number of pattern instances\.r1r\_\{1\}are retrieved from FB15k\-237\.\(a\)The number of pattern instances and body instances for the local patterns introduced in Figure[8\(a\)](https://arxiv.org/html/2609.03487#A10.F8.sf1)\.
\(b\)The number of pattern instances and body instances for the universal patterns introduced in Figure[8\(b\)](https://arxiv.org/html/2609.03487#A10.F8.sf2)\.

Figure 9:The number of pattern instances and body instances for the local and universal symmetry patterns introduced in Figure[8](https://arxiv.org/html/2609.03487#A10.F8)
#### Empirical Evidence for the Symmetry Pattern

To demonstrate that the over\-generalization effect is not limited to the composition pattern discussed in Section[3](https://arxiv.org/html/2609.03487#S3), we conduct an additional analysis on the symmetry pattern using the same framework in Section[3](https://arxiv.org/html/2609.03487#S3), as follows\.

Similar to the composition pattern, PairRE also induces a relation\-level pattern condition for symmetry patterns\. If a relationr1r\_\{1\}is symmetric, PairRE satisfies the following condition:

\(r1H\)2=\(r1T\)2\\displaystyle\\textstyle\(r\_\{1\}^\{H\}\)^\{2\}=\(r\_\{1\}^\{T\}\)^\{2\}\(18\)
This condition is determined only by relation embeddings\. Therefore, once this condition is learned from observed triples, the model can generalize the symmetry pattern to other entities, even when the pattern is supported by only a small number of instances\. This can lead to pattern over\-generalization for local symmetry patterns\. We empirically verify this by measuring the distribution ofΔ=\(r1H\)2−\(r1T\)2\\Delta=\(r\_\{1\}^\{H\}\)^\{2\}\-\(r\_\{1\}^\{T\}\)^\{2\}\. Figure[8](https://arxiv.org/html/2609.03487#A10.F8)shows the histograms of the embedding difference \(Δ\\Delta\) of local and universal symmetry patterns\. Figures[8\(a\)](https://arxiv.org/html/2609.03487#A10.F8.sf1)and[8\(b\)](https://arxiv.org/html/2609.03487#A10.F8.sf2)show that the elements ofΔ\\Deltaare concentrated near zero for both local and universal patterns, indicating that the model recognizes both as valid symmetry patterns regardless of instance frequency\. The numbers of pattern instances and body instances for these symmetry patterns are shown in Figure[9](https://arxiv.org/html/2609.03487#A10.F9)\. This analysis empirically shows that pattern over\-generalization also occurs in another pattern type\.

Table 18:Summary of differences between PogRE and the existing linear transformation models\. RESCAL is reproduced inwang2019evaluating, and TransR is reproduced inzhang2020improve\.WrW\_\{r\}andPrP\_\{r\}areℝd×d\\mathbb\{R\}^\{d\\times d\}dense linear transformations, whereddis the dimension of entities and relations\.

## Appendix KBounding Constraint Errors Under Practical Training Dynamics

In Section[4\.2](https://arxiv.org/html/2609.03487#S4.SS2), we provide a theoretical analysis showing that a dense linear transformation can address over\-generalization\. However, in practice, KGE embeddings are optimized with negative sampling and a margin\-based loss; therefore, the constraintE​ei=0Ee\_\{i\}=0is approximate\. That is, satisfying the exact linear constraintE​ei=0Ee\_\{i\}=0is challenging due to the approximate nature of margin\-based optimization with negative sampling\. However, we can mathematically guarantee that the constraint error for unseen entities is bounded by the constraint error of observed entities\. Specifically, let us assume the model is sufficiently trained such that the constraint error is minimized within a small marginϵ\\epsilonfor the observed linearly independent entitiese1,…,ede\_\{1\},\\dots,e\_\{d\}\(i\.e\., we consider the practical scenario in which‖E​ei‖<ϵ\\\|Ee\_\{i\}\\\|<\\epsilon, rather than the exact condition‖E​ei‖=0\\\|Ee\_\{i\}\\\|=0\.\) Since we empirically verified that the learned entity embeddings form a basis \(Section[6\.3](https://arxiv.org/html/2609.03487#S6.SS3.SSS0.Px1)\), any unseen entityen​e​we\_\{new\}can be represented as a linear combination of the observed entities:en​e​w=∑i=1dci​eie\_\{new\}=\\sum\_\{i=1\}^\{d\}c\_\{i\}e\_\{i\}\. By the linearity of the transformationEEand the triangle inequality, the error for the unseen entity is bounded as follows:

‖E​en​e​w‖\\displaystyle\\\|Ee\_\{new\}\\\|=‖∑i=1dci​\(E​ei\)‖\\displaystyle=\\left\\\|\\sum\_\{i=1\}^\{d\}c\_\{i\}\(Ee\_\{i\}\)\\right\\\|\(19\)≤∑i=1d\|ci\|⋅‖E​ei‖\\displaystyle\\leq\\sum\_\{i=1\}^\{d\}\|c\_\{i\}\|\\cdot\\\|Ee\_\{i\}\\\|<\(∑i=1d\|ci\|\)​ϵ\\displaystyle<\\left\(\\sum\_\{i=1\}^\{d\}\|c\_\{i\}\|\\right\)\\epsilonThis inequality demonstrates that minimizing the constraint error of observed entities \(ϵ→0\\epsilon\\to 0\) directly suppresses the constraint error for unseen entities\. Therefore, even under the approximate optimization of margin\-based loss, the constraint error of unseen entities remains bounded\. Through this analysis, we clarify the practical scope of the theoretical guarantees of our method\. In practice, exact constraint satisfaction, corresponding to the ideal zero\-error case, i\.e\.,‖E​ei‖=0\\\|Ee\_\{i\}\\\|=0, is not guaranteed after training\. Rather, when the constraint errors of observed entities are small, corresponding to the approximate case, i\.e\.,‖E​ei‖<ϵ\\\|Ee\_\{i\}\\\|<\\epsilon, the errors of unseen entities can also be bounded, thereby ensuring the practical effectiveness of our proposed method\.

#### Practical Strength of the Assumptions

Our theoretical analysis involves two assumptions: \(1\) the approximate satisfaction of pattern constraints for observed entities and \(2\) the availability ofd\+1d\+1linearly independent observed entities\. We discuss the practical strength of these assumptions below\.

First, the practical strength of the approximate pattern\-constraint assumption depends on how small the residuals become in practice\. The approximate satisfaction of pattern constraints is encouraged by the KGE training objective\. The training objective reduces the errors of observed pattern instances, thereby encouraging small residuals in the corresponding pattern constraints\. However, because training relies on mini\-batch gradient\-based optimization, it is difficult to know how small the residuals will be after training, making the practical strength difficult to assess a priori\. Consequently, our approximate analysis above is conditional on the residuals actually achieved after training\.

Second, the requirement of observingd\+1d\+1linearly independent entities should be understood as a sufficient condition for universal generalization, rather than as a condition that must always hold\. As more linearly independent entities supporting a pattern are observed, the pattern constraint applies to a larger subspace; universal generalization is guaranteed when these entities span the relevant space\. We regard this requirement as part of an inherent trade\-off\. If this condition is made less strict, the model may generalize patterns more easily\. However, this may also increase the risk of generalizing weakly supported patterns too broadly, which may lead to the over\-generalization problem that PogRE is designed to avoid\.

## Appendix LDifferences between the existing linear transformation model and PogRE

We present a comparison between existing linear transformation models and PogRE in Table[18](https://arxiv.org/html/2609.03487#A10.T18)\. Existing models that use dense linear transformation such as RESCALnickel2011threeand TransRlin2015learningare not designed for pattern modeling; moreover, when the entity and relation dimensions arenn, they assign anℝn×n\\mathbb\{R\}^\{n\\times n\}matrix to each relation, leading to high computational costs as the dimension of entities and relations increases\. While CompoundE is capable of pattern modeling and avoids these computational issues by using sparse affine transformation, it suffers from over\-generalization\. In contrast, PogRE is designed for pattern modeling, addresses the over\-generalization problem, and avoids the computational cost issue using a QR\-decomposition\-inspired method\. In addition, PogRE outperforms existing models on WN18RR and FB15k\-237, demonstrating its effectiveness\.

Table 19:MRR comparison between PogRE and transductive GNN\-based models on WN18RR and FB15k\-237\.
## Appendix MComparison with Transductive GNNs

Transductive GNN models such as R\-GCNschlichtkrull2018modeling, SACNshang2019end, and CompGCNVashishth2020Composition\-basedimprove entity representations through message passing, which incorporates local structural context into entity embeddings\. By leveraging structural context, these models may also alleviate pattern over\-generalization\. However, pattern over\-generalization has not been explicitly discussed or analyzed in transductive GNN models\. In particular, prior work has not characterized how transductive GNN models capture patterns or how the patterns captured by transductive GNN models are generalized from observed evidence\. In contrast, PogRE explicitly models patterns and is designed to improve entity representations while alleviating pattern over\-generalization\.

We further compare PogRE with these transductive GNN models in terms of MRR\. As shown in Table[19](https://arxiv.org/html/2609.03487#A12.T19), PogRE outperforms the baselines across the available benchmarks, demonstrating that PogRE remains effective compared with graph\-contextual baselines\.

Pattern\# of PatternInstances \(n\)WN18RRFB15k\-237YAGO3\-10\# of PatternsProp\. \(%\)\# of PatternsProp\. \(%\)\# of PatternsProp\. \(%\)Symmetryn = 1\-\-613\.6\-\-1<<n≤\\leq10120\.049\.1541\.710<<n≤102\\leq 10^\{2\}120\.01022\.7\-\-102<10^\{2\}<n≤103\\leq 10^\{3\}120\.01738\.6433\.3n\>103\>10^\{3\}240\.0715\.9325\.0Total5100%44100%12100%Antisymmetryn = 119\.111\.8\-\-1<<n≤\\leq10218\.21322\.8320\.010<<n≤102\\leq 10^\{2\}218\.22645\.6533\.3102<10^\{2\}<n≤103\\leq 10^\{3\}218\.21322\.8213\.3n\>103\>10^\{3\}436\.447\.0533\.3Total11100%57100%15100%Inversionn = 1330\.05218\.6635\.31<<n≤\\leq10220\.08630\.7423\.510<<n≤102\\leq 10^\{2\}550\.08329\.6423\.5102<10^\{2\}<n≤103\\leq 10^\{3\}\-\-5118\.215\.9n\>103\>10^\{3\}\-\-82\.9211\.8Total10100%280100%17100%Compositionn = 11748\.61,54626\.35617\.61<<n≤\\leq101440\.02,28839\.011235\.210<<n≤102\\leq 10^\{2\}411\.41,43924\.59128\.6102<10^\{2\}<n≤103\\leq 10^\{3\}\-\-4898\.35316\.7n\>103\>10^\{3\}\-\-1111\.961\.9Total35100%5,873100%318100%Hierarchyn = 1112\.57221\.41119\.61<<n≤\\leq10225\.09428\.01628\.610<<n≤102\\leq 10^\{2\}562\.511433\.91425\.0102<10^\{2\}<n≤103\\leq 10^\{3\}\-\-5014\.91221\.4n\>103\>10^\{3\}\-\-61\.835\.4Total8100%336100%56100%Intersectionn = 1\-\-43829\.61815\.01<<n≤\\leq10\-\-64243\.35445\.010<<n≤102\\leq 10^\{2\}\-\-37825\.54235\.0102<10^\{2\}<n≤103\\leq 10^\{3\}\-\-241\.665\.0n\>103\>10^\{3\}\-\-\-\-\-\-Total\-\-1,482100%120100%Transitiven = 1\-\-24\.1\-\-1<<n≤\\leq10116\.7816\.3533\.310<<n≤102\\leq 10^\{2\}233\.31632\.7426\.7102<10^\{2\}<n≤103\\leq 10^\{3\}350\.01122\.4213\.3n\>103\>10^\{3\}\-\-1224\.5426\.7Total6100%49100%15100%G\.Intersectionn = 1233\.32113\.0218\.21<<n≤\\leq10\-\-5433\.5545\.510<<n≤102\\leq 10^\{2\}466\.77043\.5218\.2102<10^\{2\}<n≤103\\leq 10^\{3\}\-\-148\.7218\.2n\>103\>10^\{3\}\-\-21\.2\-\-Total6100%161100%11100%B\. Transitiven = 1\-\-615\.8\-\-1<<n≤\\leq10\-\-25\.3112\.510<<n≤102\\leq 10^\{2\}133\.31128\.9225\.0102<10^\{2\}<n≤103\\leq 10^\{3\}266\.6923\.7337\.5n\>103\>10^\{3\}\-\-1026\.3225\.0Total3100%38100%8100%B\. Compositionn = 11240\.01,27828\.437\.11<<n≤\\leq101860\.01,71038\.11535\.710<<n≤102\\leq 10^\{2\}\-\-1,05323\.41228\.6102<10^\{2\}<n≤103\\leq 10^\{3\}\-\-3457\.71228\.6n\>103\>10^\{3\}\-\-1082\.4\-\-Total30100%4,494100%42100%Table 20:Distribution of inference patterns: Symmetry/Antisymmetry, Inversion, Composition, Hierarchy, Intersection, Transitive, G\.Intersection, B\. Transitive and B\. Composition according to the number of pattern instances \(nn\) across three benchmark datasets\.Table 21:Human verification results of semantically universal and semantically local patterns, categorized by the number of pattern instances\. The number next to each pattern indicates the number of pattern instances, and \(S\) and \(I\) indicate symmetry and inversion, respectively\.\# of Pattern InstancesSemantically Universal \(nn\)Semantically Local \(nn\)n≤10n\\leq 10\-owns\(4\) \(S\)isAffiliatedTo\(10\) \(S\)isKnownFor\(4\) \(S\)created\(2\) \(S\)hasAcademicAdvisor\(4\) \(S\)influences,created\(3\) \(I\)influences,isInterestedIn\(8\) \(I\)participatedIn,isCitizenOf\(1\) \(I\)created,hasAcademicAdvisor\(1\) \(I\)isAffiliatedTo,isMarriedTo\(1\) \(I\)isLocatedIn,isConnectedTo\(1\) \(I\)influences,isKnownFor\(5\) \(I\)isLocatedIn,dealsWith\(1\) \(I\)influences,hasChild\(1\) \(I\)isMarriedTo,influences\(8\) \(I\)\_synset\_domain\_topic\_of\(2\) \(S\)\_synset\_domain\_topic\_of,\_has\_part\(9\) \(I\)\_hypernym,\_derivationally\_related\_form\(17\) \(I\)\_derivationally\_related\_form,\_member\_meronym\(23\) \(I\)\_hypernym,\_also\_see\(38\) \(I\)\_hypernym,\_synset\_domain\_topic\_of\(4\) \(I\)\_also\_see,\_verb\_group\(1\) \(I\)\_hypernym,\_verb\_group\(17\) \(I\)\_has\_part,\_member\_of\_domain\_region\(1\) \(I\)\_instance\_hypernym,\_member\_of\_domain\_region\(1\) \(I\)\_derivationally\_related\_form,\_synset\_domain\_topic\_of\(23\) \(I\)/location/ …division/country\(4\) \(S\)/location/country/capital\(3\) \(S\)/film/film/prequel\(2\) \(S\)/people/person/profession\(1\) \(S\)/film/film/genre\(1\) \(S\)/film/film\_subject/films\(1\) \(S\)/base/aareas/schema/administrative\_area/capital\(1\) \(S\)/medicine/symptom/symptom\_of\(1\) \(S\)/music/instrument/family\(1\) \(S\)…/sibling\_relationship…/influenced\_by\(2\) \(I\)\.\./romantic\_relationship/celebrity\(1\) \(I\)10<n≤10010<n\\leq 100\_similar\_to\(74\) \(S\)isLocatedIn,participatedIn\(50\) \(I\)isMarriedTo,hasChild\(59\) \(I\)influences,hasAcademicAdvisor\(63\) \(I\)isLocatedIn,owns\(60\) \(I\)100<n≤1000100<n\\leq 1000dealsWith\(160\) \(S\)influences\(180\) \(S\)hasNeighbor\(550\) \(S\)hasChild\(414\) \(S\)\_also\_see\(828\) \(S\)dealsWith,hasNeighbor\(165\) \(I\)…/legislative\_sessions\(668\) \(S\)…/award\_nomination/nominated\_for\(592\) \(S\)…/military\_combatant\_group/combatants\(620\) \(S\)…/recording …/performance\_role\(402\) \(S\)…/canoodled/participant\(368\) \(S\)…/award\_honor/honored\_for\(392\) \(S\)…/marriage/spouse\(342\) \(S\)…/education/major\_field\_of\_study\(282\) \(S\)…/friendship/friend\(204\) \(S\)…/sports\_team\_roster/position\(106\) \(S\)…/location/contains,…/first\_level\_division\_of\(124\) \(I\)…/genre/titles,…film/country\(157\) \(I\)…/performance/film,…award\_winner\(963\) \(I\)…/sports\_team\_roster/team,…/american\_football/ …/position\(470\) \(I\)…location/contains,…/mailing\_address/state\_province\_region\(359\) \(I\)…/regular\_tv\_appearance/actor,…/award\_nomination/nominated\_for\(445\) \(I\)…location/contains,…/mailing\_address/country\(103\) \(I\)…/award\_nomination/nominated\_for,…/produced\_by\(497\) \(I\)…/dated/participant,\.\./romantic\_relationship/celebrity\(136\) \(I\)…/award\_nomination/nominated\_for,/film/film/music\(314\) \(I\)…/music/group\_membership/role,…/performance\_role\(246\) \(I\)…/dated/participant,…/spouse\(108\) \(I\)n\>1000n\>1000isMarriedTo\(3674\) \(S\)isLocatedIn\(5742\) \(S\)happenedIn,participatedIn\(1468\) \(I\)isLocatedIn,hasCapital\(1743\) \(I\)\_derivationally\_related\_form\(27701\) \(S\)\_verb\_group\(1060\) \(S\)…/award\_nomination/award\_nominee\(12950\) \(S\)…/award\_honor/award\_winner\(6860\) \(S\)…/track\_contribution/role\(3068\) \(S\)…/group\_membership/role\(2170\) \(S\)…/adjoining\_relationship/adjoins\(1660\) \(S\)…/friendship/participant\(1216\) \(S\)…/dated/participant\(1134\) \(S\)