Half a Link can Be Enough to Predict a Whole Link: Understanding Generalization in Knowledge Graph Foundation Models Cosimo Gregucci1 , Obaidah Theeb1 , Daniel Hernández1 , Antonio Vergari3, , Steffen Staab1,2, 1 Institute for AI, University of Stuttgart, 2 University of Southampton, 3 University of Edinburgh [email protected]
arXiv:2606.18001v1 [cs.LG] 16 Jun 2026
Abstract
(Sec. 2), a KGFM is pre-trained once and then, at test time, conditioned on a new inference graph Gi with edge set Ei , whose entities and relation names did not appear during pre-training. Then, it answers queries (h, r, ?) over a held-out edge set Et that is disjoint from Ei , but involving entities and relations in Ei , ranking every candidate entity t. Several works measure the generalization power of KGFMs in terms of aggregated raw performance, i.e., the ability to predict test triples according to ranking metrics, such as the mean reciprocal rank (MRR) averaged over several datasets (Galkin et al., 2024). At the same time, other works questioned the ability of KGFMs, and link predictors in general (Zhu et al., 2021; Trouillon et al., 2016; Sun et al., 2019; Gregucci et al., 2023; Loconte et al., 2023), to robustly generalize, also in the context of complex query answering (Gregucci et al., 2025). This is done by looking at the ability of link predictors to memorize whole links (Nickel et al., 2014), and to guarantee that training links do not leak into test KGs (Arun et al., 2025; Toutanova and Chen, 2015). In this paper, we offer a novel and finer-grained perspective on the generalization power of KGFMs by highlighting how their performance stratifies differently when predicting links for which partial information, which we call halflinks, has been observed. Predicting Eiffel Tower for the query (France, comprises, ?), the (thick) teal-colored test triple in Fig. 1(a), relies on evidence already in Gi : France is a source entity for comprises and Eiffel Tower a target one, each seen through a (thick) orange half-link in Gi . Predicting Champ de Mars for (Paris, comprises, ?), the (thick) teal-colored test triple in Fig. 1(d), has no such evidence in Gi : Paris is not a comprises-source and Champ de Mars not a comprises-target, so neither half is seen and no half-link in Gi turns orange, and the model must generalize beyond the direct evidence in Gi . We argue that the two scenarios above pose
Knowledge graph (KG) foundation models (KGFMs) are zero-shot generalizers: trained once, they can predict links on unseen graphs without retraining. However, understanding when and how they can robustly generalize across KGs is still an open question. In this paper, we shed some light on their generalization mechanisms highlighting how their performance on unseen KGs is not uniform when it comes to partially seen links, which we call half-links. In fact, we show that to predict a test triple (h, r, t) it might suffice in practice to have observed the half-link (h, r) or (r, t) in the inference graph. This yields a taxonomy of four scenarios when combinations of these half-links are observed or not. In a rigorous stratified analysis over these scenarios, we reveal that SoTA KGFMs use seen half links for predictions, while unseen half-links pose different challenges. As such, our finergrained taxonomy can be a diagnostic protocol for robust KGFM generalization and highlights where novel KGFMs can improve.
1
Introduction
Knowledge graphs (KGs) encode factual knowledge as a set of (h, r, t) triples, each linking a head entity h to a tail entity t through a relation r. KGs underpin applications such as virtual assistants and recommendation (Ilyas et al., 2022; Dong, 2018) and are used to store incomplete data (Hogan et al., 2022). Link prediction is the task of predicting unseen triples by answering queries of the form (h, r, ?), i.e., finding which entities are related to a head h through the relation name r. KG foundation models (KGFMs) make link prediction zero-shot: trained once on a collection of source graphs, they predict links on unseen graphs without retraining (Galkin et al., 2024; Lee et al., 2023). In the inductive transfer1 setting we study 1
Shared supervision. Also called fully-inductive in Geng et al. (2023).
1
(a)
SQSA
(b)
both halves seen
France
SQUA
(c)
only query half seen
France Paris
∈ Gt
comprises
Eiffel Tower
UQUA
neither half seen
Paris
Eiffel Tower
Champ de Mars
hasLandmark
(d)
France Paris
Champ de Mars
∈ Gi
only answer half seen
France Paris
Eiffel Tower
Champ de Mars
UQSA
Eiffel Tower
Champ de Mars
seen half-link
defining test triple
Figure 1: The inference graph supports test triples to fundamentally different scenarios, from both halves seen (SQSA), to one (SQUA, UQSA), to neither (UQUA). On a shared graph, the inference graph Gi (solid) and the test graph Gt (dotted), each scenario refers to a single test triple, the thick, dotted teal edge; the scenario it falls into depends on which of its halves are seen, that is, matched by a thick orange half-link in Gi . The other candidate triples stay thin and gray. Arrowhead shape encodes the relation name ( comprises, hasLandmark).
different generalization challenges to KGFMs and hence yield different performance. We formalize our argument by tracing a half-link taxonomy. We decompose each test triple (h, r, t) into a query half (h, r, ?) and an answer half (?, r, t), and call a half seen when Gi contains a witness for it, (h, r, e) ∈ Ei for the query half or (e, r, t) ∈ Ei for the answer half, and unseen otherwise; an unseen half is precisely where the model must generalize beyond the direct evidence in Gi . Crossing the two outcomes yields four scenarios, see Sec. 3.
based KGFM uses (Sec. 3). (C2) We audit the 57 benchmarks of Galkin et al. (2024) and find that the scenario composition is an artefact of split construction, with the both-halves-seen scenario dominating and the all-unseen scenario under 10% on average (Sec. 4). (C3) We disentangle what drives KGFM performance on answer-seen scenarios and on the all-unseen scenario: a frozen baseline, ULTRArand , with encoders fixed at random initialization, shows the seen-answer advantage is already architectural, and a relation-graph comparison shows that a more expressive relation graph does not always improve the all-unseen scenario (Secs. 5 and 6). (C4) We re-evaluate ULTRA, MOTIF, and TRIX over 51 zero-shot benchmarks under finer-grained reporting, showing that no KGFM is best on every scenario and that fine-tuning flips the ranking on the all-unseen scenario (Sec. 7).
Under this finer-grained view, the four scenarios pose genuinely different challenges: averaged over 51 zero-shot benchmarks, the strongest exceeds the weakest by about 0.4 MRR, a gap as large as the aggregate itself. Across the benchmarks they are unevenly present: the both-halves-seen scenario dominates and the all-unseen scenario is under 10% on average (Sec. 4). Half a link can be enough: for example, with only the answer half seen, models reach up to 1.0 MRR on NELLInductive v1 (Teru et al., 2020). This is not leakage: Ei and Et are disjoint, so the answer-half witness (e, r, t) ∈ Ei is never the test triple (h, r, t). The seen-answer advantage survives the node-degree and relation-cardinality confounders previously discussed in Mohamed et al. (2020); Bordes et al. (2013) (Sec. 5). Moreover, in the unseen answerhalves scenarios, ULTRA (Galkin et al., 2024) and MOTIF (Huang et al., 2025) are tied in overall MRR at 0.358 and 0.359, yet MOTIF clearly leads ULTRA there, 0.267 to 0.245, and almost matches TRIX (Zhang et al., 2024), the overall-best model (Table B.2, Secs. 6 and 7).
2
KGs & KGFM for link prediction
Knowledge graphs. A knowledge graph (KG) is a tuple G = (V, R, E), where V is a finite set of entities, R is a finite set of relation names, and E ⊆ V × R × V is a set of triples. Each triple (h, r, t) ∈ E links a head entity h to a tail entity t through a relation name r. Link prediction. Given a head entity h and relation name r, link prediction ranks all candidate entities e ∈ V by the predicted plausibility that (h, r, e) holds, so that the correct but unobserved target t ranks as high as possible. Under the filtered protocol (Bordes et al., 2013), letting s(h, r, e) ∈ R be the model’s score for candidate e and Fh,r,t = {t′ ∈ V : t′ ̸= t, (h, r, t′ ) ∈ E} the set of other known true candidates, the rank of the target t, denoted as rank(h, r, t) is: 1 + e ∈ V \ Fh,r,t : s(h, r, e) > s(h, r, t) .
Contributions. After establishing the KGFM setting (Sec. 2) and what it means to generalize beyond the inference graph (Sec. 3), (C1) we introduce the half-link taxonomy and its characterization in the relation graph Gir that any GNN-
Commonly both tail prediction (h, r, ?) and head 2
prediction (?, r, t) are evaluated for each test triple, and performance metrics are averaged over both directions. Performance is reported as mean reciprocal rank (MRR), the average of 1/ rank over all (triple, direction) pairs, and Hits@k (H@k), the fraction of predictions with rank ≤ k.
Gi France
Paris Eiffel Tower
Champ de Mars Gir t2t
KG foundation models for link prediction. A KG foundation model (KGFM) is pre-trained on a collection of source graphs and evaluated zero-shot on a previously unseen target KG. We focus on the inductive transfer setting, in which both entities and relation names are new at inference time. The target KG is split into the inference graph Gi = (Vi , Ri , Ei ), available at inference time, and the test graph Gt = (Vt , Rt , Et ), whose triples must be predicted. The test entities and relation names occur in the inference graph (Vt ⊆ Vi , Rt ⊆ Ri ), and the two edge sets are disjoint (Ei ∩ Et = ∅). The model receives only Gi and must rank target entities for each test triple (h, r, t) ∈ Et .
comprises
t2h
hasLandmark
Figure 2: Example inference graph Gi (top) and its ULTRA-derived relation graph Gir (bottom). For visual simplification, we omit (i) h2t edges, since they are the reverse of t2h, and (ii) trivial h2h and t2t self-loops.
occurrence motifs between relation names. MOTIF (Huang et al., 2025) extends it to higher-order (k ≥ 3) co-occurrence patterns. TRIX (Zhang et al., 2024) augments the binary relation graph in ULTRA by recording which entities participate in each co-occurrence, and additionally couples the relation and entity encoders through iterative updates. Both MOTIF and TRIX are strictly more expressive than ULTRA, extending it along orthogonal axes: the former through motif length, the latter through per-entity granularity. We exclude FLOCK (Kim et al., 2025), which replaces deterministic messagepassing with probabilistic random-walk ensembles, as it requires substantially higher computational cost while achieving comparable zero-shot performance. See App. J for a detailed cost-performance analysis.
GNN-based KGFMs use a two-encoder pipeline in which relation representations condition entitylevel reasoning. Given G, they first construct its relation graph G r — with relation names as nodes and co-occurrence motifs as directed edges (Lee et al., 2023; Galkin et al., 2024). For example, following Galkin et al. (2024), a t2t co-occurrence indicates that two relation names share a tail entity, while a t2h co-occurrence indicates that the tail of one is the head of the other. In the inference graph in Fig. 2 (top), comprises and hasLandmark share the tail Eiffel Tower, and the tail of comprises (Paris) is the head of hasLandmark, so the relation graph in Fig. 2 (bottom) links them by both a t2t and a t2h edge. A relation encoder then runs message-passing on G r to produce relation embeddings, which feed into an entity encoder that runs message-passing on G to compute entity representations (Zhu et al., 2021, 2023; Zhang and Yao, 2022).
3
What is generalizing beyond the inference graph?
Prior work questions whether KGFMs, and link predictors in general (Zhu et al., 2021; Trouillon et al., 2016; Sun et al., 2019), robustly generalize rather than memorize links (Nickel et al., 2014), also in the context of complex query answering (Gregucci et al., 2025). The held-out edge set Et is kept disjoint from Ei , so that no test triple leaks into the inference graph (Arun et al., 2025; Toutanova and Chen, 2015). This guarantee applies at the granularity of the whole triple. Yet even when (h, r, t) ∈ / Ei , Ei may still contain partial information about it. We make this notion precise and answer the following research question: (RQ1) What constitutes structural evidence in Gi for a test triple (h, r, t), and how does it translate to the relation graph Gir formulation? Half-link taxonomy. We decompose each test triple (h, r, t) into two half-links: the query half (h, r, ?) and the answer half (?, r, t). A half is seen when Gi contains a witness for it, a triple that attests it, and unseen otherwise; an unseen
At inference, both encoders run message-passing on the new Gi (Fig. 2 (top)) and its derived Gir (Fig. 2 (bottom)), computing fresh entity and relation embeddings specific to the unseen KG; these are then used to score each test triple (h, r, t) ∈ Et . Because neither encoder learns entity- or relationspecific parameters, this process generalizes zeroshot to any unseen KG with arbitrary vocabularies. We consider three GNN-based KGFMs that instantiate this framework. ULTRA (Galkin et al., 2024) builds the relation graph from binary co3
SQSA
UQUA
both halves seen
neither half seen h2h
h2h France
t2t
t2h Paris
Eiffel Tower
comprises
t2h
France
t2h Paris
hasLandmark
Champ de Mars
Eiffel Tower
comprises
t2t
t2h
hasLandmark
Champ de Mars
Gir motif
Gtr motif
induced Gir motif
induced Gtr motif
Figure 3: UQUA test triples can map to potentially missing relation-graph links. We draw the co-occurrence motifs each test triple would induce, solid if in Gir and dotted if in Gtr ; those induced by the specific test triple (teal, left graph of each pair) are thick purple. SQSA (left): the induced t2t between comprises and hasLandmark is present (purple solid), since by construction the answer entity has an incoming comprises edge. UQUA (right): the induced motifs might be missing (purple dotted), recoverable only through incidental co-occurrences (see Sec. 6).
half is precisely where the model must generalize beyond the direct evidence in Gi . Concretely, the query half (h, r, ?) is seen iff ∃ e ∈ Vi \ {t} : (h, r, e) ∈ Ei , and the answer half (?, r, t) is seen iff ∃ e ∈ Vi \ {h} : (e, r, t) ∈ Ei . Crossing the two binary outcomes yields a 2×2 half-link taxonomy, namely SQSA (Seen Query, Seen Answer), SQUA (Seen Query, Unseen Answer), UQSA (Unseen Query, Seen Answer), and UQUA (Unseen Query, Unseen Answer). The taxonomy classifies each test triple (h, r, t) ∈ Et relative to Gi , independently of any particular method.
depends on whether each leaves different structural evidence in Gir . Unlike the taxonomy itself, how it maps onto Gir is method-dependent, since it hinges on the motif vocabulary of the method. We therefore ask, for each test triple (h, r, t) ∈ Et , which of the co-occurrence motifs it induces are already present in Gir , and which might be missing. Fig. 3 illustrates the two categorical poles on the running example. For SQSA (left), the test triple (France, comprises, Eiffel Tower) would induce a t2t edge between comprises and hasLandmark, sharing its tail Eiffel Tower with the hasLandmark edge into it; this edge is already in Gir , because by definition of SQSA Eiffel Tower has an incoming comprises edge (from Champ de Mars). For UQUA (right), the test triple (Paris, comprises, Champ de Mars) would induce a t2h self-loop on comprises, since its tail Champ de Mars is the head of an existing comprises edge (out to Eiffel Tower); this edge is in Gtr , because by definition of UQUA neither half is seen, so no other comprises edge ends at Champ de Mars or starts at Paris to close the chain. In practice, though, it may still appear through incidental coverage, for example if Gi contained (Europe, comprises, France), which would chain with the comprises edge out of France to form the same t2h, while leaving both halves of the test triple unseen. We verify that this incidental coverage is common in ULTRA, and rare in TRIX (Sec. 6). For SQUA and UQSA, one half is supplied directly, the other only incidentally. The breakdown for all four scenarios on the running example appears in Fig. A.1.
Fig. 1 shows one running example: the inference graph Gi (solid) and the test graph Gt (dotted). Each scenario considers a single test triple (thick teal); a half is seen (thick orange) when Gi contains a witness triple for it. (SQSA) The test triple (France, comprises, Eiffel Tower) in Fig. 1(a) relies on evidence already in Gi : France is a comprises-source and Eiffel Tower a comprises-target, each seen through an orange half-link. (SQUA) The test triple (France, comprises, Champ de Mars) in Fig. 1(b) keeps a seen query-half, France still a comprisessource, but Champ de Mars is not a comprisestarget, so the answer half is unseen. (UQSA) The dual of SQUA (Fig. 1(c)): the answer half is seen, the query half unseen. (UQUA) The test triple (Paris, comprises, Champ de Mars) in Fig. 1(d) has no such evidence: Paris is not a comprises-source and Champ de Mars not a comprisestarget, so neither half is seen and the model must generalize beyond the direct evidence in Gi . Sec. 4 measures how many test triples fall in each scenario across existing benchmarks.
4
Relation-graph perspective. Since Gir is built exclusively from Gi , it is the only structural input the relation encoder sees, so whether the four scenarios pose different challenges to the encoder
How many links from each scenario in current benchmarks?
In this section, we analyze the scenario composition of these benchmarks and answer the following research question: (RQ2) How are test triples 4
geneous split criteria: random train/test ratios for Family 1, entity disjointness for Family 2 (Teru et al., 2020), and relation-overlap fractions for Family 3 (Lee et al., 2023). Aggregate MRR mixes the scenarios in these split-driven proportions, whereas classifying each test triple with our taxonomy makes scenario-stratified MRR independent of that mix.
Benchmarks (N ) SQSA UQSA SQUA UQUA Family 1 (16) Family 2 (18) Family 3 (23)
54.3% 21.7% 18.1% 32.5% 28.8% 28.8% 38.1% 28.0% 28.0%
5.9% 9.9% 5.9%
Overall (57)
40.9% 26.5% 25.5%
7.1%
Table 1: Fully unseen test triples (UQUA) account for under 10% of test triples on average in every family, while SQSA covers the largest share. N = number of benchmarks averaged per row (unweighted). Per-benchmark breakdown in Table K.4.
Takeaway 1. Aggregate MRR can hide where KGFMs suffer to generalize, due to different proportions of queries in SQSA, SQUA, UQSA, and UQUA. We advise reporting it along our stratified MRR, to provide a finer-grained understanding.
distributed across the four scenarios on the benchmarks used to evaluate KGFMs? Table 1 reports average scenario proportions across three benchmark families, ordered by how much of the inference graph Gi entity and relation name vocabularies already appears in the training graph G the benchmark was built from:2 “Family 1” shares both entities and relation names, “Family 2” shares relation names but introduces new entities, and “Family 3” shares neither. We evaluate every benchmark in the inductive transfer setting of Sec. 2; the family reflects how each benchmark was built, not what the model sees. For each test triple (h, r, t) ∈ Et scored in both directions, head prediction (?, r, t) and tail prediction (h, r, ?) swap which half is the query and which is the answer; consequently, every test triple labelled SQUA for one direction is labelled UQSA for the other, and the two scenarios contain the same number of triples on a per-benchmark basis. This identity is reflected in the Family 2 and Family 3 rows of Table 1; the small gap in the Family 1 row stems from three of the sixteen benchmarks that score tail predictions only (App. K). Across all three families, the largest share of test triples falls in SQSA, where both halves are seen, while fully unseen UQUA triples account for less than 10% on average. See Table K.4 for per-benchmark figures.
5
What drives per-scenario KGFM performance?
A test triple decomposes into a query half and an answer half, each seen or unseen in Gi (Sec. 3). The role each seen half plays in the link prediction performance of existing KGFMs is unclear, so we answer the following research question: (RQ3) How does each seen half-link signal, query or answer, affect KGFM link prediction performance? We aggregate scenario-stratified MRR over the 51 zero-shot KGs where every scenario is non-empty, excluding the usual pretraining graphs (FB15k237 (Toutanova and Chen, 2015), CoDEx Medium (Safavi and Koutra, 2020), WN18RR (Dettmers et al., 2018)) and the benchmarks with an empty scenario. Architectural asymmetry of the two halves. Although the two halves are defined symmetrically over Gi , they play architecturally distinct roles in any GNN-based entity encoder. A seen answer-half provides an unambiguous positive signal: when t has incoming r-typed edges in Gi , r-typed messages reach the representation of t through standard message passing regardless of the structure surrounding h. A seen query-half is structurally ambiguous: when h has outgoing r-typed edges in Gi , the representation of h encodes its existing r-tails as an implicit prior over plausible answers. This prior aligns with the true target t only if t structurally resembles those existing r-tails, a condition the architecture does not guarantee (Wu et al., 2022); otherwise the prior competes against t. The next two subsections confirm this asymmetry with a frozen baseline and a
An artefact of split design. The scenario proportions in Table 1 are a side effect of how these benchmarks were constructed, not a deliberate design target. Per-benchmark SQSA proportions range from 0% (NELL995 (Xiong et al., 2017), Metafam (Zhou et al., 2023)) to 98% (Hetionet (Himmelstein et al., 2017)), driven by hetero2
Families 1, 2, and 3 are also known as transductive, inductive entity, and fully inductive (Galkin et al., 2024), after the original setting each benchmark was created for; Tables K.1 to K.3 list the benchmarks in each.
5
Benchmarks Scenario ULTRA ULTRArand
pedia100k): the seen-query prior ranks the known answers of h above the held-out target, competing against the correct prediction rather than supporting it. See App. F and Table F.1 for more details.
∆
Fam. 1 (12)
UQSA SQUA
0.469 0.059
0.338 0.042
+0.131 +0.017
Fam. 2 (17)
UQSA SQUA
0.582 0.226
0.472 0.231
+0.110 −0.005
Fam. 3 (22)
UQSA SQUA
0.562 0.103
0.422 0.097
+0.140 +0.006
All (51)
UQSA SQUA
0.547 0.133
0.419 0.129
+0.128 +0.004
Takeaway 2. The two seen halves are asymmetric: the seen answer-half is a positive signal while the seen query-half is not. Only scenario-stratified evaluation reveals whether a model overcomes it.
Table 2: Pre-training mainly amplifies the seen answer signal. UQSA scores far above SQUA, and pretraining gains are on UQSA. ∆ = ULTRA − ULTRArand per scenario; bold marks the larger ∆ per family.
6
What drives UQUA generalization?
For a UQUA test triple, neither half-link is seen (Fig. 1(d)), so the inference graph gives the entity encoder no direct signal (Sec. 5). The relation graph can still contribute: Sec. 3 showed that the motif a test triple would induce may already appear in Gir through incidental coverage, even when both halves are unseen. Intuitively, a more expressive relation graph could raise this incidental coverage, and with it UQUA performance. We ask whether it always does: (RQ4) Does a more expressive relation graph always improve generalization in UQUA, where Gi provides no direct half-link evidence? We compare the three KGFMs of Sec. 2 (ULTRA, MOTIF, TRIX) on the same 51 zero-shot KGs as Sec. 5, and additionally evaluate the singlepass ablation TRIXnoiter (Zhang et al., 2024). ULTRA, MOTIF, and TRIXnoiter share the entity-level GNN (Zhu et al., 2021) and differ in how they construct the relation graph, so their comparison isolates the relation-graph design. Full TRIX adds iterative entity–relation refinement on top of the same relation graph of TRIXnoiter , thus highlighting the contribution of the iterative mechanism rather than of per-entity granularity. MOTIF leads UQUA among the three relation-graph designs. Among the three relationgraph designs, MOTIF posts the highest UQUA MRR in every benchmark family, while TRIXnoiter falls below ULTRA (Table 3). The pattern follows from whether the co-occurrence a UQUA test triple would induce is incidentally covered in Gir . The higher-order motifs of MOTIF are entity-agnostic, so a co-occurrence Gi exposes incidentally covers any UQUA test triple that would induce it. TRIXnoiter instead tags each co-occurrence with the entity that instantiates it in Gi . A UQUA test triple instantiates its co-occurrence at a different entity, so the tagged motif does not match: the entity tag leaves the test triple uncovered, contributing no sig-
distractor diagnostic. We verify the asymmetry is not an artefact of node degree (Mohamed et al., 2020) or of the relation-cardinality classes of Bordes et al. (2013) in Apps. G and H. Architectural vs. learned origin. We isolate the architectural contribution with a frozen baseline, ULTRArand : ULTRA with the relation encoder, the entity encoder, and the relation-graph initial features fixed at random initialization, and only the score layer trained (Degraeve et al., 2022; Bui et al., 2025), so that any separation it shows across scenarios is attributable to the architecture rather than to learning. We read each seen half off the scenario that isolates it: UQSA, where only the answer half is seen, and SQUA, where only the query half is seen. On the 51-KG average (Table 2) the frozen baseline already scores far higher on UQSA than on SQUA (0.419 vs. 0.129): even without learning, a seen answer-half helps while a seen query-half does not. Pre-training (Sec. 2) lifts UQSA from 0.419 to 0.547 (∆ = +0.128) while leaving SQUA essentially unchanged (0.129 → 0.133). Pre-training therefore amplifies the answer-half signal the architecture already supplies via the r-typed messages. Per-family rows and details on ULTRArand implementation are in App. E. Distractor diagnostic. A seen query-half has existing r-answers in Gi , the set D(h, r) = {e : (h, r, e) ∈ Ei }. To test whether the seen-query signal acts as a distractor, we count how often a member of D(h, r) outscores the held-out target t. Because the entities in D(h, r) are themselves correct answers, the filtered protocol removes them from the ranking, so we read their scores from the raw output. A member of D(h, r) outscores t on most SQUA triples across Family 1 benchmarks we selected (up to 91% on WDsinger, 66% on DB6
Model ULTRA MOTIF TRIXnoiter
51 zero-shot KGs as in Sec. 5, using the publicly released pre-trained checkpoint of each model. We ask: (RQ5) How do SoTA KGFMs perform zeroshot across the four scenarios and where does perbenchmark fine-tuning help? The per-scenario spread is as large as the aggregate. Table 4 reports scenario-stratified zeroshot MRR for ULTRA, MOTIF and TRIX. The ordering UQSA > SQSA > UQUA > SQUA holds for every model and family. It follows from the asymmetry of the two halves (Sec. 5): a seen answer-half is an unambiguous positive signal, whereas a seen query-half is a distractor that competes against the target. UQSA carries the positive signal with no distractor and ranks highest; SQSA adds the distractor and ranks second; UQUA, with neither half seen, ranks third; and SQUA, where the distractor acts with no positive signal to offset it, ranks lowest, below even the all-unseen UQUA. The spread this produces is wide: on the 51-KG average the strongest scenario exceeds the weakest by 0.37 to 0.41 MRR (UQSA vs. SQUA), as large as the Orig aggregate itself. The aggregate hides it, because Orig aggregate sits closest to SQSA, the largest scenario in current benchmarks (Table 1). This gap between the aggregate and the per-scenario view is clearest on UQUA, which isolates generalization beyond Gi . ULTRA and MOTIF tie on the aggregate (0.358 vs. 0.359) but not on UQUA, where MOTIF leads (0.267 vs. 0.245); TRIX leads on the aggregate (0.378 vs. 0.359) but its gain does not reach UQUA, where it ties MOTIF (0.268 vs. 0.267), consistent with Sec. 6. Fine-tuning reverses model ranking and partially recovers unseen answer-halves scenarios. The lower performance on the unseen answerhalves scenarios (SQUA, UQUA) has two possible sources: structure absent from Gi , or structure present in Gi that the generic pre-trained weights do not extract. We disentangle them with perbenchmark fine-tuning on Family 3, the only family on which fine-tuning preserves the inductive transfer setting (Sec. 2): there the training graph G is disjoint from Gi in both entities and relation names. Each pre-trained checkpoint is fine-tuned on the training graph G of its benchmark following the fine-tuning protocol of ULTRA,4 then message passing is performed on Gi exactly as in the zero-
Fam. 1 (12) Fam. 2 (17) Fam. 3 (22) All (51) 0.202 0.216 0.145
0.316 0.352 0.256
0.214 0.229 0.159
0.245 0.267 0.188
Table 3: MOTIF has the relation-graph design that best transfers to unseen halves, while TRIXnoiter falls below ULTRA; full TRIX closes this gap through iterative entity–relation coupling (Table D.1). Unweighted mean UQUA MRR per benchmark family on the 51 zero-shot KGs; bold marks the best within a family.
nal. We confirm this with the incidental-coverage measurement introduced in Sec. 3, computed per model and test triple (App. C). Across four benchmarks spanning different UQUA shares and performances, the relation graph Gir of ULTRA covers 90.8% to 99.5% of induced Gtr motifs, whereas the Gir of TRIXnoiter covers under 0.6% of induced Gtr motifs (Table C.1). The entity-tagged relation graph thus leaves TRIXnoiter with little usable signal, which explains the UQUA result. Iterative entity–relation coupling recovers UQUA. Full TRIX nevertheless closes this gap through a different route. Its iterative entity– relation coupling, learned during pre-training, likely aligns the representations of entities that participate in similar relations, supplying at zero-shot inference the entity-agnostic transfer that the entitytagged relation graph alone cannot. The recovery is sharp, +0.08 MRR over TRIXnoiter on the 51-KG average, far larger than the gain of iterative coupling on any other scenario (App. D, Table D.1). Improving UQUA is therefore not only a matter of a more expressive relation graph: a pre-trained architectural mechanism such as iterative entity– relation coupling is a complementary lever. Takeaway 3. Stratifying by scenario reveals that UQUA is where relation-graph design matters most. By looking at aggregate MRR alone, performance gains cannot be directly attributed to the relation-graph design.
7
KGFM performance spreads widely
We now evaluate the three KGFMs of Sec. 2 together under our scenario-stratified protocol, which we package as a reusable diagnostic,3 on the same
and benchmarks; https://github.com/cgregucci/ KG-foundation-models. 4 TRIX implements a zero-shot fallback which we do not enable so that any recovery is attributable to fine-tuning alone.
3
We bring ULTRA, MOTIF, and TRIX into a single unified repository, together with the scenario-labelling code, so the protocol can be re-applied to new KGFMs
7
Orig
SQSA
UQSA
SQUA
UQUA
Fam. 1 (12)
ULTRA MOTIF TRIX
0.293 0.286 0.314
0.269 0.259 0.285
0.469 0.444 0.491
0.059 0.079 0.088
0.202 0.216 0.225
Fam. 2 (17)
ULTRA MOTIF TRIX
0.419 0.422 0.435
0.487 0.473 0.483
0.582 0.569 0.585
0.226 0.254 0.273
0.316 0.352 0.343
Fam. 3 (22)
ULTRA MOTIF TRIX
0.346 0.349 0.368
0.385 0.372 0.370
0.562 0.545 0.579
0.103 0.142 0.166
0.214 0.229 0.235
All (51)
ULTRA MOTIF TRIX
0.358 0.359 0.378
0.392 0.379 0.388
0.547 0.529 0.561
0.133 0.164 0.184
0.245 0.267 0.268
shot setting. So comparing zero-shot and fine-tuned blocks in Table 5 isolates how much more in-Gi signal the fine-tuned weights extract (detailed results in App. I). Since only the weights change, any unseen answer-half score that fine-tuning recovers is already extractable from Gi . On the seen answer-halves scenarios SQSA and UQSA, all models improve and the leader is unchanged, as fine-tuning sharpens how each model exploits the seen-answer signal of Sec. 5. On the unseen answer-halves scenarios SQUA and UQUA the ranking instead flips: before, TRIX lead both, but after fine-tuning it falls to last on both, while the entity-agnostic ULTRA and MOTIF rise to the top: ULTRA goes from 0.214 to 0.250 on UQUA and MOTIF scores the best on SQUA. Whether a model improves or regresses on the unseen answerhalves scenarios follows the relation-graph design of Sec. 6: fine-tuning sharpens the entity-agnostic structure of ULTRA and MOTIF, but degrades the per-entity structure of TRIX.
Model
SQSA
UQSA
SQUA
UQUA
ULTRA MOTIF TRIX
0.385 0.372 0.370
0.562 0.545 0.579
0.103 0.142 0.166
0.214 0.229 0.235
ULTRA MOTIF TRIX
0.399 0.396 0.394
0.580 0.583 0.597
0.149 0.157 0.115
0.250 0.235 0.189
Table 5: Fine-tuning flips the answer-unseen model ranking. MRR on Family 3 benchmarks; in each block (zero-shot, fine-tuned) and scenario, the winning model is colored: ULTRA, MOTIF, TRIX.
as the aggregate yet invisible in it. The spread follows an asymmetry between the two halves, a seen answer-half being a positive signal and a seen query-half a distractor; the seen answer-half alone can be enough to predict a whole link. The taxonomy extends to the relation graph that GNN-based KGFMs build, where it acts as a proxy for when relation-graph expressiveness helps. Such expressiveness lifts the all-unseen scenario in particular, and only when the relation graph is entity-agnostic rather than per-entity.
Takeaway 4. Our half-link taxonomy exposes a wide spread in KGFM performance across scenarios. Fine-tuning further shows that the unseen answer-half shortfall is partly an extraction gap, and thus a target for designing future KGFMs.
8
Table 4: UQSA is the highest-MRR scenario and SQUA the lowest for every model and family, and the Orig aggregate sits closest to SQSA, the largest scenario, hiding the spread over the other three. Values are unweighted mean MRR per benchmark family on the zeroshot KGs with at least one triple in every scenario (per-family counts in parentheses); bold marks the best of the three models per family and scenario. Perdataset MRR in App. B.
Zero-shot
Model
Fine-tuned
Benchmarks
The four scenarios occur in different, split-driven proportions, which the aggregate cannot disentangle. We therefore release our scenario-stratified protocol as a reusable diagnostic and recommend reporting it alongside the aggregate, since classifying each link by scenario makes this report independent of how benchmarks are split. Finetuning further locates the unseen answer-half shortfall as partly an extraction gap, signal already in the inference graph that current models leave unused, marking it as a concrete target for future KGFMs. Our analysis covers GNN-based KGFMs; extending it to non-GNN-based approaches such as FLOCK (Kim et al., 2025), which replaces message passing with probabilistic random-walk ensembles,
Conclusion
Prior work measures KGFM generalization on the whole link, by aggregate MRR. We provide a finergrained understanding by decomposing each link into two half-links, a query half and an answer half, each seen or unseen in the inference graph, into four scenarios. Stratified over them, performance varies by about 0.4 MRR, a spread as wide 8
is a natural next step.
Edinburgh, funded by ELLIS Unit Stuttgart and by G-Research research grant CG20251209. DH was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy – EXC 2120/2 – 390831618. Halfling icon generate by Gemini.
Limitations Our analysis covers GNN-based KGFMs, among which ULTRA, MOTIF, and TRIX. The asymmetry we identify between a seen answer-half and a seen query-half, and the finding that relationgraph expressiveness lifts the all-unseen scenario only when the relation graph is entity-agnostic, are therefore established for this model family. Yet on FLOCK, an architecture with neither message passing nor relation graph, preliminary results (App. J) already reproduce the same pattern; a more extensive evaluation is needed but costly, at about 186× the inference compute of TRIX (Table J.1). The phenomenon may therefore extend beyond the family our mechanism explains; understanding why is a separate line of work, as is whether text- and language-model-based link predictors exhibit it.
References Arvindh Arun, Sumit Kumar, Mojtaba Nayyeri, Bo Xiong, Ponnurangam Kumaraguru, Antonio Vergari, and Steffen Staab. 2025. SEMMA: A semantic aware knowledge graph foundation model. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, EMNLP 2025, Suzhou, China, November 4-9, 2025, pages 31825– 31848. Association for Computational Linguistics. Antoine Bordes, Nicolas Usunier, Alberto GarcíaDurán, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multirelational data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pages 2787–2795.
Author contributions CG conceived the initial idea that the inference graph supports test triples to different scenarios in SoTA KGFMs. CG, DH, and OT designed the half-link taxonomy, and OT performed the initial experiments to validate the idea. CG wrote the first draft of the manuscript, drew all the figures, designed and ran all the experiments, with the exception of FLOCK, ULTRA, and ULTRArand , which were performed by OT. SS suggested extending the analysis to the relation level, and AV had the intuition to characterize the taxonomy in terms of the relation graph. All authors critically revised the paper. AV and SS supervised all phases of the project and gave feedback.
Thu Bui, Carola-Bibiane Schönlieb, Bruno Ribeiro, Beatrice Bevilacqua, and Moshe Eliasof. 2025. On the effectiveness of random weights in graph neural networks. CoRR, abs/2502.00190. Yihong Chen, Pasquale Minervini, Sebastian Riedel, and Pontus Stenetorp. 2021. Relation prediction as an auxiliary training objective for improving multirelational graph representations. In 3rd Conference on Automated Knowledge Base Construction, AKBC 2021, Virtual, October 4-8, 2021. Vic Degraeve, Gilles Vandewiele, Femke Ongenae, and Sofie Van Hoecke. 2022. R-GCN: the R could stand for random. CoRR, abs/2203.02424. Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2d knowledge graph embeddings. In Proceedings of the ThirtySecond AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 1811–1818. AAAI Press.
Acknowledgements AV was supported by the “UNREAL: Unified Reasoning Layer for Trustworthy ML” project (EP/Y023838/1) selected by the ERC and funded by UKRI EPSRC. CG and OT were funded by the CHIPS Joint Undertaking (JU) under grant agreement No. 101140087 (SMARTY), and by the German Federal Ministry of Education and Research (BMBF) under the sub-project with funding number 16MEE0444. CG and OT acknowledge compute time on HoreKa HPC (NHR@KIT), funded by the BMBF and the MWK of Baden-Württemberg through the NHR program, with additional support from the DFG. CG worked on the paper partially during a research stay at the University of
Boyang Ding, Quan Wang, Bin Wang, and Li Guo. 2018. Improving knowledge graph embedding using simple constraints. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers, pages 110–121. Association for Computational Linguistics. Xin Luna Dong. 2018. Challenges and innovations in building a product knowledge graph. In Proceedings
9
and Miguel A. Romero Orth. 2025. How expressive are knowledge graph foundation models? In Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, Proceedings of Machine Learning Research. PMLR / OpenReview.net.
of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018, page 2869. ACM. Mikhail Galkin, Max Berrendorf, and Charles Tapley Hoyt. 2022. An open challenge for inductive link prediction on knowledge graphs. CoRR, abs/2203.01520.
Ihab F. Ilyas, Theodoros Rekatsinas, Vishnu Konda, Jeffrey Pound, Xiaoguang Qi, and Mohamed A. Soliman. 2022. Saga: A platform for continuous construction and serving of knowledge at scale. In SIGMOD ’22: International Conference on Management of Data, Philadelphia, PA, USA, June 12 - 17, 2022, pages 2259–2272. ACM.
Mikhail Galkin, Xinyu Yuan, Hesham Mostafa, Jian Tang, and Zhaocheng Zhu. 2024. Towards foundation models for knowledge graph reasoning. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.
Jinwoo Kim, Xingyue Huang, Krzysztof Olejniczak, Kyungbin Min, Michael M. Bronstein, Seunghoon Hong, and İsmail İlkan Ceylan. 2025. Flock: A knowledge graph foundation model via learning on random walks. CoRR, abs/2510.01510.
Yuxia Geng, Jiaoyan Chen, Jeff Z. Pan, Mingyang Chen, Song Jiang, Wen Zhang, and Huajun Chen. 2023. Relational message passing for fully inductive knowledge graph completion. In 39th IEEE International Conference on Data Engineering, ICDE 2023, Anaheim, CA, USA, April 3-7, 2023, pages 1221–1233. IEEE.
Jaejun Lee, Chanyoung Chung, and Joyce Jiyoung Whang. 2023. Ingram: Inductive knowledge graph embedding via relation graphs. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, Proceedings of Machine Learning Research, pages 18796–18809. PMLR.
Cosimo Gregucci, Mojtaba Nayyeri, Daniel Hernández, and Steffen Staab. 2023. Link prediction with attention applied on multiple knowledge graph embedding models. In Proceedings of the ACM Web Conference 2023, WWW 2023, Austin, TX, USA, 30 April 2023 4 May 2023, pages 2600–2610. ACM.
Shuwen Liu, Bernardo Cuenca Grau, Ian Horrocks, and Egor V. Kostylev. 2021. INDIGO: gnn-based inductive knowledge graph completion using pair-wise encoding. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 2034–2045.
Cosimo Gregucci, Bo Xiong, Daniel Hernández, Lorenzo Loconte, Pasquale Minervini, Steffen Staab, and Antonio Vergari. 2025. Is complex query answering really complex? In Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, Proceedings of Machine Learning Research. PMLR / OpenReview.net.
Lorenzo Loconte, Nicola Di Mauro, Robert Peharz, and Antonio Vergari. 2023. How to turn your knowledge graph embeddings into generative models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.
Takuo Hamaguchi, Hidekazu Oiwa, Masashi Shimbo, and Yuji Matsumoto. 2017. Knowledge transfer for out-of-knowledge-base entities : A graph neural network approach. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, pages 1802–1808. ijcai.org.
Xin Lv, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Wei Zhang, Yichi Zhang, Hao Kong, and Suhui Wu. 2020. Dynamic anticipation and completion for multi-hop reasoning over sparse knowledge graph. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 5694–5703. Association for Computational Linguistics.
Daniel Scott Himmelstein, Antoine Lizee, Christine Hessler, Leo Brueggeman, Sabrina L Chen, Dexter Hadley, Ari Green, Pouya Khankhanian, and Sergio E Baranzini. 2017. Systematic integration of biomedical knowledge prioritizes drugs for repurposing. elife, 6:e26726.
Farzaneh Mahdisoltani, Joanna Biega, and Fabian M. Suchanek. 2015. YAGO3: A knowledge base from multilingual wikipedias. In Seventh Biennial Conference on Innovative Data Systems Research, CIDR 2015, Asilomar, CA, USA, January 4-7, 2015, Online Proceedings. www.cidrdb.org.
Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia d’Amato, Gerard de Melo, Claudio Gutierrez, Sabrina Kirrane, José Emilio Labra Gayo, Roberto Navigli, Sebastian Neumaier, Axel-Cyrille Ngonga Ngomo, Axel Polleres, Sabbir M. Rashid, Anisa Rula, Lukas Schmelzeisen, Juan F. Sequeda, Steffen Staab, and Antoine Zimmermann. 2022. Knowledge graphs. ACM Comput. Surv., 54(4):71:1–71:37.
Chaitanya Malaviya, Chandra Bhagavatula, Antoine Bosselut, and Yejin Choi. 2020. Commonsense knowledge base completion with structural and semantic context. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The
Xingyue Huang, Pablo Barceló, Michael M. Bronstein, İsmail İlkan Ceylan, Mikhail Galkin, Juan L. Reutter,
10
Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. Deeppath: A reinforcement learning method for knowledge graph reasoning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 564– 573. Association for Computational Linguistics.
Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 2925–2933. AAAI Press. Aisha Mohamed, Shameem Puthiya Parambath, Zoi Kaoudi, and Ashraf Aboulnaga. 2020. Popularity agnostic evaluation of knowledge graph embeddings. In Proceedings of the Thirty-Sixth Conference on Uncertainty in Artificial Intelligence, UAI 2020, virtual online, August 3-6, 2020, Proceedings of Machine Learning Research, pages 1059–1068. AUAI Press.
Yongqi Zhang and Quanming Yao. 2022. Knowledge graph reasoning with relational digraph. In WWW ’22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25 - 29, 2022, pages 912–924. ACM. Yucheng Zhang, Beatrice Bevilacqua, Mikhail Galkin, and Bruno Ribeiro. 2024. TRIX: A more expressive model for zero-shot domain transfer in knowledge graphs. In Learning on Graphs Conference, 26-29 November 2024, Virtual, Proceedings of Machine Learning Research, page 12. PMLR.
Maximilian Nickel, Xueyan Jiang, and Volker Tresp. 2014. Reducing the rank in relational factorization models by including observable patterns. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 1179–1187.
Jincheng Zhou, Beatrice Bevilacqua, and Bruno Ribeiro. 2023. A multi-task perspective for link prediction with new relation types and nodes. In NeurIPS 2023 Workshop: New Frontiers in Graph Learning.
Tara Safavi and Danai Koutra. 2020. Codex: A comprehensive knowledge graph completion benchmark. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 8328– 8350. Association for Computational Linguistics.
Zhaocheng Zhu, Xinyu Yuan, Michael Galkin, LouisPascal A. C. Xhonneux, Ming Zhang, Maxime Gazeau, and Jian Tang. 2023. A*net: A scalable pathbased reasoning approach for knowledge graphs. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.
Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
Zhaocheng Zhu, Zuobai Zhang, Louis-Pascal A. C. Xhonneux, and Jian Tang. 2021. Neural bellman-ford networks: A general graph neural network framework for link prediction. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 29476–29490.
Komal K. Teru, Etienne G. Denis, and William L. Hamilton. 2020. Inductive relation prediction by subgraph reasoning. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 1318 July 2020, Virtual Event, Proceedings of Machine Learning Research, pages 9448–9457. PMLR. Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, CVSC 2015, Beijing, China, July 26-31, 2015, pages 57–66. Association for Computational Linguistics. Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, JMLR Workshop and Conference Proceedings, pages 2071–2080. JMLR.org. Yingxin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua. 2022. Discovering invariant rationales for graph neural networks. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
11
A
C
Per-scenario relation-graph impact
This section reports the incidental-coverage measurement that Sec. 6 relies on, computed per model and test triple: how many of the co-occurrence motifs a test triple induces are already present in Gir , and how many are missing. For each directed test triple (h, r, t) we count the missing ones, the new relation-graph edges that appending this single triple to Gi would induce at inference time. Let R0 be the relation-graph edge set induced by the inference graph Gi (augmented with inverse relations) that the model conditions on at test time, and R1 the same edge set after appending the test triple in both directions, forward (h, r, t) and inverse (t, r+|Ri |, h); then nadded = |R1 \ R0 |. For ULTRA, edges are 3-tuples (r1 , τ, r2 ) over the four binary motif types τ ∈ {h2h, t2h, t2t, h2t}; for TRIX, edges carry the witnessing entity and become 4-tuples (r1 , r2 , e, τ ) over the same four types (Zhang et al., 2024). Each test triple is scored in both directions and so appears as two directed triples; SQSA triples have nadded ≡ 0 by construction, since both halves of the triple are already seen in Gi . Benchmark sub-selection. We select four benchmarks that span two design axes: the share of UQUA in the test set, and the gap between the UQUA MRR of ULTRA and its aggregate (Orig) MRR. UQUA shares range from 1.7% (FB15k237) through 2.8% (ILPC2022SmallInductive) and 5.7% (WKIngram:25) to 16.4% (WDsinger). The corresponding UQUA−Orig gap for ULTRA ranges from −0.144 (WKIngram:25, aggregate 0.310) through −0.061 (FB15k237) and +0.018 (ILPC2022SmallInductive) up to +0.223 (WDsinger, aggregate 0.386). Across the four the gap changes sign, placing UQUA below the aggregate on two benchmarks and above it on the other two. Per-scenario relation-graph coverage. Table C.1 reports the diagnostic split by scenario for both ULTRA and TRIX side-by-side. We report, per (benchmark, split) cell: the number of directed test triples (ntotal ); the baseline edge count |R0 | for each model (only on the Orig row, as |R0 | is a property of the benchmark, not the split); the share of triples that induce at least one co-occurrence motif absent from Gir (nadded ≥ 1, a dotted Gtr edge in Fig. 3; %≥1 ); and the mean number of such motifs per triple. For ULTRA, almost every test triple already has all its induced motifs present in Gir : its
The four scenarios leave different structural evidence in the relation graph Gir . Fig. A.1 extends the SQSA/UQUA contrast of Fig. 3 to all four scenarios on the running example, drawing for each the co-occurrence motifs the test triple induces and whether they are already present in Gir . In the two intermediate scenarios, SQUA and UQSA, exactly one half is seen, so the seen half supplies its motifs directly while the unseen half is covered only incidentally.
B
Relation-graph coverage of test triples
KGFM scenario-stratified MRR
Scenario labelling and MRR aggregation. Every test triple (h, r, t) is scored in both directions and labelled independently in each. Tail prediction (h, r, ?) has query half (h, r) and answer half (r, t); head prediction (?, r, t) is encoded with the inverse relation, giving query half (t, r+|Ri |) and answer half (r+|Ri |, h). A half is seen when its (entity, relation) pair has a matching edge in the inference graph Gi (Sec. 3), and the scenario of the directed query follows from whether its query and answer halves are seen. We report, per scenario, the mean reciprocal rank over all directed queries in the scenario; this pooling weights the head and tail directions by their counts, keeping each scenario consistent with the Orig aggregate, the original unstratified MRR pooled over all directed queries. We then average each per-scenario MRR unweighted over the benchmarks of a family. For SQSA, UQUA, and Orig the two directions are balanced (nhead = ntail ) and the weighting has no effect. SQUA and UQSA are the exception: the inverse swaps the two halves of a triple, so a triple that is SQUA under tail prediction is UQSA under head prediction; the two scenarios then cover the same triples (Sec. 4) in opposite directions (nhead ̸= ntail ), and are averaged according to their weight. Aggregate and per-dataset results. Table B.1 reports the Orig MRR per benchmark family and pooled over all 57 benchmarks; on the full suite ULTRA and MOTIF are essentially tied (0.368 vs. 0.369) and TRIX leads (0.388), consistent with the published numbers. Table B.2 gives the full per-dataset scenario-stratified MRR for ULTRA, MOTIF, and TRIX. 12
SQSA
UQSA
both halves seen
answer half seen h2h
h2h France
t2t
t2h Paris
Eiffel Tower
comprises
t2h
France Paris
hasLandmark
Champ de Mars SQUA
t2t
t2h Eiffel Tower
comprises
t2h
hasLandmark
Champ de Mars
query half seen
UQUA
neither half seen h2h
h2h France
t2t
t2h Paris
Eiffel Tower
comprises
t2h
France Paris
hasLandmark
Champ de Mars
t2t
t2h Eiffel Tower
comprises
t2h
hasLandmark
Champ de Mars
Gir motif
induced Gir motif
Gtr motif
induced Gtr motif
Figure A.1: Only in SQSA are the motifs a test triple induces present in Gir by construction, while in the other scenarios they appear only through incidental co-occurrences or are missing. For each scenario we draw the four co-occurrence motifs between comprises and hasLandmark, solid if present in Gir and dotted if only in Gtr ; those induced by the defining test triple (teal, left graph of each pair) are thick purple. In SQSA the induced t2t is present by construction, since the seen answer-half gives the answer entity an incoming comprises edge. In UQSA the induced t2t is still present, but only through an incidental comprises edge into the answer entity. In SQUA and UQUA the induced h2h or t2h self-loop is missing (purple dotted), recoverable only through incidental co-occurrences (see Sec. 6). Model
Family 1 (13)
Family 2 (18)
Family 3 (23)
Total (54)
Pretrain (3)
All (57)
ULTRA MOTIF TRIX
0.305 0.302 0.326
0.438 0.436 0.455
0.346 0.349 0.367
0.367 0.367 0.387
0.388 0.409 0.409
0.368 0.369 0.388
Table B.1: TRIX is the strongest GNN-based KGFM on aggregate. Reproduced results of ULTRA, MOTIF, TRIX using their original checkpoints. Scenario-stratified evaluation in Table 4 reveals where this gain concentrates. Unweighted mean Orig MRR per benchmark family. Family 1 (13) excludes the three pretraining graphs (FB15k237, CoDEx Medium, WN18RR); Pretrain (3) reports them separately. All (57) pools the full benchmark suite (the 54 non-pretraining benchmarks plus the 3 pretraining graphs)
aggregate %≥1 (the Orig row) stays at or below 9.2%, and SQSA is 0% by construction. For example, UQUA %≥1 peaks at 46.9% on WKIngram:25, which is also the benchmark where UQUA falls furthest below the aggregate (−0.144). For TRIX, by contrast, whenever the witnessing entity of a motif differs between Gi and Gt that motif is absent from Gir , so almost every test triple in SQUA, UQSA, and UQUA is missing at least one: %≥1 reaches exactly 100% on the three benchmarks other than FB15k237 and > 99% on FB15k237, with the mean number of missing motifs reaching 93 per UQUA triple on FB15k237. Although the |R0 | of TRIX is 4×–80× larger than that of ULTRA (each ULTRA edge can split into many TRIX edges, one per witnessing entity), the entity-tagged construction makes the per-triple coverage strictly worse. This coverage gap is the structural explanation for the UQUA regression of TRIXnoiter in Table 3: without the iterative entity–relation updates that let representations transfer across the differing witnessing entities of Gi and Gt , the relation graph
that TRIXnoiter relies on does not transfer to the UQUA scenario.
D
Iterative entity–relation coupling ablation (TRIX vs. TRIXnoiter )
The TRIXnoiter variant. TRIXnoiter is the single entity–relation pass ablation of TRIX from Zhang et al. (2024): it is identical to full TRIX, including the same entity-tagged relation graph, except that the relation and entity networks each run once in sequence rather than through the iterative entity– relation refinement that full TRIX adds. We implement it and release the checkpoint. It collapses on UQUA, where the entity that would witness a motif for the test triple differs between Gi and Gt (App. C), falling below even ULTRA and MOTIF (Table 3). Per-dataset comparison. Table D.1 gives the per-dataset comparison of TRIX against TRIXnoiter across all benchmark families, with per-family Average rows, in the same unified format as the other 13
ULTRA
Family 3
Family 2
Family 1
Dataset CoDEx Small WDsinger FB15k237_10 FB15k237_20 FB15k237_50 NELL23k AristoV4 Hetionet NELL995 CoDEx Large ConceptNet100k DBpedia100k YAGO310 FB15k237† CoDEx Medium† WN18RR†
MOTIF
Orig
SQSA UQSA SQUA UQUA
0.460 0.386 0.240 0.268 0.322 0.245 0.166 0.250 0.452 0.328 0.049 0.391 0.412 0.354 0.354 0.457
0.464 0.369 0.129 0.160 0.221 0.197 0.229 0.253 – 0.314 0.047 0.413 0.432 0.338 0.342 0.824
0.758 0.495 0.470 0.544 0.666 0.532 0.130 0.143 0.753 0.622 0.098 0.575 0.595 0.702 0.668 0.428
0.142 0.163 0.011 0.014 0.031 0.037 0.010 0.016 0.106 0.065 0.018 0.145 0.053 0.088 0.073 0.278
TRIX
Orig
SQSA UQSA SQUA UQUA
Orig
SQSA UQSA SQUA UQUA
0.307 0.609 0.033 0.074 0.177 0.242 0.012 0.014 0.526 0.338 0.039 0.333 0.241 0.293 0.553 0.355
0.474 0.397 0.236 0.259 0.312 0.220 0.096 0.256 0.491 0.339 0.019 0.382 0.441 0.346 0.363 0.518
0.472 0.370 0.127 0.152 0.213 0.175 0.141 0.260 – 0.318 0.016 0.394 0.465 0.328 0.345 0.837
0.754 0.486 0.458 0.523 0.644 0.492 0.038 0.156 0.739 0.624 0.035 0.533 0.590 0.677 0.656 0.471
0.224 0.193 0.012 0.018 0.034 0.043 0.007 0.018 0.144 0.100 0.023 0.196 0.075 0.101 0.130 0.399
0.347 0.638 0.046 0.088 0.185 0.164 0.015 0.058 0.651 0.362 0.007 0.367 0.321 0.307 0.587 0.391
0.473 0.398 0.246 0.269 0.321 0.237 0.181 0.279 0.472 0.335 0.193 0.427 0.409 0.362 0.360 0.506
0.469 0.373 0.127 0.156 0.220 0.180 0.244 0.283 – 0.310 0.216 0.422 0.425 0.339 0.337 0.826
0.756 0.506 0.481 0.548 0.663 0.535 0.148 0.204 0.799 0.625 0.202 0.623 0.600 0.704 0.660 0.447
0.236 0.202 0.014 0.017 0.036 0.049 0.021 0.023 0.162 0.104 0.026 0.252 0.079 0.128 0.140 0.402
0.337 0.592 0.043 0.081 0.179 0.188 0.036 0.189 0.444 0.351 0.010 0.413 0.277 0.306 0.575 0.370
Average
0.305
0.269
0.491
0.062
0.227
0.302
0.259
0.467
0.084
0.250
0.326
0.285
0.515
0.094
0.242
FB v1 FB v2 FB v3 FB v4 WN v1 WN v2 WN v3 WN v4 NELL v1 NELL v2 NELL v3 NELL v4 ILPC Small ILPC Large HM 1k HM 3k HM 5k IndigoBM
0.491 0.511 0.488 0.482 0.656 0.672 0.388 0.635 0.756 0.550 0.538 0.488 0.298 0.296 0.078 0.065 0.060 0.428
0.626 0.590 0.556 0.531 0.780 0.778 0.587 0.829 – 0.619 0.577 0.543 0.326 0.292 0.101 0.070 0.058 0.418
0.706 0.701 0.676 0.664 0.705 0.743 0.500 0.615 1.000 0.797 0.855 0.741 0.503 0.570 0.133 0.109 0.094 0.783
0.187 0.227 0.208 0.214 0.643 0.651 0.246 0.559 0.513 0.264 0.218 0.145 0.042 0.031 0.023 0.025 0.027 0.126
0.472 0.466 0.472 0.466 0.352 0.440 0.277 0.432 – 0.298 0.311 0.263 0.316 0.228 0.027 0.038 0.057 0.464
0.505 0.511 0.500 0.487 0.681 0.663 0.420 0.640 0.669 0.564 0.533 0.503 0.296 0.285 0.063 0.055 0.050 0.426
0.618 0.551 0.548 0.521 0.780 0.740 0.579 0.824 – 0.624 0.540 0.546 0.319 0.285 0.074 0.042 0.042 0.416
0.668 0.687 0.669 0.650 0.749 0.717 0.525 0.616 0.993 0.777 0.820 0.714 0.498 0.537 0.106 0.096 0.081 0.764
0.241 0.265 0.249 0.245 0.696 0.658 0.289 0.580 0.345 0.313 0.270 0.217 0.051 0.039 0.022 0.019 0.021 0.135
0.533 0.528 0.518 0.520 0.365 0.479 0.335 0.440 – 0.352 0.393 0.357 0.325 0.247 0.031 0.052 0.044 0.473
0.515 0.525 0.501 0.493 0.699 0.678 0.418 0.648 0.806 0.569 0.558 0.538 0.303 0.307 0.072 0.069 0.062 0.436
0.632 0.574 0.560 0.535 0.761 0.756 0.576 0.826 – 0.615 0.566 0.574 0.315 0.285 0.091 0.065 0.052 0.426
0.720 0.724 0.689 0.674 0.743 0.710 0.449 0.606 0.996 0.787 0.840 0.764 0.522 0.597 0.118 0.119 0.108 0.783
0.220 0.275 0.242 0.242 0.756 0.693 0.348 0.600 0.616 0.353 0.318 0.278 0.062 0.045 0.027 0.024 0.023 0.144
0.519 0.485 0.466 0.471 0.435 0.496 0.360 0.468 – 0.343 0.361 0.327 0.318 0.232 0.020 0.045 0.035 0.449
Average
0.438
0.487
0.605
0.242
0.316
0.436
0.473
0.593
0.259
0.352
0.455
0.483
0.608
0.292
0.343
FB-25 FB-50 FB-75 FB-100 WK-25 WK-50 WK-75 WK-100 NL-0 NL-25 NL-50 NL-75 NL-100 Metafam FBNELL Wiki MT1 tax Wiki MT1 health Wiki MT2 org Wiki MT2 sci Wiki MT3 art Wiki MT3 infra Wiki MT4 sci Wiki MT4 health
0.387 0.334 0.397 0.446 0.310 0.175 0.386 0.178 0.364 0.399 0.394 0.355 0.469 0.344 0.480 0.240 0.297 0.084 0.254 0.251 0.596 0.293 0.525
0.386 0.355 0.444 0.481 0.346 0.184 0.472 0.263 0.431 0.390 0.437 0.394 0.511 – 0.485 0.281 0.446 0.079 0.268 0.268 0.662 0.326 0.566
0.608 0.501 0.625 0.671 0.490 0.274 0.597 0.242 0.656 0.769 0.757 0.626 0.767 0.178 0.789 0.486 0.480 0.221 0.494 0.408 0.812 0.434 0.653
0.135 0.100 0.111 0.132 0.102 0.059 0.069 0.009 0.132 0.115 0.086 0.126 0.140 0.510 0.168 0.004 0.055 0.035 0.029 0.115 0.277 0.011 0.250
0.444 0.376 0.289 0.317 0.166 0.182 0.134 0.019 0.158 0.097 0.081 0.137 0.214 – 0.467 0.003 0.301 0.002 0.002 0.060 0.601 0.254 0.412
0.384 0.338 0.399 0.428 0.311 0.163 0.366 0.164 0.324 0.348 0.373 0.314 0.438 0.344 0.469 0.325 0.326 0.092 0.286 0.269 0.658 0.283 0.626
0.370 0.348 0.431 0.462 0.334 0.169 0.438 0.238 0.356 0.359 0.387 0.341 0.445 – 0.453 0.304 0.458 0.079 0.284 0.286 0.645 0.308 0.696
0.587 0.473 0.609 0.639 0.491 0.259 0.576 0.219 0.610 0.680 0.739 0.555 0.723 0.416 0.756 0.462 0.521 0.279 0.468 0.450 0.812 0.423 0.660
0.151 0.126 0.134 0.141 0.114 0.057 0.073 0.018 0.119 0.083 0.083 0.121 0.158 0.272 0.198 0.217 0.081 0.098 0.130 0.110 0.531 0.029 0.347
0.501 0.428 0.360 0.254 0.206 0.163 0.133 0.027 0.125 0.071 0.073 0.116 0.139 – 0.487 0.003 0.301 0.002 0.011 0.072 0.618 0.493 0.450
0.393 0.334 0.401 0.436 0.305 0.166 0.368 0.188 0.385 0.377 0.404 0.351 0.468 0.341 0.473 0.358 0.376 0.091 0.323 0.284 0.655 0.290 0.677
0.384 0.341 0.437 0.465 0.319 0.167 0.435 0.261 0.385 0.317 0.357 0.322 0.451 – 0.453 0.251 0.480 0.074 0.259 0.271 0.651 0.314 0.746
0.614 0.497 0.622 0.660 0.499 0.268 0.580 0.269 0.650 0.724 0.829 0.653 0.773 0.446 0.757 0.463 0.627 0.349 0.475 0.465 0.817 0.441 0.711
0.155 0.119 0.130 0.144 0.114 0.059 0.081 0.022 0.236 0.132 0.104 0.155 0.191 0.236 0.214 0.300 0.086 0.091 0.233 0.144 0.501 0.037 0.407
0.466 0.390 0.334 0.288 0.189 0.175 0.137 0.027 0.168 0.123 0.091 0.165 0.261 – 0.486 0.015 0.301 0.003 0.016 0.106 0.637 0.367 0.416
Average
0.346
0.385
0.545
0.121
0.214
0.349
0.372
0.540
0.147
0.229
0.367
0.370
0.574
0.169
0.235
Table B.2: Per-dataset scenario-stratified MRR for ULTRA, MOTIF, and TRIX. All benchmark families in one table, separated by horizontal rules with a rotated family label per block; in each cell, bold marks the best of the three models. The italicised Average row per family is the unweighted mean over the zero-shot benchmarks of that family (pretraining rows excluded). † Used during KGFM pretraining; reported for completeness, not zero-shot. Empty scenarios (no test triples) are reported as “–”.
per-dataset tables. The iterative entity–relation updates let TRIX recover on UQUA, where TRIXnoiter otherwise falls behind.
14
Benchmark
Split
ULTRA
ntotal |R0 |
TRIX
%≥1 mean
|R0 |
%≥1
mean
Orig SQSA SQUA UQSA UQUA
4,406 1,236 1,224 1,224 722
21,920 6.40% – 0.00% – 5.64% – 5.64% – 19.94%
0.48 0.00 0.32 0.32 1.86
186,500 71.95% 8.76 – 0.00% 0.00 – 100.00% 9.03 – 100.00% 9.03 – 100.00% 22.83
Orig SQSA ILPC2022SmallInductive SQUA UQSA UQUA
5,804 2,586 1,528 1,528 162
6,912 1.34% – 0.00% – 1.77% – 1.77% – 14.81%
0.06 0.00 0.07 0.07 0.74
198,938 55.44% 5.95 – 0.00% 0.00 – 100.00% 10.04 – 100.00% 10.04 – 100.00% 23.70
WKIngram:25
Orig SQSA SQUA UQSA UQUA
2,262 980 577 577 128
3,184 9.20% – 0.00% – 12.82% – 12.82% – 46.88%
0.46 0.00 0.62 0.62 2.50
15,148 56.68% 3.76 – 0.00% 0.00 – 100.00% 5.89 – 100.00% 5.89 – 100.00% 13.41
FB15k237
Orig 40,932 108,240 SQSA 27,848 – SQUA 6,203 – UQSA 6,203 – UQUA 678 –
WDsinger
0.53% 0.00% 1.35% 1.35% 7.08%
0.05 8,689,114 0.00 – 0.14 – 0.14 – 0.55 –
31.84% 0.00% 99.63% 99.63% 99.41%
17.96 0.00 54.14 54.14 93.33
Table C.1: The relation graph of TRIX misses essentially every UQ∗ and ∗UA triple. ULTRA is missing a relation-graph edge for at most 9.2% of test triples overall, whereas TRIX reaches %≥1 ≥ 99.4% across SQUA, UQSA, and UQUA and a mean of 13–93 new edges per UQUA triple. nadded = |R1 \ R0 | counts the relation-graph edges a single directed test triple induces that are missing from R0 ; %≥1 is the share of triples with nadded ≥ 1; mean is the average per-triple nadded. |R0 | is shown once per benchmark on the Orig row. SQSA is 0 by construction. The |R0 | of TRIX is 4×–80× larger than that of ULTRA because each ULTRA edge (r1 , τ, r2 ) splits into one TRIX edge per witnessing entity.
15
TRIX
Family 3
Family 2
Family 1
Dataset CoDEx Small WDsinger FB15k237_10 FB15k237_20 FB15k237_50 NELL23k AristoV4 Hetionet NELL995 CoDEx Large ConceptNet100k DBpedia100k YAGO310 FB15k237† CoDEx Medium† WN18RR†
TRIXnoiter
Orig
SQSA UQSA SQUA UQUA
Orig
SQSA UQSA SQUA UQUA
0.473 0.398 0.246 0.269 0.321 0.237 0.181 0.279 0.472 0.335 0.193 0.427 0.409 0.362 0.360 0.506
0.469 0.373 0.127 0.156 0.220 0.180 0.244 0.283 – 0.310 0.216 0.422 0.425 0.339 0.337 0.826
0.472 0.355 0.241 0.264 0.319 0.212 0.198 0.234 0.413 0.328 0.162 0.358 0.273 0.358 0.357 0.512
0.467 0.354 0.126 0.156 0.221 0.173 0.258 0.236 – 0.303 0.177 0.357 0.260 0.337 0.335 0.832
0.756 0.506 0.481 0.548 0.663 0.535 0.148 0.204 0.799 0.625 0.202 0.623 0.600 0.704 0.660 0.447
0.236 0.202 0.014 0.017 0.036 0.049 0.021 0.023 0.162 0.104 0.026 0.252 0.079 0.128 0.140 0.402
0.337 0.592 0.043 0.081 0.179 0.188 0.036 0.189 0.444 0.351 0.010 0.413 0.277 0.306 0.575 0.370
0.740 0.495 0.475 0.542 0.665 0.494 0.183 0.182 0.793 0.615 0.193 0.571 0.600 0.699 0.653 0.452
0.267 0.185 0.015 0.020 0.037 0.040 0.032 0.025 0.139 0.099 0.028 0.188 0.077 0.130 0.138 0.409
0.287 0.406 0.018 0.030 0.052 0.076 0.044 0.038 0.243 0.323 0.015 0.231 0.215 0.177 0.551 0.377
Average
0.326
0.285
0.515
0.094
0.242
0.295
0.257
0.503
0.089
0.152
FB v1 FB v2 FB v3 FB v4 WN v1 WN v2 WN v3 WN v4 NELL v1 NELL v2 NELL v3 NELL v4 ILPC Small ILPC Large HM 1k HM 3k HM 5k IndigoBM
0.515 0.525 0.501 0.493 0.699 0.678 0.418 0.648 0.806 0.569 0.558 0.538 0.303 0.307 0.072 0.069 0.062 0.436
0.632 0.574 0.560 0.535 0.761 0.756 0.576 0.826 – 0.615 0.566 0.574 0.315 0.285 0.091 0.065 0.052 0.426
0.720 0.724 0.689 0.674 0.743 0.710 0.449 0.606 0.996 0.787 0.840 0.764 0.522 0.597 0.118 0.119 0.108 0.783
0.220 0.275 0.242 0.242 0.756 0.693 0.348 0.600 0.616 0.353 0.318 0.278 0.062 0.045 0.027 0.024 0.023 0.144
0.519 0.485 0.466 0.471 0.435 0.496 0.360 0.468 – 0.343 0.361 0.327 0.318 0.232 0.020 0.045 0.035 0.449
0.470 0.498 0.467 0.466 0.693 0.666 0.353 0.649 0.810 0.513 0.530 0.515 0.300 0.296 0.075 0.072 0.066 0.412
0.599 0.566 0.547 0.525 0.795 0.776 0.467 0.843 – 0.587 0.563 0.574 0.315 0.283 0.113 0.073 0.065 0.394
0.697 0.713 0.672 0.668 0.729 0.718 0.407 0.612 1.000 0.747 0.792 0.733 0.515 0.574 0.122 0.127 0.115 0.784
0.222 0.268 0.237 0.222 0.710 0.672 0.305 0.586 0.619 0.252 0.292 0.228 0.059 0.039 0.022 0.025 0.023 0.144
0.345 0.343 0.292 0.311 0.412 0.422 0.246 0.450 – 0.208 0.241 0.188 0.291 0.220 0.010 0.015 0.023 0.339
Average
0.455
0.483
0.608
0.292
0.343
0.436
0.476
0.596
0.274
0.256
FB-25 FB-50 FB-75 FB-100 WK-25 WK-50 WK-75 WK-100 NL-0 NL-25 NL-50 NL-75 NL-100 Metafam FBNELL Wiki MT1 tax Wiki MT1 health Wiki MT2 org Wiki MT2 sci Wiki MT3 art Wiki MT3 infra Wiki MT4 sci Wiki MT4 health
0.393 0.334 0.401 0.436 0.305 0.166 0.368 0.188 0.385 0.377 0.404 0.351 0.468 0.341 0.473 0.358 0.376 0.091 0.323 0.284 0.655 0.290 0.677
0.384 0.341 0.437 0.465 0.319 0.167 0.435 0.261 0.385 0.317 0.357 0.322 0.451 – 0.453 0.251 0.480 0.074 0.259 0.271 0.651 0.314 0.746
0.614 0.497 0.622 0.660 0.499 0.268 0.580 0.269 0.650 0.724 0.829 0.653 0.773 0.446 0.757 0.463 0.627 0.349 0.475 0.465 0.817 0.441 0.711
0.155 0.119 0.130 0.144 0.114 0.059 0.081 0.022 0.236 0.132 0.104 0.155 0.191 0.236 0.214 0.300 0.086 0.091 0.233 0.144 0.501 0.037 0.407
0.466 0.390 0.334 0.288 0.189 0.175 0.137 0.027 0.168 0.123 0.091 0.165 0.261 – 0.486 0.015 0.301 0.003 0.016 0.106 0.637 0.367 0.416
0.359 0.288 0.369 0.425 0.293 0.157 0.369 0.183 0.354 0.392 0.400 0.326 0.437 0.418 0.446 0.300 0.371 0.088 0.298 0.285 0.636 0.268 0.634
0.380 0.341 0.425 0.467 0.322 0.162 0.425 0.257 0.391 0.352 0.390 0.344 0.481 – 0.452 0.248 0.490 0.070 0.223 0.282 0.628 0.291 0.721
0.603 0.479 0.613 0.647 0.485 0.257 0.597 0.260 0.644 0.768 0.809 0.579 0.726 0.560 0.735 0.456 0.620 0.350 0.452 0.462 0.806 0.408 0.633
0.157 0.116 0.120 0.125 0.082 0.053 0.096 0.018 0.174 0.109 0.094 0.138 0.113 0.276 0.181 0.177 0.076 0.104 0.211 0.140 0.478 0.029 0.343
0.179 0.116 0.108 0.137 0.161 0.140 0.088 0.013 0.096 0.127 0.064 0.111 0.202 – 0.334 0.020 0.302 0.002 0.005 0.114 0.645 0.261 0.275
Average
0.367
0.370
0.574
0.169
0.235
0.352
0.370
0.563
0.148
0.159
Table D.1: Per-dataset scenario-stratified MRR for TRIX and TRIXnoiter (TRIX without the iterative entity– relation coupling). All benchmark families in one table, separated by horizontal rules with a rotated family label per block; in each cell, bold marks the better of the two models. The italicised Average row per family is the unweighted mean over the zero-shot benchmarks of that family (pretraining rows excluded). † Used during KGFM pretraining; reported for completeness, not zero-shot. Empty scenarios (no test triples) are reported as “–”.
16
E Frozen-baseline ablation (ULTRA vs. ULTRArand )
the held-out target unless the target aligns with the seen answers D(h, r).
The ULTRArand baseline. ULTRArand is our frozen-backbone ablation of ULTRA: we keep the entire backbone at random initialization, the two NBFNets, every relational-convolution layer, all layer-norm parameters, and the relation embeddings, and train only the entity-side score head, a two-layer MLP (128→128→1) with 16,641 parameters, 9.86% of the 168,705parameter model; the remaining 90% stay random. We pre-train this head for ten epochs on the FB15k237 + WN18RR + CoDEx Medium mixture, following the exact same training protocol as ULTRA, and release the checkpoint. It isolates how much of the per-scenario behavior is fixed by the r-typed message-passing architecture before any backbone weights are learned.
G
A natural objection to the query/answer asymmetry of Sec. 5 is that it might merely reflect target popularity: if UQSA targets are popular, high-degree entities and SQUA targets unpopular, low-degree ones, a popularity bias alone could reproduce the gap, with no half-link mechanism. To disentangle this, we fix the node degree, binning test triples into gold-target-degree deciles (the degree of t in Gi ) and re-computing per-scenario MRR within each, on the 17 Family 2 benchmarks. The lower deciles are long-tail entities on which every scenario scores near the floor, so we report the upper deciles. Table G.1 shows the asymmetry survives: UQSA exceeds SQUA in every reported decile, for example 0.511 against 0.133 at degree 34–55. The gap is therefore not an artefact of target degree.
Per-dataset comparison. Table E.1 reports the per-dataset scenario-stratified MRR for ULTRA against ULTRArand across all benchmark families. Even with the backbone frozen at random initialization, the seen-answer advantage of Sec. 5 is already visible under ULTRArand on average, indicating that it originates in the r-typed message passing of the architecture rather than in learned weights.
F
The query/answer asymmetry is not a node-degree artefact
H
The query/answer asymmetry is not a relation-cardinality artefact
A second possible confound for the query/answer asymmetry of Sec. 5 is relation cardinality: the SQUA < UQSA gap could merely mirror 1-to-N queries being harder than N-to-1 ones, with no halflink mechanism. To disentangle this, we fix the relation cardinality, stratifying SQUA and UQSA MRR by the four cardinality classes of Bordes et al. (2013) (1-to-1, 1-to-N, N-to-1, N-to-N), pooled over the 17 Family 2 benchmarks. Table H.1 shows the asymmetry survives: UQSA exceeds SQUA within every cardinality class, for example 0.431 against 0.061 on 1-to-N. The gap is therefore not an artefact of relation cardinality.
Seen-query distractor diagnostic
For each SQUA test triple (h, r, t) we measure the D(h, r) distraction rate: how often the model scores the held-out target t no higher than an entity in D(h, r), the set of seen r-answers of h in the inference graph. These are exactly the entities the filtered evaluation protocol removes from the rank, so a high distraction rate is invisible to filtered MRR yet directly reflects the seen-query prior ranking the previously observed r-tails of h above the correct answer. We report it for the pre-trained ULTRA 3-graph checkpoint on three benchmarks chosen to span the range of SQUA degradation relative to aggregate MRR: WN18RRInductive v4 (near-parity), WDsinger (intermediate), and DBpedia100k (severe). Scores are unfiltered with pessimistic tie handling. Table F.1 shows the distraction rate tracks this degradation: where it stays near parity (≈ 0.5) SQUA MRR stays close to Orig (WN18RRInductive v4), and where it is high SQUA MRR falls far below (WDsinger, DBpedia100k). This is the seen-query side of the asymmetry of Sec. 5: the seen-query prior distracts from
I
Per-benchmark fine-tuning results
We report results under a per-benchmark finetuning protocol on Family 3. The pre-trained checkpoint is fine-tuned on the training graph G of each benchmark individually, then evaluated by scoring its held-out test triples Et with message passing over its inference graph Gi , exactly as in the zero-shot setting; the zero-shot baseline is the same pre-trained checkpoint applied to Gi without finetuning. For Family 3, G is disjoint from Gi in both entities and relation names, and only the weights change: the GNN and Gi do not, and Gi is still only used at inference. A fine-tuning gain on Gi 17
ULTRA
Family 3
Family 2
Family 1
Dataset CoDEx Small WDsinger FB15k237_10 FB15k237_20 FB15k237_50 NELL23k AristoV4 Hetionet NELL995 CoDEx Large ConceptNet100k DBpedia100k YAGO310 FB15k237† CoDEx Medium† WN18RR†
ULTRArand
Orig
SQSA UQSA SQUA UQUA
Orig
SQSA UQSA SQUA UQUA
0.460 0.386 0.240 0.268 0.322 0.245 0.166 0.250 0.452 0.328 0.049 0.391 0.412 0.354 0.354 0.457
0.464 0.369 0.129 0.160 0.221 0.197 0.229 0.253 – 0.314 0.047 0.413 0.432 0.338 0.342 0.824
0.355 0.294 0.215 0.225 0.239 0.171 0.062 0.124 0.164 0.179 0.075 0.196 0.379 0.227 0.229 0.391
0.362 0.266 0.096 0.113 0.150 0.153 0.083 0.125 – 0.158 0.069 0.191 0.409 0.207 0.222 0.771
0.758 0.495 0.470 0.544 0.666 0.532 0.130 0.143 0.753 0.622 0.098 0.575 0.595 0.702 0.668 0.428
0.142 0.163 0.011 0.014 0.031 0.037 0.010 0.016 0.106 0.065 0.018 0.145 0.053 0.088 0.073 0.278
0.307 0.609 0.033 0.074 0.177 0.242 0.012 0.014 0.526 0.338 0.039 0.333 0.241 0.293 0.553 0.355
0.557 0.335 0.431 0.477 0.526 0.343 0.049 0.091 0.361 0.358 0.166 0.279 0.448 0.508 0.415 0.303
0.086 0.114 0.009 0.011 0.017 0.040 0.008 0.006 0.037 0.037 0.037 0.105 0.031 0.029 0.053 0.246
0.390 0.577 0.047 0.083 0.164 0.141 0.020 0.034 0.051 0.330 0.018 0.289 0.274 0.261 0.547 0.326
Average
0.305
0.269
0.491
0.062
0.227
0.206
0.181
0.340
0.041
0.186
FB v1 FB v2 FB v3 FB v4 WN v1 WN v2 WN v3 WN v4 NELL v1 NELL v2 NELL v3 NELL v4 ILPC Small ILPC Large HM 1k HM 3k HM 5k IndigoBM
0.491 0.511 0.488 0.482 0.656 0.672 0.388 0.635 0.756 0.550 0.538 0.488 0.298 0.296 0.078 0.065 0.060 0.428
0.626 0.590 0.556 0.531 0.780 0.778 0.587 0.829 – 0.619 0.577 0.543 0.326 0.292 0.101 0.070 0.058 0.418
0.706 0.701 0.676 0.664 0.705 0.743 0.500 0.615 1.000 0.797 0.855 0.741 0.503 0.570 0.133 0.109 0.094 0.783
0.187 0.227 0.208 0.214 0.643 0.651 0.246 0.559 0.513 0.264 0.218 0.145 0.042 0.031 0.023 0.025 0.027 0.126
0.472 0.466 0.472 0.466 0.352 0.440 0.277 0.432 – 0.298 0.311 0.263 0.316 0.228 0.027 0.038 0.057 0.464
0.500 0.490 0.448 0.446 0.674 0.654 0.349 0.615 0.780 0.457 0.426 0.400 0.172 0.172 0.031 0.027 0.026 0.331
0.550 0.513 0.485 0.468 0.749 0.745 0.522 0.794 – 0.505 0.439 0.448 0.186 0.157 0.037 0.029 0.021 0.326
0.646 0.635 0.581 0.589 0.722 0.690 0.384 0.576 1.000 0.623 0.679 0.570 0.280 0.336 0.046 0.038 0.037 0.599
0.232 0.262 0.218 0.225 0.721 0.669 0.259 0.568 0.560 0.258 0.198 0.145 0.027 0.018 0.017 0.017 0.016 0.084
0.643 0.576 0.530 0.530 0.393 0.447 0.304 0.425 – 0.299 0.263 0.231 0.308 0.228 0.010 0.020 0.026 0.388
Average
0.438
0.487
0.605
0.242
0.316
0.389
0.410
0.502
0.250
0.331
FB-25 FB-50 FB-75 FB-100 WK-25 WK-50 WK-75 WK-100 NL-0 NL-25 NL-50 NL-75 NL-100 Metafam FBNELL Wiki MT1 tax Wiki MT1 health Wiki MT2 org Wiki MT2 sci Wiki MT3 art Wiki MT3 infra Wiki MT4 sci Wiki MT4 health
0.387 0.334 0.397 0.446 0.310 0.175 0.386 0.178 0.364 0.399 0.394 0.355 0.469 0.344 0.480 0.240 0.297 0.084 0.254 0.251 0.596 0.293 0.525
0.386 0.355 0.444 0.481 0.346 0.184 0.472 0.263 0.431 0.390 0.437 0.394 0.511 – 0.485 0.281 0.446 0.079 0.268 0.268 0.662 0.326 0.566
0.608 0.501 0.625 0.671 0.490 0.274 0.597 0.242 0.656 0.769 0.757 0.626 0.767 0.178 0.789 0.486 0.480 0.221 0.494 0.408 0.812 0.434 0.653
0.135 0.100 0.111 0.132 0.102 0.059 0.069 0.009 0.132 0.115 0.086 0.126 0.140 0.510 0.168 0.004 0.055 0.035 0.029 0.115 0.277 0.011 0.250
0.444 0.376 0.289 0.317 0.166 0.182 0.134 0.019 0.158 0.097 0.081 0.137 0.214 – 0.467 0.003 0.301 0.002 0.002 0.060 0.601 0.254 0.412
0.330 0.291 0.345 0.340 0.287 0.142 0.329 0.119 0.269 0.332 0.334 0.310 0.405 0.133 0.410 0.203 0.121 0.038 0.212 0.150 0.475 0.174 0.284
0.319 0.292 0.367 0.361 0.324 0.128 0.381 0.151 0.313 0.350 0.394 0.309 0.442 – 0.370 0.161 0.298 0.032 0.167 0.160 0.530 0.171 0.314
0.472 0.395 0.505 0.511 0.443 0.231 0.537 0.201 0.440 0.559 0.577 0.526 0.634 0.189 0.687 0.423 0.115 0.147 0.442 0.259 0.468 0.335 0.369
0.127 0.093 0.120 0.105 0.083 0.059 0.070 0.006 0.121 0.144 0.108 0.135 0.142 0.078 0.164 0.007 0.056 0.033 0.026 0.049 0.377 0.013 0.094
0.496 0.429 0.374 0.319 0.219 0.215 0.126 0.020 0.163 0.154 0.110 0.207 0.300 – 0.490 0.002 0.268 0.001 0.004 0.062 0.629 0.349 0.210
Average
0.346
0.385
0.545
0.121
0.214
0.262
0.288
0.411
0.096
0.234
Table E.1: Per-dataset scenario-stratified MRR for ULTRA and ULTRArand (ULTRA with the relation encoder, entity encoder, and relation-graph initial features frozen at random initialization; only the score head is trained). † Used during KGFM pretraining; reported for completeness, not zero-shot. Empty scenarios (no test triples) are reported as “–”.
therefore reflects in-Gi signal that the generic pretrained weights leave unused rather than anything carried over from G, whose entities and relation names never appear in Gi .
restrict the fine-tuning analysis to Family 3 because it is the only family on which fine-tuning stays in the inductive transfer setting (Sec. 2): there G shares neither entities nor relation names with Gi . In Family 1, G shares both with Gi , and in Family 2 it shares the relation names.
Table I.1 gives the per-dataset fine-tuned MRR for ULTRA, MOTIF, and TRIX on Family 3. We 18
nSQUA
Dataset
ULTRA MRR Orig
SQUA
DBpedia100k 17,415 0.391 WDsinger 1,224 0.386 WN18RRInductive v4 1,458 0.635
0.145 0.163 0.559
the original paper). On FB15k237_10 we apply our scenario-stratified evaluation to FLOCK and TRIX (Table J.1): they reach the same overall MRR (0.246) and similar per-scenario MRR, but FLOCK needs roughly 186× the inference time of TRIX (147 versus 0.8 minutes on a single NVIDIA A100 40 GB GPU with an Intel Xeon Platinum 8368 host). We therefore exclude FLOCK from the main comparison (Sec. 2).
D(h, r) distraction 0.664 0.916 0.515
Table F.1: On SQUA triples, SQUA MRR drops when the previously seen r-answers of h (D(h, r)) outrank the held-out target t instead of aligning with it, with the D(h, r) distraction rate near parity only on the benchmark where SQUA MRR stays close to Orig. ULTRA MRR is the ULTRA aggregate MRR over all test triples (Orig) and its MRR on SQUA triples; the gap between them is the SQUA degradation these benchmarks span. The D(h, r) distraction rate is the fraction of SQUA triples where some entity in D(h, r) scores at least as high as t under raw scores (pessimistic ties); the filtered protocol removes D(h, r) from the rank, so a high distraction rate is invisible to filtered MRR. Decile
Degree
Orig
SQUA
UQSA
5 7 9 10
9–17 34–55 129–520 523–6,411
0.350 0.377 0.587 0.839
0.158 0.133 0.325 0.721
0.413 0.511 0.665 0.859
K
Tables K.1 to K.3 list the size statistics of the Family 1, Family 2, and Family 3 benchmarks, and Table K.4 reports their per-benchmark half-link scenario proportions, the breakdown behind Table 1.
Table G.1: The query/answer asymmetry survives degree stratification: UQSA exceeds SQUA in every reported decile. ULTRA MRR by gold-target-degree decile (degree in Gi ) on the 17 Family 2 benchmarks; the lower deciles are long-tail low-degree entities on which performance is near the floor and uninformative for this contrast, so the upper deciles are reported. Orig is the aggregate MRR of ULTRA. Cardinality
SQUA
UQSA
∆
Orig (all)
0.140
0.572
+0.432
1-to-1 1-to-N N-to-1 N-to-N
0.494 0.061 0.206 0.307
0.692 0.431 0.540 0.659
+0.198 +0.370 +0.334 +0.352
Table H.1: Conditioning on relation cardinality leaves the query/answer asymmetry intact: UQSA exceeds SQUA within every cardinality class. ULTRA MRR, SQUA versus UQSA, pooled over the 17 Family 2 benchmarks, stratified by the relation-cardinality categories of Bordes et al. (2013). ∆ = UQSA − SQUA.
J
Dataset statistics
Comparison with FLOCK
FLOCK (Kim et al., 2025) replaces the deterministic message passing of ULTRA, MOTIF, and TRIX with probabilistic random-walk ensembles. Across the 54 zero-shot graphs it averages 0.391 MRR, marginally above the strongest GNN-based model, TRIX at 0.387 (the FLOCK average is taken from 19
ULTRA
Dataset FB-25 FB-50 FB-75 FB-100 WK-25 WK-50 WK-75 WK-100 NL-0 NL-25 NL-50 NL-75 NL-100 Metafam FBNELL Wiki MT1 tax Wiki MT1 health Wiki MT2 org Wiki MT2 sci Wiki MT3 art Wiki MT3 infra Wiki MT4 sci Wiki MT4 health
MOTIF
Orig
SQSA UQSA SQUA UQUA
0.386 0.335 0.407 0.439 0.307 0.149 0.371 0.163 0.340 0.400 0.418 0.378 0.478 0.999 0.490 0.368 0.378 0.102 0.323 0.314 0.662 0.310 0.698
0.378 0.345 0.447 0.475 0.321 0.156 0.444 0.230 0.407 0.440 0.438 0.404 0.505 – 0.466 0.320 0.514 0.086 0.310 0.295 0.672 0.339 0.782
0.594 0.480 0.630 0.657 0.484 0.246 0.586 0.235 0.560 0.746 0.801 0.673 0.767 0.997 0.814 0.512 0.618 0.359 0.531 0.498 0.833 0.454 0.687
0.139 0.111 0.127 0.140 0.122 0.033 0.072 0.012 0.151 0.104 0.101 0.131 0.170 1.000 0.187 0.260 0.087 0.090 0.149 0.166 0.479 0.038 0.418
0.495 0.425 0.333 0.247 0.228 0.171 0.128 0.016 0.191 0.131 0.138 0.178 0.278 – 0.527 0.021 0.301 0.003 0.017 0.194 0.631 0.406 0.440
TRIX
Orig
SQSA UQSA SQUA UQUA
Orig
SQSA UQSA SQUA UQUA
0.378 0.336 0.397 0.438 0.311 0.147 0.368 0.166 0.327 0.389 0.422 0.361 0.465 0.998 0.479 0.455 0.386 0.105 0.321 0.314 0.685 0.312 0.701
0.369 0.349 0.435 0.474 0.334 0.146 0.441 0.233 0.424 0.403 0.446 0.370 0.471 – 0.475 0.311 0.517 0.087 0.308 0.298 0.686 0.344 0.783
0.379 0.329 0.386 0.423 0.290 0.142 0.364 0.181 0.338 0.360 0.402 0.340 0.464 1.000 0.477 0.403 0.378 0.098 0.336 0.270 0.675 0.308 0.703
0.393 0.354 0.445 0.476 0.326 0.150 0.450 0.259 0.423 0.319 0.437 0.363 0.483 – 0.507 0.302 0.494 0.084 0.290 0.307 0.680 0.344 0.790
0.586 0.497 0.621 0.653 0.492 0.249 0.580 0.239 0.576 0.732 0.813 0.681 0.787 1.000 0.793 0.526 0.629 0.379 0.523 0.495 0.835 0.454 0.691
0.135 0.105 0.122 0.136 0.114 0.041 0.072 0.016 0.116 0.111 0.102 0.114 0.151 0.995 0.170 0.446 0.093 0.105 0.153 0.165 0.538 0.030 0.426
0.486 0.405 0.321 0.273 0.202 0.167 0.130 0.018 0.123 0.110 0.107 0.147 0.162 – 0.474 0.018 0.302 0.004 0.016 0.206 0.646 0.400 0.450
0.591 0.500 0.634 0.667 0.512 0.259 0.583 0.266 0.671 0.770 0.829 0.672 0.805 1.000 0.829 0.514 0.630 0.360 0.541 0.493 0.833 0.453 0.714
0.100 0.080 0.073 0.061 0.045 0.019 0.045 0.002 0.072 0.046 0.045 0.078 0.113 1.000 0.121 0.340 0.082 0.049 0.184 0.047 0.513 0.016 0.388
0.429 0.375 0.237 0.147 0.120 0.115 0.086 0.006 0.096 0.090 0.065 0.102 0.178 – 0.354 0.016 0.302 0.003 0.009 0.103 0.638 0.252 0.430
Table I.1: Per-dataset scenario-stratified MRR for ULTRA, MOTIF, and TRIX fine-tuned on Family 3. Each model is fine-tuned individually on the training graph of each benchmark; in each cell, bold marks the best of the three models. Empty scenarios (no test triples) are reported as “–”.
TRIX
FLOCK
Orig SQSA UQSA SQUA UQUA
0.246 0.127 0.481 0.014 0.043
0.246 0.127 0.476 0.018 0.055
Time (min)
0.79
147.18
Table J.1: On FB15k237_10, TRIX and FLOCK reach the same overall MRR, but FLOCK needs roughly 186× the inference time. Zero-shot MRR per scenario (top) and inference time in minutes (bottom); bold marks the best in each row (highest MRR, lowest time). Time measured on a single NVIDIA A100 (40 GB) GPU with an Intel Xeon Platinum 8368 host.
20
Table K.1: Family 1 datasets (16). Train, Valid, Test denote triples in the respective set. Task: h/t predicts both heads and tails; tails predicts tails only. Dataset
Entities
Rels
Train
Valid
Test
Task
CoDEx Small (Safavi and Koutra, 2020) WDsinger (Lv et al., 2020) FB15k237_10 (Lv et al., 2020) FB15k237_20 (Lv et al., 2020) FB15k237_50 (Lv et al., 2020) FB15k237 (Toutanova and Chen, 2015) CoDEx Medium (Safavi and Koutra, 2020) NELL23k (Lv et al., 2020) WN18RR (Dettmers et al., 2018) AristoV4 (Chen et al., 2021) Hetionet (Himmelstein et al., 2017) NELL995 (Xiong et al., 2017) CoDEx Large (Safavi and Koutra, 2020) ConceptNet100k (Malaviya et al., 2020) DBpedia100k (Ding et al., 2018) YAGO310 (Mahdisoltani et al., 2015)
2,034 10,282 11,512 13,166 14,149 14,541 17,050 22,925 40,943 44,949 45,158 74,536 77,951 78,334 99,604 123,182
42 135 237 237 237 237 51 200 11 1,605 24 200 69 34 470 37
32,888 16,142 27,211 54,423 136,057 272,115 185,584 25,445 86,835 242,567 2,025,177 149,678 551,193 100,000 597,572 1,079,040
1,827 2,163 15,624 16,963 17,449 17,535 10,310 4,961 3,034 20,000 112,510 543 30,622 1,200 50,000 5,000
1,828 2,203 18,150 19,776 20,324 20,466 10,311 4,952 3,134 20,000 112,510 2,818 30,622 1,200 50,000 5,000
h/t h/t tails tails tails h/t h/t h/t h/t h/t h/t h/t h/t h/t h/t h/t
Table K.2: Family 2 datasets (18). Triples denote the number of edges of the graph given at training, validation, or test. Valid and Test denote triples to be predicted in the respective graph. Dataset FB v1 (Teru et al., 2020) FB v2 (Teru et al., 2020) FB v3 (Teru et al., 2020) FB v4 (Teru et al., 2020) WN v1 (Teru et al., 2020) WN v2 (Teru et al., 2020) WN v3 (Teru et al., 2020) WN v4 (Teru et al., 2020) NELL v1 (Teru et al., 2020) NELL v2 (Teru et al., 2020) NELL v3 (Teru et al., 2020) NELL v4 (Teru et al., 2020) ILPC Small (Galkin et al., 2022) ILPC Large (Galkin et al., 2022) HM 1k (Hamaguchi et al., 2017) HM 3k (Hamaguchi et al., 2017) HM 5k (Hamaguchi et al., 2017) IndigoBM (Liu et al., 2021)
Rels 180 200 215 219 9 10 11 9 14 88 142 76 48 65 11 11 11 229
Training Graph
Validation Graph
Test Graph
Entities
Triples
Entities
Triples
Valid
Entities
Triples
Test
1,594 2,608 3,668 4,707 2,746 6,954 12,078 3,861 3,103 2,564 4,647 2,092 10,230 46,626 36,237 32,118 28,601 12,721
4,245 9,739 17,986 27,203 5,410 15,262 25,901 7,940 4,687 8,219 16,393 7,546 78,616 202,446 93,364 71,097 57,601 121,601
1,594 2,608 3,668 4,707 2,746 6,954 12,078 3,861 3,103 2,564 4,647 2,092 6,653 29,246 36,311 32,250 28,744 12,797
4,245 9,739 17,986 27,203 5,410 15,262 25,901 7,940 4,687 8,219 16,393 7,546 20,960 77,044 93,364 71,097 57,601 121,601
489 1,166 2,194 3,352 630 1,838 3,097 934 414 922 1,851 876 2,908 10,179 1,771 1,201 900 14,121
1,093 1,660 2,501 3,051 922 2,757 5,084 7,084 225 2,086 3,566 2,795 6,653 29,246 9,899 19,218 23,792 14,775
1,993 4,145 7,406 11,714 1,618 4,011 6,327 12,334 833 4,586 8,048 7,073 20,960 77,044 18,638 38,285 48,425 250,195
411 947 1,731 2,840 373 852 1,143 2,823 201 935 1,620 1,447 2,902 10,184 476 1,349 2,124 14,904
21
Table K.3: Family 3 datasets (23). Triples denote the number of edges of the graph given at training, validation, or test. Valid and Test denote triples to be predicted in the respective graph. Dataset
Training Graph
Validation Graph
Test Graph
Entities
Rels
Triples
Entities
Rels
Triples
Valid
Entities
Rels
Triples
Test
FB-25 (Lee et al., 2023) FB-50 (Lee et al., 2023) FB-75 (Lee et al., 2023) FB-100 (Lee et al., 2023) WK-25 (Lee et al., 2023) WK-50 (Lee et al., 2023) WK-75 (Lee et al., 2023) WK-100 (Lee et al., 2023) NL-0 (Lee et al., 2023) NL-25 (Lee et al., 2023) NL-50 (Lee et al., 2023) NL-75 (Lee et al., 2023) NL-100 (Lee et al., 2023)
5,190 5,190 4,659 4,659 12,659 12,022 6,853 9,784 1,814 4,396 4,396 2,607 1,258
163 153 134 134 47 72 52 67 134 106 106 96 55
91,571 85,375 62,809 62,809 41,873 82,481 28,741 49,875 7,796 17,578 17,578 11,058 7,832
4,097 4,445 2,792 2,624 3,228 9,328 2,722 12,136 2,026 2,146 2,335 1,578 1,709
216 205 186 77 74 93 65 37 112 120 119 116 53
17,147 11,636 9,316 6,987 3,391 9,672 3,430 13,487 2,287 2,230 2,576 1,818 2,378
5,716 3,879 3,106 2,329 1,130 3,224 1,143 4,496 763 743 859 606 793
4,097 4,445 2,792 2,624 3,228 9,328 2,722 12,136 2,026 2,146 2,335 1,578 1,709
216 205 186 77 74 93 65 37 112 120 119 116 53
17,147 11,636 9,316 6,987 3,391 9,672 3,430 13,487 2,287 2,230 2,576 1,818 2,378
5,716 3,879 3,106 2,329 1,131 3,225 1,144 4,496 763 744 859 607 793
Metafam (Zhou et al., 2023) FBNELL (Zhou et al., 2023) Wiki MT1 tax (Zhou et al., 2023) Wiki MT1 health (Zhou et al., 2023) Wiki MT2 org (Zhou et al., 2023) Wiki MT2 sci (Zhou et al., 2023) Wiki MT3 art (Zhou et al., 2023) Wiki MT3 infra (Zhou et al., 2023) Wiki MT4 sci (Zhou et al., 2023) Wiki MT4 health (Zhou et al., 2023)
1,316 4,636 10,000 10,000 10,000 10,000 10,000 10,000 10,000 10,000
28 100 10 7 10 16 45 24 42 21
13,821 10,275 17,178 14,371 23,233 16,471 27,262 21,990 12,576 15,539
1,316 4,636 10,000 10,000 10,000 10,000 10,000 10,000 10,000 10,000
28 100 10 7 10 16 45 24 42 21
13,821 10,275 17,178 14,371 23,233 16,471 27,262 21,990 12,576 15,539
590 1,055 1,908 1,596 2,581 1,830 3,026 2,443 1,397 1,725
656 4,752 10,000 10,000 10,000 10,000 10,000 10,000 10,000 10,000
28 183 9 7 11 16 45 27 42 20
7,257 10,685 16,526 14,110 21,976 14,852 28,023 21,646 12,516 15,337
184 597 1,834 1,566 2,441 1,650 3,113 2,405 1,388 1,703
22
Table K.4: Per-benchmark half-link scenario proportions (%) range from 0% to 98% SQSA, the split-driven composition discussed in Sec. 4 These proportions are a property of the test split alone; they do not depend on any model or on zero-shot vs. fine-tuned evaluation. Each row sums to 100%. N = number of test triples. † Tail-only evaluation task (predicting t only); SQUA ̸= UQSA. Ver.
Family 1
CoDEx Small WDsinger FB15k237_10† FB15k237_20† FB15k237_50† FB15k237 CoDEx Medium NELL23k WN18RR AristoV4 Hetionet NELL995 CoDEx Large ConceptNet100k DBpedia100k YAGO310
— — — — — — — — — — — — — — — —
80.6 28.1 29.7 41.1 57.6 68.0 66.3 41.7 22.1 61.6 98.0 0.0 53.0 79.1 60.2 82.2
9.2 27.8 41.8 36.0 27.9 15.2 16.3 25.0 32.8 17.8 1.0 38.2 22.6 9.9 17.4 8.4
9.2 27.8 19.3 17.2 11.7 15.2 16.3 25.0 32.8 17.8 1.0 38.2 22.6 9.9 17.4 8.4
1.0 3,656 16.3 4,406 9.2 18,150 5.7 19,776 2.8 20,324 1.6 40,932 1.1 20,622 8.3 9,904 12.3 6,268 2.8 40,000 0.0 225,020 23.6 5,636 1.8 61,244 1.1 2,400 5.0 100,000 1.0 10,000
Family 2
FB FB FB FB WN WN WN WN NELL NELL NELL NELL ILPC Small ILPC Large HM HM HM IndigoBM
v1 v2 v3 v4 v1 v2 v3 v4 v1 v2 v3 v4 — — 1k 3k 5k —
22.1 37.2 37.0 43.4 35.6 32.3 15.0 31.0 0.0 48.1 41.2 56.4 44.6 31.5 15.7 11.7 10.0 72.3
30.0 24.6 25.0 23.1 23.5 23.8 33.6 25.8 50.0 21.0 25.9 18.6 26.3 32.7 39.0 41.0 42.0 13.3
30.0 24.6 25.0 23.1 23.5 23.8 33.6 25.8 50.0 21.0 25.9 18.6 26.3 32.7 39.0 41.0 42.0 13.3
17.9 13.6 13.0 10.4 17.4 20.1 17.8 17.4 0.0 9.9 7.0 6.4 2.8 3.1 6.3 6.3 6.0 1.1
822 1,894 3,462 5,680 746 1,704 2,286 5,646 402 1,870 3,240 2,894 5,804 20,368 952 2,698 4,248 29,808
Family 3
SQSA UQSA SQUA UQUA
N
Dataset
FB-25 FB-50 FB-75 FB-100 WK-25 WK-50 WK-75 WK-100 NL-0 NL-25 NL-50 NL-75 NL-100 Metafam FBNELL Wiki MT1 Wiki MT1 Wiki MT2 Wiki MT2 Wiki MT3 Wiki MT3 Wiki MT4 Wiki MT4
— — — — — — — — — — — — — — — tax health org sci art infra sci health
51.5 40.5 48.0 60.0 43.3 41.5 44.7 43.0 22.7 20.0 25.3 32.2 34.0 0.0 37.0 8.2 16.3 87.9 19.9 25.9 43.0 67.4 64.2
18.9 22.3 20.9 18.2 25.5 26.2 25.3 25.6 30.5 35.3 32.7 28.3 32.3 50.0 27.6 44.2 41.5 5.9 38.3 34.1 27.0 15.9 17.2
18.9 22.3 20.9 18.2 25.5 26.2 25.3 25.6 30.5 35.3 32.7 28.3 32.3 50.0 27.6 44.2 41.5 5.9 38.3 34.1 27.0 15.9 17.2
10.7 14.9 10.2 3.6 5.7 6.1 4.7 5.8 16.3 9.4 9.3 11.2 1.4 0.0 7.8 3.4 0.7 0.3 3.5 5.9 3.0 0.8 1.4
11,432 7,758 6,212 4,658 2,262 6,450 2,288 8,992 1,526 1,488 1,718 1,214 1,586 368 1,194 3,668 3,132 4,882 3,300 6,226 4,810 2,776 3,406
23